commvault.com commvault.com
Solved

Reservation Status: No new readers can be allocated

  • 16 July 2021
  • 6 replies
  • 1151 views
D
Rank
Userlevel 4
Badge +13

Hello,

 

in the log of aux copy job (primary copy(disk storage) to secondary copy (tape library), there is this information: "Reservation Status: No new readers can be allocated, check for additional streams after [900] seconds, pending streams [1]".

 

Can you explain here, what does it mean and possibly how to avoid this? Can this state caused slow backup performance?

icon

Best answer by Laurent 20 July 2021, 14:34

View original

6 replies

Mike Struening RETIRED
Rank
Userlevel 7
Badge +23

Hey @drPhil !  That message is saying you have no streams left to use (at this moment), but the job will recheck in 900 seconds to see if any have freed up.

What is the stream count for that Storage Policy?  Are they combined in the Aux Copy?

Also, recalling your other thread, is there multiplexing involved in the Storage Policy?

At the root, we need to see if there’s an issue with slow reads (holding up streams), slow writes (doing the same), incorrect stream counts (you could do more if the job was allowed), or some combination of these.

Also, check cvperfmgr.log to see what our performance metrics are.

https://www.linkedin.com/in/michael-struening
D
Rank
Userlevel 4
Badge +13

Hi @Mike Struening ,

What is the stream count for that Storage Policy?  - Storage policy - Device streams: 100; Aux copy: Combine source data streams to - not in use; Multiplexing for the storage policy copy not used; Maximum number of data readers: 5 (copying from one storage array to the second one)

In my opinion, the heart of the matter seems to be slow reader time. My question is - can be the reader time somehow influenced by the Commvault settings (number of reader streams, device streams etc.) or is it just the matter of the RAID configuration or hardware issue?

Mike Struening RETIRED
Rank
Userlevel 7
Badge +23

Appreciate the details!

So if we’re just copying from one array to the other, then you should have a maximum of 100 streams on the primary, connecting to 100 streams on the Aux Copy.  

Since that destination is not tape, our only concern is really if the hardware can handle it (on both ends).

It very well can be a slow read issue which will cause this issue, however, there’s not much Commvault can do to influence that.  The only main ways would be a) having too few streams (idle hardware) or too many (causing too much work for too little hw).

Let’s heck cvperfmgr.log to see what our performance metrics are which will help shape the next action/adjustments.  Share what you see in this log and we’ll be able to make suggestions.

https://www.linkedin.com/in/michael-struening
D
Rank
Userlevel 4
Badge +13

Hi @Mike Struening , hm, it’s strange. Some storage policy aux copy jobs are quite fast, on the other hand some of them are very slow. However, there are just 2 storage arrays, where data is to be copied. We just need to duplicate data from primary copies (one storage device) to the secondary copies (the second storage device). There is just a short extract from the MA while duplicating data:

 

 

J
Rank
Userlevel 5
Badge +11

hi @drPhil , the log snip you gave is for a very short stream of an SQL backup, not an aux copy.

 

Would you be able to provide a log snip of an aux copy job with preferably a stream time in 1000’s of seconds rather than 18 seconds?

 

Also for the original stream issue, need to make sure theres available streams at SP level, library level and DDB level (if this is deduped to disk and not to tape). If tape, see if there’s free tape drives at all and whether you are using multiplexing to optimize write speeds to tape. 

L
Rank
Userlevel 6
Badge +15

Hello,

 

  1. in the log of aux copy job (primary copy(disk storage) to secondary copy (tape library),…
  1. "Reservation Status: No new readers can be allocated, check for additional streams after [900] seconds, pending streams [1]".

 

  1. Can you explain here, what does it mean and possibly how to avoid this?
  2. Can this state caused slow backup performance?
  1. So you have backup to disk (what kind of disk device is this ?), and copy from that disk device to tape library
  2. From this event, I understand that the copy from disk to tape cannot read, so mostly the library or source SP stream settings are involved.
  3. This means that, at some level, a setting/configuration has been set to limit the maximum data readers for this source copy, and that limit has been reached at the moment this event is written. The auxcopy will be automatically retried later in 900seconds hoping the limit would still not be reached at that moment.
  4.  Depends, but probably yes. I would say, not directly because of the Auxcopy to tape, but maybe just the consequence of too many streams on the disk library or the primary StoragePolicy (too many backups or streams in parallel for the running backups), reaching the limit, then the auxcopy can’t proceed. 

I’d suggest to review what’s running at the same moment (backups) for the same storage policy, and do the same at the same time for the Disk Library, and review all this.

Reply


Terms & ConditionsCookie settings