Skip to main content

Could someone explain a flow of process for allocating readers for an auxcopy job?

What does stream allocation take place in CVJobReplicatorODS exactly? 

I see that the number of readers is a function of num of CPU and RAM. By default, each CPU in a proxy can support 10 streams, and each stream requires 100 MB of memory. For e.g for VSA.

E.g. My VSA backups were backed up with x number of readers and y number of writers to HPE StoreOnce Catalyst.

How does the replication agent (auxcopy) decide and allocate readers for copying a list of N jobs to tape. 

Imagine there is a primary copy → Storeonce Catalyst

secondary copy → tape (combine streams to 2 tape drives with multiplexing factor of 25).

 

Number of readers varies from 12 to 38 in my environment. It appears that the number of readers used for the subclients plays a role in the number of readers that will be assigned in the auxcopy. My goal is to increase the readers for the auxcopy jobs for improving the performance. My auxcopy with 38 readers streams has a throughput of 1.4TB/hr while the job with 26 readers on a different MA is 255GB/hr. Note that both the auxcopies are copying VMware (VSA) backups. 

 

Would be happy to get pointers in understanding the high level flow of how readers are allocated in auxcopies? How can the number of readers for auxcopies be tuned? The goal is to get as many readers as possible to not have the tape drive wait for data from readers.

 

Catalyst API for reading data doesnt show much difference from a perf standpoint
notice that higher the num of readers higher the performance

 

have opened a TR

Hey @neuwiesener !  If you don’t mind, can you share the incident answer back on this thread once resolved?

I can adjust the Best Answer afterwards.

You are right - job readers has a direct impact on streams that can be used on the auxcopy. If you are auxcopying 1 job, and than job ran with 5 readers, then 5 streams is possible. This is why there is so much love for VSA with V2 since we can do multiple streams on the synthetic full, improving performance for the auxcopy like you have observed.

 

Hi @neuwiesener 

Just sharing my two cent here

CVJobReplicatorODS is an encapsulated process within CVODS framework which handles the Aux Copy Read operation (and other operations). 

Depending on the configuration/schedule on the CommServe, the CommServe will alocate resources and send the request to the MediaAgent 

As mentioned by Damian, the way streams are set on Subclient (VSA V2) will definitely impact the amount of readers and streams distribution when running Aux Copy. 

If 5 Streams are used to backup the Subclient, the same stream account will be calculated during Aux Copy to move the Backup to Secondary Storage. 

For HPE StoreOnce Catalyst Aux Copy, as this is not a Catalyst Replication process, what happens is Commvault will send the Catalyst API to re-read the data to the MediaAgent and then the MediaAgent will write the data back to Tape. 

One thing to keep in mind is when Commvault is engaging Catalyst Library we will be utilizing data session/command sessions on the HPE StoreOnce, so when modifyingStreams we will also need to be mindful to not saturate these connection to HPE StoreOnce or read/write will halt. 

Recommendation - monitor the current data/command session on the HPE StoreOnce and then optimize the streams for Aux Copy (increase it if we still have a lot of resources on the HPE StoreOnce)

Regards

Winston 

Configuring stream randomization should also help with having more source streams, as it will randomly pick the stream ID from the available pool of streams as configured by the number of devices streams in the Storage policy

For more information on how to configure this, please have look at our documentation.

Reply


Terms & ConditionsCookie settings