commvault.com commvault.com
Solved

Questions about scheduling and AUX copies

  • 2 February 2023
  • 4 replies
  • 529 views
T
Rank
Userlevel 2
Badge +10

I was wondering:

  • Can I just rename a schedule and not have to do anything else (like re-associate it anywhere)? I have several schedules made by someone else I’d like to rename to a standard naming, and I don’t want to break anything.
  • Are there any “gotchas” with scheduling Aux copies to run every hour? Currently we have most of our Aux copies scheduled to run 2x per day (and they run for hours), but some things like DB transaction logs are scheduled to be backed up hourly, so there’s a lot of data when they are finally Aux copied.  I wasn’t sure if there was a “lower recommended limit” to running an Aux copy like if it caused any issues like “it takes quite a bit of CPU and database locking to prep and start it” or “too many aux copy jobs can make the DB or tape writing slower”, etc.
  • I notice if an Aux copy is running and another one kicks off for the same policy/copy, it is considered “failed” as it could not run.  This shows up on reports as “failed” alongside of things that actually, legitimately failed. Is there a way to schedule/set/config an Aux copy to not fail (like a different status) or just not attempt to run if its already running?
  • I’ve read about “contention” being a possible issue when you schedule/run an Aux copy and at the same time write tapes associated with the same storage policy as the Aux copy. Is this still a possible “issue” to be concerned about when scheduling tapes/Aux Copies (like in 11.24+, I’m not sure if contention was more of an issue with older versions?) and if so are there any settings/scheduling tricks to help this out so they don’t both run at the same time (like “stop Aux copy if running on Saturday 5 PM, Start tape job, restart Aux Copy when tape job ends”)
  • Can you set a schedule to kick off an Aux copy if the amount of data to be copied is > some threshold? like “if data to be copied >10 TB, run Aux copy”
icon

Best answer by Jon Vengust 3 February 2023, 02:58

View original

4 replies

Jon Vengust
Rank
Userlevel 3
Badge +5

Hi tigger2,

 

Can I just rename a schedule and not have to do anything else (like re-associate it anywhere)? I have several schedules made by someone else I’d like to rename to a standard naming, and I don’t want to break anything.

 

Renaming a schedule policy has no bearing/impact. It is simply a label. All existing associations will reflect the name change after being modified.

 

Are there any “gotchas” with scheduling Aux copies to run every hour? Currently we have most of our Aux copies scheduled to run 2x per day (and they run for hours), but some things like DB transaction logs are scheduled to be backed up hourly, so there’s a lot of data when they are finally Aux copied.  I wasn’t sure if there was a “lower recommended limit” to running an Aux copy like if it caused any issues like “it takes quite a bit of CPU and database locking to prep and start it” or “too many aux copy jobs can make the DB or tape writing slower”, etc.

 

Auxiliary Copy operations can be scheduled every hour but only one can run per storage policy copy at any given time. You may find that with enough of a backlog, you will have AuxCopy jobs in a Queued state waiting for the existing job to complete. Depending on where the AuxCopy is writing to, you may find performance gets diluted across the many jobs being launched at the same time but it is definitely worth testing to see your current capabilities.

 

I notice if an Aux copy is running and another one kicks off for the same policy/copy, it is considered “failed” as it could not run.  This shows up on reports as “failed” alongside of things that actually, legitimately failed. Is there a way to schedule/set/config an Aux copy to not fail (like a different status) or just not attempt to run if its already running?

 

This is not a typical Failed state, but rather Failed to Start (with the reason being that there is an existing job running). This is something I’ve alluded to in the previous question.

 

I’ve read about “contention” being a possible issue when you schedule/run an Aux copy and at the same time write tapes associated with the same storage policy as the Aux copy. Is this still a possible “issue” to be concerned about when scheduling tapes/Aux Copies (like in 11.24+, I’m not sure if contention was more of an issue with older versions?) and if so are there any settings/scheduling tricks to help this out so they don’t both run at the same time (like “stop Aux copy if running on Saturday 5 PM, Start tape job, restart Aux Copy when tape job ends”)

 

Contention will occur when there is a finite amount of tape drives available whereby the number of AuxCopy jobs exceeds this number. AuxCopy jobs that aren’t able to reserve a tape drive for read/write purposes will remain in a Waiting state until a resource becomes available.

 

Can you set a schedule to kick off an Aux copy if the amount of data to be copied is > some threshold? like “if data to be copied >10 TB, run Aux copy”

 

No, I do not believe that is an option.

 

Kind regards,

 

Jon Vengust

Helping organisations cope in the minefield that is enterprise data protection and management #CloudResponsibly
Jos Meijer
Rank
Userlevel 7
Badge +16

Hi @tigger2 

You can rename schedules without breaking it, no re-association needed.

Regarding your second and third points, I would use the continuous option in the schedule. This allows to start a job x minutes after the last job ended. The default schedule is 30 minutes. If a job already runs then it checks again later to see if the schedule parameters are met and a job needs to be started.

https://documentation.commvault.com/2022e/expert/6699_creating_continuous_schedule.html

Hope I interpreted this question correctly. Contention is always possible if the same drives are requested for certain jobs, but the system allows claims only when the resources are available preventing an actual issue. Jobs requesting unavailable resources will wait untill resources become available. Schedule staggering will definitely help and you could also set job priorities to give priority to certain activities.

https://documentation.commvault.com/2022e/expert/6516_job_priorities_and_priority_precedence_advanced.html

Regarding the data threshold as a trigger for an auxcopy, I do not believe there is an option out of the box, but I can check later on. Can't check this now unfortunately. If not possible out of the box, I am almost sure you could realize this via REST API. I will get back on this if no one answered this already before I can check.

Hope this helps.

Jos Meijer
Rank
Userlevel 7
Badge +16

Hi @tigger2 

Regarding the data threshold trigger, there is no out of the box option to define an amount of data.
I do have some workaround ideas which can create a similar delay / trigger:

  • Defer Auxiliary Copy for n day(s)

    Specifies that data can be copied to this copy only after an n amount of days after the data was protected. So the amount of data is not fixed, but it will allow you to perform copies in a manually set frequency based on the initial protection date without modifying your auxcopy schedule.

    This setting can be found within the Copy Policy tab which is in the properties of a storage policy copy.
     

  • Manually configure a trigger based on data amount using the Fallen Behind setting on SP Copy

    Exclude the storage policy copy from the auxcopy schedule.
    Set your storage policy copy fallen behind value to the required data amount on which you want to trigger an aux copy.
    Create a new alert specific for the SP copy involved, based on Job Management - Auxiliary Copy
    Select the SP copy
    Check the box for Auxiliary Copy fallen behind alert
    Use the Run Command to run a script to initiate the aux copy.

    This script specific for this SP Copy can be created by manually right clicking the SP involved, select All Taks - Run Auxiliary Copy. Define the copy and select Save as Script.

I also checked the REST API method, but I can't seem to find the fallen behind trigger there.
So as far as I can see this is not an option.

Hope this helps.

T
Rank
Userlevel 2
Badge +10

Thank you so much @Jon Vengust @Jos Meijer !

 

Info: All of our schedules/copies/etc. were made by several people in the past and essentially batches of similar copy types will all “kick off” at the same time. It’s very ‘simple/basic’ and I think it worked well for when we had less data to back up but it appears to causes a lot of Aux copies to run at the same time and slow down their overall throughput (we have some physical hardware limitations we will be alleviating in the future), as well as most of the tape jobs run at the same time and go to pending and seem to slow down the system as well as we have fewer drives than Aux tape jobs are configured for.  Whenever we have to do something heavy like DDB data verification or a job simply backs up (due to issues, say tape drive failures or a lot of new data is being backed up) it can take a long time for Aux copies to catch up.

I think all this information will help me make some scheduling changes and spread out the CPU/job load a bit better without complicating it too much. Thanks again!

Reply


Terms & ConditionsCookie settings