Skip to main content

In our enterprise environment, we have a primary media agent we store the backups of all of our servers. Some of these are then copied out to Azure for cloud storage, all are eventually copied to tape, and then we have a completely separate media agent running a different OS than all of our other machines where we keep an aux copy of only the most recent synthetic full of all of our servers. Used to work great, and now is not. Pretty sure I know why, and I am curious if there is a better way to do what I am doing.

Previously, we had the retention set to 1 day, 1 cycle which worked fine. We did the aux copy every evening after the main backups completed. After the aux copy there would be two sets of data for the last two days on the media agent until noon the next day when the aging job would run and the earlier day would be purged from the system. This is great when you have 18TB of data and 40TB of storage and you can fit two copies on the media agent. But growth is a thing, and suddenly you don’t have room to store two full copies of the data on the machine and so the job suspends because there isn’t enough storage available to complete it.

Ideally long term the solution is to get more storage, which is 100% the plan. But in the short term, is there any better method to select for doing this kind of backup than a standard selective aux copy to pluck out just the most recent fulls and keep them only for 1 day? Any way to make it purge the data before the aux copy so there is enough room to store everything? I realize that is highly less than ideal as you could purge the data and suddenly some calamitous serious of events kick off that leave you needing those backups you just purged because some threat actor just nuked all of your windows based backup servers and now all you have is a half finished aux copy and your tapes.

Just curious if anyone has suggestions on how best to approach this situation in the short term until we get the funding approval for additional storage. Would inline copies or some other option work better than the approach I have taken? Running 11.28 On-Prem. About to upgrade to latest LTS version as well.

Hello @Brian Barrett 

Can you confirm if you are Deduping any of this data? I feel like having 1 day 1 cycle retention on your primary copy is bad if you are deduping the data as you will  rebase line and process way more than is needed if you were to keep just one extra cycle without all that much impact to storage. 

If you are not deduping this data its worth noting that we will not age anything off unless it is not required for any other operation like Aux copy. If you manually deselect the job from being copied and the primary is past retention/cycles it will age out. 

Kind regards
Albert Williams

It’s not a great option, but in theory you could set your retention to 1 Day / 0 Cycles, delay your Auxiliary Copy operation, and schedule in a granular Data Aging operation.

For example, if your backups are finishing by 2100, you could schedule a granular Data Aging operation (targeting only the specific Storage Policy Copy or Library) to commence at 2115, then schedule your Auxiliary Copy to commence at 2315. That would provide 2 hours for Data Aging and physical pruning to occur, before the Auxiliary Copy starts. 

As you say though, it would be less than ideal.

Hello @Brian Barrett 

Can you confirm if you are Deduping any of this data? I feel like having 1 day 1 cycle retention on your primary copy is bad if you are deduping the data as you will  rebase line and process way more than is needed if you were to keep just one extra cycle without all that much impact to storage. 

If you are not deduping this data its worth noting that we will not age anything off unless it is not required for any other operation like Aux copy. If you manually deselect the job from being copied and the primary is past retention/cycles it will age out. 

Kind regards
Albert Williams

While the primary copies being drawn from are deduplicated, these particular aux copies are not. And there is nothing dependent upon these aux copies. The aux copies to Tape, Azure, and this secondary disk library all come from the primary, so I do not believe there should be anything holding up or preventing the data from properly aging.

It’s not a great option, but in theory you could set your retention to 1 Day / 0 Cycles, delay your Auxiliary Copy operation, and schedule in a granular Data Aging operation.

For example, if your backups are finishing by 2100, you could schedule a granular Data Aging operation (targeting only the specific Storage Policy Copy or Library) to commence at 2115, then schedule your Auxiliary Copy to commence at 2315. That would provide 2 hours for Data Aging and physical pruning to occur, before the Auxiliary Copy starts. 

As you say though, it would be less than ideal.

I will give this a shot and see what happens. I guess it all comes down to how Commvault quantifies ‘1 day’. Is that simply that the calendar date has incremented by a day since the job completed, or is it that 24 hours have passed since the job completed? If is the former, I could see it working. it it is the latter (pretty sure it is) it could get really messy being sure we always wait 24 hours between jobs. Either way, if it doesn’t work it just gives me more leverage to push for getting the new storage ASAP and not waiting for 2025.

 

Reply


Terms & ConditionsCookie settings