commvault.com commvault.com
Question

OnePass retention settings and backup chain

  • 28 February 2024
  • 6 replies
  • 51 views
G
Rank
Badge +6

For Archived data produced by the OnePass feature in Commvault what determines the retention time in the Storage Policy Copy?

There is a “Basic Retention Rule for All Backups” and a Basic Retention Rules for Data / Compliance Archiver Data. If I’m right the Basic Retention for All Backups setting would dictate retention for “backups” that were referenced by subclients to that storage policy and the latter would just reference data that was archived by OnePass and uses the same storage policy copy? So if the “Basic Retention Rules for Data / Compliance Archiver Data” setting is infinite data created by OnePass will never age out? 

Secondly if the schedule associated with OnePass is configured to do a nightly incremental and never a Full or Synth Full what is the impact of deleting a heap of Incremental Backups in the backup chain? A full was done only the first night this was set up.

I was never intended to retain the Archived files indefinitely and as a result 3 years of unwanted non deduped data has been retained. Will deleting these incrementals immediately after the first full and up to the point we want to retain cause any issues? I’m assuming the associated stub files will remain unless manually deleted? 

6 replies

B
Rank
Userlevel 5
Badge +12

Hi @Glenno,

 

Thank you for your question!  The ‘Basic Retention Rules for Data / Compliance Archiver Data’ was a legacy setting that has been removed in more recent versions of the software.  This was used only by the Classic Archiver agents, which aren’t supported anymore.

 

The settings for Archived data was moved to the Subclient level in the Retention Tab:

 

 

Depending on the type of client we’re talking about, there should be 2 different retention modes here:

 

  1. Job Based Retention
    1. This will only perform 1 Full backup as its first job and the rest will be incremental forever.
    2. This does not support Synthetic Fulls either, so it would be expected that only 1 Full followed by nothing but incrementals will be seen here.
  2. Object Based Retention
    1. Similar to Job Based retention, there will only be 1 Traditional Full backup here, however both Incrementals and Synthetic Fulls are supported in this scenario.

You’ll find that the retention configuration is a little bit different between these 2 options:

 

  1. Job Based Retention
    1. Specify the desired amount of time that each job should be retained.
      1. Jobs will be aged based on the longer value between the Subclient’s configuration and the Storage Policy’s retention.
        1. Example 1 - If Subclient retention is 90 days, but Storage Policy Retention is 180 days, the Storage Policy retention is honored.
        2. Example 2 - If Subclient retention is 180 days, but Storage Policy retention is 90 days, Subclient retention is honored.
  2. Object Based Retention
    1. You’ll find more granular options here where you can specify a length of time to maintain individual objects based on various types of criteria
      1. Modified time on the file
      2. Deleted time (meaning files that are deleted from the source client)
      3. File Versions
    2. These settings will dictate when we will stop including files in the subsequent Synthetic Full backup.
    3. Incrementals and Synthetic Fulls themselves will follow Storage Policy Retention.

We do offer a deeper explanation of many of these options in the below documentation:

 

https://documentation.commvault.com/2023e/expert/subclient_properties_retention.html

 

Please let me know if you have further questions.

 

-Brian Bruno

G
Rank
Badge +6

Thanks Brian! The software version is 11.24 and the subclient resides under the File System, DefaultBackupSet. There are both a Retention and an Archiving rules tab on the subclient config which I can’t see in your screenshots above. i have included screenshots of the sub client retention and storage policy settings. Yes there is only a single Full backups and rest are incremental but data has not aged after the 2555 days in the policy. We would like to change settings to allow all backups after this specified period to age out and recover library space. 

 

 

G
Rank
Badge +6

I can see in the link you sent the following: Starting from Feature Release 11.22, this topic does not apply to newly created subclients for an archive set. If you upgraded a client from FR 11.21 or an earlier release, you can still see and edit the retention settings for the upgraded subclients. Yes this instance was upgraded from a previous verison.

B
Rank
Userlevel 5
Badge +12

Hi @Glenno ,

 

Thanks for the reply!  In that configuration we are going to honor Basic Storage Policy Retention, however that subclient is going to need to be included in a Synthetic Full schedule.

 

Since Archiving is being performed you won’t be able to run Traditional Fulls backups, but should instead be leveraging Synthetic Fulls.  The Infinite retention setting on the Storage Policy’s retention screen won’t be used in this scenario.

 

Unfortunately, none of the data will age here until the Basic Retention requirements of 2555 days / 7 cycles are met.  Considering there is only a single cycle here (since there’s never been a Synthetic Full run) this would suggest that in order for the Data Aging rules to prune those jobs, 7 more Synthetic Fulls would need to age, and the most recent Incremental would need to be 2555 days old before any pruning will occur.

 

The alternative would be to perform some manual pruning of the older jobs, however please be sure to NOT do that until at least 1 Synthetic Full has been performed.  The Synthetic Full will bundle up all of the data that was contained in the initial full as well as all of the subsequent incrementals that meets the criteria that you have defined in the Retention tab of the Subclient.

 

This means that any files that were deleted directly by a user more than 2555 days ago will NOT be included in the upcoming Synthetic Full.  All other files that were deleted by an Archiving job, or any stubs that were deleted manually by a user will be included, since the subclient defines to retain all of those items indefinitely.

 

-Brian Bruno

G
Rank
Badge +6

Hi Brian, sincere thanks for your time answering these questions. So, I f I understand correctly a Synthetic full should be included in the backup schedule. If archiving is done every night how often would it be required? Now the kicker. There are over 400 TB of backups under current policy over a period of 9 years. Just how big would a Synth full be expected to be? Our object is to save disk space :-) Post this Synth full backup you are suggesting manual deletion of the jobs older than 2555 days from the storage policy? Ongoing what modifications to settings ensure that data will age out on the 2556 day without any manual action?

 

B
Rank
Userlevel 5
Badge +12

Hi @Glenno,

 

I would generally recommend doing a Synthetic Full at least every 30 days, if not once a week.  The way our Data Aging process works is that we age jobs a full cycle at a time, meaning that we will need to wait the set number of Basic Days on the last incremental in that cycle before we will allow any other jobs within that cycle to be eligible for pruning. 

 

The more frequently we run Synthetic Fulls, the more frequently we will end each cycle, and thus the sooner that 2555 day counter starts ticking on that last incremental in order to start the aging process.

 

In terms of how large that Synthetic Full will be, that really depends on how much of that 400 TB worth of data meets the criteria configured in the subclient’s Retention tab.  A quick summary of what you have per the example screenshot you provided:

 

Files deleted by user: 2555 days (storage policy retention).

Stubs deleted by user: Infinite

Files deleted by Archiving: Infinite

 

So this means that we will essentially be rolling forward all of the data that was included in all of those prior incrementals minus any files that have been deleted by the user more than 2555 days ago.  If there is routinely a lot of files that get created and subsequently deleted on that client, it’s possible that we may not need to include much of that data in the upcoming Synthetic Full.  However, if we assume that there were no files deleted by the user, then we can expect the Synthetic Full to be near that full size.

 

That being said, assuming we’re using deduplication in this policy, then you can expect the actual media size footprint of that Synthetic Full to be quite small, as it would be all references to data that has already been protected, meaning data written will be negligible.  Although the Application Size of the job could be near that 400 TB value, assuming deduplication is used we won’t have to write any new data to storage.

 

-Brian Bruno

Reply


Terms & ConditionsCookie settings