Retention & Data aging

  • 3 February 2021
  • 5 replies

Badge +3

Short version of the question: Does modifying a file in Commvault reset the 1-year retention?

In our environment I have daily incremental that roll into a weekly synth full that roll into a monthly synth full. That monthly is kept for 1 year so at any given point I have 12 months of Monthly synth full jobs.

Our retention is set to 1 year.  “For 365 days Keep Monthly Full”

So I create a file on 01/01/21 and the daily incremental job runs 01/02/21 and backs the file up.

It then should be part of the weekly synth job and then part of the monthly synth full job for January, correct?

That file then would be available (unless modified) until January 2022 when the next monthly synth full runs, Correct?

The question that was asked is unless a file is modified will it expire after 1-year?

If the file is modified on 9/1/21 then it would go through the daily incremental > weekly synth full > monthly synth full and retained until the 9/1/22 job ran

I guess the short question is after 1-year any none modified file is aged? Unless we change the retention or archive, 1-year is the max? That sound accurate?



Best answer by Mike Struening RETIRED 4 February 2021, 15:57

View original

5 replies

Userlevel 7
Badge +23

Thanks for the question, @Bill !

Based on your description, if that file is not modified after Jan 1, it will be added to every Synth Full that runs for that subclient.  You won’t ‘lose it’ after the Synth Full from January prunes off because it was in the February Synth full as well.

This assumes the file does not change further or get deleted at the source, etc.

If the source file gets deleted some time later, then at the next Synth full, it won’t be included.  That means the subsequent Monthly Synth full won’t have it, so then you will ‘lose’ the file once that final backup ages off.

*Editing my post to clarify*

Are you referring to the Retention on your Storage Policy Copy, or on the Subclient?  Links added for easy reference.

Let me know if the above answers your question.

Badge +3

@Mike Struening  I am referring to our Storage Policy Copy and thanks for the reply. I assumed the data aging process purged files/folders that had not been accessed in a year and it makes sense now that the file would be carried over until deleted.

So the Monthly kept for 1-year is kind of confusing then. It’s safe to say it keeps that, for lack of a better term, point in time but does not delete data, correct?

Thank you!

Userlevel 7
Badge +23

The job is kept for a year, and the files within that actual job would prune; however, there’s nothing to say that subsequent jobs wouldn’t have backed up that same file.

Synth Fulls are only using the already run jobs to create a Full off of the Media Agent; in effect they should contain the same files a normal full would.

For that reason, assuming the file is still on the client, each subsequent Synth full will contain that file (and retain the job according to your retention).

With your last sentence, I want to ask if you are using One Pass?  This is where we stub files along with the backup job.  We keep passing the data into the next backup to ensure you DON’T lose files when jobs age based on what is on the client.  Otherwise, if we didn’t keep the files getting added to the synth full, you’d lose the stub and the file.

Let me know if you’re Archiving in these backup jobs so I can be sure my answer is complete :nerd:

Badge +3

@Mike Struening I am not using OnePass. Based on the fact that  “For that reason, assuming the file is still on the client, each subsequent Synth full will contain that file (and retain the job according to your retention)” it seems like we could lose files on the client based on my Last Monthly for 1-year or obviously if a file is deleted and 366 days has passed then it would not be recoverable.

What I was asked is if there was a “reset mechanism” like if a file is created and not touched does it automatically get deleted after a year OR if a file is created in May and then modified (then an incremental backup happens on said file) and that would then reset the 1-year retention date.

Also we currently are not archiving but that topic is being discussed due to the concern of possibly losing data after one year.

Thank you very much for your detailed and informative replies! I came from a TSM background where “last copy forever” existed and so even after a couple of years I’m still learning Commvault.

Userlevel 7
Badge +23

Thanks for confirming!  This community is a great place to come and discuss these exact scenarios.

At the core, your question is more about retention then.  I’m not sure what your Basic Retention is set to, but assuming you have at LEAST 1 cycle set, then you’ll always have at least 1 full backup per subclient, regardless of your Extended 1 year retention on Monthly Fulls.

The file in question will be recoverable for as long as any job exists that contained that file (which is really two distinctions in one sentence).  I’ll break it down into 2 scenarios:

  1. The file is changed/created once and is never changed/deleted - As long as you keep running backups, this file will ALWAYS be included in your Synth Full, so in essence, it will be recoverable forever since the latest Synch Full contains it as well.
  2. The file is changed/created, but is then changed/removed after a Full/Synth backup - This means that Synth Full Job 1 contains that exact version of the file and will be held for your Basic Retention (unless it is the Monthly Full, then 1 year).  

Now, it’s certainly tough to know EXACTLY when a file was removed or changed (in case it changed a few times) to know in advance which job had that exact version (though you can browse the jobs or for that file to see what options there are).

Storage Policy Copy Retention is calculated on the job itself, not the files within the job, so we’re better off looking at the jobs themselves to determine how long they will be around.  Once we have that solved, then we can see what files are within the jobs.