We want to have Azure Blob containers Immutable feature turned ON.
Primary - Incremental - 30Days - AzureBlobContainer1 Monthly Full backup Extended retention - 365Days - AzureBlobContainer1 With Commvault Deduplication.
Commvault DDB resides on the Media Agent.
How so we handle DDB and the extended retention of INCR and Full on the Container1?
Will the data on the container1 expire post INCR has reached its retention for 30Days?
Currently we are not sealing our DDB. Do we need to seal the DDB when using Azure Immutable storage?
Can we have one Azure Immutable blob and inside them have multiple folders with multiple retention?
How does Commvault Retention linked with Azure blob Immutable - Access Policy - Retention Interval?
What if Azure Retention Interval on AzureBlobcontainer1 is set for 30Days and My Commvault retention is for 365Days. what happens to the same data on 31st day?
Page 1 / 1
Hello @Kavya
You can refer to our documentation for WORM on Cloud.
Can you clarify what you mean by “handle the DDB”?
The data will prune once the Storage Policy and Azure Lock duration have been met only when the DDB is sealed.
Yes the DDB does need to be sealed at the same interval of the Policy retention. For example if the Policy has 60day retention, the DDB needs to be sealed every 60days.
Azure Retention lock is container-based. You can only have 1 retention per container.
The Azure Retention Lock duration must be set to twice the amount of time as the Policy. For example, the Policy has 60day retention, the DDB is sealed every 60 days, the Retention Lock must be set to 120 days.
The data won’t age until after 365 days and the DDB has been sealed, but after 30 days it can technically be modified.
Note:
1) All of this is automated by the workflow and all you need to do is create the Library\Storage Pool, Create the Copy and associate the storage pool, run the workflow on the storage pool.
2) All Copies associated to the storage pool must have the same retention configured to prevent issues with pruning.
Thank you,
Collin
Why and how is the DDB sealing linked to immutable backups
Thank You @Collin Harper, That was very helpful and in-depth reply
I have a follow up scenario
Primary Copy
AzureBlobContainer1 with Immutable Primary Copy 30 days incremental with dedupe Primary Copy Full backup at the end of month with dedupe
The Azure Retention Interval is 30 days Commvault retention is 30 days
How will the DDB sealing work?
Secondary Copy
Aux copy the full backup from AzureBlobContainer1 (primary) with dedupe to a AzureBlobContainer2 with dedupe with immutable.
The Azure Retention Interval is 180 days Commvault retention is 180 days
How will the DDB sealing work?
Hello @Kavya ,
I also discussed a lot with Commvault and tested in the last year regarding Immutable Copy in Azure.
At the moment we test concrete following Requirement:
7 Day Immutable Copy → All Jobs
Using Object Locking mechanism
In Azure you have the following Requirements for Using Immutable with Object Lock:
dedicated Storage Account with
Versioning Enabled (all Versions) → no cleanup Rule by Azure
Hot or Cold
If you have less than 30Days Retention you should us Hot, because on “Cold” you have “Minimum Stay Days” from Azure of 30 Days
So we use “Hot”
Create a Container with “Enable version-level immutability support”
Important to know about WORM/Immtuable :
for each Retention you must have an own Bucket → not mix Retetions in the same bucket
Snapshot Copy Jobs not supported (Job based Retention)
Micro Pruning is not supported
Immutable Object Lock always calculated Retention *2 , this means if you have 14 Days Retention the Immutable Lock is 14*2 = 28 Days
DDB Sealing necessary because of Cleaning Date must use Macro Pruning
Background is that all referenced blocks in the DDB must be locked up to the final release and thus the aging can take place only in the whole with sealed DDB.
After you added the Immutable Container as CLoud Storage you must run the WOrkflow ”Enable WORM Storage” This WF sets:
Disable Micro Pruning
Set WORM LOCK Days 2* Retention
Sealing DDB to Retetion Days (in Default - can be overwritten)
Set WORM Active on Storage Policy associated with Library → no Deletion possible
In our Case we added to our Storage Policy a “Worm Copy” “All JObs” with Retention “7Days / 0 Cycles”
Retention Days set correct
Object Lock works
DDB get sealed as configured
Deletion of Data in Container not possible
At the moment we Monitor the Costs in Azure and we have one problem open Index Jobs are not pruned and therefore Sealed DDBs not cleaned up (I asked today here in the forum for help)
The most difficult thing is cost calculation, but maybe you can get in contact we Commvault from your side with Manager, they have a Excel Cost Calculator to estimate Cost, but this Calculator so far as I know is not public.
Hope this helps a little bit.
BR
Marcel
@Kavya
The Azure Retention Lock would need to be double the duration of the Commvault retention. This is because of deduplication.
Example: A Job is written on Day1 would age on Day 30. A Job written on Day 30 would age on day 60. Since everything is deduplicated the job written on day 30 depends on the blocks written by the jobs written days 1-29. If retention lock is set to 30days and the DDB is sealed, all the blocks from Days 1-29 become eligible to prune. You would be pruning the data from the job before its retention is over and it wouldn’t be restorable.
DDB sealing is required because we cannot micro-prune the data. Micro-pruning requires the files to be modified as we delete the unnecessary data within the blocks. WORM prevents the files from being modified so we can only prune once all of the data has aged. Sealing the DDB allows it to be prunable.
Thank you,
Coilin
@Kavya short Update here: we had one problem open with Index Backup Jobs got bot pruned, this was a BUG:
we were running on V11.28 then the fix is included with V11.28.76
indexing backups are not aging from sealed DDB
5065, 5066
After installing this fix release, Index jobs got pruned and sealed DDBs were cleaned up.
But have to keep in mind Index Jobs always have different Retention settings.
BR
Marcel
Hello @Collin Harper and @Marcel Geisen,
For 365 backup (infinity retention), with backup destination a hot Blob Storage, could you please advise me about the immutability / WORM copy procedure?
Is just the WORM copy checkbox in Storage policy or are there additional steps needed?
Any Object Store (Edge Drive, third-party migration, and so on) and hybrid file
Snap copy (job-based retention)
Thank you, Collin
Hello @Kavya ,
I also discussed a lot with Commvault and tested in the last year regarding Immutable Copy in Azure.
At the moment we test concrete following Requirement:
7 Day Immutable Copy → All Jobs
Using Object Locking mechanism
In Azure you have the following Requirements for Using Immutable with Object Lock:
dedicated Storage Account with
Versioning Enabled (all Versions) → no cleanup Rule by Azure
Hot or Cold
If you have less than 30Days Retention you should us Hot, because on “Cold” you have “Minimum Stay Days” from Azure of 30 Days
So we use “Hot”
Create a Container with “Enable version-level immutability support”
Important to know about WORM/Immtuable :
for each Retention you must have an own Bucket → not mix Retetions in the same bucket
Snapshot Copy Jobs not supported (Job based Retention)
Micro Pruning is not supported
Immutable Object Lock always calculated Retention *2 , this means if you have 14 Days Retention the Immutable Lock is 14*2 = 28 Days
DDB Sealing necessary because of Cleaning Date must use Macro Pruning
Background is that all referenced blocks in the DDB must be locked up to the final release and thus the aging can take place only in the whole with sealed DDB.
After you added the Immutable Container as CLoud Storage you must run the WOrkflow ”Enable WORM Storage” This WF sets:
Disable Micro Pruning
Set WORM LOCK Days 2* Retention
Sealing DDB to Retetion Days (in Default - can be overwritten)
Set WORM Active on Storage Policy associated with Library → no Deletion possible
In our Case we added to our Storage Policy a “Worm Copy” “All JObs” with Retention “7Days / 0 Cycles”
Retention Days set correct
Object Lock works
DDB get sealed as configured
Deletion of Data in Container not possible
At the moment we Monitor the Costs in Azure and we have one problem open Index Jobs are not pruned and therefore Sealed DDBs not cleaned up (I asked today here in the forum for help)
The most difficult thing is cost calculation, but maybe you can get in contact we Commvault from your side with Manager, they have a Excel Cost Calculator to estimate Cost, but this Calculator so far as I know is not public.
Hope this helps a little bit.
BR
Marcel
Hello,
I see there a contradiction: “Immutable Object Lock always calculated Retention *2” and "If you have less than 30Days Retention you should us Hot, because on “Cold” you have “Minimum Stay Days” from Azure of 30 Days”
If we need retention of 15 days and more, Immutable Object Lock will be 30 days or more. For me it sounds that you should use cool instead of hot starting from 15 days retention. Or I'm missing something?