commvault.com commvault.com
Solved

Commvault Azure Immutability

  • 31 August 2023
  • 6 replies
  • 77 views
K
Rank
Userlevel 1
Badge +4

We want to have Azure Blob containers Immutable feature turned ON.

Primary - Incremental - 30Days - AzureBlobContainer1
Monthly Full backup Extended retention - 365Days - AzureBlobContainer1
With Commvault Deduplication.

Commvault DDB resides on the Media Agent.

  1. How so we handle DDB and the extended retention of INCR and Full on the Container1?
  2. Will the data on the container1 expire post INCR has reached its retention for 30Days?
  3. Currently we are not sealing our DDB. Do we need to seal the DDB when using Azure Immutable storage?
  4. Can we have one Azure Immutable blob and inside them have multiple folders with multiple retention?
  5. How does Commvault Retention linked with Azure blob Immutable - Access Policy - Retention Interval?
  6. What if Azure Retention Interval on AzureBlobcontainer1 is set for 30Days and My Commvault retention is for 365Days. what happens to the same data on 31st day?
icon

Best answer by Marcel Geisen 1 September 2023, 14:04

View original

6 replies

C
Rank
Userlevel 5
Badge +12

Hello @Kavya 

You can refer to our documentation for WORM on Cloud.

Configuring WORM Storage Mode on Cloud Storage - https://documentation.commvault.com/2022e/expert/9251_configuring_worm_storage_mode_on_cloud_storage.html

  1. Can you clarify what you mean by “handle the DDB”?
  2. The data will prune once the Storage Policy and Azure Lock duration have been met only when the DDB is sealed.
  3. Yes the DDB does need to be sealed at the same interval of the Policy retention. For example if the Policy has 60day retention, the DDB needs to be sealed every 60days.
  4. Azure Retention lock is container-based. You can only have 1 retention per container.
  5. The Azure Retention Lock duration must be set to twice the amount of time as the Policy. For example, the Policy has 60day retention, the DDB is sealed every 60 days, the Retention Lock must be set to 120 days.
  6. The data won’t age until after 365 days and the DDB has been sealed, but after 30 days it can technically be modified.

Note:

1) All of this is automated by the workflow and all you need to do is create the Library\Storage Pool, Create the Copy and associate the storage pool, run the workflow on the storage pool.

2) All Copies associated to the storage pool must have the same retention configured to prevent issues with pruning.

 

Thank you,

Collin

K
Rank
Userlevel 1
Badge +4

Why and how is the DDB sealing linked to immutable backups

K
Rank
Userlevel 1
Badge +4

Thank You @Collin Harper, That was very helpful and in-depth reply

I have a follow up scenario

Primary Copy

AzureBlobContainer1 with Immutable
Primary Copy 30 days incremental with dedupe
Primary Copy Full backup at the end of month with dedupe

The Azure Retention Interval is 30 days
Commvault retention is 30 days

How will the DDB sealing work?

Secondary Copy 

Aux copy the full backup from AzureBlobContainer1 (primary) with dedupe to a AzureBlobContainer2 with dedupe with immutable.

The Azure Retention Interval is 180 days
Commvault retention is 180 days


How will the DDB sealing work?

M
Rank
Userlevel 2
Badge +5

Hello @Kavya ,

I also discussed a lot with Commvault and tested in the last year regarding Immutable Copy in Azure.

At the moment we test concrete following Requirement:

  • 7 Day Immutable Copy → All Jobs
  • Using Object Locking mechanism

In Azure you have the following Requirements for Using Immutable with Object Lock:

  • dedicated Storage Account with
    • Versioning Enabled (all Versions) → no cleanup Rule by Azure
  • Hot or Cold
    • If you have less than 30Days Retention you should us Hot, because on “Cold” you have “Minimum Stay Days” from Azure of 30 Days
    • So we use “Hot”
  • Create a Container with “Enable version-level immutability support”

Important to know about WORM/Immtuable :

  • for each Retention you must have an own Bucket → not mix Retetions in the same bucket
  • Snapshot Copy Jobs not supported (Job based Retention)
  • Micro Pruning is not supported
  • Immutable Object Lock always calculated Retention *2 , this means if you have 14 Days Retention the Immutable Lock is 14*2 = 28 Days
  • DDB Sealing necessary because of Cleaning Date must use Macro Pruning
    • Background is that all referenced blocks in the DDB must be locked up to the final release and thus the aging can take place only in the whole with sealed DDB.

After you added the Immutable Container as CLoud Storage you must run the WOrkflow ”Enable WORM Storage” This WF sets:

  • Disable Micro Pruning
  • Set WORM LOCK Days 2* Retention
  • Sealing DDB to Retetion Days (in Default - can be overwritten)
  • Set WORM Active on Storage Policy associated with Library → no Deletion possible

 

In our Case we added to our Storage Policy a “Worm Copy”  “All JObs” with Retention “7Days / 0 Cycles”

  • Retention Days set correct
  • Object Lock works
  • DDB get sealed as configured
  • Deletion of Data in Container not possible

At the moment we Monitor the Costs in Azure and we have one problem open Index Jobs are not pruned and therefore Sealed DDBs not cleaned up (I asked today here in the forum for help)

 

The most difficult thing is cost calculation, but maybe you can get in contact we Commvault from your side with Manager, they have a Excel Cost Calculator to estimate Cost, but this Calculator so far as I know is not public.

Hope this helps a little bit.

BR

Marcel

 

C
Rank
Userlevel 5
Badge +12

@Kavya 

The Azure Retention Lock would need to be double the duration of the Commvault retention. This is because of deduplication. 

Example: A Job is written on Day1 would age on Day 30. A Job written on Day 30 would age on day 60. Since everything is deduplicated the job written on day 30 depends on the blocks written by the jobs written days 1-29. If retention lock is set to 30days and the DDB is sealed, all the blocks from Days 1-29 become eligible to prune. You would be pruning the data from the job before its retention is over and it wouldn’t be restorable.

DDB sealing is required because we cannot micro-prune the data. Micro-pruning requires the files to be modified as we delete the unnecessary data within the blocks. WORM prevents the files from being modified so we can only prune once all of the data has aged. Sealing the DDB allows it to be prunable.

 

Thank you,

Coilin

M
Rank
Userlevel 2
Badge +5

@Kavya short Update here: we had one problem open with Index Backup Jobs got bot pruned, this was a BUG:

we were running on V11.28 then the fix is included with V11.28.76

 

indexing backups are not aging from sealed DDB

 5065, 5066

 

After installing this fix release, Index jobs got pruned and sealed DDBs were cleaned up.

But have to keep in mind Index Jobs always have different Retention settings.

BR

Marcel

Reply


Terms & ConditionsCookie settings