Skip to main content
commvault.com commvault.com
Solved

S3 Object Locking and Versioning

  • January 3, 2023
  • 5 replies
  • 3012 views
S
Bit
Forum|alt.badge.img+1

There is a related topic regarding my question but I don’t feel it was adequately answered: 

 

Essentially there is a conflict in documentation regarding versioning and how object locking works. Firstly the public cloud architecture guide for AWS states that bucket and object versioning is not supported in Commvault (p. 85 of AWS Cloud Architecture Guide - Feature Release 11.25 (commvault.com))

However, when enabling object locking in AWS, versioning is enabled by default on the bucket and objects - it is by design. Under ‘Enabling S3 Object Lock’ section (https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock-overview.html) it states that versioning is enabled.

So if Commvault doesn’t support versioning (and will result in orphaned objects), how can Commvault support object locking, which also enables versioning?

We have also tested this in the lab and can confirm when object locking is enabled, we do not see any data prune from the cloud library. We have run the workflow, stores have sealed and as far as Commvault is concerned, data is gone. However, the cloud library grows and grows.

Best answer by Josh Perkoff

Hello @ScottN 

 

Sorry the information about versioning support is above that section.  What I was intending to inform you is how we would go about deleting the versions.

 

If the data is Deduplicated, the versions will not be deleted until the entire DDB Engine has sealed and is deleted.

 

If the data is non-Deduplicated, the version is deleted once all reference headers are also pruned with normal data aging.

 

If you would like greater clarity in this documentation regarding versioning, please feel free to submit a Documentation Feedback request referencing this conversation.

 

As for versioning without WORM, by default no CommVault will not clean up versions without WORM pruning or a DDB Macro-prune being triggered.  If the data is non-deduplicated, a macro-prune cannot be forced and thus a life cycle rule would need to be implemented.  If the data is deduplicated, sealing the DDB will force a macro-prune once all jobs age.

 

 

View original
O
1 person likes this
  • O
    Orazan
Did this answer your question?

5 replies

J
Vaulter
Forum|alt.badge.img+6

Hello @ScottN 

 

Thank you for reaching out on this question.

 

The document referenced is for a version of the product that is End of Life.  As of 11.28 and beyond we support deletion of versioned objects for WORM storage as outlined in the Data Aging and Pruning section of this document (see the Results header):

https://documentation.commvault.com/2022e/expert/9251_configuring_worm_storage_mode_on_cloud_storage.html

 

For any release pre 11.28 you would have to set up a life-cycle rule manually to delete the versioned objects.

 

Let me know if you have any other questions.

 

Regards,

 

Josh

S
Bit
Forum|alt.badge.img+1
  • ScottN
  • Author
  • Bit
  • 2 replies
  • January 3, 2023

Thanks for the info - we were drawing conclusions around the lifecycle policy in S3 as that would be our only option. It’s disappointing the lifecycle rule requirement didn’t make its way into documentation prior to FR28 (I certainly couldn’t find anything) as it’s a significant issue if data cannot prune.

I also can’t find your reference to the fact that versioned objects are now deleted in the link you mentioned. A complete copy and paste is below. Am I missing something?

Finally, will Commvault properly clean up versions if versioning is enabled on the bucket but we don’t also use the WORM workflow / object locking? Just curious as some customers enable versioning by ‘accident’.

 

  • Data aging and pruning (both the object level, and bucket / container level) will be performed as follows:

    • For deduplicated data, the data will be pruned from the cloud when the DDB is sealed and all the jobs in the DDB are aged. By this time, the WORM retention time in the cloud vendor side will expire, so the deletion will be allowed.

      For non-deduplicated data, the data will be pruned from the cloud when the job is aged

J
Vaulter
Forum|alt.badge.img+6
  • Josh PerkoffVaulter
  • Vaulter
  • 34 replies
  • Answer
  • January 3, 2023

Hello @ScottN 

 

Sorry the information about versioning support is above that section.  What I was intending to inform you is how we would go about deleting the versions.

 

If the data is Deduplicated, the versions will not be deleted until the entire DDB Engine has sealed and is deleted.

 

If the data is non-Deduplicated, the version is deleted once all reference headers are also pruned with normal data aging.

 

If you would like greater clarity in this documentation regarding versioning, please feel free to submit a Documentation Feedback request referencing this conversation.

 

As for versioning without WORM, by default no CommVault will not clean up versions without WORM pruning or a DDB Macro-prune being triggered.  If the data is non-deduplicated, a macro-prune cannot be forced and thus a life cycle rule would need to be implemented.  If the data is deduplicated, sealing the DDB will force a macro-prune once all jobs age.

 

 

S
1 person likes this
  • S
    ScottN
S
Bit
Forum|alt.badge.img+1
  • ScottN
  • Author
  • Bit
  • 2 replies
  • January 3, 2023

Thanks Josh, that’s cleared up the issue for us. Much appreciated.

J
Vaulter
Forum|alt.badge.img+6

Hello Scott, Happy to assist!

Reply


Most helpful members this week

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings