commvault.com commvault.com
Solved

How does Commvault work with Amazon S3 API - need to calcuate 'retrieval request' operations

  • 17 June 2022
  • 5 replies
  • 540 views
ZachHeise
Rank
Userlevel 2
Badge +9

Hello. We're interested in creating a new MediaAgent (I'll call it MediaAgent03 or MA03 from now on) in Amazon Web Services, using commvault's "combined storage tier" functionality with S3 Glacier Deep Archive being the coldest (and cheapest) possible data option.

This mediaagent would only be used for Data Restores in the event of a catastrophic event in our datacenter where MediaAgent01, our primary onsite MA, is stored. Otherwise, it will sit there quietly in the AWS cloud as our insurance policy.

I was looking at Amazon's pricing page here - https://aws.amazon.com/s3/pricing/ under Requests & Data Retrievals Tab - and was wondering if commvault can please explain or provide documentation as to how to calculate 'retrieval request' operations. I of course understand 'data retrievals per GB' cost because I know in a DR situation I'd be restoring basically everything, 100% from MA03 - about 200TB of data at the moment.

However, what I don't understand is how commvault software interacts with the S3 API and how many operations, billed at $.025 per 1,000 operations, commvault restores take. Is a single file being restored an operation? Is a media path mount an operation? Please help me to understand how I can figure this out so I can report likely operation expenses to my boss. Thanks!

icon

Best answer by Collin Harper 17 June 2022, 16:05

View original

5 replies

C
Rank
Userlevel 5
Badge +14

Hello @ZachHeise 

Thank you for your question. I hope this below link answers your questions.

https://documentation.commvault.com/11.26/expert/118855_cloud_storage_building_block_guide.html

ZachHeise
Rank
Userlevel 2
Badge +9

Hello @Collin Harper - so based on https://documentation.commvault.com/11.26/expert/139246_combined_storage_tier.html - if the backup data in my locally-hosted mediaagent is 100TB right now, I could expect 90TB to live in the cold/archive tier, and 10TB to live in the hot, more expensive storage tier? I am trying to build a price estimate for how much moving to the cloud would cost.

When commvault does a restore operation from a combined storage location, does the data from the archive storage need to be temporarily stored on the hot storage at all, meaning that we would be charged by the cloud storage provider both for the archive tier 90TB, but also a hot storage cost of 90TB, even if it’s just for a few days?

Or does the data from the archive tier get sent directly to my commserve and bypass the hot tier?

C
Rank
Userlevel 5
Badge +14

Hello @ZachHeise 

The amount of data stored in hot vs cold tier is completely based on data vs metadata. We store the metadata in the hot tier so we can readily access the index and so we know what is there, but all the data blocks get stored in the archive tier.

When we do a restore the data is recalled from the archive tier and hydrated into the hot tier so we can read it. Due to the nature of archive storage we cannot directly read from it. Reading data from the archive tier will incur egress charges but it would be best to contact the vendor you are looking to use for better guidance on costs.

Thank you,
Collin

ZachHeise
Rank
Userlevel 2
Badge +9

Thanks Collin. When using Combined Storage tiers with commvault + AWS, am I given the option in a restore to choose “buk” or “standard” as my retrieval type, or does Commvault select one of those automatically and I cannot change it?

C
Rank
Userlevel 5
Badge +14

@ZachHeise You can configure this.

By default, when running a restore for data in the cloud, the restore will report a specific error indicating the data is in archive storage. This error triggers an alert that triggers a workflow. The Cloud Storage Archive Recall can be configured with the retrieval options you prefer. You can also set the retrieval options on a one-time basis if you run the workflow manually to recall a specific Job ID.

Performing On-Demand Recalls from Archive Cloud Storage - https://documentation.commvault.com/11.26/expert/102379_performing_on_demand_recalls_from_archive_cloud_storage.html

Viewing Alerts - https://documentation.commvault.com/11.26/expert/5206_viewing_alerts.html

Reply


Terms & ConditionsCookie settings