Amazon MALZ integration with Commvault

@Iggy , I’m going to convert this to a conversation to encourage various ideas and inputs.

Tagging in @Laurent , @Onno van den Berg , @christopherlecky, and @Jos Meijer who often have insightful thoughts!

We do have quite some experience with AWS and environment based on multi-account. So far everything that Commvault supports on AWS is also supported when you use multi-account. There might be some issues when you want to move data between accounts but generally speaking this is most of the times an issue related to permissions or something that is just not possible. Sometimes AWS still needs to build some bridges or Commvault has to implement so functionality to make things work. 

I do however encourage you to run the latest feature release and once a newer version reaches GA to move over as soon as possible. As AWS is moving, so is Commvault,  and so is it pushing a lot of improvements and enhancements in a newer version. To be sure I'm not missing anything else I also tag @mikevg and @Michel Scheepers if they have some additional comments to make.

Also tagging in @Mathew Ericson who is the PM for AWS related functionality. 

 

Thanks for stepping in @Mathew Ericson ! One question from my side → where is the Commvault Public Cloud Architecture Guide for AWS? Can't find it anymore on the documentation side. This document was very helpful! In the past you could find a link on https://documentation.commvault.com/11.26/expert/109587_cloud_feature_support_for_amazon.html


 

Thanks @Iggy but I 'doubt if this is the most recent version. It is based on FR20, so it's already 1.5 years old ;-) 

@Onno van den Berg@Mathew Ericson - Thanks for the responses in terms of MALZ. That is quite a help in clearing up some grey areas for me.

Cheers.

Ignes

@Iggy I see a lot of people already told you it’s possible and no problem…. so I would like to add a bit more technical depth.

Commvault requires a virtual server agent (vsa) to interact with any cloud environment (hypervisor). Now where this agent is installed is the important question and is also highly dependant on your (restore) requirements.

If you only would like to create AWS cloud native integrated snapshots and orchestrate that you could do that from any vsa as long as it can talk to the AWS API’s. If you use an agent outside AWS you need access keys and secret keys per AWS Account.
I would however go a different route and create 2 (Highly available) ec2 instances in your root/services AWS account and set those up to access that AWS account and then setup assume role IAM configurations for all other (child) AWS accounts.
You can enable scale out if you want to be really dynamic in your workloads so your TCO remains lower.

You can find various permissions sets (json) here: https://documentation.commvault.com/11.26/essential/30960_amazon_web_services_user_permissions_for_backups_and_restores.html

All the above is for AWS cloud native integrations, if you want to look at agent based (installing an agent on a ec2 instance to protect something) I would strongly recommend using ‘storage accelerator’ to an S3 bucket in that region and make sure the private S3 endpoint is attached to the VPC (otherwise it will cost you egress on the vpc).
Where you place the DDB is up to you as the only communication left in the storage accelerator scenario is control traffic.

Finally I’d like to mention that a few restore scenario’s, especially across AWS accounts, is under development as it requires ‘sharing of snapshots’ etc. If you have a use case that does not work or is unsupported I’d strongly ask you to open a support case or CMR(/CCR) as that will help all customers.

I hope this helps and if you have any questions, feel free to reach out.

Hello Ignes,

If you have at least 11.20 (they backported this post 11.23 release) and the correct IAM policies in place it would use EBS DIRECT API during backup copies. For this you would need to add the EBS VPC endpoint on the VPC to ensure traffic stays local.

Enabling the S3 endpoint on the VPC is a generic best practice regardless of Commvault. Whether it is agent based backups or vsa backup copies as long as storage accelerator is installed it will talk to S3 locally.

So whether these ec2 instances are MA or not does not matter as long as you ensure storage accelerator is installed the traffic will stay local and the only traffic left is control traffic.

That said the use of EBS DIRECT API for writing and some actions is still under development so there are corner cases Commvault does a hot-add style and creates a new volume from a snapshot and attaches it to the ec2 instance. In my book these scenario’s work OK, the only caveat with this is snapshots (and the volumes created from them) that have marketplace codes attached to them. By design AWS does not allow these to be hot-added to a running ec2 instance and backup copies for those volumes fail if it cannot use EBS DIRECT API and falls back to hot-add style.

Regards,

Mike

Hello Ignes,

I was not aware that AWS launched AMS in a locked down fashion.

From my initial reading on the subject there are 2 modes: accelerate and advanced. In the advanced scenario all current AWS API’s are blocked and any third party, Commvault included, will not work when you want to protect cloud natively as far as I can see.

Personally I get that the ‘advanced’ option might be appealing to certain customers, but when you already have Commvault or want to use Commvault I do not see why you would opt to use the ‘advanced’ mode here.

Can you share some details on this ?