AWS EC2 snapshot backup copy performance

@xiwen have you checked you resource utilization on you MAs? you should definitely stick to EBS-Direct-API as this is the most efficient way of performing backup copies in AWS. it does not require you to have access nodes mount the VM disks to be able to perform backup copies but instead it requests the unique changed blocks directly from the EBS API and than copies those blocks directly to the backend storage. 

it could be that the figures as presented are not correct or accurate hence I would first check to see if you are running into a system wide bottleneck like CPU or network interface congestion. 

which version are you currently running? 

Is it a single stream or is it the entire job that is showing a throughput of 50GB/hour? I assume you also checked your DDB latency?  

Have you checked the documentation and do you meet the requirements → https://documentation.commvault.com/2022e/expert/130078_ebs_direct_api_backups_for_amazon_web_services.html

EBS-Direct traffic is a HTTP stream so also make sure you are not passing the traffic through a proxy. 

Otherwise please open a ticket and let them check it out.

Currently I run EBS Direct API for one testing server only, EBS size is 120GB, actually size is about 20GB.

Q1, I don’t see permission request on PutSnapshotBlock from Commvault document, shall we add it on IAM role?

You can run API requests concurrently. Assuming PutSnapshotBlock latency is 100ms, then a thread can process 10 requests in one second. Furthermore, assuming your client application creates multiple threads and connections (for example, 100 connections), it can make 1000 (10 * 100) requests per second in total. This will correspond to a throughput of around 500 MB per second.

 https://documentation.commvault.com/11.26/expert/130078_ebs_direct_api_backups_for_amazon_web_services.html

Q2, I got below word from word from below link, which quota should be changed? GetSnapshotBlock ?PutSnapshotBlock? what is proposal number? or how to calculate it?

The service quota for GetSnapshotBlock requests per account per Region is 1,000 per second by default. To increase the service quota limit, you must open a ticket with AWS.

 https://documentation.commvault.com/11.26/expert/130078_ebs_direct_api_backups_for_amazon_web_services.html

@Mathew Ericson looking at it from a MSP perspective but have you added some clear event notification during the processing of jobs that the software is hitting "walls” that require the end-customer to open a ticket at AWS to have their limits being increased?