AWS EBS Direct Read API denied

Hi @Anthony.Hodges 

Here are some things to check, courtesy of the Support team  :

Is EBS direct API supported in the AWS region?

https://docs.aws.amazon.com/general/latest/gr/ebs-service.html

Is ec2:DescribeInstanceAttribute enabled?

There are 4 permissions needed, you have mentioned 3, please also check ec2:DescribeInstanceAttribute.

Amazon Web Services Permission Usage

Please check the permissions have been granted specifically to this AWS region.

The access node needs access to the EBS endpoints.

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-accessing-snapshot.html

Changed Block Tracking for Amazon

Thanks,

Stuart

Is EBS direct API supported in the AWS region?

Yes, ap-southeast-2.

Is ec2:DescribeInstanceAttribute enabled?

Yes.

Please check the permissions have been granted specifically to this AWS region.

In my testing we used Resource *, but can try narrowing to this region.

The access node needs access to the EBS endpoints.

The EBS vpc interface endpoint has not been set up yet whilst doing small VM testing, but it "should" have access.

https://documentation.commvault.com/11.22/expert/130078_direct_read_backups_for_amazon.html

Direct read backups use the EBS service, and requires the following AWS permissions:

• ebs:GetSnapshotBlock

• iam:SimulatePrincipalPolicy -- Is this permission added too?

Thanks and yes "iam:SimulatePrincipalPolicy” is set too.

Couple of other things to check @Anthony.Hodges 

1. Make sure any previously configured registry/additional settings keys for this feature are removed - “bAmazonDirectReadEnabled” under “VirtualServer”
    
2. Try to disable the check itself with additional setting (it seems that in some cases, API requests could be throttled and therefore giving it a false return that permissions are missing)
   
   Name: bAmazonSkipDirectReadPermissionsCheck
   Path: VirtualServer
   Type: Boolean
   Value: true

Ah, sorry I accidentally hit the answer button and I haven't tested it yet. 🤪 Perhaps, a confirmation dialog would be useful for us intermittently cross-eyed/uncoordinated people.

Thanks @Damian Andre I will try these.

@Anthony.Hodges , unmarked for you!

I have a support ticket open and it was stated that CV calls simulate-prinicipal-policy for some operations. I suspect that there are additional permissions required to call this API (Testing IAM policies with the IAM policy simulator - AWS Identity and Access Management (amazon.com), & IAM: Access the policy simulator API - AWS Identity and Access Management (amazon.com)).

For example, the simulate-principal policy gives an implicit deny to any of the permissions granted, for example S3 list all my buckets.

# aws iam simulate-principal-policy --policy-source-arn arn:aws:iam::11111111111:role/VSA-Role-Name --action-names s3:ListAllMyBuckets

{

    "EvaluationResults": [

        {

            "EvalActionName": "s3:ListAllMyBuckets",

            "EvalResourceName": "*",

            "EvalDecision": "implicitDeny",

            "MatchedStatements": [],

            "MissingContextValues": [],

            "OrganizationsDecisionDetail": {

                "AllowedByOrganizations": false

            }

        }

    ]

}
 

CV Support provided that we can do commands like list snapshot blocks, despite the implicit deny.

# aws ebs list-snapshot-blocks --snapshot-id snap-041e89c92ab0745a3 

{

    "Blocks": [

        {

            "BlockIndex": 0,

            "BlockToken": "AA8BAQ96rUz...

Hi @Anthony.Hodges 

No worries at all, these discussions are all valuable, showing troubleshooting steps and ideas for the Community to check and follow if anyone else sees something similar in future.

I’ll go ahead and mark your post as Best answer.

Thanks,

Stuart

Absolutely!  This is no bother, it’s a solid conversation and a great source of information thanks to you, @Anthony.Hodges !