Solved

out of place restore AWS

  • 30 April 2021
  • 9 replies
  • 309 views

Userlevel 4
Badge +14

Hello,

 

I try to restore an AWS instance using the out of place option.

I have modified only the name of the instance and leave all default settings.

I am restoring on the same Availability zone.

I have this error message

does anyone have some experience on AWS restorations ?

 

vol-064d51e6343cc439b-Volume type [gp3] could not be used for volume [vol-064d51e6343cc439b]. Volume has been restored using volume type [gp2].; vol-061d8c0b7a9efc2da-Volume type [gp3] could not be used for volume [vol-061d8c0b7a9efc2da]. Volume has been restored using volume type [gp2].; IAM Role [ce-ec2-standard-role] does not exist and cannot be restored to [myinstance-restore]

 

thank you !

icon

Best answer by Mike Struening RETIRED 30 July 2021, 18:01

View original

9 replies

Userlevel 4
Badge +8

Hello Bloopa!

Can you ensure that the IAM Role “ce-ec2-standard-role” is attached to the VSA EC2 instance that you are using for the restore in the AWS console?

How to attach IAM Role to EC2 Instance

If this has already been completed, can you verify that this IAM Role has the proper permissions LISTED HERE

 

(Note: The JSON permissions do change after Feature Release updates. Be sure you have the latest JSON)

Userlevel 4
Badge +14

Hello,

Sorry to reply again on this post but, the new instance (out of place) is created by commvault. how to attach the IAM role again to the instance while it is not yet  created on aws ?

 

 

Thank you !

Userlevel 2
Badge +4

Hello Bloopa,
I believe what Dan White was referring to is to attach the IAM role to the VSA proxy/EC2 instance that is handling the restore, not to the created Instance from the restore.

When you’re doing an out of place restore for AWS instances, when you say default option, is there an IAM role being associated to the instances when you’re doing restore?  If so I would set that to default or blank if possible.

Would you be able to send a screenshot of the options you’re selecting?

Userlevel 4
Badge +14

Hello, thank you for the clarifications.

Here is step by step what I do to restore the full instance (out of place)

 

 

 

 

I am using another instance-name

 

added inteface and security groups

 

 

 

if I continue the restore, the instance is restored with the error message I sent upper. if I click on the IAM role list, I got his error message.

 

 

Thanks !

Userlevel 2
Badge +4

Hello Bloopa,

Thank you for the update and screenshots!  So the IAM role under instance settings will attach that specific IAM role onto those new instances that were created, which if it does not exist within the region you’re restoring the Instances you can see that failure in attaching the IAM roles on the newly created instances.

The proxy client that is specified at the beginning of the restore is the machine that will basically handle all the API work into Amazon, so you need to verify and make sure that the IAM role attached to it has all the proper JSON permissions as recommended by our documentation.

Amazon Backup and Restore JSON permissions:

https://documentation.commvault.com/11.23/expert/30960_amazon_web_services_user_permissions_for_backups_and_restores.html

Also CVD on the proxy you specified should contain the failure when you hit the drop down list for IAM roles on the instance setting, it maybe able to provide us with the reason why we’re failing to get Virtual Machine destination list.

It may be beneficial to generate a support ticket if you’re hitting more hurdles since we may need to have a deeper log dive.

 

-Chi Miao

Userlevel 4
Badge +14

Hello,

 

I had opened a case 210526-163 

Here is what I have on the CVD log from the media agent

 

 

cvd.log 5636 8d0 05/26 10:16:19 ##### AmazonCompute::GetRoleInfoForRegion() - Exception - Amazon.Runtime.AmazonServiceException: A WebException with status ConnectFailure was thrown. ---> System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 52.94.225.3:443

 

Thanks !

 

 

Userlevel 4
Badge +14

I have no answer for the moment from Commvault support but the media agent try to reach the endpoint ec2 through its public IP.

Our media agents can’t connect to internet for security reason.

 

We had this kind of issue on other media agents which uses the STS assume role endpoint.

Commvault has deployed a patch to allow us to force the media agent to use the private IP of the endpoint.

 

the patch is now official and included in 11.22.25 and onwards

for example here is the host file of a media agent that use sts assumerole     

10.8.13.42 sts.amzonaws.com

 

I think we need to do the same to force the MA to reach the ec2 endpoint through its private IP.

I’ll keep you informed.

 

 

 

Userlevel 7
Badge +23

Hey @Bloopa !  hope all is well.

Following up on the incident updates.

I’ve consolidated the updates below for ease of review:

Dev had asked if you can confirm the user has this permission “iam:GetRole". You will have to check the iam role which you are using to give permissions to the proxy. Then in console, find the iam role and view it in json format. In the json format, check if "iam:GetRole' is present under actions.

Do not check the iam role that is failing to restore. You need to check the iam role that is controlling permissions for commvault.

Can you please refer to this documentation from AWS on permissions?

https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_change-permissions.html

Userlevel 7
Badge +23

Adding in the latest:

Hope you are well. Dev have confirmed that we need to give * to resource in order to fix this. They have explained the reason as below.

 

'Yes, we need to give '*' for resources to make it work. This is because before restoring an IAM role we first query the role using GetRole.

Currently, under resources the customer has "arn:aws:iam::*:role/CF_BackupRestore". So, we will have permission to call GetRole only on resources matching that template arn.

In this case, the customer is trying to restore "arn:aws:iam::697502770565:role/ce-ec2-standard-role" which does not match the template arn and hence we do not have permission to call GetRole on the IAM Role.

 

The customer just needs to give all resources for action iam:GetRole, they need not change resources for iam:PassRole.

The customer need not worry about giving ‘*’ under resources for ‘iam:GetRole’, because we cannot modify any role using that permission. It is just used to query details about a role.'

 

Let me know if you’ve been able to get this configured.

Thanks!!!

Reply