getting started with AWS and commvault


Userlevel 2
Badge +8

We are starting a new project with everything in AWS and planning to use commvault as our backup system via AWS.   So Im still a novice with commvault and Im wondering when deploying the commvault image to an instance is it like everything wrapped up in one instance.?  Such as the commserve, media agent etc…

Or do I need to deploy various instances for each.?  Do I need a server for commservr and another one for media agent etc…?

 

 


19 replies

Userlevel 4
Badge +6

We have a single image to get started here in the AWS marketplace:

https://aws.amazon.com/marketplace/pp/Commvault-Backup-Recovery-BYOL/B0725QYS9Q

 

You can certainly start with a minimal image and scale up as your data volumes increases.

Commvault is made of of a central command and control server (CommServe), a data mover component (MediaAgent) and one or many Access Node (which talk to your workloads)

 

Your MediaAgents can be separated out as data gets larger and just powered down automatically to save cost

https://documentation.commvault.com/11.22/essential/101313_cloud_mediaagent_power_management.html

 

Your Access Nodes can be automagically created by commvault during backup and then automagically deleted after the backup completes - so nothing to manage there at all :)

https://documentation.commvault.com/11.22/essential/111430_automatic_scaling_for_amazon_access_nodes.html

Userlevel 4
Badge +9

Hi BC1410,

You can start with one instance all in one! If you need more media agents and different plans then you can add more instances of course. 

https://aws.amazon.com/marketplace/pp/Commvault-Backup-Recovery-BYOL/B0725QYS9Q

https://aws.amazon.com/marketplace/seller-profile?id=88cecb14-a8b2-49bd-ba1f-58be76108f48

Userlevel 2
Badge +8

Mathew and Alireza Thanks for both the Replies.   Great info for me.. as Im still familarizing my self with AWS and Commvault..     

 

I was hoping to use the Backup and Recovery BYOL but its on 2016 server…  According to the higer ups in my organization they want everything running on 2019 server. 

 

So if I was to deploy a 2019 server in AWS how would I go about getting commvault on it..  And can I still have everything on a single instance.   We have purchased 2 commvault license for backup and recovery for 10 Virtual machines.    How would I partition the single instance on a 2019 server.   I guess I mean what partitons/ drives are needed to have everything on one 2019 aws instance 


Thanks again 

BC

 

 

Userlevel 7
Badge +23


So if I was to deploy a 2019 server in AWS how would I go about getting commvault on it..  And can I still have everything on a single instance.   We have purchased 2 commvault license for backup and recovery for 10 Virtual machines.    How would I partition the single instance on a 2019 server.   I guess I mean what partitons/ drives are needed to have everything on one 2019 aws instance 

 

Hey @bc1410  - It should be straight forward! - provision an EC2 machine in AWS and then grab a Commvault installer from https://cloud.commvault.com - for 10 machines, this looks like a small deployment and I think you can go with these specs from the cloud architecture guide.

 

 

I wasn't sure if you had an on-premises deployment already. If you do, you can likely have a smaller machine in the cloud that acts as the Media Agent and virtual server proxy only, and connect it to your existing CommServe deployment. This will also allow you to leverage the power management feature that @Mathew Ericson mentioned to save on costs. But if you have a requirement to run it all in the cloud, then this is the way to go (or, way to start). One of the great things about cloud is the flexibility to change the instance to meet your needs as you go, without having to rebuild anything.

Userlevel 2
Badge +8

Thanks Damian - 

 

We are doing everything in the cloud.  We dont have a on-premises commvault environment.   Thanks for your Reply…  I might have more questions once I get started..  Thanks again

Userlevel 2
Badge +8

Why is it I dont see any info for performing a “regular” commvault install on a AWS Instance?  All I see if information that tells you to use the commvault AMI in the AWS environment.   I guess they rather you just use the AMI which make sense..  probably much easier and its probably already set up and partition etc..   Our dilema was 2016 server..   We need 2019..  Also how does commvault VSA work in a nuttshell.   Sorry for all the novice questions.   This Forum has been great so far..  I definitly get better answers then I do from google…  Thanks again 

Userlevel 4
Badge +9

I don’t think we have 2019 in Market place. Generally speaking we do support 2019 though.

In terms of VSA, a quick answer is: VSA proxy sends virtualization related APIs to AWS to create snapshot of each instance volumes. Then creates volumes from those snaps and attach them to the VSA proxy. Now we can read those disks and after that we will detach and delete them.

For more details:

https://documentation.commvault.com/commvault/v11/article?p=30992.htm

 

Userlevel 7
Badge +23

Why is it I dont see any info for performing a “regular” commvault install on a AWS Instance?  

 

Hey @bc1410 - Once the instance is deployed, it's all the same; same installer, process etc. Is there something specific you were looking for?

Userlevel 2
Badge +8

Hey Damian - Nothing special..  Im probably just second guessing myself on the the install…  I just downloaded the Media Kit Feature Release 11.22 Comvault.   

I guess I was just looking for straight forward instructions for installing Commvault on a single AWS instance - with what option to select etc..

We had a commvault on site which was 2 servers..  one acted as a media server and the other was commserve.  So with this AWS instance I will have everything on one server (commcell | media agent | DB  etc)

So I will proceed with the install on this AWS instance.  I slated 3 drives.   C:\ drive as 200GB  ;   D:\ drive as 200GB for the dedpulication and E:\ drive for 400GB for the Index Cache.   

We will use S3 Buckets for our Backups.   Am I missing anything else as far a drive partition layout

 

Thanks again for all the Replys - been a great Help

BC

Userlevel 7
Badge +23

Hey Damian - Nothing special..  Im probably just second guessing myself on the the install…  I just downloaded the Media Kit Feature Release 11.22 Comvault.   

I guess I was just looking for straight forward instructions for installing Commvault on a single AWS instance - with what option to select etc..

We had a commvault on site which was 2 servers..  one acted as a media server and the other was commserve.  So with this AWS instance I will have everything on one server (commcell | media agent | DB  etc)

So I will proceed with the install on this AWS instance.  I slated 3 drives.   C:\ drive as 200GB  ;   D:\ drive as 200GB for the dedpulication and E:\ drive for 400GB for the Index Cache.   

We will use S3 Buckets for our Backups.   Am I missing anything else as far a drive partition layout

 

Thanks again for all the Replys - been a great Help

BC

Hey @bc1410 -

 

Apologies for getting to this late. I don’t think you’re missing anything! Looks good to me. You may want to have additional space for the software cache, but you can always resize the drive later if needed (gotta love cloud).

With Commvault it is as easy to install an all-in-one as it is to separate out the roles.

You may end up using less index cache space because VSA does not do very granular indexing by default - so keep an eye on that as you maybe be able to reduce the size, later on, to save a tiny bit on storage.

Userlevel 2
Badge +8

Thanks Damian - you and everyone else has been a great help.

you probably figured I was going to ask another question.  So our environment will be small..  we will have a handful of AWS windows servers and a handful of AWS linux servers and like one AWS MS SQL server and maybe a AWS Oracle instance.   

MY question is with dedplication.  I know I slated a D:/ drive for the deduplication.  And I have read that the deduplication should not be stored on a s3.   So if I create my cloud pool or cloud library that points to s3 bucket(s) (still learning) and  we want to use global deduplication I just have to set the path to my D: drive.  Essential the D: drive is cloud storage also since our whole environment is in AWS so when I read https://documentation.commvault.com/commvault/v11/article?p=93056.htm it states that the deduplication should reside on a disk library or its recommeneded to be on a disk library.  Not sure what they mean since everything is in the cloud for us.

Userlevel 7
Badge +23

Thanks Damian - you and everyone else has been a great help.

.

You are most welcome! This is why we are here :grinning:

 I know I slated a D:/ drive for the deduplication.  And I have read that the deduplication should not be stored on a s3

 

The good news is that its not possible to store the deduplication database on s3 - its a database that needs a file system, so when you create the deduplication policy, place your ‘DDB’ (deduplication database) on the D:\. The backup library can absolutely be a cloud library pointing to S3 - no issue with that. The DDB gets used to store signatures and identify duplicate blocks. Its not needed for restore so if you lose it later on, there no issue recovering your data (although its backed up automatically anyway). The link you posted is referring to the automatic backup of the DDB - as long as its not archive class (glacier) then you are good.

You dont want to make a disk library in the cloud on local disks attached to an EC2 instance - that would cost you WAY more than S3, and dare I say less reliable :thinking: .    

The recomended s3 type for most workloads is S3-ia, which will save you a bit on costs vs traditional hot tier.

 

So deduplication database, put it on d:\

Setup a cloud library with an S3-IA bucket

Userlevel 2
Badge +8

Thanks for the great info and knowledge Damian.   Since Im a rookie you know I have more questions.

 

Now this might sound like a dumb question but oh well..  I mentioned before we will have a hand ful of aws windows instances and aws linux server instances to backup along with 1 or 2 MS SQL Always ons servers via our AWS commvualt instance.  So when I go to add a new client.  Im assumming Im treating each server as a client?  correct or am I way off?  Do I just select Windows filesystem type Windows for the AWS windows Server 2019 servers we are wanting to backup?  or ?  

 

Thanks again  BC

 

Userlevel 4
Badge +9

So when I go to add a new client.  Im assumming Im treating each server as a client?

Correct. Each of your Servers will be a Client for CV Commserve to protect. Based on the data you need to protect you will pick an agent and in this case WFS agent as you mentioned. 

Userlevel 6
Badge +15

It all depends if you need granular backup/restore (then file level/database level) or full VM protection (then VSA).

With granular backup methods, each client is a server/VM. 

In VSA mode, then the VSA client is by default the Hypervisor, so in your case your whole AWS environment, and in this case you can have all the VMs grouped inside this client. But you can always create ‘subclients’ in this client, where you would have your VMs one by one. This can be interesting if you want to have different backup schedules between all VMs, for instance (but not only). 

Userlevel 7
Badge +23

This thread is absolutely awesome.  I have to imagine the next person to come along trying to put a CommCell in AWS will get all of their questions answered thanks to the collective wisdom on display here!!

Userlevel 7
Badge +23

Yes the thread has been super helpful for me.  Cant thank eveyone enough for everyone sharing the commvault knowledge.

 

This question - Im not sure if its a AWS question or commvault.   
We wanted to add a AWS S3 bucket as our Storage Cloud library.  In doing so we kept on getting “access denied”  when attempting to add the aws s3.  The way we were finally able to add our S3 bucket to our commvault commserve environment as a cloud library was to DISABLE the AWS S3 bucket encryption option.  We used the AWS IAM ROLE Policy to add the S3 bucket.   Does anyone know why we cant have the encryption on the AWS s3 bucket enabled.  I feel like we should have the encytpion enabled but of course it didnt work when adding the S3.

 

Thanks again  -  BC

I’m going to split this out into a new topic so we can track the answer separately.  Nothing you need to do!

 

Userlevel 7
Badge +23

Hey @bc1410 , I’ll split this question off into a new thread so we can track the answer better.

Userlevel 7
Badge +23

Moved here:

 

Reply