Best practices for number of proxies for ESX / AHV

Hi @Jarvis 

In terms of Sizing for Hardware of the VSA Proxy, you can refer here: Hardware Specifications for Virtual Server Agent (commvault.com)

I’d suggest calculating how many VM’s you need to backup within your backup window. Also check the transport method you’d prefer to use for backup (HotAdd if Virtual) and ensure you use CBT on incremental backups.

The amount of streams a VSA Proxy can handle depends on the resources available to it.

To give you Files & Folders level recovery for Windows VM’s you’d need a Windows VSA Proxy/MA.
Also check if you need to recover Files & Folders from Linux VM’s, If so then you will need a Linux FREL which you could also use as a VSA Proxy.

Best Regards,

Michael

Maybe you misunderstood the question, or I wasn’t clear in my question.  I understand the clear documentation on the sizing requirements for an individual proxy.  I’m asking about the rule to follow for the number of proxies to deploy.  Is there a general rule of thumb for that?  If I have 1500 guest VM’s, should I plan on 15 proxies?  That’s what I’m looking for.

Hello Jarvis,
In terms of the amount of proxies there really isn't a recommendation that you can say applies to all environments. In general we have the sizing guides to give you a rough idea of what to size a proxy should be. This varies heavily on a few things. 

The first being the number of subclients and streams allocated across the active jobs. If you have ten actives streams but fifteen proxies setup then ultimately your going to have five machines idle as they won't have a stream allocated. So in large part you are going to want to have more streams than the number of proxies. As of 11.16 we default the subclients to five streams so that usually isn't a concern as much.

https://documentation.commvault.com/commvault/v11_sp20/article?p=97216.htm

The transport mode in question matters quite a bit as well for the amount of proxies and the configuration settings on them. Hotadd has limitations in regards to the amount of disks attached vs the amount of controllers added to the machine. A controller has have up to sixteen disks attached so in terms of Hotadd if you have one base controller that proxy can usually handle about five virtual machines mounted at once comfortably. This is assuming the virtual machines on average have about two disks. In general however I would say you would want at least two-three controllers to ensure you have amble room to mount disks.

https://documentation.commvault.com/commvault/v11_sp20/article?p=32048.htm

Something like SAN or NBD this isn't as much of a concern at least from the Vmware side, but Nutanix I believe has similar concerns regarding the amount of disks attached to each proxy if the proxies are housed inside Nutanix.

Ultimately there really isn't a clean answer saying that x number of proxies are fine for x number of machines. In your example of 15 proxies per 1500 machines that is probably a bit low on the number of proxies, but that really depends on how large the machines are, how many parallel backups you have running, the amount of change on a daily basis, and how long the backup window is. Similar to the amount of streams, the number of proxies is a value that you usually have to adjust over time as you expand and shrink.

Regards,
Nickalas
 

This is a bit of a difficult one to give an exact rule to go by, All environments are not the same.

In terms of Sizing VSA Proxies, a VSA can have 10 Streams per CPU and 1 Stream per 100MB RAM.
Ref: Performance Tuning for Backups (commvault.com)

Theoretically if you backed up all of the 1500 VM’s (in FULL) at exactly the same time in a short backup window, you’d need 1500 streams. - So 15x Large proxies in theory “could” cover this Full Backup using SAN Transport, but you would need Media Agents and Storage to be able to handle this many streams at the same time.

If you’re running Incremental backups the the Jobs should have a reduced runtime, therefore you can run more Jobs within the backup window and in turn, require/consume less VSA resources.
 

Things you’d need to consider are:

Backup Method - If using HotAdd there are disk limits per VM/SCSI controller. NBD will mean one connection to the ESX per-disk.

Streams - How many streams can the MA’s/Storage Handle? and how many of these streams are in use during the backup window?
(This can often be a limiting factor, Once a stream is released it can then be given to the next Agent/VM)

Performance - The rate of data transfer from VM Storage to the VSA Proxy (VMware: NBD/SAN/NAS/HotAdd, AHV: NAS/HotAdd)

Time - How long is the backup window?

Data Size - What are the sizes of the virtual machines? What is the rate of change/growth?

Best Regards,

Michael
 

Great suggestions here. FET is definitely the way to go. 1500 VMs could be 15 TB or 150 TB - we stopped using # of VMs for sizing a long time ago as it made it difficult to predict (customers average VM size varies a LOT) - FET is a better measure.

As others have stated, its always ‘depends’ - transport modes play a big role too. For example, if you planned to use hotadd, you could deploy much smaller proxies per ESX host and distribute load that way (commvault will auto-assign the local proxy where possible). If you have physical Media Agents then you would more likely strictly follow the FET guidelines as you’re more likely to have fewer physical boxes around. As always, the FET value is provided as a guide - there is always a generous helping of ‘it depends’, but FET should be a good starting point and should work for the vast majority of deployments.