Skip to main content
commvault.com commvault.com

I'm trying to wrap my head around the following. I'm setting up VM-level protection in a VMware environment and I'm using the VCD hypervisor because we want to deliver VM-level protection to our cloud customers. So I configured a vCenter hypervisor first and added the VCD hypervisor afterwards, which leverages the vCenter hypervisor. Right now there are 4 active clusters with a few others coming soon. I deployed access nodes on all clusters and they are all listed as access node. Our backup method should be hot-add, because we'll have to close down/limit connectivity between clusters because of security, so NBD will not be an option anymore. The "first" access node in the list acting as the communicator is able to communicate to all other access nodes via a network topology, no errors what so ever to be seen when it comes to the communication between the nodes.

Now when setting the transport mode to hot-add all jobs so far failed with the error that the access node cannot mount the disk, which after having a more closer look makes sense because Commvault picked the wrong access node. It (randomly) selects access nodes from remote clusters to backup VMs. This is silly of course and shouldn't be the case, because it can very easily make the correct decision which access node to pick based on information from VCD and vCenter to perform the hot-add operation. 

When i change it back to auto it perform NBD backups (currently this is still possible) and sometimes it assigns the correct access node which results in a hot-add backup being created.

I'm curious to hear from other customers using VMware Cloud Director with Commvault what their experience is.... 

PS: If you are using the VCD plug-in then you should have a look in the software store. They just released a new plug-in with a different, more native/modern VCD look-and-feel.

Hmmz odd…

I configured it in a different order, but should basically work the same.

I have:

  • Configured the pseudoclient for vCloud Director.
    This asked for the workload vCenter and configured it automatically with the credentials I provided.
  • Configured a VSA on each host within the workload domain, these VSA's have connectivity towards vCloud Director, the workload vCenter and all hosts within the workload environment.
  • Enabled communication between VSA's and configured them in a client group which I then added to the vCenter config access node list, allowing me to remove the seperate VSA nodes in the list for more flexibel management of involved VSA's.
  • Configured Hot-Add.
  • Tested backup and works as expected..

Curious to see what your coordinator VSA shows in the log whilst selecting the corresponding VSA to a VM which needs to be backed up.

So whatever I do it still randomly picks the access node. I checked the communication from the access node to all related component (vCenter, VCD, Commvault backend and other access nodes). Sometimes when it selects the correct one than it works, but I'm unable to steer it and this should also not be necessary because it should be done automatically.

So I decided to do a small test using the vCenter hypervisor which is consumed by the VCD-hypervisor. Added some VMs who are spread across various clusters and configured the transport mode to hot-add. Started the backup and it worked flawlessly! This means it has something to do with how the information from VCD is pulled in and how it is processed. 

@Jos Meijer are you also using VCD organization hypervisors? which version are you running right now? 

 

 

VCD organization hypervisors, what do you mean by that? vDC's?

VCD version = 10.3

I was referring to the following capability, which creates a bridge between a Commvault company and a VCD organization. 

It allows the company users to configure their VM groups themselves and makes sure the VCD VMs are pulled into the CommCell with strict segregation (multi-tenancy) in place. 

This results in:

  • Regular vCenter Hypervisor.
  • A VCD hypervisor which creates a pseudo/copy vCenter Hypervisor of the one that was selected during the creation. 
  • A VCD "Company” hypervisor.

 

Yes I use that 🙂🙂

Edit: lol just noticed I used dutch language in my previous response. Corrected that now 😇

Mmmmmmmm ok. We run on FR28.17 and we will move to FR28.26 next Monday. The access nodes re based on the FREL OVF templates as provided by Commvault. The VMware environment is based on the latest 7 update 3x version, with VCD running on 10.3.3.2.

We currently have 6 ESXi clusters and more are being enrolled. Each cluster contains at least one access node and we have validated, by performing a test backup using the vCenter backup, that it automatically selects the right access node depending on the location of the VM. Once we move upwards in the chain and use the VCD Organization hypervisor the it just distributes the jobs towards the access nodes causing backups to fail when being set on hot-add explicitly. Sometimes it picks the right one and it performs a backup….

We are running 11.24 on latest patch for this environment, will double check tomorrow to see if I have weird behaviour. But haven't noticed it so far whilst using the organization hypervisor. Will let you know

Hi @Onno van den Berg 

I have created a new VM group for one of the Organization based Hypervisor configs, selected a random VM, configured Hot-Add and started a backup.

Initially during the discovery phase the VM admin job assigns the workload domain vCenter as the host and selects a VSA, then the host changes to the actual ESXi host.
The backup then completed successfully.

Checked logs to verify the underlying settings/procedures.
Can see the correct transport mode, identifying the VM via workload domain vCenter, assigning the VSA and retrieving data from correct VSAN datastore and ESXi Host.

All working here as it should.

 

I opened a ticket and received the following update from support:

This is a known issue that we have seen before with vCloud. - vCloud backups are actually dispatched to VSA Proxies using round-robin method, since they do not (currently) share the same dispatch logic as VMware vCenter based Backups.

In the documentation it does state "The VSA proxy coordinator assigns virtual machines to proxies for backup using round-robin logic." here: https://documentation.commvault.com/2022e/expert/101051_backing_up_virtual_machines_for_vmware_cloud_director.html

There is currently a CMR open with Development to correct the dispatch logic for vCloud assignment to VSA on the same ESX host. Ref 360653.

I can add your CommCell ID to this CMR for tracking if required here?

I just can't imagine how MSPs protect customers data who use VCD and deliver self-service VM-level data protection in large multi-cluster environments. This would mean a lot of us have to rely on NBD. 

 

As discussed with Onno directly, our environment differs from Onno's environment.
To support the CMR I have requested to be added as well, this solution should work the same as a regular vCenter config.

Received an update from support that the fix which adds support for hot-add in environments with multiple clusters is in testing phase for FR28!

I can confirm that the enhancement for VCD hypervisors enabling hot-add support across multiple cluster is about to the added to the upcoming maintenance release for FR30. 

Reply


Terms & ConditionsCookie settings