Solved

Coordinator proxy rules in VSA backup

  • 8 April 2022
  • 6 replies
  • 398 views

Badge +4

Hello,

Can I know what is the process or algorithm of VSA coordinator proxy node on how it load balances between the VSA proxy server in the list .. Any link or explanation would be helpful.. I have seen cases where 20-25 VSA proxy nodes are used in very large environment… Hence would like to know how the proxy nodes are spanned across each VM during the backup process..  

 

Thanks

 

icon

Best answer by Prem 26 May 2022, 14:23

View original

If you have a question or comment, please create a topic

6 replies

Userlevel 5
Badge +8

Hi Prem,

Please refer here

https://documentation.commvault.com/11.24/expert/32272_vm_dispatch_and_proxy_selection_for_vmware_backups.html

https://documentation.commvault.com/11.24/expert/97216_backup_workload_distribution.html

https://documentation.commvault.com/11.24/expert/31115_performance_tuning_for_backups.html

 

Regards

Gopinath

Badge +4

Hi @Gopinath ,

Thanks, I went through this link already but couldn’t find what I was looking for.. Missed  to update this above.. The reason I was looking for this in detail because I have a case where there is 17 VSA proxy node and customer is complaining on multiple creation/deletion snapshots when backup is triggered, ideally in the last attempt of snapshot backup gets successful.. However I am trying to look on vsbkp log on specific VSA proxy where the first snap was attempted and since there is no clue during the first snap which proxy node it took I have to check the vsbkp log in all the proxy to find the which one it would have taken and further review it.. This is quite time consuming. 

 

One of the cases  where VSA proxy was identified manually by checking all the vsbkp log as below for reference..

 

Failed snap attempt that took random node out of 17..Couldnt find this info in GUI or Jobmgr.log. Had to check each node’s vsbkp log to find it.. :::

9701 8631 04/04 20:05:06 4455883 CVMWareInfo::CreateVMSnapshot() - Successfully created Snapshot [snapshot-585185] of VM [KBNCTSQL31N2] Guid [503a3a5f-fd19-d0d8-0ee5-f8e2ae8aaf73] duration [00:00:09]
 

 

 

Then moving forward, it backed up 5 disks out of 8 and failed to open/read disk of the three other disks:

 29701 5c30 04/04 20:13:12 4455883 VSBkpWorker::_BackupVMDisk() - Done Backing up disk/volume [KBNCTSQL31N2.vmdk] [12570329088] bytes.
29701 60aa 04/04 20:13:21 4455883 VSBkpWorker::_BackupVMDisk() - Done Backing up disk/volume [KBNCTSQL31N2_4.vmdk] [2699034624] bytes.
29701 5f39 04/04 20:14:47 4455883 VSBkpWorker::_BackupVMDisk() - Done Backing up disk/volume [KBNCTSQL31N2_3.vmdk] [10012852224] bytes.
29701 5ec6 04/04 20:14:51 4455883 VSBkpWorker::_BackupVMDisk() - Done Backing up disk/volume [KBNCTSQL31N2_2.vmdk] [27045920768] bytes.
29701 6264 04/04 20:15:57 4455883 VSBkpWorker::_BackupVMDisk() - Done Backing up disk/volume [KBNCTSQL31N2_5.vmdk] [22064136192] bytes.
 29701 5daf 04/04 20:11:39 4455883 VSBkpWorker::BackupVMDisk() - Failed to open virtual disk [KBNCTSQL31N2_1.vmdk],  [0x80070306:{CArchiveVirtualDiskFile::Open(535)} + {CArchiveVirtualDiskFile::OpenVmdkFile(580)} + {CVmdkVcb4ExtentReader::OpenDisk(167)} + {CVmdkVcb4ExtentReader::BuildExtentTable(950)} + {CVmdkVcb4ExtentReader::_BuildExtentTable(1005)/W32.774.(One or more errors occurred while processing the request. (ERROR_ERRORS_ENCOUNTERED.774))-Received VMDK file extents for incorrect position}]

 

Later the last snap in which backup was successful, here in below case the identification of proxy was done from GUI virtual machine status itself..

 Created snapshot:

55246 dba7 04/04 22:16:09 4455883 CVMWareInfo::CreateVMSnapshot() - Successfully created Snapshot [snapshot-586200] of VM [KBNCTSQL31N2] Guid [503a3a5f-fd19-d0d8-0ee5-f8e2ae8aaf73] duration [00:00:08]

  

 

And this time successfully backed up all 8 disks:

55246 3638 04/04 22:31:07 4455883 VSBkpWorker::_BackupVMDisk() - Done Backing up disk/volume [KBNCTSQL31N2.vmdk] [12650020864] bytes.
55246 75ff 04/04 22:31:44 4455883 VSBkpWorker::_BackupVMDisk() - Done Backing up disk/volume [KBNCTSQL31N2_2.vmdk] [27053260800] bytes.
55246 a07b 04/04 22:33:15 4455883 VSBkpWorker::_BackupVMDisk() - Done Backing up disk/volume [KBNCTSQL31N2_3.vmdk] [10933501952] bytes.
55246 cd48 04/04 22:35:48 4455883 VSBkpWorker::_BackupVMDisk() - Done Backing up disk/volume [KBNCTSQL31N2_4.vmdk] [2888826880] bytes.
55246 efa5 04/04 22:41:05 4455883 VSBkpWorker::_BackupVMDisk() - Done Backing up disk/volume [KBNCTSQL31N2_5.vmdk] [22114467840] bytes.
55246 5a86 04/04 22:41:53 4455883 VSBkpWorker::_BackupVMDisk() - Done Backing up disk/volume [KBNCTSQL31N2_1.vmdk] [20350763008] bytes.
55246 1c8e 04/04 22:52:58 4455883 VSBkpWorker::_BackupVMDisk() - Done Backing up disk/volume [KBNCTSQL31N2_7.vmdk] [16281239552] bytes.
55246 fa31 04/04 22:54:01 4455883 VSBkpWorker::_BackupVMDisk() - Done Backing up disk/volume [KBNCTSQL31N2_6.vmdk] [20927479808] bytes.
 

Keeping the above thing in mind want to know how shall i figure out the first proxy it took to create/delete snap..

Sorry for the long post.. :)

Userlevel 5
Badge +8

Hi,

Please rise a ticket to get it reviewed with support team to help on this.

Regards

Gopinath

Userlevel 7
Badge +23

@Prem , can you confirm if you were able to resolve this issue?

Badge +4

Hi Team, I have an update as below :

Best way probably would be checking Coordinator Proxy (First Proxy in the list) logs and find the first attempt of the job. it'll tell you which proxy it sent the VM to. Something like:

Sending VM [VM1] to proxy 1

or

Adding VM [VM1]

I verified from the logs and it does gives the above said information.. This might be useful for infra where we are using more than 15 VSA proxy nodes..

Userlevel 5
Badge +8

Hi Prem,

yes, same details and explanation was provided in dispatch logic in below BOL page

https://documentation.commvault.com/11.24/expert/32272_vm_dispatch_and_proxy_selection_for_vmware_backups.html

 

Regards

Gopinath