commvault.com commvault.com
Question

Failed Backup VMWare Consolidation

  • 17 October 2023
  • 12 replies
  • 574 views
T
Rank
Badge +3

We currently have the problem that the backup is no longer running on 2 VMs. Hard drives need to be consolidated. If this is done, 1 - 3 backups will run through and then it will run into the same error again.

Error Code: [91:399] Description: The virtual machine [cs-at00-201] reports that it has virtual disks that need consolidation and this could not be performed automatically. In order to prevent a further buildup of delta disks and possible stability issues with the virtual machine, further backups of this virtual machine will not be performed until this condition is corrected. Check if there are any locks on the virtual disks or if the disks are still attached to a backup proxy. []

 

This all happened after the update to 11.28.86. Before the update, the jobs ran without errors. The consolidation only affects the VSA Agent and one other VM

 

Automount on the VSA Proxy is enabled.

12 replies

A
Rank
Userlevel 2
Badge +3

Few things to check

  1. Make sure VM hardware version is latest.
  2. Make sure VMware tools are running and up to date in status.
T
Rank
Badge +3

Few things to check

  1. Make sure VM hardware version is latest.
  2. Make sure VMware tools are running and up to date in status.

Checked. All up to date.

Onno van den Berg
Rank
Userlevel 7
Badge +19

Could be related to a fix that was part of MR85 see → 

Did you already try the following:

  • Perform consolidation
  • Remove all snapshots from the VM
  • Rerun the backup job

I have a few VMs in my environment FR30 who experienced something similar and managed to fix it by applying previous steps. 
 

T
Rank
Badge +3

@Onno van den Berg we've already tried that. One or two jobs then run through but then this error occurs again.

Rajiv
Rank
Userlevel 4
Badge +11

Hello @turbo1781 

After the backup fails or gets to a point where it is stuck, can you please check /vmfs/volumes/<vm-datastore>/<vm-name>/vmware.log ? If there are locks getting generated, this log will be helpful. 

Secondly, you mentioned automount is enabled? As per my understanding, it should be disabled. Can you re-confirm?

To end with, what is the version of vCenter and VDDK? They both should of same version, you can check vsbkp.log on the proxy/access node and check the VDDK version in it. 

Best,

Rajiv Singal

T
Rank
Badge +3

@Rajiv now we have disabled automount on the VSA Proxy. 

Rajiv
Rank
Userlevel 4
Badge +11

@turbo1781 please use automount scrub as well as it will remove some stale mounts if any.

Best,

Rajiv Singal

T
Rank
Badge +3

@Rajiv unfortunately the same game again. a job ran through and now backup failed again. what else could be the cause here?

 

We also have other machines that run without problems and, as I said, there were no problems before the update.

Rajiv
Rank
Userlevel 4
Badge +11

Hello @turbo1781 Did you happen to check  /vmfs/volumes/<vm-datastore>/<vm-name>/vmware.log ? 

Also, what is the version of vCenter and VDDK? They both should of same version, you can check vsbkp.log on the proxy/access node and check the VDDK version in it.

Best,

Rajiv Singal

T
Rank
Badge +3

I have now moved the Access Node to a different schedule. Unfortunately, that only achieved limited results. Now the backups run about 5 times and then the error occurs again. There are always other machines involved.

S
Rank
Userlevel 6
Badge +14

@turbo1781 

You can log a case with VMware & Support so they/we can check.

Best Regards,

Sebastien

J
Rank
Userlevel 1
Badge +2

Maybe related:

 

Reply


Terms & ConditionsCookie settings