Hi @liorkrispin 

Thank you for the question.

Checking expected behaviour: Snapshot Cleanup, we would expect these VMware snapshots to get cleaned up after a successful backup.

Perhaps you can provide some more information to help:

What version of Commvault software are you running on the Commserve, MA and VSA clients?

Are backup jobs completing successfully for each VM?

Thanks,

Stuart

Hi,

our CommCell version is 11.20

MA - 11.9

regarding VSA version, can you help me to find it?

Hi @liorkrispin 

Thanks for the additional info.

If your Commserve is running 11.20, please ensure the Media Agent and VSA client, which is the VSA proxy used to perform VMware backups, are both updated to the same version.

Please take a look at CommCell Environment Upgrades for help on upgrading your Commcell.

It’s important that infrastructure clients are on the same version, which might just resolve this issue.

Thanks,

Stuart

Please check the events (view events) on the job as there may be more information there. Also, check the tasks & events on the VM in vCenter where the snapshot remains. In 99% of cases there is an issue on the VMware side removing the snapshot(s), since we only send the request, but VMware needs to process it.

Sometimes snapshots are removed but the VM fails to consolidate. This means its logically removed from VMware, but VMware has not performed the physical consolidation of the data between the snap and the live VM data - this is a background process and generally takes several hours to complete for a large VM. Unfortunately if there is ever an issue with snap consolidation then VMware rarely retrospectively fixes it, and you could end up with a lot of unconsolidated snapshots that has to be manually cleaned up.

If you see a snapshot consolidation error you can also try this VMware KB to see if it will help.

I definitely echo what @Stuart Painter said in regards to versions. Sometimes old versions of the VDDK (software from VMware that Commvault uses as a framework to do the backups) could have bugs that result in snapshot consolidation issues - best to get the MA/VSA up to the same level as your CommServe (11.20).

Fix the version diff and make sure you run the latest FR20 maintenance release. Personally I would even consider moving to FR24! 

My personal experience has been that when commvault abandons snapshots its because the vCenter is not responding in a timely fashion.

there is a specific event id generated when this happens.

i don’t remember the Id off the top of my head but ill look for it.

As @Damian Andre  said this is because commvault makes the request but cannot guarantee that vmware will honor it.

in the past wherever I have seen this I would run a daily a report that generates a list of vms for the corresponding event ids and forward them to your friendly neighborhood vmware admin.

I spent about 10 minutes yesterday looking for the event ID, in Commvault’s error list. Turns out there are an absolute ton of event errors related to snapshots.

I didn't have the time to go through them or look through old notes.

Just look in your job history and examine events and you should find it.