Backup on Red Hat Enterprise Virtualization Management (RHEV-M)

we have many customers using that..do you have any specific questions or concerns?

Good,

i have 2 question/problems.
i have two environments rhev-m one 4.1 and one 4.3.
In the first environment commvault send command to rhev-m to delete snapshot, but the rhev-m fails and in some case force restart node. This envirnment is in dismission (backup suspended). In the other env. in some case have the same problem whtout  reboot node, but is not possible delete the snapshot in any way (opened case to redhat). The vm backup whit this snapshot not work.

Thanks Andrea

Hey @ITStorageEWL, sorry to hear that you are having issues.

One quick thing to check - Are you using linux proxies? If so, is lvmetad disabled?

yes, using linux proxies and lvmetad is disabled.

Got it - thank you.

When no backups are running, check the proxy to see if it has any additional disks attached from other VMs - that could cause snapshots to fail to be removed. If so, remove the disks (detach, not delete) and attempt the snapshot deletion again. Beyond that, I know there were some specific issues in early RHV releases that caused this problem, but I am not aware of issues with 4.1+ It will be interesting to hear what Red Hat find?

The vsbkp.log on the proxy should have more info about the snapshot delete requests.

when the backup starts there are no snapshots hanging on the proxy, from the log you can see that the backup starts and wants to delete this snapshot on the rhev-m that you cannot delete in any way.
We opened a case to red-hat, unfortunately they are not as fast as cv support.

3687718 384559 03/08 00:10:52 94675 RhevSDKWrapper::CVRhevOperations::pollForRhevJobCompletion() - job [98218c82-241a-4ad2-8d8c-eb584bbb05ce]:[Removing Snapshot _GX_BACKUP_xxxipa4.it.xxxxxxxx.local_92072_1868119_vsarhevprinfra of VM xxxipa4.it.xxxxxxxx.local] - current state [started] - waiting for [FINISHED]...
3687718 384559 03/08 00:11:02 94675 RhevSDKWrapper::CVRhevOperations::pollForRhevJobCompletion() - job [98218c82-241a-4ad2-8d8c-eb584bbb05ce]:[Removing Snapshot _GX_BACKUP_xxxipa4.it.xxxxxxxx.local_92072_1868119_vsarhevprinfra of VM xxxipa4.it.xxxxxxxx.local] - current state [started] - waiting for [FINISHED]...
3687718 384559 03/08 00:11:12 94675 RhevSDKWrapper::CVRhevOperations::pollForRhevJobCompletion() - job [98218c82-241a-4ad2-8d8c-eb584bbb05ce]:[Removing Snapshot _GX_BACKUP_xxxipa4.it.xxxxxxxx.local_92072_1868119_vsarhevprinfra of VM xxxipa4.it.xxxxxxxx.local] - current state [failed] - waiting for [FINISHED]...
 

Hey @ITStorageEWL - We’re the vendor to beat on support response ;)

Did you get anywhere with Red Hat? from the logs it does look like a failure to remove on their side, so I think you did the right thing reaching out to them.

@ITStorageEWL , following up on @Damian Andre’s note.  Any update from Red Hat support?

For this moment only steps to manual remove snapshot, we are waiting for root cause.

Appreciate it.  I’ll check back with you occassionally to see (barring you beating me to the punch  )

@ITStorageEWL , gentle follow up to see if Red Hat support were able to give you a solution yet.

Understood, and thanks for the reply!  Are the backups now completing?

Yes, backup working without errors.

Hi Mike, we had experienced this issue as well in my environment. Unfortunately for us we cannot do file system agent backup because it’s quite a lot of VMs(about 500) in RHEV. Has commvault found any solution to this? When we ran RHEV VM backups, the snapshots didn’t get deleted causing the VMs to freeze and quite a messy service interruption. Manual deletion of snapshots is quite a task for all the VMs. Is there any known issue on this.

Hi @Shirley , what version of rhev-m do you have installed?

@Shirley With vm backups with VSA on rhev-m 4.1 old infrastructure and rhev-m 4.3 new, we had to change the vm type from thin to thick. This is because of how red hat handles snapshots.

Hi @ITStorageEWL We are running 4.3.10. So we have to do this for all the VMs? We have about 500 of them. Are the RHEV VM backups successful now?

Also which Commvault Service Pack are you on? Previously support had told us it’s a service pack support problem but I don’t think this was the cause. I’m currently on 20.53, I last attempted the backup while on SP17.

11.20.32

we save 250 or so VMs with VSA on rhev, now we have no problems. We certainly allocated a lot more space this way. Make some tests.