@Gilles SCHMIDT , can you confirm where the snaps are located? In prior incidents with this message, we found that the snaps are aged in Commserve database - Since the setup is based on OCUM, we only mark the snaps for expiry in OCUM, and deleting on snap is handled by it.
Intellisnap is configured with Open Replication (Ocumless). What’s weird is that Intellisnap snapshot SP_xxxxxx_xxx became common snapmirror snapshot in Snapmirror relationship and cannot be deleted (see below the output of snapshot show command).
The snap involved became a relationship snapshot on the snapmirror destination :
s-nsg-fssnap::> snap show -snapshot SP_2_530332_19758_1651518145 -instance
Vserver: vserver_name
Volume: Volume_name
Snapshot: SP_2_530332_19758_1651518145
Creation Time: Mon May 02 21:02:34 2022
Snapshot Busy: true
List of Owners: snapmirror
Snapshot Size: 399.5MB
Percentage of Total Blocks: 0%
Percentage of Used Blocks: 0%
Comment: -
7-Mode Snapshot: false
Label for SnapMirror Operations: IntelliSnap
Snapshot State: -
Constituent Snapshot: false
Expiry Time: -
SnapLock Expiry Time: -
CV Logs :
Failed to delete snapshot for FileServer:svm_name.domain_name] Volume:s/vol/volume_name/Qtree_name]. SnapName:iSP_2_530332_19758_1651518145]. Error returned:13012]This Snapshot copy is currently used as a reference Snapshot copy by one or more SnapMirror relationships. Deleting the Snapshot copy can cause future SnapMirror operations to fail. ]
CV has been upgraded from 11.20.46 to 11.24.43 to solve 2 issues :
- Tapes not recycled when retention is expired
- snapshots not pruned as well as CV and Netapp). After upgrade, snapshots have been pruned on CV but not on Netapp. (Case number 220506-276)
The issues are not resolved.
@Gilles SCHMIDT , as I was reading, I was going to suggest opening a case, though thankfully you already have one.
I’ll follow 220506-276 for progress on the combined issues.
@Mike Struening Maybe an issue on Netapp, I have requested Ontap logs to our final customer after Data Aging. Do you know whether the issue on “snapshot never pruned” is with retroactive effect or not after patch upgrade ?
More than likely not retroactive, though I’d need to check. Do you know the specific patch that was involved (within the Maintenance Release you were told to upgrade to)? I can look it up and try and get more detail.
Here is the reply from Wisam regarding the hotfix :
For this issue, we have a fix in Hotfix 4081 and it's part of SP24.21
NetApp snapshot deletion may complete without deleting the snapshot if replication status is snapshot transfer in progress (SM_REPL_STATUS_TRANSFER_IN_PROGRESS) | 4081 |
So the hotfix SP24.34 is also good to go.
We have applied a newer version : 11.24.43
Thanks, @Gilles SCHMIDT ! I found the form ID (3089) and reviewed the notes.
It looks like the adjusted behavior is only going forward.
Hello .Mike,
The issue related to “reference snapshot” has been solved by an update of the snapmirror relationship from Ontap : snapmirror update -destination-path …. -source-snapshot …. But it’s not a good thing because CV is not aware that the snapshot is already transferred. But the issue with snap retention and snap transfer still occurs since CVSnapEngine tries to transfer 3 snapshots which are not listed in Ontap (snaphot show) as well in CV (list of snapshots). An idea to resolve this?
In attachment, list of snapshots + logs CVReplicator….
@Gilles SCHMIDT , I would let the case owner know about this. I don’t have any initial ideas...
@Mike Struening here is a small description as a workaround proposed:
As per the secondary copy "2.Snapvault" these jobs are showing as partially copied and on the Snap Primary it is showing as available however, snapshots are not listed as they are no longer present.
Solution (I prefer to use the work “workaround”):
You can mark these jobs as "Do not copy" from the secondary copy and bad from the primary copy so that it will not be referenced.
I hope this workaround will help Commvault community.
Bye for now.
Thanks for sharing @Gilles SCHMIDT ! Definitely agree the community will find this useful.