after upgrading from V11FR20 to V11FR24 I noticed a new schedule policy named “System Created DDB Space Reclamation schedule policy” which was disabled by default.
I basically know what the Space Reclamation functionality is about, and the policy has all our deduplication engines assigned. But when initializing this policy it finished in less than a quarter of an hour and from the logs only one dedup engine was processed. Another manual start just gives below error message.
Can anybody explain to me what this schedule policy is about and how it is supposed to work?
Best answer by jgeorgesView original
Space reclamation is a feature only for storage libraries using NAS shares that do not support sparse. ‘Drill holes’ means the ability to take a 1 MB file, and purge a 128K block from the middle of it - in order to do that, the file system needs to support sparse so that the file system understands there is a free block in the middle of the file. Without sparse, we can only delete the ends of a file, but nothing from the middle.
On those types of systems, space reclamation will read the chunk and re-create it without the middle bit, allowing space to be free’d up. There are some minimum thresholds to allow the job to run successfully.
It looks like in your case, your disk library supports sparse, OR you do not have enough fragmentation (i.e chunks with enough blocks in the middle that need to be purged) for space reclamation to run. There is a threshold because reading and rewritign the chunks is a taxing operation, so you don’t want to run it all the time, and Commvault will wait until there is enough blocks to free to make the operation ‘worth running’. You can read more about it here - it also explains how you can adjust that threshold.
To add to Damians comment, you can actually check the drill holes support from the Library within the Java GUI. Against each Mountpath there is a check box for ‘Sparse Support’ which should be checked if Commvault believes we can drill holes.
The Watermark or Threshold that triggers the space reclamation is actually dependent on the low watermark threshold set at the Library Properties:
The fragmentation Damian refers to is also important though, as space reclamation can run at 4 levels:
20%, 40%, 60% or 80% unused blocks.
I believe the system created schedule runs at default Level 3 (40% unused data blocks within a file). If none of your files have atleast 40% fragmentation than the reclamation will complete without processing any files.
thanks for the information provided. It helps me to better understand what is going on and what influences apply.
Personally I do not give too much on the checkbox because it shows confusing information in our environment. Like libraries on the same type of NAS system sometimes being shown as supporting Sparse and sometimes not. Even worse it shows for e.g. a library with 3 mountpaths on the same NAS system different information for the mountpaths.
I will play around a bit with the the settings to get a better understanding when the function will be triggered and when not.
Sounds good, keep us posted if you have any further questions!
If those MP’s were created at different times, ie. additional MPs added as the environment grew?, than this might explain WHY some show sparse support and others do not.
I know that when we create a MP, from SP20, we have specific logging within CVMA.log which creates and tests the drill holes functionality. If this fails, we clear the ‘sparse support’ flag. The test was there before SP20, but not entirely sure how much further back.
In eariler SPs (before the sparse support test), we would assume sparse support was there, until drill holes failed. At which point we’d stop attempting to drill holes.
You can check the SIDBPrune.log to confirm if ‘Drill Holes’ is occuring OR if you are receiving errors when we attempt to, something similar to:
You can also check with the hardware vendor if they support Sparse Data and if they support the specific Windows APIs:
Alternatively, log out a ticket with Support and we can check logs on session togther.