I recently carried out a large delete from a Storage Policy copy, the Total Application Size is bigger than Total Data Size on Disk. I have waited a day before rechecking these stats. I can see there are no pending deletes so am assuming the Commvault pruning cycle has completed.
My object storage carries out the physical deletes once a week. But I would not think this would affect how Commvault displays the data on disk size.
Any ideas why my Total application Size is smaller than Total Data Size on disk (running 2022E or 11.28 for us old farts)
Is the Storage associated with this DDB configured as a Cloud library in Commvault? If so we may not be displaying the correct size on these paths.
So check the following and enable if it’s not enabled, set to 1 to enable - https://documentation.commvault.com/2022e/expert/11022_media_management_configuration_service_configuration.html
The library is an internal S3 Object Store and the setting recommended was already
set to 1.
Thanks for confirming. I see there are 4 partitions for this Engine, can you check each one under this Engine and note the Total number of unique blocks?
Each partition should have ~250 million’ish blocks - let me know if you see any partitions that show much higher from the rest
I’m not a mathematician but I get the feeling one of my partitions has significantly more blocks than the others
The bottom two partitions were added in the last two months and they may explain the size difference.
We are also doing a massive amount of pruning.
Thanks for confirming, we see this behavior when there is a corrupt record in the DDB index.
Can you check the SIDBEngine.log on the Media Agent hosting that 513,806,304 record partition - do a find for the word ISAM.
If you see it, please copy and paste the log lines here. Or feel free to attach the log to the post here and we can confirm.
373302 5b248 03/17 11:27:19 ######## 141-0-317-0 LogErr 945 ISAM Error , ISAM file , UERR , SYSIOCOD 
374781 5b815 03/17 11:38:13 ### 141-0-317-0 LogErr 945 ISAM Error , ISAM file , UERR , SYSIOCOD 
375731 5bbca 03/17 11:48:14 ######## 141-0-317-0 LogErr 945 ISAM Error , ISAM file , UERR , SYSIOCOD 
376298 5be04 03/17 11:54:14 ### 141-0-317-0 LogErr 945 ISAM Error , ISAM file , UERR , SYSIOCOD 
377218 5c194 03/17 11:59:12 ######## 141-0-317-0 LogErr 945 ISAM Error , ISAM file , UERR , SYSIOCOD 
Yep, that is what I had suspected - there’s a corrupt DDB index that’s preventing the pruning of records on this partition. To fix this, you’ll need an hour or 2 of downtime on the MA hosting this partition.
The DDB will need to be re-indexed, this is a command line operation. Stop the CV services on the MA in question before running it.
See the doc above and, ignore the steps other than the Re-indexing section:
You can track the Re-inedxing process via the Command line output. Once the reindexing is completed, start the CV services back up and run a DDB Space reclaim job with Clean Orphan Data selected:
Once that completes all of the space should be reclaimed and data should be pruning normally moving forward.
Let me know how it goes