Total application Size is smaller than Total Data Size on Disk

Hi @DaveK 

Is the Storage associated with this DDB configured as a  Cloud library in Commvault? If so we may not be displaying the correct size on these paths.

So check the following and enable if it’s not enabled, set to 1 to enable - https://documentation.commvault.com/2022e/expert/11022_media_management_configuration_service_configuration.html

The library is an internal S3 Object Store and the setting recommended was already

set to 1.

Thanks for confirming. I see there are 4 partitions for this Engine, can you check each one under this Engine and note the Total number of unique blocks?

Each partition should have ~250 million’ish blocks - let me know if you see any partitions that show much higher from the rest

I’m not a mathematician but I get the feeling one of my partitions has significantly more blocks than the others

513,806,304
 33,120,917
 10,643,339
 10,598,259

The bottom two partitions were added in the last two months and they may explain the size difference.

We are also doing a massive amount of pruning.

Thanks for confirming, we see this behavior when there is a corrupt record in the DDB index. 

Can you check the SIDBEngine.log on the Media Agent hosting that 513,806,304 record partition - do a find for the word ISAM.

If you see it, please copy and paste the log lines here. Or feel free to attach the log to the post here and we can confirm.

373302 5b248 03/17 11:27:19 ######## 141-0-317-0 LogErr            945  ISAM Error [4], ISAM file [4], UERR [0], SYSIOCOD [0]
374781 5b815 03/17 11:38:13 ### 141-0-317-0 LogErr            945  ISAM Error [4], ISAM file [4], UERR [0], SYSIOCOD [0]
375731 5bbca 03/17 11:48:14 ######## 141-0-317-0 LogErr            945  ISAM Error [4], ISAM file [4], UERR [0], SYSIOCOD [0]
376298 5be04 03/17 11:54:14 ### 141-0-317-0 LogErr            945  ISAM Error [4], ISAM file [4], UERR [0], SYSIOCOD [0]
377218 5c194 03/17 11:59:12 ######## 141-0-317-0 LogErr            945  ISAM Error [4], ISAM file [4], UERR [0], SYSIOCOD [0]

Yep, that is what I had suspected - there’s a corrupt DDB index that’s preventing the pruning of records on this partition. To fix this, you’ll need an hour or 2 of downtime on the MA hosting this partition. 

The DDB will need to be re-indexed, this is a command line operation. Stop the CV services on the MA in question before running it.

https://documentation.commvault.com/11.24/expert/137434_converting_transactional_deduplication_database_to_non_transactional_deduplication_database_procedure.html

See the doc above and, ignore the steps other than the Re-indexing section:
 

You can track the Re-inedxing process via the Command line output. Once the reindexing is completed, start the CV services back up and run a DDB Space reclaim job with Clean Orphan Data selected:
 

Once that completes all of the space should be reclaimed and data should be pruning normally moving forward.

Let me know how it goes