Following best practices is important for any aspect of our product, but this is especially important with regard to the Deduplication Database. A well-specced Media Agent that meets and exceeds our hardware recommendations hosting a DDB that is configured properly can achieve incredible feats of space savings and backup performance. Conversely, without proper planning and optimal configuration the performance and storage usage can be negatively impacted.
Since the inception of deduplication in Commvault one of these Best Practices has always been that Extended Retention should not be used on a DDB being used for a Primary Copy. To prevent a DDB from becoming too ‘bloated’, our recommendation has been to segregate long-term retention into its own DDB, leaving only Basic Retention on your Primary Copy. Although it may appear obvious that longer retention means a larger DDB, the true reason for this may not be immediately apparent.
Prior to DDB V5 (or DDB V4 GEN2 depending on who you ask), the Secondary Table Files contain 16 Archive Files (which we will call ‘afiles’ for short). These afiles in many cases will be affiliated with different Job ID’s. In order for any of the files in the Secondary Table to become obsolete and get deleted, ALL 16 ‘afiles’ must be eligible for deletion.
A Primary Copy using only short-term retention will be creating and subsequently deleting Secondary Table files at regular intervals, which keeps the DDB lean and fast. How about if we mix-and-match short term and long term retention? Adding Extended Retention into the scenario, some of those afiles will be kept for a much longer period of time, preventing those secondary files from being deleted. The result is that more and more secondary files will be created as new Primary backups are written, but those files will get deleted very infrequently due to the long-term retention. The result is the DDB will continue to grow in size which will consume expensive SSD disk space, as well as eventually slow down performance.
The above scenario is no longer true in an environment running DDB Version 5, as we’ve changed the way that the Secondary Table is structured. Whereas in older versions each file in the Secondary Table contained 16 afiles (which may correlate to multiple jobs), in DDB V5 1 file = 1afile. This seemingly small change has a massive impact here, and allows for short term and long term retention to co-exist on a single DDB without resulting in unnecessary bloat. As short-term retention jobs meet their retention and age, we have the ability to clean up the corresponding files in the DDB immediately once they are no longer needed.
If you are interested in learning more about the advancements made in DDB V5, I urge you to watch this incredible video created by our Training Department that goes into much greater detail about our latest DDB platform :
Have an older version 4 DDB and want to leverage these newer features without sealing and starting over? No problem! We’ve created a Workflow that can be used to upgrade your existing DDB :
Have some additional questions? Be sure to reach us on Community and we will be glad to help!
