Skip to main content
commvault.com commvault.com
Solved

Does anyone have a write-up on DDB Version 5?

DaysAndCycles
Byte

Hi!

Commvault FR 11.20.    We’ll be upgrading to FR 11.24 in the Fall.    

We’ll also be standing up new Media Agents, with new DDBs.   

Is anyone aware of a write-up on DDB Version 5?   All I can find are references to upgrading from DDB Version 4 to Version 5 (though it doesn’t tell you how to identify what version you’re currently using).  

https://documentation.commvault.com/11.24/expert/134342_upgrading_deduplication_database_to_v5.html

I also can’t find any references in it in the release notes, though I think it first shows up in FR11.20.   

It appears to be far more scalable than previous versions.   I just want to read an explanation of how it works. 

 

Thanks!

 

 

Best answer by Blaine Williams

Hi mikeymac, 

V4 Gen 2 also known as V5 has been from 11.14. 

With the introduction of the second generation of V4 deduplication, the table structure was modified for both the primary table and the secondary table files.

Primary Table
In the previous deduplication generation, the performance was impacted due to an increase or decrease of numerous counters when a backup job was running or aged out. This made the I/O process intensive. With DDB V4 Gen 2, counters are removed. When aging out data, secondary table files are scanned to create the list of obsolete blocks.

Secondary Table Files
In the previous deduplication generation, the DDB secondary table was built on files containing up to 16 archive files (AF). This means that as soon as the 16th archive file was written in the secondary file, the secondary file closed and a new one was created. This process continued creating as many secondary table files as were needed. In the new generation of deduplication, each archive file is now written as a standalone secondary table file.

Structure Change Advantages and Additional Improvements
The structure changes that were introduced in V4 Gen 2 improved many aspects of the deduplication database (DDB) operations:

  • DDB reconstruction restartability
  • Concurrent reconstruction for multiple DDB partitions (optional)
  • Growth management
  • Faster full and synthetic full backups
  • Garbage collection (optional)
  • Faster reconstruction (optional)
  • Memory increase
  • NVMe SSD optimization

 

View original
Did this answer your question?

5 replies

B
Vaulter
Forum|alt.badge.img+8

Hi mikeymac, 

V4 Gen 2 also known as V5 has been from 11.14. 

With the introduction of the second generation of V4 deduplication, the table structure was modified for both the primary table and the secondary table files.

Primary Table
In the previous deduplication generation, the performance was impacted due to an increase or decrease of numerous counters when a backup job was running or aged out. This made the I/O process intensive. With DDB V4 Gen 2, counters are removed. When aging out data, secondary table files are scanned to create the list of obsolete blocks.

Secondary Table Files
In the previous deduplication generation, the DDB secondary table was built on files containing up to 16 archive files (AF). This means that as soon as the 16th archive file was written in the secondary file, the secondary file closed and a new one was created. This process continued creating as many secondary table files as were needed. In the new generation of deduplication, each archive file is now written as a standalone secondary table file.

Structure Change Advantages and Additional Improvements
The structure changes that were introduced in V4 Gen 2 improved many aspects of the deduplication database (DDB) operations:

  • DDB reconstruction restartability
  • Concurrent reconstruction for multiple DDB partitions (optional)
  • Growth management
  • Faster full and synthetic full backups
  • Garbage collection (optional)
  • Faster reconstruction (optional)
  • Memory increase
  • NVMe SSD optimization

 

MichaelCapon
YYZ
2 people like this
  • MichaelCapon
    MichaelCapon
  • YYZ
    YYZ
DaysAndCycles
Byte

Thanks for your reply, Blaine Williams!   

It’s good to know how your software works, and I was having a dickens of a time trying to figure out what version of DDB I’m using in my environment (I still am).    Our new ones will obviously be Version 4 Gen2/ AKA Version 5, but what of our existing ones?    I still haven’t located this answer, but there must be an indicator squirreled away in the interface somewhere.   I think I’ll open a support ticket for that item.  

On a similar thread from a few months ago (which I was able to locate thanks to your revelation of the original name of DDB Version 5), a Damian Andre posted the following informative video from around the 11.14 release describing the differences in DDB Version 4 Gen2/AKA DDB Version 5.

https://commvaultondemand.atlassian.net/wiki/spaces/ODLL/pages/501350567/Tech+Talks?preview=/501350567/501022936/DDB%20V4%20Gen%202.mp4#TechTalks-DeduplicationGeneration4Version2

Thanks for your help!

B
Vaulter
Forum|alt.badge.img+8

Hey mikeymac, 

I wasn’t entirely sure that site was public. Check out this page https://commvaultondemand.atlassian.net/wiki/spaces/ODLL/pages/358580314/Deduplication+V4+Gen+2+V5

Glad to have helped

DaysAndCycles
Byte

One more DDB version question.   I can’t find any indication of a DDB Version 4 or DDB Version 4, gen 2 or DDB Version 5 in the software anywhere.    But I do see “DDB Version 11” on each Dedupe database (as seen via the Storage Resources section of the CommCell Console). 

The procedure for compacting DDBs references the following:  

Supported DDB versions:

v10 DDBs on v11 MediaAgent

v11 DDBs on v11 MediaAgent        

That documentation can be seen here for 11.24:

https://documentation.commvault.com/11.24/expert/12613_compacting_deduplication_database.html

Might “v11 DDB” be  another name for “V4 Gen2/AKA Gen 5”?

Thanks again for the information!

Mike Struening
Vaulter
Forum|alt.badge.img+23

Not exactly.  A DDB created in Version 11 is known as a V4 DDB.

The V5 is the garbage collection feature.  If you want to enable garbage collection, then use the Convert DDB To v5 workflow detailed below:

https://documentation.commvault.com/11.24/expert/108822_optimizing_deduplicated_database_ddb_pruning_garbage_collection.html

https://www.linkedin.com/in/michael-struening
DaysAndCycles
B
2 people like this
  • DaysAndCycles
    DaysAndCycles
  • B
    Blaine Williams

Reply


Most helpful members this week

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings