Question on Horizontal scaling

Hi ​@mach123 ,

When horizontal scaling is enabled for a Deduplication Database (DDB) in Commvault, it primarily impacts how data is handled on primary copies. Once a DDB reaches a predefined threshold (such as size or number of jobs), Commvault automatically creates a new DDB, and new subclients begin using this new DDB for backups.

For auxiliary (aux) copies—which don’t involve subclients directly—the behavior differs slightly:

Existing DDBs : Auxiliary copies continue to use the original DDBs (e.g., DDB1) to read and copy data that was backed up before the scale-out occurred. These DDBs remain active as long as data associated with them exists.

New DDBs : When a new DDB (e.g., DDB2) is created due to horizontal scaling, any newly backed-up data will be written to this DDB. The aux copy process will automatically utilize the appropriate DDB (in this case, DDB2) to read and transfer the new data to the secondary storage.

Seamless Transition : The transition from old to new DDBs for auxiliary copies is automatic and managed by the system. Commvault intelligently determines which DDB to use based on the original backup job’s association—ensuring that the right deduplication data is read during the copy process.

Jobs backed up using DDB1: Auxiliary copy uses DDB1 to read deduplicated data and copy it to secondary.
Jobs backed up using DDB2: Auxiliary copy uses DDB2 for reading and copying.

No manual reassignment is needed, and deduplication remains effective throughout the process.

Overall, the auxiliary copy mechanism seamlessly adapts to changes in the DDB structure, maintaining deduplication efficiency and simplifying data management across both primary and secondary copies.

Regards,
Dheeraj

Thanks Dheeraj,

A follow up question. say you create a new copy using a new dedupe engine which has horizontal scaling from the start.  This will create a DDB for filesystem, VM and database separately.

The question is say your source copy is an old DDB which was just converted to horizontal DDB. There will be 4 DDBs in there filesystem, VM , database and original. With original containing most of the records/unique blocks prior to conversion 

When I initiate an aux copy will Commvault seamlessly create the record for each data type on the specific DDB on the secondary copy ( e.g. filesystem backup goes to filesystem DDB and so forth and so on ).  Is this assumption correct?

Hi ​@mach123 ,
 

Yes, that's correct. When using a horizontally-scaled secondary dedupe copy, Commvault automatically routes each Auxiliary Copy Job to the appropriate DDB partition based on workload type—filesystem, VM, or database—even if the source includes both original and horizontally-scaled DDBs.

Commvault uses job metadata (such as client type, subclient, and backup set) to determine the correct DDB for each job. This ensures that data is efficiently deduplicated by directing it to the corresponding DDB partition. Even if the source DDB contains mixed data types, Commvault handles the separation automatically as part of its horizontal scaling functionality.

Regards,

Dheeraj