Skip to main content
commvault.com commvault.com
Solved

DDB Verification and Aux Copy Jobs impact on DDB disk

E
Byte
Forum|alt.badge.img+9

Could the DDB Verification and the Auxiliary Copy jobs impact on DDB disk in terms of Query & Insert time (the calculation)? We have some Database's Average Q&I Times across Partitions in the last 3 and 14 days with very high values. 
Backups jobs are running fine. No Performance degradation during backups with a good Deduplication ratio. There are no significant events. Most of the time we’re doing Database Log backups (So, no dedup here) and running Admin Jobs to copy data onto tapes or to create secondary copies. At night, we're doing incremental backups (VMware and FS agents). We have an issue with Synthetic Full backups (https://community.commvault.com/commvault-q-a-2/synthetic-full-is-not-running-based-on-automatic-schedule-settings-2744) and I'm wondering about how this issue also can impact on those times. 

Some info:

VMware VM Media Agents are running updated Linux. 8 CPU cores, 32 GB RAM. DDB Flash disk. Kernel Parameter Configuration done. 

DDB Verification: Incremental DDB Verification (not quick), 3 Streams. It runs for aprox. 5 hours. Sometimes it runs for 10 or more hours.

Auxiliary Copy: Allow Maximum Number Of Streams. Use Scalable Resource Allocation. 

Best answer by christopherlecky

Eduardo Braga wrote:

I’m doing it at this moment and I’ll share results here.

Another question: can I schedule a quick incremental verification daily and a regular incremental verification once per week? Is there a bit of strong advice against that? 

Personally , I would push the verification schedules out, CommVault primary is goal is to protect your data, but the truth is that auxiliary copies will typically catch %90 of the potential corruptions issues that you will run into since it physically has to copy that data to another location. Because of its incremental nature it will not catch issues that are currently on disk and so the ddb verification becomes necessary.

 

that said in situations such as lost power or some kind of storage outage that is when you should definitely do a verification. 
 

Typically there will be warning signs of impending storage problems that the verifications are meant to catch, meaning within your logs you may see a storage adapter flapping or host disconnects, assuming your media agents are well monitored for this kind of thing, you can save yourself some cycles by toning down the aggression of the verifications.
 

the default settings of aggressive verification checks are meant for set it and forget it scenarios but once your org gets sufficiently large you have to put a bit more thought into it.

View original
Did this answer your question?

6 replies

christopherlecky
Byte
Forum|alt.badge.img+17

Under most circumstances the short answer is no, at least for auxiliary copies. 

DISCLAIMER 1

Please note that the views, thoughts, and opinions expressed in the text belong solely to the author, and not necessarily to the author's employer, organization, committee or other group or individual.

 

The query and insert times are primarily a function of the number of records in the DDB.

As the number of records increase the Query and Insert times degrade but it is not linear as it approaches the maximum number of records Q&I times fall off a cliff. 

Typically your options for lowering Q&I is to 

  1. Get faster DDB disks.
  2. Lower the amount of data being retained
  3. Split up your DDBS.

There is a DDB calculator floating around that can give you a good idea what to expect, but that is most likely a document that changes regularly and as such you should probably talk to your CommVault (tm) rep about what numbers to expect.

 

Disclaimer 2

de-duplication is a complex topic and I have zero idea about your underlying configuration. 

 

 

I blame auto-correct for all misspellings, grammatical errors, and half-baked opinions.
Mike Struening
1 person likes this
  • Mike Struening
    Mike Struening
O
Vaulter
Forum|alt.badge.img+15
  • OrazanVaulter
  • Vaulter
  • 630 replies
  • May 3, 2022

When Q&I times show as slow, the first thing we do is test to make sure the IOPs for the disk that the DDB resides on are within specifications.  Do do this, please run iometer against the DDB disk as shown here:

https://documentation.commvault.com/11.24/expert/8825_testing_iops_of_deduplication_database_disk_on_windows_01.html

 

Update this thread with the output please.

E
Byte
Forum|alt.badge.img+9

I’m doing it at this moment and I’ll share results here.

Another question: can I schedule a quick incremental verification daily and a regular incremental verification once per week? Is there a bit of strong advice against that? 

O
Vaulter
Forum|alt.badge.img+15
  • OrazanVaulter
  • Vaulter
  • 630 replies
  • May 4, 2022

There shouldn’t be a need to do both.  I would just do the quick verification as it will find any issues that may have occurred since the last verification. 

christopherlecky
1 person likes this
  • christopherlecky
    christopherlecky
christopherlecky
Byte
Forum|alt.badge.img+17
Eduardo Braga wrote:

I’m doing it at this moment and I’ll share results here.

Another question: can I schedule a quick incremental verification daily and a regular incremental verification once per week? Is there a bit of strong advice against that? 

Personally , I would push the verification schedules out, CommVault primary is goal is to protect your data, but the truth is that auxiliary copies will typically catch %90 of the potential corruptions issues that you will run into since it physically has to copy that data to another location. Because of its incremental nature it will not catch issues that are currently on disk and so the ddb verification becomes necessary.

 

that said in situations such as lost power or some kind of storage outage that is when you should definitely do a verification. 
 

Typically there will be warning signs of impending storage problems that the verifications are meant to catch, meaning within your logs you may see a storage adapter flapping or host disconnects, assuming your media agents are well monitored for this kind of thing, you can save yourself some cycles by toning down the aggression of the verifications.
 

the default settings of aggressive verification checks are meant for set it and forget it scenarios but once your org gets sufficiently large you have to put a bit more thought into it.

I blame auto-correct for all misspellings, grammatical errors, and half-baked opinions.
S
E
2 people like this
  • S
    Stippett
  • E
    Eduardo Braga
christopherlecky
Byte
Forum|alt.badge.img+17

I forgot to add this disclaimer.

All of this contextual to your environment, so if you are unclear on anything I suggest be sure to run it by your TAM so you don’t get into trouble monkeying around with default settings intended to protect you.

 

Thanks.

Chris.

 

I blame auto-correct for all misspellings, grammatical errors, and half-baked opinions.
Mike Struening
1 person likes this
  • Mike Struening
    Mike Struening

Reply


Most helpful members this week

Terms & ConditionsCookie settings

Cookie policy

We use cookies to enhance and personalize your experience. If you accept you agree to our full cookie policy. Learn more about our cookies.

 
Cookie settings