CIFS backup with large number of small files

Question

I have a customer who has 15 million files in a CIFS share and the size of each share is 2TB. They have about 111 shares that we need to backup via CIFS in Commvault. Customer has two XL Windows media agents running 11.28.24. Both media agents are also use as access nodes.

We ran a backup with the default setting, and it took about 40+ hours to complete a full backup on a share. We adjusted the data readers from 4 to 25 to 50 and even to 100. We’re still not getting the time we need to meet RPO. Incremental are also taking too long to complete. Last one finished at 17 hours with 25 data readers

Question is what is Commvault’s recommendation with backing up large number of small files for a CIFS environment?

Also SnapDiff is not supported in this environment since they are running ONTAP 9.10.1P5.

Block based and Snapshot are also not an option here.

Thanks in advance!

christopherlecky · Accepted Answer

@christopherlecky, thanks but I think NDMP is just another form of block backup. Due to compliance reason for this customer, block backup is not an option here.

@ak2, SnapDiff v3 is not supported in v11.28. I validated this in my lab and confirmed this with Commvault.

Well is that's the case my suggestion it to use a very large number of sub clients, with the sub clients themselves being based on name differentiation.

So something like

Subclient 1 backs up files that start with a-A

Subclient 2 backs up files that start with b-B

Through to 26.

My suggestion is to break it down much more granularly than this.

christopherlecky · Answer

NDMP is probably a better option in this case with intellisnap and block based backups ruled out.

Reply

Sign up

Login to the community

Scanning file for viruses.

This file cannot be downloaded