Highly fragmented library/DAS - solutions?

Just for clarification, are you using Secondary Copies / Auxiliary Copies within Commvault for your DR copy, or some other process?

I'm seeing some confusion in what I'm reading.
I'll start by saying that I've always used deduplication on Commvault disk libraries. Is this also the case for you?
I imagine the Windows server has NTFS-formatted volumes, and that Commvault uses these volumes to store data. Typically, Commvault writes blocks of 128KB, if I'm not mistaken. For NTFS, it's talking about blocks of multiples of 512bytes (not MB). https://www.blueskysystems.co.uk/about-us/knowledge-base/windows/ntfs-max-partition-size-limits
The issue of blocks and defragmentation might be relevant if there were only a few large files within the CV libraries that needed to be read sequentially. This doesn't seem to be true for Commvault, which writes many small files.

If I understand correctly, copy the files written to the filesystem to LTO tapes.
To make data copies in Commvault, you don't need to physically copy the library files; instead, you use AuxiliaryCopy to transfer the backup data to tapes: Commvault will read the data from the backup and transfer it to tape in the most accurate format.
This allows you to have directly recoverable data, managed by Commvault.
And once again, filesystem data fragmentation isn't an issue.

Blocklevel backup applies to the clients you want to back up: for Windows, using VSS, and for other systems, such as LVM, etc.
But this doesn't apply to data within the Commvault library; that must be copied using the Commvault functions themselves.

My advice is to find a professionally trained consultant who can provide you with the right guidance for managing your data protection environment in a timely manner.

No, we don’t. I explicitly _don’t_ want that. I want to have a full “dr” backup of my backup server, respectively the local/attached disk libraries. An Aux-Copy would just give me a 2nd copy of a current backup, which would require me to keep a bazillion tapes, as we have unlimited retention time on some jobs.

The “backup all files from FS” was the solution we got implemented as the “only known option” to realize what we want to do.

Yes, of course we do, otherwise we’d need several PB of storage.

You’re correct, and I alreay set the volumes to 64 kb cluster size to optimize IO. That’s the limit with the running version of Windows Server. More current ones can go higher and ideally both match.

Correct!

That would require to make an aux copy of the current disk librarys (and keeping the consistency somehow, which is beeing done on fs-level by VSS), as explained above: Aux Copy of the jobs would be nuts and not matching the goal.

Also: If you make an exact copy of the libraries, fragmentation should still be an issue.

Exactly that solution was provided us / implemented by VAD support. I heard it’s a rare solution, but I can’t believe that noone else just wants to keep all data on disks while having frequent backups of those libraries.

I'm sorry, but I believe the solution you've implemented is incorrect and impossible to fix.

If you can't find anyone to clarify your doubts, it's because the solution is wrong and therefore no one is talking about it.

Copy the data to another location (NAS, Server, Cloud, etc.), configure Commvault DR, and restore the software and data. This is the solution to follow.

“Impossible to fix” is nonsense. It’s not broken or unusuable. Also it’s originally about fragmentation on NTFS-based disk libraries.

Commvault DR is running, but that’s only a few GB and no containing the data of the attached disk libraries. Cloud is totally not an option and probably will never be - for financial and practical reasons.

You’re very confident in what you communicate. Thx anways for trying, I’ll try to find a ‘better’ or different solution. Aux copies of current jobs are nuts if you want to have a current backup of all your libraries and data aging/staging would also break the 3-2-1(-0) rule.

I’m a little confused by this process. Are you literally taking a copy of the raw chunk files from the Commvault disk library and moving them to tape? With the reason being you don’t want to rehydrate the jobs to create a true 2nd copy of the data on tape?

Thanks,
Scott
 

Hey Scott,

the goal we want to reach is: Having frequent offsite/offline copies of the whole Disk Libraries (which are the B2D targets for various jobs). So we have the daily Commvault DR-backups with 2 copies (one tape, one to a secured fileshare) and a scheduled job making, indeed, “raw” copies of the Disk library paths, backing them up to tape.

So - technically - we should be able to use the Commvault DR backups and the “raw” data to restore a dead or otherwise “affected” Commvault server (we don’t split functions).

The problem on 2nd copies is: I want the deduplicated/compressed storage beeing backed up, not thousands of “old” jobs individually archived on tapes. It also means we don’t need to care about retention times of the jobs, as the “current state” of the DB will have all we need on just a few tapes.

Regards 

- Christof

I completely understand the logic behind the approach, but I would not recommend it. I imagine you would fall under “best effort” should ever need any Commvault support. There’s certainly more moving parts and additional risk with this design.

Has this even been tested to see if it works?

To be clear, best case scenario, this is a solution for DR only. For operational issues, like accidental deletions, disk corruption, etc., this solution will not help you. There’s no way you would be able to recover a subset of the chunks from the disk library.

The real answer here is investment in a legitimate 2nd copy. Yes, it will have additional cost, but there’s a great amount of risk being taken in this above scenario.

Thanks,
Scott

I would have some serious concerns about this in a Disaster Recovery scenario.

Instead, I would be looking at leveraging multiple Storage Policies / Storage Policy Copies to prevent jobs with short-term retention requirements residing on the same tapes as jobs with long-term or infinite retention requirements. Tape sprawl can be managed. 

To answer part of your question though, the fragmentation issues you are experiencing are to be expected when using deduplication. Where it is impacting performance, Commvault recommends performing defragmentation using a suitable tool that supports online volume defragmentation.

Did you ever look into SILO Copy?