+4Hello
I am planning to install a new Media Agent with DAS, this one will be Server 2022 and here is my question - should I consider ReFS for data, DDB and Index?
Commvault support does not provide any 100% yes\no answer, Commvault documentation has zero information.
Thoughts?
Best answer by Jos Meijer
We do have DDB (RAID1) and Index (RAID5) on SSDs but there is no 100% answer regarding the file system and cluster size.
The last info I have discovered is that DDB and Index disks should be 32k cluster formatted and data drives are .. nobody knows, but I believe it is 64k.
Quote:
"For Windows, we recommend that the DDB needs to be on a fast, dedicated disk formatted at 32KB and dedicated disk libraries formatted at 64 K or with a higher block size up to 512 K, if supported by the operating system"
Index isn't defined as where 32KB is needed, default block size is fine.
Depending on your block size definitions in Commvault in combination with the type of data your backing up and restoring will define the best disklibrary block size for you.
Regarding NTFS vs ReFS, both should be possible.
But ReFS has internal mechanisms which can correct data corruption for instance, even though I never tested it, I expect this will interfere with for example the ransomware protection on a MA.
ReFS mechanisms may interfere with DDB performance as DDB integrity is checked by Commvault periodically so no need for ReFS to manage that also.
As resilience should be managed on a hardware level and not software level I don't see the advantage there for ReFS, multiple raid 6 groups and NTFS will do fine.
Looking at NTFS sparse file support is available.
ReFS Sparse VDL is mostly for virtualized environments where you need to zero big files.
All that being said I don't see direct advantages to use ReFS anywhere in a MA.
As said earlier, never tested it so if anyone has other experiences I would love to read them. But logic dictates that you want Commvault to manage it's own data. Plus side is that these Commvault processes can be reviewed via job history and reports, ReFS doesn't provide you that kind of insight.
+23For the Index, they are checking.
Will be in touch!
+4We do have DDB (RAID1) and Index (RAID5) on SSDs but there is no 100% answer regarding the file system and cluster size.
The last info I have discovered is that DDB and Index disks should be 32k cluster formatted and data drives are .. nobody knows, but I believe it is 64k.
+4For the Index, they are checking.
Will be in touch!
Hi! Do you have any updates?
+19Looking at the characteristics of ReFS is seems to be a perfect match to be used as backup target. It also supports sparse file and the recommended block sizes so it should be able to facilitate DDB, index and data.
For Windows, we recommend that the DDB needs to be on a fast, dedicated disk formatted at 32KB and dedicated disk libraries formatted at 64 K. For Linux MediaAgents, we recommend to use DDB disks formatted at 4KB block size.
+17We do have DDB (RAID1) and Index (RAID5) on SSDs but there is no 100% answer regarding the file system and cluster size.
The last info I have discovered is that DDB and Index disks should be 32k cluster formatted and data drives are .. nobody knows, but I believe it is 64k.
Quote:
"For Windows, we recommend that the DDB needs to be on a fast, dedicated disk formatted at 32KB and dedicated disk libraries formatted at 64 K or with a higher block size up to 512 K, if supported by the operating system"
Index isn't defined as where 32KB is needed, default block size is fine.
Depending on your block size definitions in Commvault in combination with the type of data your backing up and restoring will define the best disklibrary block size for you.
Regarding NTFS vs ReFS, both should be possible.
But ReFS has internal mechanisms which can correct data corruption for instance, even though I never tested it, I expect this will interfere with for example the ransomware protection on a MA.
ReFS mechanisms may interfere with DDB performance as DDB integrity is checked by Commvault periodically so no need for ReFS to manage that also.
As resilience should be managed on a hardware level and not software level I don't see the advantage there for ReFS, multiple raid 6 groups and NTFS will do fine.
Looking at NTFS sparse file support is available.
ReFS Sparse VDL is mostly for virtualized environments where you need to zero big files.
All that being said I don't see direct advantages to use ReFS anywhere in a MA.
As said earlier, never tested it so if anyone has other experiences I would love to read them. But logic dictates that you want Commvault to manage it's own data. Plus side is that these Commvault processes can be reviewed via job history and reports, ReFS doesn't provide you that kind of insight.
+17Lol just wrote a completely different opinion regarding ReFS at the same time as Onno đ€Ł
+4We do have DDB (RAID1) and Index (RAID5) on SSDs but there is no 100% answer regarding the file system and cluster size.
The last info I have discovered is that DDB and Index disks should be 32k cluster formatted and data drives are .. nobody knows, but I believe it is 64k.
Quote:
"For Windows, we recommend that the DDB needs to be on a fast, dedicated disk formatted at 32KB and dedicated disk libraries formatted at 64 K or with a higher block size up to 512 K, if supported by the operating system"
Index isn't defined as where 32KB is needed, default block size is fine.
Depending on your block size definitions in Commvault in combination with the type of data your backing up and restoring will define the best disklibrary block size for you.
Regarding NTFS vs ReFS, both should be possible.
But ReFS has internal mechanisms which can correct data corruption for instance, even though I never tested it, I expect this will interfere with for example the ransomware protection on a MA.
ReFS mechanisms may interfere with DDB performance as DDB integrity is checked by Commvault periodically so no need for ReFS to manage that also.
As resilience should be managed on a hardware level and not software level I don't see the advantage there for ReFS, multiple raid 6 groups and NTFS will do fine.
Looking at NTFS sparse file support is available.
ReFS Sparse VDL is mostly for virtualized environments where you need to zero big files.
All that being said I don't see direct advantages to use ReFS anywhere in a MA.
As said earlier, never tested it so if anyone has other experiences I would love to read them. But logic dictates that you want Commvault to manage it's own data. Plus side is that these Commvault processes can be reviewed via job history and reports, ReFS doesn't provide you that kind of insight.
Thank you very much! So, I will contimnue using NTFS with DDBs on 32k, Index on 32k and probably will increase data drives from 64 to 128k blocks.
+19
As in your concerns:
Te ransomware feature is a filter driver and resides above the actual filesystem, so it should not interfere. As for DDB integrity do mind there is also a difference between something external that is causing issues on DDB integrity like human involvement through software updates, scripts or malware and underlying data integrity issues for example bit-rot. Also here I do not see any disadvantages, but instead if you are using large disk libraries (DAS) it will add some form of protection ;-)
Anyway I think the only way to get formal confirmation is that someone from development delivers an answer and that the documentation is updated accordingly to remove the fuzzđ
+17You could be right
Never tested it and very curious what developments answer on this will be.
Regarding the DDB, wasn't referring to the data integrity functionality.
Only that ReFS process could interfere with performance đ , my expectation is that this can occur in high load environments.
You could be right
I think youâre wrong here.
The Commvault integrity tests will go and recover from backups (it found an inconsistent/broken DDB here once), if they detect something broken. It does not include auto-heal by checksum etc. DBs usually donât do that, itâs a FS/hardware job.
Instead ReFS is a very good choice for big datasets, as bitrot can happen and errors may also occure on FS, which canât be detected on RAID level.
Weâre using ReFS for DDB for years now and I canât say any bad. As we have to change HDDs, itâs probably going to be a ReFS for that too - for now itâs NTFS.
Tehnically you should also check NTFS consistency now and then, so no real difference to ReFS.
People use ZFS for decades and itâs a very popular, very good FS, comparable to the rather new ReFS.
As Onno stated out: Commvault doesnât fiddle between FS and hardware. Thereâs also no need to and it would be pretty weird to do that.
And donât forget: Go double parity for your large datasets. RAID5 is not enough.
(I know the thread is old, but I wasnât sure if I should go for NTFS or ReFS for storage, until now).
Enter your E-mail address. We'll send you an e-mail with instructions to reset your password.
