Solved

NTFS vs REFS for DAS


Badge +4

Hello

I am planning to install a new Media Agent with DAS, this one will be Server 2022 and here is my question - should I consider ReFS for data, DDB and Index?

Commvault support does not provide any 100% yes\no answer, Commvault documentation has zero information. 

Thoughts? 

icon

Best answer by Jos Meijer 2 August 2022, 20:08

View original

10 replies

Userlevel 7
Badge +23

@Roman Melekh , a few of our devs are looking into this, though note that the DDB should be on an SSD.

For the Index, they are checking.

Will be in touch!

Badge +4

We do have DDB (RAID1) and Index (RAID5) on SSDs but there is no 100% answer regarding the file system and cluster size. 

 

The last info I have discovered is that DDB and Index disks should be 32k cluster formatted and data drives are .. nobody knows, but I believe it is 64k.

Badge +4

@Roman Melekh , a few of our devs are looking into this, though note that the DDB should be on an SSD.

For the Index, they are checking.

Will be in touch!

Hi! Do you have any updates? 

Userlevel 7
Badge +15

Looking at the characteristics of ReFS is seems to be a perfect match to be used as backup target. It also supports sparse file and the recommended block sizes so it should be able to facilitate DDB, index and data.

  • For Windows, we recommend that the DDB needs to be on a fast, dedicated disk formatted at 32KB and dedicated disk libraries formatted at 64 K. For Linux MediaAgents, we recommend to use DDB disks formatted at 4KB block size.

Userlevel 6
Badge +14

We do have DDB (RAID1) and Index (RAID5) on SSDs but there is no 100% answer regarding the file system and cluster size. 

 

The last info I have discovered is that DDB and Index disks should be 32k cluster formatted and data drives are .. nobody knows, but I believe it is 64k.

Quote:

"For Windows, we recommend that the DDB needs to be on a fast, dedicated disk formatted at 32KB and dedicated disk libraries formatted at 64 K or with a higher block size up to 512 K, if supported by the operating system"

Index isn't defined as where 32KB is needed, default block size is fine.

Depending on your block size definitions in Commvault in combination with the type of data your backing up and restoring will define the best disklibrary block size for you.

Regarding NTFS vs ReFS, both should be possible.

But ReFS has internal mechanisms which can correct data corruption for instance, even though I never tested it, I expect this will interfere with for example the ransomware protection on a MA.

ReFS mechanisms may interfere with DDB performance as DDB integrity is checked by Commvault periodically so no need for ReFS to manage that also.

As resilience should be managed on a hardware level and not software level I don't see the advantage there for ReFS, multiple raid 6 groups and NTFS will do fine.

Looking at NTFS sparse file support is available.

ReFS Sparse VDL is mostly for virtualized environments where you need to zero big files.

All that being said I don't see direct advantages to use ReFS anywhere in a MA.

As said earlier, never tested it so if anyone has other experiences I would love to read them. But logic dictates that you want Commvault to manage it's own data. Plus side is that these Commvault processes can be reviewed via job history and reports, ReFS doesn't provide you that kind of insight.

Badge +4

Ok, what about the Index volume block size? 32k?

Userlevel 6
Badge +14

Lol just wrote a completely different opinion regarding ReFS at the same time as Onno 🤣

Badge +4

We do have DDB (RAID1) and Index (RAID5) on SSDs but there is no 100% answer regarding the file system and cluster size. 

 

The last info I have discovered is that DDB and Index disks should be 32k cluster formatted and data drives are .. nobody knows, but I believe it is 64k.

Quote:

"For Windows, we recommend that the DDB needs to be on a fast, dedicated disk formatted at 32KB and dedicated disk libraries formatted at 64 K or with a higher block size up to 512 K, if supported by the operating system"

Index isn't defined as where 32KB is needed, default block size is fine.

Depending on your block size definitions in Commvault in combination with the type of data your backing up and restoring will define the best disklibrary block size for you.

Regarding NTFS vs ReFS, both should be possible.

But ReFS has internal mechanisms which can correct data corruption for instance, even though I never tested it, I expect this will interfere with for example the ransomware protection on a MA.

ReFS mechanisms may interfere with DDB performance as DDB integrity is checked by Commvault periodically so no need for ReFS to manage that also.

As resilience should be managed on a hardware level and not software level I don't see the advantage there for ReFS, multiple raid 6 groups and NTFS will do fine.

Looking at NTFS sparse file support is available.

ReFS Sparse VDL is mostly for virtualized environments where you need to zero big files.

All that being said I don't see direct advantages to use ReFS anywhere in a MA.

As said earlier, never tested it so if anyone has other experiences I would love to read them. But logic dictates that you want Commvault to manage it's own data. Plus side is that these Commvault processes can be reviewed via job history and reports, ReFS doesn't provide you that kind of insight.

Thank you very much! So, I will contimnue using NTFS with DDBs on 32k, Index on 32k and probably will increase data drives from 64 to 128k blocks.

 

 

Userlevel 7
Badge +15

@Jos Meijer that's indeed funny :-) to be honest I have no experience with it as well and the biggest reason for that is that we're not using any disk-based library, so no large datasets.

As in your concerns:
Te ransomware feature is a filter driver and resides above the actual filesystem, so it should not interfere. As for DDB integrity do mind there is also a difference between something external that is causing issues on DDB integrity like human involvement through software updates, scripts or malware and underlying data integrity issues for example bit-rot. Also here I do not see any disadvantages, but instead if you are using large disk libraries (DAS) it will add some form of protection ;-) 

Anyway I think the only way to get formal confirmation is that someone from development delivers an answer and that the documentation is updated accordingly to remove the fuzz😋

Userlevel 6
Badge +14

You could be right @Onno van den Berg , but still the data file is being altered to correct data even if it is on a file system level. I don’t know, it's all a bit of a grey area.

Never tested it and very curious what developments answer on this will be.

Regarding the DDB, wasn't referring to the data integrity functionality.
Only that ReFS process could interfere with performance 😉 , my expectation is that this can occur in high load environments.

Reply