0

When I had my 18TB drive filled I noticed I could not enable NTFS compression because the function was grayed out. I followed the directions here "Compress contents to save disk space" option not available for one partition and emptied out the drive to format it (took 3 weeks to find places to put the data), then realized that 4k cluster size is not an option for formatting a drive this large (options start from 8k). Was all this trouble for nothing? Partitioning is out of the question.

phuclv
  • 26,555
  • 15
  • 113
  • 235
Bricktop
  • 209
  • 1
  • 12
  • “then realized that 4k cluster size is not an option for formatting.” - Cluster size is dependent on hardware. What cluster sizes were offered? Probably should have asked before you formatted the drive. – Ramhound May 17 '23 at 11:50
  • @Ramhound Everything starting from 8k. Thank you Captain Hindsight, it was very unexpected. – Bricktop May 17 '23 at 11:54
  • What file system are you using? – Ramhound May 17 '23 at 11:57
  • @Ramhound NTFS. Is another file system an option? Should be Linux compatible. – Bricktop May 17 '23 at 12:00
  • 1
    @Bricktop ExFAT is Linux compatible too and can handle such large disks, but if you insist on NTFS compression you will obviously need NTFS. And that also means 4K clusters or smaller which isn't going to fly on a partition > 16 TB. – Tonny May 17 '23 at 12:15
  • @Tonny It doesn't have to be NTFS compressions specifically. Any file system compression compatible with 18tb, Windows and Linux will do. – Bricktop May 17 '23 at 12:19
  • @Bricktop I think you are out of luck in that case. I'm not aware of any filesystem that meets those criteria. The only option I see is to split that disk in 2 partitions to get under the 16 TB limit for 4K clusters. – Tonny May 17 '23 at 12:27
  • 1
    @Bricktop - I misspoke earlier, cluster size is file system dependent, while sector size is hardware dependent. I hope you realize the compression you are looking to use won't be realized in a large percentage of your disk usage, even if you could enable it, on a 18 TB disk. My last statement is clearly limited to NTFS. – Ramhound May 17 '23 at 13:14

2 Answers2

3

Yes, it was for nothing .

NTFS partitions bigger than 16TB have a minimum cluster size of 8 KB (and partitions bigger than 2TB also have a minimum of 4 KB.)

As far as using a different file system, which might very well allow you larger cluster sizes, it's not going to help you for your goal, since the compression you are referring to is NTFS Compression which certainly won't be available for other file systems.

r2d3
  • 3,298
  • 1
  • 8
  • 24
Yisroel Tech
  • 9,687
  • 3
  • 23
  • 37
  • OK. So there might be another file system capable of compression that is compatible with Windows, Linux and 18tb drives? – Bricktop May 17 '23 at 12:18
  • Maybe (not sure) but for someone to help you with such a quest you really first need to lay out in detail what your goal exactly is (and any other requirements/needs around it, and not things like *"out of the question"* without explaining why...) But I think that's for another question. – Yisroel Tech May 17 '23 at 12:22
  • 1
    But for the meantime to give you some reference, the **Btrfs** file system may be an option. – Yisroel Tech May 17 '23 at 12:31
  • 2
    If you have to support Windows machines, Btrfs support, must be achieved through third-party software. – Ramhound May 17 '23 at 13:12
1

NTFS supports maximum 232 - 1 clusters, so for 4KB cluster the maximum size is (232 - 1)*4KB = 16TB. Therefore partitions larger than that can't use the default transparent compression. That doesn't mean compression can't be used because there are multiple compression algorithms in modern NTFS and many can be used on NTFS with large clusters with the caveats that

  • You can't enable automatic compression on the drive or folders and have to compress files individually
  • You can't edit the files on-the-fly, so compression will be much better but is only suitable for read-only files

For more details read the link above

There are other file systems that support transparent compression, however only Btrfs and ZFS have good Windows drivers: WinBtrfs and ZFSin

phuclv
  • 26,555
  • 15
  • 113
  • 235