4

Backstory:

I use a Mac (10.6.4), and to this day when I download a torrent using Vuze, no matter what configuration I try Vuze allocates space for the entire file before it starts downloading said file. For a long time, a quick google led me to believe it was a Vuze bug, but after years have passed and nothing has changed a more detailed search tells me that it's because HFS+ doesn't support sparse files.

Question:

The sparse file wiki says that Apple's HFS+ doesn't support sparse files, and Apples' Time Machine and FileVault uses "sparse images" and "sparse bundles".

I understand what a sparse bundle is (image split into 8Meg chunks) but aside from that, is there a difference between these "sparse images" and just regular sparse files? How can HFS+ not support sparse files and yet still support sparse images and sparse bundles?

Community
  • 1
user52250
  • 43
  • 3

3 Answers3

7

It's because the sparseness is handled at a different level of abstraction for a sparse image vs. a sparse file. If you're storing a file "directly" on disk, the filesystem (e.g. HFS+) handles mapping the sequence of bytes that make up the file into a sequence of bytes on disk. A sparse file is one that has gaps in its sequence of bytes, and a filesystem that supports this will skip those missing bytes when it maps the file to disk.

When you're using a disk image, the mapping is quite a bit more complicated: you have the bytes in the file being mapped (by the inner filesystem) to bytes in the (virtual) disk. These are then mapped (by the image mounter) to bytes in the disk image (file), which are then mapped (by the outer filesystem) to bytes on the actual disk. The image mounter handles sparse mappings (and AIUI only between/beyond files, not within them), but that doesn't imply that the other layers (the filesystems) can. As far as the outer filesystem is concerned, the image file is just a (non-sparse) sequence of bytes.

BTW, the image mounter can also add other features that the filesystem alone doesn't deal with. For instance, Mac disk images have supported compression roughly forever, but native support for compression wasn't added to HFS+ until OS X v10.6.

Gordon Davisson
  • 34,084
  • 5
  • 66
  • 70
2

Gordon Davisson explanation was enlightening: this is how I am using sparsebundle images as a torrents download directory.

First of all create the image (I set it to 50GB, but any size would work)

hdiutil create -size 50g -type SPARSEBUNDLE -nospotlight -volname "Torrents repository" -fs "Case-sensitive Journaled HFS+" Torrents.sparsebundle

Then double click Torrents.sparsebundle image just created to mount it.

Setup your favorite torrent client to use the volume mounted called "Torrents repository" as temporary repository, I suggest to set it up so that completed files would be moved out of the image (to Downloads folder for example).

It works very well, especially when you select only a few files in a big torrent.

I also added a startup script to my system:

hdiutil compact Torrents.sparsebundle 
hdiutil attach Torrents.sparsebundle

First command will reclaim the unused diskspace, while second will automatically mount the image. Unfortunately the compact operation cannot be done while the image is mounted, so I thought the best chance was on system startup before mounting it.

Marco Baldassarre
  • 21
  • 1
  • "this is how I am using sparsebundle images as a torrents download directory." — ok, this is how you're using it; the q-n is why are you using it that way? O_o – poige Jul 23 '17 at 15:03
-3

APFS which is new default FS now does support sparse files.

poige
  • 409
  • 5
  • 12
  • This question was asked 7 years before APFS was even introduced. This does not answer the author's question. APFS and Apple's HFS+ are different file systems. Please read the question again – Ramhound Oct 01 '17 at 12:05
  • that's why my answer has word "now". Try finding it more thoroughly may be – poige Oct 01 '17 at 13:57
  • Your answer isn’t relevant to this question though – Ramhound Oct 01 '17 at 14:49
  • it's an answer to the subject: "Apple has “sparse bundles” but not sparse files?" – poige Oct 01 '17 at 16:01
  • if you're so concerned in regards on having irrelevant things here fix the subject first may be – poige Oct 01 '17 at 16:04
  • The author's question was specifically about sparse files on 10.6.4 while using HFS+. I wish you luck. I am not going to spend anymore time on this discussing the relevant of your answer. – Ramhound Oct 01 '17 at 17:10
  • I wish I could say I need you to wish me anything but I'd be a hypocrite if I'd follow the same line you did. I just don't care. That's normal though. – poige Oct 02 '17 at 10:01