Usually loop device (v-node disk) is based on a file with pre-allocated size (e.g. an image of a floppy drive or ISO disk). The device is then treated as a block device and can be formatted with any filesystem supported by the operating system, e.g. ext3.
On the other hand, FUSE allows to mount archives with writing support, i.e. archivemount. It's possible to write files to the mounted drive and the underlying archive will grow to accommodate all files written to it. So in a way it behaves like a block device but without a specific filesystem - the filesystem is determined by the archive type, e.g. zip, tar.
I am looking for an implementation of a file-backed device that can grow dynamically to accommodate any amount of files (behaves like an archive mounted with FUSE) but with efficient implementation of read/write/seek operations (as it would be the case with a filesystem in a block device).
By efficient I mean more efficient than linearly searching through the file, so possibly with some sort of distributed indexing of files contained in the file.
As an example of what I mean lets consider an uncompressed tar archive mounted with archivemount. It supports read/write but the seek operation isn't efficient since tar archives dont's support TOC - they are supposed to work with tape archives.
In case of zip archives mounted with archivemount read and seek is efficient because zip supports TOC, however write isn't efficient becauset it often involves recompressing the whole archive.
Another example would be a sparse file mounted through loop device - it's sparse so it can grow dynamically but up to a specified fixed maximum size, which is not fulfilling my requirement.
Yet another example is the file-backed of the CouchDB database or the MBox file format. In both cases data is always appended to the file (e.g. instead of deleting a file it marks it as deleted and appends a new version). And to reclaim the space used by deleted files from time to time the file needs to be compacted. With such file all operations are fast, including seek, because it always reads the last version of the file. However, I don't know of any such implementation to store arbitrary files.
So, does such implementation of a file-backed device exist?
