4

I've got a bootable usb drive with a live linux distro on it. We set it up to support persistence, so normally the changes make are kept between reboots. Booting into ram seems to make linux a lot faster than normal, which makes it a convenient way to use the live usb, especially since the live usb is a lot slower and sometimes hangs compared to a normal installation on a sata drive.

We can boot the live usb into ram by adding the boot parameters toram in the grub menu. However, obviously this breaks the persistence feature, as the filesystem is loaded onto ram, so all changes are lost upon reboot.

Once we made changes and decide we want to keep them, how can we flush the filesystem changes from the ram back to the usb drive?

For the record, I followed these instructions to make the live usb support persistence: https://docs.kali.org/downloading/kali-linux-live-usb-persistence

chtenb
  • 1,755
  • 2
  • 15
  • 17
  • An alternative procedure in [detailed here](https://www.howtogeek.com/howto/14912/create-a-persistent-bootable-ubuntu-usb-flash-drive/), which might give better results. – harrymc Nov 04 '17 at 16:22
  • 1
    That blog post does not deal with booting into ram, whereas my question is specifically about that. If the question is not clear enough, please tell me so I can improve the formulation! – chtenb Nov 04 '17 at 17:48
  • This is done afterward by adding `toram` as you did. Here is another article with one more method : [Create a persistent Ubuntu USB which boots to RAM](https://calvin.me/create-persistent-ubuntu-16-04-live-usb-even-boots-ram/). – harrymc Nov 04 '17 at 18:05
  • @ChieltenBrinke I think you misunderstood what the `toram` option does. As far as I understand it, just the readonly filesystem is treated as ramdisk. When booting with persistence enabled, the changes you make are directly written to the usb drive, not to ram as you seem to think. – the blizz Nov 05 '17 at 11:48
  • @theblizz: I agree. In addition, USB is slower than an internal disk, so the poster is not going to get more performance, but rather less, than with a classic disk-based installation. The only advantage of a persistent boot USB is that it's a portable Linux installation that can be run on any computer, which is what I initially thought the poster was trying to achieve (but maybe I was wrong). – harrymc Nov 05 '17 at 12:50
  • 1
    Allright that makes sense. I got the impression that when booting with the `toram` option the overlay read-write filesystem would also be copied to the ram. Is there a way to actually do that? – chtenb Nov 05 '17 at 13:30
  • You don't need to do anything - disk data is automatically read and kept in cache memory when accessed or written. Writes are delayed and done at the discretion of the OS and in optimal order. These optimizations are an extremely important part of any OS, many times judged by the efficiency of these algorithms. Efficiency is chiefly improved by adding more RAM, which allows allocating a bigger memory cache for the disk. – harrymc Nov 05 '17 at 13:37
  • That might depend on the caching settings for the drive though. It may be that the default caching behavior for usb drives differs from e.g. sata drives due to their removable character, not sure. – the blizz Nov 05 '17 at 18:16
  • 1
    @theblizz: Normally data written to removable devices are physically flushed faster. The OS does not take the chance that such a device will be removed without being dismounted first. – harrymc Nov 05 '17 at 18:44
  • Right so to improve the speed something probably has to be tweaked there I assume. – chtenb Nov 05 '17 at 19:01
  • @ChieltenBrinke: Tweaks are unnecessary - the OS does it very well. There was a post here from a guy who tweaked his removable disk, then started losing data that stayed unwritten when he turned off his computer. Nobody knew what he tweaked, and he didn't remember, so he got no help. Trust in the OS to have good algorithms, more sophisticated than anything you or I could do. – harrymc Nov 06 '17 at 07:47
  • "Normally data written to removable devices are physically flushed faster. " If the caching settings were to be tweaked to reflect non-removable drive behavior, would that result in losing data? – chtenb Nov 06 '17 at 08:24
  • Any tweaking of settings that you don't FULLY understand can lose data. And by "fully understand" I mean down to the level of the disk driver and cache management, which needs you to go into the source code. Don't tweak that which works well. – harrymc Nov 06 '17 at 16:45
  • @harrymc, I guess who tweaks do not think it works well enough. – Martian2020 Dec 02 '21 at 10:01
  • Hi, have you found a way by now? I'm currently trying to use longer route and not sure it would work - made changes on persistent USB, now prepare to add them to squashfs and make new USB. – Martian2020 Dec 02 '21 at 10:09

2 Answers2

2

Here are my 50 cents.

When running a live linux distro from a CD-rom, the filesystem is readonly. This makes sense because you can't write to a CD-rom. To allow you to create files and make modifications, the live OS uses an additional overlay filesystem on a ramdisk to store your modifications. (There are several ways of implementing this, just google around on linux overlay filesystem to learn more). These changes are lost upon shutdown however, as they are not stored on the CD-rom, but on ram.

When booting a live distro from an USB drive however, you have the possibility to put this overlay filesystem on the USB drive instead of a ramdisk. This is what's usually called a live usb with persistent storage. The advantage of this is that the changes you make are persisted. The drawback is that the OS writes frequently to your usb drive, which may impact your performance on non-duplex usb drives (e.g. result in frequent freezes). Furthermore this deteriorates the durability of your usb drive, as usb drives aren't really designed to be written often.

To address these two latter issues, you might want to create a ramdisk and manually create an overlay filesystem in there. To persist your changes, save this filesystem in a file on your USB drive upon shutdown, and restore it upon startup.

Creating an overlay filesystem manually isn't hard. Read this as a start: https://wiki.archlinux.org/index.php/Overlay_filesystem

the blizz
  • 171
  • 3
  • 1
    Could you please share more details on how to store and restore this overlay filesystem from the USB drive? – chtenb Nov 10 '17 at 15:03
  • Creating overlay is not hard. No access upper part of it when it was shadowed by new root and `systemd` replaced `init script` is (at least I have not found a sure way to do that yet). – Martian2020 Dec 02 '21 at 10:05
1

Booting into RAM from USB only seems faster because only a minimal Linux version is charged into memory.

USB is slower than an internal disk, so one will not get more performance, but rather less, than with a classic disk-based installation. The only advantage of a persistent boot USB is that it's a portable Linux installation that can be run on any computer.

In addition, removable devices are physically flushed more, as the OS does not take any chance that such a device will be removed without being dismounted first. This means more writes and more often for USB than for an internal disk.

No tweaks are necessary to improve memory usage. Linux automatically maintains disk data in cache memory when read or written. Writes are delayed and done at the discretion of the OS and in optimal order. These optimizations are an extremely important part of any OS, many times judged by the efficiency of these algorithms.

Efficiency is chiefly improved by adding more RAM, which allows allocating a bigger memory cache for the disk.

OS algorithms for the memory cache and disk management are highly sophisticated, and tweaking them without a deep understanding of the effects is highly dangerous to the health of the disk and OS.

harrymc
  • 455,459
  • 31
  • 526
  • 924
  • 1
    "so one will not get more performance, but rather less, than with a classic disk-based installation" I never said you would get more performance with an USB installation, quoting myself "since the live usb is a lot slower and sometimes hangs compared to a normal installation on a sata drive" – chtenb Nov 08 '17 at 09:43
  • 2
    "No tweaks are necessary to improve memory usage." You sort of contradicted yourself with what you said earlier "removable devices are physically flushed more". In my experience, especially when the usb does not (properly) implement duplex I/O as the USB 3.0 standard prescribes, this flush of writes really slows the OS down, and the computer hangs every few seconds with applications that do frequent writes, like browsers. Preventing the OS to flush writes here *would* make a performance difference. – chtenb Nov 08 '17 at 09:48
  • 1
    Tweaking disk policies is an extremely risky way to go - you can find examples on this forum. If you are trying to get appreciably more speed out of a slow computer, you should invest in hardware. I know that this is not what you would like to hear, but unfortunately there are no magical solutions (or they would have been already incorporated in Linux). – harrymc Nov 08 '17 at 10:01
  • 1
    I agree upon it being risky – chtenb Nov 08 '17 at 10:39