6

I am using a SSD connected to a Mac over a sata to usb bridge to archive relatively big files. After I transfer it to the SSD, I unmount it and and unplug it.

This SSD has a DRAM cache. From my understanding, this cache is both used to cache writes and store a copy of the file mapping. If I unplug it without unmounting or when its currently writing, I would assume that it would be very possible to lose data that is cached in DRAM and maybe even lose files due to pointers to any data that changed in the mapping being lost as well.

My question is: If I unmount the drive and immediately unplug after it successfully unmounts, is there a chance that I remove power from the SSD prior to it finishing writing whatever is in the DRAM to disk?

Does the OS / APFS manage the DRAM and ensures that its flushed prior to it being unmounted or does the firmware in the SSD do this?

agz
  • 8,088
  • 21
  • 70
  • 112
  • 1
    Without knowing the specific SSD model it is impossible to tell you what will happen. Your question is exactly why there are consumer versus enterprise drives. https://techcommunity.microsoft.com/t5/storage-at-microsoft/don-t-do-it-consumer-grade-solid-state-drives-ssd-in-storage/ba-p/425914 – MonkeyZeus Feb 16 '22 at 14:16
  • It's an Inland SATA SSD. I am not even sure if it has DRAM as the only info I can find online are conflicting. When transferring 1 huge file, I see it run at a higher throughput until it settles in at a much slower one (don't know if this is DRAM caching or some caching macOS is doing). I assume in the case it doesn't have DRAM, it can only write to stable storage. – agz Feb 16 '22 at 18:09
  • [Drive Listing] (https://www.microcenter.com/product/623042/inland-professional-512gb-ssd-3d-tlc-nand-sata-30-6-gbps-25-inch-7mm-internal-solid-state-drive) – agz Feb 16 '22 at 18:15
  • @agz: How huge and how fast? Many SSDs use some of their cells as a fast SLC write buffer (1 bit per cell), later compacting that to TLC or QLC or whatever. If you write more than a few (dozen) GiB (depending on how much free space the drive has, given trim etc.) at once, that will fill up. https://phisonblog.com/the-benefits-of-using-slc-buffers-with-ssds/ / https://www.anandtech.com/show/16087/the-samsung-980-pro-pcie-4-ssd-review/3 for example. – Peter Cordes Feb 17 '22 at 01:07
  • What's stopping you from performing such tests? Back up your stuff elsewhere and then run your mount/unmount tests and assess the results. – MonkeyZeus Feb 17 '22 at 13:46
  • @PeterCordes, 8GB single file, the first ~5 seconds I get writes of ~200 MB/s, it then drops to a steady 50 MB/s. – agz Feb 17 '22 at 18:16
  • @MonkeyZeus in a middle of a move right now, don't have a way to backup the stuff on there – agz Feb 17 '22 at 18:16
  • 1
    Sounds like you're suffering from an SLC cache scenario as explained in https://phisonblog.com/the-benefits-of-using-slc-buffers-with-ssds/. The SSD you linked doesn't look very special so as long as your OS tells you it's safe to unplug the drive then it should be safe. I assume that trying to unmount it during a transfer results in the system telling you "NO", correct? – MonkeyZeus Feb 17 '22 at 18:53
  • 200MB/s is pretty slow for an SSD, but that might be the USB bottleneck. It's way below SATA3 transfer speed of 600MB/s. If it is a USB bottleneck, that doesn't fully rule out DRAM, but an SLC cache should be keeping up with that slow 200MB/s write speed. (And most budget drives do have an SLC write cache.) If it's actually an SLC-write bottleneck rather than the USB interface, that would rule out DRAM. – Peter Cordes Feb 17 '22 at 19:57
  • @PeterCordes If the source drive is a traditional hard drive then 200MB/s is as fast as you can expect it to go. Going down to to 50MB/s is weird unless this HDD also hosts the OS and is battling for the seek head. – MonkeyZeus Feb 17 '22 at 20:31
  • @MonkeyZeus: Oh right, the querent is probably talking about copying, not a file-write benchmark that just generates data in memory that would just test transfer *to* the SSD. I wouldn't suggest using up more of the drive's write endurance with further benchmarking, though. – Peter Cordes Feb 17 '22 at 20:41
  • @PeterCordes We usually call them "OP" =) and in the comments is buried " 8GB single file, the first ~5 seconds I get writes of ~200 MB/s, it then drops to a steady 50 MB/s." – MonkeyZeus Feb 17 '22 at 20:46
  • @MonkeyZeus: Yeah, that was the comment I saw; I assumed it was just about *writing* the file, not reading it from a source slower than an SSD. e.g. `dd if=/dev/zero of=testfile bs=64k count=131072` to read zero bytes from the kernel. (Modern SSDs don't usually do compression, but if your FS does then you'd want a fast PRNG). That's what I'd do if I wanted to time SSD writes. It's only other comments and part of the question that hint at this being about *copying* a file without saying where from. – Peter Cordes Feb 17 '22 at 20:51
  • @PeterCordes Based on that drive's rated 345 TBW endurance rating, you would have to write an 8GB file 43,125 times in order to make the drive fail. That's 94GB every day for 10 years. I think OP can safely test this drive a few dozen times if they wanted to. – MonkeyZeus Feb 17 '22 at 21:03
  • Doubt source drive is a bottleneck, its a M1 MacbookPro w/ ssd with nothing IO intensive going on during the transfer. I am using a sabrant usb3.0 to sata adapter. I'm less interested about its steady 50 MB/s transfer speed. More interested in if the initial burst was stored in DRAM somewhere and when I unmount at the end, I am not giving it enough time for it to write to stable storage. The SLC cache scenario said above seems to make sense. – agz Feb 18 '22 at 19:17

2 Answers2

26

The DRAM on the SSD is managed by the firmware, but the OS can tell the firmware to flush all pending data to stable storage using the "barrier" function calls. The OS can also tell the SSD whether write caching should be enabled at all.

If you cleanly unmount the drive, the OS will issue the required function calls to make sure all data is fully written.

If you unplug it mid-write without cleanly unmounting, all bets are off, and the outcome will depend on the SSD firmware, file system and various other factors.

Gordan Bobić
  • 3,330
  • 1
  • 18
  • 23
  • I think the asker wants to know if, *after* the OS reports that the volume is successfully unmounted, there might still be DRAM writes possible. (I assume not) – MiG Feb 16 '22 at 07:18
  • 3
    The point in this answer is that the OS can use the barrier function in the disk I/O protocol to make sure all the writes are done before it tells the user that it is safe to remove... That barrier function will be called as part of the unmount. – user10489 Feb 16 '22 at 12:36
  • The last line could be removed (or moved forward) though, that confuses the message. – MiG Feb 16 '22 at 12:45
  • 5
    @MiG: The OS can only report what the SSD tells it to. There have been drives in the past which reported to the OS that they have committed the write to permanent storage when, in fact, they were still in the process of writing it. There is nothing the OS can do about that. So, the answer to "I think the asker wants to know if, after the OS reports that the volume is successfully unmounted, there might still be DRAM writes possible." is "Yes, especially if the drive lies to the OS". – Jörg W Mittag Feb 16 '22 at 15:09
  • 1
    If the hardware is lying to you, it is defective and all bets are well and truly off. – Gordan Bobić Feb 16 '22 at 15:40
  • @JörgWMittag 1. oh man. 2. I think that's very useful info to the asker! – MiG Feb 16 '22 at 15:43
  • 4
    @GordanBobić or badly designed. If what it does doesn't correspond to what it tells, I'd call that a bug. – MiG Feb 16 '22 at 15:44
  • Thanks, all of this has been very helpful information. As mentioned in an above comment, the ssd is an Inland SATA SSD. It's a cheap consumer one, but hopefully not one that lies to me :) – agz Feb 16 '22 at 18:11
  • @GordanBobić "Defective"? No. Probably "malicious", definitely "dangerous". – Josh Part Feb 16 '22 at 18:45
  • @JoshPart: Why malicious? There are many defective products that exist merely because lousy companies made them. – user21820 Feb 17 '22 at 07:09
  • @GordanBobić I don't think it's apt to say the hardware is lying. The described situation is causing undefined behaviour and thus the specification (written for defined behaviour under known circumstances) simply no longer applies. – Mast Feb 17 '22 at 07:40
  • @user21820 if a storage device is purposely made to deceive its users, without any regards of what could happen to their information, then I'd definitely call it "malicious" – Josh Part Feb 17 '22 at 15:54
  • @JoshPart: Of course. I thought it was clear that agz didn't mean "*intentionally* lies to me". =) – user21820 Feb 17 '22 at 16:07
  • 1
    @user21820: I wouldn't assume that at all. Lying to boost benchmark results is quite intentional. – Ben Voigt Feb 17 '22 at 23:04
  • 1
    Any SSD that tells the OS it has completed writes when it hasn't is defective. Any device that does this intentionally is maliciously defective and should be returned to the manufacturer. – user10489 Feb 18 '22 at 01:29
  • And **why** do you think it is more likely that a cheap consumer SSD would lie in order to boost benchmark results than simply 'accidentally lie' because of a bug in firmware? Of all defective products I have seen, 99% of them are defective solely because some bug or fault rather than any intent to harm the consumer. – user21820 Feb 18 '22 at 12:04
5

There is DRAM involved both in the host controlled by the operating system and DRAM inside the SSD controlled by the SSD's firmware.

As stated in the other answer, if you eject properly, all of that should be taken care of by well defined mechanisms that are part of the SATA and USB protocols to synchronize actions between the host and the SSD.

If you yank it out mid write, it is likely that data in both the DRAM of the host and the SSD will remain unwritten, and it is kind of moot if the unwritten data was in the DRAM in the host or the DRAM in the SSD, as it will be lost either way. In that situation, there's probably even data on the bus that has been sent by the host but not yet received by the SSD.

user10489
  • 1,173
  • 1
  • 5
  • 10