2

For me, the chief fear of using SD cards for backup is reliability rather than storage capacity. SD cards are known to offer relatively high storage for little expense, compared to CDs and DVDs. However, they may not be considered reliable enough for long term archives, for instance in the time span of decades in storage. I'm investigating the use of SD memory cards to store small batches of data like Text Files, spreadsheets and an occasional scanned document, for personal use and family history, not business. The total archive size would be perhaps 500MB.

Ive scoured the internet looking for special SD file systems designed to build redundant copies of files at the file level, or that use redundant binary (many bits to represent one bit), but can't find anything at all. I'm also searching for software project sites that might have begun to tackle this problem.

This is an area outside my expertise. I know that SD cards are a block device but my understanding of that is shallow. It seems they control their own memory management, but I'm lost after that.

To anyone who understands the operations of SD cards memory, and how this memory commonly fails, would having redundant copies of files on one card solve the problem of reliability, or would the card failure corrupt the entire card all at one time? Or, would having some sort of file system that (instead of having copies of files) has redundant memory bits. For example, if I had 500MB of archives and a 32GB card to store them own, would 64 copies of the files be better preserved, or would a hypothetical file system that writes 64 bits for every bit, then if say 62 bits indicate "0" and the other 2 bits indicate "1" then the file system interprets this as a "0". (after many years of storing this card, not the next day!)

So my ultimate question is. Are SD cards physically able to become reliable archive mediums using redundancy of data? Or are they just too poor quality to be relied on, even with redundancy?

user12711
  • 153
  • 1
  • 9
  • Relevant: https://superuser.com/q/1025044/141595 – Saaru Lindestøkke Nov 03 '22 at 00:24
  • 1
    “Or are they just too poor quality to be relied on, even with redundancy?” They are considered to be disposable media by the manufacturer unless you purchase high-end SD cards. It would be better if you just got an SSD drive to store the data on. Using your example a 32GB SSD drive costs around $15 (U.S.). Heck, a USB flash drive might be a better form of storage but the cost/risk benefit is not worth it. – Giacomo1968 Nov 03 '22 at 02:00
  • 3
    Does this answer your question? [What's the life expectancy of an SD card?](https://superuser.com/questions/17350/whats-the-life-expectancy-of-an-sd-card) – Giacomo1968 Nov 03 '22 at 02:00
  • And more info here as well: https://superuser.com/q/1334494/167207 – Giacomo1968 Nov 03 '22 at 02:18
  • It won't "solve" the problem, but it'll probably more or less increase the chance of successful recovery when it rots. (I mean, given that the archive is not gonna be larger than 1G and the smallest card you can get these days are 16G ones. Why not?) – Tom Yan Nov 03 '22 at 04:49
  • that's all helpful information, but this is an archive with Write Once, read-every-once-in-a-awhile use, and many copies of the data /folder1/ /folder2/ ... /folder16/ all with the same data. I have a good answer below I'm going by now – user12711 Nov 03 '22 at 19:59
  • I think you're approaching this from the wrong direction. Even if all your SD cards survived the timescale, would they survive a fire? Buy some online storage space & keep it there, plus two small spinny rust HDs, one at home & one at a relative's. Don't compress, don't encrypt. – Tetsujin Nov 06 '22 at 13:08
  • What if the online storage site goes out of business and gets unplugged? (the first ebooks by George Orwell were removed by Amazon from folks devices AFTER they bought 1984, due to discovering later they didn't have publishing rights -- an example) – user12711 Nov 06 '22 at 19:19
  • So if I rip my CDs to back them up, post them to online storage, and the government says, delete it all, then it's gone. That's about what happened in the case above when Amazon removed the novel 1984 from devices, after the copies were purchased. – user12711 Nov 06 '22 at 19:43
  • It's not vaguely similar at all. Amazon was a top-down managed system. "The government" has no way to ever know what's in your personal storage. I don't know of any online storage system that's gone down without at least a year's grace. Microsoft & Google are unlikely to go down overnight. Dropbox, Mega, Backblaze etc are pretty well established. Use two or three of them for total paranoia coverage. – Tetsujin Nov 07 '22 at 09:58

1 Answers1

3

SD Cards are totally unfit for long term storage as NAND memory 'bleeds' data. Simplified, a certain charge at the cell level 'decides' if the value is 0 or 1. As charge leaks or bleeds, at some point it will drop below a threshold at which point the bit 'flips'.

I suppose you could compare it to a rechargeable battery losing it's charge over time. To a degree the NAND controller can compensate for this. First of all a number of bitflips per sector can be detected and recovered from by means of ECC. How many flipped bits that can be recovered from depends on ECC algorithm and size of the ECC code.

NAND recovery software emulating controller read-retries

At a lower level controllers employ a technique referred to as 'read-retry' (RR) which allows for the controller to experiment with alternative threshold values (probe levels in below image). It should not be confused with simply trying a read several times. If lower threshold for example results in less bitflips, enough to ECC correct, the data can be read/recovered. Data is then written to a fresh page to preserve it. In above example the controller is bypassed and RR registers are used by NAND data recovery software, showing the dramatic effects of trying different thresholds.

lowering threshold may recover bitflips

Now if you put multiple copies of the data on the card which would give some protection, but it's not just user data that is affected by data bleed, the firmware that's stored on the same NAND suffers from it too. This could lead to total in ability to access the card, in fact firmware corruption, often as result of degraded NAND, is a very common cause for SD card failure.

In which we still can try recover the data by dumping the NAND and emulating controller algorithms for ECC, mixing and scrambling (example shows UFD, process for SD Card is identical):

enter image description here

It's not the question if it will happen, but when it will. It depends on several factors ranging from quality of components used, type of NAND, use the device saw (p/e cycles), 'strength' of the ECC algorithms, length of ECC codes and also environmental factors like storage temperature. But even if all these work in your favor, the SD Card will at some point fail.

From personal experience (my job is data recovery from small flash based devices such as memory cards and USB pendrives) I can share that I have seen cheap USB flash drives being accessible though with massive data corruption that were in some drawer less than a year, upto NAND based drives that lasted several years, SD cards that were completely unresponsive/inaccessible, etc..

In this case NAND corruption right in area where file allocation tables were located (inefficient wear leveling in combination with an area that is frequently written to is a bad idea) :

enter image description here

In summary I would strongly recommend against using SD Cards for (semi) permanent storage.

Joep van Steen
  • 4,730
  • 1
  • 17
  • 34
  • 1
    so after 20 years in a safe, all the bits would be bleed down to a useless charge, or is there still going to be a percentage of bits that are good? 20% ? 50% ? ...but the firmware on the SD would be useless, so it couldn't be salvaged, even with some sort of theoretical virtual file system that uses a 64bits to represent 1bit (and figures out which bit it's supposed to be?) --- I'm just about to drop my idea into the trash bin, but one last try question --- anyhow, it sounds like all the bits on NAND SD are slowly decaying at the same rate (not like random atomic decay??) – user12711 Nov 03 '22 at 20:05
  • 2
    Corrupt firmware will 'choke' the controller, we could still attempt unsoldering NAND chips or tapping into NAND chips on monolithic device. We recover those every day by doing what you propose, we take raw dump and try reconstruct a logical file system. But wait long enough and we will not be able to read meaningful data. I can not tell you if this happens after 2 years, or 5 or 10. No, not decaying, escaping would be better term AIUI. – Joep van Steen Nov 03 '22 at 20:50
  • 1
    One last question: if they stay "plugged in" to power, would that solve most of the problems. Could a tiny battery be used to provide just enough energy to keep everything charged up. And what would be the minimum amount? and I've heard or PRAM, could they design a single chip that has PRAM for firmware, and the rest being NAND? thnx2u!! – user12711 Nov 04 '22 at 16:06
  • 2
    @user12711, It's not about powered up that somehow keeps cells charged. They will bleed even with power, but power allows firmware to do maintenance IF the firmware does this kind of maintenance which I doubt on SD Cards. A modern SSD however probably will. But I'd still not recommend it for long term storage for reasons I can not explain in limited space of a comment. – Joep van Steen Nov 09 '22 at 01:16