Recover a file after shred command

Question

Let's say I have some sensitive files in my computer and I want to completely delete them so that no one can ever retrieve them. For that, I can use the rm command, but it's still easily recoverable. So I choose to use the shred command that will overwrite my file with noise and then delete it.

> shred -n 120 -u my_file.txt

However, I'm still not sure my method is completely safe.
What can be done in order to recover my file after I shred it ?

Prerequisite

let's say I have access to every forensic software available and lab equipment to recover my data.
Let's also say I want to preserve the physical integrity of my hard drive (so no microwave, fire, or any physical damage).

Research

I also tried :

dd if=/dev/zero of=my_file.txt bs=16M status=progress

In order to replace the content of my file with 0s. But I don't know if this method is better than shred.

Answer 1

I'm still not sure my method is completely safe.

It isn't:

Shred basically relies on a very important assumption: that the file system overwrites data in place. This is the traditional way to do things, but many modern file system designs do not satisfy this assumption

Source: https://unix.stackexchange.com/questions/656146/how-to-use-the-shred-command-the-right-way#comment1234353_656147, comment by Daniel Kelley

The only way to make recovering of your sensitive data nearly impossible is to overwrite (“wipe” or “shred”) the data with several defined patterns.

CAUTION: The use of wiping or shredding tools relies on a very important assumption: that the filesystem overwrites the data in place. This is the traditional way to do things, but many modern filesystem designs do not satisfy this assumption for example Ext4, XFS, ReiserFS, etc.

Source: SystemRescue - Secure Deletion of Data

Shred does not work on SSDs:

Writing a block to an SSD does not overwrite the old block. That's because all recent SSDs use something called "wear levelling".

To write a block to an SSD, you need to erase it first, and then you can write the new data. But erasing is an operation that can only be executed a limited number of times; each time you do an erase, you "weaken" the hardware, until the block cannot be properly erased any more.

So instead of erasing and overwriting the same block, wear levelling will make the SSD pick a different, unused block, and will write to this block, leaving the data on the old block in place.

And if the data on the old block is in place, that means it still can be read.

So any of the commands you can use to "overwrite" a file (cp, dd, cat, shred, and many more) have this weakness: It does not actually overwrite the file at all, instead it writes zeroes, random data or whatever to new blocks.

So unlike for HDs, this is not a good way to make sure your data is gone, and cannot be read by someone else.

Source: Is shred bad for erasing SSDs? - Unix & Linux Stack Exchange, answer by dirkt

---

Further Reading:

• destruction - Security of linux shred utility - Information Security Stack Exchange

Answer 2

Let's say I have some sensitive files in my computer and I want to completely delete them so that no one can ever retrieve them.

This in itself is a request that can never be satisfied. If we assume a conventional, magnetic drive then all research and literature I have ever read says it's impossible to recover meaningful data even after only a single pass of zeros.

Example: https://www.researchgate.net/publication/221160815_Overwriting_Hard_Drive_Data_The_Great_Wiping_Controversy

This study has demonstrated that correctly wiped data cannot reasonably be retrieved even if it is of a small size or found only over small parts of the hard drive. Not even with the use of a MFM or other known methods. The belief that a tool can be developed to retrieve gigabytes or terabytes of information from a wiped drive is in error.

Can not be recovered, now!

But one could still argue I didn't locate the correct research yet or although it may be true that this is the current state of affairs, it is not a guarantee that no one in the distant feature may be able to recover the data.

Is overwrite/shred in-place?

Since OP is discussing a single file, to overwrite the contents of the file (or shred or whatever you want to call it), we have to assume the file is overwritten in-place at file system level. So file is in clusters 10 - 20, shredder now gets to actually save the shredded file to clusters 10-20. If file system however decides to save to clusters 100 - 110 instead for whatever reason, the original file contents survive, the file system simply no longer points to it. In such an event the simplest of en user file recovery software can potentially recover the data you intended to overwrite/shred.

Modern drives may not all do what you expect them to..

Even IF we assume file system cooperates, a modern drive like and SSD or SMR drive will almost by definition write the shredded data to a different physical location. Added to that, modern drives may before writing the blocks to a 'final destination' cache the 'file', or better file data. For example many SMR drives reserve a CMR area for fast writes (Media Cache) where the data is written to, and later written to the slower SMR areas. Until the data in the CMR area is overwritten, 2 copies of your data exist.

The process for hybrid drives (SSHD) is similar.

On an SSD even multiple copies of your file may exist ..

SSD's are even worse than that due to the way data is organized, 'programmed' and erased. Due to characteristics of NAND, and how much can programmed at once (a page) and erased at once (an erase block) and that only an erased page can be programmed, it almost by definition never writes to the physical location you read from. IOW 'your shredder' reads file, scrambles the data and saves that. But the SSD will thus not overwrite the original data since the scrambled data is written somewhere else.

In addition, without you knowing, multiple copies of your file may exist as SSD's tend to shuffle data around all over the place (scrubbing, wear leveling and whatnot) with the intention to erase the stale copy at some point. Researchers found upto 16 copies of a file that was saved to SSD only once! Since only one copy is mapped to LBA, to to us, the end users it is as if only one copy of this data exists.

See: https://www.usenix.org/legacy/events/fast11/tech/full_papers/Wei.pdf

Figure 1: Multiple copies This graph shows The FTL duplicating files up to 16 times.

So even if you succeeded in shredding your file in-place, many stale copies of the file may still exist, and may be recoverable with the right tools!

So, no your method is not safe.

However, I'm still not sure my method is completely safe. What can be done in order to recover my file after I shred it ?

After reading the previous, it should be obvious.

All we need is a way to access this space in which these 'stale' copies of data still exist.

Such modern drives typically work with a 'translator' or 'translation layer' (FTL) that maps LBA addresses to physical addresses, keeps track of status of page, wear, etc..

If an SSD saves your file, and even if file system writes to the exact same location, it finds an erased page to write the data to. The page the data was read from is not by definition. However for the OS it seems as if data was written the same range of LBA blocks. The SSD accomplishes this by simply mapping a different physical location to those LBA blocks.

At the NAND level we now have two copies of your file, the original in a location no longer mapped to any LBA address, we call this stale data, and the new file (the shredded variant).

So, all we need is a tool that allows us to access the data that's outside the 'LBA domain'.

Option 1:

PC3000 SSD by Acelab. Almost every data recovery lab has this tool. To access your 'stale data' it is required though that the tool 'knows' the SSD model. Even if a specific model is not supported at this point, it may be in the future. New models are added constantly.

See: https://blog.elcomsoft.com/2019/01/life-after-trim-using-factory-access-mode-for-imaging-ssd-drives/

Option 2:

Dump the NAND and bypass controller altogether.

Example: http://www.flash-extractor.com/library/SSD/IDX110M00/IDX100M01-LC%20v2__2c_68_04_46__16x1

On modern NAND this impossible though as although we can read the NAND, the data is encrypted and can (at this point) not be decrypted since we lack assistance of the controller.

Example: http://www.flash-extractor.com/library/SSD/SandForce/SF-2281VB1-SDC-S03__2c_88_04_4b__4x4

With regards to SMR drives, the most recent update (at day of writing this) of PC3000 further enhances support for such drives: https://youtu.be/PylbEVGWo3M

Answer 3

It is useful to look at different levels of security.

1. You are having user rights. Them a simple rm will be enough. Take care of other copies, e.g., in backups and filesystem accessible snapshots.

2. You have root privileges. Then you may access any snapshots and any raw data on the disk. On some filesystems shred will work, on others not. Especially COW filesystems like btrfs, ZFS and nilfs2 are a problem. ext3/4 in journal write-back mode is also a problem.

3. You're hacking the drive's firmware. Then you may have the possiblity to access remapped blocks and wear-leveling blocks. There may be alternative firmwares that make it easy and there may be homebrew firmwares or raw-access modes for that. This will especially allow to access spare blocks on SSDs.

4. You're accessing the drive on hardware level. Then you can, for example, try to read bad blocks, that cannot even be read with a firmware that ignores the block being marked as bad because they are actually bad. If the blocks are indeed defect (and not only hard to read) they are almost impossible to restore on modern drives. In the past there were methods reading blocks with more expensive equipment, but with drives getting more and more storage with increasing complex methods most of these methods won't work anymore. If someone tells you to wipe 35 times, they are talking about 100 megabyte drives and not 10 terabyte drives.

The most common threat model would be a root-level access. And there you have to deal with the filesystem features. I'd recommend encrypting the drive and preventing people from getting root access on the running system. Then you're fine in most scenarios. Encryption will also prevent problems in the more sophisticated scenarios.

Answer 4

Take your SSD and use various tools to physically "shred" the device. Hammers, angle grinders etc are useful.

Then chuck the fragments in the nearest river.