24

cp -l hard links files instead of copying them, saving filesystem space. I need to use rsync instead of cp because of its --exclude capabilities.

So my question is, how do I get rsync to hard link files instead of copying them? Obviously, this is a local filesystem copy. I've read the docs for rsync's -H option, but it's unclear to me if this behaves the same way as cp -l.

nnyby
  • 1,359
  • 3
  • 13
  • 22
  • 1
    It looks like I can get this behavior with `--link-dest=$SOURCE_DIR`: https://slaptijack.com/system-administration/rsync-backups-conserve-disk-space-with-hard-links/ I'm testing this out now. – nnyby Sep 22 '18 at 18:33
  • Generally, `rsync` is used for back-up or archive purposes. Why would you want a back-up of file names without their contents? – AFH Sep 22 '18 at 18:37
  • 1
    @AFH my backup script creates rolling backups of gigabytes of data at many points in time, but most of this data is the same. I save disk space by copying these between time points with `cp -al`. Without using hardlinks, all these different copies of data (if they were real files) wouldn't fit on my 2TB drives. See: https://slaptijack.com/system-administration/rsync-backups-conserve-disk-space-with-hard-links/ – nnyby Sep 22 '18 at 18:42
  • 1
    So you can recover from accidental deletions, thought not overwrites, but you'll never be able to use your back-up to recover from a disc failure or partition loss. I use `cp -au`, which copies only changed files, so the process doesn't take too long, but it does give me an external copy of the latest version of every file for recovery purposes. Of course, to back up Gigabytes of data you need Gigabytes of storage, but if your files are important you need to make that investment. Only unimportant files are not backed up, by definition. – AFH Sep 22 '18 at 19:02

2 Answers2

24

The answer has turned out to be yes, with rsync's --link-dest option. It's just not obvious because it's not a simple on/off flag like cp -l, because it takes a path argument.

So, cp -l a/ b/ can also be done like this:

rsync  -a  --link-dest=../a  a/  b/

Note that the destination needs to be ../a, because the path is relative to the relative to the target directory.
(So if you were syncing to b/c/, it would be ../../a.)

mwfearnley
  • 7,172
  • 5
  • 26
  • 38
nnyby
  • 1,359
  • 3
  • 13
  • 22
  • 2
    Did you try this command yourself? The man page says that --link-dest option, if relative, is relative to the target directory. This answer is more correct: https://serverfault.com/a/210058/251416 – Yaroslav Nikitenko Jan 31 '20 at 18:11
  • I tried it out; you're right. @nnyby would want to do `rsync -a --link-dest=../a a/ b/` – Edward Falk Feb 22 '21 at 19:33
2

I've been using https://github.com/rsnapshot/rsnapshot for a long time. It does exactly what you are describing.

ericpe
  • 21
  • 1
  • My rsnapshot job seems to be creating full copies (ie different inodes). And my free space is going down by non-trivial amounts. Any idea what I'm missing? – Sridhar Sarnobat Aug 13 '21 at 05:51
  • 1
    @SridharSarnobat Check your rsnapshot config file and make sure you have the link dest feature enabled. – jw013 Dec 21 '21 at 22:14