38

You can easyly backup your home folder on an external harddrive with

rsync -a --exclude=.cache --progress /home/$USER /media/linuxbackup/home/$USER

I excluded the .cache folder cause I think I will never need it when I have to re-install from this backup.

I found this list of all folders that I could exclude in a normal backup here:
What files and directories can be excluded from a backup of the home directory?

I created a list of this answer that contains some coments in this form:

#These directories may be excluded:

.gvfs                           # contains mounted file systems?
.local/share/gvfs-metadata
.Private                        # contains the actual encrypted home directory
.dbus                           # session-specific
.cache
.Trash                          # do I need to say more?
.local/share/Trash
.cddb                           # cached info about audio CDs
.aptitude                       # cached packages lists

#Flash-specific:

.adobe                          # Cache for flash, maybe others?
.macromedia   # except for Flash persistence, there is no reason to keep this

#Files:

.xsession-errors            # contains errors from the current graphical session
.recently-used              # recently used files
.recently-used.xbel
.thumbnails

Here is the full list at gist

How can I add this list to my rsync command?

wjandrea
  • 14,109
  • 4
  • 48
  • 98
rubo77
  • 31,573
  • 49
  • 159
  • 281
  • 1
    besides `.Trash`, there's frequently also `.Trash-1000` (under current Ubuntu-MATE 14 at least), so better settle for `.Trash-*`? – Frank N Feb 26 '17 at 12:43

4 Answers4

58

The exclude list may only contain filenames, foldernames and lines starting with #. A comment behind the foldername is not allowed. I created a Git repository with all known files and folders that are superfluous:

Download this ignorelist to /var/tmp/ignorelist

wget https://raw.githubusercontent.com/rubo77/rsync-homedir-excludes/master/rsync-homedir-excludes.txt -O /var/tmp/ignorelist

Then start the rsync with

rsync -aP --exclude-from=/var/tmp/ignorelist /home/$USER/ /media/$USER/linuxbackup/home/

Note:
In the ignorelist there is a commented section at the start with folders, that are probably not worth a backup either. Uncomment those, you don't need.

rubo77
  • 31,573
  • 49
  • 159
  • 281
  • 2
    You can combine the two commands, since `rsync` can read from `stdin`: `wget https://... -O - | rsync -a --progress --exclude-from=- ...` – muru Nov 04 '14 at 18:08
  • 10
    good point, but I think it is recommendable first to look at the ignorelist before starting the rsync – rubo77 Nov 04 '14 at 18:17
  • surprisingly `node_modules` isn't on the list – jchook Nov 03 '18 at 21:11
  • don't you want to keep `node_modules` in some of your config directories? Imagine, you want to move your home directory to a new computer, you would need to keep those then. – rubo77 Nov 04 '18 at 15:28
5

From man rsync:

 --exclude-from=FILE     read exclude patterns from FILE
          This option is related to the --exclude option, but it specifies
          a FILE that contains exclude patterns  (one  per  line).   Blank
          lines  in  the  file  and  lines  starting  with  ’;’ or ’#’ are
          ignored.  If FILE is -, the list  will  be  read  from  standard
          input.
muru
  • 193,181
  • 53
  • 473
  • 722
  • There is such a list, I edited my question – rubo77 Nov 04 '14 at 17:40
  • @rubo77 I stand by my statement that there's no list of *all* folders you *could* exclude. That's a recommendation, and I doubt even Lekenstyn would say it's a complete list. – muru Nov 04 '14 at 17:43
  • The list doesn''t have to be **complete** I just would like to create my backup faster so skip obvious useless folders – rubo77 Nov 04 '14 at 17:44
  • @rubo77 I think you'll have to shift the comments to different lines. – muru Nov 04 '14 at 17:52
  • thx, I updated the gist file so the comments are on single lines – rubo77 Nov 04 '14 at 18:04
0

Use rsync-homedir-excludes on Github ->

  1. download the excluded list (rsync-homedir-local.txt)->

    wget https://raw.githubusercontent.com/rubo77/rsync-homedir-excludes/master/rsync-homedir-excludes.txt -O rsync-homedir-local.txt

  2. Edit the file to meet your needs ->

    nano rsync-homedir-local.txt

  3. Create a backup directory (some examples)

    BACKUPDIR=/media/workspace/home/$USER

    BACKUPDIR=/media/$USER/linuxbackup/home/$USER

    BACKUPDIR=/media/$USER/USBSTICK/backup/home/$USER

  4. Use the "-n" parameter so rsync will simulate its operation. You should use this before you start:

    rsync -naP --exclude-from=rsync-homedir-local.txt /home/$USER/ $BACKUPDIR/

  5. Check for permission denied errors in your homedir ->

    rsync -naP --exclude-from=rsync-homedir-local.txt /home/$USER/ $BACKUPDIR/ | grep denied

  6. if it is all fine, actually perform your backup ->

    rsync -aP --exclude-from=rsync-homedir-local.txt /home/$USER/ $BACKUPDIR/

Aindriú
  • 141
  • 1
  • 6
  • I get `install: missing destination file operand after 'npkill'` – rubo77 May 03 '22 at 08:27
  • How is this related to the Question? – rubo77 May 03 '22 at 08:27
  • If you have done a lot of programming exercises and saved them in your home folder you are going to have lots of node_module folders.... you can remove them by running the commands above. Be careful to delete only node_modules in your programming exercises – Aindriú May 03 '22 at 12:17
0

Could try this if directories and files within are all you want backing-up. Excludes all hidden directories.

rsync -aP --exclude=.* /home/$USER/ /media/$USER/folder

arth
  • 9
  • 2
  • The shell could expand the `*` (maybe not in that place?). The paths should be quoted - the user name can not contain a space, but `/media/` could be `/Old Media/` or so... – Volker Siegel May 17 '15 at 17:02
  • I apologize I am not getting it. The '.*' is to not include any '.' pattern which are hidden directories at home directory '/home/$USER/' to backup destination: external hdd or pen drive '/media/$USER/folder'. By doing this, only the actual directories and files are being backup. If you're pointing something else, please elaborate. – arth May 17 '15 at 18:23
  • 6
    It is not really sensefull to exclude all directories starting with a dot, for example .thunderbird directory is one of the most important directories – rubo77 May 17 '15 at 21:34
  • @arth Try this in `bash`: `echo foo --exclude=.* bar`, `shopt -s nullglob`, `echo foo --exclude=.* bar`. – Volker Siegel May 17 '15 at 23:03
  • But I have to apologize for being terse and confusing. In part, I just had a reflex-like reaction when seeing unquoted globs and variables - it's a very frequent problem - but as you say, the variables may be actually save. – Volker Siegel May 17 '15 at 23:10
  • @Volker I was just making a suggestion. No need for condescending. Thanks. – arth May 18 '15 at 00:41
  • @arth Oh, absolutely not meant as condescending tone - it was a honest attemt to be polite, for making up for the initial terseness... (English is not my native language) – Volker Siegel May 18 '15 at 01:38
  • @arth Adding to the example above: `shopt -s nullglob`, `echo foo --exclude=.* bar` - here, the shell is interpreting the `*` (the command never sees it), `echo foo --exclude='.*' bar` - in this case, the command is seeing the `*`. With the default bash options unchanged, the shell tries to interpret the `*` in the first case, fails, and keeps it - so the command can see it. `nullglob` makes it insert nothing if nothing is found for a glob. – Volker Siegel May 18 '15 at 01:57
  • From this, I would expect that `rsync -aP --exclude='.*' /home/$USER/ /media/$USER/folder` would prevent the `*` problem - but I did not test it. – Volker Siegel May 18 '15 at 02:05
  • @Volker I see. I tested yours as stated and resulted the same as mine. However, I'll add that to my additional solution notes as my files grows and gets complicated. Thanks. – arth May 18 '15 at 02:45