2

Background

What is the "correct" method to use rsync to backup data? The reason why I ask for the "correct" method is because my past experience of using rsync has resulted in permissions, file/folder mask, and timestamp related issues. Therefore, this question seeks to understand how can I use rsync to backup data without running into any permission, file/folder mask, or timestamp related issues, primarily when rsync is being used on a Windows system.

Another motivation for this question is to ensure that I don't into my previous issue where I was unable to access any files on my external drive which was backed up with rsync via cygwin. This was a nightmare because I had actually lost data on my main PC and was trying to restore it from the backup.

Below are similar issues I have ran into when using rysnc:

My understanding of rsync usage is intermediate and I have referred to the guides listed below. However, my biggest confusion of rsync usage comes from permissions, file/folder masking, Posix, etc.

Requirements/Assumptions

  • I want to use rsync in the context of "backup" and NOT in the conext of "sync". Often other users asking questions related to rsync may be interested in "syncing" data to ensure they have a working copy of files on another PC in sync. However, my intention is not for "syncing" but simply "backing up", such that if anything happens to my PC, I can get a last backup copy and restore it.
  • The original files that need to be backed up could exist on any operating system (ex: Windows, Ubuntu, MacOS), but primarily on Windows.
  • On Windows, rsync is used using Windows Subsystem for Linux (WSL) and NOT through cygwin
  • The destination for the backup could be an external drive formatted as NTFS, FAT32, or exFAT or it can be a network drive (ex: Samba share).
  • This is not required, but destination could be connected to using SSH

My current method & issues

I am using rsync through Windows Subsystem for Linux (WSL) on Windows 10. Below are example example issues I have run into:

Example 1: Not working

rsync -avhP --delete /mnt/c/Users/MyUsername/ProjectA /mnt/my_usb_mount/Backup

This method results in files NOT being transferred and errors like below:

rsync: chgrp "/mnt/my_usb_mount/ProjectA/src" failed: Operation not permitted (1)

and 

rsync: mkstemp "/mnt/my_usb_mount/ProjectA/src/.mydata.json.6cfTCf" failed: Operation not permitted (1)

Example 2: Partially working

rsync -avhP --no-p --delete /mnt/c/Users/MyUsername/ProjectA /mnt/my_usb_mount/Backup

or 

rsync -avhP --no-p --chmod=ugo=rwX --delete /mnt/c/Users/MyUsername/ProjectA /mnt/my_usb_mount/Backup

This method of using rysnc results in files being transferred but the timestamp (created, modified, accessed) are all set to the time of the transfer. If I run the rsync command again, all the files will be detected as modified and get re-transferred because of the incorrect timestamp. In addition I get errors like below:

rsync: chgrp "/mnt/my_usb_mount/ProjectA/src" failed: Operation not permitted (1)

Example 3: Correctly working. 

sudo rsync -avhP --delete /mnt/c/Users/MyUsername/ProjectA /mnt/my_usb_mount/Backup

This method work perfectly, however, I don't understand why I have to use sudo? Is it because I am using rsync on Windows through WSL? Would I have to use sudo if I were backing up files from a Linux PC (ex: Ubuntu)?

Although example 3 is a working solution, I still have many questions:

  • How do I know when sudo is required?
  • When do I use --no-p or --chmod=ugo=rwX
  • What if I don't have permissions to use sudo?
  • Will it work if I were backing up files to a network drive?
  • What if I am using SSH?
  • When using rsync from Windows through WSL, will I ever run into weird permission or file/folder mask issues as I previously have?
Zythyr
  • 271
  • 1
  • 3
  • 12
  • You need to use "sudo" because you are not root, and backing up files created with users other then yourself, including correct metadata requires root access. SSH won't directly help you, although rsync can (and often does) run over ssh. You would typically use sudo if backing up files you do not own in Linux. – davidgo Dec 04 '18 at 08:29
  • The --chmod=ugo=rwX would probably be required when your filesystem does not support Posix permissions, or you have read permissions but not the appropriate permissions for a file. If you don't have permission to use sudo, you get it or know you will loose some functionality for files you do not own. If you back up on a network drive then the permissions are limited by what the network drive supports. You can use tar archives if you want to preserve perms - but you may want to back up /etc/passwd as well which holds user mappings. – davidgo Dec 04 '18 at 08:32

1 Answers1

1

Ntfs and fat don't care about linux permission, you must make you're backup on a partition ext4

or make a backup and compress it into tar.gz and then decompress it on a linux system file and restore it

lejurassien
  • 91
  • 6