syncing two folders into one, removing deleted files

Question

Given folders named Drive1/Folder1 and /Drive2/Folder2 I want to have a folder /Drive3/folder3 that contains the synced merged contents of both the source folders, but does not contain any files or directories that have been removed from the source folders.

/Drive1/Dolder1/File1
                File2
/Drive2/Folder2/File6
                File7
                File8
/Drive3/Folder1/File1
                File2
                File6
                File7

When File7 is deleted, the next sync removes it from the target Drive3 directory while leaving all the other files behind.

The only solution I can think of that could work is to create a /tmp/folder3 folder and then make hard links in that folder and rsync that folder with --delete, but that won’t work because /Drive1 an /Drive2 are not the same volume.

I thought I could do this with rsync. but everything I try either removes all of the Drive1 files or the Drive2 files each time.

Possible approach: create [some kind of union mount](https://superuser.com/a/1281559/432690) of the two source locations and sync *it*. — Kamil Maciorowski, Feb 05 '21 at 09:14
... where has file8 gone? Also, is scripting in powershell an option, perhaps? — GChuf, Feb 05 '21 at 09:29
bash/rsync/ other unix tools, no powerall. Reading the union mount link, but I don’t think I have any of the suggested tools available on this server (It’s a Synology NAS) — lbutlr, Feb 05 '21 at 09:47

score 0 · Accepted Answer · answered Feb 20 '21 at 14:58

The following will work for files that don't contain newlines in their names

Example

mkdir -p {Drive1/Folder1,Drive2/Folder2,Drive3/Folder}
touch Drive1/Folder1/File{1,2} Drive2/Folder2/File{6,7,8}

# Copy files from Drive1
rsync -av Drive1/Folder1/ Drive3/Folder/

# Copy files from Drive2
rsync -av Drive2/Folder2/ Drive3/Folder/

# Generate file lists
( cd Drive1/Folder1 && find -type f ) >d1f1
( cd Drive2/Folder2 && find -type f ) >d2f2
( cd Drive3/FolderM && find -type f ) | LC_ALL=C sort >d3f

# Compare them to get the list of files to delete from the destination (optional)

# Delete the list of files
comm -13 <(LC_ALL=C sort d1f1 d2f2) d3f |
    while IFS= read -r file
    do
        echo "Deleting $file" >&2
        rm -f "Drive3/Folder/$file"
    done

# Tidy up
rm -f d1f1 d2f2 d3f

syncing two folders into one, removing deleted files

1 Answers1