15

I try to copy all files from an NTFS to an external drive and want to preserve all permissions and attributes. The external drive has already an NTFS and contains other files.

I've considered using the following tools:

  • ntfsclone does not work since it operates on sector level and would clone the whole FS, thus destroying the old data. If used to create an image file, I can't access the content from a Windows machine.
  • rsync does not preserve all meta data.

An alternative would be to use robocopy inside a running Windows, but I would prefer to copy the files without. (The source partition is used as the system's C:\.)

How can I copy the files?

Scolytus
  • 334
  • 1
  • 4
  • 17
  • 5
    What exact metadata are you trying to retain here that RSYNC won't retain with the appropriate parameters? Are you referring to music file software type metadata or actual FS permissions, structure, time, owner, etc.? I'm not sure what you've tried already with RSYNC but it would seem that using the correct parameters such as `-AXogt` for example should retain the FS metadata for the most part. Please explain what you have tried that does not work and what "metadata" you're seeing that is not retained between NTFS partitions. – Vomit IT - Chunky Mess Style Jan 24 '16 at 17:33
  • I know the files themselves can be copied exactly byte-for-byte in linux, but is it all the dates like created, modified, accessed that's missing? Or NTFS user/owner & permissions? Is there no way to change these dates & permissions on files after they're copied back to windows NTFS? – Xen2050 Jan 26 '16 at 11:17

4 Answers4

9

There is no way to exactly copy NTFS files, passing through Linux. Even Wine, the Windows-compatibility layer on Linux, works by converting permissions back-and-forth between Linux and Windows, and so is limited to their (rather small) least common denominator.

The only solution I can see is running Windows on Linux inside a virtual machine (or physical).

I believe you have mentioned this possibility in your post and would prefer to avoid it, but I do not see another possibility. Only Windows can exactly copy NTFS files; Linux is only capable of copying NTFS partitions.

This article might help: How to install and run Microsoft Windows for free on Linux, using free virtual machines made available for download by Microsoft.

harrymc
  • 455,459
  • 31
  • 526
  • 924
  • I'm not sure about it, but according to the [manpage](https://linux.die.net/man/8/ntfs-3g) of ntfs-3g, the option `streams_interface=xattr` is set by default. If I understand it correctly, this means that NTFS file attributes (including Windows permissions) are mapped to `xattr` attributes in linux. Since `cp -a` and `rsync -X` does copy this attributes, shouldn't it be an exact copy? – JojOatXGME Apr 15 '18 at 11:07
  • @JojOatXGME: The permissions models of Windows and Linux are completely different. Only the most basic attributes have an exact counterpart. – harrymc Apr 15 '18 at 11:25
  • I know. I don't claim that Linux does understand this attributes, but [xattr](http://man7.org/linux/man-pages/man5/attr.5.html) attributes are key-value-paris that doesnt have to be understood by the operating system. *ntfs-3g* seems to make Windows permissions available as xattr attribute `system.ntfs_acl`. The same seems to be true for other matadata in NTFS. Since xattr attributes are copied when using `rsync -X` or `cp -a`, it might copy the metadata as well. It depends on the concrete limitations of `xattr` and the implementation of `ntfs-3g`, but it looks possible to me. – JojOatXGME Apr 15 '18 at 14:51
  • so how can we test this properly? thats what we need with a statement like this. It could is only useful if it can be tested.... – U.V. Apr 22 '20 at 21:44
  • The xattrs on NTFS files, such as `ntfs_acl`, will not be returned when listing xattrs because they are in the system namespace. So `rsync -X` will not work by default. If ntfsprogs has been compiled with `--enable-xattr-mappings` (eg. Ubuntu) one can use a [special mapping file](https://github.com/tuxera/ntfs-3g/wiki/Using-Extended-Attributes) to map the system xattrs into the user namespace where they will be visible by the likes of `rsync -X`. But then the xattrs will be copied to the user and not system namespace, not exactly what you want. – crass Oct 21 '22 at 22:35
  • However, a script could be written to check all possible NTFS xattrs and copy them over using `getfattr` and `setfattr`. Its not clear to me that this is sufficient to resolve the OP's question (eg. is there other NTFS data associated with a file that is not exposed via the [xattrs](https://github.com/tuxera/ntfs-3g/wiki/Using-Extended-Attributes)). – crass Oct 21 '22 at 22:40
  • Actually, the `ntfs-3g` driver creates virtual xattrs, eg. `ntfs_acl`, and currently does not return them when listing xattrs, although I don't see why it couldn't. So its not that the xattrs are in the system namespace. Since these ntfs xattrs are not listed, rsync doesn't find any to copy. – crass Oct 22 '22 at 05:42
1

Linux can copy NTFS files contents fine, just not all the modified/created/accessed dates & attributes (I'm assuming that's what you want preserved).

So why don't you just make note of the current dates & attributes (dir should be able to display them), then copy the files in linux, and once you're running Windows again change the dates & attributes back to the originals.

Use a tool in Windows that can change the files, like one of these ones:

Apparently Windows' own File Explorer doesn't even preserve all the file dates properly either. But zip & cygwin's tar commands should save file dates, so using one of those in Windows to create an archive first should work too, then just copy the archive any old way.

If the attributes are super-important, and difficult to copy even in Windows, they should probably be backed up in a text file or database, or made into part of the filename...

Community
  • 1
Xen2050
  • 13,643
  • 4
  • 24
  • 42
1

First thing is, you need to make up your mind if you want to copy files, ...

or you want to make a complete, sector-by-sector, copy of your NTFS partition, including the "old data", the "other files", aswell "all the metadata" you have mention above very broadly.

For the latter, use dd:

dd if=/dev/sda2 of=/mnt/usbdisk/my-ntfs-partition-D.img bs=1M

Have fun.

Paxsali
  • 193
  • 1
  • 8
  • 2
    Well, I've made up my mind, and it's already in the question. I want to **copy** the files with all associated meta data. This should be possible because source FS as well as destination FS is NTFS, thus having the same capabilities. Using `dd` would be a poor choice since it does not take FS information into account. If you have an NTFS you should always use `ntfsclone` for image creation. – Scolytus Apr 01 '14 at 21:46
  • In this case use robocopyor ntfscopy (https://tzworks.net/prototype_page.php?proto_id=9), but neither of those are available on linux. – Paxsali Apr 01 '14 at 22:09
  • Well, I also mentioned `robocopy` in my question. So I appreciate your afford, but it's not really an answer to my question. – Scolytus Apr 01 '14 at 22:18
0

UPDATE 2023.08.08: cp and rsync still don't work out of the box. But tar still does and tar can be piped. This will keep timestamps. (Except maybe for ctime.)

See How to tar/untar the output on the fly

Dennis
  • 1
  • 1