How do I recursively copy/download a whole webdav directory?

Question

When I attempt to copy a folder from a webdav server to a local disk using Nautilus, it copies what appeas to be a manifest file (xml with the directory listing etc..). With cadaver I get an empty file.

I would like to be able to recursively copy a whole directory tree. Does anyone know how I can do this?

ps: I'm using Ubuntu 11.04 with Nautilus 2.32.2.1 and Cadaver 0.23.3

Do you need a tool with a GUI or is wget "good enough" for you? — Ocaso Protal, Feb 14 '12 at 07:26
I had a peek at wget, and it didn't claim to do webdav... It would work for me if it did — drevicko, Feb 14 '12 at 07:35
Try this: wget --user={username} --password={password} --wait=20 --limit-rate=20K -r -p -U Mozilla http://{website} If it works Ill add it as an answer ;) — Rinzwind, Feb 14 '12 at 07:52
But webdav is just plain https, so wget should work. a quick google search gave me http://www.astron.nl/northstar/nswiki/lib/exe/fetch.php?id=quickstarts%3Aconfigure_repository&cache=cache&media=quickstarts:webdav_manual.pdf to get a quick overview over wget with webdav — Ocaso Protal, Feb 14 '12 at 07:58
interesting.. when I point it to a webdav directory/collection, it grabs the html file for that directory (ie: with the list of things there as a webpage) then complains: "localhost/webdav/calibre-library: Not a directorylocalhost/webdav/calibre-library/index.html: Not a directory". When I point it at the webdav root, its rather random... ends up starting to download the whold webserver.. Note that it's https only and self-signed, so I had to add '--no-check-certificate' — drevicko, Feb 14 '12 at 08:08
@Ocaso: following those instructions I tried "wget -nH -np --cut-dirs=1 --user={uname} --password={pwd} --no-check-certificate --wait=1 -r -U Mozilla https://my-host/my-webdav-dir/my-dir-in-webdav" ... not perfect (downloaded lots of 'index.html?C=M;O=D' and the like) but otherwise worked ok. Should be able to drop '--wait=1' I guess. Rinzwind: the '-np -nH' were handy.. One of you want to post an answer? (: — drevicko, Feb 14 '12 at 08:32

score 7 · Accepted Answer · answered Feb 28 '12 at 10:42

7

This answer summarises suggestions given in comments by @Ocaso and @Rinzwind.

I used this:

wget -r -nH -np --cut-dirs=1 --no-check-certificate -U Mozilla --user={uname} 
    --password={pwd} https://my-host/my-webdav-dir/my-dir-in-webdav

Not perfect (downloaded lots of 'index.html?C=M;O=D' and the like) but otherwise worked ok.

The "-r" downloads recursively, following links.

The "-np" prevents ascending to parent directories (else you download the whole website!).

The "-nH" prevents creating a directory called "my-host" (which I didn't want).

The "--cut-dirs=1" prevents creating a directory called "my-webdav-dir".

The "--no-check-certificate" is because I'm using a self-signed certificate on the webdav server (I'm also forcing https).

The "-U Mozilla" sets the user agent in the http request to "Mozilla" - my webdav server didn't actually need this, but I've included it anyway.

answered Feb 28 '12 at 10:42

drevicko

4,303
7
34
42

Using this to download a file from nextcloud it does download a file with the content: ```This is the WebDAV interface. It can only be accessed by WebDAV clients such as the Nextcloud desktop sync client.```. Any idea why that happens? – Micromegas Apr 14 '20 at 20:26
My guess is that nextcloud expects a user agent that looks like a WebDAV client (as opposed to Mozilla, which is a browser). If you have their client and a packet sniffer (eg: wireshark) you could find out what user agent it uses in it's requests. – drevicko Apr 16 '20 at 23:02
Alternatively find out that it's user agent could be "Mozilla/5.0 (Android) Nextcloud-android/3.8.0" from [the web](https://github.com/nextcloud/android/issues/4522). – drevicko Apr 16 '20 at 23:05
Thank you very much drevicko!! I'll do this – Micromegas Apr 16 '20 at 23:14
Also [here](https://github.com/nextcloud/android/blob/master/src/main/res/values/setup.xml#L20). Good luck! – drevicko Apr 16 '20 at 23:20

score 5 · Answer 2 · answered Feb 09 '17 at 15:42

5

Alternatively you can mount it as a path to be accessed as part of your own file system.

sudo mount -t davfs https://your.remote/path /your/local/mount/point

Note: /your/local/mount/point has to be a real existing directory for this to work.

As far as I know you only need to run the following to get the command to work: sudo apt-get install davfs2 (If more configuration is required I apologise, it was a long time ago that I did this.)

(I added this as an answer because I feel Liam's answer didn't give enough info.)

answered Feb 09 '17 at 15:42

JasoonS

482
1
4
15

using `sudo` to download a file doesn't really sound right. – Equidamoid Jan 16 '18 at 16:07
2

@Equidamoid Mounting filesystems requires root access because it can allow you to bypass local UNIX permissions in some cases. – Resigned June 2023 Dec 21 '19 at 19:06

score 3 · Answer 3 · answered Jun 04 '20 at 00:34

I realize that this is an old question, but I was wanting to do this and I found rclone which is like rsync for the cloud and supports a lot of different protocols including WebDAV. You use rclone config to configure a remote and then just do rclone copy remote:path . and int will download everything. You can also use rclone sync remote:path . and it will do checking of the files and only copy updates. It is a golang program so it is very portable.

score 3 · Answer 4 · edited Sep 08 '12 at 09:05

3

Actually with Cadaver you can cd to directory from which you want to download files and mget *.

edited Sep 08 '12 at 09:05

Eliah Kagan

116,445
54
318
493

answered Sep 08 '12 at 08:21

viktor denischik

31
1

2

For this to be a useful answer, you need to tell the user how to do this. – RolandiXor Sep 25 '12 at 19:20
1

Hm, `mget * .` in a folder or `mget foldername` results in `501 Not Implemented`. Did it really work for you on box.com? – alexanderadam Jan 07 '16 at 11:22
@Wurstsalat You mustn't put the extra '.' ; also it takes filenames as arguments, so you will need to do `mget foldername/filename'. – JasoonS Feb 09 '17 at 15:33
That doesn't work for me when there's subdirectories (the _recursive_ part). – Resigned June 2023 Dec 21 '19 at 19:05

score 3 · Answer 5 · answered Aug 09 '13 at 13:43

3

You can use dav2fs to mount the webdav server, and then you can access it as you would a local directory.

answered Aug 09 '13 at 13:43

Liam

427
4
13

also use this method, easy access to files and folders – bobbyrne01 Nov 04 '14 at 11:32
And this method is the slowest in comparison with others and won't always work (bugs related to filename-squirks). – alexanderadam Jan 07 '16 at 11:26

How do I recursively copy/download a whole webdav directory?

5 Answers5

Linked