269

I'm just learning to use sftp, and I want to copy a directory from the remote computer to my local computer. If I try

get [directory]

it gives me this error:

Cannot download non-regular file: /home/mpirocch/Documents

get -R doesn't work, either.

Matthew
  • 14,496
  • 18
  • 73
  • 119
  • 2
    Recursive put/get was added to OpenSSH in 5.4, which was released in 2010. You may consider upgrading your servers as it also adds a lot of bug fixes and better security features. – Lèse majesté Sep 28 '12 at 04:50
  • 2
    Wow, sftp is basically useless as is. [`lftp`](https://superuser.com/questions/40281/how-do-i-get-an-entire-directory-in-sftp/726866#726866) helps a lot. – Eric Duminil Dec 10 '19 at 14:40
  • sftp supports now recursive mode: `get -R [directory]` – AndreyP Feb 08 '23 at 11:39

10 Answers10

401

Use the -R (recursive) flag:

get -R .
AndreyP
  • 105
  • 4
mani-fresh
  • 4,134
  • 1
  • 15
  • 2
  • 26
    Thank you for putting an answer that actually answers the question. – aVeRTRAC Jan 19 '12 at 19:56
  • 32
    Gives me: `get: Invalid flag -r` – ghbarratt Jul 18 '12 at 21:21
  • 3
    This also gives me Invalid flag. I'm not sure this is supported on all systems. – Ben Aug 28 '12 at 07:35
  • @Ben: When I asked this question in 2009, the `-r` flag did not exist. Is your system up to date? – Matthew Dec 10 '12 at 16:24
  • It works on all current systems. Just tested it. – buschtoens Jul 25 '13 at 10:03
  • 15
    @silvinci not *all* current systems. Illegal option with CentOS 6.4 openssh-clients-5.3p1-84.1.el6.x86_64 – CrackerJack9 Nov 05 '13 at 01:25
  • 1
    For some reason *uploading* recursively does not work with `put -r *` it can't create directories. However for me the lftp solution below worked just fine. – Ciantic Dec 22 '14 at 08:24
  • 1
    Anyone know the way to get -r to give directories write permissions? Everytime I try -r on a read only directory, it will not download it because it creates the directory locally read only as well. – Rahly Nov 05 '15 at 00:52
  • Not working with openssh-clients-5.3p1-111.el6.x86_64 on 2.6.32-573.12.1.el6.x86_64 #1 SMP Tue Dec 15 08:24:23 CST 2015 x86_64 x86_64 x86_64 GNU/Linux – user637338 Feb 26 '16 at 10:54
  • 1
    more specifically `mget -r ./*` to get all files and folders in the current directory. works in MINGW64 too. https://kb.wisc.edu/cae/page.php?id=32991 – SashikaXP Jan 25 '18 at 04:09
  • It works but gives an error then skips when finding a symlink : `not a regular file` – sinekonata Feb 21 '21 at 12:37
  • Correct option name is `-R`. Man: If the `-R` flag is specified then directories will be copied recursively. Note that sftp does not follow symbolic links when performing recursive transfers. – AndreyP Feb 08 '23 at 11:43
63

Use:

scp -r mpirocch@my-server:/home/mpirocch/Documents Documents
Cristian Ciupitu
  • 5,513
  • 2
  • 37
  • 47
  • 5
    This works, but it would be nice to be able to do this from SFTP (so I can see what I copy before I copy it). Is this possible? – Matthew Sep 12 '09 at 22:53
  • @Matthew: I don't think so. I have been asking myself the same question for some time and the best answer that I have for the moment is `scp -r`. Or you could use an advanced SFTP client like Filezilla. – Cristian Ciupitu Sep 12 '09 at 23:53
  • 1
    scp -r will follow symlinks... – jsleuth Feb 13 '13 at 15:57
  • 11
    Some systems may allow `sftp`, but not `scp` access. – Turion Aug 04 '17 at 09:41
  • 3
    This doesn't answer the question. – Eric Duminil Dec 10 '19 at 14:20
  • @EricDuminil have you seen the date? Unfortunately back then this was a good _alternative_, but if you want to be pedantic, yeah it doesn't answer the question and maybe deserves a downvote. – Cristian Ciupitu Dec 10 '19 at 15:22
  • @CristianCiupitu: It's been a while indeed, but many shared hosting servers still only allow sftp. `scp` simply isn't an alternative in this case. – Eric Duminil Dec 10 '19 at 15:38
  • @EricDuminil then take the neutral approach and don't upvote or downvote? – Cristian Ciupitu Dec 10 '19 at 17:24
  • Please don't take downvotes personally. The description on the downvote icon is "This answer is not useful". And well, `scp` isn't useful for a question asking specifically about `sftp`. – Eric Duminil Dec 10 '19 at 17:54
  • Eric is right. I came here searching for the answer about the recursive sftp as an alternative to rsync/rcp/etc. The recommendation to use another tool is not helpful. – Alex Apr 15 '21 at 16:01
50

Use lftp:

lftp sftp://user@host

Then, within lftp, cd into the directory you want to copy, and use the mirror command to recursively download the selected directory, like this:

mirror

This command accepts options and arguments:

mirror [OPTIONS] [source [target]]

For example, the -R (or --reverse) option will cause it to upload the local directory tree to the remote directory:

mirror -R

See the lftp(1) man page at the project’s site or at Debian.org for other commands and options.

Scott - Слава Україні
  • 21,399
  • 46
  • 64
  • 121
bshanks
  • 639
  • 5
  • 5
26

well this little guide should help, mirror a remote server to local folder with lftp

lftp sftp://user:password@server.org:22 -e 'mirror --verbose --use-pget-n=8 -c /remote/path /local/path'

  • sftp:// = uses SFTP protocol
  • mirror = mirror mode
  • verbose = shows the files being downloaded
  • use-pget-n = number of segments, realy useful to speed up big files
  • parallel = downloads multiplier files at the same time

if you want to download files in parallel switch out use-pget-n=8 with --parallel=8

hope this helps anyone needing to mirror a remote folder to a local folder

nwgat
  • 1,091
  • 1
  • 11
  • 12
  • Thanks, this worked for me, after the other methods failed (in particular no "-r" option on sftp). The first part of the command in my case needed to be quoted, due to dollar characters in the password `lftp 'sftp://user:pa$$word@server.org:22' -e 'mirror --verbose --use-pget-n=8 -c /remote/path /local/path'` – Jason Nov 09 '14 at 22:57
  • Excellent, thanks. I wanted to copy from local to server, so I used `mirror -R ...` – Eric Duminil Dec 10 '19 at 14:29
  • If you get `mirror: Fatal error: Host key verification failed.`, the host key fingerprint probably still needs to be accepted (i.e. use `ssh` / `sftp` to make an initial connection, and confirm the fingerprint with `yes`) – mwfearnley Jan 20 '21 at 10:25
16

Don't use the sftp program directly if you can find something better. For Linux, many file managers (at least Nautilus and Dolphin, the GNOME and KDE ones) support sftp natively, and there's always sshfs. For windows, there's WinSCP, and probably others. The point of all of these is to let you access files over sftp as if they were on a regular filesytem, so you don't have to care that you're accessing them over sftp.

Ryan C. Thompson
  • 11,563
  • 9
  • 47
  • 69
  • Wow, nautilus does work very well with SFTP. I added a bookmark in Nautilus, and now it's all ridiculously easy. Thanks! – Matthew Sep 13 '09 at 14:13
  • 1
    I've had performance issue with Nautilus SFTP. Yeah, it's really convenient but I wasn't able to saturate 1 Gbit/s network via Nautilus, while with plain `sftp/ssh` command line interface I was (cca 100 MB/s file transfers). So the issue was Nautilus on GNU/Linux Mint. – stamster Jan 25 '18 at 22:15
15
get -r [directory]

gets [directory] and everything under it, where r stands for recursive. I found this just by typing help from sftp.

slhck
  • 223,558
  • 70
  • 607
  • 592
drkvogel
  • 427
  • 5
  • 9
  • 2
    IMO this is the best answer to this question. :) – ramrunner Jan 24 '13 at 18:28
  • 6
    This is a duplicate of an [answer](http://superuser.com/a/216447/2357) from *Nov 29 '10* (almost one year and a half ago). – Cristian Ciupitu Jan 04 '14 at 19:43
  • 1
    @CristianCiupitu Not really. The answer you refer to recommends `get -r *` which will recursively get *everything* at the current directory level. My suggestion gets just a specific directory, which is what the OP requested. So I would argue that my answer is more accurate than the accepted answer, and I'll have my upvotes back, thankyou! ;) – drkvogel Jul 20 '22 at 11:36
  • Worked as a charm! – ADV-IT Apr 03 '23 at 22:40
13

Try mget instead of get.

Clarification: mget will work if you are inside the directory you want to copy; if you do something like this:

sftp> cd dir_to_get
sftp> mget *

it will get all the files in that directory. However, it will not recursively get the contents of any subdirectories.

Ken Keenan
  • 335
  • 3
  • 7
1

As with cp:

scp -rp user@host:/path/to/dir dir

The above will preserve times and modes of the original files and subdirectories. This is especially useful for the retrieval of backups.

Indrek
  • 24,204
  • 14
  • 90
  • 93
Aziraphale
  • 153
  • 1
  • 1
0

I recently solved this with lftp:

lftp -c '
open sftp://USER:PASSWORD@remoteserver.example.com:22
mirror --verbose --use-pget-n=8 -c /remote/catalogue/ /local/catalogue/
'

Inspired by this post.

Orphans
  • 184
  • 1
  • 8
0

I have Java dist folder in remote server, where i have following tree:

- dist
--- Audio.jar
--- README
--- lib
----- lib.jar

Goal is: I want to use SFTP? And put them in /tmp/<>

Step 1. sftp remoteuser@ip

Step 2. cd /var/tmp

Step 2. lmkdir /tmp/dist; lmkdir /tmp/dist/lib

Step 3. lcd /tmp/dist

Step 4. mget *

Step 5. lcd /tmp/dist/lib

Step 6. mget *

Step 7. finally i have my goal

$ ls
Audio.jar  lib  README.TXT
YumYumYum
  • 1,667
  • 7
  • 42
  • 66