33

I'm trying to use a cURL command to download a file from an FTP server to a local drive on my computer. I've tried

curl "ftp://myftpsite" --user name:password -Q "CWD /users/myfolder/" -O "myfile.raw"

But it returns an error that says:

curl: Remote file name has no length!
curl: try 'curl --help' or 'curl --manual' for more information
curl: (6) Could not resolve host: myfile.raw; No data record of requested type

I've tried some other methods, but nothing seems to work.

Also, I'm not quite sure how to specify which folder I want the file to be downloaded to. How would I do that?

studiohack
  • 13,468
  • 19
  • 88
  • 118
Josiah
  • 433
  • 1
  • 4
  • 4

6 Answers6

40

Try

curl -u user:password 'ftp://mysite/%2fusers/myfolder/myfile/raw' -o ~/Downloads/myfile.raw

In FTP URLs, the path is relative to the starting directory (usually your homedir). You need to specify an absolute path, and that means using %2f to specify /. This is needed because the path in ftp: URLs is treated as a list of slash-separated names, each of which is supposed to be given to a separate CWD command. The %2f is decoded after splitting. See RFC 1738 and FTP URLs.

As for the output location, just give a path to -o.

Security suggestions:

  • Don't put your password in the URL. Storing it in ~/.netrc is not particularly secure either, but it at least is hidden from ps -ef.

  • Your password is sent in clear text. If the server supports it, use curl --ssl-reqd or curl ftps://mysite/...

  • Using SFTP (the SSH file transfer protocol) would be even better.

Community
  • 1
u1686_grawity
  • 426,297
  • 64
  • 894
  • 966
  • Worked great. Not quite sure I fully comprehend how the %2f is used, as you're also using /, but hey it works, so I'm not gonna complain. Thanks for the help! – Josiah Apr 01 '11 at 19:44
  • 1
    @Josiah: It seems that the URL "path" is split by `/`, and each argument is sent with a `CWD` command: `%2fusers/myfolder` as `CWD /users`, `CWD myfolder`. See [RFC 1738](http://tools.ietf.org/html/rfc1738#section-3.2.2) on this topic. – u1686_grawity Apr 01 '11 at 19:51
3
curl -T /users/myfolder/myfile.raw -u username:password "ftp://myftpsite/path/myfile.raw"

I use this all the time. It works like a charm.

jonsca
  • 4,077
  • 15
  • 35
  • 47
cUrly Stooge
  • 31
  • 1
  • 3
    Am I the first one to notice this answer is wrong? The -T parameter means "file upload", while OP asked for "file download" – Kar.ma Feb 01 '19 at 15:25
1

It is not cURL but commandline and works super:

If you're not bound to curl, you might want to use wget in recursive mode but restricting it to one level of recursion, try the following;

wget --no-verbose --no-parent --recursive --level=1\
--no-directories --user=login --password=pass ftp://ftp.myftpsite.com/
  • --no-parent : Do not ever ascend to the parent directory when retrieving recursively.
  • --level=depth : Specify recursion maximum depth level depth. The default maximum depth is five layers.
  • --no-directories : Do not create a hierarchy of directories when retrieving recursively.
  • --delete-after : can be added if you need to delete files after downloading.
  • --no-host-directories : to download right in '.' current folder, not create directory named by domain.
  • --no-clobber : skip downloads that would download to existing files
  • --continue : Continue getting a partially-downloaded file for more stability
  • combine with cd : to define the destination directory

So this sample can look like a bit more complicated:

cd /home/destination/folder \
&& wget --no-verbose --no-parent --recursive --level=1 \
--no-directories --no-host-directories \
--no-clobber --continue \ 
--user="login" --password="pass" ftp://ftp.myftpsite.com/

Feel free to use such command in crontab to automate delivery and local folder synchronization.

crontab -e

*/5 * * * * cd /home/destination/folder && wget --no-verbose --no-parent --recursive --level=1 --no-directories --no-host-directories --no-clobber --continue --user=login --password=pass ftp://ftp.myftpsite.com/

Or add the key --delete-after for automatic delivery of files from remote to local folder.

If you have special characters in this command (e.g. password) -- put it in a separate .sh file and configure crontab to execute .sh only. Crontab is really tricky in characters escaping.

Eugene Kaurov
  • 111
  • 4
1

Try:

curl "ftp://user:password@myftpsite/users/myfolder/myfile.raw"

(If the remote file name is 'myfile.raw')

  • When I try that, I get the following error: "curl: (9) Server denied you to change to the given directory". I believe that it why I needed to use the -Q "CWD" command. –  Mar 31 '11 at 21:22
0

If you're not bound to curl, you might want to use lftp , try the following;

lftp ftp://ftp.some.domain.com -u login@some.domain.com,'password' -e 'set ftp:ssl-allow no; mirror --Remove-source-files --verbose /remote/directory /home/destination/folder; bye'

Where

  • -u username
  • ,'password' : put your password rather in quotas
  • --Remove-source-files : deete remove files after you downloaded them
  • --verbose : good to see info but feel free to delete on production
  • set ftp:ssl-allow no; : only in case if you do not need to check certificate (not ssl)
Eugene Kaurov
  • 111
  • 4
0

As yan suggests,

curl "ftp://user:password@myftpsite/users/myfolder/myfile.raw"

should work, but some FTP servers use security policies that are not standards-compliant. In those cases, the --ftp-method singlecwd or --ftp-method nocwd option may help.

Dan
  • 26
  • 1