0

Can i run few processes in parallel of axel to download same file? Each time I start new process:

axel -a -n 10 http://image-net.org/image/ILSVRC2017/ILS                                                                                                                VRC2017_DET.tar.gz

I get notification:

Initializing download: http://image-net.org/image/ILSVRC2017/ILSVRC2017_DET.tar.                                                                                                                gz
File size: 60855983982 bytes
Opening output file ILSVRC2017_DET.tar.gz.0
State file found: 5103495158 bytes downloaded, 55752488824 to go.
Starting download

Then all of my parallel processes shows the same downloaded % on status bar.

axel -a -n 10 http://image-net.org/image/ILSVRC2017/ILSVRC2017_DET.tar.gz 

[  9%] [0    1    2     3    4    5    6    7    8    9   ] [   2,5MB/s] [ 5h57]

only speed and time left is different then

When i run in multiprocess i have 10mb/s total, otherwise 2mb/s for process only.

yourstruly
  • 117
  • 6

1 Answers1

1

According to the man page, axel will use multiple connections by itself, there is no need to run it multiple times.

Example download using 4 connection :

axel -a -n 4 http://mirror.switch.ch/ftp/ubuntu-cdimage/18.04/ubuntu-18.04-desktop-amd64.iso
Initializing download: http://mirror.switch.ch/ftp/ubuntu-cdimage/18.04/ubuntu-18.04-desktop-amd64.iso
File size: 1921843200 bytes
Opening output file ubuntu-18.04-desktop-amd64.iso.0
Starting download

[ 30%] [....0        .....1        .....2         ....3        ] [  11.1MB/s] [01:54]^C

The ....N shows clearly that it's using 4 connections

Example download using 6 connection :

axel -a -n 6 http://mirror.switch.ch/ftp/ubuntu-cdimage/18.04/ubuntu-18.04-desktop-amd64.iso
Initializing download: http://mirror.switch.ch/ftp/ubuntu-cdimage/18.04/ubuntu-18.04-desktop-amd64.iso
File size: 1921843200 bytes
Opening output file ubuntu-18.04-desktop-amd64.iso.0
State file found: 583252805 bytes downloaded, 1338590395 to go.
Starting download

[ 32%] [....0        .....1        .....2         ....3        ] [   9.9MB/s] [02:04]

There is still 4 connections! It seems that axel will re-use the partial download, but will continue using the same number of connections.

Example download using 6 connection (after erasing the temporary files):

rm ubuntu-18.04-desktop-amd64.iso*
axel -a -n 6 http://mirror.switch.ch/ftp/ubuntu-cdimage/18.04/ubuntu-18.04-desktop-amd64.iso Initializing download: http://mirror.switch.ch/ftp/ubuntu-cdimage/18.04/ubuntu-18.04-desktop-amd64.iso File size: 1921843200 bytes Opening output file ubuntu-18.04-desktop-amd64.iso Starting download

[  2%] [0        1        2         3        4        5        ] [     11.1MB/s] [02:40]

Now we can see axel using 6 connections.

Please note that the download speed may not be reduced because of the number of connection, and in this case you will only push more load to the network and to the remote host.

pim
  • 3,240
  • 3
  • 20
  • 37
  • problem here is that if u start one download u will have speed 10mb, and with second process u will have double speed so it seems normal it will attach to previous split of 4 parts, bcos its same file? i had to download 55gb, when i run 4 parallel axels, i had 90% downloaded in 1:30min with total speed 10mb/s; otherwise each process stick to this 2mb/s. Even when i had killed one process, total speed get down to 7mb/s :( – yourstruly Jun 19 '18 at 15:14
  • 1
    The man pages does not talk about running multiple instance of axel on the same file, it may go faster, but it will probably make corrupted files as well... – pim Jun 20 '18 at 07:24
  • indeed it resulted in broken file... but still it was faster than aria2c, wget and downthem all in this case of images tar.gz from imagenet – yourstruly Jun 28 '18 at 20:57