Extremely slow pypi package retrieval with i.e. pip

Question

Finding python packages using tools such as pip or easy_install takes very long, up to ten minutes, on my local desktop running Ubuntu 12.10. The same procedures are lightning fast on my cloud servers.

Things ruled out:

The download of the package itself is fast enough
regular DNS lookups fast enough
when put a super-simple pypi mirror in between, with the 'simple pypi' interface, it is much faster.
the pip log does not show anything useful, just long listings of urls visited.
--use-mirrors doesn't help from mhausler's answer

Could it have something to do with the parsing of the listings?

I would be thankful for any tips on how to properly diagnose this, or what the problem could be.

Its super fast for the last couple of days! not sure why ... but previously it was super slow and times out occasionally. Maybe they got into some money or something. — Pykler, May 09 '13 at 17:51

score 8 · Answer 1 · edited Jan 23 '13 at 22:23

8

The main pypi index may be down or having problems.

Try executing

pip install --use-mirrors

to automatically use backup mirrors.

See pip install --help for details.

edited Jan 23 '13 at 22:23

Dennis

48,917
12
130
149

answered Jan 23 '13 at 21:37

mhausler

81
1

This is a great tip for many users, but does not solve my problem. – Jasper van den Bosch Jan 23 '13 at 21:47
What if you configured it globally? That and other suggestions [here](http://jacobian.org/writing/when-pypi-goes-down/). Pypi has been experiencing some hiccups again recently. – Tom Feb 08 '13 at 14:35
Yes, have tried several tiers of config; from pip.conf to commandline arguments. It works fine one some other machines, but all my config is the same (I keep it in version control) so it must be some package conflict or firewall issue or something. – Jasper van den Bosch May 09 '13 at 22:03
3

pip still have this option? When I tried to run `pip install -r requirements/production.txt --use-mirrors --default-timeout=3600` I got an `no such option: --use-mirrors` error. – Vini.g.fer Aug 09 '17 at 14:31
Very late response but no. Pip removed this option because pypi.org is no longer known for being slow, and it seems like there are very few mirrors today. – ioi-xd Mar 25 '22 at 02:13

score 3 · Answer 2 · edited Feb 05 '21 at 21:38

3

yes, the main pypi server is slow.

pickup another mirror server on http://www.pypi-mirrors.org/, then

pip install django -i http://g.pypi.python.org/simple/

replace the url with your fastest mirror.

also you can save the setting by using pip.conf, check http://www.pip-installer.org/en/latest/configuration.html for detail

edited Feb 05 '21 at 21:38

larsks

4,053
28
36

answered Jul 14 '13 at 02:21

aufula

31
3

ioi-xd · Answer 3 · 2022-03-25T02:51:20.083

This was asked when pypi.org frequently had network problems, which is no longer the case, meaning that it's unlikely that this is actually happening unless the Python foundation reports a pypi outage or you are generally experiencing network problems on your install.

There is now two more problems that you might have come here as a result of.

An issue in Pip 21.0, explained here. If this affects you, you'll either need to upgrade pip or use get-pip.py to install the latest one.
This open issue on the GitHub that affects some people on WSL.

In my case, there seems to be a general problems that throttles downloading files from the CLI, a problem I didn't even know could happen, however it seems wget and curl are affected too.

Extremely slow pypi package retrieval with i.e. pip

3 Answers3