9

Problem:

The bin directory of my virtual environment produced with virtualenvwrapper (which uses virtualenv under the hood) is located in the subfolderlocal instead which leads to breakage across the board (mostly virtualenvwrapper and vscode for now). Basically, these tools expect the interpreter in <envname>/bin and not in <envname>/local/bin.

Infos:

I already did some research and found the following:

  • There was a change in Ubuntu 22.04 that now uses posix_local in module sysconfig instead of posix_prefix as the default scheme which was explained here
  • Virtualenv at some point introduced a patch for Python 2 that changed the bin location to local/bin inside a virtual environment in response to the posix_local prefix, which was asked in this question and resulted in this patch which was supposed to fix this bug

System:

  • Ubuntu 22.04 LTS x86_64
  • python3 - 3.10.4-0ubuntu2
  • python3-virtualenv - 20.13.0+ds-2
  • virtualenvwrapper==4.8.4

Question:

  • How do I get virtualenv to install my environment into <envname>/bin again, ignoring posix_local option?
  • Is the reason why this was fixed still applicable for Python 3? (I must admit I did not really understand the reason why the patch was necessary in the first place.)
icezyclon
  • 436
  • 4
  • 13
  • I opened a related [issue on github](https://github.com/pypa/virtualenv/issues/2340), regarding the idea to use `venv` as creator of environment via `virtualenv`. This is still broken atm. – icezyclon May 20 '22 at 10:59

3 Answers3

11

Original Answer

The user yingmanwumen over on the GitHub of virtualenv posted a solution in this issue:

Basically, they changed the sysconfig scheme by exporting the following variable in their shell of choice:

export DEB_PYTHON_INSTALL_LAYOUT='deb'

This changes the default installation scheme from posix_local, which produced the problematic behavior, to deb_system, which has the normal/expected virtual environment layout again. The current installation scheme can be checked with this snipped:

import sysconfig
print(sysconfig.get_default_scheme())

It should be noted that this is a temporary solution. A real solution would probably include changing this behavior in virtualenv itself. yingmanwumen's issue as well as my own issue address problems with the posix_local scheme. The developers of virtualenv are open to accepting a PR to resolve the underlying issue. I will update this answer if a PR is proposed and accepted that resolves it.

Update

A PR was accepted solving this issue specifically for virtualenv, meaning that all versions of virtualenv > 20.16.5 should not have this problem anymore.

Additionally, it seems the bug happens specifically if:

[...] python3-distutils is not installed (or when not using the deadsnakes fork)

according to the proposer of the PR. So either installing python3-distutils or using the version of the deadsnakes fork should also fix this problem if it is encountered.

Lastly, efforts are taken to address this issue also in Debian/Ubuntu and with making the posix_prefix scheme more robust in general. This wasn't the first issue that happened due to confusing installation schemes as can be seen in this discussion.

icezyclon
  • 436
  • 4
  • 13
1

I had the same problem as you, and whilst I dont have the answers to your questions I did find that switching to venv solved my issues within a few minutes... It seems to be the modern, preferred 'one' way to manage python virtual environments.

source

How to setup venv

make_transition
  • 111
  • 3
  • Thank you for the answer, however there are two problems that I have with switching to `venv` from `virtualenv`: 1) `venv` is seemingly a subset of `virtualenv` as explained [here](https://stackoverflow.com/a/64586483). Especially creating environments with different Python versions might be finicky otherwise. 2) I personally use `virtualenvwrapper` and it seems it is not designed to wrap `venv` instead of `virtualenv`. It is discussed [here](https://stackoverflow.com/questions/45826517/python-venv-and-virtualenvwrapper-combined) but there does not seem to be a definitive answer. – icezyclon May 04 '22 at 19:59
  • I read the same thing about virtualenvwrapper and venv - i agree its not super clear, but I can confirm it does work. Which versions of python are you using? I am only using python 3 (3.10) which maybe why my scenario is easier. I am picking this project up after a long break hence things have broken and ive forgotten things. My approach is get it working as fast as possible and the understanding will follow... Im definitely not satisfied yet! – make_transition May 05 '22 at 09:38
  • FWIW these are my environment variables i need to set: export WORKON_HOME=$HOME/environments export VIRTUALENVWRAPPER_PYTHON=/usr/bin/python3 export VIRTUALENVWRAPPER_VIRTUALENV=/usr/local/bin/virtualenv source /usr/local/bin/virtualenvwrapper.sh export VIRTUALENVWRAPPER_ENV_BIN_DIR=bin Seems that virtualenv and virtualenvwrapper are still very much being used in someway! My confusion continues :) – make_transition May 05 '22 at 09:50
  • VIRTUALENVWRAPPER_ENV_BIN_DIR is being overwritten in virtualenvwrapper.sh, so you have to set if *after* sourcing, as you do, but I had to set it to VIRTUALENVWRAPPER_ENV_BIN_DIR="local/bin" because `virtualenv` still puts the bin directory into /local. This fixes `virtualenvwrapper` but other programs, like `vscode`, still expect the interpreter in /bin. I am not convinced that working around the local/bin directory is the correct move for me. My original question still stands. – icezyclon May 05 '22 at 11:42
  • I am using `virtualenvwrapper` which I still want to use, so I tried the following, inspired by your use of `venv`: Tried using an alias -`alias mkvirtualenv="mkvirtualenv --creator=venv"` - using `venv` as creator for `virtualenv`, because `venv` does not install into `local/bin`. This is [completely broken](https://github.com/pypa/virtualenv/issues/2340) at the moment. I am currently overwriting it like so: `mkvirtualenv() { python3 -m venv "$HOME/.virtualenvs/$1" }`. This works for now but is *not* want I ultimately want. I *would* like to use `virtualenv`... – icezyclon May 07 '22 at 19:00
1

A "solution" that worked for me was to downgrade setuptools: pip install setuptools==59.6.0

Tim
  • 11
  • 1