quick question - I have a big requirements file. On one system, I have a couple requirements (pytorch, torchvision) which don't install on the a particular machine. Is there a way I can still use the file to install everything BUT these? Something like
pip install -r requirements.txt --except=pytorch,torchvision
I don't see anything like this in the pip options but maybe there's another way.
There is no way. You have to somehow process the list and exclude packages before passing them to pip. Something like
pip install `grep -v 'pytorch\|torchvision' requirements.txt`
If you are working in Bash, it might be a better idea to use process substitution:
pip install -r <(grep -E -v 'pytorch|torchvision' requirements.txt)
This way, you don't have to worry about comments etc.
Related
Most of you may know OpenAI playground, so I built an function generator app and followed all the instructions but I can't launch it via venv-python. I can actually install the requirements using pip while venv is not active and can launch the site with flask, but when I try to install requirements after activating venv, I get this error;
User#lalec ~
$ cd openai-quickstart-python
User#lalec ~/openai-quickstart-python (master)
$ . venv/Scripts/activate
(venv)
User#lalec ~/openai-quickstart-python (master)
$ pip install -r requirements.txt
Fatal error in launcher: Unable to create process using '"C:\Users\celal\openai-quickstart-python\venv\Scripts\python.exe" "C:\Users\User\openai-quickstart-python\venv\Scripts\pip.exe" install -r requirements.txt': The system cannot find the file specified.
How do I fix this? I added every possible script locations into PATH I thought it would help but no result.
I just realized two directories in the error doesn't match, and "C:\Users\celal\openai-quickstart-python\venv\Scripts\python.exe" actually does not even exist. Maybe that's the problem... How can I change this ??
Sorry if I'm asking dumb questions I'm new to all this... Also, I don't get why I need venv activated while I can just launch it by accessing the directory and type flask run in cmd, would appreciate any answers.
ss of the project's directory
ss of venv/Scripts/
ss of bash screen with errors
The error message suggests that the requirements.txt file can't be read. Check that you definitely can read it and it's in the expected location.
The perferred command for running pip is:
python3 -m pip install -r requirements.txt
This makes sure that you know which python3 binary you're running pip with - some systems have more than one python3 binary in various locations.
You could achieve similar without activating the virtual environment:
./venv/bin/python3 -m pip install -r requirements.txt
I am wondering if it is possible to install a conda package and pass arguments to the package being built. In homebrew, one can do brew install emacs --with-cocoa, brew install ffmpeg --with-x265, etc. However, conda install -c conda-forge emacs --with-cocoa tries to pass --with-cocoa to conda install, which fails. Is it possible to pass these sorts of arguments to conda's build process? Perhaps something like conda install -c conda-forge emacs --build-args="--with-cocoa".
From what I can tell the answer is no you can not. However, you may use something like Conda build recipes to customize your installations.
For your emacs example, this may be of use: https://github.com/conda/conda-recipes/tree/master/emacs to getting a better understanding.
I do not know if this satisfies your use case requirement. If not, I'll knock this answer down and see if there is a more plausible solution. Hope that it was of some help!
I have installed some packages with -e
> pip install -e git+https://github.com/eventray/horus.git#2ce62c802ef5237be1c6b1a91dbf115ec284a619#egg=horus-dev
I with pip freeze I see
> pip freeze
...
-e git+https://github.com/eventray/horus.git#2ce62c802ef5237be1c6b1a91dbf115ec284a619#egg=horus-dev
...
when I try to uninstall the packages I get errors:
> pip uninstall horus-dev
Cannot uninstall requirement horus-dev, not installed
> pip uninstall horus
Cannot uninstall requirement horus, not installed
How do I uninstall such a package?
At {virtualenv}/lib/python2.7/site-packages/ (if not using virtualenv then {system_dir}/lib/python2.7/dist-packages/)
remove the egg file (e.g. distribute-0.6.34-py2.7.egg) if there is any
from file easy-install.pth, remove the corresponding line (it should be a path to the source directory or of an egg file).
An easier way to do the same with the new version of setup_tools is to run the following:
python setup.py develop -u
Which basically does the same as what #glarrain describes in his answer.
Here's a demonstration, showing that eg you don't want to substitute a package name into that command:
.../pytest-migration$ python setup.py develop -u
running develop
Removing /home/me/virtualEnvs/automation/lib/python2.7/site-packages/pytest-migration.egg-link (link to .)
Removing pytest-migration 1.0.155 from easy-install.pth file
.../pytest-migration$
Install a dev package use cmd:
pip install --editable .
Uninstall:
rm -r $(find . -name '*.egg-info')
Now you can use:
pip uninstall package_name
or python setup.py develop --uninstall or python setup.py develop -u
Simply uninstall the package you installed in 'editable' mode:
pip uninstall yourpackage
it works for recent pip-versions (at least >=19.1.1).
It turns out that my installation was somehow corrupt.
I could find the entry in:
/usr/local/lib/python2.7/site-packages/easy-install.pth
To solve the problem I removed the line in the .pth file by hand!
import sys; sys.__plen = len(sys.path)
...
/absolute-path-to/horus # <- I removed this line
...
This is a bug on debian/ubuntu linux using OS-installed pip (v8.1.1 for me), which is what you'll invoke with sudo pip even if you've upgraded pip (e.g. get-pip.py). See https://github.com/pypa/pip/issues/4438
For a discussion on how to clean up see https://askubuntu.com/questions/173323/how-do-i-detect-and-remove-python-packages-installed-via-pip, though the solutions there are of the "remove everything" variety.
...pip packages [go] to /usr/local/lib/python2.7/dist-packages, and apt packages to /usr/lib/python2.7/dist-packages
...a few packages were installed in ~/.local/lib too.
For my system all I needed to remove was /usr/local/lib/python2.7/dist-packages/{package_name}.egg-link
I think I have something to add to all the answers here:
Using pip list you'll see all your installed packages, and there is a little trickery: a single pip install can create several entries in this list. In particular when you do an editable install, you'll have your <package_name> listed besides the location of the source on your disc.
This <package_name> is only used for pip and is never called in python as far as I understand, it is configured in your pyproject.toml, setup.cfg or setup.py.
Thus, to properly uninstall your package using pip, you should use this name and not the named of individual modules included in your package.
Hope it helps!
I am trying to remove and install again a package, 'python-keystoneclient' using pip. The problem is, when I run pip uninstall python-keystoneclient, it is showing successfully uninstalled:
root#openstack1:~/cinder# pip uninstall python-keystoneclient
Uninstalling python-keystoneclient:
Proceed (y/n)? y
Successfully uninstalled python-keystoneclient
but when I do a pip search python-keystoneclient, it shows:
root#openstack1:~/cinder# pip search python-keystoneclient
python-keystoneclient - Client library for OpenStack Identity API
(Keystone)
INSTALLED: 0.2.1 (latest)
I have tried installing the package again, but it shows requirement already satisfied. Not sure why this is happening.
Chances are it's installed somewhere else via a package or mechanism that PIP isn't entirely aware of. Plus last I heard, pip uninstall worked in the simple cases, but it was easy to fool and not entirely consistent on all platforms.
I've resorted to doing something like
python -c "import sys;import pprint;pprint.pprint(sys.path)"
to show me where python is potentially finding it's modules - that will show you if you've done something like left a local directory somewhere in the system path for python (happens to me after doing a "python setup.py develop" from a directory when I'm root and not paying attention).
You can also just ask keystoneclient where it's been imported from with a similar setup, which you can use to find and nuke it:
python -c "import keystoneclient;print keystoneclient.__file__"
You have to manually uninstall the keystone-client.
On my server, under
/usr/local/lib/python2.7/dist-packages/python_keystoneclient-0.3.2-py2.7.egg-info (you may have a different version number on your server)
you can find a installed-files.txt file, which records all the files installed by keystone-client.
Then, delete them and the related directories in dist-packages.
cat installed-files.txt | xargs sudo rm -rf
cd ..
sudo rm -rf keystoneclient python_keystoneclient-0.3.2-py2.7.egg-info
Now, keystoneclient has been fully uninstalled.
I have a Virtual env for my django project, but when I hit pip freeze, I get what must be a global site package list, includes too many packages, like ubuntu packages and so much irrelevant stuff. This happens whether virtualenv is active or not. My site packages list looks a bit slim too, so I wonder whether venv has been working at all.
(env)~/code/django/ssc/dev/env/lib/python2.7/site-packages> ls
django
Django-1.4-py2.7.egg-info
easy-install.pth
pip-1.0.2-py2.7.egg
setuptools-0.6c11-py2.7.egg
setuptools.pth
What's my problem?
If your virtual environment has access to the system's site-packages dir (ie. you used virtualenv --system-site-packages) then it's normal for the list to be a rather long one.
Compare the following:
$ virtualenv --system-site-packages v1 && source v1/bin/activate
$ (v1) pip freeze | wc -l # 100
$ virtualenv v2 && source v2/bin/activate
$ (v2) pip freeze | wc -l # 2
Can you try recreating the virtualenv?
Alternatively, adding a no-global-site-packages.txt file should tell pip to ignore the global site-packages:
$ touch $VIRTUAL_ENV/lib/python${version}/no-global-site-packages.txt
I don't understand why the most concise option was just left in the comments. Since I have just almost missed it I will put it here as a separate answer with some tweaks.
You can add --local flag with your pip freeze if you are running a virtual env with system-site-packages enables.
So, if you had:
py -m venv --system-site-packages env
To make sure you are not getting all system deps into your requirements.txt, just run:
python -m pip freeze --local > requirements.txt
Another, a bit more elaborate option, but still viable because dependencies are not supposed to change all that often, would be to go into pyvenv.cfg file located in your virtual env library and manually change:
include-system-site-packages = true/false