Pip: install --find-links on multiple folder - pip

I create a virtual environment with the command:
python -m venv ./my_venv
After activating this virtual env, I want to install some packages with .tar.gz archives.
My method to install my packages is explained in the following part:
PACKAGE1='/path/to/the/folder1'
PACKAGE2='/path/to/the/folder2'
PACKAGE3='/path/to/the/folder3'
PACKAGE4='/path/to/the/folder4'
pip install --find-links="${PACKAGE1}" package_1==1.0.0
pip install --find-links="${PACKAGE2}" package_2==1.0.0
pip install --find-links="${PACKAGE3}" package_3==1.0.0
pip install --find-links="${PACKAGE4}" package_4==1.0.0
Now I want to use a file requirements.txt like:
package_1==1.0.0
package_2==1.0.0
package_3==1.0.0
package_4==1.0.0
and run the command:
pip install --find-links="${PACKAGE}" -r requirements.txt
but
I don't find the way to tell pip to install using multiple folder.
Here is an example of what I want:
PACKAGE1='/path/to/the/folder1'
PACKAGE2='/path/to/the/folder2'
PACKAGE3='/path/to/the/folder3'
PACKAGE4='/path/to/the/folder4'
pip install --find-links="${PACKAGE1}" --find-links="${PACKAGE2}" --find-links="${PACKAGE3}" --find-links="${PACKAGE4}" -r requirements.txt
Obviously, I didn't work.
Can you help me ? :)

Have a look at the --find-links documentation again:
If a local path ... that’s a directory, then look for archives in the directory listing.
So /path/to/the/folder4 for example will need to contain an archive/wheel for package4. This will not work if /path/to/the/folder4 is just the source directory/root for package4.

Related

How to upgrade pyenv (macOS) such that a fresh .venv contains an up-to-date pip?

I'm trying to prevent this warning every time I create a fresh .venv:
> /Users/pi/.pyenv/versions/3.10.0/bin/python -m venv .venv
> . .venv/bin/activate
> pip install ipykernel # or anything
WARNING: You are using pip version 21.2.3; however, version 22.2.2 is available.
You should consider upgrading via the '/Users/pi/code/foo/.venv/bin/python -m pip install --upgrade pip' command.
Somehow pyenv has populated my fresh .venv with an out-of-date pip.
If I execute the suggested command it will upgrade my .venv's pip. But I don't want to be doing that every time I create a .venv.
I figured this might fix it, but it doesn't:
> /Users/pi/.pyenv/versions/3.10.0/bin/python -m pip install --upgrade pip
Requirement already satisfied: pip in /Users/pi/.pyenv/versions/3.10.0/lib/python3.10/site-packages (22.2.1)
Collecting pip
Using cached pip-22.2.2-py3-none-any.whl (2.0 MB)
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 22.2.1
Uninstalling pip-22.2.1:
Successfully uninstalled pip-22.2.1
Successfully installed pip-22.2.2
What is actually happening when I execute the above command? I was expecting it to update the pip for the python version created/maintained by pyenv. Which it seems to be doing:
🧢 pi#pPro18-4 ~/.pyenv/versions/3.10.0
> find . -name 'pip*'
./bin/pip3
./bin/pip
./bin/pip3.10
./lib/python3.10/site-packages/pip
./lib/python3.10/site-packages/pip-22.2.2.dist-info
🧢 pi#pPro18-4 ~/.pyenv/versions/3.10.0
> ./bin/pip --version
pip 22.2.2 from /Users/pi/.pyenv/versions/3.10.0/lib/python3.10/site-packages/pip (python 3.10)
So why isn't this pip getting copied into my .venv when I create it?
I thought that was the way .venv creation worked.
How to clean up my pyenv Python installation so that it spawns up-to-date .venvs?
EDIT:
Insight from #python on IRC/Libera:
grym: I don't think you can; i just get in the habit of python -m venv somevenv && somevenv/bin/python -m pip install --upgrade pip setuptools wheel
jinsun: python -m venv --upgrade-deps .venv is a simple solution if you were just annoying by the pip warning (...) it is updating the pip inside the venv, forget about the base python, I don't even have pip in the base python
This is the use case for pyenv-hooks
pyenv-hooks are scripts that are executed by pyenv whenever certain commands are run. You can create hooks for regular commands like: exec, rehash, which, but it can also be a plugin command, like virtualenv. The scripts can be written in any language.
Here is the wiki with official instructions.
You can have a hook by creating a script at the following location:
$PYENV_ROOT/pyenv.d/<hook-name>/<your-script-name>
For example, to create a hook that upgrades pip, create a new script within this path:
$PYENV_ROOT/pyenv.d/virtualenv/after.bash
With contents:
after_virtualenv 'PYENV_VERSION="$VIRTUALENV_NAME" pyenv-exec pip install --upgrade pip'
after_virtualenv is the command that tells pyenv when to execute. First, it sets the pyenv version to the name of the virtualenv we just created. with the variable $VIRTUALENV_NAME. Then it upgrades pip itself.
More details in this article.
I originally posted it as a comment, but was suggested to make it a proper answer.
An easier approach is to use the upgrade-deps flag when you create a virtual environment. Like this:
python3 -m venv --upgrade-deps .venv
It was added on python3.9, and according to the official docs:
--upgrade-deps
Upgrade core dependencies (pip, setuptools) to the latest version in PyPI
So, in other words, it will install pip and upgrade right away.

Ansible installation folder

I have installed Ansible on Ubuntu but it has installed it in the following directory:
/home/phill/.local/lib/python3.8/site-packages
rather than the expected /etc/ansible.
The command I used to install it was " python3 -m pip install --user ansible"
How do I get it to install in the /etc/ansible directory?
You don't want to "install" it to /etc/ansible. /etc is used for configuration files. You should read about the Filesystem Hierarchy Standard (FHS).
However, you can install ansible globally and create the /etc/ansible directory
sudo apt-get install ansible

Install subset of packages listed in a requirements.txt file with PIP

I'm wondering if it's possible to install a subset of the packages listed in a requirements file using the PIP CLI command or do I have to wrangle stuff in a makefile to get something like this?
pip install -r [package1, package2] requirements.txt
Regards
Not sure it would do exactly what you have in mind, but you could try something like:
python -m pip install Something Another --constraint requirements.txt
See pip' documentation section on constraints files.

pip install -r requirement.txt on virtualenv doesn't install locally but globally, even after activation

I tried to install all dependencies in my requirements.txt (a bunch of packages list).
What I did:
created virtual env virtualenv my_env
activated the new virtual env, I'm able to see my virtual env before the prompt (my_env) $
ran pip install -r requirements.txt
All packages got installed, but when checking with pip freeze I have nothing. Tried to deactivate the virtual env, and made pip freeze again, here I have all installed.
I'm a bit confused because, I'm very sure my virtual env was activated, and I have the right pip path when doing which pip inside it (/home/virtual_env/my_env/bin/pip). Plus, I tried to install one by one the dependency, and they got installed right inside the virtual env & displayable with pip freeze
I cannot do all of them one by one, and I need to reproduce the installation somewhere. Could someone helps on this?
Still no clean solution for this so far, but what would work is to copy-edit (search & replace the return character in the requirements.pip to && pip install. Meaning, edit it from this format
package1==vX.Y
package2==vU.V
...
into this
package1==vX.Y && pip install package2==vU.V ...
Add pip install at the beginning then make a copy to all for install command like
pip install package1==vX.Y && pip install package2==vU.V ...

How do I install pip modules on google compute engine?

I am trying to run some python script using ssh to log into the google compute engine but all the installed pip modules are not found as I do not have permission to the .cache/pip folder in my user is there a correct way to do this?
You should be running this with the root user.
Also, if you need pip inside your GCP Instance, you can use the following commands:
sudo curl "https://bootstrap.pypa.io/get-pip.py" -o "get-pip.py"
sudo python get-pip.py
[Source]
Use:
sudo apt-get install python3-pip
sudo runs this command as an administrator
apt-get is the standard package manager used on Debian Linux distributions
python3-pip is the package name for pip3
Once installed, you can install PIP modules with:
pip3 install MODULE_NAME
for example:
pip3 install tensorflow
I'm not entirely sure there is one correct way to do this, but an easy way would be to use the conda python package manager.
The lighter version of it is miniconda. You can get a minimal python installation with pip preinstalled, and virtual environments capability if you need. Assuming you are running on linux and want python 3, you'll have to run
wget https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
and then install conda with
bash Miniconda3-latest-Linux-x86_64.sh
At the end of this process you should have a minimal python installation (that includes pip) and you'll be able to install packages with pip as you are used to.
You might want to install some basic libraries first -
sudo apt-get install bzip2 libxml2-dev
Then install miniconda as given by #teoguso and restart your shell
source ~/.bashrc
You can then use conda or pip to install your packages

Resources