I am trying to migrate from pip to pipenv and have my projects running on a pipenv environment. The steps I'm taking to set up a pipenv environment are listed below:
Create a folder name "Project" & set the dir to "Project"
pip freeze > requirements.txt to create a requirements.txt file containing packages installed under pip environment.
pipenv --python 3.9 to set a specific version of python
interpreter to use under pipenv environment
pipenv shell to run pipenv environment
pipenv install -r ./requirements.txt to install packages listed in the requirements.txt
Questions...
Q1.
On step 3, I am not sure if I should have executed pipenv install --python 3.9 instead of pipenv --python 3.9. Are they doing the same thing?
Q2.
I am also curious why I'm supposed to run step 3 before 4, not after 4. To me, it seems more reasonable to install python version after you're inside the pipenv environment.
Please let me know if the steps above are incorrect or additional steps need to be taken.
Related
I'm trying to prevent this warning every time I create a fresh .venv:
> /Users/pi/.pyenv/versions/3.10.0/bin/python -m venv .venv
> . .venv/bin/activate
> pip install ipykernel # or anything
WARNING: You are using pip version 21.2.3; however, version 22.2.2 is available.
You should consider upgrading via the '/Users/pi/code/foo/.venv/bin/python -m pip install --upgrade pip' command.
Somehow pyenv has populated my fresh .venv with an out-of-date pip.
If I execute the suggested command it will upgrade my .venv's pip. But I don't want to be doing that every time I create a .venv.
I figured this might fix it, but it doesn't:
> /Users/pi/.pyenv/versions/3.10.0/bin/python -m pip install --upgrade pip
Requirement already satisfied: pip in /Users/pi/.pyenv/versions/3.10.0/lib/python3.10/site-packages (22.2.1)
Collecting pip
Using cached pip-22.2.2-py3-none-any.whl (2.0 MB)
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 22.2.1
Uninstalling pip-22.2.1:
Successfully uninstalled pip-22.2.1
Successfully installed pip-22.2.2
What is actually happening when I execute the above command? I was expecting it to update the pip for the python version created/maintained by pyenv. Which it seems to be doing:
🧢 pi#pPro18-4 ~/.pyenv/versions/3.10.0
> find . -name 'pip*'
./bin/pip3
./bin/pip
./bin/pip3.10
./lib/python3.10/site-packages/pip
./lib/python3.10/site-packages/pip-22.2.2.dist-info
🧢 pi#pPro18-4 ~/.pyenv/versions/3.10.0
> ./bin/pip --version
pip 22.2.2 from /Users/pi/.pyenv/versions/3.10.0/lib/python3.10/site-packages/pip (python 3.10)
So why isn't this pip getting copied into my .venv when I create it?
I thought that was the way .venv creation worked.
How to clean up my pyenv Python installation so that it spawns up-to-date .venvs?
EDIT:
Insight from #python on IRC/Libera:
grym: I don't think you can; i just get in the habit of python -m venv somevenv && somevenv/bin/python -m pip install --upgrade pip setuptools wheel
jinsun: python -m venv --upgrade-deps .venv is a simple solution if you were just annoying by the pip warning (...) it is updating the pip inside the venv, forget about the base python, I don't even have pip in the base python
This is the use case for pyenv-hooks
pyenv-hooks are scripts that are executed by pyenv whenever certain commands are run. You can create hooks for regular commands like: exec, rehash, which, but it can also be a plugin command, like virtualenv. The scripts can be written in any language.
Here is the wiki with official instructions.
You can have a hook by creating a script at the following location:
$PYENV_ROOT/pyenv.d/<hook-name>/<your-script-name>
For example, to create a hook that upgrades pip, create a new script within this path:
$PYENV_ROOT/pyenv.d/virtualenv/after.bash
With contents:
after_virtualenv 'PYENV_VERSION="$VIRTUALENV_NAME" pyenv-exec pip install --upgrade pip'
after_virtualenv is the command that tells pyenv when to execute. First, it sets the pyenv version to the name of the virtualenv we just created. with the variable $VIRTUALENV_NAME. Then it upgrades pip itself.
More details in this article.
I originally posted it as a comment, but was suggested to make it a proper answer.
An easier approach is to use the upgrade-deps flag when you create a virtual environment. Like this:
python3 -m venv --upgrade-deps .venv
It was added on python3.9, and according to the official docs:
--upgrade-deps
Upgrade core dependencies (pip, setuptools) to the latest version in PyPI
So, in other words, it will install pip and upgrade right away.
I use a different virtual environment for every project I work on (using pyenv and pyenv-virtualenv), and I often come across a message like then when installing a python package using pip:
WARNING: You are using pip version 21.1.3; however, version 21.2.4 is available.
You should consider upgrading via the '/Users/.../.pyenv/versions/3.9.6/envs/sdge/bin/python -m pip install --upgrade pip' command.
However, when I run the pip install --upgrade pip command, it only upgrades pip in that particular virtual environment (which is expected, and that is behavior that I want), and it is annoying to see this across all my different virtual environments.
Is there some kind of shortcut (either with pyenv, pip, or the shell) that I could use to run pip install --upgrade pip on every virtual environment?
I don't use pyenv so I created this code after reading documentation and source code (not tested):
for env in $(pyenv virtualenvs --bare --skip-aliases); do
pyenv activate $env
pip install --upgrade pip
pyenv deactivate
done
Can anybody recommend how to do the following smartest:
Make a new environment called "wells"
in addition to anaconda packages like matplotlib, numphy, pandas ++,
install packages like
"pip install lasio", "pip install dlisio", ..
Tried cloning anaconda, but after 30+ minutes I stopped it.
In terms of speed, I recommend beginning with Miniconda3 and immediately after installing, running these commands to ensure that all packages are installed from conda-forge wherever possible:
conda config --add channels conda-forge
conda config --set channel_priority strict
To create the environment, I recommend using an environment file. Create a text file named environment.yml, where you can specify packages which are available via conda on e.g. conda-forge (numpy, matplotlib, pandas in this case) as well as a separate list for packages available via pip on e.g. PyPI: lasio, dlisio in this case.
Example environment.yml file:
name: examplename
channels:
- conda-forge
dependencies:
- python
- numpy
- matplotlib
- pandas
- pip:
- lasio
- dlisio
Then you can create the environment wells using this command:
conda env create -n wells -f environment.yml
Here is one approach, it uses Python3.x. I ran this to verify it would be faster than the 30+ minutes you experienced. It took less than 4 minutes. So hopefully this will work for you.
# Create a work directory
mkdir wells
cd wells
# Create a virual environement named 'wells'
python3 -m venv wells
# Start the virual environment
source wells/bin/activate
# Update the basic virtual environment tools
pip install -U pip setuptools
# Create a requirements.txt with all the packages to install
echo matplotlib >>requirements.txt
echo numpy >> requirements.txt
echo pandas >> requirements.txt
echo lasio >> requirements.txt
echo dlisio >> requirements.txt
# Install the packages
pip install -r requirements.txt
When done working with the virtual environment, deactivate it with:
deactivate
Additionally, if using a Conda enviroment instead of a Python -m venv virtual env refer to these links for steps:
https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html
https://www.anaconda.com/blog/using-pip-in-a-conda-environment
Also, it will be faster with fewer unneeded pkgs to use Miniconda instead of full Anaconda.
I am using conda(anaconda 2). Before installing anaconda, my Ubuntu16.04 already had pip installed. I recently installed virtualenv using pip install virtualenv. I did the following steps while trying to install coala in my virtual environment.
Create a new virtual environment using virtualenv venv
Activate environment using source activate path/to/activate
Clone the coala repository.
Run pip3 install -e . while inside the repository
After all this, I get the following error from coala:
There is a conflict in the version of a dependency you have installed and the requirements of coala. This may be resolved by creating a separate virtual environment for coala or running `pip install "yapf~=0.14.0"`. Be aware that the latter solution might break other python packages that depend on the currently installed version.
I am already in a virtualenv, so I tried installing yapf as per the command: pip install "yapf~=0.14.0". After that, when I check my pip list, it still showed yapf (0.15.2), To solve this, I did:
pip uninstall yapf
pip install "yapf~=0.14.0"
Now when I check my pip list, I get the desired results. BUT, the error is still there. It still shows the same dependency error. I am unable to understand what is happening? Which pip is installing what and where, what is conda doing behind the scenes, how is virtualenv behaving with conda, and the most important, why this error and how to resolve it?
first create a virtualenv pertaining to python3.x
template
virtualenv -p {location of python3 version } {name of virtualenv folder}
like this
virtualenv -p /usr/local/bin/python3 venv
then activate it
source venv/bin/activate
then run
pip install {library you want to use}
pip install "yapf~=0.14.0"
This should install yapf into the venv folder.
try that let us know.
Answering my own question, what I found out was that conda and virtualenv do not go hand in hand. condas has the ability to create it's own virtual environment, and if using condas, we must create a conda virtual environment(See this answer).
What I did was uninstalled conda and make a virtual environment using virtualenv. What I could have done as well is uninstall virtualenv and create condas environment and work in that.
How can I add a package to an existing conda environment?
If it is a python package I can use pip install <package>, but what if pip does not work?
Is it sufficient to activate the environment and use conda install <package>?
You've answered your own question. In fact you really want to do conda install ... instead of using pip if you can.
You can install a conda package also without activating the environment. Just use conda install -n <env_name> <package> or conda install -p <path/to/env> <package>.
If you want to install a specific package inside a specific conda environment, you can use the following command.
First activate the conda environment and then do:
$ conda install --name <conda_env_name> -c <channel_name> <package_name>
For a concrete example, let's assume that you want to install chainer from the channel anaconda to an already created conda environment named chainerenv, then you can do:
$ conda install --name chainerenv -c anaconda chainer
If you want to install a package in the environment, you can use
conda install -p /path/to/env package
example:
conda install -p /users/dekstop/env-test Django
There's an alternative way to do this and I have just tested it on my own mac:
example: i want to install a non-conda package at my python2.7 environment:
go to terminal
activate the desired environment by: source activate py27
after you successfully activated the environment, you can install the package you wanted by: pip install package
The answer is yes (usually).
One example is that you can activate your conda environment and then directly do conda install pandas.tar.bz2 on the existing tar.bz2 files from /conda_envs/.pkgs (leftovers from other environments)
If you don't have a tarball package like that but you have the src with setup.py you can just do the usual install by python setup.py install (or python setup.py develop to link the src)