To install to my own directory I can use
pip install --user package
Alternatively I can use
conda install package
How do I ask conda to install to my home directory since conda does not take a --user flag?
Installing pip packages to $HOME folder
I don't think it's possible. Use virtual environments (conda create).
See -p option here:
-p PATH, --prefix PATH
Full path to environment prefix.
So to install to, say, local under your home directory, do:
conda install -p $HOME/local packagename
Note, however, this is not a "clean" install as it adds a bunch of conda-related files.
To install conda packages on your own directory you can follow these steps:
Create a blank environment
conda create -y -n my-conda-env
Replace the name my-conda-env with any name you want to give the environment.
Activate the environment
source activate my-conda-env
Don't forget to replace my-conda-env with the name you gave the conda environment from the previous step
Install your package
conda install -c bioconda epa-ng
And that's it, the package should be installed on your own directory
Simply:
sudo conda install -c conda-forge package
Or:
sudo chmod -R 777 ./
conda install -c conda-forge package
I don't know of an exact match for the --user flag, but a reasonable analogue is to use a virtual environment.
What I do when I have to install to a shared CentOS server where I don't have admin access:
First I run
conda env list
will list all conda virtual environments and display the path to each. Once you have the environment created and can see it in the conda env list, copy the path to the environment.
If you need to create one, you can do that with conda create or by running anaconda-navigator and using the GUI.
Activate your environment (if not active) with
conda activate [environment_name]
or
activate [environment_name]
depending on your system (most linux systems use the first, Windows and CentOS use the latter).
Now you can use
conda install -p [environment_path] [package_name]
and you are off to the races.
This is really a work around; it's not the best but it does install the package to the selected virtual environment.
The current Anaconda Install Individual Edition, when run in a linux local account, installs in a local directory. So all the subsequent installs should install there, too.
According to the documentation:
--use-local
Related
I tried to create virtual environment using pyenv virtualenv 3.8.2 myenv, but it failed, i don't know why, i even changed the python version or the virtual environment name, but it still not working.
Some detail:
I tried this in macos and zsh.
➜ ~ pyenv virtualenv 3.8.2 myenv
Looking in links: /var/folders/_9/l8m14fgs6ts9wx0nl1qbzbkm0000gn/T/tmpe9l25o2_
Requirement already satisfied: setuptools in /Users/fitz/.pyenv/versions/3.8.2/envs/myenv/lib/python3.8/site-packages (41.2.0)
Requirement already satisfied: pip in /Users/fitz/.pyenv/versions/3.8.2/envs/myenv/lib/python3.8/site-packages (19.2.3)
rm: /Users/fitz/.pyenv/shims/shims: is a directory
➜ ~ pyenv versions
system
3.7.3
* 3.8.2 (set by /Users/fitz/.python-version)
3.9.5
I had the same issue. For me it was caused by two empty folders in my shims directory. I deleted the folders and it fixed the issue. So, just running the following would fix your issue.
rm -rf ~/.pyenv/shims/shims
rm -rf ~/.pyenv/shims/versions
Try using 'venv' instead of pyenv or pyvenv or virtualenv. Venv is a library that already comes with your python installation. Virtualenv is an external one.pyenv is similar to venv in that it lets you manage multiple python environments. However with pyenv you can't conveniently rollback library installs to some start state and you will likely need admin privileges at some point to update libraries. So I think it is also best to use venv.
First, make a directory :
mkdir testing
Then, moved to this directory named testing :
cd testing
When you type following command in this directory:
python3 -m venv env
You got error like :
The virtual environment was not created successfully because ensurepip is not
available. On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.
apt install python3.8-venv
Type the following command but before that keep an eye on the version of python you installed on the machine; in my case its python3.8
sudo apt install python3.8-venv
Now, we can create a virtual environment and store its tools in the "bhandari" folder .
python3 -m venv bhandari
Note: you can named this "bhandari" folder; anyname you like( Standard practice is to name it "env" ...)
Now to activate your virtual environment, from the directory of your folder, type the following command this will activate our virtual environment in the “bhandari” folder
source bhandari/bin/activate
If you have successfully activated your virtual environment, you should see the (bhandari) word indicating that we are working in a virtual environment.
After this, we can install anything that will be isolated from the rest of the system....
In pip there is an -f option which does the following:
-f, --find-links : If a url or path to an html file, then parse for links to archives. If a local path or file:// url that's a directory, then look for archives in the directory listing.
This is the preferred way of installing PyTorch, by setting the link to their overview website, e.g.:
pip3 install torch===1.3.0 -f https://download.pytorch.org/whl/torch_stable.html
For my virtual environments I use pipenv but I haven't found an option that does the same as -f. In the meantime, I can just look up the direct link to the package that is relevant for my system, but that is cumbersome.
Does pipenv provide a way to do the same thing as pip's -f?
In the new version of pipenv (I tested with version 2020.11.15), you can install packages in this way:
pipenv install https://download.pytorch.org/whl/cpu/torch-1.3.0%2Bcpu-cp36-cp36m-linux_x86_64.whl
The link can be found in the this page: https://download.pytorch.org/whl/torch_stable.html
This will be added to the Pipfile as well.
[packages]
torch = {file = "https://download.pytorch.org/whl/cpu/torch-1.3.0%2Bcpu-cp36-cp36m-linux_x86_64.whl"}
you need to manually check the link with your compute platform, os and python version.
It is possible to use environment variables recognised by pip to tweak its behaviour within pipenv's execution. E.g.:
PIP_FIND_LINKS=https://download.pytorch.org/whl/torch_stable.html pipenv install torch==1.5.1+cu101
See:
Advanced Usage of Pipenv - Configuration With Environment Variables
pip's User Guide - Environment Variables
For now, I haven't found a work-around. What you can do, of course, is enabling the pipenv shell and doing what you must with pip, e.g.
pipenv shell
python -m pip install torch===1.3.0 -f https://download.pytorch.org/whl/torch_stable.html
This will install torch in the pipenv environment but torch will not be added to the Pipfile (nor to the lock file).
I have installed caffe and pytorch0.3 in my system path(environment), and there is a project which only works under caffe and history version of pytorch0.2. To solve that I install pytorch0.2 in a Conda environment and I wonder if there is any way can save me from installing a caffe again in this conda environment.
In other words, can I use pytorch0.2 in this Conda environment meanwhile import the system caffe? and how?
Activate your environment and try the following
conda install package-name --offline
Also, in case you wish to clone the root or some other environment to a conda environment, you can use -- clone. For instance, when you wish to clone the root -
conda create -n pytorch_02 --clone root
I create a conda environment without specifying any packages using the following command:
conda create --name test_env
I can then use all the packages in the root environment inside test_env (but they do not appear in the outputs of conda list and conda env export). This is already unexpected to me but the real problems begin when I then install something inside that environment, e.g.:
conda install pywavelets
Afterwards, pywavelets is usable but all the other packages which are no dependencies of pywavelets disappear inside the environment (e.g. pandas). I don't understand why that happens. Does anybody have an explanation for that?
More importantly, what does this mean for best practices for working with conda environments? Should I always create my environments specifying at least python (conda create --name test_env python)? However, then I have to install everything by hand in that environment which is quite cumbersome. So, my idea now is to specify anaconda for all environments I create:
conda create --name test_env anaconda
The disadvantage, however, is that the list of dependencies listed by conda list and conda env export gets unnecessarily long (e.g. even listing the Anaconda Navigator). Does anybody have a better solution for this?
The reason you can use all the packages from the root environment when you don't specify a Python version during environment creation is because you're actually using the root environment's Python executable! You can check with which python or python -c "import sys; print(sys.executable)". See also my other answer here.
When you install pywavelets, one of the dependencies is (probably) Python, so a new Python executable is installed into your environment. Therefore, when you run Python, it only picks up the packages that are installed in the test_env.
If you want all of the packages from another environment, you can create a file that lists all the packages and then use that file to create a new environment, as detailed in the Conda docs: https://conda.io/docs/user-guide/tasks/manage-environments.html#building-identical-conda-environments
To summarize
conda list --explicit > spec-file.txt
conda create --name myenv --file spec-file.txt
or to install into an existing environment
conda install --name myenv --file spec-file.txt
Since that's just a text file, you can edit and remove any packages that you don't want.
How can I install a package under development to an Anaconda environment?
With pip:
pip install -e /path/to/mypackage
or with regular setuptools:
python /path/to/mypackage/setup.py develop
There is also conda develop available now.
http://conda.pydata.org/docs/commands/build/conda-develop.html
Update in 2019: conda develop hasn't been maintained and is not recommended. See https://github.com/conda/conda-build/issues/1992
Recommendation is to use python setup.py develop or pip install -e .
Using either of those will work with Anaconda. Make sure that you have pip or setuptools installed into the conda environment you want to install into, and that you have it activated.
This is the equivalent to pip install -e .
conda install conda-build
conda develop .
As explained in this gh issue thread, because of build isolation and dependency installation, Anaconda developers recommend using:
pip install --no-build-isolation --no-deps -e .
Build / Host Environment
To create build and host environments and a build script go to your recipe directory and use
conda debug /path/to/your/recipe-directory
as documented here. This will print an instructive message like
################################################################################
Build and/or host environments created for debugging. To enter a debugging environment:
cd /home/UserName/miniconda3/conda-bld/debug_1542385789430/work && source /home/UserName/miniconda3/conda-bld/debug_1542385789430/work/build_env_setup.sh
To run your build, you might want to start with running the conda_build.sh file.
################################################################################
(The message might tell you incorrectly, that it created a test environment.) Your source code has been copied to the .../work directory and there is also a conda_build.sh script. Note, that sourcing the build_env_setup.sh will load both build and host environments.
You can work on your code and your recipe and build with the conda_build.sh, but you won't get a proper conda package, as far as I know. When you are finished, you can remove the debug environment:
conda deactivate # maybe twice
conda build purge
Test Environment
To get the test environment, you have to build the package first and then debug that. This might be useful to fix your test files.
conda build /path/to/your/recipe-directory # creates mypackage*.tar.bz2
# find file location of mypackage*.tar.bz2 with:
conda search --info --use-local mypackage # look at the url row for the path
cd /path/to/miniconda3/conda-bld/linux-64/ # go to that path, can be different
conda debug mypackage*.tar.bz2
This will print e. g.:
################################################################################
Test environment created for debugging. To enter a debugging environment:
cd /home/UserName/miniconda3/conda-bld/debug_1542385789430/test_tmp && source /home/UserName/miniconda3/conda-bld/debug_1542385789430/work/conda_test_env_vars.sh
To run your tests, you might want to start with running the conda_test_runner.sh file.
################################################################################
Again, remove with
conda deactivate
conda build purge
Run Environment
This is actually no debugging, but the general process of building and installing a local package. With the run environment you can check, whether all dependencies are specified in the requirements/run section. Also pinning can be an issue.
(base) $ conda build /path/to/your/recipe-directory
(base) $ conda create --name package-env --use-local mypackage
(base) $ conda activate package-env
(package-env) $ python
>>> import mypackage
You can also list the dependencies of your package with (man page)
conda search --info --use-local mypackage
A last hint: If you want to know the versions of the dependencies and see, whether pinning works, try (man page)
conda render /path/to/your/recipe-directory