I'm installing packages in several envs which depends on cudatoolkits in docker.
My docker image is pull from nvidia/cuda which already have cuda libs, but conda doesn't know, then conda will install cudatoolkits again.
I wonder how to let conda know that I have cuda libs in /usr/local/cuda.
My solution is to install all packages with --no-deps, but it is so complicated, I have to fetch all dependencies, then install them except cudatoolkit ONE BY ONE.
I wonder how to cheat conda, let conda know cudatoolkit has already installed.
Related
I'm a little confused with package install in anaconda environment.
I can install my python packagse in the following ways.
open anaconda prompt and do : conda install tensorflow
launch an jupyter notebook from anaconda prompt, choose the default python kernel, and do:
!pip install tensorflow
Can someone tell me what is the difference between these two cases? Where are the python packages installed? What happens when I pip install python packages in default kernel in jupyter notebook?
I was able to install tensorflow in jupyter notebook in default kernel (python3), but trying to import tensorflow give me module note found error. Does anyone know why that happened? What is happening under the hood? Very much appreciate any help to clarification.
pip is the default package manager that ships with python. Conda is also a package manager, but it is third party. Conda was made especially for data science libraries. Libraries installed with conda usually give much better performance than pip. In pip, the packages are stored in python/scripts and conda stores them at /anaconda/pkgs/. As for the module not found error. I would need more information about it, but you can check out this video. I learnt how to install TensorFlow here and would highly recommend it.
I installed Anaconda3 (version 2019.10-Windowsx86_64) on my PC last week. It comes with networkx v2.3, but I would like to upgrade to v2.4, which should be available on Anaconda according to conda search and Anaconda's website.
I first tried the Anaconda navigator, but can't get it to work. It indicates that networkx can be updated. However, when I click Apply the navigator spends a bit of time "solving package specifications" and then gives me the empty pop-up shown below.
I also tried using conda prompt. Using conda install networkx, conda update networkx, conda install -c anaconda networkx does not result in v2.4 being installed. I also tried conda install networkx=2.4, which takes forever and then reports a very long list of package conflicts. I find this confusing because I just installed Anaconda and haven't tampered with any configs yet.
Grateful for any help on understanding what is going on and how I can attempt to fix it!
I also tried conda install networkx=2.4, which takes forever and then reports a very long list of package conflicts.
This is the key here. anaconda comes with many packages pre-installed and some of these might depend on networkx==2.3 since they might not be compatible with the newest version.
What you can do is create a new environment that has the required version of networkx:
conda create -n myenv python=<your desired python version> networkx==2.4
which will create a new environment where you can only install the packages that you need and make sure that networkx==0.24 is satisfied.
You can try this in jupyter notebook in case you are working on jupyter notebook.
!pip install networkx==2.4
I had 2.5 v installed. But something wasn't working and had to degrade it to 2.4 and above method worked.
Try this:
pip install --upgrade networkx
I know there are a lot of questions on the coexistence and interchangeability/non-interchangeability of pip and conda. That is not my question: I know I need both for my work, I use both, and for the most part, my conda envs are a manageable mess.
But here's the thing: there's many ways to install pip. I happened to get conda going first, so my pip is through anaconda/bin/pip. It is the only pip on my machine. Here are my questions:
Is this sensible? Do I want my pip to be usr/bin/pip and be independent of global conda? It feels not-sensible.
If I install a new pip through say brew or easy_install, should I start downloading packages through this new pip? Would that be awful and mess everything up?
Thanks!
Pip always requires a version of Python to be installed, and is associated with that specific Python installation. By default, pip installs packages for its own Python, into the related site-packages directory inside the Python library directory. The exact location of this directory depends on your operating system and how you installed conda.
If you install pip via Homebrew or with another installation of Python, you should not use that pip and expect it to install for conda. For that matter, if you create a new conda environment, you should not expect that the pip in that environment will install packages into another environment.
There is the --user option to pip, which installs packages into a directory in your user account (on *nix systems, this is ~/.local; I can't recall for Windows where this is). These packages will be able to be found by all Python versions with the same major and minor version number. However, it is not recommended to install packages with the intent of sharing them among several Pythons this way, because if the different Pythons were compiled with different compilers, you may run into trouble.
Forgive me in case this too easy a question.
Background:
I installed Python 3 and Anaconda as I want to start programming Python and want to dive into machine learning.
Unfortunately all this shell based installation thing is absolutely new to me.
I managed to install python 3.6. and Ananconda 3.
Problem(s):
Unfortunately now I have several Python instances installed. One with Anaconda under user/anaconda and some under library/Python/
Now I need for a course to install PyML and PyBrain. I tried to it the "normal way". i.e. what is written on the webpage and as so often I get multiple errors during the installation.
Goal:
I'd like to have it under the Anaconda packages. Is there a way to do that?
If no: how can I get my other Python 3.6. installation to know the location of the Anaconda packages?
edit 12.08.17
The content of $PATH:
/Users/ak/anaconda/bin:/Users/ak/miniconda2/bin:/Users/ak/miniconda3/bin:/opt/local/bin:/opt/local/sbin:/Library/Frameworks/Python.framework/Versions/2.7/bin:/Library/Frameworks/Python.framework/Versions/3.6/bin:/Library/Frameworks/Python.framework/Versions/3.5/bin:/Users/ak/.rbenv/shims:/Library/Frameworks/EPD64.framework/Versions/Current/bin:/Library/Frameworks/Python.framework/Versions/Current/bin:/Library/Frameworks/Python.framework/Versions/2.7/bin:/Users/ak/Library/Application Support/GoodSync:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/MacGPG2/bin:/Library/TeX/texbin:/Users/ak/.rvm/bin
edit 12.08.17 - 2:
When I try to "conda" this libraries this happens though I started it from the unzipped, downloaded PyML:
AKs-MacBook-Pro:PyML-0.7.14 ak$ conda install -c manmadescience pyml
Fetching package metadata ...........
PackageNotFoundError: Packages missing in current channels:
pyml
And when I do this: conda install -c manmadescience pyml (found here https://anaconda.org/manmadescience/pyml)
I get the same result.
I already have Python 2.7 installed but I wanted to try IPython so I installed IPython via Anaconda as recommended on the ipython website (although not sure what the pros/cons of doing this are). Now I would like to use ipdb debugger. I guess I need to make sure it installs underneath the Anaconda version of python rather than the normal python.
How do I install this? In general if I want to install some arbitrary python module under Anaconda how do I do this?
Actually I think in the case of ipdb it's already installed with Anaconda. But in general it appears you can just install stuff via either pip or easy_install as necessary. The key that I was missing is to make sure you are using the pip/easy_install that comes with Anaconda (which are .bat files in the Scripts directory) rather than the system Python's pip/easy_install. So:
Anaconda\Scripts\easy_install somepackage
This will install somepackage in Anaconda\lib\site-packages\ and not in the system python. This appears to work and I can now import somepackage from my anaconda python. This seems to work. It wasn't clear to me from reading Anaconda documentation if everything needed to be in a conda package or not.
This answer seems to support this idea: Installing Anaconda into a Virtual Environment
Generally the first thing to check is whether someone else has already built it for your version of python and uploaded it to anaconda.org:
anaconda search -t conda ipdb
then find a repository with ipdb built for your OS, and try
conda install -c <repository> ipdb
e.g. conda install -c conda-forge ipdb
You might need to try a few different ones to find one built for your version of python. There is a feature request to make this easier.
If that doesn't work, then pip install ipdb will