I'm a little confused with package install in anaconda environment.
I can install my python packagse in the following ways.
open anaconda prompt and do : conda install tensorflow
launch an jupyter notebook from anaconda prompt, choose the default python kernel, and do:
!pip install tensorflow
Can someone tell me what is the difference between these two cases? Where are the python packages installed? What happens when I pip install python packages in default kernel in jupyter notebook?
I was able to install tensorflow in jupyter notebook in default kernel (python3), but trying to import tensorflow give me module note found error. Does anyone know why that happened? What is happening under the hood? Very much appreciate any help to clarification.
pip is the default package manager that ships with python. Conda is also a package manager, but it is third party. Conda was made especially for data science libraries. Libraries installed with conda usually give much better performance than pip. In pip, the packages are stored in python/scripts and conda stores them at /anaconda/pkgs/. As for the module not found error. I would need more information about it, but you can check out this video. I learnt how to install TensorFlow here and would highly recommend it.
Related
I installed Anaconda3 (version 2019.10-Windowsx86_64) on my PC last week. It comes with networkx v2.3, but I would like to upgrade to v2.4, which should be available on Anaconda according to conda search and Anaconda's website.
I first tried the Anaconda navigator, but can't get it to work. It indicates that networkx can be updated. However, when I click Apply the navigator spends a bit of time "solving package specifications" and then gives me the empty pop-up shown below.
I also tried using conda prompt. Using conda install networkx, conda update networkx, conda install -c anaconda networkx does not result in v2.4 being installed. I also tried conda install networkx=2.4, which takes forever and then reports a very long list of package conflicts. I find this confusing because I just installed Anaconda and haven't tampered with any configs yet.
Grateful for any help on understanding what is going on and how I can attempt to fix it!
I also tried conda install networkx=2.4, which takes forever and then reports a very long list of package conflicts.
This is the key here. anaconda comes with many packages pre-installed and some of these might depend on networkx==2.3 since they might not be compatible with the newest version.
What you can do is create a new environment that has the required version of networkx:
conda create -n myenv python=<your desired python version> networkx==2.4
which will create a new environment where you can only install the packages that you need and make sure that networkx==0.24 is satisfied.
You can try this in jupyter notebook in case you are working on jupyter notebook.
!pip install networkx==2.4
I had 2.5 v installed. But something wasn't working and had to degrade it to 2.4 and above method worked.
Try this:
pip install --upgrade networkx
The Jupyter notebook worked initially, but I tried importing tensorflow and that would not work, so that led to me messing up everything.
I basically messed everything up, and I feel like the only way out now is to just nuke my device and restart. I had no idea what pip and anaconda are (still don't really), tried a bunch of funky updates and installations and whatever and now everything is just dead. My jupyter notebook cannot even run the normal python kernel.
How can I hard reset everything?
As a bonus, if someone were to ELI5 the difference between conda, pip, gitbash, and PowerShell are. And what versions of stuff does Jupyter run on (since my conda and device had different versions of things I think?). I use Windows 10.
My first piece of advice is to not use Windows, though I'll probably get downvote spam for that. On Ubuntu, I could stuff Jupyter setup into one line:
# update, install python3, python3-dev, and pip3; get pip packages
sudo apt-get update && sudo apt-get install -y python3 python3-dev python3-pip && sudo -H python3 -m pip install jupyter notebook ipykernel tensorflow
Once the packages are installed, it's as easy as running jupyter notebook in the terminal.
Anaconda is a distribution of Python that includes a ton of pre-built packages, including Jupyter and scipy, numpy, pandas, etc. It's an "out of the box" solution basically, that comes with most of the tools you need. "Pip" is a package manager for Python; pip install [package] lets you use a package in your script, like import [package]. In this case, that's tensorflow.
ipykernel is a package that will open up a Python kernel for Jupyter. You could run a Jupyter notebook on a Python3.7 backend but do stuff with Python2 code by installing ipykernel with Python2's pip, usually (on Ubuntu) sudo apt-get update && sudo apt-get install -y python-pip && sudo -H python -m pip install ipykernel.
What happens when you run jupter notebook? Do you get errors? Can you get the notebook to open, but there's just no kernel to attach to a notebook?
I have just set up a new Windows 10 machine for Python, Jupyter, and Tensorflow. I did the set-up without anaconda. I did the normal set-up procedure with some special steps:
1) Python 3.8 und Jupyter as installed by "pip install" does not work. You need to add three lines of code in a module that is installed as dependency when you install Jupyter. change asyncio.py
2) Current Tensorflow does not work with Python 3.8. You need to install Python 3.7. You don't need to delete your Python 3.8 if you have one. Create a virtual environment with virtualenv as described here and give the Path to your Python 3.7 Special Python in virtualenv
3) If you want to use GPU for NVIDIA in Tensorflow, you need to deal with the fact that two things do not fit together: current Tensorflow and the current version of ‘NVIDIA GPU Computing Toolkit’ (a tools you need for GPU support). Take a look here for the fix: cudart64_XYZ.dll not found
Let's start with the basics:
As a bonus, if someone were to ELI5 the difference between conda, pip, gitbash, and powershell are
You probably know the classical cmd.exe which opens a basic terminal where you can use different commands and call programs from. It is basically a text based way to interact with your operating system.
Powershell is in my understanding just an extension of this (I don't use it myself) and has more capabilities of what you can do and also better scripting support.
gitbash is an optional tool that you probably installed when you installed git on your computer. It emulates a bash shell that many people are used to from different operating system like ubuntu where bash is often the default terminal and therefore makes it easier to use, as all the syntax and commands are then the same as these ppl are used.
Neither of these is in any way directly related to using python on your computer other than being able to type python or jupyter notebook into these terminals to start the applications.
To the more python specific questions:
conda is a package and virtual environment management tool. It can be used to install a variety of software and also create virtual environments to keep different set ups seperate from one another (e.g. different python versions on the same machine). But it is not limited to python. It is pre-installed when you download and install miniconda or anaconda which are two python distributions.
pip is a package manager only for python packages and comes pre-installed with most python distributions.
anaconda/miniconda , often times confused with conda are two python distributions, i.e. what you would consider as "I installed python on my system" that come with the conda package manager pre-installed. miniconda does thereby not ship any other packages while anaconda comes with a long list of useful packages pre-installed and is therefore a popular choice when you want an easy acces into using python for your research
For more info, you can also read understanding-conda-and-pip
How can you save your system now
I basically fucked everything up
Difficult to access the current state of your system, but I would suggest you try the following steps to get to a working condition again:
Go into Setting -> Apps and remove everything that is related to python or anaconda. Make sure that everything is deleted by also searching (using windows search feature) for python or conda folders somewhere in C:\Users. This should make sure that everything about your setup is purged
Make sure that neither python, pip or jupyter commands are working anymore in your cmd (confirming the purge)
Download and install miniconda
Now Create a virtual environment and install tf. This is a good way to go because if you should manage to f*k up the environment, you can just delete and recreate it without much trouble:
conda create -n venv pip python=3.7 #create environment
conda activate venv #activate the environment
conda install jupyter #for jupyter notebook
pip install https://storage.googleapis.com/tensorflow/windows/gpu/tensorflow_gpu-2.1.0-cp37-cp37m-win_amd64.whl
Start jupyter notebook: jupyter notebook. Since it only exists in this environment, same as tensorflow, there should be no more issues to use tensorflow normally
after following the instruction in http://docs.h2o.ai/h2o/latest-stable/h2o-docs/downloading.html#install-in-python, I was able to install h2o v 3.16.0.2. I was also able to use command line instructions mentioned in the above website and test that it is working.
python
import h2o
h2o.init()
h2o.demo("glm")
However, when I launch anaconda spyder, I am not able to import h2o. How do I link the h2o I have installed and bring it into Spyder python?
Update:
I have already tried {conda install -c anaconda h2o } which is mentioned on Anaconda.org, but that installs older ver 3.10 of h2o and that did not work either.
thanks for your help.
I believe the problem is due to your python environment. When you install Anaconda, you need to use anaconda pip, to ensure that the installed packages are available in conda.
The short answer is you can install the lastest stable version of H2O (3.16.0.2 as of today) using conda via h2oai channel instead of anaconda channel (h2o maintains it's own channel):
conda install -c h2oai h2o
this should solve your issue.
But more generally, the packages will appear in conda if you use anaconda pip. You can check which pip is being used by doing
which pip
and making sure that the path to the pip is within your anaconda distribution; something like /home/<userdir>/anaconda/bin/pip instead of /usr/bin/pip
Same is true also for Python. Try checking if starting Python in terminal points to anaconda Python by doing which python. If that's not the case, than you would need to add the conda installation of Python to your PATH variable. Please refer to conda docs for instructions https://conda.io/docs/user-guide/install/index.html
It would have been helpful if you had included information regarding your operating system in the question.
I just installed PyTorch after installing Anaconda, and when I run iPython using Anaconda Python, it won't find PyTorch.
However I can verify PyTorch is in the pkgs directory of my anaconda folder.
What's going on?
While the framework name is Pytorch, it must be imported using the torch namespace like so import torch. If you look at the official documentation here, you will see any pytorch related module import like nn or autograd is done through the torch namespace. I admit it is a bit confusing but I guess it was done to instill a sense of familiarity for old torch7 users. Also, there is a very good introduction tutorial which you could probably use to familiarize yourself better with the ins and outs of pytorch found here. Hope this helps!
The Py Torch package might have installed to a different python environment that you might have on your system.
You can see the entire list of python packages installed in your current environment using
pip freeze
Ensure that the pip command being executed belongs to Anaconda distribution. The following command will provide the path of the pip executable.
which pip
I already have Python 2.7 installed but I wanted to try IPython so I installed IPython via Anaconda as recommended on the ipython website (although not sure what the pros/cons of doing this are). Now I would like to use ipdb debugger. I guess I need to make sure it installs underneath the Anaconda version of python rather than the normal python.
How do I install this? In general if I want to install some arbitrary python module under Anaconda how do I do this?
Actually I think in the case of ipdb it's already installed with Anaconda. But in general it appears you can just install stuff via either pip or easy_install as necessary. The key that I was missing is to make sure you are using the pip/easy_install that comes with Anaconda (which are .bat files in the Scripts directory) rather than the system Python's pip/easy_install. So:
Anaconda\Scripts\easy_install somepackage
This will install somepackage in Anaconda\lib\site-packages\ and not in the system python. This appears to work and I can now import somepackage from my anaconda python. This seems to work. It wasn't clear to me from reading Anaconda documentation if everything needed to be in a conda package or not.
This answer seems to support this idea: Installing Anaconda into a Virtual Environment
Generally the first thing to check is whether someone else has already built it for your version of python and uploaded it to anaconda.org:
anaconda search -t conda ipdb
then find a repository with ipdb built for your OS, and try
conda install -c <repository> ipdb
e.g. conda install -c conda-forge ipdb
You might need to try a few different ones to find one built for your version of python. There is a feature request to make this easier.
If that doesn't work, then pip install ipdb will