I'm a little confused with package install in anaconda environment.
I can install my python packagse in the following ways.
open anaconda prompt and do : conda install tensorflow
launch an jupyter notebook from anaconda prompt, choose the default python kernel, and do:
!pip install tensorflow
Can someone tell me what is the difference between these two cases? Where are the python packages installed? What happens when I pip install python packages in default kernel in jupyter notebook?
I was able to install tensorflow in jupyter notebook in default kernel (python3), but trying to import tensorflow give me module note found error. Does anyone know why that happened? What is happening under the hood? Very much appreciate any help to clarification.
pip is the default package manager that ships with python. Conda is also a package manager, but it is third party. Conda was made especially for data science libraries. Libraries installed with conda usually give much better performance than pip. In pip, the packages are stored in python/scripts and conda stores them at /anaconda/pkgs/. As for the module not found error. I would need more information about it, but you can check out this video. I learnt how to install TensorFlow here and would highly recommend it.
The Jupyter notebook worked initially, but I tried importing tensorflow and that would not work, so that led to me messing up everything.
I basically messed everything up, and I feel like the only way out now is to just nuke my device and restart. I had no idea what pip and anaconda are (still don't really), tried a bunch of funky updates and installations and whatever and now everything is just dead. My jupyter notebook cannot even run the normal python kernel.
How can I hard reset everything?
As a bonus, if someone were to ELI5 the difference between conda, pip, gitbash, and PowerShell are. And what versions of stuff does Jupyter run on (since my conda and device had different versions of things I think?). I use Windows 10.
My first piece of advice is to not use Windows, though I'll probably get downvote spam for that. On Ubuntu, I could stuff Jupyter setup into one line:
# update, install python3, python3-dev, and pip3; get pip packages
sudo apt-get update && sudo apt-get install -y python3 python3-dev python3-pip && sudo -H python3 -m pip install jupyter notebook ipykernel tensorflow
Once the packages are installed, it's as easy as running jupyter notebook in the terminal.
Anaconda is a distribution of Python that includes a ton of pre-built packages, including Jupyter and scipy, numpy, pandas, etc. It's an "out of the box" solution basically, that comes with most of the tools you need. "Pip" is a package manager for Python; pip install [package] lets you use a package in your script, like import [package]. In this case, that's tensorflow.
ipykernel is a package that will open up a Python kernel for Jupyter. You could run a Jupyter notebook on a Python3.7 backend but do stuff with Python2 code by installing ipykernel with Python2's pip, usually (on Ubuntu) sudo apt-get update && sudo apt-get install -y python-pip && sudo -H python -m pip install ipykernel.
What happens when you run jupter notebook? Do you get errors? Can you get the notebook to open, but there's just no kernel to attach to a notebook?
I have just set up a new Windows 10 machine for Python, Jupyter, and Tensorflow. I did the set-up without anaconda. I did the normal set-up procedure with some special steps:
1) Python 3.8 und Jupyter as installed by "pip install" does not work. You need to add three lines of code in a module that is installed as dependency when you install Jupyter. change asyncio.py
2) Current Tensorflow does not work with Python 3.8. You need to install Python 3.7. You don't need to delete your Python 3.8 if you have one. Create a virtual environment with virtualenv as described here and give the Path to your Python 3.7 Special Python in virtualenv
3) If you want to use GPU for NVIDIA in Tensorflow, you need to deal with the fact that two things do not fit together: current Tensorflow and the current version of ‘NVIDIA GPU Computing Toolkit’ (a tools you need for GPU support). Take a look here for the fix: cudart64_XYZ.dll not found
Let's start with the basics:
As a bonus, if someone were to ELI5 the difference between conda, pip, gitbash, and powershell are
You probably know the classical cmd.exe which opens a basic terminal where you can use different commands and call programs from. It is basically a text based way to interact with your operating system.
Powershell is in my understanding just an extension of this (I don't use it myself) and has more capabilities of what you can do and also better scripting support.
gitbash is an optional tool that you probably installed when you installed git on your computer. It emulates a bash shell that many people are used to from different operating system like ubuntu where bash is often the default terminal and therefore makes it easier to use, as all the syntax and commands are then the same as these ppl are used.
Neither of these is in any way directly related to using python on your computer other than being able to type python or jupyter notebook into these terminals to start the applications.
To the more python specific questions:
conda is a package and virtual environment management tool. It can be used to install a variety of software and also create virtual environments to keep different set ups seperate from one another (e.g. different python versions on the same machine). But it is not limited to python. It is pre-installed when you download and install miniconda or anaconda which are two python distributions.
pip is a package manager only for python packages and comes pre-installed with most python distributions.
anaconda/miniconda , often times confused with conda are two python distributions, i.e. what you would consider as "I installed python on my system" that come with the conda package manager pre-installed. miniconda does thereby not ship any other packages while anaconda comes with a long list of useful packages pre-installed and is therefore a popular choice when you want an easy acces into using python for your research
For more info, you can also read understanding-conda-and-pip
How can you save your system now
I basically fucked everything up
Difficult to access the current state of your system, but I would suggest you try the following steps to get to a working condition again:
Go into Setting -> Apps and remove everything that is related to python or anaconda. Make sure that everything is deleted by also searching (using windows search feature) for python or conda folders somewhere in C:\Users. This should make sure that everything about your setup is purged
Make sure that neither python, pip or jupyter commands are working anymore in your cmd (confirming the purge)
Download and install miniconda
Now Create a virtual environment and install tf. This is a good way to go because if you should manage to f*k up the environment, you can just delete and recreate it without much trouble:
conda create -n venv pip python=3.7 #create environment
conda activate venv #activate the environment
conda install jupyter #for jupyter notebook
pip install https://storage.googleapis.com/tensorflow/windows/gpu/tensorflow_gpu-2.1.0-cp37-cp37m-win_amd64.whl
Start jupyter notebook: jupyter notebook. Since it only exists in this environment, same as tensorflow, there should be no more issues to use tensorflow normally
I downloaded and installed Anaconda3-2019.03-MacOSX-x86_64.pkg which should normally install python 3.7. But oddly this is not the case :
(base) iMac-de-Bibi:~ ac$ which python
/Users/ac/anaconda3/bin/python
(base) iMac-de-Bibi:~ ac$ /Users/ac/anaconda3/bin/python --version
Python 2.7.16 :: Anaconda, Inc
Any help would be appreciated
Note : it's my first anaconda installation on this machine
You might have mistakenly downloaded and installed the wrong Anaconda distribution.
You might have changed the Python version in the base environment after the initial installation, by doing something like this:
conda activate base
conda install python=2.7
Update: As reported in a comment, installing pyside downgrades Python.
Have you correctly activated the base environment, and not messed with PATH or PYTHONPATH? I'm not sure about it, but python might be a script that calls the actual Python interpreter. It could pick the wrong one if search paths are off.
In general, I recommend to install Miniconda and create custom environments. Then you can choose whichever Python version you want for each environment, and don't care which one is in the base environment.
after following the instruction in http://docs.h2o.ai/h2o/latest-stable/h2o-docs/downloading.html#install-in-python, I was able to install h2o v 3.16.0.2. I was also able to use command line instructions mentioned in the above website and test that it is working.
python
import h2o
h2o.init()
h2o.demo("glm")
However, when I launch anaconda spyder, I am not able to import h2o. How do I link the h2o I have installed and bring it into Spyder python?
Update:
I have already tried {conda install -c anaconda h2o } which is mentioned on Anaconda.org, but that installs older ver 3.10 of h2o and that did not work either.
thanks for your help.
I believe the problem is due to your python environment. When you install Anaconda, you need to use anaconda pip, to ensure that the installed packages are available in conda.
The short answer is you can install the lastest stable version of H2O (3.16.0.2 as of today) using conda via h2oai channel instead of anaconda channel (h2o maintains it's own channel):
conda install -c h2oai h2o
this should solve your issue.
But more generally, the packages will appear in conda if you use anaconda pip. You can check which pip is being used by doing
which pip
and making sure that the path to the pip is within your anaconda distribution; something like /home/<userdir>/anaconda/bin/pip instead of /usr/bin/pip
Same is true also for Python. Try checking if starting Python in terminal points to anaconda Python by doing which python. If that's not the case, than you would need to add the conda installation of Python to your PATH variable. Please refer to conda docs for instructions https://conda.io/docs/user-guide/install/index.html
It would have been helpful if you had included information regarding your operating system in the question.
Forgive me in case this too easy a question.
Background:
I installed Python 3 and Anaconda as I want to start programming Python and want to dive into machine learning.
Unfortunately all this shell based installation thing is absolutely new to me.
I managed to install python 3.6. and Ananconda 3.
Problem(s):
Unfortunately now I have several Python instances installed. One with Anaconda under user/anaconda and some under library/Python/
Now I need for a course to install PyML and PyBrain. I tried to it the "normal way". i.e. what is written on the webpage and as so often I get multiple errors during the installation.
Goal:
I'd like to have it under the Anaconda packages. Is there a way to do that?
If no: how can I get my other Python 3.6. installation to know the location of the Anaconda packages?
edit 12.08.17
The content of $PATH:
/Users/ak/anaconda/bin:/Users/ak/miniconda2/bin:/Users/ak/miniconda3/bin:/opt/local/bin:/opt/local/sbin:/Library/Frameworks/Python.framework/Versions/2.7/bin:/Library/Frameworks/Python.framework/Versions/3.6/bin:/Library/Frameworks/Python.framework/Versions/3.5/bin:/Users/ak/.rbenv/shims:/Library/Frameworks/EPD64.framework/Versions/Current/bin:/Library/Frameworks/Python.framework/Versions/Current/bin:/Library/Frameworks/Python.framework/Versions/2.7/bin:/Users/ak/Library/Application Support/GoodSync:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/opt/X11/bin:/usr/local/MacGPG2/bin:/Library/TeX/texbin:/Users/ak/.rvm/bin
edit 12.08.17 - 2:
When I try to "conda" this libraries this happens though I started it from the unzipped, downloaded PyML:
AKs-MacBook-Pro:PyML-0.7.14 ak$ conda install -c manmadescience pyml
Fetching package metadata ...........
PackageNotFoundError: Packages missing in current channels:
pyml
And when I do this: conda install -c manmadescience pyml (found here https://anaconda.org/manmadescience/pyml)
I get the same result.