I am using linux-datascience-svm VM provided on Azure in my batch GPU pool. At first I tried to pip install some libraries like so:
pip install --upgrade pip;
pip install docopt;
pip install pubnub;
pip install azure;
pip install glob2;
pip install theano>=0.8.2
pip freeze;
However when my application tries to import theano it gives a Module Not Found error for theano.
I tried leveraging Anaconda so I tried activating base environment in the pool start task, then running the following task cmdline:
/bin/bash -c "set -e;
source activate base;wait"
however I get the following error:
/bin/bash: line 1: activate: No such file or directory
I tried to put the conda environment activation statement in a bash script and running it but I get this error:
./run.sh: line 3: source: activate: file not found
How can I access my installed libraries like theano after they've been installed on the pool in conda or in the general environment?
Try replacing activate with the absolute path to the activate script in conda. It would look something like
source /data/username/miniconda2/bin/activate base
Related
I tried to create virtual environment using pyenv virtualenv 3.8.2 myenv, but it failed, i don't know why, i even changed the python version or the virtual environment name, but it still not working.
Some detail:
I tried this in macos and zsh.
➜ ~ pyenv virtualenv 3.8.2 myenv
Looking in links: /var/folders/_9/l8m14fgs6ts9wx0nl1qbzbkm0000gn/T/tmpe9l25o2_
Requirement already satisfied: setuptools in /Users/fitz/.pyenv/versions/3.8.2/envs/myenv/lib/python3.8/site-packages (41.2.0)
Requirement already satisfied: pip in /Users/fitz/.pyenv/versions/3.8.2/envs/myenv/lib/python3.8/site-packages (19.2.3)
rm: /Users/fitz/.pyenv/shims/shims: is a directory
➜ ~ pyenv versions
system
3.7.3
* 3.8.2 (set by /Users/fitz/.python-version)
3.9.5
I had the same issue. For me it was caused by two empty folders in my shims directory. I deleted the folders and it fixed the issue. So, just running the following would fix your issue.
rm -rf ~/.pyenv/shims/shims
rm -rf ~/.pyenv/shims/versions
Try using 'venv' instead of pyenv or pyvenv or virtualenv. Venv is a library that already comes with your python installation. Virtualenv is an external one.pyenv is similar to venv in that it lets you manage multiple python environments. However with pyenv you can't conveniently rollback library installs to some start state and you will likely need admin privileges at some point to update libraries. So I think it is also best to use venv.
First, make a directory :
mkdir testing
Then, moved to this directory named testing :
cd testing
When you type following command in this directory:
python3 -m venv env
You got error like :
The virtual environment was not created successfully because ensurepip is not
available. On Debian/Ubuntu systems, you need to install the python3-venv
package using the following command.
apt install python3.8-venv
Type the following command but before that keep an eye on the version of python you installed on the machine; in my case its python3.8
sudo apt install python3.8-venv
Now, we can create a virtual environment and store its tools in the "bhandari" folder .
python3 -m venv bhandari
Note: you can named this "bhandari" folder; anyname you like( Standard practice is to name it "env" ...)
Now to activate your virtual environment, from the directory of your folder, type the following command this will activate our virtual environment in the “bhandari” folder
source bhandari/bin/activate
If you have successfully activated your virtual environment, you should see the (bhandari) word indicating that we are working in a virtual environment.
After this, we can install anything that will be isolated from the rest of the system....
To install to my own directory I can use
pip install --user package
Alternatively I can use
conda install package
How do I ask conda to install to my home directory since conda does not take a --user flag?
Installing pip packages to $HOME folder
I don't think it's possible. Use virtual environments (conda create).
See -p option here:
-p PATH, --prefix PATH
Full path to environment prefix.
So to install to, say, local under your home directory, do:
conda install -p $HOME/local packagename
Note, however, this is not a "clean" install as it adds a bunch of conda-related files.
To install conda packages on your own directory you can follow these steps:
Create a blank environment
conda create -y -n my-conda-env
Replace the name my-conda-env with any name you want to give the environment.
Activate the environment
source activate my-conda-env
Don't forget to replace my-conda-env with the name you gave the conda environment from the previous step
Install your package
conda install -c bioconda epa-ng
And that's it, the package should be installed on your own directory
Simply:
sudo conda install -c conda-forge package
Or:
sudo chmod -R 777 ./
conda install -c conda-forge package
I don't know of an exact match for the --user flag, but a reasonable analogue is to use a virtual environment.
What I do when I have to install to a shared CentOS server where I don't have admin access:
First I run
conda env list
will list all conda virtual environments and display the path to each. Once you have the environment created and can see it in the conda env list, copy the path to the environment.
If you need to create one, you can do that with conda create or by running anaconda-navigator and using the GUI.
Activate your environment (if not active) with
conda activate [environment_name]
or
activate [environment_name]
depending on your system (most linux systems use the first, Windows and CentOS use the latter).
Now you can use
conda install -p [environment_path] [package_name]
and you are off to the races.
This is really a work around; it's not the best but it does install the package to the selected virtual environment.
The current Anaconda Install Individual Edition, when run in a linux local account, installs in a local directory. So all the subsequent installs should install there, too.
According to the documentation:
--use-local
I was attempting to use anaconda to download tensorflow. I followed the guide character by character. Anaconda downloaded and installed. I used the command:
c:>conda create -n tensorflow python=3.5
which worked, then I used:
c:> activate tensorflow
Which failed to change to a # prompt. So I tried using pip install and got an error message:
'pip' is not recognized as an internal or external command, operable
program or batch file.
Does anyone have any suggestions on how to correct this?
Did you mean to use:
conda create -n tensorflow tensorflow python=3.5
the conda command:
conda install -n <env_name> <package>
translates your code
conda install -n tensorflow pythong-3.5
tells conda to:
- create a new environment,
- that you want your new environment to be named tensorflow, and to
- install python version 3.5 in the environment you just created.
You did not actually tell conda to install TensorFlow.
Personally, I prefer to name my environment, then change into it to install packages:
conda create -n new_env_name python=3.5
source activate new_env_name
conda install tensorflow numpy pandas matplotlib
* Note: if you are on Windows, you may need to use activate my_env_name instead of source activate my_env_name to start your environment.
Which command to use is dependent on what terminal window you are using:
- Powershell requires activate my_env_name,
- Git Bash requires source activate my_env_name.
Often instructions naively state the the former is always used when on a Windows system.
Try source activate tensorflow.
On mac and in some windows environments source activate <env_name> is required. activate <env_name> is used instead in some Windows' environments.
For example, on Windows, if you're in a Git Bash terminal window, you must use source activate <env_name>, but if you're in a Powershell terminal window, then activate <env_name> would be required.
Linux/Mac will always (so far as I know) require source activate <env_name>
Run Anaconda Prompt as an administrator
I just have the same problem and by this way it's fixed.
I have been trying to run Javascript from a Jupyter Notebook on Windows 10 but failed miserably. It seems I cannot install IJavascript to make it available.
All installation guides say to use Anaconda for IJavascript - so I did, but I CANNOT FIND IJAVASCRIPT anywhere within Anaconda, only the js packages, i searched everywhere there was a search bar available. So, because I am stubborn i tried the hard way:
Installed all javascript packages except mocha (which cannot be found) listed here:
https://anaconda.org/javascript/repo
then tried
npm install -g ijavascript
but keep getting this error:
c:\users\ryuuzako\anaconda3\scripts\node_modules\ijavascript\node_modules\nan\nan_json.h(89): error C2660: 'v8::JSON::P
arse': function does not take 2 arguments [C:\Users\ryuuzako\Anaconda3\Scripts\node_modules\ijavascript\node_modules\ze
romq\build\zmq.vcxproj]
"Javascript" type does not appear when creating a new file in Jupyter, my guess is because ijavascript is not installed..
ANY suggestion to make it work is greatly appreciated.
I can provide the whole npm log but it is humongous.
Feel free to assume i am a complete idiot who didn't work with node before.
I've just forked ijavascript and edited the documentation for the Windows installation section as shown below.
Windows
Install Python3 or the Anaconda3 Python distribution.
In the command line:
pip3 install --upgrade pip
pip3 install jupyter
npm install -g ijavascript
ijsinstall
If the ijinstall command is not recognized, you can execute it manually by navigating to your npm install directory and running the ijinstall batch file. If you do not know where your npm install is located, try looking for it in the default install location: C:\Users\USERNAME\AppData\Roaming\npm.
Then you can run jupyter notebook in your terminal to load Jupyter Notebook. When you create a new Jupyter Notebook, you should see the Javascript (Node) kernel available.
If using the Anaconda alternative to the standard Python distribution, it comes pre-installed with Jupyter Notebook. If using Anaconda, you can skip the pip3 install jupyter step.
This picture might also help, when it comes to the ijinstall part.
I struggled with this as well. After installing by running commands:
pip install jupyter
npm install -g ijavascript
You will need to update the environment variable named 'Path' in the 'System Variables' section.
In Windows 10, Python 3.10, the path to add looks like:
C:\Users\your-windows-user\AppData\Roaming\Python\Python310\Scripts
After changing the environment variable, you will need to re-start the computer for it to take effect.
Once this is done, run:
ijsinstall
in the command prompt and you should be good to go.
This also happeed to me.
First thing install anaconda after that(by default you have installed jupyter),
then search in your menu apps for anaconda prompt, then install ijavascript from anaconda prompt:
npm install -g ijavascript
ijsinstall
then run jupyter from anaconda prompt
jupyter notebook
environments: Ubuntu 14.04(64bit) Python2.7.11
Firstly, I installed tensorflow in the way of Virtualenz installation.
$ sudo apt-get install python-pip python-dev python-virtualenv
$ virtualenv --system-site-packages ~/tensorflow
$ source ~/tensorflow/bin/activate
$export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.12.1-cp27-none-linux_x86_64.whl
$ export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-0.12.1-cp27-none-linux_x86_64.whl
$ pip install --upgrade $TF_BINARY_URL
and then, I test my installation and some issue appear. I know I didn't install tensorflow successfully.
import tensorflow
Traceback (most recent call last):
File "", line 1, in
ImportError: No module named tensorflow
import tensorflow as tf
Traceback (most recent call last):
File "", line 1, in
ImportError: No module named tensorflow
I don't know how to solve the problem. Please help me, it cost me one day. I tried to uninstall tensorflow and then I installed in the way of pip installation. But I get the same error.
The protocbuf is 3.1.0.
Are you running python in the same virtual environment you installed tensorflow in?
To access your tensorflow installation, you have to first "activate" the virtualenv in any new terminals, as follows:
source ~/tensorflow/bin/activate
python
import tensorflow as tf
If you run the above in a new terminal, does it solve your problem?
When you did
$ export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-0.12.1-cp27-none-linux_x86_64.whl
this step you are specifying that you are going to use Nvidia card.
To run tensorflow with GPU(Nvidia graphics card) you need to satisfy all Nvidia requirements
Nvidia requires some special privileges to its CUDA cores
You also need to check for Cuda pathnames to the LD_LIBRARY_PATH environment variable, check in Nvidia Documentation.Also, you need to install an profiling support, this can be done by libcupti-dev library, which is the NVIDIA CUDA Profile Tools Interface. This library provides advanced profiling support. To install this library, issue the following command:
sudo apt-get install libcupti-dev
But if you want to run tensorflow in CPU mode only, do not specify $ export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-0.12.1-cp27-none-linux_x86_64.whl.With this you are overriding TF_BINARY_URL variable to use Nvidia CUDA core
So, to use CPU from all your steps remove $ export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-0.12.1-cp27-none-linux_x86_64.whl and include only $export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.12.1-cp27-none-linux_x86_64.whl and reinstall
I hope this should clear the problem
In case, your prerequisite python packages are not installed properly,
check several things.
$ source $HOME/tensorflow/bin/activate
$ which python
$ which pip
please check these binaries are in the path $HOME/tensorflow/bin/activate. If so, try
$ pip install -I --upgrade $TF_BINARY_URL
where -I option forces to install packages.
INSTALLATION OF TENSORFLOW ON UBUNTU 18.04
download anaconda python package
install it via shell using bash
$bash anaconda*.sh
editing the .bashrc script //location home
$sudo apt-get install python3-pip
$sudo apt-get update
$cd
$nano .bashrc
nano is the text editor
insert the given line at the end of the file
export PATH=-/anaconda3/bin:$PATH
create a virtual environment
using conda
$conda create -n myenv python=3.5
//SPECIFY THE VERSION REQUIRED DO NOT USE 3.7 AS THERE IS A COMPATIBLITY ISSUE WITH TENSORFLOW 10
$source activate myenv
$pip install -U tensorflow
$python
>>import tensorflow as tf
>> //get this prompt without an error it means the installation is successful
>>exit()
source deactivate
fully tested if an issue arises do let me know
whenever you install python packages i would suggest to do it in a virtual environment