How to install AutoKeras on aws-ec2 - amazon-ec2

I tried to install autokeras on aws ec2 (p2.xlarge) with the environment python 3.6 & tensorflow. I get following error after "pip install autokeras":
tensorflow 1.10.0 has requirement numpy<=1.14.5,>=1.13.3, but you'll have numpy 1.15.4 which is incompatible.
Installing collected packages: imageio, autokeras
Found existing installation: imageio 2.3.0
Cannot uninstall 'imageio'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
I uninstalled numpy 1.15.4 and installed numpy 1.14.5. With "conda list", I can see the numpy has the correct version.
But after "pip install autokeras" I get the same error and numpy 1.15.4 is still there.
Has anyone successfully installed autokeras on aws ec2? What shall I do to install autokeras correctly?
(Maybe the 'imageio' is the next problem?)
Thank you!

I think you need tensorflow 1.14. Here my notes for AutoKeras installation:
Autokeras Installation Notes in the Deep Learning AMI:
We launched a new deep learning AMI with ubuntu.
The deep learning AMI didn't worked using the "tensorflow + keras + py3.6" environment (so no need for a DL AMI probably, you can save space on disk using a normal clean AMI), so we managed o install autokeras doing the following:
Create a new environment with Anaconda: $ conda create -n autokeras python=3.6.
1.1. Remember that only python 3.6 is working with autokeras
Activate virtual env: It didn't work $ conda activate autokeras, but it works using $ source activate autokeras.
installation of all the packages as required by pyimagesearch.
3.1. A new problem arised here, which, long story short, was solved using the next post (note that I chenged the order, since urllib3 needs jsonschema to be installed first):
$ pip uninstall urllib3```
$ pip uninstall jsonschema
$ pip install jsonschema==2.6.0
$ pip install urllib3==1.24.1
3.2. Finally I was able to install all 3 packages:
$ pip install tensorflow # or tensorflow-gpu
$ pip install keras
$ pip install autokeras
3.3. Autokeras worked fine at this point, but it raised a warning:
>>> import autokeras
Better speed can be achieved with apex installed from https://www.github.com/nvidia/apex
So I just went to the webpage and followed the installation steps. Now it works without warnings (so far):
$ git clone https://github.com/NVIDIA/apex
$ cd apex
$ pip install -v --no-cache-dir --global-option="--pyprof" --global-option="--cpp_ext" --global-option="--cuda_ext" ./

Related

PyTorch dll issues for Caffe2

I am using a Windows 10 Machine and after re-installing Anaconda and all of the packages I had previously, including torchvision, torch and necessary dependencies, I am still getting this error:
OSError: [WinError 127] The specified procedure could not be found. Error loading "C:\Users\XXX\Anaconda3\envs\XXX\lib\site-packages\torch\lib\caffe2.dll" or one of its dependencies.
I am using python 3.7.9 and:
torchaudio=0.6.0=py37
torchvision=0.7.0=py37_cpu
tornado=6.0.4=py37he774522_1
traitlets=5.0.5=py_0
I've looked into it quite a bit but feel like this should be an easy solve...
I do not have CUDA and have used this:
conda install pytorch torchvision torchaudio cpuonly -c pytorch
as per instructed on the official website of pytorch
After a long time trying many things with Anaconda I decided to use bare python instead and I installed Python 3.8.6 and installed PyTorch from the link you provided and it finally worked even with CUDA support. Make sure to completely remove all Anaconda/Other Python version scripts from your path to ensure only the 3.8.6 version is used by your prompt.
I had the same problem, so I uninstalled the pytorch in my machine using
"conda uninstall pytorch torchvision torchaudio cudatoolkit=10.2 -c pytorch"
Then did a clean installation using the following
"pip install torch===1.7.0 torchvision===0.8.1 torchaudio===0.7.0 -f https://download.pytorch.org/whl/torch_stable.html"
I did not use conda for installation as it was persistently giving me the cpu version of pytorch.
I tried to install PyTorch with Anaconda and had the same error.
For me what solved the issue was to uninstall Pytorch using Pip (even though I installed it with conda) and then installing again with Pip as explains at PyTorch guide.
Uninstall:
pip uninstall torch
pip uninstall torchvision
pip uninstall torchaudio
Install:
pip3 install torch torchvision torchaudio

Unable to install coremltools

I am trying to convert a pb file to a coreml file. to do this i need to install coremltools. However, when i try to install it by pip it comes up with this error :
ERROR: Could not find a version that satisfies the requirement coremltools (from versions: none)
ERROR: No matching distribution found for coremltools
i have tried to install it in a python 2.7 environment, still no joy
pip install coremltools
Collecting coremltools
ERROR: Could not find a version that satisfies the requirement coremltools (from versions: none)
ERROR: No matching distribution found for coremltools
Rorys-MBP:~ roryhodgson$
The only reason I could found out that explains why this is happening is that coremltools require python 2.7, make sure you are running it pip --version. If you just typed pip install coremltools the chances are that your machine (assuming it is running macOS) pip command is running the default version of macOS python which probably is 3.5.2 or greater.
I could fix this issue by creating an environment in which my python version was 2.7:
pip install virtualenv
Create a virtual environment:
virtualenv --python=/usr/bin/python2.7 py27
Activate it:
source py27/bin/activate
Lastly, install coremltools:
pip install -U coremltools
When you are done just deactivate the environment running deactivate in the terminal and that's it.
All this is available at the following source: satoshi.blogs.com
If you install from GitHub, then you will not need to install Python 2.7 or fiddle with virtual environments.
pip install "git+https://github.com/apple/coremltools"
The code above will let you install coremltools by cloning the Git repository.

Issues installing Tensorflow on mac

I am following instructions from tensorflow website using virtualenv to install and having following issue when validating installation using import tensorflow as tf in python interactive shell.
Please note, I am running Python 3.7
Here are list of packages I have installed.
Can someone help?
I also tried to remove protobuf suggested by stackoverflow issue using pip uninstall protobuf, but doesn't help.
I was able to resolve above issue after some research. It looks like, tensorflow is not compatible with python 3.7 (as of July 30th 18) which is what I had installed on my computer .
Following github issue helped me resolved the issue.
I installed python 3.6.6 using pyenv and setup virtualenv. Here is what I did:
# Install python 3.3.6 and create virtualenv
$ pyenv install 3.6.6
$ pyenv virtualenv 3.6.6 tensorflow_image_recognition_3_6_6
$ pyenv activate tensorflow_image_recognition_3_6_6
# update pip
$ easy_install -U pip
#Install tensorflow
$ pip3 install --upgrade tensorflow
# Test if tensorflow is installed properly
# open python interactive shell
$ python
import tensorflow as tf
hello = tf.constant('Hello, TensorFlow!')
sess = tf.Session()
print(sess.run(hello))

Installing tensorflow on anaconda/mac?

I can not find a solution to install tensorflow on anaconda/mac.
When i try: conda install tensorflow,
I get:
UnsatisfiableError: The following specifications were found to be in conflict:
- blaze -> numba -> numpy=1.13
- tensorflow
Use "conda info " to see the dependencies for each package.
I tried reinstalling numpy, and could not find any relevant documentation.
On mac use following command:
conda install -c conda-forge tensorflow
This will install the latest Tensorflow on your system. if you wish to upgrade it to newer version then you can use the following command
conda update -f -c conda-forge tensorflow
I have had success installing tensorflow through pip...
pip install tensorflow
or, if you have all of the proper libraries...
pip install tensorflow-gpu
But if you have already removed numpy form the anaconda install, you might be in for a long day because almost all of the numeric packages require it. If the above pip install doesn't work, you might want to start from a clean anaconda install.
I have installed tensorflow using -
pip install tensorflow
Or, you can use Anaconda in following way -
Open Anaconda Navigator
On Left side go to Environments
Create a new environment (eg :- tensorflow_tf), select python 3.7
then select Not installed and Search "tensorflow"
click on tensorflow and apply

Tensorflow installation error: not a supported wheel on this platform

when I try to install TensorFlow by cloning from Git, I run into the error "no module named copyreg," so I tried installing using a virtualenv. However, I then run into this error:
pip install https://storage.googleapis.com/tensorflow/mac/tensorflow-0.5.0-py2-none-any.whl
tensorflow-0.5.0-py2-none-any.whl is not a supported wheel on this platform.
I don't see this under the common problems section.
I am using OS X v10.10.5 (Yosemite) and Python 3.4.3, but I also have Python 2.7 (I am unsure if pip differentiates between these or how to switch between them).
I too got the same problem.
I downloaded get-pip.py from https://bootstrap.pypa.io/get-pip.py and then ran python2.7 get-pip.py for installing pip2.7.
And then ran the pip install command with python2.7 as follows.
For Ubuntu/Linux:
python2.7 -m pip install https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.5.0-cp27-none-linux_x86_64.whl
For Mac OS X:
python2.7 -m pip install https://storage.googleapis.com/tensorflow/mac/tensorflow-0.5.0-py2-none-any.whl
This should work just fine as it did for me :)
I followed these instructions from here.
After activating the virtualenv, be sure to upgrade pip to the latest version.
(your_virtual_env)$ pip install --upgrade pip
And now you'll be able to install TensorFlow correctly (for Linux):
(your_virtual_env)$ pip install --upgrade https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-0.7.0-py2-none-linux_x86_64.whl
I was trying to do the Windows-based install and kept getting this error.
It turns out you have to have Python 3.5.2. Not 2.7, not 3.6.x-- nothing other than 3.5.2.
After installing Python 3.5.2, the pip install worked.
Make sure that the wheel is, well, supported by your platform. Pip uses the wheel's filename to determine compatibility. The format is:
tensorflow-{version}-{python version}-none-{your platform}.whl
I didn't realize that x86_64 refers to x64, I thought it meant either x86 or x64, so I banged my head against this futilely for some time. TensorFlow is not available for 32-bit systems, unless you want to compile it yourself.
It seems that TensorFlow only works on Python 3.5 at the moment. Try to run this command before running the pip install
conda create --name tensorflow python=3.5
After this, run the following lines:
For CPU:
pip install --ignore-installed --upgrade https://storage.googleapis.com/tensorflow/windows/cpu/tensorflow-1.1.0-cp35-cp35m-win_amd64.whl
For GPU:
pip install --ignore-installed --upgrade https://storage.googleapis.com/tensorflow/windows/gpu/tensorflow_gpu-1.1.0-cp35-cp35m-win_amd64.whl
It should work like a charm.
On Windows 10, with Python 3.6.X version I was facing the same. Then after checking deliberately, I noticed I had Python-32 bit installation on my 64-bit machine. Remember TensorFlow is only compatible with a 64-bit installation of Python. Not 32 bit of Python
If we download Python from python.org, the default installation would be 32 bit. So we have to download the 64 bit installer manually to install Python 64 bit. And then add
C:\Users<username>\AppData\Local\Programs\Python\Python36
C:\Users<username>\AppData\Local\Programs\Python\Python36\Scripts
Then run gpupdate /Force on the command prompt. If the Python command doesn’t work for 64 bit, restart your machine.
Then run the Python interpreter on the command prompt. It should show 64 bit
python
Python 3.6.3 (v3.6.3:2c5fed8, Oct 3 2017, 18:11:49) [MSC v.1900 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
Then run the below command to install the TensorFlow CPU version (recommended)
pip3 install --upgrade tensorflow
The pip wheel file contains the Python version in its name (cp34-cp34m). If you download the .whl file and rename it to say py3-none or instead, it should work. Can you try that?
The installation won't work for Anaconda users that choose Python 3 support, because the installation procedure is asking to create a Python 3.5 environment and the file is currently called cp34-cp34m. So renaming it would do the job for now.
sudo pip3 install --upgrade https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-0.7.0-cp34-cp34m-linux_x86_64.whl
This will produce the exact error message you got above. However, when you will download the file yourself and rename it to "tensorflow-0.7.0-py3-none-linux_x86_64.whl", then execute the command again with the changed filename, it should work fine.
For Windows 10 64 bit:
I have tried all the suggestions here, but finally I got it running as follows:
Uninstall all current versions of Python
Remove all Python references in the PATH system and user environment variables
Download the latest 64-bit version of Python 3.8: Python 3.8.7 currently, not the latest 3.9.x version which is the one I was using, and not 32 bit.
Install with all options selected, including pip, and including the PATH environment variable
pip install tensorflow (in an administrator CMD prompt)
Upgrade pip if prompted (optional)
Actually, you can use Python 3.5.*.
I successfully solved this problem with Python 3.5.3. Modify the Python version to 3.5.* in Conda. See Managing Python.
Then go to https://www.tensorflow.org/install/install_windows, and repeat from "Create a Conda environment named tensorflow by invoking the following command" bla, bla...
Maybe you are installing the wrong pre-build binary?
Check on https://github.com/lakshayg/tensorflow-build
For my Coffee Lake processor on Ubuntu 18.04 (Bionic Beaver) the download URL was:
https://github.com/lakshayg/tensorflow-build/releases/download/tf1.12.0-ubuntu18.04-py2-py3/tensorflow-1.12.0-cp36-cp36m-linux_x86_64.whl
pip install --ignore-installed --upgrade <PATH>
resolved the issue for me.
I was trying to install from source and got that error. (Why would a wheel built on this machine not be compatible with it?)
For me, the tag --ignore-installed made all the difference.
pip install --ignore-installed /tmp/tensorflow_pkg/tensorflow-1.8.0-cp36-cp36m-linux_x86_64.whl
worked, while
pip install /tmp/tensorflow_pkg/tensorflow-1.8.0-cp36-cp36m-linux_x86_64.whl
threw the abovementioned error.
Context: Conda environment; it might have been a problem specific to this
I was trying to install CPU TensorFlow on Ubuntu 18.04 (Bionic Beaver), and the best way (for me...) I found for it was using it on top of Conda, for that:
To create a Conda ‘tensorflow’ environment. Follow How to Install Anaconda on Ubuntu 18.04
After all is installed, see Getting started with conda. And use it according to Managing environments
conda create --name tensorflow
source activate tensorflow
pip install --upgrade pip
pip uninstall tensorflow
For CPU: pip install tensorflow-cpu, for GPU: pip install tensorflow
pip install --ignore-installed --upgrade tensorflow
Test TF E.g. on 'Where' with:
python
import tensorflow as tf
>>> tf.where([[True, False], [False, True]])
Expected result:
<tf.Tensor: shape=(2, 2), dtype=int64, numpy=
array([[0, 0],
[1, 1]])>
After the Conda upgrade, I got:
DeprecationWarning: 'source deactivate' is deprecated. Use 'conda deactivate'.
So you should use:
‘conda activate tensorflow’ / ‘conda deactivate’
I faced the same issue and tried all the solutions that folks suggested here and other links (like Platform not supported for TensorFlow on Ubuntu 14.04.2).
It was so frustrating because using print(wheel.pep425tags.get_supported()) I could see that my Ubuntu supported ('cp37', 'cp37m', 'linux_x86_64') and that was exactly what I was trying to install (from https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.14.0-cp37-cp37m-linux_x86_64.whl).
What at the end fixed it was to simply download the package first and then
pip install tensorflow-1.14.0-cp37-cp37m-linux_x86_64.whl
It means that the version of your default Python interpreter (python -V) and the version of your default pip (pip -V) do not match. You have built TensorFlow with your default Python interpreter and trying to use a different pip version to install it.
In Mac, delete /usr/local/bin/pip and rename (copy) pipx.y (whatever x.y version that matches your Python version) to pip in that folder.
This worked for me.
system requirement Python 3.7–3.10
macOS 10.12.6 (Sierra) or later (no GPU support)
pip install tensorflow-macos

Resources