Problems in using pip to install tensorflow on iMac El Capitan? - macos

I've followed the installation guide but the final install step failed.
My installed python version:
python 2.7.10 (default, Oct 23 2015, 19:19:21)
[GCC 4.2.1 Compatible Apple LLVM 7.0.0 (clang-700.0.59.5)] on darwin
I have used the following two commands to install tensorflow
sudo pip install --upgrade $TF_BINARY_URL
sudo -H pip install --upgrade $TF_BINARY_UR
Somehow I found Protobuf was not installed under my Python 2.7,
so I followed the suggestion from the installation site,
pip install protobuf
It failed to install protobuf, see the error message below.
Please help
Thanks in advance
When using the first command, got the following messages:
The directory '/Users/David/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/Users/David/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Then as suggested, I used the second command, then error messages were:
Exception:
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/pip-8.1.2-py2.7.egg/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/Library/Python/2.7/site-packages/pip-8.1.2-py2.7.egg/pip/commands/install.py", line 317, in run
prefix=options.prefix_path,
File "/Library/Python/2.7/site-packages/pip-8.1.2-py2.7.egg/pip/req/req_set.py", line 736, in install
requirement.uninstall(auto_confirm=True)
File "/Library/Python/2.7/site-packages/pip-8.1.2-py2.7.egg/pip/req/req_install.py", line 742, in uninstall
paths_to_remove.remove(auto_confirm)
File "/Library/Python/2.7/site-packages/pip-8.1.2-py2.7.egg/pip/req/req_uninstall.py", line 115, in remove
renames(path, new_path)
File "/Library/Python/2.7/site-packages/pip-8.1.2-py2.7.egg/pip/utils/__init__.py", line 267, in renames
shutil.move(old, new)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 299, in move
copytree(src, real_dst, symlinks=True)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 208, in copytree
raise Error, errors
Error: [('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.py', '/tmp/pip-9CKE1y-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.py', "[Errno 1] Operation not permitted: '/tmp/pip-9CKE1y-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.pyc', '/tmp/pip-9CKE1y-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-9CKE1y-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/__init__.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.py', '/tmp/pip-9CKE1y-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.py', "[Errno 1] Operation not permitted: '/tmp/pip-9CKE1y-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.pyc', '/tmp/pip-9CKE1y-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.pyc', "[Errno 1] Operation not permitted: '/tmp/pip-9CKE1y-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib/markers.pyc'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib', '/tmp/pip-9CKE1y-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib', "[Errno 1] Operation not permitted: '/tmp/pip-9CKE1y-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/_markerlib'")]
For the command 3, the error message:
Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/h5/vh7y894j0js3vqkv_hz4qk6w0000gn/T/pip-build-KWqDAq/protobuf/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/h5/vh7y894j0js3vqkv_hz4qk6w0000gn/T/pip-5RoEnu-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/h5/vh7y894j0js3vqkv_hz4qk6w0000gn/T/pip-build-KWqDAq/protobuf/

I think this is a result of System Integrity Protection in El Capitan. I was able to install using the following:
# Note the lack of "sudo" and the addition of "--user"
pip install --user --upgrade $TF_BINARY_URL
# And then to ensure that the user-specific packages are used,
# I had to start python from the user-site packages directory
cd $(python -m site --user-site)

I had a similar problem when installing awsebcli,
try:
sudo pip install protobuf --ignore-installed six

Related

DLVM - pip install error

Trying to pip install on Azure Deep Learning VM is giving this error-
Complete output from command python setup.py egg_info:
ERROR:root:Error parsing
Traceback (most recent call last):
File "/anaconda/envs/py35/lib/python3.5/site-packages/pbr/core.py", line 111, in pbr
attrs = util.cfg_to_args(path, dist.script_args)
File "/anaconda/envs/py35/lib/python3.5/site-packages/pbr/util.py", line 267, in cfg_to_args
wrap_commands(kwargs)
File "/anaconda/envs/py35/lib/python3.5/site-packages/pbr/util.py", line 569, in wrap_commands
cmdclass = ep.resolve()
File "/anaconda/envs/py35/lib/python3.5/site-packages/setuptools-27.2.0-py3.5.egg/pkg_resources/__init__.py", line 2264, in resolve
module = __import__(self.module_name, fromlist=['__name__'], level=0)
ImportError: No module named 'setuptools.command.build_clib'
error in setup command: Error parsing /tmp/pip-build-9ucgxr1d/cliff/setup.cfg: ImportError: No module named 'setuptools.command.build_clib'
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-9ucgxr1d/cliff/
Given the paths above it appears this is happenning on the Python 3.5 (py35) conda environment on the Linux edition of the Azure Data Science VM (DSVM) and Deep Learning VM (DLVM).
Based on the error message, the general guidance to resolve this is to update the setuptools and then try to install the library.
The commands to do that on the DSVM/DLVM on the global py35 conda environment is:
sudo /anaconda/envs/py35/bin/pip install --upgrade setuptools
sudo /anaconda/envs/py35/bin/pip install <<package name>>
For the root (Python 2.7) environment on the DSVM the commands are:
sudo /anaconda/bin/pip install --upgrade setuptools
sudo /anaconda/bin/pip install <<package name>>
Hope this resolves the issue.

"OSError: mysql_config not found" when trying to "pip install mysqlclient" - Django

I'm relatively new to Django, and Web Development in general.
I am trying to
pip install mysqlclient
in my
virtualenv -p python3
to hook up Django 2.0 to mySQL. However, I'm getting this error:
Collecting mysqlclient
Using cached mysqlclient-1.3.12.tar.gz
Complete output from command python setup.py egg_info:
/bin/sh: mysql_config: command not found
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/var/folders/43/md5vpqrx0mx8627sq04slbz00000gn/T/pip-build-l8ea3vja/mysqlclient/setup.py", line 17, in <module>
metadata, options = get_config()
File "/private/var/folders/43/md5vpqrx0mx8627sq04slbz00000gn/T/pip-build-l8ea3vja/mysqlclient/setup_posix.py", line 44, in get_config
libs = mysql_config("libs_r")
File "/private/var/folders/43/md5vpqrx0mx8627sq04slbz00000gn/T/pip-build-l8ea3vja/mysqlclient/setup_posix.py", line 26, in mysql_config
raise EnvironmentError("%s not found" % (mysql_config.path,))
OSError: mysql_config not found
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/43/md5vpqrx0mx8627sq04slbz00000gn/T/pip-build-l8ea3vja/mysqlclient/
I have looked around for answers for hours, and yes, I have checked out this and this but nothing seems to be working.
Any help would be much appreciated!
the python package "mysqlclient" is depended on C system library to drive mysql connect,if you on ubuntu 16.4, just do as below:
sudo apt-get install libmysqlclient-dev
sudo pip install mysqlclient
If you get OSError: mysql_config not found when installing the mysqlclient Python package in macOS, then just do:
brew install mysql-client
export PATH="/usr/local/opt/mysql-client/bin:$PATH"
and retry.
You should start with cloning the mysqlclient repo:
git clone https://github.com/PyMySQL/mysqlclient-python
Then you will have to get mysql-connector-c. You can get it by doing:
brew install mysql-connector-c
Then open /usr/local/bin/mysql_config in a text editor.
From the Github issue:
There are lines in mysql_config like following:
# Create options
libs="-L$pkglibdir"
libs="$libs -l "
It should be:
#Create options
libs="-L$pkglibdir"
libs="$libs -lmysqlclient -lssl -lcrypto"
Save that file and change back to the directory where you downloaded the repo. Now open site.cfg file.
uncomment the line (remove the #)
#mysql_config = /usr/local/bin/mysql_config
Save that and run:
python3 setup.py install
For CentOS here's what made it work for me:
yum install gcc mysql-devel openssl-devel
#Install library explicitly, otherwise the python3 setup.py install will fail
pip3 install "mysqlclient==2.0.3" --global-option=build_ext --global-option="-L/usr/lib64/mysql/"

How can I resolve this issue regarding upgradation of pip on ubuntu 16.04?

Here is what happened.I even tried to remove pip, but it shows the same problem.My python version is 2.7.12
saikat#saikat:~$ pip install --upgrade pip
Collecting pip
Using cached pip-9.0.1-py2.py3-none-any.whl
Installing collected packages: pip
Found existing installation: pip 8.1.2
Uninstalling pip-8.1.2:
Exception:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/usr/local/lib/python2.7/dist-packages/pip/commands/install.py", line 317, in run
prefix=options.prefix_path,
File "/usr/local/lib/python2.7/dist-packages/pip/req/req_set.py", line 736, in install
requirement.uninstall(auto_confirm=True)
File "/usr/local/lib/python2.7/dist-packages/pip/req/req_install.py", line 742, in uninstall
paths_to_remove.remove(auto_confirm)
File "/usr/local/lib/python2.7/dist-packages/pip/req/req_uninstall.py", line 115, in remove
renames(path, new_path)
File "/usr/local/lib/python2.7/dist-packages/pip/utils/__init__.py", line 267, in renames
shutil.move(old, new)
File "/usr/lib/python2.7/shutil.py", line 303, in move
os.unlink(src)
OSError: [Errno 13] Permission denied: '/usr/bin/pip'
You are using pip version 8.1.2, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
try this.
wget https://pypi.python.org/packages/e7/a8/7556133689add8d1a54c0b14aeff0acb03c64707ce100ecd53934da1aa13/pip-8.1.2.tar.gz
tar -xzvf pip-8.1.2.tar.gz
cd pip-8.1.2
sudo python setup.py install
It should help

Install Tensorflow from source

I'm trying to install tensorflow on a cluster (I have no sudo rights).
If I try to use pip I first set the environment variable:
export TF_BINARY_URL=https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-0.9.0-cp27-none-linux_x86_64.whl
and then I execute:
pip install --upgrade $TF_BINARY_URL -b=/home/js668623/tools/tensorflow
The output is:
You are using pip version 7.1.0, however version 8.1.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Collecting tensorflow==0.9.0 from https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-0.9.0-cp27-none-linux_x86_64.whl
Using cached https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow-0.9.0-cp27-none-linux_x86_64.whl
Collecting six>=1.10.0 (from tensorflow==0.9.0)
Using cached six-1.10.0-py2.py3-none-any.whl
Collecting protobuf==3.0.0b2 (from tensorflow==0.9.0)
Using cached protobuf-3.0.0b2-py2.py3-none-any.whl
Collecting wheel (from tensorflow==0.9.0)
Using cached wheel-0.29.0-py2.py3-none-any.whl
Collecting numpy>=1.8.2 (from tensorflow==0.9.0)
Using cached numpy-1.11.1.zip
Collecting setuptools (from protobuf==3.0.0b2->tensorflow==0.9.0)
Using cached setuptools-23.1.0-py2.py3-none-any.whl
Installing collected packages: six, setuptools, protobuf, wheel, numpy, tensorflow
Found existing installation: six 1.9.0
Uninstalling six-1.9.0:
Exception:
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/pip/basecommand.py", line 223, in main
status = self.run(options, args)
File "/usr/lib/python2.7/site-packages/pip/commands/install.py", line 308, in run
strip_file_prefix=options.strip_file_prefix,
File "/usr/lib/python2.7/site-packages/pip/req/req_set.py", line 640, in install
requirement.uninstall(auto_confirm=True)
File "/usr/lib/python2.7/site-packages/pip/req/req_install.py", line 726, in uninstall
paths_to_remove.remove(auto_confirm)
File "/usr/lib/python2.7/site-packages/pip/req/req_uninstall.py", line 125, in remove
renames(path, new_path)
File "/usr/lib/python2.7/site-packages/pip/utils/__init__.py", line 314, in renames
shutil.move(old, new)
File "/usr/lib64/python2.7/shutil.py", line 299, in move
rmtree(src)
File "/usr/lib64/python2.7/shutil.py", line 252, in rmtree
onerror(os.remove, fullname, sys.exc_info())
File "/usr/lib64/python2.7/shutil.py", line 250, in rmtree
os.remove(fullname)
OSError: [Errno 13] Permission denied: '/usr/lib/python2.7/site-packages/six-1.9.0-py2.7.egg-info/PKG-INFO'
If you don't have sudo rights on the machine, personally I find the "virtualenv" installation easiest. It should just work without root access.
https://www.tensorflow.org/versions/r0.9/get_started/os_setup.html#virtualenv-installation
You might also look into the "anaconda" installation, but I have no experience with that.
If you also need a specific version of python, you 'll need more than the script I give you, but with a little bit of diggig, that script should work for you ;)
module load Python/2.7.11-foss-2016a
curl -O https://bootstrap.pypa.io/get-pip.py
python get-pip.py --user --upgrade
rm get-pip.py
.local/bin/pip install --user --upgrade pip
git clone https://github.com/tensorflow/tensorflow
git clone https://github.com/bazelbuild/bazel.git
wget --no-check-certificate --no-cookies --header "Cookie: oraclelicense=accept-securebackup-cookie" http://download.oracle.com/otn-pub/java/jdk/8u92-b14/jdk-8u92-linux-x64.tar.gz
tar -xf jdk*
rm jdk*.gz
cd bazel
export JAVA_HOME=~/jdk1.8.0_92/
module unload GCC/4.9.3-2.25
module load GCC/4.8.2
./compile.sh
cd ../tensorflow
export PATH=/home/ulg/sysmod/gmath/bazel/output/:$PATH
./configure
bazel build -c opt //tensorflow/tools/pip_package:build_pip_package 2>/dev/null
bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
cd
.local/bin/pip install --user --upgrade /tmp/tensorflow_pkg/tensorflow-0.8.0-py2-none-any.whl

Error [Errno 89] Function not implemented while executing command python setup.py egg_info

Update 6/25/2013) I see the same error when trying to install different package using pip, indicating that the package is likely to have nothing to do with the issue.
On QNX I'm seeing the following error upon running pip.
# pip install -U catkin_pkg
Downloading/unpacking catkin-pkg
Running setup.py egg_info for package catkin-pkg
Requirement already up-to-date: argparse in /usr/pkg/lib/python2.7/site-packages (from catkin-pkg)
Requirement already up-to-date: docutils in /usr/pkg/lib/python2.7/site-packages (from catkin-pkg)
Requirement already up-to-date: python-dateutil in /usr/pkg/lib/python2.7/site-packages (from catkin-pkg)
Requirement already up-to-date: six in /usr/pkg/lib/python2.7/site-packages (from python-dateutil->catkin-pkg)
Installing collected packages: catkin-pkg
Running setup.py install for catkin-pkg
Error [Errno 89] Function not implemented while executing command /usr/qnx650/host/qnx6/x86/usr/bin/python -c "import setuptools;__file__='/tmp/pip-build-n130s/catkin-pkg/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-XhRdBe-record/install-record.txt --single-version-externally-managed
Exception:
Traceback (most recent call last):
File "/usr/pkg/lib/python2.7/site-packages/pip-1.3-py2.7.egg/pip/basecommand.py", line 139, in main
status = self.run(options, args)
File "/usr/pkg/lib/python2.7/site-packages/pip-1.3-py2.7.egg/pip/commands/install.py", line 271, in run
requirement_set.install(install_options, global_options, root=options.root_path)
File "/usr/pkg/lib/python2.7/site-packages/pip-1.3-py2.7.egg/pip/req.py", line 1185, in install
requirement.install(install_options, global_options, *args, **kwargs)
File "/usr/pkg/lib/python2.7/site-packages/pip-1.3-py2.7.egg/pip/req.py", line 592, in install
cwd=self.source_dir, filter_stdout=self._filter_install, show_stdout=False)
File "/usr/pkg/lib/python2.7/site-packages/pip-1.3-py2.7.egg/pip/util.py", line 627, in call_subprocess
cwd=cwd, env=env)
File "/usr/pkg/lib/python2.7/subprocess.py", line 711, in __init__
errread, errwrite)
File "/usr/pkg/lib/python2.7/subprocess.py", line 1205, in _execute_child
self.pid = os.fork()
OSError: [Errno 89] Function not implemented
Storing complete log in /root/.pip/pip.log
The first time I ran this command when the depended components such as argparse, docutils, python-dateutil and six were not installed, the same error happened, but at the part where these required components are checked. Now that I've installed them one by one, it seems to be happening at the pkg I'm installing itself (ie. catkin_pkg).
On python's console, os.fork(), which I assume the source of the cause, behaves differently. Thus I'm not even sure if the error occurs at os.fork() or not.
# python
Python 2.7.4 (default, Jun 21 2013, 22:33:48)
[GCC 4.4.2] on qnx6
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.fork()
18804790
>>> 0
sem_wait: Invalid argument
How can I avoid this error? Thank you!
Using QNX 6.5.0 SDP SP1, pip version is 1.3 (its source on github), also source of catkin_pkg.
I somehow figured it out. Running pip WITHOUT -U option gets the package installed (install log is at the bottom for the reference).
...But why...? Help message of pip's doesn't seem to make sense to me:
$ pip install --help
:
Install Options:
:
-U, --upgrade Upgrade all packages to the newest available version. This process is recursive regardless of whether a dependency is already satisfied.
Log of installation by pip:
$ pip install catkin_pkg
Downloading/unpacking catkin-pkg
Running setup.py egg_info for package catkin-pkg
Requirement already satisfied (use --upgrade to upgrade): argparse in /usr/pkg/lib/python2.7/site-packages (from catkin-pkg)
Requirement already satisfied (use --upgrade to upgrade): docutils in /usr/pkg/lib/python2.7/site-packages (from catkin-pkg)
Requirement already satisfied (use --upgrade to upgrade): python-dateutil in /usr/pkg/lib/python2.7/site-packages (from catkin-pkg)
Requirement already satisfied (use --upgrade to upgrade): six in /usr/pkg/lib/python2.7/site-packages (from python-dateutil->catkin-pkg)
Installing collected packages: catkin-pkg
Running setup.py install for catkin-pkg
changing mode of build/scripts-2.7/catkin_create_pkg from 664 to 775
changing mode of build/scripts-2.7/catkin_generate_changelog from 664 to 775
changing mode of build/scripts-2.7/catkin_tag_changelog from 664 to 775
changing mode of build/scripts-2.7/catkin_test_changelog from 664 to 775
changing mode of /usr/pkg/bin/catkin_create_pkg to 775
changing mode of /usr/pkg/bin/catkin_generate_changelog to 775
changing mode of /usr/pkg/bin/catkin_tag_changelog to 775
changing mode of /usr/pkg/bin/catkin_test_changelog to 775
Successfully installed catkin-pkg
Cleaning up...

Resources