IPython-Parallel: ipcluster and ipcontroller-client.json not found - parallel-processing

I'm trying to follow these instructions on using ipyparallel in order to speed up some Python scripts in Jupyter Notebook. When doing
import ipyparallel as ipp
IPPC = ipp.Client()
I get the following error:
Using matplotlib backend: TkAgg
Waiting for connection file: ~/.ipython/profile_default/security/ipcontroller-client.json
---------------------------------------------------------------------------
OSError Traceback (most recent call last)
<ipython-input-11-bf8e572b63bc> in <module>()
18 display(HTML("<style>.container {width:100% !important;}</style>"))
19
---> 20 IPPC = ipp.Client()
21
22
/home/guest/.local/lib/python3.7/site-packages/ipyparallel/client/client.py in __init__(self, url_file, profile, profile_dir, ipython_dir, context, debug, sshserver, sshkey, password, paramiko, timeout, cluster_id, **extra_args)
415 no_file_msg,
416 ])
--> 417 raise IOError(msg)
418 if url_file is None:
419 raise IOError(no_file_msg)
OSError: Connection file '~/.ipython/profile_default/security/ipcontroller-client.json' not found.
You have attempted to connect to an IPython Cluster but no Controller could be found.
Please double-check your configuration and ensure that a cluster is running.
The file ~/.ipython/profile_default/security/ipcontroller-client.json does not exist (folder is empty!!) and doing sudo find / -name ipcontroller-client doesn't return any results.
Furthermore, when trying to start ipcluster in the terminal, I get
user#laptop:~$ ipcluster start -n 2
ipcluster: command not found
and the "Clusters" tab in Jupyter Notebook is empty: Screenshot
However, ipyparallel is definitely installed:
user#laptop:~$ pip3 install ipyparallel
Requirement already satisfied: ipyparallel in ./.local/lib/python3.7/site-packages (6.3.0)
Requirement already satisfied: jupyter-client in /usr/lib/python3/dist-packages (from ipyparallel) (5.2.3)
Requirement already satisfied: traitlets>=4.3 in /usr/lib/python3/dist-packages (from ipyparallel) (4.3.2)
Requirement already satisfied: tornado>=4 in /usr/lib/python3/dist-packages (from ipyparallel) (5.1.1)
Requirement already satisfied: pyzmq>=13 in /usr/lib/python3/dist-packages (from ipyparallel) (17.1.2)
Requirement already satisfied: ipykernel>=4.4 in /usr/lib/python3/dist-packages (from ipyparallel) (4.9.0)
Requirement already satisfied: ipython-genutils in /usr/lib/python3/dist-packages (from ipyparallel) (0.2.0)
Requirement already satisfied: decorator in /usr/lib/python3/dist-packages (from ipyparallel) (4.3.0)
Requirement already satisfied: ipython>=4 in /usr/lib/python3/dist-packages (from ipyparallel) (5.8.0)
Requirement already satisfied: python-dateutil>=2.1 in /usr/lib/python3/dist-packages (from ipyparallel) (2.7.3)
Requirement already satisfied: pexpect in /usr/lib/python3/dist-packages (from ipython>=4->ipyparallel) (4.6.0)
and there are three file ipcluster, ipcontroller, and ipengine in /home/user/.local/bin which are Python scripts.
...what could be the problem?

Could it be that you didn't enable the clusters extension?
After installing ipyparallel you need to enable the notebook extension to be able to start clusters from the Jupyter notebook
ipcluster nbextension enable
You might need to re-install ipyparallel before:
pip uninstall notebook ipyparallel
pip install notebook ipyparallel
ipcluster nbextension enable
(the ipyparallel extension can also be enabled from the notebook's nbextensions_configurator if there's one installed)

I had to run ipcluster nbextension enable --user
Otherwise, I got the error PermissionError: [Errno 13] Permission denied: '/usr/local/etc/jupyter'

Related

cx_Freeze is installed using pip, but I am still getting an unresolved reference

So according to my terminal, I am on python --version 3.9.9. I have been trying to turn a large python GUI project into a .exe file by using cx_Freeze. I installed it using the command:
pip install cx_Freeze
This results in what I believe to be a successful installation because I am getting the following result:
Collecting cx-Freeze
Using cached cx_Freeze-6.11.1-cp39-cp39-macosx_10_9_x86_64.whl (14.7 MB)
Requirement already satisfied: packaging>=21.0 in /opt/anaconda3/lib/python3.9/site-packages (from cx-Freeze) (21.3)
Requirement already satisfied: importlib-metadata>=4.8.3 in /opt/anaconda3/lib/python3.9/site-packages (from cx-Freeze) (4.11.3)
Requirement already satisfied: setuptools<=60.10.0,>=59.0.1 in /opt/anaconda3/lib/python3.9/site-packages (from cx-Freeze) (60.10.0)
Requirement already satisfied: zipp>=0.5 in /opt/anaconda3/lib/python3.9/site-packages (from importlib-metadata>=4.8.3->cx-Freeze) (3.7.0)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /opt/anaconda3/lib/python3.9/site-packages (from packaging>=21.0->cx-Freeze) (3.0.4)
Installing collected packages: cx-Freeze
Successfully installed cx-Freeze-6.11.1
The problem I am having is that in my setup.py file, I have an undefined reference error under cx_Freeze, setup, and Executable in the following line:
from cx_Freeze import setup, Executable
I have tried installing cx_Freeze through the GUI on pyCharm but it fails. Here is my interpreter information on pyCharm:
Python 3.9 located at /opt/anaconda3/envs/LADS/bin/python
Any insight would be much appreciated. Thanks.

Ansible is not upgrading even when it says successful

I ran the following to upgrade Ansible (I'm on 2.8.4):
s-MacBook-Pro:~ albertski$ sudo pip install ansible --upgrade
Password:
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
WARNING: The directory '/Users/albertski/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
WARNING: The directory '/Users/albertski/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting ansible
Downloading https://files.pythonhosted.org/packages/04/25/48fee5f8048360d9375e01846fcf395dda58242ed1f25a2106b6794452eb/ansible-2.8.5.tar.gz (14.4MB)
|████████████████████████████████| 14.4MB 14.7MB/s
Requirement already satisfied, skipping upgrade: jinja2 in /Library/Python/2.7/site-packages (from ansible) (2.10)
Requirement already satisfied, skipping upgrade: PyYAML in /Library/Python/2.7/site-packages (from ansible) (3.13)
Requirement already satisfied, skipping upgrade: cryptography in /Library/Python/2.7/site-packages (from ansible) (2.4.2)
Requirement already satisfied, skipping upgrade: MarkupSafe>=0.23 in /Library/Python/2.7/site-packages (from jinja2->ansible) (1.1.0)
Requirement already satisfied, skipping upgrade: six>=1.4.1 in /Library/Python/2.7/site-packages (from cryptography->ansible) (1.11.0)
Requirement already satisfied, skipping upgrade: cffi!=1.11.3,>=1.7 in /Library/Python/2.7/site-packages (from cryptography->ansible) (1.11.5)
Requirement already satisfied, skipping upgrade: enum34; python_version < "3" in /Library/Python/2.7/site-packages (from cryptography->ansible) (1.1.6)
Requirement already satisfied, skipping upgrade: idna>=2.1 in /Library/Python/2.7/site-packages (from cryptography->ansible) (2.7)
Requirement already satisfied, skipping upgrade: asn1crypto>=0.21.0 in /Library/Python/2.7/site-packages (from cryptography->ansible) (0.24.0)
Requirement already satisfied, skipping upgrade: ipaddress; python_version < "3" in /Library/Python/2.7/site-packages (from cryptography->ansible) (1.0.22)
Requirement already satisfied, skipping upgrade: pycparser in /Library/Python/2.7/site-packages (from cffi!=1.11.3,>=1.7->cryptography->ansible) (2.19)
Installing collected packages: ansible
Found existing installation: ansible 2.7.13
Uninstalling ansible-2.7.13:
Successfully uninstalled ansible-2.7.13
Running setup.py install for ansible ... done
Successfully installed ansible-2.8.5
It shows at the end Successfully installed ansible-2.8.5.
When I check the version it still shows 2.8.4.
Alberts-MacBook-Pro:~ albertski$ ansible --version
ansible 2.8.4
config file = /Users/albertski/.ansible.cfg
configured module search path = ['/Users/albertski/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/ansible
executable location = /Library/Frameworks/Python.framework/Versions/3.7/bin/ansible
python version = 3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018, 03:13:28) [Clang 6.0 (clang-600.0.57)]
Edit
Tried to downgrade to 2.7.13 and I ran into the same issue:
sudo pip install ansible==2.7.13
Password:
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
WARNING: The directory '/Users/albertski/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
WARNING: The directory '/Users/albertski/Library/Caches/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting ansible==2.7.13
Downloading https://files.pythonhosted.org/packages/71/25/90546592a692fcb8b3e6e578e743dcba19f1ecdb2ef47860dc5bf0a828aa/ansible-2.7.13.tar.gz (11.9MB)
|████████████████████████████████| 11.9MB 7.7MB/s
Requirement already satisfied: jinja2 in /Library/Python/2.7/site-packages (from ansible==2.7.13) (2.10)
Requirement already satisfied: PyYAML in /Library/Python/2.7/site-packages (from ansible==2.7.13) (3.13)
Requirement already satisfied: paramiko in /Library/Python/2.7/site-packages (from ansible==2.7.13) (2.4.2)
Requirement already satisfied: cryptography in /Library/Python/2.7/site-packages (from ansible==2.7.13) (2.4.2)
Requirement already satisfied: setuptools in /Library/Python/2.7/site-packages (from ansible==2.7.13) (40.6.2)
Requirement already satisfied: MarkupSafe>=0.23 in /Library/Python/2.7/site-packages (from jinja2->ansible==2.7.13) (1.1.0)
Requirement already satisfied: pyasn1>=0.1.7 in /Library/Python/2.7/site-packages (from paramiko->ansible==2.7.13) (0.4.4)
Requirement already satisfied: bcrypt>=3.1.3 in /Library/Python/2.7/site-packages (from paramiko->ansible==2.7.13) (3.1.4)
Requirement already satisfied: pynacl>=1.0.1 in /Library/Python/2.7/site-packages (from paramiko->ansible==2.7.13) (1.3.0)
Requirement already satisfied: six>=1.4.1 in /Library/Python/2.7/site-packages (from cryptography->ansible==2.7.13) (1.11.0)
Requirement already satisfied: cffi!=1.11.3,>=1.7 in /Library/Python/2.7/site-packages (from cryptography->ansible==2.7.13) (1.11.5)
Requirement already satisfied: enum34; python_version < "3" in /Library/Python/2.7/site-packages (from cryptography->ansible==2.7.13) (1.1.6)
Requirement already satisfied: idna>=2.1 in /Library/Python/2.7/site-packages (from cryptography->ansible==2.7.13) (2.7)
Requirement already satisfied: asn1crypto>=0.21.0 in /Library/Python/2.7/site-packages (from cryptography->ansible==2.7.13) (0.24.0)
Requirement already satisfied: ipaddress; python_version < "3" in /Library/Python/2.7/site-packages (from cryptography->ansible==2.7.13) (1.0.22)
Requirement already satisfied: pycparser in /Library/Python/2.7/site-packages (from cffi!=1.11.3,>=1.7->cryptography->ansible==2.7.13) (2.19)
Installing collected packages: ansible
Found existing installation: ansible 2.8.5
Uninstalling ansible-2.8.5:
Successfully uninstalled ansible-2.8.5
Running setup.py install for ansible ... done
Successfully installed ansible-2.7.13
Alberts-MacBook-Pro:ansible albertski$ ansible --version
ansible 2.8.4
config file = /Users/albertski/Sites/ansible/ansible.cfg
configured module search path = ['/Users/albertski/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/ansible
executable location = /Library/Frameworks/Python.framework/Versions/3.7/bin/ansible
python version = 3.7.1 (v3.7.1:260ec2c36a, Oct 20 2018, 03:13:28) [Clang 6.0 (clang-600.0.57)]
Operating System: OSX Mojave
You're installing Ansible with python2 pip and Ansible in your shell's path appears to be using python3.
Try the following:
sudo pip3 install ansible --upgrade
or better yet, keep it in your user homedir
pip3 install --user ansible --upgrade

Unable to install scikit-image

I'm working on a Mac with python 2.7.
Whenever i'm trying to install scikit-image
Command used to install : sudo pip install scikit-image, i'm getting the below error :
Requirement already satisfied: python-dateutil in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python (from matplotlib>=1.3.1->scikit-image) (1.5)
Requirement already satisfied: tornado in /Library/Python/2.7/site-packages (from matplotlib>=1.3.1->scikit-image) (5.0.2)
Requirement already satisfied: pyparsing>=1.5.6 in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python (from matplotlib>=1.3.1->scikit-image) (2.0.1)
Requirement already satisfied: nose in /Library/Python/2.7/site-packages (from matplotlib>=1.3.1->scikit-image) (1.3.7)
Requirement already satisfied: futures in /Library/Python/2.7/site-packages (from tornado->matplotlib>=1.3.1->scikit-image) (3.2.0)
Requirement already satisfied: singledispatch in /Library/Python/2.7/site-packages (from tornado->matplotlib>=1.3.1->scikit-image) (3.4.0.3)
Requirement already satisfied: backports_abc>=0.4 in /Library/Python/2.7/site-packages (from tornado->matplotlib>=1.3.1->scikit-image) (0.5)
Installing collected packages: numpy, PyWavelets, scipy, six, decorator, networkx, scikit-image
Found existing installation: numpy 1.8.0rc1
Cannot uninstall 'numpy'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.

-bash: pylint: command not found

I have been trying to install pylint to be used on terminal, but have been unsuccessful in using it. The installation gets successful, but whenever I try to run pylint command, it returns the following error -
-bash: pylint: command not found
I have tried using following commands -
pip install pylint
python -m pip install pylint
sudo pip install pylint
sudo -H pip install pylint
I have uninstalled it, before trying each of the above command, but everytime I install it, the installation gets successful, but I am unable to use it on command line. I know it's something silly I am missing.
Here's the output I get on installation -
My-Mac:Dev noob$ sudo -H pip install pylint
Collecting pylint
Using cached pylint-1.6.5-py2.py3-none-any.whl
Requirement already satisfied: isort>=4.2.5 in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from pylint)
Requirement already satisfied: six in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from pylint)
Requirement already satisfied: mccabe in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from pylint)
Requirement already satisfied: configparser; python_version == "2.7" in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from pylint)
Requirement already satisfied: backports.functools-lru-cache; python_version == "2.7" in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from pylint)
Requirement already satisfied: astroid<1.5.0,>=1.4.5 in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from pylint)
Requirement already satisfied: wrapt in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from astroid<1.5.0,>=1.4.5->pylint)
Requirement already satisfied: lazy-object-proxy in /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages (from astroid<1.5.0,>=1.4.5->pylint)
Installing collected packages: pylint
Successfully installed pylint-1.6.5
PS: I am using macOS Sierra 10.12.3 on a Macbook pro Late 2011, in case this is relevant.
The issue is installed python executables aren't being added to your path. Our paths don't look exactly the same (probably because different python versions, I'm not sure) but mine is
host:~ williamhammond$ pip install pylint
Requirement already satisfied: pylint in /Users/williamhammond/Library/Python/3.7/lib/python/site-packages (2.3.1)
Requirement already satisfied: astroid<3,>=2.2.0 in /Users/williamhammond/Library/Python/3.7/lib/python/site-packages (from pylint) (2.2.5)
Requirement already satisfied: isort<5,>=4.2.5 in /Users/williamhammond/Library/Python/3.7/lib/python/site-packages (from pylint) (4.3.21)
Requirement already satisfied: mccabe<0.7,>=0.6 in /Users/williamhammond/Library/Python/3.7/lib/python/site-packages (from pylint) (0.6.1)
Requirement already satisfied: wrapt in /Users/williamhammond/Library/Python/3.7/lib/python/site-packages (from astroid<3,>=2.2.0->pylint) (1.11.2)
Requirement already satisfied: six in /Users/williamhammond/Library/Python/3.7/lib/python/site-packages (from astroid<3,>=2.2.0->pylint) (1.12.0)
Requirement already satisfied: typed-ast>=1.3.0; implementation_name == "cpython" in /Users/williamhammond/Library/Python/3.7/lib/python/site-packages (from astroid<3,>=2.2.0->pylint) (1.4.0)
Requirement already satisfied: lazy-object-proxy in /Users/williamhammond/Library/Python/3.7/lib/python/site-packages (from astroid<3,>=2.2.0->pylint) (1.4.1)
I was seeing the same issues as you until I looked here
host:~ williamhammond$ ls /Users/williamhammond/Library/Python/3.7/bin/
dmypy epylint isort mypy pep8 pylint pyreverse stubgen symilar
Once you add this to your path like
export PATH=$PATH:/Users/williamhammond/Library/Python/3.7/bin/
you should be able to use pylint. To make this change permanent make sure to add the command to your ~/.bashrc
For the very last comment in the response given by William Hammond on making this change permanent make sure to add the command to your ~/.bashrc, you'd find the hyperlink above a useful guide to get this done:

OSX El Capitan: sudo pip install OSError: [Errno: 1] Operation not permitted

When I run:
sudo pip install ipython
I get the following error
OSError: [Errno: 1] Operation not permitted: '/System/Library/Frameworks/Python.framework/Versions/2.7/share'
The last command executed tries to create the directory given above.
Also, the following command fails to install iPython without providing any errors.
sudo pip install --user python
(I am on Mac OS X El Capitan in case other folks on this OS see the same issue.)
Instructions telling people to use sudo pip install are inherently wrong.
If there is any tutorial out there which says you should use sudo pip then please file a bug against this package. The author is dis-educating the Python community, as time has proven sudo pip to be a broken practice.
OSX El Capitan introduced mechanisms to prevent damaging the operating system files. /System/Library/Frameworks/Python.framework/Versions/2.7/share is one of the protected locations. A normal user has no reason to put or write any files there. This is because the operating system itself relies on these files and sudo pip, with all force given from the above, would unconditionally overwrite them. Usually bad things would not happen, but the chances are there. Apple wants to protect their OS users from accidentally bricking their installation.
Instead, you need to install a Python package, like IPython, locally to the home folder of your user. The easiest way is to create a virtual environment, activate it, and then run pip in the virtual environment.
Example:
cd ~ # Go to home directory
virtualenv my-venv
source my-venv/bin/activate
pip install IPython
More info
Official Python package installation tutorial.
How to create virtual environments.
Alternatively, one should be able to use pip install --user. But again, sudo is not needed and you need to manually set up PATH environment variable.
I had the same problems, but using the easy_install "module" solved the problem for me.
I am not sure why, but pip and easy_install use different install locations, and easy_install chose the right ones.
Edit: without re-checking but because of the comments; it seems that different (OSX and brew-installed) installations interfere with each other which is why the tools mentioned point to different locations (since they belong to different installations). I understand that usually those tools from one install point to the same folder.
You should reinstall Python:
brew reinstall python
To get brew see the brew homepage.
pip install --ignore-installed six
This will do the job, then you can try your first command.
Via http://github.com/pypa/pip/issues/3165
Used pip3 install <package> instead and solved the permission problem with pip.
TL;DR $PATH fix
Use pip install --user package_name to install a package that should include CLI executables.
Launch a python shell and import package_name
Find where lib/python/... occurs in the output and replace it all with bin
It's likely to be $HOME/Library/Python/2.7/bin
Details
Because of the new System Integrity Protection in macOS 10.11 El Capitan, you can no longer sudo pip install. We won't debate the merits of that here.
Another answer explains that you should pip install --user which is correct. But they sent you to the back alleys to figure out what to do about your $PATH so that you could get access to installed executables. Luckily, I've already solved a similar need for an unrelated question.
Here is a transcript of how I solved the problem on one of my systems. I'm including it all rather just than the $PATH that worked for me, because your system may be different from mine. This process should work for everybody.
$ pip install --user jp
Collecting jp
Downloading jp-0.2.4.tar.gz
Installing collected packages: jp
Running setup.py install for jp ... done
Successfully installed jp-0.2.4
$ python -c 'import jp; print jp'
<module 'jp' from '/Users/bbronosky/Library/Python/2.7/lib/python/site-packages/jp/__init__.pyc'>
$ find /Users/bbronosky/Library/Python -type f -perm -100
/Users/bbronosky/Library/Python/2.7/bin/jp
$ which jp
$ echo -e '\n''export PATH=$HOME/Library/Python/2.7/bin:$PATH' >> ~/.bashrc
$ bash # starting a new bash process for demo, but you should open a new terminal
$ which jp
/Users/bbronosky/Library/Python/2.7/bin/jp
$ jp
usage: jp <expression> <filepath>
Same error
Installing collected packages: six, pyparsing, packaging, appdirs, setuptools
Exception:
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/commands/install.py", line 342, in run
prefix=options.prefix_path,
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_set.py", line 784, in install
**kwargs
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_install.py", line 851, in install
self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/req/req_install.py", line 1064, in move_wheel_files
isolated=self.isolated,
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/wheel.py", line 345, in move_wheel_files
clobber(source, lib_dir, True)
File "/Library/Python/2.7/site-packages/pip-9.0.1-py2.7.egg/pip/wheel.py", line 323, in clobber
shutil.copyfile(srcfile, destfile)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 83, in copyfile
with open(dst, 'wb') as fdst:
IOError: [Errno 13] Permission denied: '/Library/Python/2.7/site-packages/six.py'
and here I use --user without sudo to solve this issue
$ pip install --user scikit-image h5py keras pygame
Collecting scikit-image
Downloading http://mirrors.aliyun.com/pypi/packages/65/69/27a1d55ce8f77c8ac757938707105b1070ff4f2ae47d2dc99461bfae4491/scikit_image-0.13.0-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl (28.1MB)
100% |████████████████████████████████| 28.1MB 380kB/s
Collecting h5py
Downloading http://mirrors.aliyun.com/pypi/packages/b7/cc/1c29b0815b12de2c92b5323cad60f724ac8f0e39d0166d0b9dfacbcb70dd/h5py-2.7.0-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl (4.5MB)
100% |████████████████████████████████| 4.5MB 503kB/s
Requirement already satisfied: keras in /Library/Python/2.7/site-packages
Requirement already satisfied: pygame in /Library/Python/2.7/site-packages
Requirement already satisfied: matplotlib>=1.3.1 in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python (from scikit-image)
Requirement already satisfied: six>=1.7.3 in /Library/Python/2.7/site-packages (from scikit-image)
Requirement already satisfied: pillow>=2.1.0 in /Library/Python/2.7/site-packages (from scikit-image)
Requirement already satisfied: networkx>=1.8 in /Library/Python/2.7/site-packages (from scikit-image)
Requirement already satisfied: PyWavelets>=0.4.0 in /Library/Python/2.7/site-packages (from scikit-image)
Collecting scipy>=0.17.0 (from scikit-image)
Downloading http://mirrors.aliyun.com/pypi/packages/72/eb/d398b9f63ee936575edc62520477d6c2353ed013bacd656bd0c8bc1d0fa7/scipy-0.19.0-cp27-cp27m-macosx_10_6_intel.macosx_10_9_intel.macosx_10_9_x86_64.macosx_10_10_intel.macosx_10_10_x86_64.whl (16.2MB)
100% |████████████████████████████████| 16.2MB 990kB/s
Requirement already satisfied: numpy>=1.7 in /Library/Python/2.7/site-packages (from h5py)
Requirement already satisfied: theano in /Library/Python/2.7/site-packages (from keras)
Requirement already satisfied: pyyaml in /Library/Python/2.7/site-packages (from keras)
Requirement already satisfied: python-dateutil in /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python (from matplotlib>=1.3.1->scikit-image)
Requirement already satisfied: tornado in /Library/Python/2.7/site-packages (from matplotlib>=1.3.1->scikit-image)
Requirement already satisfied: pyparsing>=1.5.6 in /Users/qiuwei/Library/Python/2.7/lib/python/site-packages (from matplotlib>=1.3.1->scikit-image)
Requirement already satisfied: nose in /Library/Python/2.7/site-packages (from matplotlib>=1.3.1->scikit-image)
Requirement already satisfied: olefile in /Library/Python/2.7/site-packages (from pillow>=2.1.0->scikit-image)
Requirement already satisfied: decorator>=3.4.0 in /Library/Python/2.7/site-packages (from networkx>=1.8->scikit-image)
Requirement already satisfied: singledispatch in /Library/Python/2.7/site-packages (from tornado->matplotlib>=1.3.1->scikit-image)
Requirement already satisfied: certifi in /Library/Python/2.7/site-packages (from tornado->matplotlib>=1.3.1->scikit-image)
Requirement already satisfied: backports_abc>=0.4 in /Library/Python/2.7/site-packages (from tornado->matplotlib>=1.3.1->scikit-image)
Installing collected packages: scipy, scikit-image, h5py
Successfully installed h5py-2.7.0 scikit-image-0.13.0 scipy-0.19.0
Hope it will help someone who encounter similar issue!
I had the same issues.
As others have mentioned, don't run pip install with sudo.
Run
brew doctor
and fix the warnings and you should be able to proceed with your pip install.
It is hard to get pip working on El Capitan for several reasons:
OS X doesn't set some distutils variables correctly, so pip tries to install ancillary files in locations under /System/Library/. El Capitan blocks this, which is the error you are running into.
OS X includes a number of outdated packages under /System/Library/. pip often wants to upgrade these but cannot on El Capitan.
OS X places /System/Library/ higher in the python search order than /Library/Python/2.7/site-packages (the system-wide python package location), so even if you manage to install newer versions of some packages, the old ones still get loaded, breaking some dependencies.
There are workarounds for all of these at https://apple.stackexchange.com/a/223163/143849 . But you may be best off installing your own version of Python via the standard Python installer, Homebrew or Anaconda.
I guess you have some conflict with other package. For me it was six. So you need to use a command like this:
pip install google-api-python-client --upgrade --ignore-installed six
or
pip install --ignore-installed six
I fully agree with Mikko, but if you still want to do it, here is the way:
Restart in recovery mode (Hold cmd + R)
Open terminal from utilities
Use the command csrutil disable
I have python2.7 installed via brew and the following solved my problem
brew install numpy
It installs python3, but it still works and sets it up for 2.7 as well.

Resources