I have recently updated to OSX El Capitain upgrading an exhisting working osx installation. Now I have a module, installed via pip on an anaconda distribution which is now broken. Or better the module can be correctly imported from python
Python 2.7.10 |Anaconda 2.3.0 (x86_64)| (default, Sep 15 2015, 14:29:08)
[GCC 4.2.1 (Apple Inc. build 5577)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import MDSplus as mds
whereas it does not load on ipython, claiming not to finding a library.
Python 2.7.10 |Anaconda 2.3.0 (x86_64)| (default, Sep 15 2015, 14:29:08)
Type "copyright", "credits" or "license" for more information.
IPython 4.0.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
Using matplotlib backend: MacOSX
In [1]: import MDSplus as mds
Error importing MDSplus package: Error finding library: MdsShr
The library is correctly installed, it is listed in /etc/profile, it is located in /usr/local (so there should not be a problem with the new security system of OSX el Captain) and the python interpreted which is called via python or ipython is the same (Anaconda 2.3.0).
How can I diagnose which is the problem?
EDIT:
the solution proposed in https://groups.google.com/a/continuum.io/forum/#!topic/anaconda/PEuOKEM5fdc does not worked for me. It does not seem to be an environment problem and the command
echo $DYLD_LIBRARY_PATH
points to the correct path
EDIT:
which -a ipython
ipython is /Users/vianello/anaconda/bin/ipython
which -a python
python is /Users/vianello/anaconda/bin/python
python is /usr/bin/python
>>> import sys
>>> for x in sys.path: print x
>>>/Users/vianello/anaconda/lib/python2.7/site-packages/pyhht-0.0.1-py2.7.egg
>>>/Users/vianello/anaconda/lib/python2.7/site-packages/MitDevices-0.3-py2.7.egg
>>>/Users/vianello/anaconda/lib/python2.7/site-packages/MDSplus-alpha_7.0.157-py2.7.egg
>>>//anaconda/lib/python2.7/site-packages/MDSplus-alpha_7.0.147-py2.7.egg/MDSplus
>>>/Users/vianello/anaconda/lib/python27.zip
>>>/Users/vianello/anaconda/lib/python2.7
>>>/Users/vianello/anaconda/lib/python2.7/plat-darwin
>>>/Users/vianello/anaconda/lib/python2.7/plat-mac
>>>/Users/vianello/anaconda/lib/python2.7/plat-mac/lib-scriptpackages
>>>/Users/vianello/anaconda/lib/python2.7/lib-tk
>>>/Users/vianello/anaconda/lib/python2.7/lib-old
>>>/Users/vianello/anaconda/lib/python2.7/lib-dynload
>>>/Users/vianello/anaconda/lib/python2.7/site-packages
>>>/Users/vianello/anaconda/lib/python2.7/site-packages/PIL
>>>/Users/vianello/anaconda/lib/python2.7/site-packages/Sphinx-1.3.1-py2.7.egg
>>>/Users/vianello/anaconda/lib/python2.7/site-packages/aeosa
>>>/Users/vianello/anaconda/lib/python2.7/site-packages/lmfit-0.8.3-py2.7.egg
>>>/Users/vianello/anaconda/lib/python2.7/site-packages/setuptools-18.3.2-py2.7.egg
>>>/Users/vianello/anaconda/lib/python2.7/site-packages/suds-0.4-py2.7.egg
in iPython the same give the results
/Users/vianello/anaconda/bin
/Users/vianello/anaconda/lib/python2.7/site-packages/pyhht-0.0.1-py2.7.egg
/Users/vianello/anaconda/lib/python2.7/site-packages/MitDevices-0.3-py2.7.egg
/Users/vianello/anaconda/lib/python2.7/site-packages/MDSplus-alpha_7.0.157-py2.7.egg
/Users/vianello/anaconda/lib/python27.zip
/Users/vianello/anaconda/lib/python2.7
/Users/vianello/anaconda/lib/python2.7/plat-darwin
/Users/vianello/anaconda/lib/python2.7/plat-mac
/Users/vianello/anaconda/lib/python2.7/plat-mac/lib-scriptpackages
/Users/vianello/anaconda/lib/python2.7/lib-tk
/Users/vianello/anaconda/lib/python2.7/lib-old
/Users/vianello/anaconda/lib/python2.7/lib-dynload
/Users/vianello/anaconda/lib/python2.7/site-packages/Sphinx-1.3.1-py2.7.egg
/Users/vianello/anaconda/lib/python2.7/site-packages/lmfit-0.8.3-py2.7.egg
/Users/vianello/anaconda/lib/python2.7/site-packages/setuptools-18.3.2-py2.7.egg
/Users/vianello/anaconda/lib/python2.7/site-packages/suds-0.4-py2.7.egg
/Users/vianello/anaconda/lib/python2.7/site-packages
/Users/vianello/anaconda/lib/python2.7/site-packages/PIL
/Users/vianello/anaconda/lib/python2.7/site-packages/aeosa
/Users/vianello/anaconda/lib/python2.7/site-packages/IPython/extensions
/Users/vianello/.ipython
Thus the MDSplus-alpha_7.0.157-py2.7.egg can be seen both from python and ipython
My first thought is that you are running afoul of the new Apple System Integrity Protection. iPython is launched using a shell script that loads via /bin/bash. That will result in your DYLD_LIBRARY_PATH environment variable being stripped when python launches. You can check this by importing os and then attempting to print the contents of os.environ["DYLD_LIBRARY_PATH"]. If that print throws an exception then you know what the problem is.
A simple fix that works for me is to edit the first line of the ipython script so that the #! calls your python binary directly rather than going via bash. This is not a long term solution as it will have to be redone each time ipython is updated until upstream changes (to be fair I'm not sure why bash is involved).
I have written a report on python and library paths at http://dmtn-001.lsst.io
Related
I have installed python 3.8.5 and in ipython 7.19.0 via Anaconda installer on Windows 10 machine and I get the following error in Ipython when I import keras and then run ls or any other commands.
It happens when I import other commands or run other commands. Can anybody propose any solution? Is this to do with latest python or Ipython? Or something else.
C:\Users\AA>ipython
Python 3.8.5 (default, Sep 3 2020, 21:29:08) [MSC v.1916 64 bit (AMD64)]
Type 'copyright', 'credits' or 'license' for more information
IPython 7.19.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: from keras.models import load_model
2021-01-06 10:36:43.729592: I tensorflow/stream_executor/platform/default/dso_loader.cc:48] Successfully opened dynamic library cudart64_101.dll
In [2]: ls
Unhandled exception in event loop:
File "C:\ProgramData\Anaconda3\lib\asyncio\proactor_events.py", line 768, in _loop_self_reading
f.result() # may raise
File "C:\ProgramData\Anaconda3\lib\asyncio\windows_events.py", line 808, in _poll
value = callback(transferred, key, ov)
File "C:\ProgramData\Anaconda3\lib\asyncio\windows_events.py", line 457, in finish_recv
raise ConnectionResetError(*exc.args)
Exception [WinError 995] The I/O operation has been aborted because of either a thread exit or an application request
Press ENTER to continue...
It seems to be a bug in the ipython prompt toolkit (version >= 3)
Try doing this for your environment:
conda install prompt-toolkit~=2.0
Generally we can fix this by doing something like this:
py -m pip install -U prompt-toolkit~=2.0
If that does not seem to work, there is an extensive discussion of the bug here with a few other solutions: https://github.com/prompt-toolkit/python-prompt-toolkit/issues/1023#issue-534396318
(Discussed in this issue: https://github.com/jquast/blessed/issues/121)
According to this issue: https://github.com/prompt-toolkit/python-prompt-toolkit/issues/1023
Upgrading python to at least 3.8.7 should fix this.
I'm trying to install a module from withing a python script using pip. Here is the contents of script.py:
#/usr/bin/python2.7
# I'm the file called `script.py`
import sys, importlib, pip
print(sys.version); print(sys.path) # For debugging
try:
importlib.import_module('docopt')
except ImportError:
pip.main(['install', '-U', 'docopt'])
finally:
globals()[pack] = importlib.import_module('docopt')
Runnting this script, e.g. using python2.7 script.py gives me:
$ python2.7 script.py
2.7.9 (default, Oct 3 2016, 17:42:24)
[GCC 4.3.2 20081105 (Red Hat 4.3.2-7)]
['/usr/lib/python27.zip', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-linux2', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/lib/python2.7/site-packages']
No module named docopt
Downloading/unpacking docopt
Cannot fetch index base URL https://pypi.python.org/simple/
Could not find any downloads that satisfy the requirement docopt
Cleaning up...
No distributions at all found for docopt
Storing debug log for failure in /root/.pip/pip.log
Traceback (most recent call last):
[...]
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
ImportError: No module named docopt
So problems with internet connection/ssl or something transport related. However, the same commands work perfectly fine in the python interactive interpreter, when I enter them by hand (copy&paste them):
$ python2.7
Python 2.7.9 (default, Oct 3 2016, 17:42:24)
[GCC 4.3.2 20081105 (Red Hat 4.3.2-7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pip, sys
>>> print(sys.path)
['/usr/lib/python27.zip', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-linux2', '/usr/lib/python2.7/lib-tk', '/usr/lib/python2.7/lib-old', '/usr/lib/python2.7/lib-dynload', '/usr/lib/python2.7/site-packages']
>>> pip.main(['install', '-U', 'docopt'])
Downloading/unpacking docopt
Downloading docopt-0.6.2.tar.gz
Running setup.py (path:/tmp/pip_build_root/docopt/setup.py) egg_info for package docopt
Installing collected packages: docopt
Running setup.py install for docopt
Successfully installed docopt
Cleaning up...
0
Again, the thing that weirds me out is, that it works interactively i.e. running python2.7 in the shell and then entering the very same code by hand. However, not when I run the script file with python2.7 script.py. So In one case, on the same machine, the same interpreter has internet access in the other case it does not.
I'm running out of troubleshooting ideas. I used the same machine and user to get above results. There is no python-startup file, so that is not making the commands magically work interactively. The contents of sys.path are the same in both cases. I'm not behind a proxy. Any ideas what could be missing in the script?
Cannot fetch index base URL https://pypi.python.org/simple/
indicates that for some reason your python interpreter cannot access to internet. There can be plenty of reasons for that. Do you use a HTTP proxy ?
How do I make all modules be available for both python 2.7 and 3.4?
I executed:
$ brew install <prerequisites for pygame>
<snip snip>
<snip snip>
$ brew install pygame
$ pip freeze
<snip snip>
pygame==1.9.2a0
<snip snip>
and can see pygame is installed. When I open a new shell/terminal and run python 2.7 and type import pygame, I have no problems:
Python 2.7.9 |Anaconda 2.2.0 (x86_64)| (default, Dec 15 2014, 10:37:34)
[GCC 4.2.1 (Apple Inc. build 5577)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://binstar.org
>>> import pygame
>>>
However, when I run python 3.4 and type import pygame, it says there's no module named pygame.
/usr/local/bin/python3
Python 3.4.2 (default, Dec 11 2014, 17:48:01)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.56)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pygame
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ImportError: No module named 'pygame'
>>>
What do I need to do to make 3.4 see/link with all the modules that 2.7 sees?
P.S. I don't know how but pip is running from anaconda, which I think I might have installed when installing numpy or panda or something.
You cannot use modules installed for Python 2 in Python 3. Python 2 and Python 3 are not compatible, that is why most libraries offer a Python 2 and a Python 3 version.
So if you have PyGame installed with pip2, you can only use it with Python 2. To use PyGame in Python 3, you need to install it with pip3. Afterwards you will have both versions on your computer, so that a project that uses your Python 2 interpreter uses the PyGame installed by pip2 and a project that uses Python 3 interpreter uses the pyGame installed by pip3.
Example:
# test.py
import pygame
In the console:
> python3 test.py # imports pygame from /usr/lib/Python3.x/site-packages
> python2 test.py # imports pygame from /usr/lib/Python2.x/site-packages
You should not use Python 2 modules with Python 3 (and vice versa). Instead, install those separately, e.g.
pip3 install pygame
Sometimes, the code bases for a package will be compatible between 2 and 3, but quite often, they are not (especially when shared object modules, *.so files, come into play).
Also, Python 2 will not have the Python 3 packages library in sys.path by default, and vice versa. For a good reason.
All in all, consider Python 2 and Python 3 two different languages, for which you have to do everything separately. They just happen to be very much alike.
(Personal opinion/bit of advocacy: use and write your code in Python 3, unless you have a very good reason no to (unported essential libraries, chance of losing your job). It's relatively straightforward in 99% of the cases to make it Python 2 compatible later on, if really needed.)
I'm having trouble listing available serial ports and I really need help debugging this.
In Python 2.7.5 the COM-ports are listed correctly while PySerial returns an empty list in Python 3.3.5.
I found one other lonely soul with the same problems on the internet (no answers), but the problem doesn't seem to be popular at all - maybe it's my system?
I'm using Mac OS X 10.9.2 and installed python and python3 via homebrew. I updated everything just now. PySerial is at version 2.7 in both pip and pip3.
The output:
Python 2.7.5 (default, Nov 4 2013, 18:04:45)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.2.79)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from serial.tools import list_ports
>>> list_ports.comports()
[['/dev/cu.Bluetooth-Incoming-Port', 'n/a', 'n/a'], ['/dev/cu.Bluetooth-Modem', 'n/a', 'n/a']]
Python 3.3.5 (default, Mar 10 2014, 13:25:50)
[GCC 4.2.1 Compatible Apple LLVM 5.0 (clang-500.2.79)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from serial.tools import list_ports
>>> list_ports.comports()
[]
For the moment I'm resorting to the following method. The PySerial tools are broken.
>>> import glob
>>> glob.glob('/dev/tty.*')
['/dev/tty.Bluetooth-Incoming-Port', '/dev/tty.Bluetooth-Modem']
The root cause of the problem stems from the fact that python3 encodes strings as unicode (wide) strings, and python2 encodes strings as narrow strings.
So the pyserial code needs to pass down narrow strings when calling the API functions from iokit.
I discovered that somebody else also ran into this and posted a patch. You can find his patch at the end of this issue:
http://sourceforge.net/p/pyserial/patches/38/
Using that patch, I now get the same behaviour from python3 as I do from python2.
I had installed python3 using macports, and my installation of pyserial for python3 was found in this directory:
/opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/serial
so I executed the following:
cd /opt/local/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/site-packages/serial/tools
sudo cp list_ports_osx.py list_ports_osx_orig.py
sudo curl -O http://sourceforge.net/p/pyserial/patches/_discuss/thread/603bd426/55a8/attachment/list_ports_osx.py
and that made list_ports.comports() work for me.
What EXACTLY do I have to do to get numpy to work? I've read that it's supported in 3.2.3 and that it should work by using setup.py. I'm getting errors talking about os_path. Is there something I'm missing?
Here is the full traceback:
Python 3.2.3 (v3.2.3:3d0686d90f55, Apr 10 2012, 11:25:50)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "copyright", "credits" or "license()" for more information.
>>> ================================ RESTART ================================
>>>
Converting to Python3 via 2to3...
Traceback (most recent call last):
File "/Users/cameron/Downloads/numpy-master/setup.py", line 214, in <module>
setup_package()
File "/Users/cameron/Downloads/numpy-master/setup.py", line 175, in setup_package
__file__ = os.path.join(os.curdir, os.path.basename(__file__))
NameError: global name '__file__' is not defined
>>>
You are probably using the Python version pre-installed with Mac OS X. I strongly suggest you use a distribution where all the components (python, scipy, numpy, etc) are built to work together properly, like macports. Macports is a package manager for Open Source software, and it automatically resolves and installs dependencies.
After the installation of Macports, the only thing you need to do is type "sudo port install py27-numpy" or "sudo port install py33-numpy" at the command prompt, depending on whether you prefer to use Python 2 or 3 (some packages might not run on Python3 yet). This will automatically download and install the correct version of python and all other software packages that might be required for numpy to run properly.
Try to compile a Python from scratch first (by downloading the source) and not using the one shipped by Apple. See the recommendation on the related SciPy/Numpy page.