I just read the Cython Pure Python Mode documentation and I'm not sure if I understand it right. It sounds as if I could keep all my Python files as they are, add *.pxd files where I declare Cython types. In the setup.py, I still add
from setuptools import setup
from Cython.Build import cythonize
setup(
ext_modules = cythonize(
"A.py",
compiler_directives={'language_level' : "3"}
)
)
When I run python setup.py build_ext --inplace it actually builds the .so file.
What happens when I create the sdist / bdist, upload them to PyPI and a user does not have a matching platform? They will download the sdist, sure. I guess pip / setuptools will automatically try to compile the extension modules (A.py) and I guess if that works, it is fine. But what if cythonize fails? Will it still install the package and use the pure Python code?
I don't think so. I believe a failure in setup.py aborts installation completely.
You can try to declare an extension optional but there're reports that doesn't really work. Could be an issue with older setuptools.
Related
I'm trying to install/import a cython dependent module [ZigZag] into my python function on Azure functions (Linux platfrom), I'm working via Windows 8.1 (VSCode), Running Python 3.6 on both local machine and azure functions.
I tried including Cython in the requirements.txt file, this errors out during deployment (seems installation of external modules happens somewhat in parallel)
I tried creating a '.toml' file, no luck.
I also tried the idea found here Python google cloud function deployment failure - Madmom pip package still didn't work.
Finally I decided to deploy the function first (with cython included in the 'requirements.txt' file) without the zigzag module, and then in my python script use subprocess.Popen() to do a pip install zigzag and import it before the rest of the function runs.
This is where I get stuck. In my python file I have these lines included to do the above:
#function to be called in azure __init__.py file
def f....:
# check for zigzag
try:
import zigzag
except ImportError:
subprocess.Popen(["pip", "install", "ZigZag"], shell=True)
finally:
import zigzag
.
.
.
rest of the function...
This deploys successfully to azure functions, however it fails during execution with the error:
Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcException :
Result: Failure Exception: ModuleNotFoundError: No module named 'zigzag
Any idea's how to do this correctly?
NB: This issue occurs on google cloud functions as well.
I see this is related to https://learn.microsoft.com/en-us/answers/questions/152557/index.html
As per the thread, the latest version for ZigZag is 0.2.2 and the supported python version is 3.5: https://pypi.org/project/ZigZag/
The azure function supports 3.6 and above python version which will make it incompatible. You can try installing the latest cython using requirements.txt and it will work as expected. But when you try to install ZigZaG it tries to find the dependency of cython package (Cython==0.26.1) that is compatible with it therefor it throws the error that cython package not found.
When creating a Python package and uploading it to pypi, it will automatically install the requirements that are put in the setup.py file under install_requires, e.g.
from distutils.core import setup
setup(
name = 'a_package',
packages = ['a_package'],
install_requires=['another_package']
)
When the package has a cython extension (and .pyx files instead of .c/.cpp files), the setup.py file will need to import cython to create an installable extension, e.g.
from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
setup(
name = 'a_package',
packages = ['a_package'],
install_requires=['another_package'],
cmdclass = {'build_ext': build_ext},
ext_modules = [Extension('the_extension', sources=['a_file.pyx'])]
)
But since Cython is imported before executing the setup part, when trying to install this package through pip from source (rather than from a wheel) downloaded from pypi, it will fail to install due to not being able to import cython, as it has not reached the part with the requirements yet.
I’m wondering what can be done to ensure that a pip install of this package from pypi will install cython before it tries to import it. Adding a requirements.txt with cython does not seem to add automatic-install requirements for files downloaded from pypi.
Now, I realize it’s possible to just pip install cython before pip install thispackage, but I’m wondering if there’s a better fix that would allow to install the package along with cython directly from pypi when it’s not possible to run an additional command (without resorting to uploading the .c. files and ajusting the setup.py file to use them instead of the .pyx).
What you're describing is a "build time dependency", and this is precisely the use case "PEP 518 -- Specifying Minimum Build System Requirements for Python Projects" was created for.
You can specify cython as a build-time dependency by adding a pyproject.toml file like:
[build-system]
requires = ["cython"]
Then when installing your package with a modern version of pip (or another PEP 518 compatible installer), cython will be installed into the build environment before your setup.py script is run.
How do I make pip recompile Cython extensions when I have only changed setup.py when installing in editable mode. Currently it always skips the extensions. There are many questions related to this for distutils but I can't see any answers for pip.
To be clear, I have a setup.py like the following
from distutils.core import setup, Extension
from Cython.Distutils import build_ext
import numpy
ext_modules = []
# simulate_fast
ext_modules += [
Extension("adio.simulating.simulate_fast_c",
sources=["./adio/simulating/simulate_fast/simulate_fast_c.pyx",
"./adio/simulating/simulate_fast/c/simulate_fast.c",
"./adio/simulating/simulate_fast/c/matrix.c"],
include_dirs=[numpy.get_include()],
extra_compile_args=["-Ofast", "-ffast-math", "-march=native"],
language='c',
libraries=["gsl", "openblas"],
define_macros=[('FLOAT32', 1)]
)
]
setup(
name="adio",
packages=["adio"],
cmdclass={'build_ext': build_ext},
ext_modules=ext_modules
)
Now if I delete the ('FLOAT32', 1) in define_macros of course I would like the Cython extension to recompile.
However when I run
python3 -m pip install --editable -U . -v
I receive the following output as part of the output
running develop
running egg_info
writing adio.egg-info/PKG-INFO
writing dependency_links to adio.egg-info/dependency_links.txt
writing top-level names to adio.egg-info/top_level.txt
reading manifest file 'adio.egg-info/SOURCES.txt'
writing manifest file 'adio.egg-info/SOURCES.txt'
running build_ext
skipping './adio/simulating/simulate_fast/simulate_fast_c.c' Cython extension (up-to-date)
I have tried the -I and --force-reinstall flag with pip but it always skips the Cython extension. If I am not using editable mode then I can run
python3 -m pip install -U . -v
and this does recompile. How can I achieve the same thing when using the --editable flag.
Related
distutils ignores changes to setup.py when building an extension?
I have installed NLTK. Here's an image of the installation log.
When i use import nltk i get an error:
"No module named NLTK.corpus"
Here is a screenshot.
What could be the cause?
I think I had the same problem. So, downloading all the packages at once (since question didn't specify).
Start python and then import the packages, exit python and upgrade nltk. Modify the 'all' to download a specific corpus. Took me awhile to complete the 'all' download, I separately downloaded framenet_v15 and restarted the 'all' after. Upgrade nltk when the download is complete.
$ python
>>>import nltk
>>>nltk.download('all')
exit python
$ pip install --upgrade nltk
To fix this, you should rename your file to something else, say nltkXXX.py. Also make sure to remove "nltk.pyc" from your directory if it exists, since this will also be loaded (it's the byte compiled version of your code). After that, it should work fine.
If you are using the latest version of python, then try installing nltk using pip and the wheel downloaded from here:
http://www.lfd.uci.edu/~gohlke/pythonlibs/
Then in command prompt, use the command:
pip3 install
This should install nltk correctly.
After that check the installation in python using the command:
import nltk
and download the nltk data required using:
nltk.download()
If you find (Import NLTK : no module NLTK corpus) that type of error .
Make sure your saved file not be the name like (nltk.py).
so just rename your file name (like rename nltk.py to example.py ) or something else:
I hope it will help you.
thanks
If you has using PyCharm IDE, you should have install NLTK from the IDE's own tools [File -> Settings -> Projetct Interpreter -> Install (button '+') -> Install Package].
I am using Python 2.7 through Anaconda 2.7.8 and need Kapteyn 2.2 to perform Non-linear Least Squares fitting easily (it is probably an alternative to Scipy.optimize.leastsq() for dummies like me!).
After copy-pasting this from a previous post here on Stack Overflow:
conda install -c https://conda.binstar.org/dhirschfeld pyodbc
and then running on my cmd (as I did not have pyodbc installed I think, because of which maybe the command prompt on my Windows 7 64-bit system was not responding well to python setup.py install inside the Anaconda directory where I unzipped the Kapteyn .zip file downloaded from University of Groningen website.
But, after the installing pyodbc properly and running python setup.py install, the cmd gave me an error saying error: command 'C:\Users\windows 7\Anaconda\Scripts\gcc.bat' failed with exit status 1. Later, when I tried to import kmpfit module (needed for Non-linear least square fitting with Kapteyn), here is the problem:
import kapteyn
help(kapteyn)
Help on package kapteyn:
NAME
kapteyn - Kapteyn package.
FILE
c:\users\windows 7\anaconda\kapteyn\__init__.py
PACKAGE CONTENTS
_ni_support
celestial
doccer
filters
interpolation
maputils
mplutil
positions
rulers
shapes
tabarray
wcsgrat
DATA
__all__ = ['celestial', 'wcs', 'wcsgrat', 'tabarray', 'maputils', 'mpl...
__version__ = '2.2'
VERSION
2.2
As you can see, there is no module named kmpfit (or even wcs) here. But according to http://www.astro.rug.nl/software/kapteyn/intro.html, these two should be there.
Kindly help. I have never imported any module before.
Thanks in advance...:-)
I just managed to get this working (on Mac OSX, so you may have to adjust this). My steps were:
$ conda install pyodbc (didn't need to go through binstar)
Download & unarchive the kapteyn package, then navigate to its directory
$ python setup.py install, which used my OS's C compiler and Anaconda's python, and installed kapteyn to my anaconda distro's site-packages, as it should.
Check that kmpfit.so is in the kapteyn folder in site-packages, showing that kmpfit installed correctly.
>> from kapteyn import kmpfit failed, ImportError: cannot import name kmpfit. I did some digging and discovered that it was still importing kapteyn from the folder that I downloaded, not from site-packages.
Delete the downloaded kapteyn folder, then try again. It worked!