Azure functions (cloud): Install cython dependent module - installation

I'm trying to install/import a cython dependent module [ZigZag] into my python function on Azure functions (Linux platfrom), I'm working via Windows 8.1 (VSCode), Running Python 3.6 on both local machine and azure functions.
I tried including Cython in the requirements.txt file, this errors out during deployment (seems installation of external modules happens somewhat in parallel)
I tried creating a '.toml' file, no luck.
I also tried the idea found here Python google cloud function deployment failure - Madmom pip package still didn't work.
Finally I decided to deploy the function first (with cython included in the 'requirements.txt' file) without the zigzag module, and then in my python script use subprocess.Popen() to do a pip install zigzag and import it before the rest of the function runs.
This is where I get stuck. In my python file I have these lines included to do the above:
#function to be called in azure __init__.py file
def f....:
# check for zigzag
try:
import zigzag
except ImportError:
subprocess.Popen(["pip", "install", "ZigZag"], shell=True)
finally:
import zigzag
.
.
.
rest of the function...
This deploys successfully to azure functions, however it fails during execution with the error:
Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcException :
Result: Failure Exception: ModuleNotFoundError: No module named 'zigzag
Any idea's how to do this correctly?
NB: This issue occurs on google cloud functions as well.

I see this is related to https://learn.microsoft.com/en-us/answers/questions/152557/index.html
As per the thread, the latest version for ZigZag is 0.2.2 and the supported python version is 3.5: https://pypi.org/project/ZigZag/
The azure function supports 3.6 and above python version which will make it incompatible. You can try installing the latest cython using requirements.txt and it will work as expected. But when you try to install ZigZaG it tries to find the dependency of cython package (Cython==0.26.1) that is compatible with it therefor it throws the error that cython package not found.

Related

shiboken2 & PySide2 issues

I get this error when trying to run a PyQt5 Application, I keep getting it and I think it's when I use pyuic5 to export code from QT Designer and then try to run it.
from .shiboken2 import *
ImportError: DLL load failed: The specified procedure could not be found.
I have used pip freeze to find the versions of shiboken2 and pyside2 and they are both the same version.
I installed Python 3.7 + And it works.

What happens with "pure" Python + Cython packages during installation built failure?

I just read the Cython Pure Python Mode documentation and I'm not sure if I understand it right. It sounds as if I could keep all my Python files as they are, add *.pxd files where I declare Cython types. In the setup.py, I still add
from setuptools import setup
from Cython.Build import cythonize
setup(
ext_modules = cythonize(
"A.py",
compiler_directives={'language_level' : "3"}
)
)
When I run python setup.py build_ext --inplace it actually builds the .so file.
What happens when I create the sdist / bdist, upload them to PyPI and a user does not have a matching platform? They will download the sdist, sure. I guess pip / setuptools will automatically try to compile the extension modules (A.py) and I guess if that works, it is fine. But what if cythonize fails? Will it still install the package and use the pure Python code?
I don't think so. I believe a failure in setup.py aborts installation completely.
You can try to declare an extension optional but there're reports that doesn't really work. Could be an issue with older setuptools.

Creating Python 3.6 AWS Lambda package

My OS is win 10 Home.
I have a Python 3.6 script that updates an MSSql DB, using pymssql. The script works fine locally.
Now I need to upload it as aws lambda, so I followed this using the cmd:
python -m venv .
Scripts\activate
pip install pymssql
Then I copied my py function to the Lib\site-packages dir, ziped all the dir content and uploaded it to the Lambda service.
The result was this error:
Unable to import module 'validationLambda': No module named 'pymssql'
How to fix this?
I am not strong with Windows environments but you should probably try to do the opposite.
Copy the Lib\site-packages\pymssql directory to the root of your package (same level with your_function.py
Try to import cython along with pymssql.

pyinstaller gives weird errors when compiling pandas

UPDATE june 18:
I would like compiling my python code for windows.
I use to work with pyinstaller, python 3.5+, windows 7 and anaconda to control modules pandas, pyqt (v5) and pyqtgraph, but it seems something is not compatible. I would not be surprised if you tell me to change a version.
In previous versions, it worked with python 3.5, windows 7 and Pyinstaller 3.3.1- without pandas; the new version imports pandas and then this doesn’t work anymore:
with the usual config pyinstaller gives a message saying the C libraries are not built. Following the instructions, I should run python setup.py build_ext --inplace --force, except I don’t see the setup.py file, so I don’t know what to do. Forums tell me it should be a problem with pip but not with conda… but I use conda…
with python 3.6, I got an error message ‘RecursionError: maximum recursion depth exceeded’ (https://github.com/steph2016/profiles/blob/master/errorMessage). This problem seems known and solutions I found on forums is either to downgrade python or to replace my PyInstaller-3.3.1 by PyInstaller-3.4.dev0+7b3143612. But with PyInstaller-3.4.dev0+7b3143612 I get a ’’str’ object has no attribute ‘items’’ error (https://github.com/steph2016/profiles/blob/master/ErrorMessage2). I’m not sure pyinstaller is compatible with python 3.6…
with python 3.4, I got an error similar to the ‘str object’ one above. I’m not sure python3.4 is compatible with qt5…
I would be pleased by any solution with any version of whatever to windows-compile my code
I don't know if it is the issue but I didn't find PyQtGraph in Supported-Packages
I eventually applied the first solution proposed in No module named 'pandas._libs.tslibs.timedeltas' in PyInstaller in the original config (python 3.5, PyInstaller 3.3.1, pyqtgraph, pandas, windows 7,...) and it worked.
I don't know why adding def get_pandas_path ..... a.binaries) helps, so I can't justify. But just to be explicit even if it seems obvious: pyinstaller must be run on the spec file, not on the py one (for instance > pyinstaller toto.spec)

kmpfit module not in Kapteyn v2.2 module list

I am using Python 2.7 through Anaconda 2.7.8 and need Kapteyn 2.2 to perform Non-linear Least Squares fitting easily (it is probably an alternative to Scipy.optimize.leastsq() for dummies like me!).
After copy-pasting this from a previous post here on Stack Overflow:
conda install -c https://conda.binstar.org/dhirschfeld pyodbc
and then running on my cmd (as I did not have pyodbc installed I think, because of which maybe the command prompt on my Windows 7 64-bit system was not responding well to python setup.py install inside the Anaconda directory where I unzipped the Kapteyn .zip file downloaded from University of Groningen website.
But, after the installing pyodbc properly and running python setup.py install, the cmd gave me an error saying error: command 'C:\Users\windows 7\Anaconda\Scripts\gcc.bat' failed with exit status 1. Later, when I tried to import kmpfit module (needed for Non-linear least square fitting with Kapteyn), here is the problem:
import kapteyn
help(kapteyn)
Help on package kapteyn:
NAME
kapteyn - Kapteyn package.
FILE
c:\users\windows 7\anaconda\kapteyn\__init__.py
PACKAGE CONTENTS
_ni_support
celestial
doccer
filters
interpolation
maputils
mplutil
positions
rulers
shapes
tabarray
wcsgrat
DATA
__all__ = ['celestial', 'wcs', 'wcsgrat', 'tabarray', 'maputils', 'mpl...
__version__ = '2.2'
VERSION
2.2
As you can see, there is no module named kmpfit (or even wcs) here. But according to http://www.astro.rug.nl/software/kapteyn/intro.html, these two should be there.
Kindly help. I have never imported any module before.
Thanks in advance...:-)
I just managed to get this working (on Mac OSX, so you may have to adjust this). My steps were:
$ conda install pyodbc (didn't need to go through binstar)
Download & unarchive the kapteyn package, then navigate to its directory
$ python setup.py install, which used my OS's C compiler and Anaconda's python, and installed kapteyn to my anaconda distro's site-packages, as it should.
Check that kmpfit.so is in the kapteyn folder in site-packages, showing that kmpfit installed correctly.
>> from kapteyn import kmpfit failed, ImportError: cannot import name kmpfit. I did some digging and discovered that it was still importing kapteyn from the folder that I downloaded, not from site-packages.
Delete the downloaded kapteyn folder, then try again. It worked!

Resources