cx_freeze builds Mac program that runs from command line but dies when clicked - macos

I have a Python 2.7/PyQt4 program that I am attempting to freeze with cx_freeze. The program also uses requests, serial, xml.etree.ElementTree, and collections. Using the unmodified setup.py generated by cxfreeze-quickstart-2.7, I can successfully build as both a console program (python setup.py build) and an .app (python setup.py bdist_mac) in Yosemite using Python from macports. If I run the program directly either from the app bundle or the dist:
$ build/MacDISE-1.0.app/Contents/MacOS/macdise
$ dist/macdise
It runs exactly as expected. If I open from the command line:
$ open -a /Users/jeffemandel/macdise/build/MacDISE-1.0.app
I get the dreaded
LSOpenURLsWithRole() failed for the application /Users/jeffemandel/macdise/build/MacDISE-1.0.app with error -10810.
I worked through a number of potential issues raised by Dan McCombs (distutils.util.get_platform, sys.arg), but these don’t seem to be the problem. Through brute force trial and error, I found that if I put all of my code in a separate module, simply importing that module (without actually invoking it) causes the 10810 error, so I figured it was finding a library when run from the command line, but not from the app. I put the dist directory on a thumb drive and ran it on another Mac that doesn't have Python, Qt4, etc installed, and got this:
packages/cx_Freeze/initscripts/Console.py", line 27, in <module> File "macdise.py", line 4, in <module>
File "ExtensionLoader_PyQt4_QtGui.py", line 11, in <module>
ImportError: dlopen(/Volumes/NO NAME/dist/PyQt4.QtGui.so, 2): Library not loaded: /opt/local/Library/Frameworks/QtGui.framework/Versions/4/QtGui
Referenced from: /Volumes/NO NAME/dist/PyQt4.QtGui.so
Reason: image not found
When I look in dist, there is a file QtGui that is the same size as the one in /opt. So it seems the failure is dlopen(PyQt4.QtGui.so) returning a hard-coded path to the QtGui library. I'm guessing the solution is simple, but I haven't stumbled across it yet.
Update: I looked at the libraries in build/Contents/MacOS/PyQt4.*.so with otool, and these all have #executable_path (as opposed to hard-coded paths in dist). My stupid. So I repeated the process of moving the program, only using the MacOS folder rather than the dist folder, and executing macdise from the command line on my wife's MBP. The problem turned out to be in the way I was looking for the included_files. I changed this to:
if getattr(sys, 'frozen', False):
uiName = os.path.join(os.path.dirname(sys.executable), "tabDISE.ui" )
else:
uiName = "tabDISE.ui"
and it runs. What would have saved me a day would be a way to automagically dump the error message generated when executing from the command line to the console log. If someone knows how to do this, it would be a big help.

Related

cx_Freeze Mac-build stopped at _ctypes for Homebrew-and-Python

We use cx_Freeze to produce standalone binary build of our python application under Mac OS. The build runs well under the build machine (which has Homebrew-and-Python installed), but failed in client machine with following error messages.
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/cx_Freeze/initscripts/__startup__.py", line 14, in run
module.run()
File "/usr/local/lib/python2.7/site-packages/cx_Freeze/initscripts/Console.py", line 26, in run
exec(code, m.__dict__)
File "./utest2.py", line 15, in <module>
from ttLib import *
File "ttLib.py", line 848, in init ttLib
import ctypes
File "/usr/local/Cellar/python#2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ctypes/__init__.py", line 7, in <module>
from _ctypes import Union, Structure, Array
ImportError: dlopen(/Users/gff/src/TextSeek_test/build/test_build/lib/_ctypes.so, 2): Symbol not found: __PySlice_AdjustIndices
Referenced from: /Users/gff/src/TextSeek_test/build/test_build/lib/_ctypes.so
Expected in: flat namespace
in /Users/gff/src/TextSeek_test/build/test_build/lib/_ctypes.so
By browsing stackoverflow answers about "__PySlice_AdjustIndices" and "flat namespace", we suspect this error was caused by the python conflict between system-default version and homebrew version.
Then we use "brew install python#2" in client machine, and the ctypes error disappeared. We run "brew uninstall python#2", this error returned back.
The question is : how can we embed necessary part of homebrew-python to bypass this ctype error?
We used "otool -L lib/_ctypes.so" to know the depenency of dynamic-linked file, it only showed out one file "/usr/lib/libSystem.B.dylib". This file seems to have no relation with python.
Any advice to move forward?
Dev Machine Info: Mac High Sierra, python 2.7.15 64bit. Use homebrew-python to build.
setup.py for cx_Freeze :
build_exe_options = {
"packages": ["wcwidth", "watchdog", "xlrd", "jinja2", "subprocess"],
"excludes": [ "AppKit", "Carbon", "CoreFoundation", "Finder", "Foundation", "FSEvents", "objc"],
"include_msvcr": True,
"zip_include_packages":["winshell", "wcwidth", "watchdog", "pyhk", "xlrd", "jinja2",\
"argh", "ctypes", "email", "encodings" ]
}
exeList = [Executable( "utest2.py", base = None, targetName= "utest") ]
setup( name = "XXX",
description = u"XXX desc",
options = {
"build_exe": build_exe_options,
'bdist_mac': {
'bundle_name': "XXX",
}
},
executables = exeList)
I found the solution by myself.
This issue is caused by wrongly linked dylib files. We use "otool -L" to find the dependency and re-link them with "install_name_tool -change" one by one. Finally the program works.
On the client machine, after installation of the frozen application, try to manually copy all dynamic libraries (.so, .dylib, ...?) which are in the lib subdirectory of the build directory into the build directory itself. Copy also all dynamic libraries which are in the build directory into its lib subdirectory. Does this solve the problem? On the build machine, you should of course use the same python version to produce the frozen application than you use to run the unfrozen application.
If 1. solves the problem, find out which dynamic libraries need to be copied in order that the application works (i.e. find out which of the manual copy actions you've done are really necessary). Use the include_files list of the build_exe options in your setup script to let cx_Freeze include the dynamic library automatically at the build step. You can use a tuple (source, destination) as item in the include_files list to let cx_Freeze copy a file from source to a specific destination into the build directory.
Explanation: I don't know Mac/OS. But I believe the issue you describe could be related to a similar issue with Microsoft Visual C++ Redistributable DLLs under Windows: cx_Freeze 5.1.1 includes packages in a lib subdirectory of the build directory, in the previous versions of cx_Freeze they used to be in the build directory itself. Some dynamic libraries need to be found in the build directory but are erroneously included in the lib subdirectory by cx_Freeze or vice versa. On the build machine it is usually not a problem, because a copy of the library is usually found in the system path, but on the client machine often no copy of the library can be found on the system path, or an incompatible one.
By the way the build_exe option "include_msvcr": True does not seem to work in cx_Freeze5.1.1, see this issue.

Manually specify library when pyinstaller sees conflicting versions

Is it possible to manually replace or specify the location of a .dylib on Mac OSX when using pyinstaller?
I'm getting the error when trying to open my completed .app
Traceback (most recent call last):
File "DeepMeerkat/main.py", line 3, in <module>
import cv2
File "/Library/Python/2.7/site-packages/PyInstaller/loader/pyimod03_importers.py", line 546, in load_module
module = imp.load_module(fullname, fp, filename, ext_tuple)
ImportError: dlopen(/Users/ben/Documents/DeepMeerkat/Installer/dist/Lib/cv2.so, 2): Library not loaded: #loader_path/libpng16.16.dylib
Referenced from: /Users/ben/Documents/DeepMeerkat/Installer/dist/Lib/libopencv_imgcodecs.3.3.dylib
Reason: Incompatible library version: libopencv_imgcodecs.3.3.dylib requires version 48.0.0 or later, but libpng16.16.dylib provides version 45.0.0
Failed to execute script main
I have no problem loading cv2 in python outside of the app, or any other problems with open, which was installed with homebrew.
Poking around, I think it is extremely likely the error comes from pyinstaller grabbing libpng from X11 (/opt/X11/include/libpng16) when it needs to be grabbing from Homebrew's folders (/usr/local/Cellar/libpng/1.6.32/)
How can I use a hook to specify which libpng I want?
I had the same problem, the answer lay in editing the .spec file. Add the following line after a = Analysis...:
a.binaries = a.binaries - TOC([('libpng16.16.dylib',None,None)])
This has the effect of removing the offending dylib from the relevant TOC file which solves the conflict.

Installing Perl and PDL

I am currently trying - unsuccessfully to install perl PDL as I would like to automate some of my data analysis and graphics generation.
Unfortunately I seem unable to even install the necessary modules without issue, let alone create a small script to deal with my data files.
I have some rudementary experience with Perl but Nothing fancy and am using Windows 10 x64.
Initially I went for the x64 strawberry perl v5.24 installation, followed by ppm installations of Astro::FITS::Header, Convert::UU, OpenGL and PDL - as instructed here http://pdl.perl.org/?page=install. However, there were issues with the installation of Inline::C and after trying the example listed on page 5 of the PDL_Book_Latest http://sourceforge.net/projects/pdl/files/PDL/2.4.10/PDL-Book-20120205.pdf/download there wasn't any image appearing, nor image file in any location I could think to search. Back to Google and still no luck finding any information on the source of this issue so I decided to go back to square one and use an older version in the hopes that it would be more well documented.
So, uninstallation of strawberry perl through Windows installed programs dialog, deletion of strawberry folder and registry entries containing "strawberry". Then a new installation of perl, with the Padre IDE, v5.14 - maybe some debugging fonctionality will help pinpoint the errors. Started installing the same modules (Astro::FITS::Header, Convert::UU, OpenGL, PGPLOT). Astro and Convert worked fine, but the OpenGL started throwing errors about ExtUtils::F77 which I am apparently missing after a few more hours searching. I go to install this module and it keeps giving errors about 'gfortan' being an unknown command... It was my understanding that the idea of PDL was to avoid fortran and C, and I have found next to no information about this installation problem so far.
So i decided to try an alternative approach. I installed the ppm module from cpan and repeated the initial installation process through the ppm command. This time i get:
"Unknown element 'PROVIDE' found inside SOFTPKG. at C:/Dwimperl/perl/site/lib/PPM.pm line 1462."
So my question is as follows:
Am I missing some critical step in the installation process? Is my system just handing this really badly, and if so is there any sort of possible work around? Is the fact that I tried two different versions liable to create path issues that could be at the root of this problem?
Could anyone provide an alternative and preferably not too complex route to install PDL with a given perl installation on Windows 10?
EDIT 1:
Here is the code for the error I got trying to install Inline::C, an apparent pre-requisite for PDL
code deleted to make space for updates...
EDIT 3:
I reinstalled GnuWin32 in the directory specified by PATH, rather that changing PATH, and the tried the installation again. Inline::C gave the same 'diff' and 'rm' errors. So i ran "cpanm --notest 'package'" to install Inline::C and then PDL, both said they were successful. However, even using '--notest' cpanm could not install OpenGL and PGPLOT.
The final objectif is to be able to produce graphics using the fonctions from PGPLOT: for example
# use PDL;
# use PDL::Graphics::PGPLOT;
# imag(sin(rvals(200,200)+1));
yet this doesn't work of course if PGPLOT can't install.
EDIT 4:
Tried the fix found by #Dr.Avalange at sourceforge https://sourceforge.net/p/pogl/bugs/26/ and noticed that I seem to have multiple copies of this file.
Here
Is this normal...?
EDIT 5:
So i uninstalled perl, deleted any lingering files I could find in %APPDATA%, %USERPROFILE% and C:/DWIM, and then ran ccleaner just in case I missed anything.
I then downloaded Strawberry perl 5.24.1 x64 for Windows and installed it.
After this, I ran 'cpanm Astro::FITS::Header', cpanm 'Convert-UU' and 'cpanm OpenGL' as per http://pdl.perl.org/?page=install. This all worked fine. Then I ran 'cpanm PDL' which installed Inline, Pegex, Win32::Mutex and Module::Compile dependencies. However, if had the same error with Inline::C that you mentioned previously.
I already had GetGnuWin32 installed before so not sure why this isn't working. The Windows 'Path' environment (not PATH - this doesn't exist according to the advaced system dialogue) is noted as
%USERPROFILE%\AppData\Local\Microsoft\WindowsApps;
In this directory I have the folder GetGnuWin32 which should have installed all the packages from the link you provided, unless there is a supplementary step that I missed to install this correctly? So I redownloaded the DiffUtils setup and ran that, installing to \programfiles(x86) (default option). I then reran 'cpanm PDL'.
I still get the same 'diff' error message. So is DiffUtils, not installing properly? (I did close and re-open the commandline between tries and after the installation)
EDIT 6:
So after installing PDL and PGPLOT with --notest I tried to run the following test and it failed. See below
C:\Windows\system32>perldl
perlDL shell v1.357
PDL comes with ABSOLUTELY NO WARRANTY. For details, see the file
'COPYING' in the PDL distribution. This is free software and you
are welcome to redistribute it under certain conditions, see
the same file for details.
ReadLines, NiceSlice, MultiLines enabled
Reading PDL/default.pdl...
Found docs database C:/Strawberry/perl/site/lib/PDL/pdldoc.db
Type 'help' for online help
Type 'demo' for online demos
Loaded PDL v2.017 (supports bad values)
Note: AutoLoader not enabled ('use PDL::AutoLoader' recommended)
pdl> use PDL::Graphics//PLplot
Unknown regexp modifier "/P" at (eval 62) line 4, at end of line
Unknown regexp modifier "/L" at (eval 62) line 4, at end of line
Unknown regexp modifier "/t" at (eval 62) line 4, at end of line
BEGIN not safe after errors--compilation aborted at (eval 62) line 5, <DATA> line 207.
pdl> use PDL::Graphics::PLplot
Can't locate PDL/Graphics/PLplot.pm in #INC (you may need to install the PDL::Graphics::PLplot module) (#INC contains: C:/Strawberry/perl/site/lib/MSWin32-x64-multi-thread C:/Strawberry/perl/site/lib C:/Strawberry/perl/vendor/lib C:/Strawberry/perl/lib .) at (eval 72) line 5.
BEGIN failed--compilation aborted
pdl> use PDL
pdl> use lib 'C:/Strawberry/perl/site/lib/PDL/Graphics'
pdl> use PGPLOT
pdl> use PDL::Graphics::PGPLOT
pdl> imag(sin(rvals(200,200)+1))
Undefined subroutine &PDL::Graphics::PGPLOT::pgqinf called at C:/Strawberry/perl/site/lib/PDL/Graphics/PGPLOT.pm line 408, <DATA> line 90.
pdl> exit
C:\Windows\system32>cpanm PDL::Graphics::PGPLOT::pgqinf
! Finding PDL::Graphics::PGPLOT::pgqinf on cpanmetadb failed.
! Finding PDL::Graphics::PGPLOT::pgqinf () on mirror http://www.cpan.org failed.
! Couldn't find module or a distribution PDL::Graphics::PGPLOT::pgqinf
C:\Windows\system32>cpan PDL::Graphics::PGPLOT::pgqinf
Loading internal null logger. Install Log::Log4perl for logging messages
CPAN: CPAN::SQLite loaded ok (v0.211)
Database was generated on Thu, 13 Apr 2017 13:49:14 GMT
C:\Windows\system32>
There was already a directory problem but I managed to point it in th right direction I had thought. Also tried installing the PLplot 'cpanm' said the installation was successful but there is no .pm file, only a .pd in another directory and so the use PDL::Graphics::PLplot fails as there is no .pm file to find.
EDIT: 7
So retrying ppm install http://www.sisyphusion.tk/ppm/PGPLOT.ppd seemed to work this time as the links int he explanation you provided are no longer in use
http://www.kalinabears.com.au/w32perl/pgplot-5.2.2-mingw32.tar.bz2
http://jrfonseca.home.dyndns.org/projects/gnu-win32/software/ported/patches/pgplot-5.2.2-mingw32.diff.gz
And while this seemed to work, there are no traces of the files that were listed as necessary for the installation, unless they were just for this specific method?
pgplot/bin/cpgplot.dll
pgplot/bin/pgplot.dll
pgplot/bin/grfont.dat
pgplot/include/cpgplot.h
pgplot/lib/cpgplot.a
pgplot/lib/pgplot.a
finally, trying the example again gives the following; no errors, but no image either...
Microsoft Windows [version 10.0.14393]
(c) 2016 Microsoft Corporation. Tous droits réservés.
C:\Windows\system32>perldl
perlDL shell v1.357
PDL comes with ABSOLUTELY NO WARRANTY. For details, see the file
'COPYING' in the PDL distribution. This is free software and you
are welcome to redistribute it under certain conditions, see
the same file for details.
ReadLines, NiceSlice, MultiLines enabled
Reading PDL/default.pdl...
Found docs database C:/Strawberry/perl/site/lib/PDL/pdldoc.db
Type 'help' for online help
Type 'demo' for online demos
Loaded PDL v2.017 (supports bad values)
Note: AutoLoader not enabled ('use PDL::AutoLoader' recommended)
pdl> use PDL
pdl> use PDL::Graphics::PGPLOT
pdl> imag(rvals(200,200)+1)
%PGPLOT, Unable to read font file: grfont.dat
%PGPLOT, Use environment variable PGPLOT_FONT to specify the location of the PGPLOT grfont.dat file.
Displaying 200 x 200 image from 1 to 142.421356201172, using 240 colors (16-255)...
pdl> $ENV{PGPLOT_FONT} = "C:/Strawberry/perl/site/lib/PGPLOT/pgplot_supp"
pdl> imag(rvals(200,200)+1)
Displaying 200 x 200 image from 1 to 142.421356201172, using 240 colors (16-255)...
pdl>
After translating from French I can see that you don't have diff in your system. Either install diff (part of gnu32: http://gnuwin32.sourceforge.net/) or just skip the tests. See also:
https://github.com/ingydotnet/inline-c-pm/issues/60
With Strawberry 5.24.1 simply type the following from the command prompt:
cpanm --notest PDL
or if you want to watch everything that's going on:
cpanm -v --notest PDL
This will take some time to run.

Pyinotify error when building linux with yocto on mac osx

I'm trying to build a linux with yocto (morty) on my Mac OSX (El Capitan). The source step with source poky/oe-init-build-env buildsucceeded without errors, but when I execute bitbake core-image-baseI get the following error trace:
Traceback (most recent call last):
File "/Users/peterblicharski/Entwicklung/dnsPro/yocto/yoctowa/poky/bitbake/bin/bitbake", line 48, in <module>
cookerdata.CookerConfiguration()))
File "/Users/peterblicharski/Entwicklung/dnsPro/yocto/yoctowa/poky/bitbake/lib/bb/main.py", line 468, in bitbake_main
server = start_server(servermodule, configParams, configuration, featureset)
File "/Users/peterblicharski/Entwicklung/dnsPro/yocto/yoctowa/poky/bitbake/lib/bb/main.py", line 354, in start_server
cooker = bb.cooker.BBCooker(configuration, features)
File "/Users/peterblicharski/Entwicklung/dnsPro/yocto/yoctowa/poky/bitbake/lib/bb/cooker.py", line 178, in __init__
self.configwatcher = pyinotify.WatchManager()
File "/Users/peterblicharski/Entwicklung/dnsPro/yocto/yoctowa/poky/bitbake/lib/pyinotify.py", line 1759, in __init__
raise InotifyBindingNotFoundError()
pyinotify.InotifyBindingNotFoundError: Couldn't find any inotify binding
DEBUG: Removed the following variables from the environment: __CF_USER_TEXT_ENCODING, XPC_SERVICE_NAME, SECURITYSESSIONID, PS1, TERM_PROGRAM_VERSION, TMPDIR, _, TERM_PROGRAM, DISPLAY, LANG, SHLVL, GIT_PS1_SHOWDIRTYSTATE, Apple_PubSub_Socket_Render, __PYVENV_LAUNCHER__, OLDPWD, XPC_FLAGS, TERM_SESSION_ID, BUILDDIR
Mac is not a supported build host and currently does not work at all. Quoting Ross Burton from mailing list:
If you install all of the GNU tools using brew or similar and put them first
on $PATH then you can get bitbake started. Then you need to stub out the
linux-specific bits in bitbake. I've previously started on this work
already
(http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/log/?h=ross/darwin).
The next step is figuring out how to configure OE to build and link natively
on OSX using LLVM instead of GCC.
However all of this is mostly academic because in Sierra (iirc) onwards
there is tighter security on processes, which means that pseudo won't work
even if you port it to macOS.
Your error would be part of "linux-specific bits in bitbake" and seems to be worked around in ross' branch. But as you can see it's not the last issue you'll face.
Based on the discussion there are probably ways to get pseudo running but after that you would get to fix all the native builds Yocto does: it's likely that many projects we use are not frequently built on Mac and will fail there.

Getting an "invalid syntax" error from __init__.py

I'm following this example for using Flask-JWT:
https://pythonhosted.org/Flask-JWT/
And I get this error when I start uwsgi:
Traceback (most recent call last):
File "./wsgi.py", line 1, in
from main import app as application
File "./main.py", line 5, in
from auth import api_auth
File "./auth.py", line 3, in
from flask_jwt import JWT
File "/opt/mist_base/env/lib/python2.6/site-packages/flask_jwt/__init__.py", line 83
for claim in verify_claims
^
SyntaxError: invalid syntax
I've been only using Python v2.6.6 (for Centos 6.x per customer specification) for only a week so I'm still very green at it. As a result, earlier today I discovered I had multiple variations of JWT-something in my virtualenv so I removed all instances and only reinstalled Flask-JWT. I had thought the issue was library conflicts (and it may have been earlier), but I'm still getting this error and other people have used it with no issues. Am I doing something wrong or is this library simply not designed to be used with Python v2.6.x?
[UPDATE] I've gone ahead and compiled v2.7.12 from source in CentOS 6 and so far it works nicely for my needs. I found these directions:
http://toomuchdata.com/2014/02/16/how-to-install-python-on-centos/
and was able to install in an alternate directory.
This library just doesn't work on Python 2.6. The source code relies on features like dict comprehensions that only exist in 2.7 and up.

Resources