cx_Freeze Mac-build stopped at _ctypes for Homebrew-and-Python - macos

We use cx_Freeze to produce standalone binary build of our python application under Mac OS. The build runs well under the build machine (which has Homebrew-and-Python installed), but failed in client machine with following error messages.
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site-packages/cx_Freeze/initscripts/__startup__.py", line 14, in run
module.run()
File "/usr/local/lib/python2.7/site-packages/cx_Freeze/initscripts/Console.py", line 26, in run
exec(code, m.__dict__)
File "./utest2.py", line 15, in <module>
from ttLib import *
File "ttLib.py", line 848, in init ttLib
import ctypes
File "/usr/local/Cellar/python#2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ctypes/__init__.py", line 7, in <module>
from _ctypes import Union, Structure, Array
ImportError: dlopen(/Users/gff/src/TextSeek_test/build/test_build/lib/_ctypes.so, 2): Symbol not found: __PySlice_AdjustIndices
Referenced from: /Users/gff/src/TextSeek_test/build/test_build/lib/_ctypes.so
Expected in: flat namespace
in /Users/gff/src/TextSeek_test/build/test_build/lib/_ctypes.so
By browsing stackoverflow answers about "__PySlice_AdjustIndices" and "flat namespace", we suspect this error was caused by the python conflict between system-default version and homebrew version.
Then we use "brew install python#2" in client machine, and the ctypes error disappeared. We run "brew uninstall python#2", this error returned back.
The question is : how can we embed necessary part of homebrew-python to bypass this ctype error?
We used "otool -L lib/_ctypes.so" to know the depenency of dynamic-linked file, it only showed out one file "/usr/lib/libSystem.B.dylib". This file seems to have no relation with python.
Any advice to move forward?
Dev Machine Info: Mac High Sierra, python 2.7.15 64bit. Use homebrew-python to build.
setup.py for cx_Freeze :
build_exe_options = {
"packages": ["wcwidth", "watchdog", "xlrd", "jinja2", "subprocess"],
"excludes": [ "AppKit", "Carbon", "CoreFoundation", "Finder", "Foundation", "FSEvents", "objc"],
"include_msvcr": True,
"zip_include_packages":["winshell", "wcwidth", "watchdog", "pyhk", "xlrd", "jinja2",\
"argh", "ctypes", "email", "encodings" ]
}
exeList = [Executable( "utest2.py", base = None, targetName= "utest") ]
setup( name = "XXX",
description = u"XXX desc",
options = {
"build_exe": build_exe_options,
'bdist_mac': {
'bundle_name': "XXX",
}
},
executables = exeList)

I found the solution by myself.
This issue is caused by wrongly linked dylib files. We use "otool -L" to find the dependency and re-link them with "install_name_tool -change" one by one. Finally the program works.

On the client machine, after installation of the frozen application, try to manually copy all dynamic libraries (.so, .dylib, ...?) which are in the lib subdirectory of the build directory into the build directory itself. Copy also all dynamic libraries which are in the build directory into its lib subdirectory. Does this solve the problem? On the build machine, you should of course use the same python version to produce the frozen application than you use to run the unfrozen application.
If 1. solves the problem, find out which dynamic libraries need to be copied in order that the application works (i.e. find out which of the manual copy actions you've done are really necessary). Use the include_files list of the build_exe options in your setup script to let cx_Freeze include the dynamic library automatically at the build step. You can use a tuple (source, destination) as item in the include_files list to let cx_Freeze copy a file from source to a specific destination into the build directory.
Explanation: I don't know Mac/OS. But I believe the issue you describe could be related to a similar issue with Microsoft Visual C++ Redistributable DLLs under Windows: cx_Freeze 5.1.1 includes packages in a lib subdirectory of the build directory, in the previous versions of cx_Freeze they used to be in the build directory itself. Some dynamic libraries need to be found in the build directory but are erroneously included in the lib subdirectory by cx_Freeze or vice versa. On the build machine it is usually not a problem, because a copy of the library is usually found in the system path, but on the client machine often no copy of the library can be found on the system path, or an incompatible one.
By the way the build_exe option "include_msvcr": True does not seem to work in cx_Freeze5.1.1, see this issue.

Related

cannot run scons - unable to find files

So I've installed scons via Anaconda's conda install scons under Windows 10 (Python 3.6) and could not execute it via the command line so I've added C:\Users\D\Anaconda3\envs\py36\Scripts\ to my Path (even tough C:\Users\Dominik\Anaconda3 was already in there).
Now I can execute it in the Powershell, but I get an error, because it is unable find engine files:
scons
SCons import failed. Unable to find engine files in:
C:\Users\D\Anaconda3\envs\py36\Scripts\..\engine
C:\Users\D\Anaconda3\envs\py36\Scripts\scons-local-3.0.1
C:\Users\D\Anaconda3\envs\py36\Scripts\scons-local
C:\Users\D\Anaconda3\scons-3.0.1
C:\Users\D\Anaconda3\Lib\site-packages\scons-3.0.1
C:\Users\D\Anaconda3\scons
C:\Users\D\Anaconda3\Lib\site-packages\scons
Traceback (most recent call last):
File "C:\Users\D\Anaconda3\envs\py36\Scripts\scons.py", line 192, in <module>
import SCons.Script
ImportError: No module named 'SCons'
Does someone know how to further investigate/fix this problem?
I found it thanks to nekomatic.
If you use the anaconda prompt (with the correct environment activated or in my case the correct shortcut leading to anaconda prompt in that environment), scons runs just fine.
See this link explaining how anaconda uses the path on windows and whats the advantage of using the anaconda prompt (mainly it dynamically manages PATH correctly for that session only with full knowledge of conda environments).

Manually specify library when pyinstaller sees conflicting versions

Is it possible to manually replace or specify the location of a .dylib on Mac OSX when using pyinstaller?
I'm getting the error when trying to open my completed .app
Traceback (most recent call last):
File "DeepMeerkat/main.py", line 3, in <module>
import cv2
File "/Library/Python/2.7/site-packages/PyInstaller/loader/pyimod03_importers.py", line 546, in load_module
module = imp.load_module(fullname, fp, filename, ext_tuple)
ImportError: dlopen(/Users/ben/Documents/DeepMeerkat/Installer/dist/Lib/cv2.so, 2): Library not loaded: #loader_path/libpng16.16.dylib
Referenced from: /Users/ben/Documents/DeepMeerkat/Installer/dist/Lib/libopencv_imgcodecs.3.3.dylib
Reason: Incompatible library version: libopencv_imgcodecs.3.3.dylib requires version 48.0.0 or later, but libpng16.16.dylib provides version 45.0.0
Failed to execute script main
I have no problem loading cv2 in python outside of the app, or any other problems with open, which was installed with homebrew.
Poking around, I think it is extremely likely the error comes from pyinstaller grabbing libpng from X11 (/opt/X11/include/libpng16) when it needs to be grabbing from Homebrew's folders (/usr/local/Cellar/libpng/1.6.32/)
How can I use a hook to specify which libpng I want?
I had the same problem, the answer lay in editing the .spec file. Add the following line after a = Analysis...:
a.binaries = a.binaries - TOC([('libpng16.16.dylib',None,None)])
This has the effect of removing the offending dylib from the relevant TOC file which solves the conflict.

cx_freeze builds Mac program that runs from command line but dies when clicked

I have a Python 2.7/PyQt4 program that I am attempting to freeze with cx_freeze. The program also uses requests, serial, xml.etree.ElementTree, and collections. Using the unmodified setup.py generated by cxfreeze-quickstart-2.7, I can successfully build as both a console program (python setup.py build) and an .app (python setup.py bdist_mac) in Yosemite using Python from macports. If I run the program directly either from the app bundle or the dist:
$ build/MacDISE-1.0.app/Contents/MacOS/macdise
$ dist/macdise
It runs exactly as expected. If I open from the command line:
$ open -a /Users/jeffemandel/macdise/build/MacDISE-1.0.app
I get the dreaded
LSOpenURLsWithRole() failed for the application /Users/jeffemandel/macdise/build/MacDISE-1.0.app with error -10810.
I worked through a number of potential issues raised by Dan McCombs (distutils.util.get_platform, sys.arg), but these don’t seem to be the problem. Through brute force trial and error, I found that if I put all of my code in a separate module, simply importing that module (without actually invoking it) causes the 10810 error, so I figured it was finding a library when run from the command line, but not from the app. I put the dist directory on a thumb drive and ran it on another Mac that doesn't have Python, Qt4, etc installed, and got this:
packages/cx_Freeze/initscripts/Console.py", line 27, in <module> File "macdise.py", line 4, in <module>
File "ExtensionLoader_PyQt4_QtGui.py", line 11, in <module>
ImportError: dlopen(/Volumes/NO NAME/dist/PyQt4.QtGui.so, 2): Library not loaded: /opt/local/Library/Frameworks/QtGui.framework/Versions/4/QtGui
Referenced from: /Volumes/NO NAME/dist/PyQt4.QtGui.so
Reason: image not found
When I look in dist, there is a file QtGui that is the same size as the one in /opt. So it seems the failure is dlopen(PyQt4.QtGui.so) returning a hard-coded path to the QtGui library. I'm guessing the solution is simple, but I haven't stumbled across it yet.
Update: I looked at the libraries in build/Contents/MacOS/PyQt4.*.so with otool, and these all have #executable_path (as opposed to hard-coded paths in dist). My stupid. So I repeated the process of moving the program, only using the MacOS folder rather than the dist folder, and executing macdise from the command line on my wife's MBP. The problem turned out to be in the way I was looking for the included_files. I changed this to:
if getattr(sys, 'frozen', False):
uiName = os.path.join(os.path.dirname(sys.executable), "tabDISE.ui" )
else:
uiName = "tabDISE.ui"
and it runs. What would have saved me a day would be a way to automagically dump the error message generated when executing from the command line to the console log. If someone knows how to do this, it would be a big help.

How to let Python find the HTML tidy lib

I downloaded the libtidy.dll and libtidy.lib and put them in the corresponding folders of python27. Then I installed pytidylib for python.
But when I type
import tidylib
I got below error which indicates python can't find the module
File "<pyshell#0>", line 1, in <module>
import tidylib
File "C:\Python27\lib\site-packages\tidylib\__init__.py", line 69, in <module>
raise OSError("Could not libtidy using any of these names: %s" % (",".join(LIB_NAMES)))
OSError: Could not libtidy using any of these names: libtidy,libtidy.so,libtidy- 0.99.so.0,cygtidy-0-99-0,tidylib,libtidy.dylib,tidy
Anyone can help one this?
I downloaded the libtidy.dll and libtidy.lib and put them in the corresponding folders of python27
Don't do that. Use pip or easy_install to manage your packages.
Update Apparently your question is about how to install dlls on windows. Put them in any directory on the windows PATH.
Pytidylib requires tidy, which is not a python package. You have to install it yourself and make it accessable.
If using windows, just extract the tidy package somewhere and add its bin folder to windows' PATH.

making a python program executable

from distutils.core import setup
import py2exe, sys, os
sys.argv.append('py2exe')
setup(
options = {'py2exe': {'bundle_files': 1}},
windows = [{'script': "single.py"}],
zipfile = None,
)
in this setup file for py2exe where it says single.py is that where I place the name of my program?
I don't know your py2exe tool, but we usually use this way to convert py to exe:
Download and install Standard Python Software:
http://www.python.org/download/
Download PyInstaller via link below:
http://pyinstaller.python-hosting.com/
Unpack the archive, that you have downloaded!
In this examople, the directory of the unpacked files:
In the <UNPACKED_FILES_DIR> directory, run Configure.py.
It must be run before trying to build anything.
Create a spec file for your project:
python Makespec.py -F -p <PYTHON_LIB_PATH> <PYTHON_SCRIPT>
-F: Produce a single file deployment.
-p <PYTHON_LIB_PATH>: Set base path for import (like using PYTHONPATH).
( e.g.: C:\Program Files\Python24\Lib\ )
<PYTHON_SCRIPT>: Path to python script.
6 Build your project!
python Build.py <SPECFILE>
<SPECFILE>: Path to the specfile, that have been created in step 4!
The full path to <SPECFILE>:
<UNPACKED_FILES_DIR>/<PYTHON_SCRIPT>/<PYTHON_SCRIPT>.spec
The binary file will be placed in the directory of <SPECFILE>.
If you can restrict your code, then Shed Skin, PyPy, or Cython make true, fast executables.
Py2exe, PyInstaller, or bbfreeze can package Python up to 2.7 into single executables.
Cx_Freeze packages Python up to 3.x into an executable plus many other files.
Yes. Are you making a windowing application or a console application? See the example setup.py files that came with py2exe.

Resources