I've noticed that pytz misses zoneinfo folder when I try to roll a zip for Windows. Right now I have a workaround that I use after python setup.py build, namely
7z a -xr!*.py* build\exe.win32-2.7\library.zip C:\Python27\Lib\site-packages\pytz
Is there a proper way to achieve that from setup.py or something?
You could fix this, adding the following method:
def include_files():
path_base = "C:\\Python27\\Lib\\site-packages\\pytz\\zoneinfo\\"
skip_count = len(path_base)
zip_includes = [(path_base, "pytz/zoneinfo/")]
for root, sub_folders, files in os.walk(path_base):
for file_in_root in files:
zip_includes.append(
("{}".format(os.path.join(root, file_in_root)),
"{}".format(os.path.join("pytz/zoneinfo", root[skip_count:], file_in_root))
)
)
return zip_includes
Then, into setup.py file:
build_exe_options = {"packages": ["os"],
"excludes": ["tkinter"],
"zip_includes": include_files(),
...
}
Hope that helps
I've solved this problem in Python 3.4 in the following way
import pytz
setup(
...
options = {'build_exe':
{'include_files': (pytz.__path__[0],), ...},
},
)
Then pytz is included unzipped with all its time zones
Related
I am yet to use poetry to run project, so excuse lack of understanding.
I successfully installed the poetry python library manager, using:
curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python3
Next step poetry install initially returned this error:
me#LAPTOP-G1DAPU88:~/.ssh/workers-python/workers$ poetry install
RuntimeError
Poetry could not find a pyproject.toml file in /home/me/.ssh/workers-python/workers or its parents
at ~/.poetry/lib/poetry/_vendor/py3.8/poetry/core/factory.py:369 in locate
365│ if poetry_file.exists():
366│ return poetry_file
367│
368│ else:
→ 369│ raise RuntimeError(
370│ "Poetry could not find a pyproject.toml file in {} or its parents".format(
371│ cwd
372│ )
373│ )
I soon realised I needed my own made pyproject.toml file. Running poetry install again yielded:
$ poetry install
TOMLError
Invalid TOML file /home/me/.ssh/workers-python/workers/pyproject.toml: Key "json " already exists.
at ~/.poetry/lib/poetry/_vendor/py3.8/poetry/core/toml/file.py:34 in read
30│ def read(self): # type: () -> "TOMLDocument"
31│ try:
32│ return super(TOMLFile, self).read()
33│ except (ValueError, TOMLKitError) as e:
→ 34│ raise TOMLError("Invalid TOML file {}: {}".format(self.path.as_posix(), e))
35│
36│ def __getattr__(self, item): # type: (str) -> Any
37│ return getattr(self.__path, item)
38│
Above error indicates there were duplicate entries.
Running poetry install again with the now updated pyproject.toml file in cwd threw this error (in the post's title):
$ poetry install
Creating virtualenv my_project-1_EUeV5I-py3.8 in /home/me/.cache/pypoetry/virtualenvs
Updating dependencies
Resolving dependencies... (28.4s)
SolverProblemError
Because my_project depends on string (*) which doesn't match any versions, version solving failed.
at ~/.poetry/lib/poetry/puzzle/solver.py:241 in _solve
237│ packages = result.packages
238│ except OverrideNeeded as e:
239│ return self.solve_in_compatibility_mode(e.overrides, use_latest=use_latest)
240│ except SolveFailure as e:
→ 241│ raise SolverProblemError(e)
242│
243│ results = dict(
244│ depth_first_search(
245│ PackageNode(self._package, packages), aggregate_package_nodes
However, temporarily removing all instances = "*" gave me this error of \n on line 12... which doesn't appear to be there:
$ poetry install
TOMLError
Invalid TOML file /home/me/.ssh/workers-python/workers/pyproject.toml: Unexpected character: '\n' at line 12 col 5
at ~/.poetry/lib/poetry/_vendor/py3.8/poetry/core/toml/file.py:34 in read
30│ def read(self): # type: () -> "TOMLDocument"
31│ try:
32│ return super(TOMLFile, self).read()
33│ except (ValueError, TOMLKitError) as e:
→ 34│ raise TOMLError("Invalid TOML file {}: {}".format(self.path.as_posix(), e))
35│
36│ def __getattr__(self, item): # type: (str) -> Any
37│ return getattr(self.__path, item)
38│
me#LAPTOP-G1DAPU88:~/.ssh/workers-python/workers$ cat pyproject.toml
[tool.poetry]
name = "my_project"
version = "0.1.0"
description = "Top-level package for my_project."
authors = [""]
packages = [
{ include = "my_project"},
]
[tool.poetry.dependencies]
python = "^3.8"
click # Suspect line
I have reverted this.
Current pyproject.toml:
[tool.poetry]
name = "data_simulator"
version = "0.1.0"
description = "Top-level package for data_simulator."
authors = ["iotahoe <iotahoe#iotahoe.com>"] # daniel.bell#hitachivantara.com / daniel#iotahoe.com
packages = [
{ include = "data_simulator"},
]
[tool.poetry.dependencies]
python = "^3.8"
click = "*"
#logging = "*"
#os = "*"
#pathlib = "*"
#time = "*"
numpy = "*"
pandas = "*"
#json = "*"
#random = "*"
faker = "*"
transformers = "4.4.2"
#re = "*"
#itertools = "*"
#datetime = "*"
#requests = "*"
#copy = "*"
#collections = "*"
#collections.abc = "*"
#multiprocessing = "*"
#multiprocessing.dummy = "*"
nltk = "*"
#nltk.corpus = "*"
#string = "*"
[tool.poetry.dev-dependencies]
isort = "5.6.4"
black = "^20.8b1"
invoke = "^1.4.1"
coveralls = "^2.2.0"
pytest = "^3.0"
flake8 = "^3.8.3"
mypy = "^0.782"
[[tool.poetry.source]]
name = "azure"
url = "https://pkgs.dev.azure.com/iotahoe/Halo/_packaging/private-sources/pypi/simple/"
secondary = true
[build-system]
requires = ["poetry>=0.12"]
build-backend = "poetry.masonry.api"
Note: 'name', 'authors', 'include', 'url' have been censored.
As a general advise I recommend to use poetry's command line instead of creating/manipulating the pyproject.toml.
Start with a poetry init or poetry init -n and add your dependencies with poetry add.
The problem with your current pyproject.toml is, that you declare built-in packages as dependencies, like os, pathlib, string and others. This is why you receive the message Because my_project depends on string (*) which doesn't match any versions, version solving failed., which means poetry cannot find any matching package information in the repository.
tl;dr: Flush the *.egg-info directories before running poetry lock.
This answer is not strictly related to the current issue, but a similar error message can appear in other circumstances, so I think it's valuable to share it here.
If you are locking in a project where sub-dependencies are directly available on the file system, some *.egg-info directories may interfere with the locking process, causing issues when trying to run poetry install in a context where those *.egg-info files are missing. To avoid the problem: Flush the *.egg-info directories prior to locking. You should then have an updated poetry.lock file with more content.
ref: https://github.com/louking/loutilities/tree/0.14.7
I am getting an importerror after pip install of this package. When I use easy_install of the egg all is well. Any help on how to debug would be appreciated.
Used following commands to install this package to pypi, and pip install using "pip 18.1 from c:\users\lking\anaconda2\lib\site-packages\pip (python 2.7)"
python setup.py install
python setup.py sdist bdist_wheel
twine upload dist/loutilities-0.14.7.*
:
# later
pip install loutilities
When I run the project, see the following
NoAppException: While importing "run", an ImportError was raised:
Traceback (most recent call last):
File "C:\Users\lking\Documents\Lou's Software\projects\contracts\contracts\venv\lib\site-packages\flask\cli.py", line 235, in locate_app
__import__(module_name)
File "C:\Users\lking\Documents\Lou's Software\projects\contracts\contracts\run.py", line 31, in <module>
app = create_app(Development(configpath), configpath)
File "C:\Users\lking\Documents\Lou's Software\projects\contracts\contracts\contracts\__init__.py", line 82, in create_app
from contracts.views.frontend import bp as frontend
File "C:\Users\lking\Documents\Lou's Software\projects\contracts\contracts\contracts\views\frontend\__init__.py", line 18, in <module>
import frontend
File "C:\Users\lking\Documents\Lou's Software\projects\contracts\contracts\contracts\views\frontend\frontend.py", line 21, in <module>
from loutilities.flask_helpers.blueprints import add_url_rules
ImportError: No module named flask_helpers.blueprints
find_packages seems to be finding the appropriate packages:
from setuptools import find_packages
find_packages()
Out[4]: ['loutilities', 'tests', 'loutilities.flask_helpers']
setup.py:
#!/usr/bin/python
# [irrelevant comments deleted]
import glob
import pdb
# home grown
from loutilities import version
from setuptools import setup, find_packages
def globit(dir, filelist):
outfiles = []
for file in filelist:
filepath = '{0}/{1}'.format(dir,file)
gfilepath = glob.glob(filepath)
for i in range(len(gfilepath)):
f = gfilepath[i][len(dir)+1:]
gfilepath[i] = '{0}/{1}'.format(dir,f) # if on windows, need to replace backslash with frontslash
outfiles += [gfilepath[i]]
return (dir, outfiles)
setup(
name = "loutilities",
version = version.__version__,
packages = find_packages(),
# include_package_data = True,
scripts = [
'loutilities/agegrade.py',
'loutilities/apikey.py',
'loutilities/applytemplate.py',
'loutilities/filtercsv.py',
'loutilities/makerst.py',
],
# Project uses reStructuredText, so ensure that the docutils get
# installed or upgraded on the target machine
install_requires = [
'unicodecsv>=0.13.0',
],
# If any package contains any of these file types, include them:
data_files = ([
globit('loutilities', ['*.conf','*.pyc','*.pyd','*.dll','*.h','*.xlsx']),
globit('doc/source', ['*.txt', '*.rst', '*.html', '*.css', '*.js', '*.png', '*.py', ]),
globit('doc/build/html', ['*.txt', '*.rst', '*.html', '*.css', '*.js', '*.png', ]),
globit('doc/build/html/_sources', ['*.txt', '*.rst', '*.html', '*.css', '*.js', '*.png', ]),
globit('doc/build/html/_static', ['*.txt', '*.rst', '*.html', '*.css', '*.js', '*.png', ]),
globit('doc/build/html/_images', ['*.png', ]),
]),
entry_points = {
'console_scripts': [
'agegrade = loutilities.agegrade:main',
'apikey = loutilities.apikey:main',
'applytemplate = loutilities.applytemplate:main',
'filtercsv = loutilities.filtercsv:main',
'makerst = loutilities.makerst:main',
],
},
zip_safe = False,
# metadata for upload to PyPI
description = 'some hopefully useful utilities',
long_description=open("README.md").read(),
license = 'Apache License, Version 2.0',
author = 'Lou King',
author_email = 'lking#pobox.com',
url = 'http://github.com/louking/loutilities',
# could also include long_description, download_url, classifiers, etc.
)
After pip install loutilities I see an unknown directory loutilities at the top-level of my virtual environment. The directory contains a few *.pyc but not any subdirectory. I suspect import loutilities.flask_helpers tries to find a subdirectory loutilities/flask_helpers and fails.
I think the top-level directory is from the code
globit('loutilities', ['*.conf','*.pyc','*.pyd','*.dll','*.h','*.xlsx']),
I suspect there is a bug in globit() that puts the data in the wrong directory.
This a section in a schema file
imports:
- path: configs/folder1/resources/gcpresource/test/*
I am trying to import all the files in a folder using the template's schema file.
I know that this does not work.
My question is,
what is a better way to import all the files using * or something that's suitable so that deployment manager can import all of the files in a folder without having to explicitly specify them ?
You could do so in python if they are all yaml config files
# my-template.yaml
imports:
- path: omni-importer.py
And in the middleware-like python template do something like:
# omni-importer.py
import yaml
from os import listdir
from os.path import isfile, join
def generate_config(ctx):
mypath = ctx.properties['path']
files = [f for f in listdir(mypath) if isfile(join(mypath, f)) and f.endswith('yaml')]
yamls = map(lambda f: yaml.load(open('test.txt')['resources']), files)
return {
'resources': yamls,
}
This is not working code, but rather a proof of concept
from cx_Freeze import setup, Executable
# Dependencies are automatically detected, but it might need
# fine tuning.
buildOptions = dict(packages = ["idna","lib","gui","plugins"], excludes = ["Tcl","tcl"]
import sys
base = 'Win32GUI' if sys.platform=='win32' else None
executables = [
Executable('electrum-xvg', base=base, targetName = 'Electrum XVG',icon="electrum.icns")]
setup(name='electrum-xvg',
version = '1.0',
description = '',
options = dict(build_exe = buildOptions),
executables = executables])
I have the above setup.py file which I am using to try build application on OSX Sierra. But when I use python setup.py bdist_mac it raises error
#rpath/libQtDeclarative.4.dylib
error: can't copy '#rpath/libQtDeclarative.4.dylib': doesn't exist or not a regular file
libQtDeclarative.4.dylib is present in ~/anaconda/envs/pyqtapp/lib on my system but when I used otool -D libQtDeclarative.4.dylib it raised error that no such file exists, so I used
install_name_tool -id "#rpath/libQtDeclarative.4.dylib" libQtDeclarative.4.dylib
in ~/anaconda/envs/pyqtapp/lib now when I run otool -D libQtDeclarative.4.dylib I get
libQtDeclarative.4.dylib:
#rpath/libQtDeclarative.4.dylib
but cx_Freeze still raises the error
error: can't copy '#rpath/libQtDeclarative.4.dylib': doesn't exist or not a regular file
Try explicitly setting includes (list of relative paths):
includefiles = ['README.txt', 'CHANGELOG.txt', 'helpers\uncompress\unRAR.exe', , 'helpers\uncompress\unzip.exe']
I write the application using modules PyQt4, cv2(v.2.4.13), numpy(v.1.11.1) and etc. I use Python (2.7.11 on win32), Windows7 (x64).
Before the compiling (using py2exe) on Windows my application work correctly (run from python).
In setup.py:
...
options = {
'py2exe': {
...
'compressed': True,
'includes': ['cv2', 'numpy', 'sip', 'lxml._elementpath', 'PyQt4.QtCore', 'PyQt4.QtGui' ],
'dll_excludes': ['MSVCP90.dll', 'w9xpopen.exe', 'Qwt.pyd', 'tcl85.dll', 'tk85.dll']
}
}
...
After compiling on Windows and trying to start the apps (*.exe file) the system return this error:
File "cv2.pyc", line 12, in module
File "cv2.pyc", line 10, in __load
ImportError: numpy.core.multiarray failed to import.
The exception raised in line:
import cv2
In the folder named 'dist' there is file numpy.core.multiarray.pyd.
In library.zip there are file cv2, folder numpy.
SOLVED
The problem was in setup.py in the py2exe section.
Also I copied numpy-atlas.dll from C:\Python27\Lib\site-packages\numpy\core to site-packages for succesful build.
setup.py:
excludes = [
'Tkconstants', 'Tkinter', 'tcl', '_ssl', 'bz2',
'_testcapi', 'pyexpat', 'select'
]
options = {
'py2exe': {
'compressed': True,
'excludes': excludes,
'includes': ['sip', 'lxml._elementpath', 'PyQt4.QtCore', 'PyQt4.QtGui', 'cv2', 'numpy'],
'dll_excludes': ['MSVCP90.dll', 'w9xpopen.exe', 'Qwt.pyd', 'tcl85.dll', 'tk85.dll', 'MSVCR90.DLL']
}
}
I had a similar problem for me the solution was as easy as moving the image file to the folder where the executable was created. This worked for both py2exe and pyinstaller.