Running CI tests with PyQT - continuous-integration

I am setting up a CI Bitbucket pipeline for my team. We are using pytest and pytest-qt to test our software. The tests run locally without any issue, but the build fails using pipeline. Here is the pipeline yml:
image: python:3.9
pipelines:
default:
- parallel:
- step:
name: Test
caches:
- pip
script:
- if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- pip install pytest
- pip install pytest-qt
- pytest -v tools/IntegrationTests.py --junitxml=test-reports/report.xml
The file requirements.txt contains several modules including:
PyQt5==5.15.4
PyQt5-Qt5==5.15.2
And here is the output when trying to build:
+ pytest -v tools/IntegrationTests.py --junitxml=test-reports/report.xml
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR> File "/usr/local/lib/python3.9/site-packages/_pytest/main.py", line 265, in wrap_session
INTERNALERROR> config._do_configure()
INTERNALERROR> File "/usr/local/lib/python3.9/site-packages/_pytest/config/__init__.py", line 982, in _do_configure
INTERNALERROR> self.hook.pytest_configure.call_historic(kwargs=dict(config=self))
INTERNALERROR> File "/usr/local/lib/python3.9/site-packages/pluggy/hooks.py", line 308, in call_historic
INTERNALERROR> res = self._hookexec(self, self.get_hookimpls(), kwargs)
INTERNALERROR> File "/usr/local/lib/python3.9/site-packages/pluggy/manager.py", line 93, in _hookexec
INTERNALERROR> return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR> File "/usr/local/lib/python3.9/site-packages/pluggy/manager.py", line 84, in <lambda>
INTERNALERROR> self._inner_hookexec = lambda hook, methods, kwargs: hook.multicall(
INTERNALERROR> File "/usr/local/lib/python3.9/site-packages/pluggy/callers.py", line 208, in _multicall
INTERNALERROR> return outcome.get_result()
INTERNALERROR> File "/usr/local/lib/python3.9/site-packages/pluggy/callers.py", line 80, in get_result
INTERNALERROR> raise ex[1].with_traceback(ex[2])
INTERNALERROR> File "/usr/local/lib/python3.9/site-packages/pluggy/callers.py", line 187, in _multicall
INTERNALERROR> res = hook_impl.function(*args)
INTERNALERROR> File "/usr/local/lib/python3.9/site-packages/pytestqt/plugin.py", line 203, in pytest_configure
INTERNALERROR> qt_api.set_qt_api(config.getini("qt_api"))
INTERNALERROR> File "/usr/local/lib/python3.9/site-packages/pytestqt/qt_compat.py", line 104, in set_qt_api
INTERNALERROR> self.QtGui = _import_module("QtGui")
INTERNALERROR> File "/usr/local/lib/python3.9/site-packages/pytestqt/qt_compat.py", line 100, in _import_module
INTERNALERROR> m = __import__(_root_module, globals(), locals(), [module_name], 0)
INTERNALERROR> ImportError: libGL.so.1: cannot open shared object file: No such file or directory
What I get from that is that Qt is not properly installed remotely, thus the tests cannot run.
My question is: can GUI be tested in CI, and if yes, how?
EDIT:
I have broke down to test only the import of PyQt and it boiled down to this test suite:
def test_import_widgets():
from PyQt5 import QtWidgets
def test_import_core():
from PyQt5 import QtCore
def test_import_gui():
from PyQt5 import QtGui
def test_import_qt():
from PyQt5.QtCore import Qt
Which have resulted in the following results:
2 / 4 tests failed
BasicsTests.test_import_guitools
<1s
ImportError: libGL.so.1: cannot open shared object file: No such file or directory
def test_import_gui():
> from PyQt5 import QtGui
E ImportError: libGL.so.1: cannot open shared object file: No such file or directory
tools/BasicsTests.py:50: ImportError
BasicsTests.test_import_widgetstools
<1s
ImportError: libGL.so.1: cannot open shared object file: No such file or directory
def test_import_widgets():
> from PyQt5 import QtWidgets
E ImportError: libGL.so.1: cannot open shared object file: No such file or directory
tools/BasicsTests.py:42: ImportError

You have to install the Qt dependencies:
image: python:3.9
pipelines:
default:
- parallel:
- step:
name: Test
caches:
- pip
script:
- apt-get update && apt-get autoclean
- apt-get install -y '^libxcb.*-dev' libx11-xcb-dev libglu1-mesa-dev libxrender-dev libxi-dev libxkbcommon-dev libxkbcommon-x11-dev
- if [ -f requirements.txt ]; then pip install -r requirements.txt; fi
- pip install pytest
- pip install pytest-qt
- pytest -v tools/IntegrationTests.py --junitxml=test-reports/report.xml

Related

pip install command creates syntax error : Traceback (most recent call last)

I've been trying to install bs4 and selenium from the windows powershell, and here is the error message I get :
ERROR: Exception:
Traceback (most recent call last):
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_internal\cli\base_command.py", line 160, in exc_logging_wrapper
status = run_func(*args)
^^^^^^^^^^^^^^^
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_internal\cli\req_command.py", line 247, in wrapper
return func(self, options, args)
^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_internal\commands\install.py", line 326, in run
session = self.get_default_session(options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_internal\cli\req_command.py", line 98, in get_default_session
self._session = self.enter_context(self._build_session(options))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_internal\cli\req_command.py", line 125, in _build_session
session = PipSession(
^^^^^^^^^^^
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_internal\network\session.py", line 343, in __init__
self.headers["User-Agent"] = user_agent()
^^^^^^^^^^^^
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_internal\network\session.py", line 175, in user_agent
setuptools_dist = get_default_environment().get_distribution("setuptools")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_internal\metadata\importlib\_envs.py", line 188, in get_distribution
return next(matches, None)
^^^^^^^^^^^^^^^^^^^
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_internal\metadata\importlib\_envs.py", line 183, in <genexpr>
matches = (
^
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_internal\metadata\base.py", line 612, in iter_all_distributions
for dist in self._iter_distributions():
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_internal\metadata\importlib\_envs.py", line 176, in _iter_distributions
for dist in finder.find_eggs(location):
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_internal\metadata\importlib\_envs.py", line 146, in find_eggs
yield from self._find_eggs_in_zip(location)
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_internal\metadata\importlib\_envs.py", line 123, in _find_eggs_in_zip
from pip._vendor.pkg_resources import find_eggs_in_zip
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 3251, in <module>
#_call_aside
^^^^^^^^^^^
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 3235, in _call_aside
f(*args, **kwargs)
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 3277, in _initialize_master_working_set
tuple(
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 3278, in <genexpr>
dist.activate(replace=False)
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 2785, in activate
declare_namespace(pkg)
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 2284, in declare_namespace
_handle_ns(packageName, path_item)
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pip\_vendor\pkg_resources\__init__.py", line 2217, in _handle_ns
loader.load_module(packageName)
File "C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pygoogle\google.py", line 113
print """PyGoogle %(__version__)s
^^^^^^^^^^^^^^^^^^^^^^^^^^
SyntaxError: Missing parentheses in call to 'print'. Did you mean print(...)?
I cannot install any pip package in my pc it returns that error
pip version is : pip 22.3.1
python version is : Python 3.11.1
Appreciate Your support .
I tried to install any pip package
You've installed a package PyGoogle which is very old (2006) and works only with Python 2. You need to uninstall it:
pip uninstall -y pygoogle
If that doesn't work — remove the directory C:\Users\T470\AppData\Local\Programs\Python\Python311\Lib\site-packages\pygoogle

Django Celery issues within Docker

Having a major issue I cannot track down and I'm spinning my wheels. Any help would be greatly appreciated. I have django, celerybeat and celeryworker containers that all use the same base image of django.
local.yml
version: "3"
volumes:
local_postgres_data: {}
local_postgres_data_backups: {}
services:
django: &django
build:
context: .
dockerfile: ./compose/local/django/Dockerfile
image: local_django
container_name: django
depends_on:
- postgres
- mailhog
- redis
volumes:
- .:/app
env_file:
- ./.envs/.local/.django
- ./.envs/.local/.postgres
ports:
- "8000:8000"
command: /start
restart: always
postgres:
build:
context: .
dockerfile: ./compose/production/postgres/Dockerfile
image: production_postgres
container_name: postgres
volumes:
- local_postgres_data:/var/lib/postgresql/data
- local_postgres_data_backups:/backups
env_file:
- ./.envs/.local/.postgres
ports:
- "5432:5432"
mailhog:
image: mailhog/mailhog:v1.0.0
container_name: mailhog
ports:
- "8025:8025"
logging:
driver: "none" # disable saving logs
redis:
image: redis:6
container_name: redis
celeryworker:
<<: *django
image: local_celeryworker
container_name: celeryworker
depends_on:
- redis
- postgres
- mailhog
ports: []
command: /start-celeryworker
restart: always
celerybeat:
<<: *django
image: local_celerybeat
container_name: celerybeat
depends_on:
- redis
- postgres
- mailhog
ports: []
command: /start-celerybeat
restart: always
node:
build:
context: .
dockerfile: ./compose/local/node/Dockerfile
image: local_node
container_name: node
depends_on:
- django
volumes:
- .:/app
# http://jdlm.info/articles/2016/03/06/lessons-building-node-app-docker.html
- /app/node_modules
command: npm run dev
ports:
- "3000:3000"
# Expose browsersync UI: https://www.browsersync.io/docs/options/#option-ui
- "3001:3001"
compose/local/django/Dockerfile
ARG PYTHON_VERSION=3.9-slim-bullseye
# define an alias for the specfic python version used in this file.
FROM python:${PYTHON_VERSION} as python
# Python build stage
FROM python as python-build-stage
ARG BUILD_ENVIRONMENT=local
# Install apt packages
RUN apt-get update && apt-get install --no-install-recommends -y \
# dependencies for building Python packages
build-essential \
# psycopg2 dependencies
libpq-dev
# Requirements are installed here to ensure they will be cached.
COPY ./requirements .
# Create Python Dependency and Sub-Dependency Wheels.
RUN pip wheel --wheel-dir /usr/src/app/wheels \
-r ${BUILD_ENVIRONMENT}.txt
# Python 'run' stage
FROM python as python-run-stage
ARG BUILD_ENVIRONMENT=local
ARG APP_HOME=/app
ENV PYTHONUNBUFFERED 1
ENV PYTHONDONTWRITEBYTECODE 1
ENV BUILD_ENV ${BUILD_ENVIRONMENT}
WORKDIR ${APP_HOME}
# Install required system dependencies
RUN apt-get update && apt-get install --no-install-recommends -y \
# psycopg2 dependencies
libpq-dev \
# Translations dependencies
gettext \
# cleaning up unused files
&& apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false \
&& rm -rf /var/lib/apt/lists/*
# All absolute dir copies ignore workdir instruction. All relative dir copies are wrt to the workdir instruction
# copy python dependency wheels from python-build-stage
COPY --from=python-build-stage /usr/src/app/wheels /wheels/
# use wheels to install python dependencies
RUN pip install --no-cache-dir --no-index --find-links=/wheels/ /wheels/* \
&& rm -rf /wheels/
COPY ./compose/production/django/entrypoint /entrypoint
RUN sed -i 's/\r$//g' /entrypoint
RUN chmod +x /entrypoint
COPY ./compose/local/django/start /start
RUN sed -i 's/\r$//g' /start
RUN chmod +x /start
COPY ./compose/local/django/celery/worker/start /start-celeryworker
RUN sed -i 's/\r$//g' /start-celeryworker
RUN chmod +x /start-celeryworker
COPY ./compose/local/django/celery/beat/start /start-celerybeat
RUN sed -i 's/\r$//g' /start-celerybeat
RUN chmod +x /start-celerybeat
COPY ./compose/local/django/celery/flower/start /start-flower
RUN sed -i 's/\r$//g' /start-flower
RUN chmod +x /start-flower
# copy application code to WORKDIR
COPY . ${APP_HOME}
ENTRYPOINT ["/entrypoint"]
config/celery_app.py
import os
from celery import Celery
# set the default Django settings module for the 'celery' program.
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "config.settings.local")
app = Celery("myapp")
# Using a string here means the worker doesn't have to serialize
# the configuration object to child processes.
# - namespace='CELERY' means all celery-related configuration keys
# should have a `CELERY_` prefix.
app.config_from_object("django.conf:settings", namespace="CELERY")
# Load task modules from all registered Django app configs.
app.autodiscover_tasks()
config/__init__.py
# This will make sure the app is always imported when
# Django starts so that shared_task will use this app.
from .celery_app import app as celery_app
__all__ = ("celery_app",)
compose/local/django/celery/beat/start
#!/bin/bash
set -o errexit
set -o nounset
rm -f './celerybeat.pid'
celery -A config.celery_app beat -l INFO
compose/local/django/celery/worker/start
#!/bin/bash
set -o errexit
set -o nounset
watchgod celery.__main__.main --args -A config.celery_app worker -l INFO
When running docker-compose up without celeryworker or celerybeat the modules are found fine in the django container. But as soon as I add the celeryworker or celeryworker I get the following about not being able to find modules.
celery beat v5.2.7 (dawn-chorus) is starting.
2022-08-09 14:07:41,879 [celery.utils.dispatch.signal:280] ERROR - Signal handler <bound method DjangoFixup.on_import_modules of <celery.fixups.django.DjangoFixup object at 0x7f6cff68e670>> raised: ModuleNotFoundError("No module named 'customadmin'")
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/celery/utils/dispatch/signal.py", line 276, in send
response = receiver(signal=self, sender=sender, **named)
File "/usr/local/lib/python3.9/site-packages/celery/fixups/django.py", line 82, in on_import_modules
self.worker_fixup.validate_models()
File "/usr/local/lib/python3.9/site-packages/celery/fixups/django.py", line 120, in validate_models
self.django_setup()
File "/usr/local/lib/python3.9/site-packages/celery/fixups/django.py", line 116, in django_setup
django.setup()
File "/usr/local/lib/python3.9/site-packages/django/__init__.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File "/usr/local/lib/python3.9/site-packages/django/apps/registry.py", line 91, in populate
app_config = AppConfig.create(entry)
File "/usr/local/lib/python3.9/site-packages/django/apps/config.py", line 224, in create
import_module(entry)
File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'customadmin'
2022-08-09 14:07:41,880 [celery.utils.dispatch.signal:280] ERROR - Signal handler <promise#0x7f6cff6491f0 --> <bound method Celery._autodiscover_tasks of <Celery bahamar at 0x7f6cff68eee0>>> raised: AppRegistryNotReady("Apps aren't loaded yet.")
Traceback (most recent call last):
File "/usr/local/lib/python3.9/site-packages/celery/utils/dispatch/signal.py", line 276, in send
response = receiver(signal=self, sender=sender, **named)
File "/usr/local/lib/python3.9/site-packages/vine/promises.py", line 160, in __call__
return self.throw()
File "/usr/local/lib/python3.9/site-packages/vine/promises.py", line 157, in __call__
retval = fun(*final_args, **final_kwargs)
File "/usr/local/lib/python3.9/site-packages/celery/app/base.py", line 689, in _autodiscover_tasks
return self._autodiscover_tasks_from_fixups(related_name)
File "/usr/local/lib/python3.9/site-packages/celery/app/base.py", line 698, in _autodiscover_tasks_from_fixups
return self._autodiscover_tasks_from_names([
File "/usr/local/lib/python3.9/site-packages/celery/app/base.py", line 701, in <listcomp>
for pkg in fixup.autodiscover_tasks()
File "/usr/local/lib/python3.9/site-packages/celery/fixups/django.py", line 92, in autodiscover_tasks
return [config.name for config in apps.get_app_configs()]
File "/usr/local/lib/python3.9/site-packages/django/apps/registry.py", line 145, in get_app_configs
self.check_apps_ready()
File "/usr/local/lib/python3.9/site-packages/django/apps/registry.py", line 136, in check_apps_ready
raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.
Traceback (most recent call last):
File "/usr/local/bin/celery", line 8, in <module>
sys.exit(main())
File "/usr/local/lib/python3.9/site-packages/celery/__main__.py", line 15, in main
sys.exit(_main())
File "/usr/local/lib/python3.9/site-packages/celery/bin/celery.py", line 217, in main
return celery(auto_envvar_prefix="CELERY")
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1055, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.9/site-packages/click/core.py", line 760, in invoke
return __callback(*args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/celery/bin/base.py", line 134, in caller
return f(ctx, *args, **kwargs)
File "/usr/local/lib/python3.9/site-packages/celery/bin/beat.py", line 72, in beat
return beat().run()
File "/usr/local/lib/python3.9/site-packages/celery/apps/beat.py", line 75, in run
self.init_loader()
File "/usr/local/lib/python3.9/site-packages/celery/apps/beat.py", line 124, in init_loader
self.app.loader.init_worker()
File "/usr/local/lib/python3.9/site-packages/celery/loaders/base.py", line 111, in init_worker
self.import_default_modules()
File "/usr/local/lib/python3.9/site-packages/celery/loaders/base.py", line 105, in import_default_modules
raise response
File "/usr/local/lib/python3.9/site-packages/celery/utils/dispatch/signal.py", line 276, in send
response = receiver(signal=self, sender=sender, **named)
File "/usr/local/lib/python3.9/site-packages/celery/fixups/django.py", line 82, in on_import_modules
self.worker_fixup.validate_models()
File "/usr/local/lib/python3.9/site-packages/celery/fixups/django.py", line 120, in validate_models
self.django_setup()
File "/usr/local/lib/python3.9/site-packages/celery/fixups/django.py", line 116, in django_setup
django.setup()
File "/usr/local/lib/python3.9/site-packages/django/__init__.py", line 24, in setup
apps.populate(settings.INSTALLED_APPS)
File "/usr/local/lib/python3.9/site-packages/django/apps/registry.py", line 91, in populate
app_config = AppConfig.create(entry)
File "/usr/local/lib/python3.9/site-packages/django/apps/config.py", line 224, in create
import_module(entry)
File "/usr/local/lib/python3.9/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
File "<frozen importlib._bootstrap>", line 984, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'customadmin'
customadmin just so happens to be the first app of many apps in my LOCAL_APPS.
Django 3.2.14
celery 5.2.7
django-celery-beat 2.3.0
django-celery-results 2.2.0

Can't install gevent on termux, anyone can help me?

~ $ pip install gevent
Collecting gevent
Using cached gevent-21.12.0.tar.gz (6.2 MB)
Installing build dependencies ... done Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... done
Collecting zope.event
Using cached zope.event-4.5.0-py2.py3-none-any.whl (6.8 kB)
Requirement already satisfied: setuptools in /data/data/com.termux/files/usr/lib/python3.10/site-packages (from gevent) (58.1.0)
Collecting zope.interface
Using cached zope.interface-5.4.0-cp310-cp310-linux_aarch64.whl
Requirement already satisfied: greenlet<2.0,>=1.1.0 in /data/data/com.termux/files/usr/lib/python3.10/site-packages (from gevent) (1.1.2)
Building wheels for collected packages: gevent
Building wheel for gevent (pyproject.toml) ... error
error: subprocess-exited-with-error
× Building wheel for gevent (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [63 lines of output]
running bdist_wheel
running build
running build_py
running build_ext
generating cffi module 'build/temp.linux-aarch64-3.10/gevent.libuv._corecffi.c'
Running '(cd
"/data/data/com.termux/files/usr/tmp/pip-install-55j041qz/gevent_018671f4e2174ad19170736008c5782a/deps/libev" && sh ./configure -C
configure-output.txt )' in /data/data/com.termux/files/usr/tmp/pip-install-55j041qz/gevent_018671f4e2174ad19170736008c5782a
configure: error: no acceptable ld found in $PATH
Traceback (most recent call last):
File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 363, in
main()
File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 345, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/data/data/com.termux/files/usr/lib/python3.10/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 261, in build_wheel
return _build_backend().build_wheel(wheel_directory, config_settings,
File "/data/data/com.termux/files/usr/tmp/pip-build-env-l1wl_veu/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 230, in build_wheel
return self._build_with_temp_dir(['bdist_wheel'], '.whl',
File "/data/data/com.termux/files/usr/tmp/pip-build-env-l1wl_veu/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 215, in _build_with_temp_dir
self.run_setup()
File "/data/data/com.termux/files/usr/tmp/pip-build-env-l1wl_veu/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 267, in run_setup
super(_BuildMetaLegacyBackend,
File "/data/data/com.termux/files/usr/tmp/pip-build-env-l1wl_veu/overlay/lib/python3.10/site-packages/setuptools/build_meta.py", line 158, in run_setup
exec(compile(code, __file__, 'exec'), locals())
File "setup.py", line 481, in <module>
run_setup(EXT_MODULES)
File "setup.py", line 348, in run_setup
setup(
File "/data/data/com.termux/files/usr/tmp/pip-build-env-l1wl_veu/overlay/lib/python3.10/site-packages/setuptools/__init__.py", line 155, in setup
return distutils.core.setup(**attrs)
File "/data/data/com.termux/files/usr/tmp/pip-build-env-l1wl_veu/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 148, in setup
return run_commands(dist)
File "/data/data/com.termux/files/usr/tmp/pip-build-env-l1wl_veu/overlay/lib/python3.10/site-packages/setuptools/_distutils/core.py", line 163, in run_commands
dist.run_commands()
File "/data/data/com.termux/files/usr/tmp/pip-build-env-l1wl_veu/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 967, in run_commands
self.run_command(cmd)
File "/data/data/com.termux/files/usr/tmp/pip-build-env-l1wl_veu/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
cmd_obj.run()
File "/data/data/com.termux/files/usr/tmp/pip-build-env-l1wl_veu/overlay/lib/python3.10/site-packages/wheel/bdist_wheel.py", line 299, in run
self.run_command('build')
File "/data/data/com.termux/files/usr/tmp/pip-build-env-l1wl_veu/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/data/data/com.termux/files/usr/tmp/pip-build-env-l1wl_veu/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
cmd_obj.run()
File "/data/data/com.termux/files/usr/tmp/pip-build-env-l1wl_veu/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/data/data/com.termux/files/usr/tmp/pip-build-env-l1wl_veu/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/data/data/com.termux/files/usr/tmp/pip-build-env-l1wl_veu/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
cmd_obj.run()
File "/data/data/com.termux/files/usr/tmp/pip-build-env-l1wl_veu/overlay/lib/python3.10/site-packages/cffi/setuptools_ext.py", line 143, in run
ext.sources[0] = make_mod(self.build_temp, pre_run)
File "/data/data/com.termux/files/usr/tmp/pip-build-env-l1wl_veu/overlay/lib/python3.10/site-packages/cffi/setuptools_ext.py", line 128, in make_mod
pre_run(ext, ffi)
File "/data/data/com.termux/files/usr/tmp/pip-install-55j041qz/gevent_018671f4e2174ad19170736008c5782a/_setuputils.py", line 364, in pre_run
action()
File "/data/data/com.termux/files/usr/tmp/pip-install-55j041qz/gevent_018671f4e2174ad19170736008c5782a/_setuplibev.py", line 55, in configure_libev
system(libev_configure_command)
File "/data/data/com.termux/files/usr/tmp/pip-install-55j041qz/gevent_018671f4e2174ad19170736008c5782a/_setuputils.py", line 195, in system
if _system(cmd, cwd=cwd, env=env, **kwargs):
File "/data/data/com.termux/files/usr/tmp/pip-install-55j041qz/gevent_018671f4e2174ad19170736008c5782a/_setuputils.py", line 191, in _system
return check_call(cmd, cwd=cwd, env=env, **kwargs)
File "/data/data/com.termux/files/usr/lib/python3.10/subprocess.py", line 369, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '(cd "/data/data/com.termux/files/usr/tmp/pip-install-55j041qz/gevent_018671f4e2174ad19170736008c5782a/deps/libev" && sh ./configure -C > configure-output.txt )' returned non-zero exit status 1.
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
ERROR: Failed building wheel for gevent
Failed to build gevent
ERROR: Could not build wheels for gevent, which is required to install pyproject.toml-based projects
~ $
try lower versions
https://pypi.org/project/gevent/#history
I have tried all the versions I see 20.4.0 Is the best
Try:
pkg install binutils
This worked for me.

Why is pipenv failing to install dependencies in production environment but not locally?

So I attempting to debug a deployment on AWS EB -> Elastic BeanStalk Django Error in Pipfile
And it seems like the issue is being caused by pipenv
error Command /bin/sh -c /usr/bin/python3.8 -m pipenv install --skip-lock
However, when I run that command locally it works fine, so I dont know what to do to fix this or investigate further on the cause of the issue.
My pipfile is:
[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true
[dev-packages]
[packages]
appdirs = "==1.4.4"
asgiref = "==3.3.1"
"backports.entry-points-selectable" = "==1.1.0"
certifi = "==2020.11.8"
chardet = "==3.0.4"
distlib = "==0.3.1"
django-cors-headers = "==3.6.0"
djangorestframework = "==3.12.2"
filelock = "==3.0.12"
idna = "==2.10"
pipenv = "==2020.8.13"
platformdirs = "==2.2.0"
pytz = "==2020.4"
requests = "==2.25.0"
six = "==1.15.0"
sqlparse = "==0.4.1"
stripe = "==2.55.1"
urllib3 = "==1.26.2"
virtualenv = "==20.7.2"
virtualenv-clone = "==0.5.6"
Django = "==3.0.8"
Pillow = "==8.0.1"
[requires]
python_version = "3.8"
And the full error log is:
----------------------------------------
/var/log/eb-engine.log
----------------------------------------
Downloading Pillow-8.0.1-cp38-cp38-manylinux1_x86_64.whl (2.2 MB)
Collecting pipenv==2020.11.15
Downloading pipenv-2020.11.15-py2.py3-none-any.whl (3.9 MB)
Collecting pytz==2020.4
Downloading pytz-2020.4-py2.py3-none-any.whl (509 kB)
Collecting requests==2.25.0
Downloading requests-2.25.0-py2.py3-none-any.whl (61 kB)
Collecting six==1.15.0
Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting sqlparse==0.4.1
Downloading sqlparse-0.4.1-py3-none-any.whl (42 kB)
Collecting stripe==2.55.1
Downloading stripe-2.55.1-py2.py3-none-any.whl (204 kB)
Collecting urllib3==1.26.2
Downloading urllib3-1.26.2-py2.py3-none-any.whl (136 kB)
Collecting virtualenv==20.2.1
Downloading virtualenv-20.2.1-py2.py3-none-any.whl (4.9 MB)
Collecting virtualenv-clone==0.5.4
Downloading virtualenv_clone-0.5.4-py2.py3-none-any.whl (6.6 kB)
Requirement already satisfied: pip>=18.0 in /var/app/venv/staging-LQM1lest/lib/python3.8/site-packages (from pipenv==2020.11.15->-r requirements.txt (line 12)) (21.1.3)
Requirement already satisfied: setuptools>=36.2.1 in /var/app/venv/staging-LQM1lest/lib/python3.8/site-packages (from pipenv==2020.11.15->-r requirements.txt (line 12)) (57.1.0)
Installing collected packages: urllib3, sqlparse, six, pytz, idna, filelock, distlib, chardet, certifi, asgiref, appdirs, virtualenv-clone, virtualenv, requests, Django, stripe, pipenv, Pillow, djangorestframework, django-cors-headers
Successfully installed Django-3.0.8 Pillow-8.0.1 appdirs-1.4.4 asgiref-3.3.1 certifi-2020.11.8 chardet-3.0.4 distlib-0.3.1 django-cors-headers-3.6.0 djangorestframework-3.12.2 filelock-3.0.12 idna-2.10 pipenv-2020.11.15 pytz-2020.4 requests-2.25.0 six-1.15.0 sqlparse-0.4.1 stripe-2.55.1 urllib3-1.26.2 virtualenv-20.2.1 virtualenv-clone-0.5.4
2021/08/13 12:49:39.051999 [INFO] Installing dependencies with Pipfile.lock
2021/08/13 12:49:39.052013 [INFO] Running command /bin/sh -c /usr/bin/python3.8 -m pipenv install --ignore-pipfile
2021/08/13 12:49:40.248089 [INFO] Installing dependencies with Pipfile
2021/08/13 12:49:40.248118 [INFO] Running command /bin/sh -c /usr/bin/python3.8 -m pipenv install --skip-lock
2021/08/13 12:49:40.976863 [INFO] Installing dependencies from Pipfile…
2021/08/13 12:49:40.976902 [ERROR] An error occurred during execution of command [app-deploy] - [InstallDependency]. Stop running the command. Error: fail to install dependencies with Pipfile file with error Command /bin/sh -c /usr/bin/python3.8 -m pipenv install --skip-lock failed with error exit status 1. Stderr:Traceback (most recent call last):
File "/usr/lib64/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib64/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.8/site-packages/pipenv/__main__.py", line 5, in <module>
cli()
File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/decorators.py", line 73, in new_func
return ctx.invoke(f, obj, *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pipenv/cli/command.py", line 232, in install
retcode = do_install(
File "/usr/local/lib/python3.8/site-packages/pipenv/core.py", line 2052, in do_install
do_init(
File "/usr/local/lib/python3.8/site-packages/pipenv/core.py", line 1305, in do_init
do_install_dependencies(
File "/usr/local/lib/python3.8/site-packages/pipenv/core.py", line 827, in do_install_dependencies
lockfile = project.get_or_create_lockfile(from_pipfile=True)
File "/usr/local/lib/python3.8/site-packages/pipenv/project.py", line 783, in get_or_create_lockfile
lockfile_dict.update({"_meta": self.get_lockfile_meta()})
File "/usr/local/lib/python3.8/site-packages/pipenv/project.py", line 817, in get_lockfile_meta
sources = self.lockfile_content.get("_meta", {}).get("sources", [])
File "/usr/local/lib/python3.8/site-packages/pipenv/project.py", line 662, in lockfile_content
return self.load_lockfile()
File "/usr/local/lib/python3.8/site-packages/pipenv/project.py", line 1044, in load_lockfile
j = json.load(lock)
File "/usr/lib64/python3.8/json/__init__.py", line 293, in load
return loads(fp.read(),
File "/usr/lib64/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python3.8/json/decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 4 column 1 (char 35)
2021/08/13 12:49:40.976920 [INFO] Executing cleanup logic
2021/08/13 12:49:40.977012 [INFO] CommandService Response: {"status":"FAILURE","api_version":"1.0","results":[{"status":"FAILURE","msg":"Engine execution has encountered an error.","returncode":1,"events":[{"msg":"Instance deployment failed to install application dependencies. The deployment failed.","timestamp":1628858980,"severity":"ERROR"},{"msg":"Instance deployment failed. For details, see 'eb-engine.log'.","timestamp":1628858980,"severity":"ERROR"}]}]}
Additionally, I am operating inside a virtualenv and the command I mentioned gives me the following warning
Courtesy Notice: Pipenv found itself running within a virtual environment, so it will automatically use that environment, instead of creating its own for any project. You can set PIPENV_IGNORE_VIRTUALENVS=1 to force pipenv to ignore that environment and create its own instead. You can set PIPENV_VERBOSITY=-1 to suppress this warning.
I have tried to run it outside the virtualenv but can't seem to get the correct version of pipenv I have version 2021.5.29 and I need 2020.8.13 as specified by AWS in their documentation.
There is also a small difference between the python version as I use python3.8.11 and they use 3.8.5 does that make a difference?
Not a python export but the error message explicitly notes.
“Instance deployment failed to install application dependencies”
And reading the stack trace it notes.
“ json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 4 column 1 (char 35)”
Looks like you have malformed JSON in your lock file.

Failed to pip install georasters in MacOSX

It always failed to pip install georasters in my macbook.
The following is the Exception:
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/pip/basecommand.py", line 215, in main
status = self.run(options, args)
File "/Library/Python/2.7/site-packages/pip/commands/install.py", line 324, in run
requirement_set.prepare_files(finder)
File "/Library/Python/2.7/site-packages/pip/req/req_set.py", line 380, in prepare_files
ignore_dependencies=self.ignore_dependencies))
File "/Library/Python/2.7/site-packages/pip/req/req_set.py", line 634, in _prepare_file
abstract_dist.prep_for_dist()
File "/Library/Python/2.7/site-packages/pip/req/req_set.py", line 129, in prep_for_dist
self.req_to_install.run_egg_info()
File "/Library/Python/2.7/site-packages/pip/req/req_install.py", line 412, in run_egg_info
self.setup_py, self.name,
File "/Library/Python/2.7/site-packages/pip/req/req_install.py", line 387, in setup_py
import setuptools # noqa
File "build/bdist.macosx-10.12-intel/egg/setuptools/__init__.py", line 12, in <module>
File "build/bdist.macosx-10.12-intel/egg/setuptools/version.py", line 1, in <module>
File "build/bdist.macosx-10.12-intel/egg/pkg_resources/__init__.py", line 72, in <module>
File "build/bdist.macosx-10.12-intel/egg/packaging/requirements.py", line 59, in <module>
TypeError: __call__() takes exactly 2 arguments (1 given)
pip --version:
pip 9.0.1 from /Library/Python/2.7/site-packages (python 2.7)
After many times trying, I resolved the problem.
I used the command:pip install --no-deps georasters
https://pip.pypa.io/en/stable/reference/pip_install/#installation-order[enter image description here][1]

Resources