Why is pipenv failing to install dependencies in production environment but not locally? - pip

Question

So I attempting to debug a deployment on AWS EB -> Elastic BeanStalk Django Error in Pipfile
And it seems like the issue is being caused by pipenv
error Command /bin/sh -c /usr/bin/python3.8 -m pipenv install --skip-lock
However, when I run that command locally it works fine, so I dont know what to do to fix this or investigate further on the cause of the issue.
My pipfile is:
[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true
[dev-packages]
[packages]
appdirs = "==1.4.4"
asgiref = "==3.3.1"
"backports.entry-points-selectable" = "==1.1.0"
certifi = "==2020.11.8"
chardet = "==3.0.4"
distlib = "==0.3.1"
django-cors-headers = "==3.6.0"
djangorestframework = "==3.12.2"
filelock = "==3.0.12"
idna = "==2.10"
pipenv = "==2020.8.13"
platformdirs = "==2.2.0"
pytz = "==2020.4"
requests = "==2.25.0"
six = "==1.15.0"
sqlparse = "==0.4.1"
stripe = "==2.55.1"
urllib3 = "==1.26.2"
virtualenv = "==20.7.2"
virtualenv-clone = "==0.5.6"
Django = "==3.0.8"
Pillow = "==8.0.1"
[requires]
python_version = "3.8"
And the full error log is:
----------------------------------------
/var/log/eb-engine.log
----------------------------------------
Downloading Pillow-8.0.1-cp38-cp38-manylinux1_x86_64.whl (2.2 MB)
Collecting pipenv==2020.11.15
Downloading pipenv-2020.11.15-py2.py3-none-any.whl (3.9 MB)
Collecting pytz==2020.4
Downloading pytz-2020.4-py2.py3-none-any.whl (509 kB)
Collecting requests==2.25.0
Downloading requests-2.25.0-py2.py3-none-any.whl (61 kB)
Collecting six==1.15.0
Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting sqlparse==0.4.1
Downloading sqlparse-0.4.1-py3-none-any.whl (42 kB)
Collecting stripe==2.55.1
Downloading stripe-2.55.1-py2.py3-none-any.whl (204 kB)
Collecting urllib3==1.26.2
Downloading urllib3-1.26.2-py2.py3-none-any.whl (136 kB)
Collecting virtualenv==20.2.1
Downloading virtualenv-20.2.1-py2.py3-none-any.whl (4.9 MB)
Collecting virtualenv-clone==0.5.4
Downloading virtualenv_clone-0.5.4-py2.py3-none-any.whl (6.6 kB)
Requirement already satisfied: pip>=18.0 in /var/app/venv/staging-LQM1lest/lib/python3.8/site-packages (from pipenv==2020.11.15->-r requirements.txt (line 12)) (21.1.3)
Requirement already satisfied: setuptools>=36.2.1 in /var/app/venv/staging-LQM1lest/lib/python3.8/site-packages (from pipenv==2020.11.15->-r requirements.txt (line 12)) (57.1.0)
Installing collected packages: urllib3, sqlparse, six, pytz, idna, filelock, distlib, chardet, certifi, asgiref, appdirs, virtualenv-clone, virtualenv, requests, Django, stripe, pipenv, Pillow, djangorestframework, django-cors-headers
Successfully installed Django-3.0.8 Pillow-8.0.1 appdirs-1.4.4 asgiref-3.3.1 certifi-2020.11.8 chardet-3.0.4 distlib-0.3.1 django-cors-headers-3.6.0 djangorestframework-3.12.2 filelock-3.0.12 idna-2.10 pipenv-2020.11.15 pytz-2020.4 requests-2.25.0 six-1.15.0 sqlparse-0.4.1 stripe-2.55.1 urllib3-1.26.2 virtualenv-20.2.1 virtualenv-clone-0.5.4
2021/08/13 12:49:39.051999 [INFO] Installing dependencies with Pipfile.lock
2021/08/13 12:49:39.052013 [INFO] Running command /bin/sh -c /usr/bin/python3.8 -m pipenv install --ignore-pipfile
2021/08/13 12:49:40.248089 [INFO] Installing dependencies with Pipfile
2021/08/13 12:49:40.248118 [INFO] Running command /bin/sh -c /usr/bin/python3.8 -m pipenv install --skip-lock
2021/08/13 12:49:40.976863 [INFO] Installing dependencies from Pipfile…
2021/08/13 12:49:40.976902 [ERROR] An error occurred during execution of command [app-deploy] - [InstallDependency]. Stop running the command. Error: fail to install dependencies with Pipfile file with error Command /bin/sh -c /usr/bin/python3.8 -m pipenv install --skip-lock failed with error exit status 1. Stderr:Traceback (most recent call last):
File "/usr/lib64/python3.8/runpy.py", line 194, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/lib64/python3.8/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/usr/local/lib/python3.8/site-packages/pipenv/__main__.py", line 5, in <module>
cli()
File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 829, in __call__
return self.main(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 782, in main
rv = self.invoke(ctx)
File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 1259, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/decorators.py", line 73, in new_func
return ctx.invoke(f, obj, *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/core.py", line 610, in invoke
return callback(*args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pipenv/vendor/click/decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "/usr/local/lib/python3.8/site-packages/pipenv/cli/command.py", line 232, in install
retcode = do_install(
File "/usr/local/lib/python3.8/site-packages/pipenv/core.py", line 2052, in do_install
do_init(
File "/usr/local/lib/python3.8/site-packages/pipenv/core.py", line 1305, in do_init
do_install_dependencies(
File "/usr/local/lib/python3.8/site-packages/pipenv/core.py", line 827, in do_install_dependencies
lockfile = project.get_or_create_lockfile(from_pipfile=True)
File "/usr/local/lib/python3.8/site-packages/pipenv/project.py", line 783, in get_or_create_lockfile
lockfile_dict.update({"_meta": self.get_lockfile_meta()})
File "/usr/local/lib/python3.8/site-packages/pipenv/project.py", line 817, in get_lockfile_meta
sources = self.lockfile_content.get("_meta", {}).get("sources", [])
File "/usr/local/lib/python3.8/site-packages/pipenv/project.py", line 662, in lockfile_content
return self.load_lockfile()
File "/usr/local/lib/python3.8/site-packages/pipenv/project.py", line 1044, in load_lockfile
j = json.load(lock)
File "/usr/lib64/python3.8/json/__init__.py", line 293, in load
return loads(fp.read(),
File "/usr/lib64/python3.8/json/__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "/usr/lib64/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/usr/lib64/python3.8/json/decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 4 column 1 (char 35)
2021/08/13 12:49:40.976920 [INFO] Executing cleanup logic
2021/08/13 12:49:40.977012 [INFO] CommandService Response: {"status":"FAILURE","api_version":"1.0","results":[{"status":"FAILURE","msg":"Engine execution has encountered an error.","returncode":1,"events":[{"msg":"Instance deployment failed to install application dependencies. The deployment failed.","timestamp":1628858980,"severity":"ERROR"},{"msg":"Instance deployment failed. For details, see 'eb-engine.log'.","timestamp":1628858980,"severity":"ERROR"}]}]}
Additionally, I am operating inside a virtualenv and the command I mentioned gives me the following warning
Courtesy Notice: Pipenv found itself running within a virtual environment, so it will automatically use that environment, instead of creating its own for any project. You can set PIPENV_IGNORE_VIRTUALENVS=1 to force pipenv to ignore that environment and create its own instead. You can set PIPENV_VERBOSITY=-1 to suppress this warning.
I have tried to run it outside the virtualenv but can't seem to get the correct version of pipenv I have version 2021.5.29 and I need 2020.8.13 as specified by AWS in their documentation.
There is also a small difference between the python version as I use python3.8.11 and they use 3.8.5 does that make a difference?

Answer 1

Not a python export but the error message explicitly notes.
“Instance deployment failed to install application dependencies”
And reading the stack trace it notes.
“ json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 4 column 1 (char 35)”
Looks like you have malformed JSON in your lock file.