How to use pip-tools to install from a github repo? - pip

Here is a line in a req.in file (for pip-compile) to get a package from a github repository:
-e git://github.com/dfrankow/django-editorjs-fields.git#egg=django-editorjs-fields
Running pip-compile req.in gives a req.txt with
-e git+git://github.com/dfrankow/django-editorjs-fields.git#egg=django-editorjs-fields
Using pip to install (pip install -r req.txt) gives this error:
ERROR: Command errored out with exit status 1:
command: .../.venv/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'.../.venv/src/django-editorjs-fields/setup.py'"'"'; __file__='"'"'.../.venv/src/django-editorjs-fields/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps
cwd: .../.venv/src/django-editorjs-fields/
Complete output (3 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'setuptools'
I have setuptools (and in fact all the other libraries) installed. See, no errors:
$ .venv/bin/python -c 'import io, os, sys, setuptools, tokenize;'
$
If I install directly (without the -e) it works:
$ pip install git+git://github.com/dfrankow/django-editorjs-fields.git#egg=django-editorjs-fields
... (works)
But when I take the -e out of req.in, pip-compile complains:
$ pip-compile req.in
...
pip._internal.exceptions.InstallationError: Invalid requirement: 'git://github.com/dfrankow/django-editorjs-fields.git#egg=django-editorjs-fields' (from line 1 of req.in)
Hint: It looks like a path. File 'git://github.com/dfrankow/django-editorjs-fields.git#egg=django-editorjs-fields' does not exist.
What the heck, I'm trapped.
How do I put a github repo into req.in, compile it with pip-compile into req.txt, and pip install req.txt without errors?
I need it to work automatically because we have a toolchain that depends on it (github workflows, dependabot, ..).
I'm using python 3.9.1_2, pip 21.1.2, pip-tools 6.2.0 on OS X. Also happened with pip-tools 6.0.1 (not surprising).

Related

Poetry: always have to install setuptools manually in CI

Im building my Circle CI flow with Poetry. Everything works well, except the thing that i should install setuptools always manually. It looks like this:
steps:
- checkout
- restore_cache:
keys:
- pip-{{ checksum "poetry.lock" }}-{{ .Environment.CACHE_VERSION }}
- run:
name: install dependencies
command: |
poetry config virtualenvs.in-project true
poetry config virtualenvs.path /project
poetry run pip install --upgrade pip setuptools
poetry install
poetry show -v
steps:
- run:
name: pytest
command: |
poetry run pip install setuptools
poetry run pytest -sv tests/unittests
If im removing manual installation of setuptools, installing of some packages are failing with the same traceback. Example:
Installing googleads (29.0.0): Failed
EnvCommandError
Command ['/home/circleci/project/.venv/bin/pip', 'install', '--no-deps', '/home/circleci/.cache/pypoetry/artifacts/05/59/83/1fa9c7bc74df8656d6fa40f159fa169c10a9a110501f2ec8260c8cd3e1/googleads-29.0.0.tar.gz'] errored with the following return code 1, and output:
Processing /home/circleci/.cache/pypoetry/artifacts/05/59/83/1fa9c7bc74df8656d6fa40f159fa169c10a9a110501f2ec8260c8cd3e1/googleads-29.0.0.tar.gz
ERROR: Command errored out with exit status 1:
command: /home/circleci/project/.venv/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-e9n99b4s/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-e9n99b4s/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-8t_uwvq3
cwd: /tmp/pip-req-build-e9n99b4s/
Complete output (3 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'setuptools'
----------------------------------------
WARNING: Discarding file:///home/circleci/.cache/pypoetry/artifacts/05/59/83/1fa9c7bc74df8656d6fa40f159fa169c10a9a110501f2ec8260c8cd3e1/googleads-29.0.0.tar.gz. Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
WARNING: You are using pip version 21.2.4; however, version 22.3.1 is available.
You should consider upgrading via the '/home/circleci/project/.venv/bin/python -m pip install --upgrade pip' command.
at /usr/local/lib/python3.8/site-packages/poetry/utils/env.py:1183 in _run
1179│ output = subprocess.check_output(
1180│ cmd, stderr=subprocess.STDOUT, **kwargs
1181│ )
1182│ except CalledProcessError as e:
→ 1183│ raise EnvCommandError(e, input=input_)
1184│
1185│ return decode(output)
1186│
1187│ def execute(self, bin, *args, **kwargs):
Tests are also failing if manual installation of setuptools gonna be removed, if i will not install setuptools manually on the test step. All tests are also failing with the same trace:
collection failure
ImportError while importing test module '/home/circleci/project/tests/unittests/hsplib/test_spark.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/local/lib/python3.8/importlib/__init__.py:127: in import_module
return _bootstrap._gcd_import(name[level:], package, level)
tests/unittests/hsplib/test_spark.py:9: in <module>
from tests.unittests.testlib.asserts import assert_df_equal_2
tests/unittests/testlib/asserts.py:5: in <module>
import chispa
.venv/lib/python3.8/site-packages/chispa/__init__.py:28: in <module>
from .dataframe_comparer import DataFramesNotEqualError, assert_df_equality, assert_approx_df_equality
.venv/lib/python3.8/site-packages/chispa/dataframe_comparer.py:1: in <module>
from chispa.prettytable import PrettyTable
.venv/lib/python3.8/site-packages/chispa/prettytable.py:44: in <module>
import pkg_resources
E ModuleNotFoundError: No module named 'pkg_resources'
I want to get rid from this manual setuptools installation, but i cant understand whats wrong with it. It feels like a code smell. What i tryed:
Adding setuptools as one of the dependencies in pyproject.toml file - no effect
Adding setuptools to build-system requires - also no effect.
Adding virtualenvs.options.no-setuptools = false to poetry config. It should be by default, but i tryed to setup it manually. No effect.
The only issue, that looks like a similar one that i found is this one:
https://github.com/python-poetry/poetry/issues/4242
But there is still no answer there what to do.
My poetry version 1.3.2
pip version 22.0.4
Seems like your project depends on some libraries that are poorly packaged. You should raise up the issue towards their maintainers on their issue trackers:
https://github.com/googleads/googleads-python-lib/issues
https://github.com/MrPowers/chispa/issues

pip install xlwings fails

I'm on MacOS Mojave Version 10.14.6, 2.8 GHz Intel Core i7 processor. Python version: 2.7.
"pip install xlwings" fails for me. I have tried to resolve this by upgrading setuptools with pip install -U setuptools
However, this does not fix the issue. The full error message for "pip install xlwings" is as follows:
Collecting xlwings
Using cached xlwings-0.16.6.tar.gz (634 kB)
Collecting psutil>=2.0.0
Using cached psutil-5.9.0.tar.gz (478 kB)
Collecting appscript>=1.0.1
Using cached appscript-1.2.0.tar.gz (289 kB)
ERROR: Command errored out with exit status 1:
command: /Library/Frameworks/Python.framework/Versions/2.7/Resources/Python.app/Contents/MacOS/Python -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/hm/pt6_h7xs52z7_89jqr614q300000gn/T/pip-install-KTcWya/appscript/setup.py'"'"'; __file__='"'"'/private/var/folders/hm/pt6_h7xs52z7_89jqr614q300000gn/T/pip-install-KTcWya/appscript/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/hm/pt6_h7xs52z7_89jqr614q300000gn/T/pip-pip-egg-info-fmr2EU
cwd: /private/var/folders/hm/pt6_h7xs52z7_89jqr614q300000gn/T/pip-install-KTcWya/appscript/
Complete output (5 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/var/folders/hm/pt6_h7xs52z7_89jqr614q300000gn/T/pip-install-KTcWya/appscript/setup.py", line 8, in <module>
raise RuntimeError("Python 3.x required.")
RuntimeError: Python 3.x required.
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
xlwings declares appscript as a dependency but it doesn't limit version; appscript doesn't properly declare Python version compatibility (xlwings does it properly) so pip is trying to install the latest version which is not compatible with Python 2.7. Try to limit versions this way:
pip install "xlwings<0.17" "appscript<1.2"

pip install error when I install fbprophet

(I'm not good at English sorry)
I have an error when I install fbprophet for windows command.
I already installed pystan and cython.
I use Windows 10, python 3.9 version.
maybe, Am I having a problem with the compiler?
The error comment is (korea language in error is not really important thing), -->
=== error in building with wheel ===
ERROR: Command errored out with exit status 1:
command: 'c:\users\kstar\appdata\local\programs\python\python39\python.exe' -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\Users\kstar\AppData\Local\Temp\pip-install-la81h33s\fbprophet_b85d314a94fb4910b19f0b36b012fb78\setup.py'"'"'; file='"'"'C:\Users\kstar\AppData\Local\Temp\pip-install-la81h33s\fbprophet_b85d314a94fb4910b19f0b36b012fb78\setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' bdist_wheel -d 'C:\Users\kstar\AppData\Local\Temp\pip-wheel-b98qkky1'
cwd: C:\Users\kstar\AppData\Local\Temp\pip-install-la81h33s\fbprophet_b85d314a94fb4910b19f0b36b012fb78
Complete output (44 lines):
running bdist_wheel
running build
running build_py
creating build
creating build\lib
creating build\lib\fbprophet
creating build\lib\fbprophet\stan_model
Traceback (most recent call last):
.
.
.
=== error in running setup.py ===
ERROR: Command errored out with exit status 1:
command: c:\users\kstar\appdata\local\programs\python\python39\python.exe -u -c import io, os, sys, setuptools, tokenize; sys.argv[0] = C:\Users\kstar\AppData\Local\Temp\pip-install-la81h33s\fbprophet_b85d314a94fb4910b19f0b36b012fb78\setup.py; file=C:\Users\kstar\AppData\Local\Temp\pip-install-la81h33s\fbprophet_b85d314a94fb4910b19f0b36b012fb78\setup.py;f = getattr(tokenize, '"'"'open, open)(file) if os.path.exists(file) else io.StringIO('"'"'from setuptools import setup; setup());code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, file, '"'"'exec'"'"'))' install --record 'C:\Users\kstar\AppData\Local\Temp\pip-record-avm6oqpp\install-record.txt' --single-version-externally-managed --compile --install-headers 'c:\users\kstar\appdata\local\programs\python\python39\Include\fbprophet'
cwd: C:\Users\kstar\AppData\Local\Temp\pip-install-la81h33s\fbprophet_b85d314a94fb4910b19f0b36b012fb78
Complete output (288 lines):
running install
running build
running build_py
creating build
creating build\lib
creating build\lib\fbprophet
creating build\lib\fbprophet\stan_model
INFO:pystan:COMPILING THE C++ CODE FOR MODEL anon_model_f5236004a3fd5b8429270d00efcc0cf9 NOW.
WARNING:pystan:MSVC compiler is not supported
stanfit4anon_model_f5236004a3fd5b8429270d00efcc0cf9_296405994888268896.cpp
c:\users\kstar\appdata\local\programs\python\python39\lib\site-packages\numpy\core\include\numpy\npy_1_7_deprecated_api.h(14) : Warning Msg: Using deprecated NumPy API, disable it with #define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION
following has worked for me before:
pip install localpip
localpip install fbprophet
from fbprophet import Prophet

PKG_CONFIG_PATH environment variable (install ipopt error)

I tried to install ipopt trough pip install ipopt
Here is my error:
Collecting ipopt
Using cached ipopt-0.2.0.tar.gz (162 kB)
ERROR: Command errored out with exit status 1:
command: /usr/local/opt/python/bin/python3.7 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/3w/zf3x85h55tv7qk2yg8ykztwc0000gn/T/pip-install-i9o4pqdg/ipopt/setup.py'"'"'; __file__='"'"'/private/var/folders/3w/zf3x85h55tv7qk2yg8ykztwc0000gn/T/pip-install-i9o4pqdg/ipopt/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/3w/zf3x85h55tv7qk2yg8ykztwc0000gn/T/pip-pip-egg-info-_gej3ysb
cwd: /private/var/folders/3w/zf3x85h55tv7qk2yg8ykztwc0000gn/T/pip-install-i9o4pqdg/ipopt/
Complete output (11 lines):
Package ipopt was not found in the pkg-config search path.
Perhaps you should add the directory containing `ipopt.pc'
to the PKG_CONFIG_PATH environment variable
No package 'ipopt' found
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/var/folders/3w/zf3x85h55tv7qk2yg8ykztwc0000gn/T/pip-install-i9o4pqdg/ipopt/setup.py", line 102, in <module>
**pkgconfig('ipopt'))]
File "/private/var/folders/3w/zf3x85h55tv7qk2yg8ykztwc0000gn/T/pip-install-i9o4pqdg/ipopt/setup.py", line 58, in pkgconfig
raise OSError(msg.format(list(packages)))
OSError: pkg-config was not able to find any of the requested packages ['ipopt'] on your system. Make sure pkg-config can discover the .pc files associated with the installed packages.
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
Command pkg-config --variable pc_path pkg-config gives:
/usr/local/lib/pkgconfig:/usr/local/share/pkgconfig:/usr/lib/pkgconfig:/usr/local/Homebrew/Library/Homebrew/os/mac/pkgconfig/10.15
Made this (but I think that the link is strange), but didn't solve the problem
export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:/usr/local/share/pkgconfig:/usr/lib/pkgconfig:/usr/local/Homebrew/Library/Homebrew/os/mac/pkgconfig/10.15
You must the following commands:
sudo apt-get install gcc g++ gfortran git patch wget pkg-config liblapack-dev libmetis-dev

an error shows when I try to install submit50

C:\Users\BISMILLAH>pip install submit50
Collecting submit50
Using cached submit50-3.0.2.tar.gz (5.5 kB)
ERROR: Command errored out with exit status 1:
command: 'c:\users\bismillah\appdata\local\programs\python\python36-32\python.exe' -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'C:\\Users\\BISMILLAH\\AppData\\Local\\Temp\\pip-install-enj227ls\\submit50\\setup.py'"'"'; __file__='"'"'C:\\Users\\BISMILLAH\\AppData\\Local\\Temp\\pip-install-enj227ls\\submit50\\setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base 'C:\Users\BISMILLAH\AppData\Local\Temp\pip-install-enj227ls\submit50\pip-egg-info'
cwd: C:\Users\BISMILLAH\AppData\Local\Temp\pip-install-enj227ls\submit50\
Complete output (5 lines):
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "C:\Users\BISMILLAH\AppData\Local\Temp\pip-install-enj227ls\submit50\setup.py", line 2, in <module>
raise RuntimeError("submit50 does not support Windows directly. Instead, you should install the Windows Subsystem for Linux (https://learn.microsoft.com/en-us/windows/wsl/install-win10) and then install submit50 within that.")
RuntimeError: submit50 does not support Windows directly. Instead, you should install the Windows Subsystem for Linux (https://learn.microsoft.com/en-us/windows/wsl/install-win10) and then install submit50 within that.
----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.
Turns out submit50 doesn't run on Windows but this solution worked for me:
https://github.com/cs50/submit50/issues/196#issuecomment-617907204

Resources