We have a local pypiserver and up to now we are happy with it.
But psycopg2 makes trouble. It wants pg_config to be installed. I don't see a reason why it should be installed on the mirror server. I understand that it needs to be on the client where I install psycopg2.
Is this a bug in psycopg2? Or do I use the wrong options the get the package to the mirror?
pypi#gray:~> pip install --no-deps --no-install -d packages psycopg2
Downloading/unpacking psycopg2
File was already downloaded packages/psycopg2-2.5.1.tar.gz
Running setup.py egg_info for package psycopg2
Error: pg_config executable not found.
Please add the directory containing pg_config to the PATH
or specify the full executable path with the option:
python setup.py build_ext --pg-config /path/to/pg_config build ...
or with the pg_config option in 'setup.cfg'.
Complete output from command python setup.py egg_info:
running egg_info
creating pip-egg-info/psycopg2.egg-info
writing pip-egg-info/psycopg2.egg-info/PKG-INFO
writing top-level names to pip-egg-info/psycopg2.egg-info/top_level.txt
writing dependency_links to pip-egg-info/psycopg2.egg-info/dependency_links.txt
writing manifest file 'pip-egg-info/psycopg2.egg-info/SOURCES.txt'
warning: manifest_maker: standard file '-c' not found
Error: pg_config executable not found.
Please add the directory containing pg_config to the PATH
or specify the full executable path with the option:
python setup.py build_ext --pg-config /path/to/pg_config build ...
or with the pg_config option in 'setup.cfg'.
----------------------------------------
Command python setup.py egg_info failed with error code 1 in /home/pypi/tmp/pip-build/psycopg2
Storing complete log in /home/pypi/.config/pip/pip.log
We use this pypiserver: https://pypi.python.org/pypi/pypiserver
I know this question is over a year old but the answer is pg_config is a linux-only module. You would need to install postgresql-devel to get it.
Related
I can customized the installing process in develop mode by setup.py like below:
from setuptools import setup
from setuptools.command import develop
class CustomDevelop(develop.develop, object):
"""
Class needed for "pip install -e ."
"""
def run(self):
super(CustomDevelop, self).run()
print("CustomDevelop=====================")
install_requires = ["numpy"]
setup(
name="example_package",
version="0.0.1",
packages=["example_package"],
cmdclass={'develop': CustomDevelop},
)
When running python3 -m pip install -vvv -e ., the output is
Running setup.py develop for example-package
Running command python setup.py develop
running develop
running egg_info
writing example_package.egg-info/PKG-INFO
writing dependency_links to example_package.egg-info/dependency_links.txt
writing top-level names to example_package.egg-info/top_level.txt
reading manifest file 'example_package.egg-info/SOURCES.txt'
writing manifest file 'example_package.egg-info/SOURCES.txt'
running build_ext
Creating /home/example_package/venv_example_package/lib/python3.8/site-packages/example-package.egg-link (link to .)
Adding example-package 0.0.1 to easy-install.pth file
Installed /home/lai/example_package
CustomDevelop=====================
However, it's not possible to do it with PEP518 which introduces pyproject.toml. Is there a workaround for this issue?
How do I make pip recompile Cython extensions when I have only changed setup.py when installing in editable mode. Currently it always skips the extensions. There are many questions related to this for distutils but I can't see any answers for pip.
To be clear, I have a setup.py like the following
from distutils.core import setup, Extension
from Cython.Distutils import build_ext
import numpy
ext_modules = []
# simulate_fast
ext_modules += [
Extension("adio.simulating.simulate_fast_c",
sources=["./adio/simulating/simulate_fast/simulate_fast_c.pyx",
"./adio/simulating/simulate_fast/c/simulate_fast.c",
"./adio/simulating/simulate_fast/c/matrix.c"],
include_dirs=[numpy.get_include()],
extra_compile_args=["-Ofast", "-ffast-math", "-march=native"],
language='c',
libraries=["gsl", "openblas"],
define_macros=[('FLOAT32', 1)]
)
]
setup(
name="adio",
packages=["adio"],
cmdclass={'build_ext': build_ext},
ext_modules=ext_modules
)
Now if I delete the ('FLOAT32', 1) in define_macros of course I would like the Cython extension to recompile.
However when I run
python3 -m pip install --editable -U . -v
I receive the following output as part of the output
running develop
running egg_info
writing adio.egg-info/PKG-INFO
writing dependency_links to adio.egg-info/dependency_links.txt
writing top-level names to adio.egg-info/top_level.txt
reading manifest file 'adio.egg-info/SOURCES.txt'
writing manifest file 'adio.egg-info/SOURCES.txt'
running build_ext
skipping './adio/simulating/simulate_fast/simulate_fast_c.c' Cython extension (up-to-date)
I have tried the -I and --force-reinstall flag with pip but it always skips the Cython extension. If I am not using editable mode then I can run
python3 -m pip install -U . -v
and this does recompile. How can I achieve the same thing when using the --editable flag.
Related
distutils ignores changes to setup.py when building an extension?
I am trying to install sentry by following the instruction from the link:
However when I try to execute command pip install -U sentry, I receive the error InstallationError: Command python setup.py egg_info failed with error code 1 in /webapps/sentry/env/build/symsynd
Here is the detail from log file:
self.run_setup(setup_script, setup_base, args)
File "/webapps/sentry/env/local/lib/python2.7/site-packages/setuptools/command/easy_install.py", line 1003, in run_setup
raise DistutilsError("Setup script exited with %s" % (v.args[0],))
distutils.errors.DistutilsError: Setup script exited with 1
Complete output from command python setup.py egg_info:
unable to execute 'clang': No such file or directory
unable to execute 'clang': No such file or directory
No working compiler found, or bogus compiler options
passed to the compiler from Python's distutils module.
See the error messages above.
It would be nice if someone can figure out what am I doing wrong.
Thank you,
Vady
You need a C compiler to build the symsynd dependency.
If you are on a Debian based distribution (Ubuntu, etc.) just do :
sudo apt-get install clang
It seems that specifying a C compiler will be ok, but not. In some packages clang is hard-coded. The only way is to install clang.
I have a GPU instance running on EC2 and I'm trying to install cudamat.
I've downloaded a copy to my home directory:
git clone https://github.com/cudamat/cudamat
Then I cd into that directory and run:
sudo pip install .
Error is "command 'nvcc' failed with exit status 1.
Heres the full message:
Unpacking /home/ubuntu/cudamat
Running setup.py (path:/tmp/pip-s7db_8yv-build/setup.py) egg_info for package from file:///home/ubuntu/cudamat
PATH = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
/tmp/pip-s7db_8yv-build
Installing collected packages: cudamat
Running setup.py install for cudamat
PATH = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
/tmp/pip-s7db_8yv-build
building 'cudamat.libcudamat' extension
nvcc -I/usr/include/python3.4m -c cudamat/cudamat.cu -o build/temp.linux-x86_64-3.4/cudamat/cudamat.o -O --ptxas-options=-v --compiler-options=-fPIC
error: command 'nvcc' failed with exit status 1
Complete output from command /usr/bin/python3 -c "import setuptools, tokenize;__file__='/tmp/pip-s7db_8yv-build/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-nulrhjnl-record/install-record.txt --single-version-externally-managed --compile:
PATH = /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
/tmp/pip-s7db_8yv-build
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.4
creating build/lib.linux-x86_64-3.4/cudamat
copying cudamat/learn.py -> build/lib.linux-x86_64-3.4/cudamat
copying cudamat/__init__.py -> build/lib.linux-x86_64-3.4/cudamat
copying cudamat/cudamat.py -> build/lib.linux-x86_64-3.4/cudamat
running egg_info
creating cudamat.egg-info
writing cudamat.egg-info/PKG-INFO
writing dependency_links to cudamat.egg-info/dependency_links.txt
writing top-level names to cudamat.egg-info/top_level.txt
writing manifest file 'cudamat.egg-info/SOURCES.txt'
warning: manifest_maker: standard file '-c' not found
reading manifest file 'cudamat.egg-info/SOURCES.txt'
writing manifest file 'cudamat.egg-info/SOURCES.txt'
copying cudamat/cudamat.cu -> build/lib.linux-x86_64-3.4/cudamat
copying cudamat/cudamat_kernels.cu -> build/lib.linux-x86_64-3.4/cudamat
copying cudamat/learn.cu -> build/lib.linux-x86_64-3.4/cudamat
copying cudamat/learn_kernels.cu -> build/lib.linux-x86_64-3.4/cudamat
copying cudamat/rnd_multipliers_32bit.txt -> build/lib.linux-x86_64-3.4/cudamat
running build_ext
building 'cudamat.libcudamat' extension
creating build/temp.linux-x86_64-3.4
creating build/temp.linux-x86_64-3.4/cudamat
nvcc -I/usr/include/python3.4m -c cudamat/cudamat.cu -o build/temp.linux-x86_64-3.4/cudamat/cudamat.o -O --ptxas-options=-v --compiler-options=-fPIC
error: command 'nvcc' failed with exit status 1
There is something wrong with the path. since nvcc was not found.
heres when i type:
echo $PATH
/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
Anyone has an idea?
cheers
The error manifests itself only when sudo is used.
sudo changes the $PATH variable, depending on the security policy.
The problem is fixed if the following is used:
sudo env "PATH=$PATH" pip3 install .
nvcc is not configured for sudo access(i may be wrongly phrasing this sentence, but in short nvcc doesnt work with sudo)
there are two solutions to this problem:
Give sudo access to nvcc(which you have mentioned) which may be the inefficient way.
Don't use sudo while installing cudamat instead use the following command when you are inside the cudamat directory.
python setup.py install --user
Note:
While testing after installation dont use import cudamat from the cudamat directory which will through an error saying cudamat/cudamat not found.
It will look in the local directory.
Go to home and use import cudamat it will work
Clone from github
git clone https://github.com/cudamat/cudamat.git
This is only compatible with python 3
I am trying to debug a small cython project following the instructions from the official Cython page. but the command:
python-dbg setup.py build_ext --inplace
fails with the error below. I have seen responses to a similar issue here but I don't think it applies for me as I am running Cython installed via apt-get. Any help would be much appreciated.
Traceback (most recent call last):
File "build.py", line 4, in
from Cython.Build import cythonize
File "/usr/lib/python2.7/dist-packages/Cython/Build/init.py", line 1, in from Dependencies import cythonize
File "/usr/lib/python2.7/dist-packages/Cython/Build/Dependencies.py", line 51, in from Cython.Compiler.Main import Context, CompilationOptions, default_options
File "/usr/lib/python2.7/dist-packages/Cython/Compiler/Main.py", line 17, in from Scanning import PyrexScanner, FileSourceDescriptor
ImportError: /usr/lib/python2.7/dist-packages/Cython/Compiler/Scanning.so: undefined symbol: Py_InitModule4_64
[35101 refs]
I had the same problem.
In this article the cause of it is explained:
The problem [..] is the python-dbg binary however is incompatible with
modules not compiled with them
http://hustoknow.blogspot.de/2013/06/why-your-python-program-cant-start-when.html
My solution was to uninstall cython (which was installed via pip), clone the cython github repo and manually install it with python-dbg:
git clone git#github.com:cython/cython.git
cd cython
sudo python-dbg setup.py install
Afterwards I was able to python-dbg setup.py build_ext --inplace in my own code.