There seems to be an issue installing scikit-learn on Mac M1 chips for Big Sur. I tried including the environment variable flags in this answer to no avail.
The specific error is:
Collecting scikit-learn
Using cached scikit-learn-1.0.2.tar.gz (6.7 MB)
Installing build dependencies ... error
error: subprocess-exited-with-error
× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> [5382 lines of output]
Ignoring numpy: markers 'python_version == "3.7" and platform_machine != "aarch64" and platform_system != "AIX" and platform_python_implementation != "PyPy"' don't match your environment
...clip...
Running from SciPy source directory.
/private/var/folders/4s/pdgwwvsn2wd9192jdyz2lk8w0000gn/T/pip-install-7gy3iln3/scipy_92a96fbb9138448d8d153c3fef5c5cb4/scipy/linalg/_generate_pyx.py:674: DeprecationWarning: The distutils package is deprecated and slated for removal in Python 3.12. Use setuptools or check PEP 632 for potential alternatives
...clip...
libraries mkl_rt not found in ['/Library/Frameworks/Python.framework/Versions/3.10/lib', '/usr/local/lib', '/usr/lib']
libraries mkl_rt not found in ['/Library/Frameworks/Python.framework/Versions/3.10/lib', '/usr/local/lib', '/usr/lib']
NOT AVAILABLE
openblas_lapack_info:
C compiler: clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch arm64 -arch x86_64 -g -falign-functions=8
creating /var/folders/4s/pdgwwvsn2wd9192jdyz2lk8w0000gn/T/tmpqd0a75ee/var
creating /var/folders/4s/pdgwwvsn2wd9192jdyz2lk8w0000gn/T/tmpqd0a75ee/var/folders
...clip...
I'm trying to deploy a python app using Heroku, but I'm not able to install pycairo, which is added to requirements.txt file. I have used https://buildpack-registry.s3.amazonaws.com/buildpacks/heroku-community/apt.tgz build-package to install required apt packages.
When I tried to install this package in Heroku bash using pip, it returned the same error.
Building wheel for pycairo (setup.py): started
Building wheel for pycairo (setup.py): finished with status 'error'
ERROR: Command errored out with exit status 1:
command: /app/.heroku/python/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-6rkzsy5d/pycairo/setup.py'"'"'; __file__='"'"'/tmp/pip-install-6rkzsy5d/pycairo/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-0eopb8bo
cwd: /tmp/pip-install-6rkzsy5d/pycairo/
Complete output (35 lines):
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.8
creating build/lib.linux-x86_64-3.8/cairo
copying cairo/__init__.py -> build/lib.linux-x86_64-3.8/cairo
copying cairo/__init__.pyi -> build/lib.linux-x86_64-3.8/cairo
copying cairo/py.typed -> build/lib.linux-x86_64-3.8/cairo
running build_ext
building 'cairo._cairo' extension
creating build/temp.linux-x86_64-3.8
creating build/temp.linux-x86_64-3.8/cairo
gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -fPIC -DPYCAIRO_VERSION_MAJOR=1 -DPYCAIRO_VERSION_MINOR=19 -DPYCAIRO_VERSION_MICRO=1 -I/tmp/build_5896cd7c39526a1767b861fb4950464e/.apt/usr/include/cairo -I/tmp/build_5896cd7c39526a1767b861fb4950464e/.apt/usr/include -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/pixman-1 -I/usr/include/freetype2 -I/usr/include/libpng16 -I/usr/include/freetype2 -I/usr/include/libpng16 -I/app/.heroku/python/include/python3.8 -c cairo/device.c -o build/temp.linux-x86_64-3.8/cairo/device.o -Wall -Warray-bounds -Wcast-align -Wconversion -Wextra -Wformat=2 -Wformat-nonliteral -Wformat-security -Wimplicit-function-declaration -Winit-self -Winline -Wmissing-format-attribute -Wmissing-noreturn -Wnested-externs -Wold-style-definition -Wpacked -Wpointer-arith -Wreturn-type -Wshadow -Wsign-compare -Wstrict-aliasing -Wundef -Wunused-but-set-variable -Wswitch-default -Wno-missing-field-initializers -Wno-unused-parameter -Wno-unused-command-line-argument -fno-strict-aliasing -fvisibility=hidden -std=c99
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
error: command 'gcc' failed with exit status 1
----------------------------------------
ERROR: Command errored out with exit status 1: /app/.heroku/python/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-6rkzsy5d/pycairo/setup.py'"'"'; __file__='"'"'/tmp/pip-install-6rkzsy5d/pycairo/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-lbe5cexs/install-record.txt --single-version-externally-managed --compile --install-headers /app/.heroku/python/include/python3.8/pycairo Check the logs for full command output.
! Push rejected, failed to compile Python app.
! Push failed
Please don't link the archive file (.tgz) of the buildpack, but its homepage from here.
In the error output you posted, the build_ext function of the Python Setuptools build system fails to build cairo. In particular, the gcc compiler tells you that the ld linker failed. I guess it's because the script in the buildpack didn't link the cairo library.
I think you have 2 options:
Modify the Heroku buildpack and figure out how to install the cairo C library and link it when Pycairo runs the build_ext function.
Create a Dockerimage that includes the conda package manager (I would start from docker-miniconda) and install Pycairo as a conda recipe.
Option 2 should be far easier.
See also my other answer here.
I was trying to install a splicing program "MAJIQ". I was able to run it on Centos but unable to install on MAC OS.
Description of the MAJIQ program.
Before installation
MAJIQ installation requires the following lib/software to be installed in your system.
C++11 compiler with openMP. GCC includes that by default, but clang can be updated to include those (Clang/OMP). MAJIQ/VOILA has been tested to work with GNU GCC>=7.2, RedHat GCC>=4.7.2.
HTSlib library. This is a C library for reading/writing high-throughput sequencing data developed by Samtools organisation. MAJIQ installation assumes the library and its header files are present in the Unix default locations (/usr/local/lib, /usr/local/include). If that is not the case the appropiate locations can be specified setting the following environment variables.
$ export HTSLIB_LIBRARY_DIR=/path/to/htslib/lib
$ export HTSLIB_INCLUDE_DIR=/path/to/htslib/include
Installation
To download and install MAJIQ/Voila run the following commands:
$ python3 -m venv env
$ source env/bin/activate
$ pip install pip -U
$ pip install wheel setuptools -U
$ pip install cython numpy GitPython -U
$ pip install git+https://bitbucket.org/biociphers/majiq_stable.git#egg=majiq
If there is an error during install please verify that you're installing these packages with python 3 and the installed version of pip is current. Also, check if you have git, a c compiler (gcc, clang, etc.), and zlib installed.
Troubleshooting
1) I have tried conda install llvm, conda install gcc or conda install libgcc from https://github.com/velocyto-team/velocyto.py/issues/53
2) Also, tried in given way.
brew install llvm
LDFLAGS="-L/usr/local/opt/llvm/lib -Wl,-rpath,/usr/local/opt/llvm/lib"
echo 'export PATH="/usr/local/opt/llvm/bin:$PATH"' >> ~/.bash_profile
export LDFLAGS="-L/usr/local/opt/llvm/lib"
export CPPFLAGS="-I/usr/local/opt/llvm/include"
Error in output
In file included from /private/var/root/env/lib/python3.7/site-packages/numpy/core/include/numpy/ndarraytypes.h:1822:
/private/var/root/env/lib/python3.7/site-packages/numpy/core/include/numpy/npy_1_7_deprecated_api.h:17:2: warning: "Using deprecated NumPy API, disable it with " "#define NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-W#warnings]
#warning "Using deprecated NumPy API, disable it with " \
^
1 warning generated.
g++ -bundle -undefined dynamic_lookup -arch x86_64 -g build/temp.macosx-10.9-x86_64-3.7/majiq/src/polyfitnb.o -o build/lib/majiq/src/polyfitnb.cpython-37m-darwin.so
building 'majiq.src.build' extension
creating build/temp.macosx-10.9-x86_64-3.7/majiq/src/internals
gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -arch x86_64 -g -I./majiq/src/internals -I./voila/c -Imajiq/src/internals -Ivoila/c -I/private/var/root/env/lib/python3.7/site-packages/numpy/core/include -I/usr/local/include -I/private/var/root/env/include -I/Library/Frameworks/Python.framework/Versions/3.7/include/python3.7m -c majiq/src/build.cpp -o build/temp.macosx-10.9-x86_64-3.7/majiq/src/build.o -fopenmp -DSCYTHE_COMPILE_DIRECT -DSCYTHE_PTHREAD
clang: error: unsupported option '-fopenmp'
error: command 'gcc' failed with exit status 1
----------------------------------------
ERROR: Command "/private/var/root/env/bin/python3 -u -c 'import setuptools, tokenize;__file__='"'"'/private/tmp/pip-install-8vjv35m9/majiq/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/tmp/pip-record-sfcn1usm/install-record.txt --single-version-externally-managed --compile --install-headers /private/var/root/env/include/site/python3.7/majiq" failed with error code 1 in /private/tmp/pip-install-8vjv35m9/majiq/
I had the same issue, and setting CC and CXX as follows solved it:
CC=/usr/local/bin/gcc-9 CXX=/usr/local/bin/g++-9 python3 -m pip install git+https://bitbucket.org/biociphers/majiq_stable.git#egg=majiq
g++-9
This is based on
https://groups.google.com/forum/#!topic/majiq_voila/exTAxHb_HNY
OS: yosemite 10.10.5
using virtualenv,
.bashrc/.bash_profile/.profile all empty
i have install openblas simply using "make && make
PREFIX=/usr/local/opt/openblas install", that means it was installed
in /usr/local/opt/openblas
CUDA and CuDNN all installed and tested.
cd ~/test
ls
numpy-1.10.4 OpenBLAS-0.2.15
export PATH="/opt/local/bin:/opt/local/sbin:/usr/local/Cellar/bin:/usr/local/bin:$PATH"
export PATH="/Developer/NVIDIA/CUDA-7.5/bin:$PATH"
export DYLD_LIBRARY_PATH="/Developer/NVIDIA/CUDA-7.5/lib:$DYLD_LIBRARY_PATH"
export LDFLAGS=-L/usr/local/opt/openblas/lib
export CPPFLAGS=-I/usr/local/opt/openblas/include
after activate my virtualenv, say env1:
(env1)~>cd ~/test/numpy-1.10.4
(env1)numpy-1.10.4>cat site.cfg
[openblas]
libraries = openblas
library_dirs = /usr/local/opt/openblas/lib
include_dirs = /usr/local/opt/openblas/include
runtime_library_dirs = /usr/local/opt/openblas/lib
(env1)>python setup.py config
blas_opt_info:
blas_mkl_info:
libraries mkl,vml,guide not found in ['/Users/mac/.virtualenvs/env1/bin/../lib', '/usr/local/lib', '/usr/lib']
NOT AVAILABLE
openblas_info:
FOUND:
libraries = ['openblas', 'openblas']
library_dirs = ['/usr/local/opt/openblas/lib']
language = c
define_macros = [('HAVE_CBLAS', None)]
runtime_library_dirs = ['/usr/local/opt/openblas/lib']
FOUND:
libraries = ['openblas', 'openblas']
library_dirs = ['/usr/local/opt/openblas/lib']
language = c
define_macros = [('HAVE_CBLAS', None)]
runtime_library_dirs = ['/usr/local/opt/openblas/lib']
F2PY Version 2
lapack_opt_info:
openblas_lapack_info:
C compiler: cc
creating /var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmpvjuerN/var
creating /var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmpvjuerN/var/folders
creating /var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmpvjuerN/var/folders/mz
creating /var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmpvjuerN/var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn
creating /var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmpvjuerN/var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T
creating /var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmpvjuerN/var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmpvjuerN
compile options: '-c'
cc: /var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmpvjuerN/source.c
/var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmpvjuerN/source.c:4:13: warning: implicit declaration of function 'zungqr_' is invalid in C99 [-Wimplicit-function-declaration]
zungqr_();
^
1 warning generated.
cc /var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmpvjuerN/var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmpvjuerN/source.o -L/usr/local/opt/openblas/lib -lopenblas -o /var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmpvjuerN/a.out
FOUND:
libraries = ['openblas', 'openblas']
library_dirs = ['/usr/local/opt/openblas/lib']
language = c
define_macros = [('HAVE_CBLAS', None)]
runtime_library_dirs = ['/usr/local/opt/openblas/lib']
FOUND:
libraries = ['openblas', 'openblas']
library_dirs = ['/usr/local/opt/openblas/lib']
language = c
define_macros = [('HAVE_CBLAS', None)]
runtime_library_dirs = ['/usr/local/opt/openblas/lib']
running config
(env1)numpy-1.10.4>python setup.py build --fcompiler=gnu95
blas_opt_info:
blas_mkl_info:
libraries mkl,vml,guide not found in ['/Users/mac/.virtualenvs/env1/bin/../lib', '/usr/local/lib', '/usr/lib']
NOT AVAILABLE
openblas_info:
FOUND:
libraries = ['openblas', 'openblas']
library_dirs = ['/usr/local/opt/openblas/lib']
language = c
define_macros = [('HAVE_CBLAS', None)]
runtime_library_dirs = ['/usr/local/opt/openblas/lib']
FOUND:
libraries = ['openblas', 'openblas']
library_dirs = ['/usr/local/opt/openblas/lib']
language = c
define_macros = [('HAVE_CBLAS', None)]
runtime_library_dirs = ['/usr/local/opt/openblas/lib']
F2PY Version 2
lapack_opt_info:
openblas_lapack_info:
C compiler: cc
creating /var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmptcMLMT/var
creating /var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmptcMLMT/var/folders
creating /var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmptcMLMT/var/folders/mz
creating /var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmptcMLMT/var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn
creating /var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmptcMLMT/var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T
creating /var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmptcMLMT/var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmptcMLMT
compile options: '-c'
cc: /var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmptcMLMT/source.c
/var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmptcMLMT/source.c:4:13: warning: implicit declaration of function 'zungqr_' is invalid in C99 [-Wimplicit-function-declaration]
zungqr_();
^
1 warning generated.
cc /var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmptcMLMT/var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmptcMLMT/source.o -L/usr/local/opt/openblas/lib -lopenblas -o /var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmptcMLMT/a.out
FOUND:
libraries = ['openblas', 'openblas']
library_dirs = ['/usr/local/opt/openblas/lib']
language = c
define_macros = [('HAVE_CBLAS', None)]
runtime_library_dirs = ['/usr/local/opt/openblas/lib']
FOUND:
libraries = ['openblas', 'openblas']
library_dirs = ['/usr/local/opt/openblas/lib']
language = c
define_macros = [('HAVE_CBLAS', None)]
runtime_library_dirs = ['/usr/local/opt/openblas/lib']
running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler options
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
running build_src
build_src
building py_modules sources
building library "npymath" sources
customize Gnu95FCompiler
Found executable /usr/local/bin/gfortran
customize Gnu95FCompiler using config
C compiler: clang -fno-strict-aliasing -fno-common -dynamic -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes
compile options: '-Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c'
clang: _configtest.c
clang _configtest.o -o _configtest
success!
removing: _configtest.c _configtest.o _configtest
C compiler: clang -fno-strict-aliasing -fno-common -dynamic -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes
compile options: '-Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c'
clang: _configtest.c
_configtest.c:1:5: warning: incompatible redeclaration of library function 'exp' [-Wincompatible-library-redeclaration]
int exp (void);
^
_configtest.c:1:5: note: 'exp' is a builtin with type 'double (double)'
1 warning generated.
clang _configtest.o -o _configtest
success!
removing: _configtest.c _configtest.o _configtest
building library "npysort" sources
adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/src/private' to include_dirs.
None - nothing done with h_files = ['build/src.macosx-10.10-x86_64-2.7/numpy/core/src/private/npy_partition.h', 'build/src.macosx-10.10-x86_64-2.7/numpy/core/src/private/npy_binsearch.h']
building extension "numpy.core._dummy" sources
adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/config.h' to sources.
adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/_numpyconfig.h' to sources.
executing numpy/core/code_generators/generate_numpy_api.py
adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.h' to sources.
numpy.core - nothing done with h_files = ['build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/config.h', 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/_numpyconfig.h', 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.h']
building extension "numpy.core.multiarray" sources
adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/config.h' to sources.
adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/_numpyconfig.h' to sources.
executing numpy/core/code_generators/generate_numpy_api.py
adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.h' to sources.
adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/src/private' to include_dirs.
numpy.core - nothing done with h_files = ['build/src.macosx-10.10-x86_64-2.7/numpy/core/src/private/templ_common.h', 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/config.h', 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/_numpyconfig.h', 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.h']
building extension "numpy.core.umath" sources
adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/config.h' to sources.
adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/_numpyconfig.h' to sources.
executing numpy/core/code_generators/generate_ufunc_api.py
adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/__ufunc_api.h' to sources.
adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/src/umath' to include_dirs.
numpy.core - nothing done with h_files = ['build/src.macosx-10.10-x86_64-2.7/numpy/core/src/umath/funcs.inc', 'build/src.macosx-10.10-x86_64-2.7/numpy/core/src/umath/simd.inc', 'build/src.macosx-10.10-x86_64-2.7/numpy/core/src/umath/loops.h', 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/config.h', 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/_numpyconfig.h', 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/__ufunc_api.h']
building extension "numpy.core.umath_tests" sources
building extension "numpy.core.test_rational" sources
building extension "numpy.core.struct_ufunc_test" sources
building extension "numpy.core.multiarray_tests" sources
building extension "numpy.core.operand_flag_tests" sources
building extension "numpy.fft.fftpack_lite" sources
building extension "numpy.linalg.lapack_lite" sources
adding 'numpy/linalg/lapack_litemodule.c' to sources.
adding 'numpy/linalg/lapack_lite/python_xerbla.c' to sources.
building extension "numpy.linalg._umath_linalg" sources
adding 'numpy/linalg/umath_linalg.c.src' to sources.
adding 'numpy/linalg/lapack_lite/python_xerbla.c' to sources.
building extension "numpy.random.mtrand" sources
building data_files sources
build_src: building npy-pkg config files
running build_py
copying numpy/version.py -> build/lib.macosx-10.10-x86_64-2.7/numpy
copying build/src.macosx-10.10-x86_64-2.7/numpy/__config__.py -> build/lib.macosx-10.10-x86_64-2.7/numpy
copying build/src.macosx-10.10-x86_64-2.7/numpy/distutils/__config__.py -> build/lib.macosx-10.10-x86_64-2.7/numpy/distutils
running build_clib
customize UnixCCompiler
customize UnixCCompiler using build_clib
running build_ext
customize UnixCCompiler
customize UnixCCompiler using build_ext
running build_scripts
adding 'build/scripts.macosx-10.10-x86_64-2.7/f2py' to scripts
(env1)numpy-1.10.4>python setup.py install
Running from numpy source directory.
/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'test_suite'
warnings.warn(msg)
blas_opt_info:
blas_mkl_info:
libraries mkl,vml,guide not found in ['/Users/mac/.virtualenvs/env1/bin/../lib', '/usr/local/lib', '/usr/lib']
NOT AVAILABLE
openblas_info:
/Users/mac/test/numpy-1.10.4/numpy/distutils/system_info.py:635: UserWarning: Specified path is invalid.
warnings.warn('Specified path %s is invalid.' % d)
FOUND:
libraries = ['openblas', 'openblas']
library_dirs = ['/usr/local/opt/openblas/lib']
language = c
define_macros = [('HAVE_CBLAS', None)]
runtime_library_dirs = ['/usr/local/opt/openblas/lib']
FOUND:
libraries = ['openblas', 'openblas']
library_dirs = ['/usr/local/opt/openblas/lib']
language = c
define_macros = [('HAVE_CBLAS', None)]
runtime_library_dirs = ['/usr/local/opt/openblas/lib']
F2PY Version 2
lapack_opt_info:
openblas_lapack_info:
C compiler: cc
creating /var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmpgDbyyW/var
creating /var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmpgDbyyW/var/folders
creating /var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmpgDbyyW/var/folders/mz
creating /var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmpgDbyyW/var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn
creating /var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmpgDbyyW/var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T
creating /var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmpgDbyyW/var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmpgDbyyW
compile options: '-c'
cc: /var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmpgDbyyW/source.c
/var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmpgDbyyW/source.c:4:13: warning: implicit declaration of function 'zungqr_' is invalid in C99 [-Wimplicit-function-declaration]
zungqr_();
^
1 warning generated.
cc /var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmpgDbyyW/var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmpgDbyyW/source.o -L/usr/local/opt/openblas/lib -lopenblas -o /var/folders/mz/s14c1w0x3ql4_djr5s92pz9r0000gn/T/tmpgDbyyW/a.out
FOUND:
libraries = ['openblas', 'openblas']
library_dirs = ['/usr/local/opt/openblas/lib']
language = c
define_macros = [('HAVE_CBLAS', None)]
runtime_library_dirs = ['/usr/local/opt/openblas/lib']
FOUND:
libraries = ['openblas', 'openblas']
library_dirs = ['/usr/local/opt/openblas/lib']
language = c
define_macros = [('HAVE_CBLAS', None)]
runtime_library_dirs = ['/usr/local/opt/openblas/lib']
/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'define_macros'
warnings.warn(msg)
running install
running build
running config_cc
unifing config_cc, config, build_clib, build_ext, build commands --compiler options
running config_fc
unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
running build_src
build_src
building py_modules sources
building library "npymath" sources
customize Gnu95FCompiler
Found executable /usr/local/bin/gfortran
customize Gnu95FCompiler
customize Gnu95FCompiler using config
C compiler: clang -fno-strict-aliasing -fno-common -dynamic -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes
compile options: '-Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c'
clang: _configtest.c
clang _configtest.o -o _configtest
success!
removing: _configtest.c _configtest.o _configtest
C compiler: clang -fno-strict-aliasing -fno-common -dynamic -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.10.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes
compile options: '-Inumpy/core/src/private -Inumpy/core/src -Inumpy/core -Inumpy/core/src/npymath -Inumpy/core/src/multiarray -Inumpy/core/src/umath -Inumpy/core/src/npysort -I/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c'
clang: _configtest.c
_configtest.c:1:5: warning: incompatible redeclaration of library function 'exp' [-Wincompatible-library-redeclaration]
int exp (void);
^
_configtest.c:1:5: note: 'exp' is a builtin with type 'double (double)'
1 warning generated.
clang _configtest.o -o _configtest
success!
removing: _configtest.c _configtest.o _configtest
building library "npysort" sources
adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/src/private' to include_dirs.
None - nothing done with h_files = ['build/src.macosx-10.10-x86_64-2.7/numpy/core/src/private/npy_partition.h', 'build/src.macosx-10.10-x86_64-2.7/numpy/core/src/private/npy_binsearch.h']
building extension "numpy.core._dummy" sources
adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/config.h' to sources.
adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/_numpyconfig.h' to sources.
executing numpy/core/code_generators/generate_numpy_api.py
adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.h' to sources.
numpy.core - nothing done with h_files = ['build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/config.h', 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/_numpyconfig.h', 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.h']
building extension "numpy.core.multiarray" sources
adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/config.h' to sources.
adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/_numpyconfig.h' to sources.
executing numpy/core/code_generators/generate_numpy_api.py
adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.h' to sources.
adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/src/private' to include_dirs.
numpy.core - nothing done with h_files = ['build/src.macosx-10.10-x86_64-2.7/numpy/core/src/private/templ_common.h', 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/config.h', 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/_numpyconfig.h', 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/__multiarray_api.h']
building extension "numpy.core.umath" sources
adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/config.h' to sources.
adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/_numpyconfig.h' to sources.
executing numpy/core/code_generators/generate_ufunc_api.py
adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/__ufunc_api.h' to sources.
adding 'build/src.macosx-10.10-x86_64-2.7/numpy/core/src/umath' to include_dirs.
numpy.core - nothing done with h_files = ['build/src.macosx-10.10-x86_64-2.7/numpy/core/src/umath/funcs.inc', 'build/src.macosx-10.10-x86_64-2.7/numpy/core/src/umath/simd.inc', 'build/src.macosx-10.10-x86_64-2.7/numpy/core/src/umath/loops.h', 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/config.h', 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/_numpyconfig.h', 'build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/__ufunc_api.h']
building extension "numpy.core.umath_tests" sources
building extension "numpy.core.test_rational" sources
building extension "numpy.core.struct_ufunc_test" sources
building extension "numpy.core.multiarray_tests" sources
building extension "numpy.core.operand_flag_tests" sources
building extension "numpy.fft.fftpack_lite" sources
building extension "numpy.linalg.lapack_lite" sources
adding 'numpy/linalg/lapack_litemodule.c' to sources.
adding 'numpy/linalg/lapack_lite/python_xerbla.c' to sources.
building extension "numpy.linalg._umath_linalg" sources
adding 'numpy/linalg/umath_linalg.c.src' to sources.
adding 'numpy/linalg/lapack_lite/python_xerbla.c' to sources.
building extension "numpy.random.mtrand" sources
building data_files sources
build_src: building npy-pkg config files
running build_py
copying numpy/version.py -> build/lib.macosx-10.10-x86_64-2.7/numpy
copying build/src.macosx-10.10-x86_64-2.7/numpy/__config__.py -> build/lib.macosx-10.10-x86_64-2.7/numpy
copying build/src.macosx-10.10-x86_64-2.7/numpy/distutils/__config__.py -> build/lib.macosx-10.10-x86_64-2.7/numpy/distutils
running build_clib
customize UnixCCompiler
customize UnixCCompiler using build_clib
running build_ext
customize UnixCCompiler
customize UnixCCompiler using build_ext
running build_scripts
adding 'build/scripts.macosx-10.10-x86_64-2.7/f2py' to scripts
running install_lib
copying build/lib.macosx-10.10-x86_64-2.7/numpy/__config__.py -> /Users/mac/.virtualenvs/env1/lib/python2.7/site-packages/numpy
copying build/lib.macosx-10.10-x86_64-2.7/numpy/distutils/__config__.py -> /Users/mac/.virtualenvs/env1/lib/python2.7/site-packages/numpy/distutils
copying build/lib.macosx-10.10-x86_64-2.7/numpy/version.py -> /Users/mac/.virtualenvs/env1/lib/python2.7/site-packages/numpy
byte-compiling /Users/mac/.virtualenvs/env1/lib/python2.7/site-packages/numpy/__config__.py to __config__.pyc
byte-compiling /Users/mac/.virtualenvs/env1/lib/python2.7/site-packages/numpy/distutils/__config__.py to __config__.pyc
byte-compiling /Users/mac/.virtualenvs/env1/lib/python2.7/site-packages/numpy/version.py to version.pyc
running install_scripts
changing mode of /Users/mac/.virtualenvs/env1/bin/f2py to 755
changing mode of /Users/mac/.virtualenvs/env1/bin/f2py2.7 to 755
running install_data
copying site.cfg -> /Users/mac/.virtualenvs/env1/lib/python2.7/site-packages/numpy/distutils/
copying build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/__ufunc_api.h -> /Users/mac/.virtualenvs/env1/lib/python2.7/site-packages/numpy/core/include/numpy
copying build/src.macosx-10.10-x86_64-2.7/numpy/core/include/numpy/ufunc_api.txt -> /Users/mac/.virtualenvs/env1/lib/python2.7/site-packages/numpy/core/include/numpy
copying build/src.macosx-10.10-x86_64-2.7/numpy/core/lib/npy-pkg-config/npymath.ini -> /Users/mac/.virtualenvs/env1/lib/python2.7/site-packages/numpy/core/lib/npy-pkg-config
copying build/src.macosx-10.10-x86_64-2.7/numpy/core/lib/npy-pkg-config/mlib.ini -> /Users/mac/.virtualenvs/env1/lib/python2.7/site-packages/numpy/core/lib/npy-pkg-config
running install_egg_info
Writing /Users/mac/.virtualenvs/env1/lib/python2.7/site-packages/numpy-1.10.4-py2.7.egg-info
running install_clib
(env1)numpy-1.10.4>cd ~
(env1)~>python
Python 2.7.11 (default, Jan 23 2016, 00:33:41)
[GCC 4.2.1 Compatible Apple LLVM 6.1.0 (clang-602.0.53)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy;numpy.show_config()
lapack_opt_info:
libraries = ['openblas', 'openblas']
library_dirs = ['/usr/local/opt/openblas/lib']
define_macros = [('HAVE_CBLAS', None)]
language = c
runtime_library_dirs = ['/usr/local/opt/openblas/lib']
blas_opt_info:
libraries = ['openblas', 'openblas']
library_dirs = ['/usr/local/opt/openblas/lib']
define_macros = [('HAVE_CBLAS', None)]
language = c
runtime_library_dirs = ['/usr/local/opt/openblas/lib']
openblas_info:
libraries = ['openblas', 'openblas']
library_dirs = ['/usr/local/opt/openblas/lib']
define_macros = [('HAVE_CBLAS', None)]
language = c
runtime_library_dirs = ['/usr/local/opt/openblas/lib']
openblas_lapack_info:
libraries = ['openblas', 'openblas']
library_dirs = ['/usr/local/opt/openblas/lib']
define_macros = [('HAVE_CBLAS', None)]
language = c
runtime_library_dirs = ['/usr/local/opt/openblas/lib']
blas_mkl_info:
NOT AVAILABLE
>>>Ctrl-D
(env1)~>otool -L ~/.virtualenvs/env1/lib/python2.7/site-packages/numpy/core/multiarray.so
/Users/mac/.virtualenvs/env1/lib/python2.7/site-packages/numpy/core/multiarray.so:
/System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate (compatibility version 1.0.0, current version 4.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0)
====================
SO WHAT IS THE PROBLEM!!
why numpy still builded with apple's Accelerate Framework, rather than openblas?
if I use:
brew install numpy --with-openblas
everything will be OK, numpy.show_config() and otool will show that its been build against openblas, however I need to make sure that in a virtualenv, which brew install will not assure of it.
I had been read carefully in this http://numpy-discussion.10968.n7.nabble.com/OpenBLAS-on-Mac-td36544.html
but still I cannot get it done, I have tried recreate a virtualenv and no luck.
Any help will be appreciated!
problem solved, but I don't know how, same process:
compile openblas:
make && make install
then:
export PATH="/opt/local/bin:/opt/local/sbin:/usr/local/Cellar/bin:/usr/local/bin:$PATH"
export PATH="/Developer/NVIDIA/CUDA-7.5/bin:$PATH"
export DYLD_LIBRARY_PATH="/opt/OpenBLAS/lib:/Developer/NVIDIA/CUDA-7.5/lib:$DYLD_LIBRARY_PATH"
export LD_LIBRARY_PATH="/opt/OpenBLAS/lib:$LD_LIBRARY_PATH"
export LDFLAGS="-L/usr/local/opt/openssl/lib -L/opt/OpenBLAS/lib"
export CPPFLAGS="-I/usr/local/opt/openssl/include -I/opt/OpenBLAS/include"
then:
vi numpy-1.10.4/site.cfg
[openblas]
libraries = openblas
library_dirs = /opt/OpenBLAS/lib
include_dirs = /opt/OpenBLAS/include
runtime_library_dirs = /opt/OpenBLAS/lib
then use:
python setup.py config
python setup.py build --fcompiler=gnu95
python setup.py install
otool -L /Users/mac/.virtualenvs/env1/lib/python2.7/site-packages/numpy/core/multiarray.so
multiarray.so:
/opt/OpenBLAS/lib/libopenblas_haswellp-r0.2.15.dylib (compatibility version 0.0.0, current version 0.0.0)
/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1213.0.0)
It just can!
problem solved confirm. Maybe its because of some stupid mistake.
I am trying to install PIL inside virtual as below
pip intall PIL
And i got the below error
Downloading/unpacking PIL
Running setup.py egg_info for package PIL
WARNING: '' not a valid package name; please use only.-separated package names in setup.py
Installing collected packages: PIL
Running setup.py install for PIL
WARNING: '' not a valid package name; please use only.-separated package names in setup.py
--- using frameworks at /System/Library/Frameworks
building '_imagingft' extension
cc -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -Qunused-arguments -Qunused-arguments -pipe -Wno-error=unused-command-line-argument-hard-error-in-future -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -I/usr/local/include/freetype2 -IlibImaging -I/opt/local/include -I/Users/user/.virtualenvs/proj/include -I/usr/local/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c _imagingft.c -o build/temp.macosx-10.9-intel-2.7/_imagingft.o
_imagingft.c:73:10: fatal error: 'freetype/fterrors.h' file not found
#include <freetype/fterrors.h>
^
1 error generated.
error: command 'cc' failed with exit status 1
Complete output from command /Users/user/.virtualenvs/proj/bin/python -c "import setuptools;__file__='/Users/user/.virtualenvs/proj/build/PIL/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/x1/1bwt313j0qvgdh5pfzpbpvcw0000gn/T/pip-R5JSVs-record/install-record.txt --single-version-externally-managed --install-headers /Users/user/.virtualenvs/proj/include/site/python2.7:
WARNING: '' not a valid package name; please use only.-separated package names in setup.py
running install
running build
running build_py
running build_ext
--- using frameworks at /System/Library/Frameworks
building '_imagingft' extension
cc -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -Qunused-arguments -Qunused-arguments -pipe -Wno-error=unused-command-line-argument-hard-error-in-future -I/System/Library/Frameworks/Tcl.framework/Headers -I/System/Library/Frameworks/Tk.framework/Headers -I/usr/local/include/freetype2 -IlibImaging -I/opt/local/include -I/Users/user/.virtualenvs/proj/include -I/usr/local/include -I/usr/include -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c _imagingft.c -o build/temp.macosx-10.9-intel-2.7/_imagingft.o
_imagingft.c:73:10: fatal error: 'freetype/fterrors.h' file not found
#include <freetype/fterrors.h>
^
1 error generated.
error: command 'cc' failed with exit status 1
----------------------------------------
Command /Users/user/.virtualenvs/proj/bin/python -c "import setuptools;__file__='/Users/user/.virtualenvs/proj/build/PIL/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/x1/1bwt313j0qvgdh5pfzpbpvcw0000gn/T/pip-R5JSVs-record/install-record.txt --single-version-externally-managed --install-headers /Users/user/.virtualenvs/proj/include/site/python2.7 failed with error code 1 in /Users/user/.virtualenvs/proj/build/PIL
So how to fix it and can some one let me know why it is throwing me an error on my new macbook air ?
I suggest you try Pillow, the fork of PIL. I've not tried in a virtual machine but on Mac OS X you can do this:
brew install libtiff libjpeg webp little-cms2
pip install Pillow
Full details here: http://pillow.readthedocs.org/en/latest/installation.html#mac-os-x-installation