Let me explain the situation.
I compiled a c++ library using system g++ (under linux).
Then built a cython module which calls a function in the pre-compiled library. Building the cython module was done under an anaconada venv (but no g++ installed inside the venv. Hence cython must had used system gcc/g++).
Now under the same anaconda venv, importing the cython module results in an error as:
anaconda3/envs/hspy/lib/libstdc++.so.6: version `GLIBCXX_3.4.21' not found (required by ./libc_rd_halo.so).
It seems like the c library (libc_rd_halo.so) which was compiled by system g++ is now looking for an anaconda g++ library. And I don't have g++ installed in anaconda venv.
I can fix the problem by using anaconda g++ to compile the c library from the beginning. Build the cython module again, and then I can import the module.
But I'd like to let anaconda cython use system compiler and look for the system compiler library when the module is being imported.
My questions are:
why does anaconda cython look for a local anaconda g++ by default, but not the system compiler? any benefit doing so?
How can I make anaconda cython look for system compiler library?
why does anaconda cython look for a local anaconda g++ by default, but not the system compiler? any benefit doing so?
How can I make anaconda cython look for system compiler library?
Because libraries compiled with one version of GCC can only be linked with other libraries that were compiled with a compatible version of GCC, as you found out.
You don't. Per above, binaries build with the system compiler may or may not work with other anaconda binaries built using the anaconda compiler.
If you want your binary to use system libraries then you need to make a system package, not an anaconda package. The system package will then only work on that system (and version of).
Related
My ultimate goal is to get python package graph_tool working on my system and also on ipynb if possible. I have already brew install graph-tool, as indicated here, but that's still insufficient.
So I follow conda instructions here, and I make decent progress, until I encounter a compiler issue with the configuration.
$ conda create -n py36env python=3.6.3 anaconda
(py36env) $ conda install -c conda-forge cgal
... and so forth with the other required libraries
(py36env) Tams-MacBook-Pro:graph-tool-2.25 tamtam$ ./configure --prefix=/Users/tamtam/anaconda3/envs/py36env/ --with-python-module-path=/Users/tamtam/anaconda3/envs/py36env/lib/python3.6/site-packages
.
.
.
checking whether g++ supports C++14 features by default... no
checking whether g++ supports C++14 features with -std=gnu++14... no
checking whether g++ supports C++14 features with -std=gnu++1y... no
configure: error: *** A compiler with support for C++14 language features is required.
I'm not very familiar with compiler things, but I check my system as follows:
(py36env) Tams-MacBook-Pro:graph-tool-2.25 tamtam$ conda list | grep gcc
gcc 4.8.5 8
And outside of the conda env:
$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/Users/tamtam/anaconda3/envs/py36env/bin/../libexec/gcc/x86_64-apple-darwin11.4.2/4.8.5/lto-wrapper
Target: x86_64-apple-darwin11.4.2
Configured with: ./configure --prefix=/Users/ray/mc-x64-3.5/conda-bld/gcc-4.8_1477649012852/
Thread model: posix
gcc version 4.8.5 (GCC)
And with Homebrew
$ brew list --versions | grep gcc
gcc 7.1.0 7.2.0
$ /usr/local/bin/gcc-7 --version
gcc-7 (Homebrew GCC 7.2.0) 7.2.0
$ echo $PATH
/Users/tamtam/anaconda3/bin:/Users/tamtam/anaconda/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin
So my questions are: EDITS INCLUDED
1) What's the difference between the conda gcc 4.8.5 and Homebrew gcc 7.2 on my system? I'm confused because they're each up-to-date packages.
EDIT: Nowhere online that I perused explicitly states this, but I am understanding (aka educated guess bc it's 3am and I need sleep) conda gcc 4.8.5 to just be equivalent to brew gcc 4.8.5, and that Anaconda is just way behind on developing an 'official' gcc supporting C++14. In the meantime I conda install -c salford_systems gcc-5 and ./configure seems to find itself a C++14 compiler. YAY (but this also led me to a new problem T.B.A.)
NOTE: I had also tried running the ./configure file with both conda gcc 4.8.5 and brew gcc 7.2.0 (switching of default gcc compiler guided here), but still same error of ./configure not finding C++14 compile (so ignore this)
2) I also have Xcode 8.3.3 that also comes with clang, which is what gcc is symlinked to... ?? No idea about the setup/relation
3) How can I adjust my system (for example, paths) so that the configuration process will detect a C++14 compiler? EDIT: resolved per 1)
(This solution is also included within the edits at the bottom of my question.)
Nowhere online that I perused explicitly states this, but I am understanding (aka educated guess bc it's 3am and I need sleep) conda gcc 4.8.5 to just be equivalent to brew gcc 4.8.5, and that Anaconda is just way behind on developing an 'official' gcc supporting C++14. In the meantime I conda install -c salford_systems gcc-5 and ./configure seems to find itself a C++14 compiler. YAY (but this also led me to a new problem)
NOTE: I had also tried running the ./configure file with both conda gcc 4.8.5 and brew gcc 7.2.0 (switching of default gcc compiler guided here), but still same error of ./configure not finding C++14 compile arises (so ignore this)
As a rule of thumb, if a product provides its own build tools (compiler, C headers, static libraries, include/library paths etc) instead of using the system's ones, you should use those when building things to use within its environment.
That's because, depending on include files, compiler, compiler version, compiler flags, predefined macros etc the same sources and libraries built with different toolchains can be binary incompatible. This goes double for C++ where there are no ABI standards at all. (In C, there are de facto ones for most platforms where there are competing compilers due to the need to do system calls.)
Now, what you have on your system:
The system's stock gcc (if any) is at /usr/bin (so includes are at /usr/include, static libraries at /usr/lib etc)
Homebrew's gcc 7.2 at /usr/local/bin (includes are probably at /usr/local/include)
Anaconda also has a gcc package that, if installed, would reside somewhere like /Users/<login>/anaconda3/envs/<env name>/bin
If an environment provides an own toolchain, it provides some way to make build scripts select that toolchain when building things. A common way is to set relevant environment variables (PATH, INCLUDE, LIB) - either directly or through some tool that the environment provides. Homebrew executables reside in /usr/local/bin which is always present in UNIX PATH as per the FHS; Anaconda adds itself to PATH whenever its environment is activated.
So, as you yourself guessed, since Anaconda provides its own gcc packages (thus enforces the compatibility of their output with its binary packages via package metadata - if there are known incompatibilities, the installation will fail due to requirement conflicts), you need to install one that meets graph-tool requirements -- which is gcc 5, available via gcc-5 package from salford_systems channel. Then specify --prefix as you already have.
You may need to install other dependencies, too, if you didn't already or you need optional features that they're required for.
I'm trying to use a cgo library on Windows, namely github.com/mattn/go-sqlite3
I use Cygwin64 and installed with all "Development" packages, so gcc is availabe.
But running go get github.com/mattn/go-sqlite3 results in:
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: cannot find -lmingwex
/usr/lib/gcc/x86_64-pc-cygwin/5.3.0/../../../../x86_64-pc-cygwin/bin/ld: cannot find -lmingw32
If I search for "mingwex" and "mingw32" in the Cygwin installer, I get no results. Am I looking for the wrong names or are they not available on 64 bit systems?
Or is there a better way to use the library on Windows?
Note that the README states that
However, if you install go-sqlite3 with go install
github.com/mattn/go-sqlite3, you don't need gcc to build your app
anymore
but I get the same error message if I use go install.
$ go version
go version go1.6.2 windows/amd64
What finally worked for me (instead of Cygwin) is to download TDM MinGW-w64 from http://tdm-gcc.tdragon.net/download and set the PATH such that gcc from C:\TDM-GCC-64\bin is used.
You can also install package mingw64-i686-gcc-core from Cygwin.
The binary will be /usr/bin/i686-w64-mingw32-gcc.exe so you probably need to copy it as /usr/bin/gcc.exe.
I have encountered the same issue as well. It seems to me that cygwin is not fully compatible with cgo. Instead, I have used https://mingw-w64.org.
From the cgo documentation https://github.com/golang/go/wiki/cgo:
In order to use cgo on Windows, you'll also need to first install a gcc compiler (for instance, mingw-w64) and have gcc.exe (etc.) in your PATH environment variable before compiling with cgo will work.
I try to compile Octave 3.8.2 on CentOS 6.4 （serve）, but there's an error:
A BLAS library was detected but found incompatible with your Fortran 77 compiler settings.
I'm so confused. I compiled a BLAS and a LAPACK myself, and I added their path to bashrc. But this error appeared again. I try to change fflags to ff2c as other resources suggested, but this error also occurred.
I found the following guide and discussion of my error.
I am going through this, and I had the same issues that you are mentioning. I solved it by using the auto config options to point it to the correct library, e.g.
./configure --with-lapack=/path/to/liblapack.so.3 --with-blas=/path/to/libf77blas.so.3
It compiles and produces an executable.
I would suggest using OpenBLAS instead of the common BLAS and LAPACK packages. It is written in the Octave documentation:
"Accelerated BLAS libraries such as OpenBLAS (https://www.openblas.net/) or ATLAS (http://math-atlas.sourceforge.net) are recommended for best performance. The reference implementation (http://www.netlib.org/blas) is slow, unmaintained, and suffers from certain bugs in corner case inputs."
You can download it from https://www.openblas.net/ and install it with the classical ./configure && make %% sudo make install. After this move the libraries generated by OpenBLAS to /usr/lib64 and run:
./configure -with-blas="-lopenblas"
Add other flags and arguments to configure if needed.
I'm having trouble building the hmatrix library on OS X Lion. Looking at the .cabal file, it requires the gsl library, so I installed it with macports. The .a files are in /opt/local/lib and the .h files are in /opt/local/include/gsl
As suggested here I changed the built-type from Custom to Simple. (without that change I get a similar error).
When I use cabal configure I get the following output:
* Missing C library: gsl
This problem can usually be solved by installing the system package that
provides this library (you may need the "-dev" version). If the library is
already installed but in a non-standard location then you can use the flags
--extra-include-dirs= and --extra-lib-dirs= to specify where it is.
So I tried cabal --extra-include-dirs=/opt/local/include --extra-lib-dirs=/opt/local/lib configure, but I still get the same error. I can compile and link a c program that includes gsl. What files is cabal looking for? If I have the right files, how do I tell it how to find them?
libgsl.a is a universal binary:
$ file /opt/local/lib/libgsl.a
/opt/local/lib/libgsl.a: Mach-O universal binary with 2 architectures
/opt/local/lib/libgsl.a (for architecture x86_64): current ar archive random library
/opt/local/lib/libgsl.a (for architecture i386): current ar archive random library
ghc looks like it's 64-bit:
$ ghc --info
[("Project name","The Glorious Glasgow Haskell Compilation System")
,("GCC extra via C opts"," -fwrapv")
,("C compiler command","/usr/bin/llvm-gcc")
,("C compiler flags"," -m64 -fno-stack-protector -m64")
,("ar command","/usr/bin/ar")
,("ar flags","clqs")
,("ar supports at file","NO")
,("touch command","touch")
,("dllwrap command","/bin/false")
,("windres command","/bin/false")
,("perl command","/usr/bin/perl")
,("target os","OSDarwin")
,("target arch","ArchX86_64")
,("target word size","8")
,("target has GNU nonexec stack","False")
,("target has subsections via symbols","True")
,("Project version","7.4.2")
,("Booter version","7.4.2")
,("Stage","2")
,("Build platform","x86_64-apple-darwin")
,("Host platform","x86_64-apple-darwin")
,("Target platform","x86_64-apple-darwin")
,("Have interpreter","YES")
,("Object splitting supported","NO")
,("Have native code generator","YES")
,("Support SMP","YES")
,("Unregisterised","NO")
,("Tables next to code","YES")
,("RTS ways","l debug thr thr_debug thr_l thr_p dyn debug_dyn thr_dyn thr_debug_dyn")
,("Leading underscore","YES")
,("Debug on","False")
,("LibDir","/usr/local/Cellar/ghc/7.4.2/lib/ghc-7.4.2")
,("Global Package DB","/usr/local/Cellar/ghc/7.4.2/lib/ghc-7.4.2/package.conf.d")
,("Gcc Linker flags","[\"-m64\"]")
,("Ld Linker flags","[\"-arch\",\"x86_64\"]")
]
As an alternative to mac-ports you can use the nix package manager for mac. It does a pretty good job of taking care of the c dependancies for for the libraries available through it. In general I have been more happy with it then any other package manager on mac.
Unfortunately mac(darwin) unlike for linux does not have as many binaries available through nix so installing ghc often means waiting for it to compile.
The commands to install ghc and hmatrix after installation of nix are:
nix-env -iA nixpkgs-unstable.haskellPackages.ghc
nix-env -iA nixpkgs-unstable.haskellPackages.hmatrix
All of the needed dependencies will be taken care of for you.
I just tried it on my macbook pro and hmatrix seems to be working correctly in ghci after trying commands from the first few pages of the tutorial.
I'm not a mac person, but it really sounds like you haven't installed the "-dev" version. For a mac, I suspect you need to install gsl-devel in addition to gsl. If the problem persists, verify that you have libgsl0-dev on your library path.
I have a C++ shared library being called from my Python program. The C++ lib is compiled with HDF5 which I installed using homebrew on OS X, so it resides in /usr/local/lib. Now the problem is that I also have PyTables installed, which includes an older version of libhdf5, installed somewhere in /Library/Frameworks/EPD64.framework/Versions/Current/...
I can compile and link my library just fine, and otool -L indeed points to /usr/local/lib/libhdf5.dylib. However, when I try to run it from Python, there is a version mismatch error. Somehow the runtime linker is loading the older hdf5 library instead.
One workaround is to build hdf5 as a static library instead, but I'd like to find a neater solution - how can I persuade the runtime linker to use the newer library? I already tried setting DYLD_LIBRARY_PATH but that just broke everything else (Python and MacVim wouldn't start).
Can you provide more info? I have a few alternatives (and questions).
If I understood correctly, you have a library that uses lib_ver_a.dyld and pyTables that uses lib_ver_b.dyld. I.e. the same library but different version.
You could either:
Compile your library to use the same version as pyTables (DYLD_LIB... while you compile) and run your program with pyTables library: DYLD_LIBRARY_PATH=/correct/path/lib python myprog.py
Upgrade your xcode (which version you have? also which osx version?). This upgrades your libraries and pyTables (be careful, newest is not necessary the most stable)
You should also check which python you are calling (a custom one or apple one).
There is also a possibility to define a specific dyld while linking (Do you use gcc or ld for linking? I might miss some options here):
gcc /path/lib1.dyld myLib.c -o myLib.o
This way your library is linked to certain version of the library (caution! I don't remember the exact details, but I can dig them out).
I hope this helps.
br,
Juha