XGboost installation error in Mac OS X Sierra - macos

Am trying to install Xgboost on my Mac. Python version is 3.6
sudo pip install xgboost
am getting the following error
Complete output from command python setup.py egg_info:
rm -f -rf build build_plugin lib bin *~ /~ //*~ ///~ /.o //*.o ///.o xgboost
clang-omp++ -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/learner.o src/learner.cc >build/learner.d
/bin/sh: clang-omp++: command not found
make: * [build/learner.o] Error 127
make: Waiting for unfinished jobs....
clang-omp++ -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/logging.o src/logging.cc >build/logging.d
/bin/sh: clang-omp++: command not found
make: [build/logging.o] Error 127
-----------------------------
Building multi-thread xgboost failed
Start to build single-thread xgboost
rm -f -rf build build_plugin lib bin *~ /~ //*~ ///~ /.o //*.o ///.o xgboost
clang-omp++ -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/learner.o src/learner.cc >build/learner.d
/bin/sh: clang-omp++: command not found
make: [build/learner.o] Error 127
make: Waiting for unfinished jobs....
clang-omp++ -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/logging.o src/logging.cc >build/logging.d
/bin/sh: clang-omp++: command not found
make: * [build/logging.o] Error 127
Successfully build single-thread xgboost
If you want multi-threaded version
See additional instructions in doc/build.md
Traceback (most recent call last):
File "", line 1, in
File "/private/tmp/pip-build-TCLoWG/xgboost/setup.py", line 29, in
LIB_PATH = libpath'find_lib_path'
File "/private/tmp/pip-build-TCLoWG/xgboost/xgboost/libpath.py", line 45, in find_lib_path
'List of candidates:\n' + ('\n'.join(dll_path)))
builtin.XGBoostLibraryNotFound: Cannot find XGBoost Libarary in the candicate path, did you install compilers and run build.sh in root path?
List of candidates:
/private/tmp/pip-build-TCLoWG/xgboost/xgboost/libxgboost.so
/private/tmp/pip-build-TCLoWG/xgboost/xgboost/../../lib/libxgboost.so
/private/tmp/pip-build-TCLoWG/xgboost/xgboost/./lib/libxgboost.so
This is what happening, I don't know what's happening.

Do this:
$ brew install gcc#5
$ pip install xgboost
Info here https://github.com/dmlc/xgboost/issues/1501

Although I installed gcc 5 (with brew), the xgboost build still used the LLVM version of gcc (7 in my case). To get the build to use gcc 5 I did:
$ brew install gcc5
$ env CC=gcc-5 CXX=g++-5 pip install xgboost
and this worked for me.
(davidread's answer helps me very much! thank you #davidread)

I was also facing the same problem while installing by pip but installed successfully by this link.

Related

pip install xgboost with error Command "python setup.py egg_info" failed with error code 1 on mac (Python 2.7)

When I executed:
pip install xgboost
I got this:
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/w4/w49s3mn91plbhyjwf8qjkpxc0000gn/T/pip-build-61dms1/xgboost/
Here is my log:
liumengyaodeMacBook-Pro:~ liumengyao$ pip install xgboost
Collecting xgboost
Using cached xgboost-0.6a2.tar.gz
Complete output from command python setup.py egg_info:
rm -f -rf build build_plugin lib bin *~ */*~ */*/*~ */*/*/*~ */*.o */*/*.o */*/*/*.o xgboost
clang-omp++ -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/learner.o src/learner.cc >build/learner.d
/bin/sh: clang-omp++: command not found
make: *** [build/learner.o] Error 127
make: *** Waiting for unfinished jobs....
clang-omp++ -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/logging.o src/logging.cc >build/logging.d
/bin/sh: clang-omp++: command not found
make: *** [build/logging.o] Error 127
-----------------------------
Building multi-thread xgboost failed
Start to build single-thread xgboost
rm -f -rf build build_plugin lib bin *~ */*~ */*/*~ */*/*/*~ */*.o */*/*.o */*/*/*.o xgboost
clang-omp++ -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/learner.o src/learner.cc >build/learner.d
/bin/sh: clang-omp++: command not found
make: *** [build/learner.o] Error 127
make: *** Waiting for unfinished jobs....
clang-omp++ -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -MM -MT build/logging.o src/logging.cc >build/logging.d
/bin/sh: clang-omp++: command not found
make: *** [build/logging.o] Error 127
Successfully build single-thread xgboost
If you want multi-threaded version
See additional instructions in doc/build.md
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/private/var/folders/w4/w49s3mn91plbhyjwf8qjkpxc0000gn/T/pip-build-61dms1/xgboost/setup.py", line 29, in <module>
LIB_PATH = libpath['find_lib_path']()
File "/private/var/folders/w4/w49s3mn91plbhyjwf8qjkpxc0000gn/T/pip-build-61dms1/xgboost/xgboost/libpath.py", line 45, in find_lib_path
'List of candidates:\n' + ('\n'.join(dll_path)))
__builtin__.XGBoostLibraryNotFound: Cannot find XGBoost Libarary in the candicate path, did you install compilers and run build.sh in root path?
List of candidates:
/private/var/folders/w4/w49s3mn91plbhyjwf8qjkpxc0000gn/T/pip-build-61dms1/xgboost/xgboost/libxgboost.so
/private/var/folders/w4/w49s3mn91plbhyjwf8qjkpxc0000gn/T/pip-build-61dms1/xgboost/xgboost/../../lib/libxgboost.so
/private/var/folders/w4/w49s3mn91plbhyjwf8qjkpxc0000gn/T/pip-build-61dms1/xgboost/xgboost/./lib/libxgboost.so
----------------------------------------
Command "python setup.py egg_info" failed with error code 1 in /private/var/folders/w4/w49s3mn91plbhyjwf8qjkpxc0000gn/T/pip-build-61dms1/xgboost/
I have tried several methods:
python ez_setup.py (the file downloaded from website)
pip install --upgrade setuptools
pip install distribute
But I still cannot solve this problem.
This is how I installed it for Python 3.7:
brew install gcc#7
pip3 install
This is an old question, but I had an issue and realized pip was out of date. Solved my issue.
pip install --upgrade pip
pip install xgboost

Install xgboost on Mac - ld: library not found

I am trying to install OpenMP enabled xgboost on my mac. I installed gcc with no problem:
brew install gcc --without-multilib
then cloned git repository:
git clone --recursive https://github.com/dmlc/xgboost
cd xgboost; cp make/config.mk ./config.mk
but I get an error when I do
make -j4
Here is the error I get. I'd appreciate if you can help fixing this problem:
c++ -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -o xgboost build/cli_main.o build/learner.o build/logging.o build/c_api/c_api.o build/c_api/c_api_error.o build/common/common.o build/data/data.o build/data/simple_csr_source.o build/data/simple_dmatrix.o build/data/sparse_page_dmatrix.o build/data/sparse_page_raw_format.o build/data/sparse_page_source.o build/data/sparse_page_writer.o build/gbm/gblinear.o build/gbm/gbm.o build/gbm/gbtree.o build/metric/elementwise_metric.o build/metric/metric.o build/metric/multiclass_metric.o build/metric/rank_metric.o build/objective/multiclass_obj.o build/objective/objective.o build/objective/rank_obj.o build/objective/regression_obj.o build/tree/tree_model.o build/tree/tree_updater.o build/tree/updater_colmaker.o build/tree/updater_histmaker.o build/tree/updater_prune.o build/tree/updater_refresh.o build/tree/updater_skmaker.o build/tree/updater_sync.o dmlc-core/libdmlc.a rabit/lib/librabit.a -pthread -lm -fopenmp
c++ -std=c++0x -Wall -O3 -msse2 -Wno-unknown-pragmas -funroll-loops -Iinclude -Idmlc-core/include -Irabit/include -fPIC -fopenmp -shared -o lib/libxgboost.so build/learner.o build/logging.o build/c_api/c_api.o build/c_api/c_api_error.o build/common/common.o build/data/data.o build/data/simple_csr_source.o build/data/simple_dmatrix.o build/data/sparse_page_dmatrix.o build/data/sparse_page_raw_format.o build/data/sparse_page_source.o build/data/sparse_page_writer.o build/gbm/gblinear.o build/gbm/gbm.o build/gbm/gbtree.o build/metric/elementwise_metric.o build/metric/metric.o build/metric/multiclass_metric.o build/metric/rank_metric.o build/objective/multiclass_obj.o build/objective/objective.o build/objective/rank_obj.o build/objective/regression_obj.o build/tree/tree_model.o build/tree/tree_updater.o build/tree/updater_colmaker.o build/tree/updater_histmaker.o build/tree/updater_prune.o build/tree/updater_refresh.o build/tree/updater_skmaker.o build/tree/updater_sync.o dmlc-core/libdmlc.a rabit/lib/librabit.a -pthread -lm -fopenmp
clangclang: : warningwarning: : argument unused during compilation: '-pthread'argument unused during compilation: '-pthread'
ld: library not found for -lgomp
ld: library not found for -lgomp
clang: error: clanglinker command failed with exit code 1 (use -v to see invocation):
error: linker command failed with exit code 1 (use -v to see invocation)
make: *** [lib/libxgboost.so] Error 1
make: *** Waiting for unfinished jobs....
make: *** [xgboost] Error 1
I have the same issue and solved it by:
brew install clang-omp
export CC=clang-omp
export CXX=clang-omp++
git clone --recursive https://github.com/dmlc/xgboost
cd xgboost; cp make/config.mk ./config.mk; make -j4
cd python-package
sudo python setup.py install
If you used pip to install xgboost before, then delete the previous installed xgboost from your project. Then use
pip install xgboost
to install it again.
A new versions of OSX has a Clang as a default c\c++ compiler. Therefore your c++ command refers to clang++.
You should define a CC\CXX environmental variables for your make command like this CC=gcc CXX=g++ make -j
Also you can build an OpenMP for clang OpenMPrt and customise your shell environment (I didn't try this by own)
(I have no mac at this moment to check this solution; just linux)

"cannot find -lssl; cannot find -lcrypto" when installing mysql-python?

I'm struggling to install the mysql-python pip either systemwide or in a venv on Ubuntu 14.04 with MariaDB 10 installed. Also tried with MariaDB 5.5 and getting the same error. I don't have this issue with vanilla mysql-server installed.
I have the following installed via apt-get:
build-essential
python-dev
libmariadbclient-dev (thats the MariaDB replacement for libmysqlclient-dev)
python-mysqldb
Originally I thought this was an issue installing this into a venv but I've subsequently noticed mysql-python won't install systemwide either. Below are the cmds I used to install in a venv.
virtualenv venv
. venv/bin/activate
pip install mysql-python==1.2.5
In file included from _mysql.c:44:0:
/usr/include/mysql/my_config.h:439:0: warning: "HAVE_WCSCOLL" redefined [enabled by default]
#define HAVE_WCSCOLL
^
In file included from /usr/include/python2.7/pyconfig.h:3:0,
from /usr/include/python2.7/Python.h:8,
from _mysql.c:29:
/usr/include/x86_64-linux-gnu/python2.7/pyconfig.h:911:0: note: this is the location of the previous definition
#define HAVE_WCSCOLL 1
^x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/_mysql.o -L/usr/lib/x86_64-linux-gnu -lmariadbclient_r -lpthread -lz -lm -lssl -lcrypto -ldl -o build/lib.linux-x86_64-2.7/_mysql.so
/usr/bin/ld: cannot find -lssl
/usr/bin/ld: cannot find -lcrypto
collect2: error: ld returned 1 exit status
error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
----------------------------------------
Cleaning up...
Command /root/env/bin/python -c "import setuptools, tokenize;__file__='/root/env/build/mysql- python/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-EyhO2v-record/install-record.txt --single-version-externally-managed --compile --install-headers /root/env/include/site/python2.7 failed with error code 1 in /root/env/build/mysql-python
Storing debug log for failure in /root/.pip/pip.log
You need to have installed the development libraries of OpenSSL. It can be a libssl-dev, libssl-devel if your distribution provides separated packages for the dev libraries. Or install the complete openssl package if they don't.
(venv)➜ src pip install mysql-python==1.2.5
Downloading/unpacking mysql-python==1.2.5
Downloading MySQL-python-1.2.5.zip (108kB): 108kB downloaded
Running setup.py (path:/home/braiam/src/venv/build/mysql-python/setup.py) egg_info for package mysql-python
Installing collected packages: mysql-python
Running setup.py install for mysql-python
building '_mysql' extension
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -Dversion_info=(1,2,5,'final',1) -D__version__=1.2.5 -I/usr/include/mysql -I/usr/include/python2.7 -c _mysql.c -o build/temp.linux-x86_64-2.7/_mysql.o -DBIG_JOINS=1 -fno-strict-aliasing -g -static-libgcc -fno-omit-frame-pointer -fno-strict-aliasing
In file included from _mysql.c:44:0:
/usr/include/mysql/my_config.h:439:0: warning: "HAVE_WCSCOLL" redefined
#define HAVE_WCSCOLL
^
In file included from /usr/include/python2.7/pyconfig.h:3:0,
from /usr/include/python2.7/Python.h:8,
from _mysql.c:29:
/usr/include/x86_64-linux-gnu/python2.7/pyconfig.h:911:0: note: this is the location of the previous definition
#define HAVE_WCSCOLL 1
^
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wl,-z,relro -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/_mysql.o -L/usr/lib/x86_64-linux-gnu -lmariadbclient_r -lpthread -lz -lm -lssl -lcrypto -ldl -o build/lib.linux-x86_64-2.7/_mysql.so
Successfully installed mysql-python
Cleaning up...
But, there were two messages that you have that I didn't, namely:
/usr/bin/ld: cannot find -lssl
/usr/bin/ld: cannot find -lcrypto
Which if you did ld -lcrypto --verbose or ld -lssl --verbose wouldn't produce this results:
➜ src ld -lcrypto --verbose | grep succeeded
attempt to open //usr/lib/x86_64-linux-gnu/libcrypto.so succeeded
ld: warning: cannot find entry symbol _start; not setting start address
➜ src ld -lssl --verbose | grep succeeded
attempt to open //usr/lib/x86_64-linux-gnu/libssl.so succeeded
ld: warning: cannot find entry symbol _start; not setting start address
So, to fix this just make sure you have installed the libssl-dev package which provides both libraries.
I meet the similar error on MacOS but there is no "/usr/bin/ld: cannot find -lssl" for me.
Solved with the following steps:
Step 1. Make sure you have installed openssl with homebrew.
brew install openssl
Step 2. In a terminal:
export LIBRARY_PATH=$LIBRARY_PATH:/usr/local/opt/openssl/lib/
For Linux you can also try yum install your missing library and add them to LIBRARY_PATH.
For Debian 9.x:
apt install libssl-dev
For CentOS:
sudo yum install openssl-devel

ld cannot find -l<library>

I am having trouble installing pyipopt on ubuntu 12.04. During linking, I receive the error:
/usr/bin/ld: cannot find -lcoinhsl
Even though I know that this library is installed and the .so and .la files are available in /home/mostafa/MyBuilds/CoinIpopt/build/lib/
does anyone have a solution for this?
below is the complete return of running setup.py build:
root#ubuntu:~/MyBuilds/pyipopt# sudo python setup.py build
running build
running build_ext
building 'pyipopt' extension
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/lib/python2.7/dist-packages/numpy/core/include -I/home/mostafa/MyBuilds/CoinIpopt/build/include/coin/ -I/usr/include/python2.7 -c src/callback.c -o build/temp.linux-x86_64-2.7/src/callback.o
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/lib/python2.7/dist-packages/numpy/core/include -I/home/mostafa/MyBuilds/CoinIpopt/build/include/coin/ -I/usr/include/python2.7 -c src/pyipopt.c -o build/temp.linux-x86_64-2.7/src/pyipopt.o
src/pyipopt.c: In function ‘set_intermediate_callback’:
src/pyipopt.c:452:15: warning: variable ‘myowndata’ set but not used [-Wunused-but-set-variable]
gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relrobuild/temp.linux-x86_64-2.7/src/callback.o build/temp.linux-x86_64-2.7/src/pyipopt.o -L/home/mostafa/MyBuilds/CoinIpopt/build/lib/ -lipopt -lblas -lcoinhsl -lcoinmetis -llapack -ldl -lm -o build/lib.linux-x86_64-2.7/pyipopt.so -Wl,--rpath -Wl,/home/mostafa/MyBuilds/CoinIpopt/build/lib/
/usr/bin/ld: cannot find -lcoinhsl
collect2: ld returned 1 exit status
error: command 'gcc' failed with exit status 1
-Ldir
Add directory dir to the list of directories to be searched for -l.
You may install your coinhsl library in one of your standard libraries directories and run 'ldconfig` before doing your ppyipopt install
I had a similar problem with another library and the reason why it didn't found it, was that I didn't run the make install (after running ./configure and make) for that library. The make install may require root privileges (in this case use: sudo make install). After running the make install you should have the so files in the correct folder, i.e. here /usr/local/lib and not in the folder mentioned by you.
you can add the Path to coinhsl lib to LD_LIBRARY_PATH variable. May be that will help.
export LD_LIBRARY_PATH=/xx/yy/zz:$LD_LIBRARY_PATH
where /xx/yy/zz represent the path to coinhsl lib.

Having difficulty to install git using Homebrew

I am watching this video to install git using Homebrew. I already got Homebrew installed. The problem i have is installing git using Homebrew. I typed brew install git and have certain error message.
Error msg:
./generate-cmdlist.sh > common-cmds.h+ && mv common-cmds.h+ common-cmds.h
/usr/bin/xcrun clang -o hex.o -c -MF ./.depend/hex.o.d -MMD -MP -Os -w -pipe -march=native -Qunused-arguments -I. -DUSE_ST_TIMESPEC -DNO_GETTEXT -DHAVE_DEV_TTY -DSHA1_HEADER='<openssl/sha.h>' -DNO_MEMMEM hex.c
/usr/bin/xcrun clang -o ident.o -c -MF ./.depend/ident.o.d -MMD -MP -Os -w -pipe -march=native -Qunused-arguments -I. -DUSE_ST_TIMESPEC -DNO_GETTEXT -DHAVE_DEV_TTY -DSHA1_HEADER='<openssl/sha.h>' -DNO_MEMMEM ident.c
/usr/bin/xcrun clang -o kwset.o -c -MF ./.depend/kwset.o.d -MMD -MP -Os -w -pipe -march=native -Qunused-arguments -I. -DUSE_ST_TIMESPEC -DNO_GETTEXT -DHAVE_DEV_TTY -DSHA1_HEADER='<openssl/sha.h>' -DNO_MEMMEM kwset.c
In file included from ident.c:8:
In file included from ./cache.h:4:
./git-compat-util.h:93:10: fatal error: 'unistd.h' file not found
#include <unistd.h>
^
In file included from hex.c:1:
In file included from ./cache.h:4:
./git-compat-util.h:93:10: fatal error: 'unistd.h' file not found
#include <unistd.h>
^
In file included from kwset.c:37:
In file included from ./cache.h:4:
./git-compat-util.h:93:10: fatal error: 'unistd.h' file not found
#include <unistd.h>
^
1 error generated.
make: *** [hex.o] Error 1
1 error generated.
make: *** Waiting for unfinished jobs....
make: *** [ident.o] Error 1
1 error generated.
make: *** [kwset.o] Error 1
==> Build Environment
CPU: quad-core 64-bit sandybridge
MacOS: 10.7.4-x86_64
Xcode: 4.3.2
CC: /usr/bin/xcrun clang => /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang
CXX: /usr/bin/xcrun clang++ => /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang++
LD: /usr/bin/xcrun clang => /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang
CFLAGS: -Os -w -pipe -march=native -Qunused-arguments
CXXFLAGS: -Os -w -pipe -march=native -Qunused-arguments
MAKEFLAGS: -j4
Error: Failed executing: make prefix=/usr/local/Cellar/git/1.7.10.3 CC=/usr/bin/xcrun\ clang CFLAGS=-Os\ -w\ -pipe\ -march=native\ -Qunused-arguments LDFLAGS= install (git.rb:49)
These existing issues may help you:
https://github.com/mxcl/homebrew/issues/8643
https://github.com/mxcl/homebrew/issues/9023
https://github.com/mxcl/homebrew/issues/9618
https://github.com/mxcl/homebrew/issues/10544
https://github.com/mxcl/homebrew/issues/11481
https://github.com/mxcl/homebrew/issues/12325
https://github.com/mxcl/homebrew/issues/12344
https://github.com/mxcl/homebrew/issues/12400
https://github.com/mxcl/homebrew/issues/12478
Otherwise, please report the bug:
https://github.com/mxcl/homebrew/wiki/reporting-bugs
How do i solve this error? Need some help
You must install the command-line tools through Xcode to get the needed headers.
I suggest you to download it. Also, if you use github, I suggest you to just read this page. One year ago I started in this way.

Resources