I would like to install bazel from source, and use bazel to compile tensorflow on a cluster running redhat 6.7. When I try to install bazel, the glibc version (2.12) is too old. I do not have root access to the cluster. Is it possible to install tensorflow in this case?
My system information:
-bash-4.1$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 6.7 (Santiago)
-bash-4.1$ which gcc
/usr/bin/gcc
-bash-4.1$ gcc -v
Using built-in specs.
Target: x86_64-redhat-linux
Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --with-bugurl=http://bugzilla.redhat.com/bugzilla --enable-bootstrap --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-gnu-unique-object --enable-languages=c,c++,objc,obj-c++,java,fortran,ada --enable-java-awt=gtk --disable-dssi --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre --enable-libgcj-multifile --enable-java-maintainer-mode --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --disable-libjava-multilib --with-ppl --with-cloog --with-tune=generic --with-arch_32=i686 --build=x86_64-redhat-linux
Thread model: posix
gcc version 4.4.7 20120313 (Red Hat 4.4.7-16) (GCC)
-bash-4.1$ ldd --version
ldd (GNU libc) 2.12
The system has newer gcc installed as well. I tried using it, bazel still won't compile.
-bash-4.1$ /usr/local/gcc/4.8.4/bin/gcc -v
Using built-in specs.
COLLECT_GCC=/usr/local/gcc/4.8.4/bin/gcc
COLLECT_LTO_WRAPPER=/usr/local/gcc/4.8.4/libexec/gcc/x86_64-unknown-linux-gnu/4.8.4/lto-wrapper
Target: x86_64-unknown-linux-gnu
Configured with: ../configure --prefix=/usr/local/gcc/4.8.4
Thread model: posix
gcc version 4.8.4 (GCC)
When I was compiling bazel, I got the following error:
bazel-0.1.1/_bin/build-runfiles: /usr/lib64/libstdc++.so.6: version `GLIBCXX_3.4.14' not found
Some people also reported this issue:
https://github.com/tensorflow/tensorflow/issues/110
and https://github.com/tensorflow/tensorflow/issues/527
How can I install the missing dependency locally, and have bazel pick up the right library?
You should be able to compile from source with a newer version of Bazel exporting LD_FLAGS, CXX and CC, and tweaking the tools/cpp/CROSSTOOL file of Bazel. Please open a github issue on Bazel (https://github.com/bazelbuild/bazel/issues) if you have further questions.
I am currently working on making all that easier. Sorry for the mess.
In case anyone need to do this manually:
Compile up-to-date glibc, gcc and all their dependencies from source with the option --disable-rpath to avoid glibc path being hard coded to system default. Do NOT add glibc to LD_LIBRARY_PATH directly, or all executables including rm will stop working.
Compile python with your gcc, install pip and the official wheel
./configure --prefix=$PWD/build --enable-unicode=ucs4 --with-cxx-main=g++ && make && make install
Start python with the following command to load the correct glibc, where ${GLIBC_PATH} is your install location to glibc, make an alias
alias tensorflow='${GLIBC_PATH}/lib/ld-2.23.so --library-path ${GLIBC_PATH}/lib:${LD_LIBRARY_PATH}which python'
Import tensorflow to check no error occur
After able to import tensorflow module without any error, you can use any other computer (possibly ubuntu VM on your PC) to compile a custom wheel with machine specific option, and copy that to your cluster following the guide
bazel build -c opt --copt=-mavx --copt=-mavx2 --copt=-mfma --copt=-mfpmath=both --copt=-msse4.2 --copt=-msse4.1 //tensorflow/tools/pip_package:build_pip_package
Related
I'm on a CentOS 7 Linux system where I don't have the root access. So I have to build gcc from source. I want to build gcc 11.2 with the jit support. I invoked
contrib/download_prerequisites for GMP, MPFR, MPC and ISL. Then from a build directory I configured gcc using
../gcc-11.2.0/configure --prefix=/some/path --enable-bootstrap --enable-shared \
--enable-host-shared --enable-threads=posix --enable-languages=c,c++,jit \
--disable-multilib
then I hit make.
It failed with ld errors asking me to recompile libisl.a, libmpc.a, libmpfr.a, libgmp.a with -fPIC. Right before these errors I saw lots of jit related outputs. I assume jit is behind all of this?
Then I tried adding CFLAGS=-fPIC to the above configure command as ../gcc-11.2.0/configure CFLAGS=-fPIC *rest_of_the_options_above*. The same ld errors were emitted.
I'm aware that those dependencies each has --with-pic option to their own configure. But I'm wondering if there is a way to have gcc invoke that for me? If not, does it mean that to build gcc with jit support, one has to build the four dependencies manually with --with-pic? Thanks.
I am building a custom kernel using Buildroot 2020.02.1 for a PC mounting a AMD GX-222GC SOC.
The default external toolchain is amd-2016.11-19 from CodeBench. Kernel built with that toolchain boots correctly.
Instead, if I build the same kernel, same configuration, with the toolchain automatically built by buildroot, the kernel doen not boot! Not even a kernel panic, machine just resets.
It is clear the problem is related to the toolchain.
These are the results of gcc -v
CodeBench:
$ output/host/usr/bin/x86_64-amd-linux-gnu-gcc -v
Using built-in specs.
COLLECT_GCC=/media/AAA/tmp/buildroot_toolchain/output/host/opt/ext-toolchain/bin/x86_64-amd-linux-gnu-gcc
COLLECT_LTO_WRAPPER=/media/AAA/tmp/buildroot_toolchain/output/host/opt/ext-toolchain/bin/../libexec/gcc/x86_64-amd-linux-gnu/6.2.0/lto-wrapper
Target: x86_64-amd-linux-gnu
Configured with: /scratch/jmyers/amd-lite/src/gcc-6-2016.11/configure --build=i686-pc-linux-gnu --host=i686-pc-linux-gnu --target=x86_64-amd-linux-gnu --enable-threads --disable-libmudflap --disable-libstdcxx-pch --enable-extra-sgxxlite-multilibs --with-arch=btver2 --with-cpu=btver2 --with-gnu-as --with-gnu-ld --with-specs='-D__CS_SOURCERYGXX_MAJ__=2016 -D__CS_SOURCERYGXX_MIN__=11 -D__CS_SOURCERYGXX_REV__=19' --enable-languages=c,c++ --enable-shared --enable-lto --enable-symvers=gnu --enable-__cxa_atexit --with-glibc-version=2.24 --with-pkgversion='Sourcery CodeBench Lite 2016.11-19' --with-bugurl=https://sourcery.mentor.com/GNUToolchain/ --disable-nls --prefix=/opt/codesourcery --with-sysroot=/opt/codesourcery/x86_64-amd-linux-gnu/libc --with-build-sysroot=/scratch/jmyers/amd-lite/install/opt/codesourcery/x86_64-amd-linux-gnu/libc --with-gmp=/scratch/jmyers/amd-lite/obj/pkg-2016.11-19-x86_64-amd-linux-gnu/amd-2016.11-19-x86_64-amd-linux-gnu.extras/host-libs-i686-pc-linux-gnu/usr --with-mpfr=/scratch/jmyers/amd-lite/obj/pkg-2016.11-19-x86_64-amd-linux-gnu/amd-2016.11-19-x86_64-amd-linux-gnu.extras/host-libs-i686-pc-linux-gnu/usr --with-mpc=/scratch/jmyers/amd-lite/obj/pkg-2016.11-19-x86_64-amd-linux-gnu/amd-2016.11-19-x86_64-amd-linux-gnu.extras/host-libs-i686-pc-linux-gnu/usr --with-isl=/scratch/jmyers/amd-lite/obj/pkg-2016.11-19-x86_64-amd-linux-gnu/amd-2016.11-19-x86_64-amd-linux-gnu.extras/host-libs-i686-pc-linux-gnu/usr --enable-libgomp --enable-libitm --enable-libatomic --disable-libssp --disable-libcc1 --enable-poison-system-directories --with-python-dir=x86_64-amd-linux-gnu/share/gdb/python --with-build-time-tools=/scratch/jmyers/amd-lite/install/opt/codesourcery/x86_64-amd-linux-gnu/bin --with-build-time-tools=/scratch/jmyers/amd-lite/install/opt/codesourcery/x86_64-amd-linux-gnu/bin SED=sed
Thread model: posix
gcc version 6.2.0 (Sourcery CodeBench Lite 2016.11-19)
Buildroot:
Using built-in specs.
COLLECT_GCC=/media/AAA/tmp/buildroot_toolchain/output/host/opt/ext-toolchain/bin/x86_64-linux-gcc.br_real
COLLECT_LTO_WRAPPER=/media/AAA/tmp/buildroot_toolchain/output/host/opt/ext-toolchain/bin/../libexec/gcc/x86_64-buildroot-linux-gnu/7.5.0/lto-wrapper
Target: x86_64-buildroot-linux-gnu
Configured with: ./configure --prefix=/media/AAA/tmp/buildroot_toolchain/output/host --sysconfdir=/media/AAA/tmp/buildroot_toolchain/output/host/etc --enable-static --target=x86_64-buildroot-linux-gnu --with-sysroot=/media/AAA/tmp/buildroot_toolchain/output/host/x86_64-buildroot-linux-gnu/sysroot --enable-__cxa_atexit --with-gnu-ld --disable-libssp --disable-multilib --disable-decimal-float --with-gmp=/media/AAA/tmp/buildroot_toolchain/output/host --with-mpc=/media/AAA/tmp/buildroot_toolchain/output/host --with-mpfr=/media/AAA/tmp/buildroot_toolchain/output/host --with-pkgversion='Buildroot 2020.02.1-00007-g345f377da1-dirty' --with-bugurl=http://bugs.buildroot.net/ --enable-libquadmath --enable-tls --enable-threads --without-isl --without-cloog --with-arch=btver2 --enable-languages=c,c++ --with-build-time-tools=/media/AAA/tmp/buildroot_toolchain/output/host/x86_64-buildroot-linux-gnu/bin --enable-shared --disable-libgomp
Thread model: posix
gcc version 7.5.0 (Buildroot 2020.02.1-00007-g345f377da1-dirty)
Do you see some options that might be required for gcc and buildroot toolchain is missing?
Thanks in advance.
PS: I even tried gcc 8 without resolving.
The early 4.4.x kernels do not boot when built with binutils >= 2.31. This was fixed in commit https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=073bb7ddd35ca8f17a170258dacbe384935a43c8 in the 4.4.x stable branch, which was merged in 4.4.124. So any 4.4.x earlier than that, when built with binutils >= 2.31, will not boot.
when i run my code fyne package GUI in golang , i get this error in terminal:
C:\Go\pkg\tool\windows_amd64\link.exe: running gcc failed: exit status 1
C:\Users\AppData\Local\Temp\go-link-594045439\go.o: file not recognized: file format not recognized
collect2.exe: error: ld returned 1 exit status.
please can help me to solve this issue.
gcc -v
Using built-in specs.
COLLECT_GCC=C:\TDM-GCC-64\bin\gcc.exe
COLLECT_LTO_WRAPPER=C:/TDM-GCC-64/bin/../libexec/gcc/x86_64-w64-mingw32/5.1.0/lto-wrapper.exe
Target: x86_64-w64-mingw32
Configured with: ../../../src/gcc-5.1.0/configure --build=x86_64-w64-mingw32 --enable-targets=all --enable-languages=ada,c,c++,fortran,lto,objc,obj-c++ --enable-libgomp --enable-lto --enable-graphite --enable-cxx-flags=-DWINPTHREAD_STATIC --disable-build-with-cxx --disable-build-poststage1-with-cxx --enable-libstdcxx-debug --enable-threads=posix --enable-version-specific-runtime-libs --enable-fully-dynamic-string --enable-libstdcxx-threads --enable-libstdcxx-time --with-gnu-ld --disable-werror --disable-nls --disable-win32-registry --prefix=/mingw64tdm --with-local-prefix=/mingw64tdm --with-pkgversion=tdm64-1 --with-bugurl=http://tdm-gcc.tdragon.net/bugs
Thread model: posix
gcc version 5.1.0 (tdm64-1)
I had the same problem and finally I solved it by installing an older version of gcc.
my go version is 1.19.4 and my gcc version is 12.2 when i had the problem, so i reinstall gcc 11.4, then I recompiled once and successfully solved this problem.
This most commonly occurs when you have build tools for different architectures, such as 64bit Go tools but 32bit C compiler. If you can’t solve this please add the details of your tools and how they were installed.
I have a nearly fresh mac High Sierra installation. I installed gcc 4.9 with homebrew, and now I want to make sure that's the default gcc (not clang). So I typed the following commands and got the following outputs. And they make no sense to me. Shouldn't gcc -v and gcc-4.9 -v have the same outputs? Why are they different? It seems to me that I don't understand how which works or how symlinks work, which is a surprise because I thought I did.
$ which gcc
/usr/local/bin/gcc
$ ls -l $(which gcc)
lrwxr-xr-x 1 username admin 22 Mar 1 09:50 /usr/local/bin/gcc -> /usr/local/bin/gcc-4.9
$ which gcc-4.9
/usr/local/bin/gcc-4.9
$ gcc-4.9 -v
Using built-in specs.
COLLECT_GCC=gcc-4.9
COLLECT_LTO_WRAPPER=/usr/local/Cellar/gcc#4.9/4.9.4_1/libexec/gcc/x86_64-apple-darwin17.3.0/4.9.4/lto-wrapper
Target: x86_64-apple-darwin17.3.0
Configured with: ../configure --build=x86_64-apple-darwin17.3.0 --prefix=/usr/local/Cellar/gcc#4.9/4.9.4_1 --libdir=/usr/local/Cellar/gcc#4.9/4.9.4_1/lib/gcc/4.9 --enable-languages=c,c++,objc,obj-c++,fortran --program-suffix=-4.9 --with-system-zlib --enable-libstdcxx-time=yes --enable-stage1-checking --enable-checking=release --enable-lto --enable-plugin --with-build-config=bootstrap-debug --disable-werror --with-pkgversion='Homebrew GCC 4.9.4_1' --with-bugurl=https://github.com/Homebrew/homebrew-core/issues MAKEINFO=missing --disable-nls --enable-multilib
Thread model: posix
gcc version 4.9.4 (Homebrew GCC 4.9.4_1)
$ gcc -v
Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 9.0.0 (clang-900.0.39.2)
Target: x86_64-apple-darwin17.4.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin
For context, I'm loosely following these instructions https://github.com/discourse/discourse/blob/master/docs/DEVELOPMENT-OSX-NATIVE.md#os-x-development-tools . I don't think it really matters, but I'm including it for context's sake.
The shell environment was remembering the previous locations of the commands. Using hash -r makes the shell forget them, so now everything behaves as expected.
Thanks to Mark Setchell in the comments above.
I downloaded gcc v4.8 from homebrew so that I could update gcc and g++ to 4.8 instead of using the 4.2.1 supplied from apple.
I installed gcc48 with the following command:
$brew install gcc48 --enable-all-languages
This installed all the required dependancies and when I run,
$gcc-4.8 -v
I get:
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/local/Cellar/gcc48/4.8.1/gcc/libexec/gcc/x86_64-apple-darwin12.4.0/4.8.1/lto-wrapper
Target: x86_64-apple-darwin12.4.0
Configured with: ../configure --build=x86_64-apple-darwin12.4.0 --prefix=/usr/local/Cellar/gcc48/4.8.1/gcc --datarootdir=/usr/local/Cellar/gcc48/4.8.1/share --bindir=/usr/local/Cellar/gcc48/4.8.1/bin --enable-languages=c,c++,fortran,java,objc,obj-c++ --program-suffix=-4.8 --with-gmp=/usr/local/opt/gmp --with-mpfr=/usr/local/opt/mpfr --with-mpc=/usr/local/opt/libmpc --with-cloog=/usr/local/opt/cloog --with-isl=/usr/local/opt/isl --with-system-zlib --enable-libstdcxx-time=yes --enable-stage1-checking --enable-checking=release --enable-plugin --enable-lto --disable-werror --disable-nls --with-ecj-jar=/usr/local/opt/ecj/share/java/ecj.jar --disable-multilib
Thread model: posix
gcc version 4.8.1 (GCC)
I created the symbolic link using
$ln -s gcc-4.8 gcc
while in the /usr/local/bin folder and it created the symbolic link and
$gcc -v
gives me the same output as:
$gcc-4.8 -v
I tried doing the same thing with g++-4.8 and created a symbolic link using,
$ln -s g++-4.8 g++
and got no errors. But now, when I do
$g++ -v
I get:
g++-4.8: error trying to exec '/usr/local/bin/../Cellar/gcc48/4.8.1/bin/i686-apple-darwin11-llvm-g++-4.2': execvp: No such file or directory
Here's my $PATH:/usr/local/bin:/Library/Frameworks/Python.framework/Versions/3.3/bin:/Developer/Intel/ispc-v1.3.0-osx:/Developer/NVIDIA/CUDA-5.0/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X11/bin:/usr/texbin
It seems to be looking for 4.2 for some reason and I have no idea why! It worked perfectly fine for gcc and am just not sure what is the problem with g++?
which g++
returns
/usr/local/bin/g++
Try running:
$ hash
And check if gcc is in the list. If it is you may need to run:
$ hash -r
If you use bash or:
$ rehash
For zsh. This refreshes the hash, which provides a way for the shell to find the command without rechecking the entire path each time.
Looks to me that your g++ symbolic link points to the wrong thing.
Try:
ls -l `which g++`
And if that looks OK, see if the target is not a symbolic link too, etc..
Be mindful that symbol links use relative paths. If you move a symbolic link around, it doesn't point to the same location anymore.
Hope that helps.