I have access to a large IBM Power8 machine (running Ubuntu), and would like to build Bazel on it. However, when I try to do it as their installation instructions suggest, I get:
me#machine:~/bazel-0.1.5$ ./compile.sh
INFO: You can skip this first step by providing a path to the bazel binary as second argument:
INFO: ./compile.sh compile /path/to/bazel
🍃 Building Bazel from scratch.
Compiling Java stubs for protocol buffers...
third_party/protobuf/protoc-linux-x86_32.exe -Isrc/main/protobuf/ --java_out=/tmp/bazel.T9C83cNa/src src/main/protobuf/android_studio_ide_info.proto
scripts/bootstrap/buildenv.sh: line 63: third_party/protobuf/protoc-linux-x86_32.exe: cannot execute binary file: Exec format error
pv#sardonis:~/bazel-0.1.5$ ^C
Clearly, part of the problem is the compiler trying the 32-bit compiler. I tried the following things to no avail.
Replacing the third_party/protobuf/protoc-linux-x86_32.exe by a copy of third_party/protobuf/protoc-linux-x86_64.exe. This gave the same error.
Replacing third_party/protobuf/protoc-linux-x86_32.exe by a symbolic link to /usr/local/bin/protoc, which came with my distribution (this is version libprotoc 3.0.0 according to protoc --version). However, this gave a large amount of errors: http://pastebin.com/HN0MQiC4
Following the instructions on http://www.cnblogs.com/rodenpark/p/5007744.html to compile Protobuf from source and then building Bazel with the modifications on http://www.cnblogs.com/rodenpark/p/5007846.html but this resulted in a similar large amount of errors: http://pastebin.com/KjkseaGx for reference.
So, I'm out of inspiration. How can I compile Bazel on the IBM Power8 machines?
(PS: I've posted this as a part of resolving installing TensorFlow on the IBM power8, so it's not a duplicate question, just one aspect in order to solve it stepwise.)
The version of protobuf you're using must match the protobuf runtime that is checked in. In this case, that's protobuf-java-3.0.0-beta-1.jar [1], so you have to use the compiler version 3.0.0-beta-1.
(I work on Bazel.)
[1] https://github.com/bazelbuild/bazel/tree/master/third_party/protobuf
Related
Using Windows 10 64-bit, Cabal-3.4.0.0, ghc-8.10.7.
I installed OpenBLAS in MSYS2 environment with command
pacman -S mingw-w64-x86_64-openblas.
Than, I successfully installed hmatrix-0.20.2 with command
cabal install --lib hmatrix --flags=openblas --extra-include-dirs="C:\\ghcup\\msys64\\mingw64\\include\\OpenBLAS" --extra-lib-dirs="C:\\ghcup\\msys64\\mingw64\\bin" --extra-lib-dirs="C:\\ghcup\\msys64\\mingw64\\lib"
I am trying to build simple test project using cabal build cabalhmatrix with Main
module Main where
import Numeric.LinearAlgebra
main :: IO ()
main = do
putStrLn $ show $ vector [1,2,3] * vector [3,0,-2]
But now I am getting output
Resolving dependencies...
Build profile: -w ghc-8.10.7 -O1
In order, the following will be built (use -v for more details):
- hmatrix-0.20.2 (lib) (requires build)
- cabalhmatrix-0.1.0.0 (exe:cabalhmatrix) (first run)
Starting hmatrix-0.20.2 (lib)
Failed to build hmatrix-0.20.2. The failure occurred during the configure
step.
Build log (
C:\cabal\logs\ghc-8.10.7\hmatrix-0.20.2-6dd2e8f2795550e4dd624770ac98c326dacc0cac.log
):
Warning: hmatrix.cabal:21:28: Packages with 'cabal-version: 1.12' or later
should specify a specific version of the Cabal spec of the form
'cabal-version: x.y'. Use 'cabal-version: 1.18'.
Configuring library for hmatrix-0.20.2..
cabal-3.4.0.0.exe: Missing dependencies on foreign libraries:
* Missing (or bad) C libraries: blas, lapack
This problem can usually be solved by installing the system packages that
provide these libraries (you may need the "-dev" versions). If the libraries
are already installed but in a non-standard location then you can use the
flags --extra-include-dirs= and --extra-lib-dirs= to specify where they are.If
the library files do exist, it may contain errors that are caught by the C
compiler at the preprocessing stage. In this case you can re-run configure
with the verbosity flag -v3 to see the error messages.
cabal-3.4.0.0.exe: Failed to build hmatrix-0.20.2 (which is required by
exe:cabalhmatrix from cabalhmatrix-0.1.0.0). See the build log above for
details.
What should I do to correctly build that package?
I guess I need to somehow pass arguments --flags=openblas --extra-include-dirs="C:\\ghcup\\msys64\\mingw64\\include\\OpenBLAS" --extra-lib-dirs="C:\\ghcup\\msys64\\mingw64\\bin" --extra-lib-dirs="C:\\ghcup\\msys64\\mingw64\\lib" to hmatrix during compilation, but don't know how to do that. To be honest, I don't understand for what program exactly are those arguments (cabal, ghc, ghc-pkg or something else) and why cabal is trying to install hmatrix again. I see hmatrix in directory "C:\cabal\store\ghc-8.10.7\hmatrix-0.20.2-e917eca0fc7690010007a19f4f2a3602d86df0f0".
Created cabal.project file:
packages: .
package hmatrix
flags: +openblas
extra-include-dirs: C:\\ghcup\\msys64\\mingw64\\include\\OpenBLAS
extra-lib-dirs: C:\\ghcup\\msys64\\mingw64\\bin, C:\\ghcup\\msys64\\mingw64\\libenter code here
After adding libopenblas.dll location to PATH variable cabal project is working.
Even though there is the --lib flag, it's generally best to work under the assumption that Cabal doesn't do library installs. Never install a library, instead just depend on it – and have Cabal install, update etc. it whenever necessary.
But then how can you pass the necessary flags? With a cabal.project file.
packages: .
package hmatrix
flags: openblas
extra-include-dirs: C:\\ghcup\\msys64\\mingw64\\include\\OpenBLAS
...
Put this file in the working directory of your own project, together with cabalhmatrix.cabal. Then running cabal build in that directory will use a hmatrix install with the suitable library etc. flags.
I'm trying to compile the libxkbcommon library for kodi for my Raspberry Pi 2.
The host machine is a dedicated Server running Ubuntu 16.04 x64.
Now there are two errors when I'm trying to compile libxkbcommon, depending on what yacc I'm using:
byacc:
YACC src/xkbcomp/parser.c
yacc: e - line 219 of
"/opt/kodi/xbmc/tools/depends/target/libxkbcommon/raspberry-pi2-release/src/xkbcomp/parser.y", syntax error
%destructor { FreeStmt((ParseCommon *) $$); }
^
Makefile:1637: recipe for target 'src/xkbcomp/parser.c' failed
btyacc:
parser.y:85: syntax error
Here is the source code of libxkbcommon:
https://github.com/xkbcommon/libxkbcommon
The xbcomp/parser.y file requires a number of (very useful) bison extensions, so it can't be processed by all yacc variants.
btyacc does not support bison-compatible pure-parser declarations. (It has a different, not entirely compatible mechanism which implements the same feature.) So it fails on the first instance of one of those declarations.
It should be possible to use byacc, but not the version which is available in the Ubuntu package repository. Although the Ubuntu package repository change history seems to suggest that the intention was to include the build option which allows %destructor, the actual binary currently available in the byacc repository was built without that option. (It is also several years old, and I think it would be useful to use a more recent version.) I reported this as launchpad bug 1776270, along with a suggestion for a possible fix.
I'm sure you'll be able to build the software using Gnu bison, which is available as the Ubuntu package bison. Since that's the most popular yacc version installed on developer machines, a failure to build with bison would probably have been noticed long ago.
If you would prefer to use byacc, for whatever reason, you'll have to download and build it yourself. You can get the most recent version from Thomas Dickey's byacc page, and then build it with the usual procedure: untar, configure, make, make install. When I tested this, I used the following configure line:
./configure --enable-btyacc --program-prefix=b --prefix=/usr
Only the first option is mandatory
* --program-prefix=b Install it as `byacc` rather than `yacc`
* --enable-btyacc Necessary for %destructor support
* --prefix=/usr Install it in /usr/bin and /usr/man. The default
is /usr/local/bin and /usr/local/man, which failed on
my Ubuntu install because of a missing -D option in the
install command in the Makefile.
Trying to build go application using bazel. It fails due to multiple packages and source files present within the application.
OS: Mac OS (High Sierra)
Bazel version : 0.11.1 (homebrew)
Go rules: 0.11.0
Defined local_repository in WORKSPACE to get external dependencies
Running following command
bazel build //go-app
leads to error
Action failed to execute: java.io.IOException: Cannot run program
"/usr/bin/sandbox-exec" (in directory
"/TEMP_DIR/PATH"):
error=7, Argument list too long
Target //go-app:go-app failed to build
I came across https://docs.bazel.build/versions/master/skylark/lib/Args.html#use_param_file and thought it would fix my issue. Could anyone point me in the direction of how and where to implement this in my BUILD file and make it work? Thanks.
I think it's not in your BUILD file but rather in rules_go.
I'd recommend opening them an issue since AFAIK this is something the rule implementation needs to support.
For example in rules_scala we write the arguments to a file and then pass that file to the worker exactly like how you mention above.
https://github.com/bazelbuild/rules_scala/blob/master/scala/private/rule_impls.bzl#L274
i m getting below error while running learn-chaincode example on my local system please provide me the suitable solution for the mentioned error
$ go build
# github.com/hyperledger/fabric/vendor/github.com/miekg/pkcs11
exec: "gcc": executable file not found in %PATH%
Looks like the Fabric team introduced "nopkcs11" tag to bypass the problem - https://jira.hyperledger.org/browse/FAB-2854
I used the following command to build my chain code:
go build -tags nopkcs11
My understanding is that the Learn Chaincode example from https://github.com/IBM-Blockchain/learn-chaincode was being followed. Note that this example is for use with Hyperledger Fabric version 0.6.
The message is related to a PKCS11 package that is referenced by the example. This PKCS package requires a gcc compiler in order to build. I would suggest obtaining a gcc compiler and then trying to build the example again.
I'm trying to help build a Ruby wrapper around Tensorflow using Swig. Currently, I'm stuck at making a shared build, .so, and exposing its C/C++ headers to Ruby. So the question is: How do I build a libtensorflow.so shared build including the full Tensorflow library so it's available as a shared library on OSX El Capitan (note: /usr/lib/ is read-only on El Capitan)?
Background
In this ruby-tensorflow project, I need to package a Tensorflow .bundle file, but whenever I irb -Ilib -rtensorflow or try to run the specs rspec, I get and errors that the basic numeric types are not defined, but they are clearly defined here.
I'm guessing this happens because my .so-file was not created properly or something is not linked as it should. C++/Swig/Bazel are not my strong sides, I'd like to focus on learning Tensorflow and building a good wrapper in Ruby, but I'm pretty stuck at this point getting to that fun part!
What I've done:
git clone --recurse-submodules https://github.com/tensorflow/tensorflow
cd tensorflow
bazel build //tensorflow:libtensorflow.so (wait 10-15min on my machine)
Copied the generated libtensorflow.so (166.6 MB) to the /ext-folder
Run the ruby extconf.rb, make, and make install described in the project
Run rspec
In desperation, I've also gone through the official installation from source several times, but I don't know if that, the last sudo pip install /tmp/tensorflow_pkg/tensorflow-0.9.0-py2-none-any.whl-step even creates a shared build or just exposes a Python interface.
The guy, Arafat, who made the original repository and made the instructions that I've followed, says his libtensorflow.so is 4.5 GB on his Linux machine – so over 20X the size of the shared build on my OSX machine. UPDATE1: he says his libtensorflow.so-build is 302.2 MB, 4.5GB was the size of the entire tensorflow folder.
Any help or alternative approaches are very appreciated!
After more digging around, discovering otool (thanks Kristina) and better understanding what a .so-file is, the solution didn't require much change in my setup:
Shared Build
# Clone source files
git clone --recurse-submodules https://github.com/tensorflow/tensorflow
cd tensorflow
# Build library
bazel build //tensorflow:libtensorflow.so
# Copy the newly shared build/library to /usr/local/lib
sudo cp bazel-bin/tensorflow/libtensorflow.so /usr/local/lib
Calling from Ruby using Swig
Follow the steps here, https://github.com/chrhansen/ruby-tensorflow#install-ruby-tensorflow, to run Swig, create a Makefile and make
When you run make you should see a line saying:
$ make
$ linking shared-object libtensorflow.bundle
If your shared build is not accessible you'll see something like:
$ ld: library not found for -ltensorflow
Simple tutorial
For those starting on this adventure, using C/C++ libraries in Ruby, this post was a good tutorial for me: http://engineering.gusto.com/simple-ruby-c-extensions-with-swig/
I don't think you actually want a .so, I think you want a .dylib (see What are the differences between .so and .dylib on osx?). You're forcing Bazel to build a .so by specifying libtensorflow.so as the target, build this instead:
bazel build //tensorflow
(//tensorflow is shorthand for //tensorflow:tensorflow, which is "build the tensorflow target." Specifying an exact file you want forces Bazel to build that file, if possible.)
Once you have a .dylib, you can check its contents with otool:
otool -L bazel-bin/tensorflow/libtensorflow.dylib
Not sure if this will solve all your problems, but worth a try.