How to generate position independent code in Halide with ahead of time compilation - halide

I'm trying out halide and want to build a small shared library on top of it. But the code generated by Halide's AOT compilation seems not being position independent. How do I link the objects generated by Halide into my shared library?
[Update] extracted a small example from my code.
Specifically I used a generator with c++:
class SimpleAddition : public Halide::Generator<SimpleAddition> {
public:
SimpleAddition() : vars(4) {}
Input<Halide::Buffer<>> lhs{"lhs"};
Input<Halide::Buffer<>> rhs{"rhs"};
Output<Halide::Buffer<>> out{"out"};
std::vector<Var> vars;
void generate() {
out(vars) = lhs(vars) + rhs(vars);
}
};
HALIDE_REGISTER_GENERATOR(SimpleAddition, simple_addition)
with helpers from CMake:
halide_library(ops SRCS ${CMAKE_CURRENT_LIST_DIR}/src/simple_addition.cc
GENERATOR_NAME simple_addition
GENERATOR_ARGS lhs.type=float32 lhs.dim=4 rhs.type=float32 rhs.dim=4 out.type=float32)
and got the following error:
/usr/bin/ld: ./genfiles/halide_rt_host/halide_rt_host.a(halide_rt_host.a.o): relocation R_X86_64_PC32 against symbol `_ZN6Halide7Runtime8Internal13custom_mallocE' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: bad value
collect2: error: ld returned 1 exit status
I also tried generating object by running the generator myself (without CMake wrapper) with -e o,h, and similarly with llvm bitcode followed by llc -relocation-model=pic, also Func::compile_to_object. similar error occurred when I try to link the generated object back to shared library:
/usr/bin/ld: halide_runtime_x86.o: relocation R_X86_64_PC32 against symbol `_ZN6Halide7Runtime8Internal13custom_mallocE' can not be used when making a shared object; recompile with -fPIC
/usr/bin/ld: final link failed: bad value
collect2: error: ld returned 1 exit status
I'm running this on Ubuntu 18.10. Both Halide and my code are compiled with gcc-8.2.0. LLVM/Clang-7 is the one shipped in their repository. Halide is built with simple cmake ../ && make && make install.
For compiler flags of my testing code, command used in CMake is add_libarary(mylib SHARED ...). Non-CMake build is with flags like -fPIC -shared, nothing special since I'm still learning.

Halide should be generating PIC by default (see Codegen_Internal.cpp, make_target_machine()). What architecture are you targeting? What compiler/linker are you using, with what options, etc?

Related

Caffe build gives GCC Link Error "can not be used when making shared object.; recompile with -fPIC"

I'm trying to install caffe, using CMake, but when I run make all (after running cmake .. from a build directory) I get the following error:
me#gimli:~/Downloads/caffe/build$ make all
[ 1%] Built target caffeproto
[ 1%] Linking CXX shared library ../../lib/libcaffe.so
/usr/bin/x86_64-linux-gnu-ld: /usr/local/lib/libleveldb.a(db_impl.cc.o): relocation R_X86_64_PC32 against symbol `_ZN7leveldb10EnvWrapper8ScheduleEPFvPvES1_' can not be used when making a shared object; recompile with -fPIC
/usr/bin/x86_64-linux-gnu-ld: final link failed: Bad value
collect2: error: ld returned 1 exit status
src/caffe/CMakeFiles/caffe.dir/build.make:40060: recipe for target 'lib/libcaffe.so.1.0.0' failed
make[2]: *** [lib/libcaffe.so.1.0.0] Error 1
CMakeFiles/Makefile2:267: recipe for target 'src/caffe/CMakeFiles/caffe.dir/all' failed
make[1]: *** [src/caffe/CMakeFiles/caffe.dir/all] Error 2
Makefile:129: recipe for target 'all' failed
make: *** [all] Error 2
I don't really understand CMake, but gather that somewhere I'm supposed to add -fPIC to a gcc command. But, I have no idea where I should make this change, or if there's somewhere in Cmake where I should tell it to construct the gcc command correctly.
How can I force CMake to create/use a gcc command with the -fPIC option, or is there something else entirely I should be doing?
The error is not from CMake but from the linker. It actually tells, that:
You cannot build shared library libcaffe.so with PIC (Position independent code) feature and link it with the static library libleveldb.a compiled without this feature.
Possible solutions are:
Get shared version of the static library (libleveldb.a in your case), so it will be compiled with PIC. This is what the error message suggests you.
Instead of building shared library (Caffe in your case), build static one, without using of PIC. Note, that in this case your will face with similar issues when trying to use resulted library in the future shared libraries.
For most CMake projects forcing the building static libraries can be performed with:
cmake -DBUILD_SHARED_LIBS=OFF <other parameters>
Strictly speaking, PIC feature is independent from the type (shared or static) of a library. So you may have a static library with PIC, or build a shared library without it.
For many CMake projects you may control PIC feature of the created libraries with
cmake -DPOSITION_INDEPENDENT_CODE=<ON|OFF> <other parameter>

Linker error: access beyond end of merged section

I've built my code on a machine with gcc 6.3 before. I'm building the same code on another machine with gcc 5.3. I tried using the working oed library from a different location. But still the error persists. I went through suggestions in other questions similar to this on Stackoverflow, but they suggest that the library must be already linked. This is not the case though.
Oed is a fortran library and the code is in C and C++.
I'm at a loss to think how to figure out this problem. Any help is welcome.
/usr/bin/ld: oed__gener_nai_batch.o: access beyond end of merged section (380)
/usr/bin/ld: oed__gener_nai_batch.o: access beyond end of merged section (586)
/usr/bin/ld: oed__gener_nai_batch.o: access beyond end of merged section (580)
/usr/bin/ld: /home/umunipala/gamess/gamess-kris-simgms/simgms/oed/liboed.a(oed__gener_nai_batch.o)(.debug_info+0x203): reloc against `.debug_str': error 2
/usr/bin/ld: final link failed: Nonrepresentable section on output
collect2: error: ld returned 1 exit status
Well, there are two solutions I think.
recompile the .c file with your current library settings.
export a corresponded version LD library.
The problem is basically caused by the disagreement between .o file and your gcc(compiler).

fglut/libfglut.a(freeglut_state.o): undefined reference to symbol 'XGetWindowAttributes'

I am compiling a C++ program using make, this is the error i'm getting.
/usr/bin/ld: fglut/libfglut.a(freeglut_state.o): undefined reference to symbol 'XGetWindowAttributes'
//usr/lib/i386-linux-gnu/libX11.so.6: error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
Makefile:55: recipe for target 'morphlines' failed
make: *** [morphlines] Error 1
I'm beginner
/usr/bin/ld: fglut/libfglut.a(freeglut_state.o):
This tells me you link libfglut statically (*.a is just an archive of object files). When you do this, you must link all dependencies as well, because with the object files from the static library actually compiled into your program, your program will depend on them.
Either link libfglut dynamically (this is the default with the GNU toolchain), so your program will depend on libfglut.so which will itself depend on libX11.so -- or add -lX11 after -lfglut on the command line of your final linking step. You might need -Wl,-Bdynamic before -lX11 to switch the linker back to dynamic linking.
If this doesn't directly solve your problem, I suggest you edit your question to include the relevant parts of the Makefile you're using.

undefined reference to `CefURLRequest::Create(CefRefPtr<CefRequest>, CefRefPtr<CefURLRequestClient>)'

I am trying to build CEF in Release mode but I get the following linker error:
AR(target)
/home/adminuser/temp/build/linux/Release/obj.target/libcef_dll_wrapper.a
LINK(target) /home/adminuser/temp/build/linux/Release/cefclient
/home/adminuser/temp/build/linux/Release/ldb.a(ldbJS.o): In function
LdbURLLoader::LoadURL(std::string)': ldbJS.cpp:(.text+0x34e):
undefined reference toCefURLRequest::Create(CefRefPtr,
CefRefPtr)' collect2: error: ld returned 1 exit
status make: *** [/home/adminuser/temp/build/linux/Release/cefclient]
Error 1
I don't get this problem when building the Debug version so I am not quite sure how to resolve it. I tried various #pragma directives to stop the compiler from optimizing out this method but to no avail.
* UPDATE *
When I remove the -DNDEBUG flag from CFLAGS_Release in the libcef_dll_wrapper.target.mk file it will build. I don't want to do this in the Release build without knowing what I'm doing though.
I am using gcc 4.8.2.
It turns out forcing the LD linker to reanalyze my libraries was the solution. The CEF application already had an elaborate scheme of makefiles that used the --start-group and --end-group indicators to do this, but the library I previously added was not included with them. I added it and linked my application without any more problems!

Can't find libstdc++?

I built gcc 4.9.0 from source, and was also planning on building clang 3.4.2, however something seems to have gone awry with regards to libstdc++, as the clang build baled pretty quickly with the linker complaining about various undefined references from std.
Indeed, I then tried compiling and linking the trivial program:
#include <iostream>
int main() {
std::cout << 42;
}
and again hit linker errors:
/tmp/ccrptgVW.o:temp.cpp:function main: error: undefined reference to 'std::cout'
/tmp/ccrptgVW.o:temp.cpp:function main: error: undefined reference to 'std::ostream::operator<<(int)'
/tmp/ccrptgVW.o:temp.cpp:function __static_initialization_and_destruction_0(int, int): error: undefined reference to 'std::ios_base::Init::Init()'
/tmp/ccrptgVW.o:temp.cpp:function __static_initialization_and_destruction_0(int, int): error: undefined reference to 'std::ios_base::Init::~Init()'
collect2: error: ld returned 1 exit status
(full gcc -v output), both with my freshly-minted gcc 4.9.0 and my Ubuntu's stock gcc 4.6.3.
libstdc++.so exists, in /usr/local/lib64:
ls /usr/local/lib64/libstd*
/usr/local/lib64/libstdc++.a
/usr/local/lib64/libstdc++.so
/usr/local/lib64/libstdc++.so.6.0.20
/usr/local/lib64/libstdc++.la
/usr/local/lib64/libstdc++.so.6
/usr/local/lib64/libstdc++.so.6.0.20-gdb.py
and this directory appears in LIBRARY_PATH and as an -L argument to collect2 in the verbose gcc output.
How do I restore sanity to my system and have the linker find the shared library?
As turns out from your comment, you compiled with gcc rather than g++.
Do not take it as an error, as the compilation went fine! What's showing up is a linker error. Indeed, gcc will tell ld to link against the C standard library rather than the C++ standard one.
To solve, either go for g++ directly or pass -lstdc++.

Resources