How to link different object file with LLVM library? - compilation

I am following the LLVM tutorial : Kaleidoscope: Code generation to LLVM IR, which will use LLVM libraries like LLVMContext, Module and so on. Different from the tutorial, I am trying to write the lexer, parser and code generator in different source file and link them into one executable file.
Here is my compile command on the Ubuntu 20.04:
clang++ -g -O3 -I /home/therlf/LLVM/include -I ./ -I /home/therlf/LLVM_Temp/llvm/include `llvm-config --cxxflags --ldflags --system-libs --libs all` ast/CallExprAST.o ast/NumberExprAST.o ast/PrototypeAST.o ast/FunctionAST.o ast/BinaryExprAST.o ast/VariableExprAST.o lexer/lexer.o logger/logger.o parser/parser.o main.cpp -o main
But I only get lots of "undefined error".
Here are some of them:
/usr/bin/ld: /tmp/main-2b71c8.o:(.data+0x0): undefined reference to `llvm::DisableABIBreakingChecks
/home/therlf/MyProject/tmp/ast/CallExprAST.cpp:6: undefined reference to `llvm::Module::getFunction(llvm::StringRef) const'
/home/therlf/LLVM/include/llvm/IR/InstrTypes.h:1112: undefined reference to `llvm::Instruction::Instruction(llvm::Type*, unsigned int, llvm::Use*, unsigned int, llvm::Instruction*)'
/usr/bin/ld: /home/therlf/LLVM/include/llvm/IR/InstrTypes.h:977: undefined reference to `llvm::VectorType::get(llvm::Type*, llvm::ElementCount)'
At first I thought it's including path's error. But when I compiled and ran the source file in the tutorial successfully, which is just a whole source file with everything packed into file, I knew the including path is nothing wrong.
I have searched for this question, and some blogs say that you should link them with lld and use the -fuse-ld=lld in the compile command. But I don't have lld, and the clang++ doesn't know the argument -fuse-ld, which will report an error. The blog says that you should have lld as long as you have installed LLVM. In fact here are what I got: LLVM tools
And I know the llvm-link is used to link IR file, not the object file compiled from cpp source file.
Here is my LLVM version:
10.0.0svn
And here is my Makefile:
SOURCES = $(shell find ast kaleidoscope lexer logger parser -name '*.cpp')
HEADERS = $(shell find ast kaleidoscope lexer logger parser -name '*.h')
OBJ = ${SOURCES:.cpp=.o}
CC = clang++
# -stdlib=libc++ -std=c++11
CFLAGS = -g -O3 -I /home/therlf/LLVM/include -I ./ -I /home/therlf/LLVM_Temp/llvm/include
LLVMFLAGS = `llvm-config --cxxflags --ldflags --system-libs --libs all`
.PHONY: main
main: main.cpp ${OBJ}
${CC} ${CFLAGS} ${LLVMFLAGS} ${OBJ} $< -o $#
clean:
rm -r ${OBJ}
%.o: %.cpp ${HEADERS}
${CC} ${CFLAGS} ${LLVMFLAGS} -c $< -o $#
In fact, I follow the project structure from the repository : ghaiklor/llvm-kaleidoscope and the Makefile is nearly identical.
Sincerely thank you for your answers!

I have sloved this problem by exchanging my linker ld with linker lld.
You should install lld first by this command in ubuntu if you can't find it in the LLVM/tools directory like I did.
sudo apt-get update
sudo apt-get -y install lld
And then you can add -fuse-ld=lld to your compile command
or you can
cd /usr/bin
ln -s /path/to/ld.lld /usr/bin/ld
This should work if everything goes well.
But I still can't figure out the reason behand this situation :-(

Related

Im trying to compile program on Ubuntu and dont understand some things

Im a Windows dev who has no expirience on building C/C++ programs on Linux, but now I need to. Right way would be to go and learn Make and g++ compiler, but before I commit to that I want to figure out some basic stuff.
So I have .c program which is compiled with this makefile:
CUDA_VER=11.5
ifeq ($(CUDA_VER),)
$(error "CUDA_VER is not set")
endif
APP:= deepstream-test3-app
TARGET_DEVICE = $(shell gcc -dumpmachine | cut -f1 -d -)
NVDS_VERSION:=6.0
LIB_INSTALL_DIR?=/opt/nvidia/deepstream/deepstream-$(NVDS_VERSION)/lib/
APP_INSTALL_DIR?=/opt/nvidia/deepstream/deepstream-$(NVDS_VERSION)/bin/
ifeq ($(TARGET_DEVICE),aarch64)
CFLAGS:= -DPLATFORM_TEGRA
endif
SRCS:= $(wildcard *.c)
$(info info is $(SRCS))
INCS:= $(wildcard *.h)
PKGS:= gstreamer-1.0
OBJS:= $(SRCS:.c=.o)
CFLAGS+= -I../../../includes \
-I /usr/local/cuda-$(CUDA_VER)/include
CFLAGS+= $(shell pkg-config --cflags $(PKGS))
LIBS:= $(shell pkg-config --libs $(PKGS))
LIBS+= -L/usr/local/cuda-$(CUDA_VER)/lib64/ -lcudart -lnvdsgst_helper -lm \
-L$(LIB_INSTALL_DIR) -lnvdsgst_meta -lnvds_meta \
-lcuda -Wl,-rpath,$(LIB_INSTALL_DIR)
$(info info is $(CFLAGS))
all: $(APP)
%.o: %.c $(INCS) Makefile
gcc -c -o $# $(CFLAGS) $<
$(APP): $(OBJS) Makefile
gcc -o $(APP) $(OBJS) $(LIBS)
install: $(APP)
cp -rv $(APP) $(APP_INSTALL_DIR)
clean:
rm -rf $(OBJS) $(APP)
First thing I tried is to change this Makefile to compile it as C++ program. I changed .c file into .cpp, in makefile I change gcc to g++ everywhere and .c to .cpp everywhere. It gave me error that it couldnt find "main" entry point.
I gave up on that pretty fast and decided just to use lines output of original makefile, ending up with this:
g++ -c -o deepstream_test3_app.o -I../../../includes -I /usr/local/cuda-11.5/include -pthread -I/usr/include/gstreamer-1.0 -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include ./deepstream_test3_app.cpp
g++ -o deepstream-test3-app deepstream_test3_app.o -lgstreamer-1.0 -lgobject-2.0 -lglib-2.0 -L/usr/local/cuda-11.5/lib64/ -lcudart -lnvdsgst_helper -lm -L/opt/nvidia/deepstream/deepstream-6.0/lib/ -lnvdsgst_meta -lnvds_meta -lcuda -Wl,-rpath,/opt/nvidia/deepstream/deepstream-6.0/lib/
First question, can I combine this 2 launches of g++ into one?
Second, when I make changes to "./deepstream_test3_app.cpp" they are not noticed by compiler. I added
#include <iostream>
...
std::cout << "hello!" << std::endl;
and they are ignored. Its like g++ gets as input some other copy/older version of the file and I dont understand how to go about it.
Hope for any help, sorry if it's all sounds stupid.
Ignoring for the moment the issues surrounding compiling C code with a C++ compiler,
g++ -c -o deepstream_test3_app.o -I../../../includes -I /usr/local/cuda-11.5/include -pthread -I/usr/include/gstreamer-1.0 -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include ./deepstream_test3_app.cpp
g++ -o deepstream-test3-app deepstream_test3_app.o -lgstreamer-1.0 -lgobject-2.0 -lglib-2.0 -L/usr/local/cuda-11.5/lib64/ -lcudart -lnvdsgst_helper -lm -L/opt/nvidia/deepstream/deepstream-6.0/lib/ -lnvdsgst_meta -lnvds_meta -lcuda -Wl,-rpath,/opt/nvidia/deepstream/deepstream-6.0/lib/
First question, can I combine this 2 launches of g++ into one?
Yes. It is a common practice in makefiles to separate the compilation and linking steps, but that is not mandatory. When there are multiple sources, the separation makes it possible to limit recompilations to only the source files that have changed, but it doesn't make much difference, makefile or not, when there is only one source file.
The one-command version would be mostly a concatenation of the two commands you gave. One would omit the -c option, which instructs g++ to compile but not link, and one would omit the -o deepstream_test3_app.o, which specifies the name of the object file that we are no longer going to create. One would also omit the appearance of deepstream_test3_app.o drawn from the link (second) command, as we are going straight from source file to program. The rest of the options can be reordered to some extent, but all the -l options need to remain in the same order relative to each other and to any object files among the inputs. Here is how I would write it:
g++ -c -o deepstream_test3_app -I../../../includes -I /usr/local/cuda-11.5/include -pthread -I/usr/include/gstreamer-1.0 -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -Wl,-rpath,/opt/nvidia/deepstream/deepstream-6.0/lib/ ./deepstream_test3_app.cpp -lgstreamer-1.0 -lgobject-2.0 -lglib-2.0 -L/usr/local/cuda-11.5/lib64/ -lcudart -lnvdsgst_helper -lm -L/opt/nvidia/deepstream/deepstream-6.0/lib/ -lnvdsgst_meta -lnvds_meta -lcuda
Second, when I make changes to "./deepstream_test3_app.cpp" they are not noticed by compiler.
The compiler compiles the source file(s) you tell it to.
Its like g++ gets as input some other copy/older version of the file
It is possible that you are indeed telling it to compile a different version than the one you modified. It is also possible that compilation fails, so you don't get a new executable. And it is possible that when you try to run the result, you are not running the program you think you are running. We don't have enough information to know.
With regard to the last, however, do be aware that on Linux, unlike on Windows, the working directory is not automatically in the executable search path. If you want to run the compiled result from the above command, you would want to specify the path to it, which you could most easily do by prepending ./ to its simple name: ./deepstream-test3-app.

Why do I get a segmentation fault when running an example program compiled without cmake? (MSYS2, Mingw-w64)

In short
What are cmake's web of makefiles doing differently from a simple compile and link that is making a difference in the final executable?
I'm trying to use the bullet physics library (bullet3-2.83.7) https://github.com/bulletphysics/bullet3.
I compiled the library okay with few warnings in MSYS2 with Mingw-w64.
Afterwards I can run the example programs without problems, specifically ExampleBrowser and HelloWorld.
I've been trying to incorporate the HelloWorld source into a test project using just a Makefile but I get SIGSEGV errors whenever there is a call to dynamicsWorld in the executable. The SEGFAULT occurs at lines dynamicsWorld->AddRigidBody(body); or if those are commented out dynamicsWorld->stepSimulation
This occurs with the exact example source file compiled with the makefile (source not modified).
gdb tells me this
main (argc=1, argv=0x5f4eb0) at main.cpp:78
78 dynamicsWorld->addRigidBody(body);
(gdb) step
0x0000000000002000 in ?? ()
(gdb) step
Cannot find bounds of current function
(gdb) bt full
#0 0x0000000000002000 in ?? ()
No symbol table info available.
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
I don't know what to do with this info. I assume/hope I'm just missing a compiler or linker flag?
My original Makefile is a big mess based off http://make.mad-scientist.net/papers/advanced-auto-dependency-generation. I assumed it would be enough to just use the existing makefile on the example code by adding the libraries and include directory -lBulletDynamics_Debug -lBulletCollision_Debug -lLinearMath_Debug
I've also tried a simplified Makefile with commands and flags I found grepping the CMake directories from bullet3/examples/HelloWorld.
My PATH environment variable is clean, nothing in LD_LIBRARY_PATH (In MSYS: echo $PATH)
MSYS2 Mingw-w64
gcc 10.1.0
MSYS 20180531msys64 ? pacman updated a lot of things
CMake 3.17.3
GNU Make 4.3
Makefile
CXX_DEFINES = -DUSE_GRAPHICAL_BENCHMARK -D_CRT_SECURE_NO_DEPRECATE -D_CRT_SECURE_NO_WARNINGS -D_SCL_SECURE_NO_WARNINGS
CXX_INCLUDES = -I"C:\lib64\include\bullet3"
CXX_FLAGS = -g -fpermissive -D_DEBUG
.PHONY: all
all:
g++.exe $(CXX_DEFINES) $(CXX_INCLUDES) $(CXX_FLAGS) -o main.obj -c main.cpp
ar cr main.a main.obj
g++.exe $(CXX_FLAGS) -Wl,--whole-archive main.a -Wl,--no-whole-archive -o bulletTest.exe -Wl,--major-image-version,0,--minor-image-version,0 libBulletDynamics_Debug.a libBulletCollision_Debug.a libLinearMath_Debug.a -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32 -lole32 -loleaut32 -luuid -lcomdlg32 -ladvapi32
# -L"C:\lib64\lib"
.PHONY: run
run:
gdb -ex run bulletTest.exe -ex "bt full" -ex quit --batch
.PHONY: clean
clean:
rm -f ./bulletTest.exe ./main.obj ./main.a
Building the bullet physics library in MSYS2
In the bullet3-2.83.7 directory (tar.gz from https://github.com/bulletphysics/bullet3/releases)
mkdir build-mingw64
cd build-mingw64
cmake -G "MSYS Makefiles" \
-DBUILD_SHARED_LIBS=0 \
-DBUILD_EXTRAS=1 \
-DINSTALL_LIBS=0 \
-DUSE_GLUT=1 \
-DCMAKE_CXX_FLAGS_DEBUG="-fpermissive -g" \
-DINSTALL_EXTRA_LIBS=0 \
-DCMAKE_BUILD_TYPE=Debug ..
make -j
I ran into the same issue. In my case, it was because Bullet was compiled with USE_DOUBLE_PRECISION, so adding the following to cmakelists for my executable fixed the issue for me:
target_compile_options(<target_name> BEFORE PUBLIC -DBT_USE_DOUBLE_PRECISION)

Error linking SSL and crypto libraries installed in custom location

I am trying to create a shared library that is to be linked against OpenSSL-1.0.2p which depends on libssl1.0. The installed version of OpenSSL on my ubuntu bionic machine is however OpenSSL-1.1.1 which internally uses libssl1.1.
Since I do not wish to install OpenSSL-1.0 system-wide, I downloaded and compiled it separately installing it into a folder inside my home directory.
I wish to use this location against which I would want my shared library to link. Here is the Makefile for the shared library that I am trying to create:
APPBASE=/home/AB/Documents/APP/APP_2.17.0
OPENSSL1.0.2p_INSTALL_LOC=/home/AB/Documents/APP/OpenSSL-1.0.2p-installation
CC=gcc
#CFLAGS= -Wall -g -O -fPIC
CFLAGS= -Wall -g -O -static
RM= rm -f
.PHONY: all clean
src=$(wildcard *Generic/*.c *Linux/*.c)
$(info source=$(src))
#we use the custom compiled openssl version
#and NOT the one available on the system
#INC=-I/usr/include/openssl
INC+=-I$(OPENSSL1.0.2p_INSTALL_LOC)/include/openssl
INC+=$(foreach d,$(incdir),-I$d)
$(info includes=$(INC))
LIB=-L$(OPENSSL1.0.2p_INSTALL_LOC)/lib
LIB+=-l:libssl.a -l:libcrypto.a
# looks like we need this for proper static linking of libc
LIB+= -static-libgcc
$(info links=$(LIB))
obj=$(src:.c=.o)
#all: libAPP.so
all: libAPP.a
clean:
$(RM) *.o *.so
$(shell find $(APPBASE) -type f -iname "*.o" -exec rm -rf {} \;)
.c.o:
${CC} ${CFLAGS} $(INC) -c $< -o $#
#${CC} ${CFLAGS} $(INC) -c $< -o $#
libAPP.a: $(obj)
#ar rcs $# $^
#$(LINK.c) -shared $^ -o $#
However, make reports that the header files being included are from the system's openssl installation and hence the compile is failing (since it is expecting OpenSSL-1.0.2p). Here's a sample:
In file included from /usr/include/openssl/e_os2.h:13:0,
from /usr/include/openssl/bio.h:13,
from /usr/include/openssl/x509v3.h:13,
.... (source file 1)
gcc -static -Wall -g -O -fPIC -I/home/AB/Documents/APP/OpenSSL-1.0.2p-installation/include/openssl -I*/path/to/app/include1* -I*/path/to/app/include2* -c */path/to/src1* -L/home/AB/Documents/APP/OpenSSL-1.0.2p-installation/lib -lssl -lcrypto
sr1.c: In function ‘Get_CACertificates’: warning: implicit declaration of function ‘CRYPTO_w_lock’; did you mean ‘CRYPTO_zalloc’? [-Wimplicit-function-declaration]
CRYPTO_w_lock(CRYPTO_LOCK_X509_STORE);
^~~~~~~~~~~~~
CRYPTO_zalloc
CRYPTO_w_lock is a macro which is no longer present in the latest version of crypto.h (OpenSSL-1.1) which makes it clear that my application is still looking at the system version of OpenSSL.
Within my source files, I am including the SSL header files like so:
#include <openssl/bio.h>
#include <openssl/ssl.h>
#include <openssl/err.h>
Although angular braces are supposed to tell the compiler to look into the system's header files, doesn't the -L on the command line as well as the -I flags force it to look for them in the said directories before looking at the system files?
I do have a crypto.h file in the custom install location for OpenSSL but the compiler seems to be ignore it for some reason
ab#ab1-pc:/home/AB/Documents/APP/OpenSSL-1.0.2p-installation$ find . -iname "crypto.h"
./include/openssl/crypto.h
What am I missing here?
UPDATE 1: As suggested by Darren, removed the trailing openssl so now my include path is
INC+=-I$(OPENSSL1.0.2p_INSTALL_LOC)/include and voila I can see the shared library getting created. However when I try to find references to ssl within this newly minted shared library, I see that I have 87 entries (which includes ALL symbols having ssl as part of their name)
ab#ab1-pc:~/Documents/AB/APP_2.17.0$ nm libAPP.so | grep -i "ssl" | wc -l
87
whereas listing only the global symbols from libssl.a tells me it has 1113 globally defined symbols.
ab#ab1-pc:~/Documents/AB/APP_2.17.0$ nm -g ../OpenSSL-1.0.2p-installation/lib/libssl.a | grep -i "ssl" | wc -l
1113
Shouldn't the former count be MORE than the latter?? Has it got something to do with the fact that my 'app' is a shared library? Even then shouldn't it pull in ALL the symbols (at least the global ones) from any static libraries it links against??
UPDATE 2: Now apparently since I was facing issues with SSL symbols, I switched to creating a static library so made changes to the makefile accordingly.
In this line:
INC+=-I$(OPENSSL1.0.2p_INSTALL_LOC)/include/openssl
... try changing it to:
INC+=-I$(OPENSSL1.0.2p_INSTALL_LOC)/include
... i.e., without the 'openssl' part.
Your includes, e.g.,
#include <openssl/bio.h>
#include <openssl/ssl.h>
#include <openssl/err.h>
... expect directory search paths to terminate on the 'include/', not on the 'openssl'.

Creating .o file with make

I'm trying to learn how to write makefiles. I have started reading the manual of gnu make: https://www.gnu.org/software/make/manual/html_node/Simple-Makefile.html#Simple-Makefile
I have 3 files in the same directory:
main.cpp: which creates a rectangle and prints some information. Therefor it includes Rectangle.h
Rectangle.h: header file for rectangle class
Rectangle.cpp: implementation of rectangle class
I am having troubles with the include of Rectangle.h in main.cpp. My makefile is:
main: main.o rectangle.o
g++ -o main.exe main.o rectangle.o
main.o: main.cpp
g++ main.cpp
rectangle.o: Rectangle.cpp
g++ Rectangle.cpp
clean:
rm main.exe main.o rectangle.o
I know something is missing to create main.o but I can't find out what it is. I tried adding various variations of Rectangle.h/.o/.cpp and finding something on the internet but I was unable to find something.
Help will be much appreciated :)
PS: The code is fine, I can compile it with the command:
g++ -o main.exe main.cpp Rectangle.cpp
man g++
When you invoke GCC, it normally does preprocessing, compilation, assembly and linking. The "overall options" allow you to stop this process at an intermediate stage. For example, the -c option says not to run the linker. Then the output consists of object files output by the assembler.

How to set gcc flags in Emscripten

I compile with the following command:
gcc -Wall -march=native -O3 -ffast-math -I/usr/local/include -I/usr/local/include -o waon main.o notes.o midi.o analyse.o fft.o hc.o snd.o -L/usr/local/lib -L/usr/local/lib -lfftw3 -L/usr/local/lib -lsndfile -lm
I now would like to compile with Emscripten. How do I convert the above gcc command into an emcc command?
The command you have described in the question is linking rather than compiling. However in general you should just be able to replace gcc with emcc and it will do the right thing. In this case you will need to replace not only this linking command but also the commands used to compile the sources to the .o files.
It would probably be a good idea to take out the -march option.
It looks like your project is using libsndfile and FFTW. You will probably need to compile these libraries yourself using emscripten. Both of them are using autotools so with a bit of luck you can compile them with emscripten simply by adding the following parameters when you run the configure script:
./configure --prefix=$HOME/emscripten-libs CC=emcc
make && make install
Then when you link your program you can specify -L$HOME/emscripten-libs/lib instead of -L/usr/local/lib.
Make research about emsdk download&setup on your computer.
Download emsdk instruction
Next interest link is :
emcc or em++ instruction
https://emscripten.org/docs/tools_reference/emcc.html
When you setup emcc in command line you can see this project (i make emcc final look based on python script runner.py etc.):
c-cpp-to-javascript
Basic and useful example's :
Pretty analog with gcc :
Args:
-lGL for openGL
-s TOTAL_MEMORY=512MB --memory-init-file 1 Memory staff
--preload-file folderWithImages/--use-preload-plugins If you use assets
-I forInclude/someheader.h
-L libraryFolder/someLib.lib
-std=c11
Simple run:
./emcc -O2 a.cpp -o a.js
or
./emcc -O2 a.cpp -o a.html
Links:
./emcc -O2 a.cpp -o a.bc
./emcc -O2 b.cpp -o b.bc
./emcc -O2 a.bc b.bc -o project.js
Or :
to get JS
emcc -s WASM=1 myAdds.a myLib.a source1.c source2.cpp -o build.js
to get html
emcc -s WASM=1 myAdds.a myLib.a source1.c source2.cpp -o build.html
Link together the bitcode files:
emcc project.bc libstuff.bc -o allproject.bc
Compile the combined bitcode to HTML
emcc allproject.bc -o final.html
Important note :
You can't take an existing .a library and convert it. You must build lib with emcc also.

Resources