Error linking SSL and crypto libraries installed in custom location - makefile

I am trying to create a shared library that is to be linked against OpenSSL-1.0.2p which depends on libssl1.0. The installed version of OpenSSL on my ubuntu bionic machine is however OpenSSL-1.1.1 which internally uses libssl1.1.
Since I do not wish to install OpenSSL-1.0 system-wide, I downloaded and compiled it separately installing it into a folder inside my home directory.
I wish to use this location against which I would want my shared library to link. Here is the Makefile for the shared library that I am trying to create:
APPBASE=/home/AB/Documents/APP/APP_2.17.0
OPENSSL1.0.2p_INSTALL_LOC=/home/AB/Documents/APP/OpenSSL-1.0.2p-installation
CC=gcc
#CFLAGS= -Wall -g -O -fPIC
CFLAGS= -Wall -g -O -static
RM= rm -f
.PHONY: all clean
src=$(wildcard *Generic/*.c *Linux/*.c)
$(info source=$(src))
#we use the custom compiled openssl version
#and NOT the one available on the system
#INC=-I/usr/include/openssl
INC+=-I$(OPENSSL1.0.2p_INSTALL_LOC)/include/openssl
INC+=$(foreach d,$(incdir),-I$d)
$(info includes=$(INC))
LIB=-L$(OPENSSL1.0.2p_INSTALL_LOC)/lib
LIB+=-l:libssl.a -l:libcrypto.a
# looks like we need this for proper static linking of libc
LIB+= -static-libgcc
$(info links=$(LIB))
obj=$(src:.c=.o)
#all: libAPP.so
all: libAPP.a
clean:
$(RM) *.o *.so
$(shell find $(APPBASE) -type f -iname "*.o" -exec rm -rf {} \;)
.c.o:
${CC} ${CFLAGS} $(INC) -c $< -o $#
#${CC} ${CFLAGS} $(INC) -c $< -o $#
libAPP.a: $(obj)
#ar rcs $# $^
#$(LINK.c) -shared $^ -o $#
However, make reports that the header files being included are from the system's openssl installation and hence the compile is failing (since it is expecting OpenSSL-1.0.2p). Here's a sample:
In file included from /usr/include/openssl/e_os2.h:13:0,
from /usr/include/openssl/bio.h:13,
from /usr/include/openssl/x509v3.h:13,
.... (source file 1)
gcc -static -Wall -g -O -fPIC -I/home/AB/Documents/APP/OpenSSL-1.0.2p-installation/include/openssl -I*/path/to/app/include1* -I*/path/to/app/include2* -c */path/to/src1* -L/home/AB/Documents/APP/OpenSSL-1.0.2p-installation/lib -lssl -lcrypto
sr1.c: In function ‘Get_CACertificates’: warning: implicit declaration of function ‘CRYPTO_w_lock’; did you mean ‘CRYPTO_zalloc’? [-Wimplicit-function-declaration]
CRYPTO_w_lock(CRYPTO_LOCK_X509_STORE);
^~~~~~~~~~~~~
CRYPTO_zalloc
CRYPTO_w_lock is a macro which is no longer present in the latest version of crypto.h (OpenSSL-1.1) which makes it clear that my application is still looking at the system version of OpenSSL.
Within my source files, I am including the SSL header files like so:
#include <openssl/bio.h>
#include <openssl/ssl.h>
#include <openssl/err.h>
Although angular braces are supposed to tell the compiler to look into the system's header files, doesn't the -L on the command line as well as the -I flags force it to look for them in the said directories before looking at the system files?
I do have a crypto.h file in the custom install location for OpenSSL but the compiler seems to be ignore it for some reason
ab#ab1-pc:/home/AB/Documents/APP/OpenSSL-1.0.2p-installation$ find . -iname "crypto.h"
./include/openssl/crypto.h
What am I missing here?
UPDATE 1: As suggested by Darren, removed the trailing openssl so now my include path is
INC+=-I$(OPENSSL1.0.2p_INSTALL_LOC)/include and voila I can see the shared library getting created. However when I try to find references to ssl within this newly minted shared library, I see that I have 87 entries (which includes ALL symbols having ssl as part of their name)
ab#ab1-pc:~/Documents/AB/APP_2.17.0$ nm libAPP.so | grep -i "ssl" | wc -l
87
whereas listing only the global symbols from libssl.a tells me it has 1113 globally defined symbols.
ab#ab1-pc:~/Documents/AB/APP_2.17.0$ nm -g ../OpenSSL-1.0.2p-installation/lib/libssl.a | grep -i "ssl" | wc -l
1113
Shouldn't the former count be MORE than the latter?? Has it got something to do with the fact that my 'app' is a shared library? Even then shouldn't it pull in ALL the symbols (at least the global ones) from any static libraries it links against??
UPDATE 2: Now apparently since I was facing issues with SSL symbols, I switched to creating a static library so made changes to the makefile accordingly.

In this line:
INC+=-I$(OPENSSL1.0.2p_INSTALL_LOC)/include/openssl
... try changing it to:
INC+=-I$(OPENSSL1.0.2p_INSTALL_LOC)/include
... i.e., without the 'openssl' part.
Your includes, e.g.,
#include <openssl/bio.h>
#include <openssl/ssl.h>
#include <openssl/err.h>
... expect directory search paths to terminate on the 'include/', not on the 'openssl'.

Related

How to link different object file with LLVM library?

I am following the LLVM tutorial : Kaleidoscope: Code generation to LLVM IR, which will use LLVM libraries like LLVMContext, Module and so on. Different from the tutorial, I am trying to write the lexer, parser and code generator in different source file and link them into one executable file.
Here is my compile command on the Ubuntu 20.04:
clang++ -g -O3 -I /home/therlf/LLVM/include -I ./ -I /home/therlf/LLVM_Temp/llvm/include `llvm-config --cxxflags --ldflags --system-libs --libs all` ast/CallExprAST.o ast/NumberExprAST.o ast/PrototypeAST.o ast/FunctionAST.o ast/BinaryExprAST.o ast/VariableExprAST.o lexer/lexer.o logger/logger.o parser/parser.o main.cpp -o main
But I only get lots of "undefined error".
Here are some of them:
/usr/bin/ld: /tmp/main-2b71c8.o:(.data+0x0): undefined reference to `llvm::DisableABIBreakingChecks
/home/therlf/MyProject/tmp/ast/CallExprAST.cpp:6: undefined reference to `llvm::Module::getFunction(llvm::StringRef) const'
/home/therlf/LLVM/include/llvm/IR/InstrTypes.h:1112: undefined reference to `llvm::Instruction::Instruction(llvm::Type*, unsigned int, llvm::Use*, unsigned int, llvm::Instruction*)'
/usr/bin/ld: /home/therlf/LLVM/include/llvm/IR/InstrTypes.h:977: undefined reference to `llvm::VectorType::get(llvm::Type*, llvm::ElementCount)'
At first I thought it's including path's error. But when I compiled and ran the source file in the tutorial successfully, which is just a whole source file with everything packed into file, I knew the including path is nothing wrong.
I have searched for this question, and some blogs say that you should link them with lld and use the -fuse-ld=lld in the compile command. But I don't have lld, and the clang++ doesn't know the argument -fuse-ld, which will report an error. The blog says that you should have lld as long as you have installed LLVM. In fact here are what I got: LLVM tools
And I know the llvm-link is used to link IR file, not the object file compiled from cpp source file.
Here is my LLVM version:
10.0.0svn
And here is my Makefile:
SOURCES = $(shell find ast kaleidoscope lexer logger parser -name '*.cpp')
HEADERS = $(shell find ast kaleidoscope lexer logger parser -name '*.h')
OBJ = ${SOURCES:.cpp=.o}
CC = clang++
# -stdlib=libc++ -std=c++11
CFLAGS = -g -O3 -I /home/therlf/LLVM/include -I ./ -I /home/therlf/LLVM_Temp/llvm/include
LLVMFLAGS = `llvm-config --cxxflags --ldflags --system-libs --libs all`
.PHONY: main
main: main.cpp ${OBJ}
${CC} ${CFLAGS} ${LLVMFLAGS} ${OBJ} $< -o $#
clean:
rm -r ${OBJ}
%.o: %.cpp ${HEADERS}
${CC} ${CFLAGS} ${LLVMFLAGS} -c $< -o $#
In fact, I follow the project structure from the repository : ghaiklor/llvm-kaleidoscope and the Makefile is nearly identical.
Sincerely thank you for your answers!
I have sloved this problem by exchanging my linker ld with linker lld.
You should install lld first by this command in ubuntu if you can't find it in the LLVM/tools directory like I did.
sudo apt-get update
sudo apt-get -y install lld
And then you can add -fuse-ld=lld to your compile command
or you can
cd /usr/bin
ln -s /path/to/ld.lld /usr/bin/ld
This should work if everything goes well.
But I still can't figure out the reason behand this situation :-(

Im trying to compile program on Ubuntu and dont understand some things

Im a Windows dev who has no expirience on building C/C++ programs on Linux, but now I need to. Right way would be to go and learn Make and g++ compiler, but before I commit to that I want to figure out some basic stuff.
So I have .c program which is compiled with this makefile:
CUDA_VER=11.5
ifeq ($(CUDA_VER),)
$(error "CUDA_VER is not set")
endif
APP:= deepstream-test3-app
TARGET_DEVICE = $(shell gcc -dumpmachine | cut -f1 -d -)
NVDS_VERSION:=6.0
LIB_INSTALL_DIR?=/opt/nvidia/deepstream/deepstream-$(NVDS_VERSION)/lib/
APP_INSTALL_DIR?=/opt/nvidia/deepstream/deepstream-$(NVDS_VERSION)/bin/
ifeq ($(TARGET_DEVICE),aarch64)
CFLAGS:= -DPLATFORM_TEGRA
endif
SRCS:= $(wildcard *.c)
$(info info is $(SRCS))
INCS:= $(wildcard *.h)
PKGS:= gstreamer-1.0
OBJS:= $(SRCS:.c=.o)
CFLAGS+= -I../../../includes \
-I /usr/local/cuda-$(CUDA_VER)/include
CFLAGS+= $(shell pkg-config --cflags $(PKGS))
LIBS:= $(shell pkg-config --libs $(PKGS))
LIBS+= -L/usr/local/cuda-$(CUDA_VER)/lib64/ -lcudart -lnvdsgst_helper -lm \
-L$(LIB_INSTALL_DIR) -lnvdsgst_meta -lnvds_meta \
-lcuda -Wl,-rpath,$(LIB_INSTALL_DIR)
$(info info is $(CFLAGS))
all: $(APP)
%.o: %.c $(INCS) Makefile
gcc -c -o $# $(CFLAGS) $<
$(APP): $(OBJS) Makefile
gcc -o $(APP) $(OBJS) $(LIBS)
install: $(APP)
cp -rv $(APP) $(APP_INSTALL_DIR)
clean:
rm -rf $(OBJS) $(APP)
First thing I tried is to change this Makefile to compile it as C++ program. I changed .c file into .cpp, in makefile I change gcc to g++ everywhere and .c to .cpp everywhere. It gave me error that it couldnt find "main" entry point.
I gave up on that pretty fast and decided just to use lines output of original makefile, ending up with this:
g++ -c -o deepstream_test3_app.o -I../../../includes -I /usr/local/cuda-11.5/include -pthread -I/usr/include/gstreamer-1.0 -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include ./deepstream_test3_app.cpp
g++ -o deepstream-test3-app deepstream_test3_app.o -lgstreamer-1.0 -lgobject-2.0 -lglib-2.0 -L/usr/local/cuda-11.5/lib64/ -lcudart -lnvdsgst_helper -lm -L/opt/nvidia/deepstream/deepstream-6.0/lib/ -lnvdsgst_meta -lnvds_meta -lcuda -Wl,-rpath,/opt/nvidia/deepstream/deepstream-6.0/lib/
First question, can I combine this 2 launches of g++ into one?
Second, when I make changes to "./deepstream_test3_app.cpp" they are not noticed by compiler. I added
#include <iostream>
...
std::cout << "hello!" << std::endl;
and they are ignored. Its like g++ gets as input some other copy/older version of the file and I dont understand how to go about it.
Hope for any help, sorry if it's all sounds stupid.
Ignoring for the moment the issues surrounding compiling C code with a C++ compiler,
g++ -c -o deepstream_test3_app.o -I../../../includes -I /usr/local/cuda-11.5/include -pthread -I/usr/include/gstreamer-1.0 -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include ./deepstream_test3_app.cpp
g++ -o deepstream-test3-app deepstream_test3_app.o -lgstreamer-1.0 -lgobject-2.0 -lglib-2.0 -L/usr/local/cuda-11.5/lib64/ -lcudart -lnvdsgst_helper -lm -L/opt/nvidia/deepstream/deepstream-6.0/lib/ -lnvdsgst_meta -lnvds_meta -lcuda -Wl,-rpath,/opt/nvidia/deepstream/deepstream-6.0/lib/
First question, can I combine this 2 launches of g++ into one?
Yes. It is a common practice in makefiles to separate the compilation and linking steps, but that is not mandatory. When there are multiple sources, the separation makes it possible to limit recompilations to only the source files that have changed, but it doesn't make much difference, makefile or not, when there is only one source file.
The one-command version would be mostly a concatenation of the two commands you gave. One would omit the -c option, which instructs g++ to compile but not link, and one would omit the -o deepstream_test3_app.o, which specifies the name of the object file that we are no longer going to create. One would also omit the appearance of deepstream_test3_app.o drawn from the link (second) command, as we are going straight from source file to program. The rest of the options can be reordered to some extent, but all the -l options need to remain in the same order relative to each other and to any object files among the inputs. Here is how I would write it:
g++ -c -o deepstream_test3_app -I../../../includes -I /usr/local/cuda-11.5/include -pthread -I/usr/include/gstreamer-1.0 -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -Wl,-rpath,/opt/nvidia/deepstream/deepstream-6.0/lib/ ./deepstream_test3_app.cpp -lgstreamer-1.0 -lgobject-2.0 -lglib-2.0 -L/usr/local/cuda-11.5/lib64/ -lcudart -lnvdsgst_helper -lm -L/opt/nvidia/deepstream/deepstream-6.0/lib/ -lnvdsgst_meta -lnvds_meta -lcuda
Second, when I make changes to "./deepstream_test3_app.cpp" they are not noticed by compiler.
The compiler compiles the source file(s) you tell it to.
Its like g++ gets as input some other copy/older version of the file
It is possible that you are indeed telling it to compile a different version than the one you modified. It is also possible that compilation fails, so you don't get a new executable. And it is possible that when you try to run the result, you are not running the program you think you are running. We don't have enough information to know.
With regard to the last, however, do be aware that on Linux, unlike on Windows, the working directory is not automatically in the executable search path. If you want to run the compiled result from the above command, you would want to specify the path to it, which you could most easily do by prepending ./ to its simple name: ./deepstream-test3-app.

How to generate dependency file for executable (during linking) with gcc

gcc has -M-class options (-MMD, -MF, etc.) that allows to generate dependency file during compiling source file. The dependency file contains Makefile rules describing on which source files and headers the generated object file depends on. The dependency file may be included into Makefile and then make will automatically recompile source file when headers are changed.
I need a similar option but for generating dependency file during linking an executable. The dependency file should contain list of libraries used for linking an executable, so if any of libraries is updated, make will re-execute linking of the executable automatically.
I tried to use the same flags (-MMD, -MF), but they doesn't work for linking. It seems they are only for generating dependency files during compiling.
Is there any other flags or means for generating dependency file for executable?
So far I have not found dedicated gcc options for generating dependency file for executable, but found the --trace option (-Wl,--trace when used with gcc). This option generates list of libraries used during linking. Its output has the next format:
gcc -Wl,--trace myprog.c -o myprog -L. -lmylib
-lmylib (./libmylib.a)
-lgcc_s (/usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.5.2/libgcc_s.so)
...
The list of libraries then may be converted to Makefile rules using sed:
echo "myprog: " > myprog.dep
gcc -Wl,--trace myprog.c -o myprog -L. -lmylib \
| sed -n 's/.*(\(.*\)).*/\1 \\/p' >> myprog.dep
So myprog.dep will have the following content:
myprog: \
./libmylib.a \
/usr/lib/x86_64-linux-gnu/gcc/x86_64-linux-gnu/4.5.2/libgcc_s.so \
...
This dependency file may be included to Makefile and make will relink myprog if any of libraries are updated.
The dependency file doesn't contain list of object files, but the object files are usually known inside Makefile-script without help of compiler:
myprog: $(OBJS)
gcc -Wl,--trace $^ -o myprog -L. -lmylib | sed -n 's/.*(\(.*\)).*/\1 \\/p' >> myprog.dep

error while loading shared libraries: libnsd.so: cannot open shared object file: No such file or directory

I'm writing two C programs. One of them uses library libnsd.so. I compile two C programs using makefile which looks like this:
CC=gcc
CFLAGS= -Wall -g -O -fPIC
RM= rm
HLAV=main
HLAVO=main.o
all:lib $(HLAV)
cc c2.c -o c2
main: $(HLAVO)
$(CC) -L. -o $(HLAV) $(HLAVO) -lnsd
lib: libnsd.so
libnsd.so: nsd.o nd.o
$(CC) -shared $< -o $#
nsd.o: nsd.c nsd.h nd.h
nd.o: nd.c nsd.h nd.h
clean:
$(RM) -rf *.o *.so main
When I try to run an aplication I get an error:
error while loading shared libraries: libnsd.so: cannot open shared
object file: No such file or directory
Anyone knows how to solve it?
The error msg means your program can't find the dynamic library libnsd.so.
You need to find the library path from your system.
If the lib is not on the regular path, I suggest put it on the regular path.
whereis libnsd.so
mv your_dir/libnsd.so /usr/local/lib
Note: If the library is does not exist on your system, you should install it first.
Then, use ldconfig to write the path in the config file:
sudo echo "/usr/local/lib" >> /etc/ld.so.conf
sudo ldconfig
Or if you don't have root priviledge in your workstation, you can simply change user environment path:
export LD_LIBRARY_PATH=/usr/local/lib:$LD_LIBRARY_PATH

On mac, how to add path to environment variable?

I'm using opencv on mac, every time I compile the program, I have to type:
g++ -I /usr/local/include -L /usr/local/lib main.cpp
What can I do to avoid typing -I and -L params?
Create a Makefile:
CXXFLAGS=-I /usr/local/include -O3 -DSOMETHING
LDFLAGS=-L /usr/local/lib
LIBS=-lwhatever
main: main.o
$(LD) -o $# $* $(LDFLAGS) $(LIBS)
main.o: main.cpp
And then just type make at the command prompt:
$ make
trojanfoe is almost right, but the makefile doesn't use the conventional names. If it did, it would be even simpler:
CXXFLAGS=-I /usr/local/include -O3 -DSOMETHING
LDFLAGS=-L /usr/local/lib
LDLIBS=-lwhatever
With that makefile you can just type make main and make will use its implicit rules for compiling a C++ file

Resources