How to link libs once only in GCC? - gcc

Pardon my question, I am a beginner to GCC. I have a framework project that holds source code for multiple subcomponents.
The structure is below:
Framework/
makefile //Master makefile in root
Component1/
src/
bin/
makefile
Component2/
src/
bin/
makefile
...
...
...
ComponentN/
src/
bin/
makefile
Now each makefiles in ComponentN/ each of directories will compile the code in its respective src/ and output .o to bin/ directory.
The root makefile however searches all the .o files recursively and links them all into one executable named 'framework'
Problem:
For code dependencies like glib,gdbus,gio I have to link them once when creating .o objects, in each of the component projects.
Plus I have to link the dependencies again when linking all the .o into one executable at root level.
Why do I have to do it twice? I am interested in understanding the internal mechanics.
As per request I am putting in makefile of the individual component libs that products *.o files
CC = gcc
CFLAGS = -g3
LIBS = `pkg-config --cflags --libs glib-2.0`
BINDIR = bin
OUTOBJ = $(addprefix $(BINDIR)/, objex.o)
$(BINDIR)/%.o : %.c
$(CC) -c $< $(CFLAGS) -o $# $(LIBS)
all: $(OUTOBJ)
$(OUTOBJ): | $(BINDIR)
$(BINDIR):
mkdir $(BINDIR)
.PHONY: clean
clean:
rm bin/*

Object files (.o) are created by compilation commands, e.g.
gcc -c -o foo.o foo.c ...
g++ -c -o baz.o baz.cpp ...
-c means compile; don't link. No linkage happens in the creation of
object files by the compiler. Any linkage options that you add to a compilation
command, e.g.
gcc -c -o foo.o foo.c -L/my/libs -lbar -lgum
are simply ignored.
Linkage options are acted on by a linkage command, which creates a program, or shared/dynamic
library, by linking together object files and libraries, e.g.
gcc -o prog foo.o baz.o -L/my/libs -lbar -lgum
gcc -shared -o libfoobaz.so foo.o baz.o -L/my/libs -lbar -lgum
So:
For code dependencies like glib,gdbus,gio I have to link them once when creating .o objects, in each of the component projects.
No you don't, and you can't.
Later
With sight of the problem makefile it is quite clear how to eliminate
the $(LIBS) reference from the compilation recipe, and what has been stopping you. The makefile defines:
LIBS = `pkg-config --cflags --libs glib-2.0`
which is a mistake. That makes $(LIBS) expand to the standard output of the
command:
pkg-config --cflags --libs glib-2.0
which is a single string containing both the compilation options required
for compiling source that #include-s the glib-2.0 API (on account of --cflags)
and also the linkage options required for linking a program or shared library
against libglib-2.0 (on account of --libs). On my system that is:
$ pkg-config --cflags --libs glib-2.0
-I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -lglib-2.0
of which the compilation options alone would be output by:
$ pkg-config --cflags glib-2.0
-I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include
and the linkage options alone would be output by:
$ pkg-config --libs glib-2.0
-lglib-2.0
But because both sets of options are available only together through the expansion
of $(LIBS) you cannot successfully compile without passing the
linkage option -lglib-2.0, which is redundant and ignored.
As your make tool is evidently GNU Make, the makefile (which BTW is not that bad!) would be better written as:
Makefile
CC := gcc
CFLAGS := -g3 $(shell pkg-config --cflags glib-2.0)
BINDIR := bin
SRCS := objex.c
OUTOBJ := $(addprefix $(BINDIR)/, $(SRCS:.c=.o))
.PHONY: all clean
all: $(OUTOBJ)
$(BINDIR)/%.o : %.c
$(CC) -c $< $(CFLAGS) -o $#
$(OUTOBJ): | $(BINDIR)
$(BINDIR):
mkdir -p $(BINDIR)
clean:
$(RM) $(OUTOBJ)
which dispenses with LIBS and runs from scratch like:
$ make
mkdir -p bin
gcc -c objex.c -g3 -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -o bin/objex.o
Note a few other improvements:-
The use of immediate expansion (:=) wherever applicable in preference to unnecessary
recursive expansion (=). See 6.2 The Two Flavors of Variables
The use of direct shell substitution by make - $(shell command) - in preference to backtick-expansion in
recipe execution. See 8.13 The shell Function.
all, like clean is a phony target
and you need to tell make that it is, to avoid the booby-trap in which something creates a file called all in
the project directory without you noticing and make mysteriously stops detecting any work for it to do.
With your clean receipe:
clean:
rm bin/*
make clean will fail if ever run except following a successful build. The recipe
is replaced with $(RM) $(OUTOBJ), using GNU Make's predefined delete macro, which
won't fail.
Finally, remember that your linkage recipe, wherever it is, does need the library options for glib-2.0,
which you should provide in its makefile with:
LIBS := $(shell pkg-config --libs glib-2.0) # ...and any more library options required
for use in a recipe similar to:
prog: $(OBJS)
$(CC) -o $# $(LDFLAGS) $^ $(LIBS)
[1] Strictly, preprocessor options should appear in the definition of CPPFLAGS
(C PreProcessor Flags), not to be confused with CXXFLAGS (C++ compilation options).
[2] Strictly, linkage options other than libraries should appear in the definition
of LDFLAGS.

Related

Can't use LAPACK in makefile

I have program (in fortran) where I'm using three custom modules, which make use of LAPACK. Until now I've compiled my program using the following shell script:
filestring="main"
gfortran -c mod_exp.f90 mod_genmat.f90 mod_print.f90 $filestring.f90
gfortran mod_exp.o mod_genmat.o mod_print.o $filestring.o -llapack -lblas
rm mod_exp.o mod_genmat.o mod_print.o $filestring.o exponentiate.mod genmat.mod printing.mod printing_subrtns.mod
mv a.out $filestring
Since I've been using more and more modules and different programs using them, I've decided to start using makefiles. Following a tutorial, I managed to write the following:
FC = gfortran
FFLAGS = -Wall -Wextra -llapack -lblas #-fopenmp
SOURCES = mod_print.f90 mod_genmat.f90 mod_exp.f90 main.f90
OBJ = ${SOURCES:.f90=.o} #substitute .f90 with .o
%.o : %.f90 #creation of all *.o files DEPENDS on *.f90
$(FC) $(FFLAGS) -c -O $< -o $#
main: $(OBJ)
$(FC) $(FFLAGS) -o $# $(OBJ)
clean:
#rm -f *.o *.mod main
However, when executing make, it says that the LAPACK functions are not recognized. One such mistake is the following:
/usr/bin/ld: mod_exp.o: in function `__exponentiate_MOD_diagun':
mod_exp.f90:(.text+0x37f): undefined reference to `zgees_'
...
collect2: error: ld returned 1 exit status
One possible mistake I've seen is that I need to specify the location of the libraries. However, it would seem strange since I didn't need to do it before; also, I don't know how to find it.
Please show the link command that make invoked, that caused the error to be generated.
I'm confident that if you cut and paste that exact command line to your shell prompt, you will get the same error you see when make runs it. So the problem is not make, but your link command.
The problem is that you have put the libraries before the objects in the link line. Libraries should come at the end, after the objects, else when the linker examines the libraries it doesn't know what symbols will need to be included (because no objects have been parsed yet to see what symbols are missing).
This is why LDLIBS is traditionally a separate variable:
FC = gfortran
FFLAGS = -Wall -Wextra #-fopenmp
LDLIBS = -llapack -lblas
SOURCES = mod_print.f90 mod_genmat.f90 mod_exp.f90 main.f90
OBJ = ${SOURCES:.f90=.o} #substitute .f90 with .o
%.o : %.f90 #creation of all *.o files DEPENDS on *.f90
$(FC) $(FFLAGS) -c -O $< -o $#
main: $(OBJ)
$(FC) $(FFLAGS) -o $# $(OBJ) $(LDLIBS)

Makefile compiles all the files everytime

My Makefile compiles all the files everytime I run it though the files have not been changed. I know that this question has been asked several times but none of the provided solutions seem to work for me. I am new to Makefile and most of the times I do not understand the jargon used in the solution. Also, I want to save all the generated .o files under the folder 'obj'
Here is my folder structure
project (-)
gen (-)
display (-)
.c and .h files
logic (-)
.c and .h files
lib (-)
include (-)
.h files
.lib files
man (-)
.c and .h files
obj (-)
want to save all the .o files here
I am running this on Windows OS using MinGW
Here is my Makefile:
ALL: demo
SRCS:= filename1.o filename2.o filename3.o filename4.o and so on till filename27.o
demo: display.o logic.o man.o
gcc $(SRCS) -lglut32 -loglx -lopengl32 -Llib -o demo
display.o:
gcc -Igen/display -Igen/logic -Iman -Ilib/include gen/display/*.c -lglut32 -loglx -lopengl32 -Llib -c
logic.o:
gcc -Igen/display -Igen/logic -Iman -Ilib/include gen/logic/*.c -lglut32 -loglx -lopengl32 -Llib -c
man.o:
gcc -Igen/display -Igen/logic -Iman -Ilib/include man/*.c -lglut32 -loglx -lopengl32 -Llib -c
clean:
#echo "Cleaning up.."
-rm -rf *.o
-rm *.exe
NOTE: glut and oglx files are present in the lib folder. Display.o, lib.o and man.o do not have corresponding .c files. They are just folder names with many c files in them.
I understand this could be the problem. As there are no display.o, logic.o and man.o files created, MAKE complies the rule associated with it eveytime. SO how do I tell it to check for the actual .o filename1.o, filename2.o etc for the timestamp and recompile ONLY if they are older than the corresponding c files and h files maybe even the lib files they depend on.
I tried the following to create dependencies and avoid compiling of files everytime. But this did not help.
%.d: %.c
#set -e; rm -f $#; \
$(CC) -M $(CFLAGS) $< > $#.$$$$; \
sed 's,\($*\)\.o[ :]*,\1.o $# : ,g' < $#.$$$$ > $#; \
rm -f $#.$$$$
At a basic level, make is looking for lines like:
target: dependency
command
If target does not exist, it calls the rule for dependency and then runs command. If target does exist, it tests if dependency is newer or does not exist. If so, it calls the rule for dependency and then runs command. Otherwise, it stops.
Significantly, the rule for dependency will only be called if (a) dependency doesn't exist, or (b) dependency is newer than target.
In the question, assume we run make demo. Then make looks for the line that begins demo: and notices it declares dependencies. So it looks at each dependency in turn to see if they require action. It first discovers display.o. It notices that display.o: does not exist, so it runs the associated rule. It does the same for the other *.o.
To avoid the *.o rules always being run because no associated file exists, you could rewrite like:
ALL: demo
SRCS:= filename1.o filename2.o filename3.o filename4.o and so on till filename27.o
demo: display.ts logic.ts man.ts
gcc $(SRCS) -lglut32 -loglx -lopengl32 -Llib -o demo
display.ts: gen/display/*.c
gcc -Igen/display -Igen/logic -Iman -Ilib/include gen/display/*.c -lglut32 -loglx -lopengl32 -Llib -c
echo . > display.ts
logic.ts: gen/logic/*.c
gcc -Igen/display -Igen/logic -Iman -Ilib/include gen/logic/*.c -lglut32 -loglx -lopengl32 -Llib -c
echo . > logic.ts
man.ts: man/*.c
gcc -Igen/display -Igen/logic -Iman -Ilib/include man/*.c -lglut32 -loglx -lopengl32 -Llib -c
echo . > man.ts
clean:
#echo "Cleaning up.."
-rm -rf *.o *.ts
-rm *.exe
Problem is that your binary object targets (like display.o) do not actually match files produced by their rules. If you tell make it needs to make target display.o, it (normally, except for phony targets, but those always rerun) expect the corresponding file to be produced by the rule's recipe and it can track if the target needs to be remade. If no such file is produces, this target always evaluates as outdated and needing remaking.
A bit of a silly example of this would be the following tree:
.
├── Makefile
├── main.c
└── test
└── file.c
and Makefile:
main: test.o main.o
$(CC) -o main *.o
test.o:
$(CC) $(CFLAGX) -c test/*.c
There is no test.o file and target needs to be remade... the rule runs, produces file.o (again). Since this target was remade and is prerequisite of main... everything always gets remade.
Now with this small modification:
main: test.o main.o
$(CC) -o main *.o
test.o:
$(CC) $(CFLAGX) -o $# -c test/*.c
test.o target indeed produces test.o file and the rule needs no remaking if test.c does not change... and with test.o unchanged and main.c perhaps as well, we get:
$ make
make: 'main' is up to date.
It still is not entirely correct as it really should read:
main: test.o main.o
$(CC) -o main $+
test.o: test/*.c
$(CC) $(CFLAGX) -o $# -c $^
Where I declare depend prerequisites of test.o and reference both them and the target by automatic variable in the rule's recipe. And Same goes for prerequisites for linking. Of course in this simple example I could just rely on implicit pattern rules and do this:
main: test/file.o main.c
test/file.o: test/*.c
What does this mean for your makefile? When you compile your object files, have a look what do they actually produce and match your target to that or (with -o $# for instance) tell them to produce exactly the file matching your target.
I've extended the silly example a bit and there are now two files in test/:
.
├── Makefile
├── main.c
└── test
├── file.c
└── other.c
And the Makefile can look something like this:
main: obj/file.o obj/other.o main.c
obj/%.o: test/%.c
mkdir -p obj
$(CC) $(CFLAGS) -c -o $# $^
It now stores object files in obj/ and make still understand what needs what and can track changes. Of course your setup is more complex and will require more rules, perhaps also divining actual sources or intermediate targets from the directory tree and define few variables to work with that information, e.g.:
OBJS := $(patsubst test/%.c,obj/%.o,$(wildcard test/*.c))
main: $(OBJS) main.c
obj/%.o: test/%.c
mkdir -p obj
$(CC) $(CFLAGS) -c -o $# $^
But the principles remain the same.

Dynamic library cannot load when running program outside build directory

I have the following Makefile:
OBJ=main.o other.o other1.o other2.o
LINKDIVSUF=-L libdivsufsort-master/build/lib/ -ldivsufsort64 -Wl,-R libdivsufsort-master/build/lib/
INCDIVSUF=-I libdivsufsort-master/build/include -ldivsufsort64
EXE=program
COMPFLAGS=-MMD -fopenmp -std=c++17 -O3
CXX=g++
$(EXE):$(OBJ)
$(CXX) $(COMPFLAGS) $(OBJ) -o $(EXE) $(LINKDIVSUF) -lz -lboost_regex -lboost_program_options
%.o: %.cpp
$(CXX) $(COMPFLAGS) $(INCDIVSUF) -c $<
-include $(OBJ:.o=.d)
The program links to a dynamic library, libdivsufsort64.so.3, located from the build directory at ./libdivsufsort-master/build/lib/.
Use of -Wl,-R libdivsufsort-master/build/lib/is to avoid having to concatenate absolute/path/to/libdivsufsort-master/build/lib/ to LD_LIBRARY_PATH in order to run program. Indeed, when I make program
without -Wl, -R libdivsufsort-master/build/lib/, and without setting the LD_LIBRARY_PATH as mentioned and subsequently run program, I get the following error message:
./program: error while loading shared libraries: libdivsufsort64.so.3: cannot open shared object file: No such file or directory
With -Wl, -R libdivsufsort-master/build/lib/, program runs successfully with no alteration to LD_LIBRARY_PATH, but only when
I run program from the same directory in which it was built.
If I try to run program when compiled with -Wl, -R libdivsufsort-master/build/lib/ from any other directory, it fails to run, terminating with
the aforementioned error message.
How can I change the g++ compilation options (or anything else at compilation time) to enable program to run from
any directory whilst avoiding the need to alter LD_LIBRARY_PATH? The only "solution" I have found is to concatenate libdivsufsort-master/build/lib/ to LD_LIBRARY_PATH. By doing so, I can run program from any
directory, thus removing the need to compile with Wl,-R libdivsufsort-master/build/lib/, however, this of course requires the user of program to manually set their LD_LIBRARY_PATH, which I specifically want to avoid.
Solution
Reading this post that discusses the use of relative or absolute paths with -R (-rpath) I came up with this solution.
Append the following lines, such that the Makefile is now:
libdivsufsort_lib = $(addprefix $(shell pwd), /libdivsufsort-master/build/lib/)
libdivsufsort_include = $(addprefix $(shell pwd), /libdivsufsort-master/build/include/)
OBJ=main.o other.o other1.o other2.o
LINKDIVSUF=-L libdivsufsort-master/build/lib/ -ldivsufsort64 -Wl,-R libdivsufsort-master/build/lib/
INCDIVSUF=-I libdivsufsort-master/build/include -ldivsufsort64
EXE=program
COMPFLAGS=-MMD -fopenmp -std=c++17 -O3
CXX=g++
$(EXE):$(OBJ)
$(CXX) $(COMPFLAGS) $(OBJ) -o $(EXE) $(LINKDIVSUF) -lz -lboost_regex -lboost_program_options
%.o: %.cpp
$(CXX) $(COMPFLAGS) $(INCDIVSUF) -c $<
-include $(OBJ:.o=.d)
This avoids the use of $ORIGIN, to produce an absolute path to programs
directory, which is not supported on some systems. The two additional
lines produce the absolute path irrespective of the binary's location -
it just needs to kept in the build directory and compiled again if the build
directory moves. Importantly, program can now be called from outside the
build directory.
You need to use $ORIGIN with -Wl,-R to locate the library in relative path:
LINKDIVSUF = ... -Wl,-R,'$ORIGIN/libdivsufsort-master/build/lib'

Compiling SDL project on Raspberry Pi

I am trying to build a project with make (gcc on Raspbian)
Here is the makefile (I removed some unnecessary parts):
objects = 3d.o Affichage.o [...]
cflags = -I/usr/local/include/SDL2 -L/usr/local/lib -lSDL2
poly : %(objects)
gcc $(cflags) $(objects) -o poly
($objects) : types.h
[...]
When running Make, I got:
cc -c -o Affichage.o Affichage.c
fatal error: SDL.h: No such file or directory
#include <SDL.h>
I checked the folders, everything seems ok. SDL.h is indeed in /usr/local/include/SDL2. I tried to remove options one by one in cflags, no luck...
What am I missing?
Make told you exact command it tried to execute:
cc -c -o Affichage.o Affichage.c
This don't have -I path, which is the source of an error.
You have target for your resulting executable but not for object files. Make have builtin rule to compile object files from C sources, but it isn't aware of your cflags variable. So far your options are:
Define your own pattern rule
e.g:
%.o: %.c
gcc $(cflags) -c $< -o $#
However, your cflags contains -lSDL2, which is linking flag, which should be specified only on linking phase (so technically it isn't cflag). Move it to separate variable (usually LIBS, which may then be enfolded into make's semi-standard LDFLAGS).
Use variables that make is aware of
In that case, it is CFLAGS:
CC:=gcc
CFLAGS:=-I/usr/local/include/SDL2
LIBS:=-lSDL2
LDFLAGS:=-L/usr/local/lib $(LIBS)
objects:=3d.o Affichage.o
poly: $(objects)
$(CC) $^ -o $# $(LDFLAGS)
$(objects): types.h
The rest will be done by implicit rules.

extra CFLAGs with Makefile

I'm reading through Foundations of GTK+ and in so doing decided to write a simple makefile that would let me run "make " to compile the example program I'd just written. I also stumbled upon a list of compiler directives here that the Gnome team specified will help moving from GTK2 to GTK3, so I wanted to include those.
I'm a make noob for all intents and purposes, so this is what I came up with:
CC = gcc
CFLAGS += -Wall
GTK_DFLAGS = -DGTK_DISABLE_SINGLE_INCLUDES -DGDK_DISABLE_DEPRECATED -DGTK_DISABLE_DEPRECATED -DGSEAL_ENABLE
GTK_CFLAGS = $(shell pkg-config --cflags gtk+-3.0)
GTK_LDFLAGS = $(shell pkg-config --libs gtk+-3.0)
%.o: %.c
$(CC) $(CFLAGS) $(GTK_DFLAGS) $(GTK_CFLAGS) -c -o $# $<
%: %.o
$(CC) $(CFLAGS) $(GTK_DFLAGS) $(GTK_CFLAGS) $(GTK_LDFLAGS) -o $# $<
.PHONY: clean
clean:
rm -f *.o *~
And as you might guess, it doesn't work quite right. I know running pkg-config from inside the makefile isn't an ideal solution, but this is for my small-scale learning projects and not for deployment of any sort. That said, the output is weird to me; it seems like make just ignores any variables after CFLAGS.
Something like:
[patrick#blackbox ch2]$ make helloworld
gcc -Wall helloworld.c -o helloworld
helloworld.c:1:21: fatal error: gtk/gtk.h: No such file or directory
#include <gtk/gtk.h>
^
compilation terminated.
<builtin>: recipe for target 'helloworld' failed
make: *** [helloworld] Error 1
If I add have the contents of GTK_DFLAGS simply tacked onto the end of CFLAGS, they appear on the command line, but the pkg-config variables are still missing.
It's obvious to me that I messed something simple up, but after an hour of vaguely worded Googling, I'm fresh out of ideas as to what it is.
Found the answer, and of course the vocabulary I was missing when asking this question/doing earlier searches.
CC = gcc
CFLAGS += -Wall -std=c11
GTK_DFLAGS = -DGTK_DISABLE_SINGLE_INCLUDES -DGDK_DISABLE_DEPRECATED -DGTK_DISABLE_DEPRECATED -DGSEAL_ENABLE
GTK_CFLAGS := $(shell pkg-config --cflags gtk+-3.0)
GTK_LDFLAGS := $(shell pkg-config --libs gtk+-3.0)
%: %.c
$(CC) $(CFLAGS) $(GTK_DFLAGS) $(GTK_CFLAGS) $(GTK_LDFLAGS) -o $* $*.c
.PHONY: clean
clean:
rm -f *~
This does what I want, which is to compile a single .c file of any name into a program of the same name with the GTK flags I was looking to use.
Thanks to those who contributed!
You need a target for helloworld in your Makefile. Something like this:
helloworld: helloworld.o
$(CC) -o helloworld helloworld.o $(LDFLAGS) $(GTK_LDFLAGS)

Resources