Make: Setting Target Specific Variables With Different Goals - makefile

I'm building a makefile that will be used to build a release or debug target library. I want to place the object and auto-generated dependancy files into either a debug or release directory structure, depending on the requested makefile goal. I don't want to specify a testable make command-line argument (i.e. DBG=1), but would prefer to run make -f Makefile, or make -f Makefiel dbg for release and debug target goals, respectively. Got that part down. I understand that I can't assign a target-specific variable containing the name of the object dir (either release or debug) that can be used as part of the Target specification in a rule, like I did in the example shown below. In this example, OBJDIR is the target-specific variable I would like to set depending on the build goal. For that reason, in this example, $(OBJDIR) is empty in the target rule $(OBJDIR)/%.o. Any recommendations on how to perform the suggested steps nicely? (The example shown is simply a copy/paste unverified example...syntax is not verified...in fact, I can't get the tabs to appear correctly...I'm hoping to get some implementation ideas). (Also, $(OBJDIR) is not set in the clean target as shown...since it is not in the dbg/all target dependancy heirarchy...thoughts?) Thanks in advance.
Makefile:
OBJS := a.o b.o c.o
SRCS := $(OBJS:.o=.c)
-- Set up the release and the debug directory paths and object filenames
RELEASE_DIR := ./release
RELEASE_OBJ := $(OBJS:%=$(RELEASE_DIR)/%)
DEBUG_DIR := ./debug
DEBUG_OBJ := $(OBJS:%=$(DEBUG_DIR)/%)
.PHONY : all dbg
all: CFLAGS = -O3
all: OBJDIR := RELEASE_DIR
all: df := $(RELEASE_DIR)/$(*F)
all: init_release lib1.so
dbg: CFLAGS = -g -O0
dbg: OBJDIR := DEBUG_DIR
dbg: df := $(DEBUG_DIR)/$(*F)
dbg: init_debug lib1.so
Lib1.so: $(OBJ)
init_release:
-#mkdir -p $(RELEASE_DIR)
init_debug:
-#mkdir -p $(DEBUG_DIR)
lib1.so: $(OBJ)
#echo '--------------------------------------------------------------'
#echo linking $#
#gcc -shared -o lib1.so $(OBJ)
-Compile including advance dependancy generation alg, per Tom Tromey:
# http://make.paulandlesley.org/autodep.html
$(OBJDIR)/%.o: %.c
echo $#
echo $(OBJDIR)
echo compiling $#
$(COMPILE.c) -MD -o $# $<
cp $(df).d $(df).P; \
sed -e 's/#.*//' -e 's/^[^:]*: *//' -e 's/ *\\$$//' \
-e '/^$$/ d' -e 's/$$/ :/' < $(df).d >> $(df).P; \
rm -f $(df)/$*.d
# If the goal is "clean", don't include these to avoid trying to build them
ifneq($(MAKECMDGOALS),clean)
-include $(SRCS:%.c=$(OBJDIR)/%.P)
endif
clean:
-#rm -f $(OBJDIR)/*.[Pdo] lib1.so

Target specific variables can be tricky. Use indirection instead. Make has lots of syntax to cut-down on boilerplate text. .SECONDEXPANSION is often good. A sketch:
.SECONDEXPANSION:
${DEBUG_OBJ} ${RELEASE_OBJ}: $$(patsubst %.o,%.c,$${#F})
gcc ${copts-${#D}} -c $< -o $#
Here we tell make that ./release/a.o depends on a.c. When make decides to build ./release/a.o it expands the shell line. As it does so, ${#D} is naturally release, so make carries on and expands ${copts-release} which you will have defined usefully.
Similarly, when producing ./debug/a.o make expands ${copts-debug}.
Copious use of $(warning [blah]), $(error [blah blah]) and the mandatory --warn-undefined-variables will help you get this right.

The Makefile you wrote is not valid, and it will not generate your targets as you expect. For instance, you cannot set the CFLAGS variable in the targets definitions all and dbg.
The only solution I can think of is to call make with the same Makefile defining the DBG variable as you wish. E.g.:
ifdef DBG
CFLAGS = -O0 -ggdb
OBJDIR = dbgdir
else
CFLAGS = -O2
OBJDIR = reldir
endif
all: $(OBJDIR)/target
#Your commands here
dbg:
$(MAKE) DBG=1
With this, if you call make, you have the release build. If you call make dbg you have the release build.

Related

Makefile with automatic dependency generation

I'm reading Managing Projects with GNU Make while trying to write a slim and general Makefile for my Fortran project. I'm lost at the point where the following code is presented,
define make-depend
$(CC) $(CFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -M $1 | \
$(SED) 's,\($$(notdir $2)\) *:,$$(dir $2) $3: ,' > $3.tmp
$(MV) $3.tmp $3
endef
%.o: %.c
$(call make-depend,$<,$#,$(subst .o,.d,$#))
$(COMPILE.c) -o $# $<
about which I read
Now, the object file and dependency file are logically linked: if one
exists the other must exist. Therefore, we don’t really care if a
dependency file is missing. If it is, the object file is also missing
and both will be updated by the next build.
Well, I don't understand the part in bold; how is it possible to compile a source file ($(COMPILE.c) -o $# $<) and generate its dependency file ($(call make-depend,$<,$#,$(subst .o,.d,$#))) at the same time? I mean, how can it work?
Maybe my Fortran experience misleads me, since I'm used to the fact that I can look in one source file to find which other source files it depends upon (looking for USE statements), then compile those source files (or repeating the process) and only after I can compile that one source file.
If needed, I can upload the short Makefile I'm trying to write, but it's obviously non-working. Specifically,
make pick one of the .f90 files, say some.f90, creates its associated dependency file some.d with $(call make-depend,$<,$#,$(subst .o,.d,$#)), then fails when trying to compile some.f90 with $(COMPILE.c) -o $# $<, since some.o depends on someother.o, which is indeed listed in some.d at this point.
Running make again will result in an attempt to compile someother.f90, which will end in either
2.1) success, if someother.o does not depend on any other .o file, in which case make would go back to step 1., picking another .f90 file.
2.2) or in error, but with someother.d being produced, and I should go manually to step 2.
Brutally, I keep on running make several times by for ((i = 1; i <= 10; i++)); do make; done, if ten times is enough.
This is the non-yet-working attempt:
# ====================================================================================================
FC = mpifort
FLIBS = -llapack -lblas
FCFLAGS = -ffree-form -ffree-line-length-0 -fimplicit-none -fdefault-real-8
# ====================================================================================================
PROGRAM := main
SRCDIR := src
OBJDIR := obj
MODDIR := mod
DEPDIR := dep
SRC := $(wildcard $(SRCDIR)/*.f90)
OBJ := $(patsubst $(SRCDIR)/%,$(OBJDIR)/%,$(SRC:.f90=.o))
MOD := $(shell echo $(patsubst $(SRCDIR)/%,$(MODDIR)/%,$(SRC:.f90=.mod)) | tr '[:upper:]' '[:lower:]')
DEP := $(patsubst $(SRCDIR)/%,$(DEPDIR)/%,$(SRC:.f90=.d))
# ====================================================================================================
$(PROGRAM): $(OBJ)
$(FC) $(FCFLAGS) -o $# $^ $(LDFLAGS) $(FLIBS)
# ====================================================================================================
ifneq "$(MAKECMDGOALS)" "clean"
-include $(DEP)
endif
# $(call make-depend,source-file,object-file,depend-file)
define make-depend
./dep-gen.sh < $1 | sed -e 's,\([^ ]*\),'"$(OBJDIR)"'/\1,g' \
-e 's,.*,'"$2 $3"': &,' > $(DEPDIR)/$(notdir $3)
endef
# ====================================================================================================
$(OBJDIR)/%.o: $(SRCDIR)/%.f90
#echo '************************************************************ target %.o'
$(call make-depend,$<,$#,$(DEPDIR)/$(subst .o,.d,$(notdir $#)))
$(FC) $(FCFLAGS) $(FLIBS) -c $< -J $(MODDIR) -o $#
# ====================================================================================================
.PHONY: clean veryclean
clean:
$(RM) $(OBJ) $(MOD) $(DEP)
veryclean: clean
$(RM) $(PROGRAM)
# ====================================================================================================
The bash script dep-gen.sh takes a file.f90 file from standard input and sends to standard output the space-separated list of files.o files from which the file.o file depends.

How to avoid this recursive Makefile to relink?

today I'm requesting your help about a Makefile that's driving me crazy. There it is:
# Executable name
NAME = libft.a
# Compiler and archive linker settings
CC = gcc
AR = ar
CFLAGS = -Wall -Wextra -Werror -O3 -g3
ARFLAGS = -rsc
IFLAGS = -I./includes/
# Project layout
SRC_DIR = ./src/
INC_DIR = ./inc/
OBJ_DIR = ./obj/
OBJ = $(shell grep -r .o ./obj | awk '{print $$3}' | tr '\n' ' ')
.PHONY: all clean fclean re
#------------------------------------------------------------------------------#
all: $(OBJ_DIR) $(NAME)
$(OBJ_DIR):
mkdir -p $(OBJ_DIR)
$(NAME): compile $(OBJ) $(INC_DIR)libft.h
#echo "Linking library $(NAME).\n"
#$(AR) $(ARFLAGS) $(NAME) $(OBJ)
#echo " ✧ $(AR) $(ARFLAGS) $(NAME) object files: OK! √\n"
compile:
make -C src/io
make -C src/lists
make -C src/memory
make -C src/strings
make -C src/tests
I've tried multiple combination of dependencies, rules, etc but I just don't get it. Sometimes I got it to stop relinking but in thoses cases it wouldn't re-compile object files because $(OBJ) was empty and wasn't updated after I ran compile.
This version is close to be good, but everytime I run make it executes the recipe $(NAME) and does the ar -rsc %(OBJ) .. How can I put them in dependencies to $(NAME) ?
Well, basically your entire approach here cannot succeed. Just for one example: you are trying to find the object files using grep (honestly I don't understand that shell command at all; what does printing the $3 word from the output of grep -r do??? Don't you just mean find $(OBJ_DIR) -name \*.o here?) This will expand to all the object files found in your subdirectories. But, that shell command runs when your top-level makefile is parsed, and that parsing happens before make runs any rules... so, no object files have been built yet! So, this target doesn't depend on anything. Even after some object files have been built, it only depends on object files that already exist, not on object files that are created during the build.
Really if I were you I'd do this completely differently. However, the simplest way to make your makefile work properly as written is to build $(NAME) using a recursive make as well; change your makefile like this:
all: compile
$(NAME): $(OBJ) $(INC_DIR)libft.h
#echo "Linking library $(NAME).\n"
#$(AR) $(ARFLAGS) $# $^
#echo " ✧ $(AR) $(ARFLAGS) $# object files: OK! √\n"
compile:
mkdir -p $(OBJ_DIR)
$(MAKE) -C src/io
$(MAKE) -C src/lists
$(MAKE) -C src/memory
$(MAKE) -C src/strings
$(MAKE) -C src/tests
$(MAKE) $(NAME)
Here all doesn't depend on $(NAME); instead, the compile step first builds everything then at the end it recursively invokes itself to build $(NAME); at this point we know everything is up to date and we can depend on the object files existing.
Other things: note I used the automatic variable $^ here not $(OBJ); that variable is a simple variable that runs a shell script: it's expensive! Every time you expand the $(OBJ) variable you pay that cost, so you only ever want to do it one time. Alternatively, you can use := to set OBJS instead so it's only invoked once per make instance. That's still one more time than you need but avoiding this will be painful.
I also moved the mkdir into the compile rule. It's cleaner there than as a prerequisite of all.
Finally, you should never invoke sub-makes using the make command directly. Always use the $(MAKE) variable, or various things will not work correctly.
The question was obvioulsy solved by the previous post.
You need to use the $(MAKE) variable to call recursively your make file with the $(NAME) rule instead of putting $(NAME) as a all dependency, after subsequent calls to your underlying Makefiles using the $(MAKE) variable again.

makefile target specific variables as prerequisites

Related: Target-specific Variables as Prerequisites in a Makefile
I'm trying to craft a Makefile which uses a target-specific-variable to specify the output directory for the object files and the final executable. The idea is to maintain two separate binary versions, a 'release' version and a 'debug' version with extra debugging information.
My problem is that 'make' does a clean build every time, even if I haven't changed a thing. I'm pretty sure it's because 'make' is evaluating the prerequisites of the target 'corewars' before the variable declaration in the prerequisites for the 'debug' or 'release' target.
The Makefile is presented below.
CXX=g++
LD=g++
LDFLAGS=
CXXFLAGS=-Iinclude -Wall -Wextra
OBJECTS=main.o Machine.o Core.o ProcessQueue.o Instruction.o
OUTPUT_DIR:=Test/
.PHONY: default
.PHONY: all
.PHONY: release
default: release
all: release
release: OUTPUT_DIR:=Release/
release: corewars
.PHONY: debug
debug: CXXFLAGS+=-DDEBUG -g
debug: OUTPUT_DIR:=Debug/
debug: corewars
corewars: $(OUTPUT_DIR) $(addprefix $(OUTPUT_DIR),$(OBJECTS))
$(LD) -o $(addprefix $(OUTPUT_DIR),corewars) $(addprefix $(OUTPUT_DIR),$(OBJECTS))
Release:
mkdir -p $#
Debug:
mkdir -p $#
%.o: %.cpp include/%.h
$(CXX) -c $(CXXFLAGS) $< -o $(OUTPUT_DIR)$#
.PHONY: clean
clean:
$(RM) -r Release
$(RM) -r Debug
First of all, a non-phony recipe must create a target, $#, not $(OUTPUT_DIR)$#. Also consider converting directory dependencies into order-only prerequisites.
In order to get a proper value of $(OUTPUT_DIR) inside the list of prerequisites, you would have to use secondary expansion, because otherwise, during the primary expansion, the global definition OUTPUT_DIR:=Test/ is used instead of the target-specific one.
Unfortunately, I can't think of a sane way to make it work using target specific variables, without resorting to secondary expansion and vpath magic. Personally I would rather setup the environment first (find out the value of OUTPUT_DIR, etc.) and then re-execute Make with the proper values.
ifndef OUTPUT_DIR
.PHONY: default all release debug
default all: release
release: export OUTPUT_DIR := Release/
debug: export OUTPUT_DIR := Debug/
debug: export EXTRA_CXXFLAGS := -DDEBUG -g
release debug:
#$(MAKE)
else
# ...
CXXFLAGS := -Iinclude -Wall -Wextra $(EXTRA_CXXFLAGS)
PROGRAM := $(OUTPUT_DIR)corewars
OBJECTS := $(addprefix $(OUTPUT_DIR), \
main.o Machine.o Core.o ProcessQueue.o Instruction.o)
# Default target.
$(PROGRAM): $(OBJECTS) | $(OUTPUT_DIR)
$(LD) -o $# $<
$(OUTPUT_DIR)%.o: %.cpp | $(OUTPUT_DIR)
$(CXX) -c $(CXXFLAGS) $< -o $#
$(OUTPUT_DIR):
mkdir -p $#
endif # OUTPUT_DIR
The two parts could them be split into separate makefiles, the root (starter) one, and the one that does the real work, to make the whole thing more manageable.
Target-specific variables are only available within the context of the recipes of the target and its recursive prerequisites. That is, target-specific variables cannot be used as targets nor prerequisites.
One workaround is the makefile there.

Change Makefile variable value inside the target body

Is there a way to reassign Makefile variable value inside of the target body?
What I am trying to do is to add some extra flags for debug compilation:
%.erl: %.beam
$(ERLC) $(ERLFLAGS) -o ebin $<
test: clean debug_compile_flag compile compile_test
debug_compile:
$(ERLCFLAGS) += -DTEST
So if I invoke test target I would like to clean up my environment, add some new flags (like -DTEST to the existing ones), compile the whole code once again (first sources, then test modules).
I do not want to copy/paste the code for compiling with some new flags set since there is a lot of logic put here and there.
Is there some easy way to redefine the variable value so I can reuse the existing code?
Yes, there is an easy way to do it, and without rerunning Make. Use a target-specific variable value:
test: clean debug_compile
debug_compile: ERLCFLAGS += -DTEST
debug_compile: compile compile_test;
Another answer is here: Define make variable at rule execution time.
For the lazy, you can have rules like the following (FLAG and DEBUG are my variables):
.DBG:
$(eval FLAG += $(DEBUG))
Here is the solution I use:
PASSWORD = abc123
main: sub
#echo "in main" $(PASSWORD)
sub:
#echo "in sub" $(PASSWORD)
$(eval PASSWORD=qwerty)
#echo "in sub" $(PASSWORD)
If you run make main then the output is:
in sub abc123
in sub qwerty
in main qwerty
You can see that the original value "abc123" is overwritten in the sub and the new value "qwerty" is visible at the main level.
To override on the command line try something like:
make prefix=<path to new dir> install
This won't change Makefile, but will alter the variable.
I wanted to add a target in a makefile to run tests, which implied recompiling the source code with some debug flags. Ian's answer: https://stackoverflow.com/a/15561911/ was the only solution that worked.
Here's the Makefile I came up with, which guaranties the order of execution when running make tests:
TARGET = a.out
CC = g++
GENERIC_F = -Wall -Wextra -I. -Idoctest/doctest/
CFLAGS = -O0 -std=c++11 $(GENERIC_F)
DEBUG_MODE = -DDEBUG
LINKER = g++
LFLAGS = $(GENERIC_F) -lm
SRCDIR = src
OBJDIR = build
BINDIR = bin
SOURCES = $(wildcard $(SRCDIR)/*.cc)
INCLUDES = $(wildcard $(SRCDIR)/*.h)
OBJECTS = $(SOURCES:$(SRCDIR)/%.cc=$(OBJDIR)/%.o)
rm = rm -f
.PHONY: clear_screen tests extend_cflags
$(BINDIR)/$(TARGET): $(OBJECTS) $(INCLUDES)
$(LINKER) $(OBJECTS) $(LFLAGS) -o $#
#echo -e "Linking complete!\n"
$(OBJECTS): $(OBJDIR)/%.o : $(SRCDIR)/%.cc $(INCLUDES)
#mkdir -p $(OBJDIR) $(BINDIR)
$(CC) $(CFLAGS) -c $< -o $#
#echo -e "Compiled "$<" successfully!\n"
.PHONY: clean
clean:
#$(rm) $(OBJECTS)
#echo "Cleanup complete!"
.PHONY: remove
remove: clean
#$(rm) $(BINDIR)/$(TARGET)
#echo "Executable removed!"
clear_screen:
#clear
extend_cflags:
$(eval CFLAGS += $(DEBUG_MODE))
tests: | remove extend_cflags $(BINDIR)/$(TARGET) clear_screen
#$(BINDIR)/$(TARGET)
Edit: As explained by Beta in the other answer, it is possible.
No. There is no way to do this in the Makefile. You can however change the value of a variable on the make command line. If you rewrite your Makefile as follows:
ERLCFLAGS += $(ERLCFLAGSADDED)
%.erl: %.beam
$(ERLC) $(ERLCFLAGS) -o ebin $<
test: clean compile compile_test
Then, you can invoke make to perform your tests using:
make ERLCFLAGSADDED=-DTEST test

Generate all project dependencies in a single file using gcc -MM flag

I want to generate a single dependency file which consists of all the dependencies of source files using gcc -M flags through Makefile. I googled for this solution but, all the solutions mentioned are for generating multiple deps files for multiple objects.
DEPS = make.dep
$(OBJS): $(SOURCES)
#$(CC) -MM $(SOURCEs) > $(DEPS)
#mv -f $(DEPS) $(DEPS).tmp
#sed -e 's|.$#:|$#:|' < $(DEPS).tmp > $(DEPS)
#sed -e 's/.*://' -e 's/\\$$//' < $(DEPS).tmp | fmt -1 | \
sed -e 's/^ *//' -e 's/$$/:/' >> $(DEPS)
#rm -f $(DEPS).tmp
But it is not working properly. Please tell me where i'm making the mistake.
Something along these lines is what I use to get all my dependencies in a single file:
program_H_SRCS := $(wildcard *.h)
program_C_SRCS := $(wildcard *.c)
DEPS = make.deps
make.deps: $(program_C_SRCS) $(program_H_SRCS)
$(CC) $(CPPFLAGS) -MM $(program_C_SRCS) > make.deps
include $(DEPS)
This basically causes all the user ( as opposed to system ) dependencies to be rebuilt into a single file whenever any C or H file in the project is modified.
+++++++++ EDIT +++++++++++
I've since found a better way of doing things. I generate a separate dep file for each source file. Here is the basic makefile:
program_NAME := myprogram
program_SRCS := $(wildcard *.c)
program_OBJS := ${program_SRCS:.c=.o}
clean_list += $(program_OBJS) $(program_NAME)
# C Preprocessor Flags
CPPFLAGS +=
# compiler flags
CFLAGS += -ansi -Wall -Wextra -pedantic-errors
.PHONY: all clean distclean
all: $(program_NAME)
clean:
#- $(RM) $(clean_list)
distclean: clean
# Generate dependencies for all files in project
%.d: $(program_SRCS)
# $(CC) $(CPPFLAGS) -MM $*.c | sed -e 's#^\(.*\)\.o:#\1.d \1.o:#' > $#
clean_list += ${program_SRCS:.c=.d}
$(program_NAME): $(program_OBJS)
indent -linux -brf $(program_SRCS)
splint $(program_SRCS)
$(LINK.c) $(program_OBJS) -o $(program_NAME)
ifneq "$(MAKECMDGOALS)" "clean"
# Include the list of dependancies generated for each object file
-include ${program_SRCS:.c=.d}
endif
This does two things:
If any of the files that foo.c depend on change then foo.o is rebuilt without having to rebuild other files in the project.
The dep file itself has the same dependencies as the object file, so that if any of the deps are modified the dep file itself is also regenerated, before the object file deps are checked.
I think is is expected behaviour for gcc -M, where typically you'd do something like this:
FOO_SOURCES= \
src/foo.c \
src/bar.c
FOO_OBJECTS = $(FOO_SOURCES:.c=.o)
FOO_DEPS = $(FOO_OBJECTS:.o=.d)
(... lots of targets ...)
-include $(FOO_DEPS)
Note, -include not include as the dependencies will obviously not exist until at least one build has been run. Regardless, dependencies are generated on a per module basis.
Also note that gcc -M does not always work as you would expect it to work, broadly depending on what version of gcc you happen to be using.
I think what you want is something called makedep, which does what you want without sed hackery in the makefile.

Resources