Make 3 passes for a target - makefile

I'll explain myself, here is my scenario:
Compile my target
Do a first optimization based on the first compilation
Do a second optimization based on the first optimization
Do a third optimization based on the second optimization
So far I tried the following:
.SUFFIXES:
.SECONDARY:
OBJECTS := $(addsuffix .obj,$(SOURCES))
override OBJECTS := $(OBJECTS:$(SRC)/%.obj=$(OBJ)/%.obj)
OC1 := $(patsubst %.obj, %.oc1, $(filter %c.obj,$(OBJECTS)))
O1 := $(L166_CMD:%.lnp=%.o1)
all: $(TARGET) $(O1)
$(TARGET): $(OBJECTS)
#echo Linking $(TARGET)...
$(OBJ)/%.c.obj: $(SRC)/%.c
#echo Compiling $(<F) ...
# c.oc1 is a intermediate file
%.c.oc1: %.c.obj
#echo 1 - Optimize $<...
#touch $#
$(O1): $(OC1)
#touch $#
echo Linking O1
Result is, I modify a C file, the target will regenerate only the modified C file but the O1 pass will optimize all C files again like it was not done before (but it was).
How can I modify this behavior?

The reason is your last target, $(O1): $(OC1). That is each O1 depends on every OC1.
What is the actual value of $(O1)? Is it supposed to be a list or a single target?
I would try to replace this rule by a pattern (if it possible).

Related

How can I configure my makefile for debug and release builds to more complex projects with multiple executables?

In the post How can I configure my makefile for debug and release builds?, I find the answer https://stackoverflow.com/a/20830354/5922947 perfect!
But it seems work with only one final target.
I've tried unsuccessfully to extend this makefile to my projects where there are several executables as targets.
What changes should I make to this example so that I can have multiple final targets and with
make
or
make all
I can produce all targets (in debug or release modes), or
make test
for a specific target?
EDIT:
One of the attempts I made but which did not produce any results, but serves to better illustrate what is intended:
(...)
OBJS = $(SRCS:.c=.o)
EXE=$($(TARGET)_EXE)
sample1_EXE=sample1
sample2_EXE=sample2
sample1: TARGET=sample1
sample1: debug
sample2: TARGET=sample2
sample2: debug
all: prep debug sample1 sample2
The problem has two parts: specifying the mode, and coordinating the build.
(This will be tricky, so we will take it in stages.)
First the mode. Setting the value of mode in the command, and using debug as the default, is easy. We put this in the makefile:
mode = debug
And a command like make sailfish mode=release will override it.
Now to use the mode:
mode = debug
ifeq ($(mode),debug)
BUILDDIR := debug
CFLAGS += -g -O0 -DDEBUG
else
ifeq ($(mode),release)
BUILDDIR := release
CFLAGS += -O3 -DNDEBUG
else
$(error unknown mode: $(mode))
endif
endif
Note that I added the error statement to catch mistakes like make mode=test and make mode=releas. Such errors are much easier to catch and correct if they cause Make to abort on the spot. (Also note that the leading whitespace is not necessary, it has no effect on execution, but it makes the makefile easier to read.)
Now for the build. Suppose we execute make sailfish, so that the mode is debug. First notice that although we give sailfish as the target, we are not actually building sailfish, we are building debug/sailfish. This is important, so we make sailfish a PHONY target that requires the file we actually want:
.PHONY: sailfish
sailfish: $(BUILDDIR)/sailfish
The relevant objects are sailfish.o and seaThing.o. (You must compose this list yourself, it is almost impossible for Make to deduce it.) We could put the objects in the makefile as a distinct variable:
sailfish_OBJS := sailfish.o seaThing.o
but it will make things simpler later if we make this a target-specific variable:
$(BUILDDIR)/sailfish: $(addprefix $(BUILDDIR)/,sailfish.o seaThing.o)
$(CC) $(CFLAGS) -o $# $^
We still need a rule to build the object files, and we notice that the two pattern rules in the linked answer collapse into one:
$(BUILDDIR)/%.o: %.c
$(CC) -c $(CFLAGS) -o $# $<
This is enough to build debug/sailfish and release/sailfish. To add the executable catamaran, we add:
.PHONY: catamaran
catamaran: $(BUILDDIR)/catamaran
$(BUILDDIR)/catamaran: $(addprefix $(BUILDDIR)/,catamaran.o boatThing.o seaThing.o)
$(CC) $(CFLAGS) -o $# $^
But we notice some redundancy here, so we combine the PHONY declarations:
.PHONY: sailfish catamaran
Then we notice that if we put the names of the executables in a variable:
EXECUTABLES := sailfish catamaran
we can use it in the PHONY declaration:
.PHONY: $(EXECUTABLES)
and we can also combine the two PHONY rules into a static pattern rule:
$(EXECUTABLES): %: $(BUILDDIR)/%
and we can separate the prerequisite lines from the recipes and use one recipe for both:
$(BUILDDIR)/sailfish: $(addprefix $(BUILDDIR)/,sailfish.o seaThing.o)
$(BUILDDIR)/catamaran: $(addprefix $(BUILDDIR)/,catamaran.o boatThing.o seaThing.o)
$(addprefix $(BUILDDIR)/,$(EXECUTABLES)):
$(CC) $(CFLAGS) -o $# $^
Now, to add another executable like seagull, we must only add seagull to the EXECUTABLES := ... line, and write another line:
$(BUILDDIR)/seagull: $(addprefix $(BUILDDIR)/,seagull.o birdThing.o seaThing.o)
A few more refinements are possible, but this should be enough for now.

Makefile. Special chars

I have a question to this expression:
%.out: %.cpp Makefile
g++ $< -o $# -std=c++0x
What does it mean? I know, that it is defined target for *.o files but what does it mean %.cpp Makefile and $< and $#?
And:
What is differenece between:
all: $(patsubst %.cpp, %.o, $(wildcard *.cpp))
and:
all:
$(patsubst %.cpp, %.o, $(wildcard *.cpp))
The second doesn't works.
For the first part of your question:
%.out: %.cpp Makefile
g++ $< -o $# -std=c++0x
This is a pattern rule, and means: "for all files with a .cpp extension, compile (if needed) a corresponding .out file using the command g++ $< -o $# -std=c++0x
In this line, $< is the prerequisite (the .cpp file) , $# is the name of the target (the .out file). See here.
The rule also adds the makefile itself as a prerequisite, which means that all the files will be rebuild (even if they are already compiled) when you issue a make target command, if you make changes to the makefile.
For the second part of the question, your are mixing two things. A make rule is made of three parts:
target: dependencies
commands
The second one you show cannot work because there is no command. The line just produces a bunch of filenames, that your shell cannot understand.
The first one adds to the list of dependencies all the object files, whose names are deduced from all the .ccp files. But you are missing a command, so nothing should happen (unless you didn't give us the whole rule ?)
Edit: ouch, missed something, this rule actually should work fine, as make will evaluate all the prerequisite targets, thus call the pattern rule described above. I got confused by the fact that this structure is usually written like this:
targetname: $(OUTFILES)
#echo "- Done target $#"
with the variable defined above as:
OUTFILES = $(patsubst %.cpp, %.o, $(wildcard *.cpp))
or even as:
INFILES = $(wildcard *.cpp)
OUTFILES = $(patsubst %.cpp, %.o, $(INFILES))
I suggest you find a good make tutorial, or read the manual, you seem to have lots of concepts to learn...

Gnu make: is it possible to delay include directive to secondary expansion?

I need to delay inclusion of dependency fragments until second expansion time because the make file I'm editing is itself an include file and I will not have a list of source files to generate the includes until secondary expansion.
.SECONDEXPANSION:
AUTO_DEPENDENCY_FILES = $(patsubst %.cc, depend/%.d, $(CC_SRC_FILES))
# the following does the work because the include argument is not a rule
# prerequisite therefore no secondary expansion occurs
include $$(AUTO_DEPENDENCY_FILES)
depend:
-mkdir depend
all: autodepend
autodepend: depend autodepend_include
autodepend_include: $$(AUTO_DEPENDENCY_FILES)
#echo \"$^\"
$$(AUTO_DEPENDENCY_FILES): depend
depend/%.d: | %.cc
# generate .d files that do not exist
$(COMPILE.cc) -E $*.cc > /dev/null
%.o: %.cc
# update .d files that exist
$(COMPILE.cc) -o $# $<
Note the COMPILE.cc is a very long string that includes -MP -MMD -MFdepend/$*.d flags for auto dependency generation.
I don't know that there's a clean solution to this problem but with a bit of a hack you can get you what you want.
Given a main Makefile of:
$(info main one)
include depend.mk
$(info main two)
CC_SRC_FILES := $(addsuffix .c,a b c d e f)
$(info main three)
and a depend.mk of:
$(info depend one)
AUTO_DEPENDENCY_FILES = $(patsubst %.c,%.d,$(CC_SRC_FILES))
$(info AUTO_DEPENDENCY_FILES := $(AUTO_DEPENDENCY_FILES))
$(info MAKE_RESTARTS := $(MAKE_RESTARTS))
$(info CC_SRC_FILES := $(CC_SRC_FILES))
$(info depend two)
you get the following output when you run make:
main one
depend one
AUTO_DEPENDENCY_FILES :=
MAKE_RESTARTS :=
CC_SRC_FILES :=
depend two
main two
main three
make: `all' is up to date.
Which isn't surprising given the order of assignment and inclusion of files, etc.
Here's where the horrible hack comes in.
When make encounters an include directive that references a file that doesn't exist make sticks the file in a list of "missing include files" and continues parsing the makefile.
When it gets to the end of the makefile(s) it then tries to treat each entry in that list as a potential goal target1 and attempts to make the file.
Once the makefiles have been built make restarts itself and tries again.
You can use this to capture the value of CC_SRC_FILES in an built makefile that your makefile includes and have it be visible when you need it.
If we make depend.mk look like this:
$(info depend one)
include hack.mk
AUTO_DEPENDENCY_FILES = $(patsubst %.c,%.d,$(CC_SRC_FILES))
$(info AUTO_DEPENDENCY_FILES := $(AUTO_DEPENDENCY_FILES))
$(info MAKE_RESTARTS := $(MAKE_RESTARTS))
$(info CC_SRC_FILES := $(CC_SRC_FILES))
$(info depend two)
hack.mk: $(if $(MAKE_RESTARTS),,force)
#echo creating hack.mk
#echo 'CC_SRC_FILES := $(CC_SRC_FILES)' > '$#'
force: ;
Then our output from make becomes:
main one
depend one
depend.mk:3: hack.mk: No such file or directory
AUTO_DEPENDENCY_FILES :=
MAKE_RESTARTS :=
CC_SRC_FILES :=
depend two
main two
main three
creating hack.mk
main one
depend one
AUTO_DEPENDENCY_FILES := a.d b.d c.d d.d e.d f.d
MAKE_RESTARTS := 1
CC_SRC_FILES := a.c b.c c.c d.c e.c f.c
depend two
main two
main three
make: `all' is up to date.
Which gives us the value where we want it.
This isn't pretty but it does work.

GNU Make get the list of all files in a directory that were generated by previous rule

I am looking for Makefile macro to get list of all files in a directory that were generated as rule1 processing and using this list for rule2 processing.
Here's what I am trying to achieve :
Rule 1: Generate source .c files (using xml files) and place them in $(MYDIR) directory.
Rule 2: Get the list of all files in $(MYDIR) and create object files and place them in $(OBJDIR).
Problem is, I want to update list of files in Rule2 after Rule 1 has been processed, else list of files in $(MYDIR) will be empty.
all : rule_1 rule_2
rule1 : $(MYDIR)/generated_source1.c $(MYDIR)/generated_source2.c
$(MYDIR)/generated_source1.c:
xsltproc generator1.xml style_generator.xsl -o $(MYDIR)/generated_source_1.c
$(MYDIR)/generated_source2.c:
xsltproc generator2.xml style_generator.xsl -o $(MYDIR)generated_source_2.c
#Get list of all $(MYDIR).*c , create corresponding $(OBJDIR)/*.o list.
SOURCES := $(wildcard $(MYDIR)/*.c)
OBJECTS := $(notdir ${SOURCES})
GENERATED_OBJS := $(patsubst %.c,$(OBJDIR)/%.o,$(OBJECTS))
#This rule is compiling of all .c generated in rule1.
rule2 : $(GENERATED_OBJS)
ld -r -o $(OBJDIR)/generated_lib.o $(GENERATED_OBJS)
$(OBJDIR)/%.o: $(MYDIR)/%.c
gcc $(CFLAGS) -c -o $# $<
$(SOURCES) is shown empty, but actually it should contain generated_source1.c and generated_source2.c
I am not sure how .SECONDEXPANSION rule will work for my case.
You can't really (and don't really want to) play around with getting make to re-evaluate file existence during the running of the make process.
What you want to do is track the files from start to finish in make and then you have all your lists.
You can start at either direction but starting with the initial source tends to be easier.
So start with
MYDIR:=dir
OBJDIR:=obj
XML_SOURCES := $(wildcard $(MYDIR)/*.xml)
then translate from there to the generated source files
SOURCES := $(subst generator,generated_source,$(XML_SOURCES:.xml=.c))
and from there to the generated object files
GENERATED_OBJS := $(patsubst $(MYDIR)/%.c,$(OBJDIR)/%.o,$(SOURCES))
At which point you can define the default target
all: $(OBJDIR)/generated_lib.o
and then define the rules for each step
$(MYDIR)/%.c:
cat $^ > $#
$(OBJDIR)/%.o: $(MYDIR)/%.c
cat $^ > $#
$(OBJDIR)/generated_lib.o: $(GENERATED_OBJS)
ld -r -o $# $^
The $(MYDIR)/%.c rule needs a bit of extra magic to actually work correctly. You need to define the specific input/output pairs so that they are used correctly by that rule.
$(foreach xml,$(XML_SOURCES),$(eval $(subst generator,generated_source,$(xml:.xml=.c)): $(xml)))
This .xml to .c step would be easier if the input and output files shared a basename as you could then just use this and be done.
%.c: %.xml
cat $^ > $#

No rule to make target consoleio.c

In a recent issue, I've found that DJGPP can only accept the DOS command line character limit. To work around this limitation, I've decided to try to write a makefile to allow me to pass longer strings. In the process of hacking together a makefile and testing it, I've come across a strange error. The makefile is as follows:
AS := nasm
CC := gcc
LD := ld
TARGET := $(shell basename $(CURDIR))
BUILD := build
SOURCES := source
CFLAGS := -Wall -O -fstrength-reduce -fomit-frame-pointer -finline-functions \
-nostdinc -fno-builtin -I./include
ASFLAGS := -f aout
export OUTPUT := $(CURDIR)/$(TARGET)
CFILES := $(foreach dir,$(SOURCES),$(notdir $(wildcard $(dir)/*.c)))
SFILES := $(foreach dir,$(SOURCES),$(notdir $(wildcard $(dir)/*.s)))
SOBJS := $(SFILES:.s=.o)
COBJS := $(CFILES:.c=.o)
OBJS := $(SOBJS) $(COBJS)
build : $(TARGET).img
$(TARGET).img : $(TARGET).bin
concat.py
$(TARGET).bin : $(OBJS)
$(LD) -T link.ld -o $# $^
$(SOBJS) : %.o : %.asm
$(AS) $(ASFLAGS) $< -o $#
$(COBJS) : %.o : %.c
$(CC) -c $< $(CFLAGS) -o $#
When attempting to run it, I receive this error:
make: *** No rule to make target `consoleio.c', needed by `consoleio.o'. Stop.
What I don't understand is why it's trying to find a rule for .c files. From what I understand, if the file is there, it should just use it. How do I make make not need a rule for .c files?
What you are trying to do will not work without VPATH, and since you are still learning makefiles, I would avoid using VPATH.
The rule is looking for "consoleio.c", which if I understood your makefile correctly does not exist; what exists is "source/consoleio.c". You probably should change it to something like "$(SOURCES)/%.c" instead of "%c".
I didn't check your syntax for that rule, however. If it's incorrect, the builtin "%.o: %.c" rule will be used instead, which would have the same problem.
The way you are doing is not the usual way I've seen, however. The usual way is to:
Create an implicit rule "%.o: %.c" (or in your case "%.o: $(SOURCES)/%.c")
Explicit list the dependencies for each file: "foo.o: foo.c bar.h baz.h" (with no command, the implicit rule has the command)
Let's try a non-comment answer...
Possibility A:
Your macro for SFILES is looking for files ending in '.s'.
Your rule for compiling SOBJS is looking for files ending in '.asm'.
Possibility B:
Your rule for SOBJS and COBJS is in a notation I don't recognize.
According to the GNU Make manual, you can write implicit rules as:
%.o : %.c ; command
You seem to have a list of targets $(SOBJS) that depends on '%.o : %.asm'.
I'm not sure how make will interpret that.
Personally, I would not trust wild-cards in build rules. I'd much rather spend the time listing exactly which source files are needed to build the code. I don't often run into this problem as a result.
#CesarB seems to have nailed the issue, I'll just add a couple of observations.
I'd strongly recommend against using wildcards in build rules. The build rules should clearly define exactly what is being built, and not depend on what files happen to be in the directory.
I'd also recommend against using VPATH unless you are (1) building in a separate build directory, or (2) have your source files spread out over a large number of directories. If all your sources are in a single directory, using VPATH is only going to confuse.
The := assignment form is usually only used when the variable evaluation is known to take long time, such as when using a $(shell ...). Otherwise, "=" is preferrable.
Using "export" to propagate OUTDIR to concat.py (which I presume it is, since concat.py doesn't take any parameters) is a code smell. If possible, pass it as a parameter instead.

Resources