Gnu make: is it possible to delay include directive to secondary expansion? - include

I need to delay inclusion of dependency fragments until second expansion time because the make file I'm editing is itself an include file and I will not have a list of source files to generate the includes until secondary expansion.
.SECONDEXPANSION:
AUTO_DEPENDENCY_FILES = $(patsubst %.cc, depend/%.d, $(CC_SRC_FILES))
# the following does the work because the include argument is not a rule
# prerequisite therefore no secondary expansion occurs
include $$(AUTO_DEPENDENCY_FILES)
depend:
-mkdir depend
all: autodepend
autodepend: depend autodepend_include
autodepend_include: $$(AUTO_DEPENDENCY_FILES)
#echo \"$^\"
$$(AUTO_DEPENDENCY_FILES): depend
depend/%.d: | %.cc
# generate .d files that do not exist
$(COMPILE.cc) -E $*.cc > /dev/null
%.o: %.cc
# update .d files that exist
$(COMPILE.cc) -o $# $<
Note the COMPILE.cc is a very long string that includes -MP -MMD -MFdepend/$*.d flags for auto dependency generation.

I don't know that there's a clean solution to this problem but with a bit of a hack you can get you what you want.
Given a main Makefile of:
$(info main one)
include depend.mk
$(info main two)
CC_SRC_FILES := $(addsuffix .c,a b c d e f)
$(info main three)
and a depend.mk of:
$(info depend one)
AUTO_DEPENDENCY_FILES = $(patsubst %.c,%.d,$(CC_SRC_FILES))
$(info AUTO_DEPENDENCY_FILES := $(AUTO_DEPENDENCY_FILES))
$(info MAKE_RESTARTS := $(MAKE_RESTARTS))
$(info CC_SRC_FILES := $(CC_SRC_FILES))
$(info depend two)
you get the following output when you run make:
main one
depend one
AUTO_DEPENDENCY_FILES :=
MAKE_RESTARTS :=
CC_SRC_FILES :=
depend two
main two
main three
make: `all' is up to date.
Which isn't surprising given the order of assignment and inclusion of files, etc.
Here's where the horrible hack comes in.
When make encounters an include directive that references a file that doesn't exist make sticks the file in a list of "missing include files" and continues parsing the makefile.
When it gets to the end of the makefile(s) it then tries to treat each entry in that list as a potential goal target1 and attempts to make the file.
Once the makefiles have been built make restarts itself and tries again.
You can use this to capture the value of CC_SRC_FILES in an built makefile that your makefile includes and have it be visible when you need it.
If we make depend.mk look like this:
$(info depend one)
include hack.mk
AUTO_DEPENDENCY_FILES = $(patsubst %.c,%.d,$(CC_SRC_FILES))
$(info AUTO_DEPENDENCY_FILES := $(AUTO_DEPENDENCY_FILES))
$(info MAKE_RESTARTS := $(MAKE_RESTARTS))
$(info CC_SRC_FILES := $(CC_SRC_FILES))
$(info depend two)
hack.mk: $(if $(MAKE_RESTARTS),,force)
#echo creating hack.mk
#echo 'CC_SRC_FILES := $(CC_SRC_FILES)' > '$#'
force: ;
Then our output from make becomes:
main one
depend one
depend.mk:3: hack.mk: No such file or directory
AUTO_DEPENDENCY_FILES :=
MAKE_RESTARTS :=
CC_SRC_FILES :=
depend two
main two
main three
creating hack.mk
main one
depend one
AUTO_DEPENDENCY_FILES := a.d b.d c.d d.d e.d f.d
MAKE_RESTARTS := 1
CC_SRC_FILES := a.c b.c c.c d.c e.c f.c
depend two
main two
main three
make: `all' is up to date.
Which gives us the value where we want it.
This isn't pretty but it does work.

Related

GNU make: What does '*** missing separator. Stop' mean for a source code file?

I'm fairly new to Makefiles, and I am trying to create one for a C project I am building using gcc.
I am familiar with the error Makefile:<col>: *** missing separator. Stop. It has popped up before when I used spaces instead of tabs to precede rules in the Makefile. I just tried writing a Makefile for this particular project (being sure to use TAB character instead of spaces) and when I run the make command, I get a very nondescript error I do not understand how to fix: src/main.c:7: *** missing separator. Stop
My directory structure looks like this:
- projectfolder/
- Makefile
- bin/
- build/
- inc/
- src/
- main.c
- otherfolder/
- inc/
- common.h
- io.h
- src/
- io.c
main.c, which includes the main function, has the following imports:
#include <stdio.h>
#include <stdlib.h>
#include "common.h"
#include "io.h"
My goal is to make a bunch of .o and .d files from the files in both src/ and inc/ directories and put those into projectfolder/build/, so that they can then be used to make the final executable in projectfolder/bin/
Finally, here is the Makefile that is causing the issue:
# Compiler and linker
CC := gcc
# Target binary
TARGET := the-program
# File extensions
SRCEXT := c
# Directories
TARGETDIR := bin
BUILDDIR := build
SRCDIRS := src /home/myusername/otherfolder/src
INCDIRS := inc /home/myusername/otherfolder/inc
# gcc options
CCFLAGS := -Wall -Wextra -O3
#---------------------------------------------------------------------------------
#DO NOT EDIT BELOW THIS LINE
#---------------------------------------------------------------------------------
vpath %.$(SRCEXT) $(SRCDIRS)
vpath %.h $(INCDIRS)
CCFLAGS += $(addprefix -I,$(INCDIRS)) -MMD -MP
SRC := $(shell find $(SRCDIRS) -name *.$(SRCEXT))
OBJ := $(SRC:$(SRCDIRS)/%.$(SRCEXT)=$(BUILDDIR)/%.o)
DEP := $(OBJ:.o=.d)
TARGET := $(TARGETDIR)/$(TARGET)
# RULE: Default make
all: makedirs $(TARGET) clean
# RULE: Remake
remake: fullclean all
# RULE: Clean
clean:
rm -rf $(BUILDDIR)
# RULE: Full clean (objects and binaries)
fullclean: clean
rm -rf $(TARGETDIR)
# RULE: Make dirs
makedirs:
mkdir -p $(BUILDDIR)
mkdir -p $(TARGETDIR)
# RULE: Link
$(TARGET): $(OBJ)
$(CC) $(OBJ) -o $#
# RULE: Compile
$(OBJ): $(SRC)
$(CC) $(CCFLAGS) -c $< -o $#
# RULE: Non-file targets
.PHONY: all remake clean fullclean makedirs
# include dependencies
-include $(DEP)
What about this file would cause the described error? I am assuming that it is related to the Makefile itself rather than the C code actually contained in src/main.c since this does not appear to be a compilation error, but if I am wrong, let me know and I can change the post.
This error:
src/main.c:7: *** missing separator. Stop
is clearly being printed by make. Since we know the format of these messages is <makefile>:<linenumber>: <error>, we can see that make is trying to parse the file src/main.c as a makefile and obviously this cannot work.
How could make be doing such a thing? The obvious culprit is this line:
-include $(DEP)
If the variable DEP contains the file src/main.c, then make would try to include that as a makefile and parse it. So how is DEP set?
DEP := $(OBJ:.o=.d)
This changes all words in OBJ that end with .o to end with .d. Crucially, it should be noted that this makes no changes to words that do not end in .o. So, if OBJS contained src/main.c, this would pass it through unmodified to DEPS.
So how is OBJ set? Here are the relevant variables:
SRCDIRS := src /home/myusername/otherfolder/src
SRC := $(shell find $(SRCDIRS) -name *.$(SRCEXT))
OBJ := $(SRC:$(SRCDIRS)/%.$(SRCEXT)=$(BUILDDIR)/%.o)
Let's expand this: the SRC variable runs:
find src /home/myusername/otherfolder/src -name *.c
(you really should escape the *, either with backslash or quotes: it's very dangerous how you have this).
Now we'll say that SRC gets the value:
SRC := src/main.c /home/myusername/otherfolder/src/other.c
Now what does OBJ contain?
OBJ := $(SRC:src /home/myusername/otherfolder/src/%.$(SRCEXT)=build/%.o)
This clearly cannot work: you can't put multiple directories into SRCDIRS, but then treat it as if it contained only one word.
The problem is in the line
OBJ := $(SRC:$(SRCDIRS)/%.$(SRCEXT)=$(BUILDDIR)/%.o)
The pattern substitution works only where the pattern matches and leaves all other strings alone.
As a simple demonstration see this makefile (no rules, just variable assignments):
x := foo bar baz
y := $(x:ba%=gu%)
$(info $(x))
$(info $(y))
Output:
foo bar baz
foo gur guz
As you can see, foo still is there, unchanged although it didn't match the pattern ba%.
In your case you are passing an impossible to substitute pattern, the content of $(SRCDIRS), which is the two words: src /home/myusername/otherfolder/src to the substitution call, therefore main.c - which is either src/main.c or /home/myusername/otherfolder/src/main.c remains unchanged in your list, gets inadvertently flushed further into $(DEP) and is finally included as text with your last line -include $(DEP).
As a recommendation I want to point you to VPATH (see manual and here: https://www.cmcrossroads.com/article/basics-vpath-and-vpath), which is the better alternative to indicate the location of your sources to make.

Fortran Makefile Error and Questions

I'm trying to make my own Makefile for a Fortran code. It looks very alike to another Makefile, which can perfectly work. But unfortunately I can't find out what's wrong by myself. The configuration is described below:
I have a main folder containing sub-folders name src, lib, results, mod, obj. Makefile itself is also in the main folder. I have several fortran sources files in the src folder, say main.f90, subroutine_1.f90, subroutine_2.f90, module_1.f90 and module_2.f90. Subroutine_1 and subroutine_2 are included in module_1, module_2 in also used in module_1. In this case I only need main.f90 and module_*.f90 to be included in my Makefile(if that's not wrong). I also have a static library to be used.
# For WINDOWS
RM := del
PS := $(strip \)
#--------------------- DICTIONARY ---------------------
BUILD_DIR := .
SRC_DIR := $(BUILD_DIR)$(PS)src
LIB_DIR := -L$(BUILD_DIR)$(PS)lib
TAR_DIR := $(BUILD_DIR)$(PS)results
MOD_DIR := $(BUILD_DIR)$(PS)mod
OBJ_DIR := $(BUILD_DIR)$(PS)obj
#------------------ COMPILER AND FLAG ------------------
FC := gfortran
FCFLAGS := -g -c -fbacktrace -fno-align-commons -fbounds-check\
-J"$(MOD_DIR)"
FLFLAGS :=
#------------------- TARGET & SOURCE--------------------
TARGET := $(TAR_DIR)$(PS)main.exe
SRC := $(notdir $(wildcard $(SRC_DIR)$(PS)module_*.f90))\
main.f90
LIB := -lfftw3-3.lib
LDFLAGS := $(LIB_DIR)
OBJ := $(patsubst %.f90, $(OBJ_DIR)$(PS)%.o, $(notdir $(SRC)))
all: $(TARGET)
# Final linking
$(TARGET): $(OBJ)
$(FC) $(FLFLAGS) -o $# $(OBJ) $(LDFLAGS) $(LIB) #line a
$(OBJ_DIR)$(PS)%.o: $(SRC_DIR)$(PS)%.f90
$(FC) $(FCFLAGS) -o $# $< #line b
clean:
-$(RM) -f $(OBJ_DIR)$(PS)*.o
-$(RM) -f $(MOD_DIR)$(PS)*.mod
I got many errors from the make process but I guess they all come from the same place.
$ make
Makefile:line a: warning: overriding recipe for target '.'
Makefile:line b: warning: ignoring old recipe for target '.'
make: Circular . <- . dependency dropped.
make: *** No rule to make target 'src\%.f90', needed by '.'. Stop.
Except from the error, I also have several questions:
How to choose appropriate flags for compiling?
Since module_2 is used in module_1, do I need to compile them in order? In reality I have many modules so I don't think I'm able to specify that one by one...
When do I need to create a dependency .d file? I don't quite know how it works.
Do I need to compile .mod out of .o? For example add a line
%.mod: %.o
Thanks a lot.

substituting folder path in $(SOURCES) of a makefile

I am looking for a way to substitute the folder on a list of source files variable within makefile.
Is there something that works here?
I start off by finding my source files
program_C_SRCS := $(wildcard $(SRCDIR)/*.c)
program_CXX_SRCS := $(wildcard $(SRCDIR)/*.cpp)
the results (if I understand GNU makefiles correctly) look typically like
src/main.cpp
src/sensor.cpp
then I build by object files
program_C_OBJS := ${program_C_SRCS:.c=.o}
program_CXX_OBJS := ${program_CXX_SRCS:.cpp=.opp}
this replaces the extensions of my source files as per expected.
Finally, I want to replace "src/" with "obj/"
program_C_OBJPATH := ${subst $(SRCDIR) , $(OBJDIR) , $(program_C_OBJS)}
program_CXX_OBJPATH := ${subst $(SRCDIR) , $(OBJDIR) , $(program_CXX_OBJS)}
However, this does not work.
I have gone through the GNU makefile website to no avail. This solution Makefile to put object files from source files different directories into a single, separate directory? comes close but the objects directory must be explicitly included everywhere and the sources directory does not include source path information.
In my makefile the list of source files include the path and I would prefer that the list of object files include the corresponding object directory, too.
The rest of the makefile tries also to use variables
linking stage
$(program_NAME): $(program_OBJS)
$(CXX) $(CFLAGS) $(CXXFLAGS) $(LDFLAGS) $(program_OBJS) -o "$(program_NAME)"
compile stage
%.opp : %.cpp | mkdirobjdir
$(CXX) $(CFLAGS) $(CXXFLAGS) $(CPPFLAGS) -c -o "$#" "$<"
and finally the rules
main_enose.opp : main_enose.cpp core_enose.cpp core_enose.h
$(OBJDIR)/core_enose.opp : core_enose.cpp core_enose.h
$(OBJDIR)/core_enose.h :
Your problem here is the spaces around the commas in the $(subst) calls. make isn't ignoring them the way you expect it might. It is seeing them as literal values in the string to find, the string to replace and the to do the replacement in.
Remove them.
program_C_OBJPATH := ${subst $(SRCDIR),$(OBJDIR),$(program_C_OBJS)}
program_CXX_OBJPATH := ${subst $(SRCDIR),$(OBJDIR),$(program_CXX_OBJS)}
That said you probably either want to use $(patsubst) to limit where the replacement happens:
program_C_OBJPATH := $(patsubst $(SRCDIR)/%,$(OBJDIR)/%,$(program_C_OBJS))
program_CXX_OBJPATH := $(patsubst $(SRCDIR)/%,$(OBJDIR)/%,$(program_CXX_OBJS))
or you want to use $(notdir) and $(addprefix) to handle stripping all the directory information and adding it back:
program_C_OBJPATH := $(addprefix $(OBJDIR)/,$(notdir $(program_C_OBJS)))
program_CXX_OBJPATH := $(addprefix $(OBJDIR)/,$(notdir $(program_CXX_OBJS)))

Get include tree of GNU makefiles

In an existing GNU Make build system, I'd like to see a tree of the makefile includes. How may I do this? Like Do you know tool building tree of include files in project\file? but for GNU Make rather than C and C++.
A related but slightly different capability I'd be interested in: being able to put $(info ...) statements in a makefile and have it print out a backtrace of includes for that info statement.
If you just want a list of included makefiles, a $(info Included files: ${MAKEFILE_LIST}) at the bottom of the main makefile fits the bill.
However, if you do want a tree, then you'll have to replace all include <file> with $(call include,<file>). The include function would be something like:
space :=
space +=
${space} := ${space}
stack := $(firstword ${MAKEFILE_LIST})
define includefunc
stack := $(subst ${ },|,$(strip ${MAKEFILE_LIST} $1)) ${stack}
$(info INCLUDES $(firstword ${stack}))
include $1
stack := $(wordlist 2,$(words ${stack}),${stack})
MAKEFILE_LIST := $(subst |, ,${stack})
endef
include = $(eval $(value includefunc))
$(call include,1.mak)
$(call include,_1/1.mak)
$(call include,folder/Makefile)
.PHONY: all
all: ; echo $# success

Make 3 passes for a target

I'll explain myself, here is my scenario:
Compile my target
Do a first optimization based on the first compilation
Do a second optimization based on the first optimization
Do a third optimization based on the second optimization
So far I tried the following:
.SUFFIXES:
.SECONDARY:
OBJECTS := $(addsuffix .obj,$(SOURCES))
override OBJECTS := $(OBJECTS:$(SRC)/%.obj=$(OBJ)/%.obj)
OC1 := $(patsubst %.obj, %.oc1, $(filter %c.obj,$(OBJECTS)))
O1 := $(L166_CMD:%.lnp=%.o1)
all: $(TARGET) $(O1)
$(TARGET): $(OBJECTS)
#echo Linking $(TARGET)...
$(OBJ)/%.c.obj: $(SRC)/%.c
#echo Compiling $(<F) ...
# c.oc1 is a intermediate file
%.c.oc1: %.c.obj
#echo 1 - Optimize $<...
#touch $#
$(O1): $(OC1)
#touch $#
echo Linking O1
Result is, I modify a C file, the target will regenerate only the modified C file but the O1 pass will optimize all C files again like it was not done before (but it was).
How can I modify this behavior?
The reason is your last target, $(O1): $(OC1). That is each O1 depends on every OC1.
What is the actual value of $(O1)? Is it supposed to be a list or a single target?
I would try to replace this rule by a pattern (if it possible).

Resources