First ever attempt at writing a small Makefile, but hitting a problem - how do I stop it executing make clean after every build?
TASS64=64tass
EXOMIZER=exomizer
EXOMIZERFLAGS=sfx basic -n
VICE=/Applications/VICE/x64.app/Contents/MacOS/x64
VICEFLAGS=-sidenginemodel 1803 -keybuf "\88"
SOURCES=$(wildcard *.asm)
OBJECTS=$(SOURCES:.asm=.prg)
.PRECIOUS=Calvin.prg
all: $(TARGETS)
%.prg: %.asm
$(TASS64) -C -a -o $# -i $<
%: %.prg
$(VICE) $(VICEFLAGS) $<
.PHONY: clean
clean:
rm $(OBJECTS)
I guess, it does not execute 'make clean'. However, what may happen is that intermediate (secondary) results are deleted. GNU Make does that by default. To prevent make from doing so, mention those intermediate results X1, X2, ... in
.SECONDARY: X1 X2 ...
Or, in order to leave any secondary result in place, simply type:
.SECONDARY:
without any specific target.
So it turns out the default Make behaviour is to delete the output if there's a build problem. While it builds correctly in this case, my makefile then launches the PRG file in VICE (c64 emulator). It runs correctly, so I quit the emulator.
The quit action returns an exit code that Make treats as an unsuccessful build and thus deletes the output PRG
This is based on this thread - Why does GNU make delete a file - and subsequent testing by removing the target that launches VICE.
Related
all: ./data/for_analysis.csv ./data/tables/*.docx
./data/for_analysis.csv : ./src/convert-xls-to-gold-standard.py ./data/ED-TRAUMA-DELTA-STUDY_3_2019_total.xlsx
python3 $< --rawDataPath $(word 2,$^) --fieldCodesPath ./data/excel_field_codes.json --processedDataPath ./data/for_analysis.csv --logDir ./logs
./data/tables/%.docx : ./src/make-%.py ./data/for_analysis.csv
python3 $< --fieldCodesPath ./data/excel_field_codes.json --processedDataPath ./data/for_analysis.csv --logDir ./logs --tablesDir ./data/tables
When I update ./src/make-table-2.py, the second target isn't updated. This behavior doesn't depend on whether ./data/table/table-2.docx exists or not.
When I run make or make all even after updating the py file, I get the message make: Nothing to be done for 'all'.
It's not exactly clear from your question what the state of your targets is before you run make. But:
all: ./data/for_analysis.csv ./data/tables/*.docx
this can't really work, in general. This tells make, "go find all the files that exist with the filename matching the wildcard ./data/tables/*.docx". E.g., that's the same thing you'd get if you run ls ./data/tables/*.docx before you started make.
But of course, if you haven't built anything yet then there are no files matching that pattern, because that's what you're asking make to build. So this expands to nothing and make won't do anything with them.
You have to list the targets that you want to build explicitly, or else convert them from the source files you want them to be built from, so you can tell make what it should be building.
For example, maybe:
all: ./data/for_analysis.csv $(patsubst ./src/make-%.py,./data/tables/%.docx,$(wildcard ./src/make-*.py))
Most makefiles have a structure such as this:
.PHONY: prebuild
all: $(TARGET)
prebuild: Makefile
$(shell DEPDIR=$(DEPDIR) mkdir -p $(DEPDIR)/../common >/dev/null)
# do other work related to preparing for the object files to be built such as run a script to modify a header file included by $(TARGET).c
$(TARGET): $(TARGET).c prebuild
$(CC) $(CFLAGS) -o $(TARGET) $(TARGET).c
The implicit rules know how to build $(TARGET).o from $(TARGET).c, and doesn't do any work if $(TARGET).o is already newer than $(TARGET).c. This happens when make is run multiple times without changing the source file.
However, building the all target above will seemingly always rerun the $(CC) $(CFLAGS) -o $(TARGET) $(TARGET).c link to link the application and create the application binary. This happens even if that binary already exists and doesn't need to be recreated. In some larger projects, this process can take a long time (tens of seconds), which is sometimes not desirable.
Edit #1: The issue has to do something with an extra phony target that I do want to run ONCE before the object files are built. In my case, I'm running a script which takes Makefile variables and possibly updates a header file that is included in the C file. But, if the Makefile doesn't change, the prebuild target isn't run. However, the $(TARGET) target is still run even if prebuild doesn't do anything (for instance, because Makefile wasn't changed). FYI: because of the structure of my build system, I have prebuild run always because my build system is used for a variety of applications that can dynamically redefine prebuild.
How can this Makefile be restructured to not do the linking again when not necessary?
Edit #2:
Here's a simplified example that seems to illustrate my issue:
Before running, create a new directory and touch a b
.PHONY: prebuild main all
all: main
prebuild: a Makefile
#echo prebuild ran
main: prebuild
#echo main ran
When I run, I get this output:
prebuild ran
main ran
This is what happens no matter how many times I run make, even though the prerequisite a nor Makefile doesn't change. What I expect to happen is prebuild doesn't run (because a and Makefile don't change) and main also doesn't run because prebuild doesn't run. Clearly, I'm misunderstanding something.
The problem is that extra dependency triggering your rebuild.
Try this:
.PHONY: all
OUTPUTDIR=common/
TARGET=finalexe
all: $(OUTPUTDIR)/$(TARGET)
$(OUTPUTDIR)/$(TARGET): $(TARGET).c | $(OUTPUTDIR)
$(CC) $(CFLAGS) -o $# $(TARGET).c
$(OUTPUTDIR):
mkdir -p $#
In this above example, 'finalexe' will be created if A. it doesn't yet exist or B. if finalexe.c was modified. The timestamp on the OUTPUTDIR is not checked.
I am looking at a pre-existing, working, complex makefile for a project which will both build and deploy the code on multiple OS's.
I'm looking at some separate IDE support (Visual Studio) for the build process (i.e. half the make will already be done), so need to insert a phony target for the deploy action such that the old flow (make all) still works, including the deloy step, but that make deploy will just do the final deployment step for those using the IDE.
Not being familiar with make, I'm having difficulty seeing how/if make allows such an entry point and if so how to implement it.
The current code has:
$(BUILT_INS): git$X
$(QUIET_BUILT_IN)$(RM) $# && \
ln $< $# 2>/dev/null || \
ln -s $< $# 2>/dev/null || \
cp $< $#
whose actions are the deployment step.
So conceptually I think I need
.PHONY: deploy
$(BUILT_INS): git$X
deploy
deploy:
$(QUIET_BUILT_IN)$(RM) $# && \
ln $< $# 2>/dev/null || \
ln -s $< $# 2>/dev/null || \
cp $< $#
which is clearly not right, because the phony target can't be an action.
In summary; How to create an entry point into a makefile to do the rule's actions? (a critical desire is to avoid duplicating the action code)
Your "conceptually" solution has many more problems than using a target as a recipe (which you're right, won't work, but you could fix by using $(MAKE) deploy as the recipe to invoke a recursive make); the other thing is that this:
$(BUILT_INS):
... using $# ...
means run that recipe one time for each word in the BUILT_INS variable, and each time the automatic variable $# will be assigned to that word (the target).
Your replacement:
deploy:
... using $# ...
does an entirely different thing: it runs the recipe one time, with the value of $# set to deploy. Not going to work.
The simple answer to your question is that you just declare a new target deploy that lists the targets you want to run as prerequisites:
.PHONY: deploy
deploy: $(BUILT_INS)
Now when you run make deploy it will try to build the BUILT_INS targets, and run the install rule for each one.
However, my suspicion is that this will be a problem for you, depending on what the git$X prerequisite is... it might cause a big part of the rest of your makefile to run as well. But, you don't give any information about that so I can't say.
ETA Sure enough, git$X is a problem. So, it appears you want VS to generate your git$X file (which will be git.exe presumably). Then you want to run make deploy to copy it. The trick here is to keep make from rebuilding git$X when you use the deploy target. You can do something like this; replace the rule that builds git$X with:
.PHONY: deploy
deploy: $(BUILT_INS)
ifeq (,$(filter deploy,$(MAKECMDGOALS)))
git$X: git.o GIT-LDFLAGS $(BUILTIN_OBJS) $(GITLIBS)
$(QUIET_LINK)$(CC) $(ALL_CFLAGS) -o $# $(ALL_LDFLAGS) git.o \
$(BUILTIN_OBJS) $(LIBS)
endif
This creates a new target deploy which depends on the BUILT_INS targets, as I show above. Then, I enclose the rule to create git$X inside a test so that if you run make deploy that rule is not defined. Now make doesn't know how to build git$X at all, so if it doesn't exist then make deploy will fail, but if it does exist then make will copy it, without trying to rebuild it.
Is there a way how to ask gmake to never run two targets from a set in parallel?
I don't want to use .NOTPARALLEL, because it forces the whole Makefile to be run sequentially, not just the required part.
I could also add dependencies so that one depends on another, but then (apart from being ugly) I'd need to build all of them in order to build the last one, which isn't necessary.
The reason why I need this is that (only a) part of my Makefile invokes ghc --make, which takes care of its dependencies itself. And it's not possible to run it in parallel on two different targets, because if the two targets share some dependency, they can rewrite each other's .o file. (But ghc is fine with being called sequentially.)
Update: To give a specific example. Let's say I need to compile two programs in my Makefile:
prog1 depends on prog1.hs and mylib.hs;
prog2 depends on prog2.hs and mylib.hs.
Now if I invoke ghc --make prog1.hs, it checks its dependencies, compiles both prog1.hs and mylib.hs into their respective object and interface files, and links prog1. The same happens when I call ghc --make prog2.hs. So if they the two commands get to run in parallel, one will overwrite mylib.o of the other one, causing it to fail badly.
However, I need that neither prog1 depends on prog2 nor vice versa, because they should be compilable separately. (In reality they're very large with a lot of modules and requiring to compile them all slows development considerably.)
Hmmm, could do with a bit more information, so this is just a stab in the dark.
Make doesn't really support this, but you can sequential-ise two targets in a couple of ways. First off, a real use for recursive make:
targ1: ; recipe1...
targ2: ; recipe2...
both-targets:
${MAKE} targ1
${MAKE} targ2
So here you can just make -j both-targets and all is fine. Fragile though, because make -j targ1 targ2 still runs in parallel. You can use dependencies instead:
targ1: ; recipe1...
targ2: | targ1 ; recipe2...
Now make -j targ1 targ2 does what you want. Disadvantage? make targ2 will always try to build targ1 first (sequentially). This may (or may not) be a show-stopper for you.
EDIT
Another unsatisfactory strategy is to explicitly look at $MAKECMDGOALS, which lists the targets you specified on the command-line. Still a fragile solution as it is broken when someone uses dependencies inside the Makefile to get things built (a not unreasonable action).
Let's say your makefile contains two independent targets targ1 and targ2. Basically they remain independent until someone specifies on the command-line that they must both be built. In this particular case you break this independence. Consider this snippet:
$(and $(filter targ1,${MAKECMDGOALS)),$(filter targ2,${MAKECMDGOALS}),$(eval targ1: | targ2))
Urk! What's going on here?
Make evaluates the $(and)
It first has to expand $(filter targ1,${MAKECMDGOALS})
Iff targ1 was specified, it goes on to expand $(filter targ2,${MAKECMDGOALS})
Iff targ2 was also specified, it goes on to expand the $(eval), forcing the serialization of targ1 and targ2.
Note that the $(eval) expands to nothing (all its work was done as a side-effect), so that the original $(and) always expands to nothing at all, causing no syntax error.
Ugh!
[Now that I've typed that out, the considerably simpler prog2: | $(filter prog1,${MAKECMDGOALS})
occurs to me. Oh well.]
YMMV and all that.
I'm not familiar with ghc, but the correct solution would be to get the two runs of ghc to use different build folders, then they can happily run in parallel.
Since I got stuck at the same problem, here is another pointer in the direction that make does not provide the functionality you describe:
From the GNU Make Manual:
It is important to be careful when using parallel execution (the -j switch; see Parallel Execution) and archives. If multiple ar commands run at the same time on the same archive file, they will not know about each other and can corrupt the file.
Possibly a future version of make will provide a mechanism to circumvent this problem by serializing all recipes that operate on the same archive file. But for the time being, you must either write your makefiles to avoid this problem in some other way, or not use -j.
What you are attempting, and what I was attempting (using make to insert data in a SQLite3 database) suffers from the exact same problem.
I needed to separate the compilation from other steps (cleaning, building dirs and linking), as I wanted to run the compilation with more core processes and the -j flag.
I managed to solve this, with different makefiles including and calling each other. Only the "compile" make file is running in parallel with all the cores, the rest of the process is syncronous.
I divided my makefile in 3 separate scripts:
settings.mk: contains all the variables and flag definitions
makefile: has all the targets except the compilation one (It has .NOTPARALLEL directive). It calls compile.mk with -j flag
compile.mk: contains only the compile operation (without .NOTPARALLEL)
In settings.mk I have:
CC = g++
DB = gdb
RM = rm
MD = mkdir
CP = cp
MAKE = mingw32-make
BUILD = Debug
DEBUG = true
[... all other variables and flags needed, directories etc ...]
In makefile I have Link and compilation target as these:
include .makefiles/settings.mk
[... OTHER TARGETS (clean, directories etc)]
compilation:
#echo Compilation
#$(MAKE) -f .makefiles/compile.mk --silent -j 8 -Oline
#Link
$(TARGET): compilation
#echo -e Linking $(TARGET)
#$(CC) $(LNKFLAGS) -o $(TARGETDIR)/$(TARGET) $(OBJECTS) $(LIBDIRS) $(LIB)
#Non-File Targets
.PHONY: all prebuild release rebuild clean resources directories run debug
.NOTPARALLEL: all
# include dependency files (*.d) if available
-include $(DEPENDS)
And this is my compile.mk:
include .makefiles/settings.mk
#Defauilt
all: $(OBJECTS)
#Compile
$(BUILDDIR)/%.$(OBJEXT): $(SRCDIR)/%.$(SRCEXT)
#echo -e Compiling: $<
#$(MD) -p $(dir $#)
#$(CC) $(COMFLAGS) $(INCDIRS) -c $< -o $#
#Non-File Targets
.PHONY: all
# include dependency files (*.d) if available
-include $(DEPENDS)
Until now, it's working.
Note that I'm calling compile.mk with -j flag AND -Oline so that parallel processing doesn't mess up with the output.
Any syntax color can be setted in the makefile main script, since the -O flag invalidates escape color codes.
I hope it can help.
I had a similar problem so ended up solving it on the command line, like so:
make target1; make target2
to force it to do the targets sequentially.
Let's say you have a Makefile with two pseudo-targets, 'all' and 'debug'. The 'debug' target is meant to build the same project as 'all', except with some different compile switches (like -ggdb, for example). Since the targets use different compile switches, you obviously need to rebuild the entire project if you switch between the two. But GNUmake doesn't naturally recognize this.
So if you type make all you'll get
Building ...
...
Then if you type make debug, you get
make: Nothing to be done for `debug'.
So my question is: how do you implement a clean solution in the Makefile to notice that the last build used a different pseudo-target, or different compile switches, than the one you want currently? If they are different, the Makefile would rebuild everything.
Put the build products into different directory trees (whilst keeping one copy of the source of course). That way you are always just a short compile from an up-to-date build, be it debug or release (or even others). No possibility of confusion either.
EDIT
Sketch of the above.
src := 1.c 2.c 3.c
bare-objs := ${src:%.c=%.o}
release-objs := ${bare-objs:%=Release/%}
debug-objs := ${bare-objs:%=Debug/%}
Release/prog: ${release-objs}
Debug/prog: ${debug-objs}
${release-objs}: Release/%.o: %.c # You gotta lurve static pattern rules
gcc -c $< -o $#
${debug-objs}: Debug/%.o: %.c
gcc -c $< -o $#
Release/prog Debug/prog:
gcc $^ -o $#
.PHONY: all
all: Release/prog ; echo $# Success
.PHONY: debug
debug: Debug/prog ; echo $# Success
(Disclaimer: not tested, nor even run through make.)
There you go. It's even -j safe so you can do make -j5 all debug. There is a lot of obvious boiler plate just crying out for tidying up.
Keeping variant sets of object files (as in bobbogo's solution) is probably the best way, but if for some reason you don't want to do that, you can use empty files as markers, to indicate which way you last built the executable:
%-marker:
#rm -f $(OBJECTS) *-marker
#touch $#
debug: GCCFLAGS += -ggdb
debug: SOMEOTHERFLAG = WHATEVER
all debug: % : %-marker
#echo making $#
#$(MAKE) -S GCCFLAGS='$(GCCFLAGS)' SOMEOTHERFLAG='$(SOMEOTHERFLAG)' main
There are other variants on this idea; you could have a small file containing the flag settings, which the makefile would build and include. That would be clever, but not really any cleaner than this.
The only clean solution is to incorporate the difference into the target names.
E.g. you can define a variable $(DEBUG) and consistently use it in all targets that depend on the compile step.