Make recompiles non-updated files - makefile

I was experimenting with GNU make. Suppose, I have 3 C files with the following structure:
hellomake.c
|
|---------------------
| |
V V
hellofunc.c hellomake.h
makefile:
hellomake: hellomake.c hellofunc.c
gcc -o hellomake hellomake.c hellofunc.c -I.
When I type make for the 1-st time, it creates a program hellomake. When I run it for the 2-nd time, it prints:
make: `hellomake' is up to date.
Everything is working correctly.
I tried to use make for compiling LaTeX files. Suppose, I have 2 TeX files:
1.tex
|
V
1_data.tex
1_data is included into 1.tex internally.
Makefile:
COMMAND = pdflatex
all: 1.tex 1_data.tex
$(COMMAND) 1.tex
But it recompiles 1.tex every time I type make. Even if none of the files were modified.
What's wrong?

makesimply looks at whether or not all exists. Since it doesn't, it attempts to create it (but doesn't even notice that your commands do not create the file!)
Assuming pdflatex really creates a file somewhere, use the file name as the target name. Then it will be recreated only if it is older than the dependencies.
To recapitulate, a Makefile declares a mapping of target files, their dependencies, and how to create them from the dependencies. When a target is missing or out of date with respect to its dependencies, Make runs the commands for recreating it.
(My first attempt at articulating this answer mentioned .PHONY: targets as an aside, but that's not really useful in this context; if you declared .PHONY: all it would run the recipe even if a file named all existed, so that's the opposite behavior of what you are looking for.)

Related

Make file Doesn't detect changes in source files

I am very much new to make files , I am facing very basic problem , My Makefile doesn't detect changes I made to source files . The problem is , when I first time generate consoleapp binary from my source file i get expected output . But When I change source file again and when I run make again it says
make: 'consoleapp' is up to date , So what changes I have to give to make file so that it detects my changes
Below is my Makefile :
consoleapp:
g++ consoleapp.cpp -o consoleapp
clean:
rm -rf *.o consoleapp
This is my Source File :
#include <iostream>
using namespace std;
int main()
{
cout<<"I am ok \n"; // I am changing this line again after giving make
return 0;
}
make relies on the makefile author to tell it what each target's prerequisites are -- that is, which other targets or files affect the construction of the target in question, so that if they are newer or themselves out of date then the target is out of date and should be rebuilt. As your other answer already indicates, you do not designate any prerequisites for your targets, so make considers them out of date if and only if they don't exist at all.
That's actually problematic for both targets, albeit in different ways. For target consoleapp, which represents an actual file that you want to build, the failure to specify any prerequisites yields the problem you ask about: make does not recognize that changes to the source file necessitate a rebuild. The easiest way to fix that would be to just add the source file name to the recipe's header line, after the colon:
consoleapp: consoleapp.cpp
g++ consoleapp.cpp -o consoleapp
Generally speaking, however, it is wise to minimize duplication in your makefile code, and to that end you can use some of make's automatic variables to avoid repeating target and prerequisite names in your rule's recipe. In particular, I recommend always using $# to designate the rule's target inside its recipe:
consoleapp: consoleapp.cpp
g++ consoleapp.cpp -o $#
It's a bit more situational for prerequisites. In this case, all the prerequisites are source files to be compiled, and furthermore there is only one. If you are willing to rely on GNU extensions then in the recipe you might represent the sources via either $< (which represents the first prerequisite), or as $^ (which represents the whole prerequisite list, with any duplicates removed). For example,
consoleapp: consoleapp.cpp
g++ $^ -o $#
If you are not using GNU make, however, or if you want to support other people who don't, then you are stuck with some repetition here. You can still save yourself some effort, especially in the event of a change to the source list, by creating a make variable for the sources and duplicating that instead of duplicating the source list itself:
consoleapp_SRCS = consoleapp.cpp
consoleapp: $(consoleapp_SRCS)
g++ $(consoleapp_SRCS) -o $#
I mentioned earlier that there are problems with both of your rules. But what could be wrong with the clean rule, you may ask? It does not create a file named "clean", so its recipe will be run every time you execute make clean, just as you want, right? Not necessarily. Although that rule does not create a file named "clean", if such a file is created by some other means then suddenly your clean rule will stop working, as that file will be found already up to date with respect to its (empty) list of prerequisites.
POSIX standard make has no solution for that, but GNU make provides for it with the special target .PHONY. With GNU make, any targets designated as prerequisites of .PHONY are always considered out of date, and the filesystem is not even checked for them. This is exactly to support targets such as clean, which are used to designate actions to perform that do not produce persistent artifacts on the file system. Although that's a GNU extension, it is portable in the sense that it uses standard make syntax and the target's form is reserved for extensions, so a make that does not support .PHONY in the GNU sense is likely either to just ignore it or to treat it as an ordinary rule:
.PHONY: clean
clean:
rm -rf *.o consoleapp
because your target has no dependence. Please use this codes that rely to all cpp file in current dir to update binary.
SRCS=consoleapp.cpp
consoleapp: $(SRCS)
g++ $< -o $#

GNU make in newly created subdirectory

First - I know there are a lot of discussions similar to this, but I've spent hours without them working for me.
My makefile first creates a directory named by the current date and time. I then have the makefile append to a header file a line which creates a string with this directory name. For this reason, I first need to copy all the source files (including the header) into the newly created subdirectory, so that I can preserve the original header and only modify the header (in the subdirectory) which will be used for compilation. I would then like to build in that new directory.
My trouble is getting make to properly build the .o files in the new subdirectory. The solution I've found is to have
$(NOW)%.o: $(NOW)%.cpp
$(CC) -c $(FLAGS) $<
where $(NOW)$ is the subdirectory name. The issue is that my $(FLAGS) seem to be ignored: the output is, roughly
g++ -c -o <.o file> <.cpp file>
(Yes, there is actually extra introduced space between g++ and -c.) Whereas building in the top level directory a la
%.o: %.cpp
$(CC) -c $(FLAGS) $<
correctly outputs
g++ -c <my flags> -o <.o file> <.cpp file>
To summarize, I am unable to compile normally by transferring the source files to a newly-created subdirectory and building the .o files in that directory. TYIA.
Ad John points out, there's no way to definitively diagnose your problem with the tiny bit of makefile you provided, because the error is not in the code you provided, it's in some other part of your makefile. You need to provide a SSCCE ideally, but if not that then at least we need to see how the NOW variable is set and the linker rule so we know what make is trying to build.
I should also point out that by convention you should not use CC to hold the C++ compiler; the CC variable holds the C compiler. Use CXX for the C++ compiler and CXXFLAGS for the C++ compiler flags.
One possibility is that you are assigning the NOW variable using a recursive assignment so that the timestamp is recreated every time the variable is evaluated; it could be that the timestamp changes over the lifetime of the makefile.
The other very common problem is that you created the pattern rule, but make is not using it because the targets make wants to build don't match the pattern.
So for example, if your link line looks like this:
SRCS = foo.cpp
OBJS = $(SRC:.cpp=.o)
myprog: $(OBJS)
$(CXX) ...
$(NOW)%.o : $(NOW)%.cpp
$(CXX) ...
then your pattern will not be matched because make is trying to build the file foo.o and your rule tells it how to build $(NOW)foo.o which are not the same thing.

How to force a certain groups of targets to be always run sequentially?

Is there a way how to ask gmake to never run two targets from a set in parallel?
I don't want to use .NOTPARALLEL, because it forces the whole Makefile to be run sequentially, not just the required part.
I could also add dependencies so that one depends on another, but then (apart from being ugly) I'd need to build all of them in order to build the last one, which isn't necessary.
The reason why I need this is that (only a) part of my Makefile invokes ghc --make, which takes care of its dependencies itself. And it's not possible to run it in parallel on two different targets, because if the two targets share some dependency, they can rewrite each other's .o file. (But ghc is fine with being called sequentially.)
Update: To give a specific example. Let's say I need to compile two programs in my Makefile:
prog1 depends on prog1.hs and mylib.hs;
prog2 depends on prog2.hs and mylib.hs.
Now if I invoke ghc --make prog1.hs, it checks its dependencies, compiles both prog1.hs and mylib.hs into their respective object and interface files, and links prog1. The same happens when I call ghc --make prog2.hs. So if they the two commands get to run in parallel, one will overwrite mylib.o of the other one, causing it to fail badly.
However, I need that neither prog1 depends on prog2 nor vice versa, because they should be compilable separately. (In reality they're very large with a lot of modules and requiring to compile them all slows development considerably.)
Hmmm, could do with a bit more information, so this is just a stab in the dark.
Make doesn't really support this, but you can sequential-ise two targets in a couple of ways. First off, a real use for recursive make:
targ1: ; recipe1...
targ2: ; recipe2...
both-targets:
${MAKE} targ1
${MAKE} targ2
So here you can just make -j both-targets and all is fine. Fragile though, because make -j targ1 targ2 still runs in parallel. You can use dependencies instead:
targ1: ; recipe1...
targ2: | targ1 ; recipe2...
Now make -j targ1 targ2 does what you want. Disadvantage? make targ2 will always try to build targ1 first (sequentially). This may (or may not) be a show-stopper for you.
EDIT
Another unsatisfactory strategy is to explicitly look at $MAKECMDGOALS, which lists the targets you specified on the command-line. Still a fragile solution as it is broken when someone uses dependencies inside the Makefile to get things built (a not unreasonable action).
Let's say your makefile contains two independent targets targ1 and targ2. Basically they remain independent until someone specifies on the command-line that they must both be built. In this particular case you break this independence. Consider this snippet:
$(and $(filter targ1,${MAKECMDGOALS)),$(filter targ2,${MAKECMDGOALS}),$(eval targ1: | targ2))
Urk! What's going on here?
Make evaluates the $(and)
It first has to expand $(filter targ1,${MAKECMDGOALS})
Iff targ1 was specified, it goes on to expand $(filter targ2,${MAKECMDGOALS})
Iff targ2 was also specified, it goes on to expand the $(eval), forcing the serialization of targ1 and targ2.
Note that the $(eval) expands to nothing (all its work was done as a side-effect), so that the original $(and) always expands to nothing at all, causing no syntax error.
Ugh!
[Now that I've typed that out, the considerably simpler prog2: | $(filter prog1,${MAKECMDGOALS})
occurs to me. Oh well.]
YMMV and all that.
I'm not familiar with ghc, but the correct solution would be to get the two runs of ghc to use different build folders, then they can happily run in parallel.
Since I got stuck at the same problem, here is another pointer in the direction that make does not provide the functionality you describe:
From the GNU Make Manual:
It is important to be careful when using parallel execution (the -j switch; see Parallel Execution) and archives. If multiple ar commands run at the same time on the same archive file, they will not know about each other and can corrupt the file.
Possibly a future version of make will provide a mechanism to circumvent this problem by serializing all recipes that operate on the same archive file. But for the time being, you must either write your makefiles to avoid this problem in some other way, or not use -j.
What you are attempting, and what I was attempting (using make to insert data in a SQLite3 database) suffers from the exact same problem.
I needed to separate the compilation from other steps (cleaning, building dirs and linking), as I wanted to run the compilation with more core processes and the -j flag.
I managed to solve this, with different makefiles including and calling each other. Only the "compile" make file is running in parallel with all the cores, the rest of the process is syncronous.
I divided my makefile in 3 separate scripts:
settings.mk: contains all the variables and flag definitions
makefile: has all the targets except the compilation one (It has .NOTPARALLEL directive). It calls compile.mk with -j flag
compile.mk: contains only the compile operation (without .NOTPARALLEL)
In settings.mk I have:
CC = g++
DB = gdb
RM = rm
MD = mkdir
CP = cp
MAKE = mingw32-make
BUILD = Debug
DEBUG = true
[... all other variables and flags needed, directories etc ...]
In makefile I have Link and compilation target as these:
include .makefiles/settings.mk
[... OTHER TARGETS (clean, directories etc)]
compilation:
#echo Compilation
#$(MAKE) -f .makefiles/compile.mk --silent -j 8 -Oline
#Link
$(TARGET): compilation
#echo -e Linking $(TARGET)
#$(CC) $(LNKFLAGS) -o $(TARGETDIR)/$(TARGET) $(OBJECTS) $(LIBDIRS) $(LIB)
#Non-File Targets
.PHONY: all prebuild release rebuild clean resources directories run debug
.NOTPARALLEL: all
# include dependency files (*.d) if available
-include $(DEPENDS)
And this is my compile.mk:
include .makefiles/settings.mk
#Defauilt
all: $(OBJECTS)
#Compile
$(BUILDDIR)/%.$(OBJEXT): $(SRCDIR)/%.$(SRCEXT)
#echo -e Compiling: $<
#$(MD) -p $(dir $#)
#$(CC) $(COMFLAGS) $(INCDIRS) -c $< -o $#
#Non-File Targets
.PHONY: all
# include dependency files (*.d) if available
-include $(DEPENDS)
Until now, it's working.
Note that I'm calling compile.mk with -j flag AND -Oline so that parallel processing doesn't mess up with the output.
Any syntax color can be setted in the makefile main script, since the -O flag invalidates escape color codes.
I hope it can help.
I had a similar problem so ended up solving it on the command line, like so:
make target1; make target2
to force it to do the targets sequentially.

How can I use a makefile for D?

I have written complicated C and C++ makefiles in the past. However, I cannot seem to get my D makefile to work. It throws over a thousand lines of "undefined reference" errors, which look as if Phobos is failing to be linked. How can I fix that?
I am using GNU make and LDC2 on Fedora 19 Linux.
Edit: Compiling and linking directly using LDC2 works correctly. Only when invoked with 'make' is there an error. It seems that make is trying to invoke a separate linker.
Edit 2: Here is my makefile:
# This macro contains the source files
sources := $(wildcard *.d)
binaries := $(sources:%.d=%)
all: $(binaries)
%.o:%.d
ldc2 $< -O5 -check-printf-calls
Deleting the .o fixed it.
I don't know the intricacies of Pattern Rules, but I believe that is where your problem lies.
%.o:%.d
ldc2 $< -O5 -check-printf-calls
You've asked make to convert every .d file into a .o by calling the ldc2 command. However, you aren't requesting ldc2 to build object files, you're asking it to build an executable (I don't know which flag you want dmd/gdc: -c). Though I would have expected compiler errors from this before linker.
By removing the .o I must assume that Make is instead passing all the .d files at once rather than individually.

How are dependency files in Makefile with include resolved

I have a few questions about makefiles. I have defined my own version of a dependency file (.d) by creating it in code and called it say .dd (Looks just like a normal .c dependency file except that this is for some internal file format). Now this file I have included in the makefile with the include statement. (There are a whole bunch of these files so I've defined this as a rule which uses subst to replace the extension as required.)
Now suppose that file X depends on file Y i.e. X.dd contains a reference to file Y. Now from my understanding, whenever I run make and the program goes through the include and finds that Y has been updated, it somehow recompiles this so as to reflect the changes in X.
My question and problem is when this timestamp change in Y is noticed, does make restart itself so as to include the changes. How exactly are dependency files of this sort resolved. Also file X is not a typical .c or .cc file. So how would the file be treated when its dependency changes.
The problem I'm facing is that if file Y is changed/touched, file X does not recompile. Also if I touch file X, it recompiles just fine. Also as expected, if the makefile is touched, everything recompiles appopriately.
This bug has me scratching my head for a few days now so any help whatsoever would be appreciated.
Presuming that an X.boy is generated from an X.src (and Y.boy from Y.src) then to following works:
Makefile:
all: X.boy Y.boy
# for the example I will generate .boy with cp
%.boy: %.src
cp $< $#
include X.dd
X.dd:
X.boy: Y.src
This then gives:
$make
cp X.src X.boy
cp Y.src Y.boy
$touch X.src
$make
cp X.src X.boy
$touch Y.src
$make
cp X.src X.boy
cp Y.src Y.boy
My question and problem is when this timestamp change in Y is noticed,does make restart itself so as to include the changes.
GNU make reads the timestamps of files once, when it starts up. If you don't explicitly re-invoke it (e.g. by calling make or $MAKE from within the makefile), it won't re-check the timestamps after running commands.
So how would the file be treated when its dependency changes.
All files are treated identically; make just has a few built in rules for some types of files. Dependencies are handled the same way for all file types.
It sounds like your included .d file is creating a dependency on the source file(s), and not the target file. You can verify this by printing the rules make is using with make -p.
I'll use c files as an example, since you already know how their dependencies should work. Lets assume you have a file foo.c (source) which gets compiled to foo.o (target) and whos dependencies are specified in foo.d. For this example, lets say if bar.h or baz.h change, we want foo.o to be rebuilt.
foo.d should then contain something like:
foo.o : bar.h baz.h
And your makefile would read
%.o:%.c
$(CC) -c $< -o $#
include foo.d
Your symptoms sound like you have instead got a foo.d containing:
foo.c: bar.h baz.h
^
Or something totally unrelated on the left hand side, such that make does not see a dependency on foo.o. If you run make -p and search for foo.o in the output, you will see what make thinks it depends on from your makefile contents.

Resources