I'm trying to compile all my apps from the cmd folder but somehow this makefile is not working.
GO_BUILD_FLAGS =
APPS = my-app another-app stuff
define BUILD_BINARY =
#echo go build $(GO_BUILD_FLAGS) -o $# ./$<
go build $(GO_BUILD_FLAGS) -o $# ./$<
endef
FORCE: ;
bin/%: cmd/% FORCE
$(BUILD_BINARY)
build: $(addprefix bin/,$(APPS))
I'm continously facing following output, no matter I try.
$ make build
make: Nothing to be done for `build'.
$ make bin/my-app
make: Nothing to be done for `bin/my-app'.
$ make bin/another-app
make: Nothing to be done for `bin/another-app'.
$ make bin/stuff
make: Nothing to be done for `bin/stuff'.
Despite FORCE the bin/ targets are not executed. What am I missing?
Running this on MacOS using GNU Make 3.81.
This is how my folder looks like:
$ tree cmd
cmd
├── another-app
│ └── main.go
├── my-app
│ └── main.go
└── stuff
└── main.go
3 directories, 3 files
when running make with -d I get following:
https://pastebin.com/wh6TWxj9 (didn't fit in body as it has a 30000 char limit)
Eventually I found the issue. However I do not understand yet why the define doesn't work.
In the end I just removed:
define BUILD_BINARY =
#echo go build $(GO_BUILD_FLAGS) -o $# ./$<
go build $(GO_BUILD_FLAGS) -o $# ./$<
endef
and replaced the original bin target
bin/%: cmd/% FORCE
$(BUILD_BINARY)
with
bin/%: cmd/% FORCE
go build $(GO_BUILD_FLAGS) -o $# ./$<
Then it starts working.
I want to use make and create a static pattern rule that has the target in a output directory, and the prerequisite files are in the preceeding directory, and it has to work recursively.
I have a minimal example here:
.
├── anotherdir
│ ├── output
│ │ ├── source3.md
│ │ └── source4.md
│ ├── source3.json
│ └── source4.json
├── output
│ ├── source1.md
│ └── source2.md
├── source1.json
└── source2.json
I want to generate the output directories if they do not exist, and I want to generate *.md files from the *.json using make if they do not exist, or *.json is updated.
So far, I have the following Makefile:
SOURCE_FILES := $(shell find ./ -name "*.json")
OUTPUT_FILES := $(join $(addsuffix output/,$(dir $(SOURCE_FILES))), $(addsuffix .md,$(basename $(notdir $(SOURCE_FILES)))))
.PHONY: all
all: $(OUTPUT_FILES)
$(OUTPUT_FILES): %.md: %.json
mkdir -p $(dir $#)
# Command to create MD file from json file into the output directory here
The actual command to create the MD file from the json file doesn't matter here, because I have a script that I will call that will do this for me. The problem here, is that when I try to even run this at all, I get the following output:
> make all
make: *** No rule to make target 'anotherdir/output/source4.json', needed by 'anotherdir/output/source4.md'. Stop.
Obviously, source4.json is not in anotherdir/output, but rather, it's in the preceeding directory, which is just anotherdir. I don't know how to make it so that the pattern $(OUTPUT_FILES): %.md: %.json will match it properly.
Or is a static pattern rule not good here? I'm not sure what to do to fit my scenario.
EDIT: I tried to do something like this:
$(OUTPUT_FILES): %.md: $(join $(subst output,,$(dir %)), $(addsuffix .json,$(basename $(notdir %))))
and this doesn't work, I still get:
> make all
make: *** No rule to make target 'anotherdir/output/source4.json', needed by 'anotherdir/output/source4.md'. Stop.
Edit 2: to clarify, i start with the following files
.
├── anotherdir
│ ├── source3.json
│ └── source4.json
├── source1.json
└── source2.json
And then when i run make, i want it to generate the output folders like this
.
├── anotherdir
│ ├── output
│ │ ├── source3.md
│ │ └── source4.md
│ ├── source3.json
│ └── source4.json
├── output
│ ├── source1.md
│ └── source2.md
├── source1.json
└── source2.json
I want to use some kind of smart makefile syntax to pick up these files names without me hard coding it in myself. Hence, i looked at the documentation and saw that static pattern rules might be the solution that i want, except that i can't get the right prerequisite pattern down.
I would do it this way:
First, find the source files just as you did (with a small change to prevent the unsightly double-slash):
SOURCE_FILES := $(shell find . -name "*.json")
A pattern file would be nice, if we could use two wildcards at once, but Make can't quite do that. So I recommend using a template:
define template
TDIR := $(dir $(1))output
TARG := $$(TDIR)/$(notdir $(basename $(1))).md
$$(TARG): $(1)
mkdir -p $$#
#echo building $$# from $$<
# Command to create MD file from json file into the output directory here
endef
$(foreach SOURCE,$(SOURCE_FILES),$(eval $(call template,$(SOURCE))))
If this works, all that's left is to construct a list of output files, and a default rule that has all of them as prerequisites:
define template
TDIR := $(dir $(1))output
TARG := $$(TDIR)/$(notdir $(basename $(1))).md
OUTPUT_FILES += $$(TARG)
$$(TARG): $(1)
mkdir -p $$#
#echo building $$# from $$<
# Command to create MD file from json file into the output directory here
endef
all:
$(foreach SOURCE,$(SOURCE_FILES),$(eval $(call template,$(SOURCE))))
all: $(OUTPUT_FILES)
It isn't pretty, but it seems to work.
If it had not been proposed already in another answer I would have suggested foreach-eval-call. For completeness here are different solutions for GNU make (they may work also with other versions of make but I did not check):
Creating the output directories beforehand
If the output directories exist already you can refer to ../%.json in your pattern rule:
SOURCE_FILES := $(shell find . -name "*.json")
OUTPUT_FILES := $(join $(dir $(SOURCE_FILES)),\
$(patsubst %.json,output/%.md,$(notdir $(SOURCE_FILES))))
$(shell mkdir -p $(dir $(OUTPUT_FILES)))
.PHONY: all
all: $(OUTPUT_FILES)
%.md: ../%.json
: json2md $< -o $#
This may look strange but if you read carefully the Pattern match section of the GNU make manual you should quickly understand. The only constraint for this to work is that the output directories exist before make searches pattern rules that match the targets. If one does not exist make will complain that there is no eligible rule to build the target. This is the reason for the:
$(shell mkdir -p $(dir $(OUTPUT_FILES)))
at the beginning of the Makefile. Demonstration:
$ make
: json2md output/../source2.json -o output/source2.md
: json2md output/../source1.json -o output/source1.md
: json2md anotherdir/output/../source4.json -o anotherdir/output/source4.md
: json2md anotherdir/output/../source3.json -o anotherdir/output/source3.md
Using the secondary expansion
Secondary expansion gives you the possibility to use automatic variables in the list of prerequisites. The $$ are needed to escape the first expansion by make.
SOURCE_FILES := $(shell find . -name "*.json")
OUTPUT_FILES := $(join $(dir $(SOURCE_FILES)),\
$(patsubst %.json,output/%.md,$(notdir $(SOURCE_FILES))))
.PHONY: all
all: $(OUTPUT_FILES)
$(sort $(dir $(OUTPUT_FILES))):
mkdir -p $#
.SECONDEXPANSION:
$(OUTPUT_FILES): $$(patsubst %output,%,$$(#D))$$(basename $$(#F)).json | $$(dir $$#)
: json2md $< -o $#
Demonstration:
$ make
mkdir -p output/
mkdir -p anotherdir/output/
: json2md source2.json -o output/source2.md
: json2md source1.json -o output/source1.md
: json2md anotherdir/source4.json -o anotherdir/output/source4.md
: json2md anotherdir/source3.json -o anotherdir/output/source3.md
Note: instead of creating the output directories in the json-to-md rule (which has the drawback of creating them several times), I added them as order-only prerequisites and added a specific rule to create them.
Note: the sort function also removes duplicates.
Using recursive make
Here we invoke make (with always the same Makefile) recursively in each sub-directory (except output, of course). Each invocation handles only the local json files, which makes the paths of prerequisites and targets much simpler.
MF := $(realpath $(lastword $(MAKEFILE_LIST)))
SUB_DIRS := $(filter-out . ./output,$(shell find . -maxdepth 1 -type d))
SOURCE_FILES := $(filter-out $(SUB_DIRS),$(wildcard *.json))
OUTPUT_FILES := $(patsubst %.json,output/%.md,$(SOURCE_FILES))
.PHONY: $(SUB_DIRS) all
all: $(SUB_DIRS) $(OUTPUT_FILES)
$(OUTPUT_FILES): output/%.md: %.json | output
: json2md $< -o $#
output:
mkdir -p $#
$(SUB_DIRS):
$(MAKE) -C $# -f $(MF)
Demonstration:
$ make
make -C anotherdir -f /home/doe/json2md/Makefile
make[1]: Entering directory '/home/doe/json2md/anotherdir'
mkdir -p output
: json2md source4.json -o output/source4.md
: json2md source3.json -o output/source3.md
make[1]: Leaving directory '/home/doe/json2md/anotherdir'
mkdir -p output
: json2md source2.json -o output/source2.md
: json2md source1.json -o output/source1.md
Here's a minimal example of what I'm trying to do.
Directory structure:
.
├── Makefile
└── subdir
├── a
└── Makefile
./subdir/Makefile:
a.copy: a
cp a a.copy
./Makefile:
.PHONY: build_subdir
a.copy: subdir/a.copy
cp subdir/a.copy a.copy
build_subdir:
$(MAKE) -C subdir
subdir/a.copy: build_subdir
The first time I run make, everything's fine:
$ make
make -C subdir
make[1]: Entering directory `/home/kkourt/src/tests/make/subdir'
cp a a.copy
make[1]: Leaving directory `/home/kkourt/src/tests/make/subdir'
cp subdir/a.copy a.copy
Re-runing make is also fine (nothing happens):
$ make
make -C subdir
make[1]: Entering directory `/home/kkourt/src/tests/make/subdir'
make[1]: `a.copy' is up to date.
make[1]: Leaving directory `/home/kkourt/src/tests/make/subdir'
If I update subdir/a, however, ./a is not updated:
$ touch subdir/a
$ make
make -C subdir
make[1]: Entering directory `/home/kkourt/src/tests/make/subdir'
cp a a.copy
make[1]: Leaving directory `/home/kkourt/src/tests/make/subdir'
If I run make a second time, ./a gets updated
$ make
make -C subdir
make[1]: Entering directory `/home/kkourt/src/tests/make/subdir'
make[1]: `a.copy' is up to date.
make[1]: Leaving director
Why does this happen? shouldn't the check of whether subdir/a.copy is older than a.copy happen after the build_subdir target is finished since it's its dependency? Is there a way to fix this?
Edit:
As MadScientist suggested, I added a dummy recipe and it works:
subdir/a.copy: build_subdir
#:
Because your rule subdir/a.copy: build_subdir has no recipe, make "knows" that it can't actually change the timestamp on subdir/a.copy.
You need add a dummy recipe to it, maybe like this:
subdir/a.copy: build_subdir ;
(note I can't test this right now but I think it'll work).
I've got such a structure of files and directories:
.
├── Makefile
└── packages
├── Makefile
└── subdir
└── Makefile
and top Makefile looks like this:
define aa
make -C $1 $2
endef
packages=$(shell find ./packages -type d)
p1=$(filter-out . .., $(packages))
all:
$(foreach f,$(p1),$(call aa,$(f),compile))
and both of Makefiles in ./packages/ and ./packages/subdir/ have "compile" target.
I'm trying to invoke all of Makefiles automatically in "packages" subdirectory without adding them individually into Makefile.
When I run make in top directory I got error:
make -C ./packages compile make -C ./packages/subdir compile
make[1]: *** packages/subdir: No such file or directory. Stop.
Makefile:16: recipe for target 'all' failed
make: *** [all] Error 2
I'm wondering why both invocations of make (which should be separate invocations) are placed in one line?
When I add end-of-line at the end of the "aa" macro like this:
define aa
make -C $1 $2
endef
everything works as expected.
My question is why this macro doesn't work without this end-of-line?
Because a the variable definition starts after the newline which follows the define, and ends before the newline preceding the endef.
Quoting the gnumake manual :
The value in an ordinary assignment cannot contain a newline; but the newlines that separate the lines of the value in a define become part of the variable’s value (except for the final newline which precedes the endef and is not considered part of the value).
So your define is equivalent to aa = make -C $1 $2 and since foreach does not add anything between each expansion, you got that result.
I was investigating on the same question here, but I was not very clear of what I was asking, even for myself. Sorry for those who spent time answering my unclear question.
So let's try again with a more realistic example. We consider this structure:
.
├── Makefile
└── src/
├── bar
├── foo
└── Makefile
Where the main Makefile is:
all: src/foobar
src/foobar:
make -C $(dir $#)
And the sub-makefile is:
foobar: foo bar
join $^ > $#
If I run make for the first time (from ./) everything works as expected, foobar is produced.
$ make
make -C src/
make[1]: Entering directory '/project/src'
join foo bar > foobar
make[1]: Leaving directory '/project/src'
$ make
make: Nothing to be done for 'all'.
However if I touch any of the foobar dependencies. The parent Makefile will not regenerate the target. Here, I perfectly understand the behavior of Make, but I want to tell it to be aware of foobar' dependencies.
$ touch src/foo
$ make
make: Nothing to be done for 'all'.
My current solution which is not very neat is to ask for the dependencies. So the src/Makefile become:
src=foo bar
foobar: $(src)
#echo "Joining"
join $^ > $#
files: $(src)
#echo $^
And the ./Makefile:
all: src/foobar
src=$(addprefix src/,$(shell make --no-print-directory -C src files | tr '\n' ' '))
src/foobar: $(src)
make -C $(dir $#)
I must also say that this particular example could be simplified using a single Makefile only. My real example is quite more complex. The src/Makefile generate an executable while the parent Makefile do lots of other things (packing in a specific format, generate the documentation, build other sub-makefiles and so on). Thus, I want to keep these tasks well separated and I need to different Makefiles.
In the main Makefile create a dependency for the child target or directory that is always in need of building, and let the child Make then do the real work.
There is a good example here: http://owen.sj.ca.us/~rk/howto/slides/make/slides/makerecurs.html.
To translate for your case, change your main Makefile to be:
all: src/foobar
src/foobar: force
$(MAKE) $(MFLAGS) -C src
force:
true
I also added $(MFLAGS) which will pass same flags from parent to child make.