Second dependency in goal first-deploy not executed.
if I execute make first-deploy, make execute only first dependence - build-proxy-base with success result and exit.
Makefile:
UID=`id -u`
GID=`id -g`
NODE_N=node
PROXY_N=proxy
PY_N=py
DOCKER_D=docker
PROXY_D=skif-proxy
NODE_D=skif
PY_D=skif-proxy/template-service
# first is default
default: first-deploy docker-compose-up
docker-compose-up:
cd $(DOCKER_D) && \
docker-compose up --build && \
cd ..
build-docker-base:
docker build \
--build-arg UID=${UID} \
--build-arg GID=${GID} \
-f ${D}/Dockerfile.base \
--rm \
-t skif-${SERV_N}-base ${D}
build-node-base: D := ${NODE_D}
build-node-base: SERV_N := ${NODE_N}
build-node-base: build-docker-base
build-proxy-base:D=${PROXY_D}
build-proxy-base:SERV_N=${PROXY_N}
build-proxy-base: build-docker-base
build-py-base: D=${PY_D}
build-py-base: SERV_N=${PY_N}
build-py-base: build-docker-base
first-deploy: build-proxy-base build-py-base build-node-base
UPD
I understand that build-docker-base build only once, but how I can reuse it code in 2 goals with other paramethers?
Make definitely considered the second dependency. If it didn't "execute" then it means make decided nothing needed to be done to execute it.
I can see this:
build-proxy-base: build-docker-base
...
build-node-base: build-docker-base
Both of these targets list the same target as a prerequisite, and they don't do anything else. In make, a given target (like build-docker-base) will only ever be built one time by make per invocation. Once make builds it once, it will be considered up to date no matter how many other targets may depend on it.
So make considers build-node-base, sees that it was already brought up to date due to build-proxy-base, and decides there's nothing else to do.
Since you don't actually show us to build-node-base rule or explain what you're trying to do, there's not much more we can say for sure.
ETA Based on your updated makefile, it seems like you're trying to think of targets as if they were functions, that can be called by listing them as prerequisites. They are not, and they cannot.
It looks to me like you just want to run the same recipe multiple times with different parameters. I think you should just write it like this:
build-node-base build-proxy-base build-py-base:
docker build \
--build-arg UID=${UID} \
--build-arg GID=${GID} \
-f ${D}/Dockerfile.base \
--rm \
-t skif-${SERV_N}-base ${D}
build-node-base: D := ${NODE_D}
build-node-base: SERV_N := ${NODE_N}
build-proxy-base:D=${PROXY_D}
build-proxy-base:SERV_N=${PROXY_N}
build-py-base: D=${PY_D}
build-py-base: SERV_N=${PY_N}
Since there is no recipe for either build-proxy-base or build-node-base, and both are effectively phony, it's unclear how you are concluding that build-node-base is not, in fact, built.
I suspect that the actual observation is that target build-docker-base is built only once per make run, and that that build reflects the values of variables D and SERV_N set while building build-proxy-base. That has little to do with whether target build-node-base is built, and a lot to do with the fact that make builds each target at most once per run.
Related
I have a Makefile as below, and I am concerned with the dependency order for up-clean:
.PHONY: up
up: down
docker-compose up -d
.PHONY: up-clean
up-clean: down-clean up
.PHONY: down
down:
docker-compose down
.PHONY: down-clean
down-clean:
docker-compose down -v
Obviously it is important that in up-clean: down-clean up, down-clean must be executed before up. For ordinary make targets, the solution would be to add an entry, up: down-clean, but as these are PHONY targets, that would make up functionally equivalent to up-clean, removing volumes every time. Obviously, this is unacceptable.
In practice, GNU make respects the order of dependencies, but it does not guarantee it, and so is not entirely trustworthy, or with -j not trustworthy at all. What methods, if any, exist to ensure the order of execution of dependencies in this situation without changing the result of building other targets?
EDIT: Attempting to use order-only prerequisites does not appear to work, possibly because of an interaction with .PHONY. Adding
up: | down-clean
Causes the execution log to be:
$ make up
docker-compose down
<...>
docker-compose down -v
Removing volume <...>
Which is what is supposed to happen for normal prerequisites, not order-only ones.
The simplest answer is to use recursive invocations of make:
up-clean:
$(MAKE) down-clean
$(MAKE) up
Another alternative would be to model up-clean on up instead of making the latter a prerequisite for the former:
.PHONY: up
up: down
docker-compose up -d
.PHONY: up-clean
up-clean: down-clean
docker-compose up -d
If you want to make that a little DRYer, you could factor out the docker-compose command to a variable:
UP_COMMAND = docker-compose up -d
.PHONY: up
up: down
$(UP_COMMAND)
.PHONY: up-clean
up-clean: down-clean
$(UP_COMMAND)
Starting with GNU make 4.4 you can explicitly serialize your prerequisites with the .WAIT pseudo-target, e.g.,
.PHONY: up-clean
up-clean: down-clean .WAIT up
Before 4.4, the GNU make documentation was avoiding making any commitments about the order or execution to enable parallel execution. However, any POSIX-compliant implementation of make (emphasis mine)
... shall treat all prerequisites as targets themselves and recursively ensure that they are up-to-date, processing them in the order in which they appear in the rule. The make utility shall use the modification times of files to determine whether the corresponding targets are out-of-date.
Of course, in the parallel mode, GNU make can't be fully compliant with this requirement.
But, the 4.4 release also adds the --shuffle option, and from the research made by the implementor of this option, it is evident that the only source of non-determinism is the -j option and the parallel mode of execution, which is also witnessed by the contents of the patch and the tests, which were reviewed by other members of the GNU make project.
Therefore, in version prior to 4.4, we can safely assume that the prerequisites are executed strictly in the order in which they are specified, from left to right, as long as make is executed in non-parallel mode. In versions before 4.4, we can disable parallelism with the .NOTPARALLEL pseudo target, just add it to your file, e.g.,
.NOTPARALLEL: # ensures that all deps are executed strictly in order
.PHONY: up
up: down
docker-compose up -d
.PHONY: up-clean
up-clean: down-clean up
.PHONY: down
down:
docker-compose down
.PHONY: down-clean
down-clean:
docker-compose down -v
Notice also, that --shuffle respects the presence of the .NOTPARALLEL target, which corroborates our hypothesis the order could different from the syntactic order only because of the parallel execution.
I have a makefile that deploys x project in aws
The goal is to execute the make like this:
make deploy-production
And see output like this
Please select the project:
1) api
2) web
3) somethingelse
You press then the number and make continues, by assigning the choice to a variable named project
I have already "autogenerating" targets for supported envs, just want to build the multichoice for projects.
Sample makefile of mine with removed nonrelevant stuff:
ENVIRONMENTS := development staging production
TARGETS := $(foreach X,$(ENVIRONMENTS),deploy-$X)
$(TARGETS):
$(eval ENV:=$(subst deploy-,,$(#)))
# here want to ask for project, env i have already as resolved from above line
Well, make can run any shell script. So if you write a shell script that will ask the user for input and accept an answer you can run it from make.
You could do something like this:
deploy-production:
#echo Please select the project:; \
echo '1) api'; \
echo '2) web'; \
echo '3) somethingelse'; \
read -p 'Enter value: ' result && $(MAKE) CHOICE=$$result got-choice
There is a LOT left out here: handling invalid values, converting the CHOICE value into a target that would then be a prerequisite of the got-choice target, etc. etc. I don't really think this is a good way forward but you could make it work.
The problem I'm experiencing is an all target has dependencies on others that set a variable, then run matching dependencies.
Outcome - It will run the first dependency then stop.
Expected - Run both dependencies, setting the variable properly between each run
Is make smart enough to see pull and build were already ran and the dependency target itself has no execution, therefore it sees all dependencies as complete? Or I'm just abusing make in ways it should not be used?
Said makefile:
repo=rippio
image=default
pull:
#docker pull $(repo)/$(image):latest
build: pull
#sed -e 's/{{repo}}/$(repo)/' -e 's/{{image}}/$(image)/' Dockerfile.in > Dockerfile && \
docker build -t=$(repo)/$(image):custom .
#rm -f Dockerfile
node: image=node
node: | pull build
jdk8: image=jdk8
jdk8: | pull build
all: | node jdk8
TLDR
It is used to:
Pull the latest docker image
Run a generically designed Dockerfile against it to customize it
Tag it as :custom for internal use
Pretty handy for customizing images in a generic manner without managing many Dockerfiles.
Dockerfile template (Dockerfile.in), incase interested:
FROM {{repo}}/{{image}}:latest
... super secret sauce
UPDATE (ANSWER)
Thanks to #G.M., ended up with:
IMAGE_NAMES := node jdk8
TARGETS := $(patsubst %,build-%,$(IMAGE_NAMES))
repo=rippio
all: $(TARGETS)
build-%: pull-%
#$sed -e 's/{{repo}}/$(repo)/' -e 's/{{image}}/$*/' Dockerfile.in > Dockerfile-$* && \
$docker build -f=Dockerfile-$* -t=$(repo)/$*:custom .
#rm -f Dockerfile-$*
pull-%:
#$docker pull $(repo)/$*:latest
Which allows for:
Easy upkeep of 'all' targets, which constantly grows
Running parallel via make -j (note the Dockerfile-$* file pattern)
Much more beautiful than before
If you draw your dependency graph out long-hand you'll see that there are multiple paths from all to both pull and build -- one via each of node and jdk8. But make having reached/updated pull and build via one path will then assume that they are both up to date and, hence, not bother to update them further -- regardless of any change to target specific variables.
I think what you're trying to do (assuming I've understood correctly) might be more easily achieved using pattern rules.
IMAGE_NAMES := node jdk8
TARGETS := $(patsubst %,build-%,$(IMAGE_NAMES))
repo=rippio
all: $(TARGETS)
build-%: pull-%
#$sed -e 's/{{repo}}/$(repo)/' -e 's/{{image}}/$*/' Dockerfile.in > Dockerfile && \
$docker build -t=$(repo)/$*:custom .
#rm -f Dockerfile
pull-%:
#$docker pull $(repo)/$*:latest
Note: You currently have all build recipes using the same input/output file DockerFile. That will cause problems if you ever want to use parallel builds -- make -j etc. It might be wise to use the stem from the pattern rule match to uniquely identify the output file if that's possible.
Normally, if you invoke make with:
make all
and if none of pull, build, node, jdk8 are existing files, make should build pull and build. If you see only pull being made, it can be because you invoke make without specifying a goal. In this case make builds the first target it finds in the Makefile (pull in your case).
Anyway, there are several strange aspects in your Makefile: you use order-only prerequisites on what looks like phony targets and these phony targets are not declared as such.
I am not sure I fully understand what you are trying to do but maybe something like this would be a good starting point:
repo=rippio
image=default
.PHONY: all build node jdk8
all: node jdk8
node: image = node
jdk8: image = jdk8
build node jdk8:
#docker pull $(repo)/$(image):latest && \
sed -e 's/{{repo}}/$(repo)/' -e 's/{{image}}/$(image)/' Dockerfile.in > Dockerfile && \
docker build -t=$(repo)/$(image):custom . && \
rm -f Dockerfile
Note: if, instead of build you name the default target default you could even simplify further with:
repo=rippio
.PHONY: all default node jdk8
all: node jdk8
default node jdk8:
#docker pull $(repo)/$#:latest && \
sed -e 's/{{repo}}/$(repo)/' -e 's/{{image}}/$#/' Dockerfile.in > Dockerfile && \
docker build -t=$(repo)/$#:custom . && \
rm -f Dockerfile
Here is the related rules for the variable:
LIBAD = libadard.a
install: $(CRSLIB)/$(LIBAD)
$(CRSLIB)/$(LIBAD): $(LIBAD)
$(LIBAD): $(OBJECTS1)
OBJECTS1 = $(LIBAD)(libadardV.o)\
$(LIBAD)(a_delbb.o) $(LIBAD)(a_getbkm.o)\
...
$(LIBAD)(a_getbkm.o): a_getbkm.p \
$(KINCDIR)/dbug.h \
$(PRIMINC)/systypes.h \
$(PRIMINC)/externs.h \
$(PRIMINC)/reserrs.h \
$(KINCDIR)/ltypes.h \
$(KINCDIR)/except.h \
$(PRIMINC)/u_pr_bkmsg.h \
$(CRSINC)/sqlerrs.h \
$(PDBINC)/systypes.th \
$(PRIMINC)/u_pr_bkmsg.th \
$(INFORMINC)/sqlca.h
if i run "make install", here's what i got:
make: *** No rule to make target `/sqlca.h', needed by `libadard.a(a_getbkm.o)'. Stop.
For testing purpose, i added this rule in the makefile just to check the value of this variable $(INFORMINC):
PHONY: all
all: ; #echo $(INFORMINC)
And the output is correct:
mtang#rv02 release>make all
/informix-rv02_1/incl/esql
i also checked under the directory "/informix-rv02_1/incl/esql", the file sqlca.h is there. So what went wrong?
UPDATE:
variable $(INFORMINC) is not defined in this makefile. It is defined in a Makerules file sitting at the root level, and that Makerules is included by this Makefile:
include ../../Makerules
UPDATE 2:
Problem solved. Thanks #Roland Illig for the clue. In Makerules, INFORMINC is defined as:
INFORMIXDIR := $(MY_INFORMIXDIR)
INFORMIX := $(INFORMIXDIR)
INFORMINC := $(INFORMIX)/incl/esql
I just copied that last line where INFORMINC is defined and paste it in the makefile. And it worked. I am not sure if i totally understand the reason behind this, but that certainly gives me some experience to deal with similar problems in the future.
In BSD Make (and I think in many other implementations, too), the dependency lines are evaluated eagerly, at the time of parsing. So when you define the INFORMINC variable at a later point, it will not influence the dependency rule.
The shell command in the all target is evaluated lazily, just before executing it. Therefore you see its value.
See also https://mail-index.netbsd.org/tech-pkg/2016/05/26/msg016900.html, where I explained this topic a litle more verbosely.
The GNU make manual says:
A phony target should not be a prerequisite of a real target file; if it is, its recipe will be run every time make goes to update that file.
What if that's what I want?
For example, what if I have a phony target called lint that lints app/scripts/main.js, and I want it to run every time make goes to update (transpile & minify) dist/scripts/main.js?
I just removed the lint target and put its recipe as the first command of the recipe for the dist/scripts/main.js target.
dist/scripts/main.js : app/scripts/main.js
./node_modules/.bin/eslint $< && \
mkdir -p dist/scripts && \
./node_modules/.bin/babel $< | \
./node_modules/.bin/uglifyjs - --screw-ie8 -o $# -m -c
Updated answer: the current GNU make manual does not specify the cited "rule" from the question but instead explicit says to make use of this in the entry about forced targets, which includes a note:
As you can see, using FORCE this way has the same results as using .PHONY clean.
Using .PHONY is more explicit and more efficient. However, other versions of make do not support .PHONY; thus FORCE appears in many makefiles