Nested ifndef in Makefile - makefile

In my Makefile I am trying to step through different ways of programatically setting an environment variable in a target. However, each statement inside each ifndef appears to be being executed every time. How do I get around this happening?
repository:
ifndef REPOSITORY_URI
#$(eval REPOSITORY_URI := $(shell bash -c "aws --region $(REGION) ecr describe-repositories --repository-names $(APP) | jq -r '.repositories[0].repositoryUri'"))
ifndef REPOSITORY_URI
#$(eval REPOSITORY_URI := $(shell bash -c "aws --region $(REGION) ecr create-repository --repository-name $(APP) | jq -r '.repositories[0].repositoryUri'"))
ifndef REPOSITORY_URI
echo "Could not establish link to AWS Repository, please ensure your credentials are set and try again"
endif
endif
endif

You're trying to compute a value for the make-variable REPOSITORY_URI using
make-functions, but you mistakenly believe that computing a value for a make-variable calls for a make-target and a recipe.
The actual meaning of your recipe for repository is quite different from what
you think and explaining what it does mean would be a large digression, since
no target or recipe is needed. To do what you are after here, just write:
ifndef REPOSITORY_URI
REPOSITORY_URI := $(shell bash -c "aws --region $(REGION) ecr describe-repositories --repository-names $(APP) | jq -r '.repositories[0].repositoryUri'")
ifndef REPOSITORY_URI
REPOSITORY_URI := $(shell bash -c "aws --region $(REGION) ecr create-repository --repository-name $(APP) | jq -r '.repositories[0].repositoryUri'")
ifndef REPOSITORY_URI
$(error "Could not establish link to AWS Repository, please ensure your credentials are set and try again")
endif
endif
endif
at the place in your makefile where you want to assign a value to REPOSITORY_URI
(or fail).
This by itself now constitutes a makefile with no targets. But presumably you
want to use the value of REPOSITORY_URI in the recipe for one or more targets, e.g.
ifndef REPOSITORY_URI
REPOSITORY_URI := $(shell bash -c "aws --region $(REGION) ecr describe-repositories --repository-names $(APP) | jq -r '.repositories[0].repositoryUri'")
ifndef REPOSITORY_URI
REPOSITORY_URI := $(shell bash -c "aws --region $(REGION) ecr create-repository --repository-name $(APP) | jq -r '.repositories[0].repositoryUri'")
ifndef REPOSITORY_URI
$(error "Could not establish link to AWS Repository, please ensure your credentials are set and try again")
endif
endif
endif
.PHONY: all
all:
echo REPOSITORY_URI=$(REPOSITORY_URI)
I recommend the GNU Make documentation

Related

How do I wildcard a rule with SECONDEXPANSION in GNU Make?

# Docker Chain
src_files:=$(shell find src/ -name '*.py')
image_tags=$(shell find dkr/ -name '$(PROJECT).*.dockerfile' | xargs -n 1 basename -s '.dockerfile')
dkr/%.dockerfile: $(src_files)
docker build . -f $# -t $*
.SECONDEXPANSION:
$(PROJECT).base: dkr/$$#.dockerfile
.SECONDEXPANSION:
$(PROJECT).foo: $(PROJECT).base | dkr/$$#.dockerfile
.SECONDEXPANSION:
$(PROJECT).bar: $(PROJECT).base | dkr/$$#.dockerfile
publish-%:
$(MAKE) $*
./scripts/$#.sh $(RESOURCE_PREFIX) $* $(REGION)
publish:
$(foreach tag, $(image_tags), $(MAKE) publish-$(tag))
I would like to write the above as:
# Docker Chain
src_files:=$(shell find src/ -name '*.py')
image_tags=$(shell find dkr/ -name '$(PROJECT).*.dockerfile' | xargs -n 1 basename -s '.dockerfile')
dkr/%.dockerfile: $(src_files)
docker build . -f $# -t $*
.SECONDEXPANSION:
$(PROJECT).base: dkr/$$#.dockerfile
.SECONDEXPANSION:
$(PROJECT).%: $(PROJECT).base | dkr/$$#.dockerfile
publish-%:
$(MAKE) $*
./scripts/$#.sh $(RESOURCE_PREFIX) $* $(REGION)
publish:
$(foreach tag, $(image_tags), $(MAKE) publish-$(tag))
However, the wildcarding in the rule (via '%') does not work. I suppose I have misunderstood rule substitution, and am guessing it is due to the SECONEXPANSION.
But, my make intuition is telling me this should work, and it should be good.
How would I perform substitution in this scenario?

How to defer foreach variable expansion in makefile prerequisites?

I have the following makefile that creates a config.json file from terraform output and then parses that config.json file and creates a makefile list variable, I then loop over that list and trigger targets accordingly. The targets copies the directories from output/%/csv to current/%/csv by looping over the list abc xyz
CONFIG = config.json
tf:
terraform output -json > $(CONFIG)
config_file: tf
$(eval obj := $(shell jq -c '.objects.value[]' $(CONFIG)))
output/%/csv : current/%/csv
mkdir -p $(#D)
cp -r $< $#
apply: $(foreach X, $(obj), output/$X/csv)
The above makefile gives an empty output. I expected this result since the prerequisites are expanded immediately and have empty value, to resolve this I added .SECONDEXPANSION: to have the prerequisites expanded in the deferred phase. But then got an error No rule to make target output//csv', needed by apply'. Stop. This means the variables are still assigned an empty value.
.SECONDEXPANSION:
CONFIG = config.json
tf:
terraform output -json > $(CONFIG)
config_file: tf
$(eval obj := $(shell jq -c '.objects.value[]' $(CONFIG)))
output/%/csv : current/%/csv
mkdir -p $(#D)
cp -r $< $#
apply: $(foreach $X, $$(obj), output/$$(X)/csv)
Is there something that I am missing here or is there any better way to do this?
There is no reason to set the obj make variable in a recipe with eval. In almost all examples I saw where eval or shell make functions were used in a recipe it was an error.
What about the following?
OBJ := $(shell terraform output -json | jq -c '.objects.value[]')
TARGETS := $(patsubst %,output/%/csv,$(OBJ))
output/%/csv : current/%/csv
mkdir -p $(#D)
cp -r $< $#
.PHONY: apply
apply: $(TARGETS)

How to optimize Makefile

I write a ugly copy/paste way created Makefile:
all: download install
install: \
${EXTERNAL_MODELS_LOCAL}/squeezenet_weights_tf_dim_ordering_tf_kernels.h5 \
${EXTERNAL_MODELS_LOCAL}/resnet50_weights_tf_dim_ordering_tf_kernels.h5 \
${EXTERNAL_MODELS_LOCAL}/inception_v3_weights_tf_dim_ordering_tf_kernels.h5 \
${EXTERNAL_MODELS_LOCAL}/squeezenet_weights_tf_dim_ordering_tf_kernels.h5:
ln -s ${EXTERNAL_MODELS_ROOT}/squeezenet_weights_tf_dim_ordering_tf_kernels.h5 $#
${EXTERNAL_MODELS_LOCAL}/resnet50_weights_tf_dim_ordering_tf_kernels.h5:
ln -s ${EXTERNAL_MODELS_ROOT}/resnet50_weights_tf_dim_ordering_tf_kernels.h5 $#
${EXTERNAL_MODELS_LOCAL}/inception_v3_weights_tf_dim_ordering_tf_kernels.h5:
ln -s ${EXTERNAL_MODELS_ROOT}/inception_v3_weights_tf_dim_ordering_tf_kernels.h5 $#
download: $(EXTERNAL_MODELS_ROOT)/ \
$(EXTERNAL_MODELS_ROOT)/squeezenet_weights_tf_dim_ordering_tf_kernels.h5 \
$(EXTERNAL_MODELS_ROOT)/resnet50_weights_tf_dim_ordering_tf_kernels.h5 \
$(EXTERNAL_MODELS_ROOT)/inception_v3_weights_tf_dim_ordering_tf_kernels.h5 \
$(EXTERNAL_MODELS_ROOT)/squeezenet_weights_tf_dim_ordering_tf_kernels.h5:
wget https://github.com/OlafenwaMoses/ImageAI/releases/download/1.0/squeezenet_weights_tf_dim_ordering_tf_kernels.h5 \
-O $#
$(EXTERNAL_MODELS_ROOT)/resnet50_weights_tf_dim_ordering_tf_kernels.h5:
wget https://github.com/OlafenwaMoses/ImageAI/releases/download/1.0/resnet50_weights_tf_dim_ordering_tf_kernels.h5 \
-O $#
$(EXTERNAL_MODELS_ROOT)/inception_v3_weights_tf_dim_ordering_tf_kernels.h5:
wget https://github.com/OlafenwaMoses/ImageAI/releases/download/1.0/inception_v3_weights_tf_dim_ordering_tf_kernels.h5 \
-O $#
Biggest part skipped ,but looks the same. Is it possible to optimize this boilerplate?
A mixture of make variables, make automatic variables (e.g. $<, $#), make functions (e.g. addsuffix, addprefix) and pattern rules, maybe:
RHOST := https://github.com/OlafenwaMoses/ImageAI/releases/download/1.0/
H5STEM := squeezenet resnet50 inception_v3
H5 := $(addsuffix _weights_tf_dim_ordering_tf_kernels.h5,$(H5STEM))
H5LOCAL := $(addprefix $(EXTERNAL_MODELS_LOCAL)/,$(H5))
H5ROOT := $(addprefix $(EXTERNAL_MODELS_ROOT)/,$(H5))
.PHONY: install download
install: $(H5LOCAL)
download: $(H5ROOT)
$(EXTERNAL_MODELS_LOCAL)/%.h5: $(EXTERNAL_MODELS_ROOT)/%.h5
ln -s $< $#
$(EXTERNAL_MODELS_ROOT)/%.h5:
wget $(RHOST)/$*.h5 -O $#
And of course, if there was a way to automatically discover the list of remote *.h5 files, it would be even better. But some more information is needed to imagine how to do it (ssh, curl, wget... ?). The make shell function would be the starting point, of course:
H5 := $(shell <the-command-that-lists-the-remote-h5-files>)

Copy list of files to specific paths with Makefile

Scratching my head for a while now, but I would like to copy an arbitrary list of files with paths to under specified path in system.
File layout:
data/a/file1.ext1
data/b/randomfile.ext2
data/c/file3.ext3
data/c/subdir/randomfile.2
Running make -f Makefile deploy DESTDIR=/path/to/somewhere copies those files to:
$(DESTDIR)/a/file1.ext1
$(DESTDIR)/b/randomfile.ext2
$(DESTDIR)/c/file3.ext3
$(DESTDIR)/c/subdir/randomfile.2
Makefile:
$FILES = \
a/file1.ext1 \
b/randomfile.ext2 \
c/file3.ext3 \
c/subdir/randomfile.2
ifneq ($(filter env_check,$(MAKECMDGOALS)),$())
ifndef DESTDIR
$(error DESTDIR not defined)
endif
endif
# lots of currently broken rules :(
# check whether target directory has certain structure
# check whether all the files listed in $(FILES) are in repository
Are you looking for something like this?
FILES := ...
DST_FILES := $(addprefix $(DESTDIR)/,$(FILES))
ifneq ($(filter env_check,$(MAKECMDGOALS)),$())
ifndef DESTDIR
$(error DESTDIR not defined)
endif
endif
all: $(DST_FILES)
$(DST_FILES) : ${DESTDIR}/% : %
#echo "$< ==> $#"
#[[ -e $< ]] || (echo "some error for $<" && false)
#mkdir -p $(dir $#)
#cp $< $#
[Edit]:
Although the version somewhat worked, I still needed to do following adjustments:
Files in source repository are stored under data directory - fixed by using $addprefix call
When file in $(DESTDIR) already existed, it was never copied - used the .FORCE target. (Another option would be --always-make commandline option).
Eventually, the working Makefile looks like that:
# File are stored under data/
FILES= \
foo/file1.ext \
bar/file2.txe \
bar/dir/file3.txt
ifneq ($(filter env_check,$(MAKECMDGOALS)),$())
ifndef DESTDIR
$(error DESTDIR not defined)
endif
endif
.PHONY: deploy help
help:
#echo "Deploy stuff"
# Check whether certain directories in the output are present
env_check:
#test -d $(DESTDIR)/WEB-INF -a -d $(DESTDIR)/META-INF || \
( echo "DESTDIR: \"$(DESTDIR)\" is not proper deployment path" && exit 1 )
DST_FILES := $(addprefix $(DESTDIR)/, $(FILES))
# We need to add our path prefix to local files and FORCE to always do the copying
$(DST_FILES) : $(addprefix $(DESTDIR), %) : $(addprefix data,%) .FORCE
#cp -pv $< $#
.FORCE:
deploy: env_check $(DST_FILES)
#echo "Deployment done..."

Restart udev on Makefile

I've made a simple Makefile for an application and after install I need to restart udev rules.
INSTALLDIR=/pkt/bin
OS:=$(shell uname -v)
LBITS:=$(shell getconf LONG_BIT)
LIBDIR=/usr/lib
ifeq ($(LBITS),64)
LIBDIR64=/usr/lib64
else
LIBDIR64=/usr/lib
endif
all: usbupdater
configuracion.o: configuracion.cpp
g++ -c configuracion.cpp
main.o: main.cpp
g++ -c main.cpp
usbupdater: main.o configuracion.o
#echo "$(PATH)"
#echo "$(LIBDIR)"
g++ main.o configuracion.o $(LIBDIR)/libReadINI.a $(LIBDIR64)/chilkat/li
bchilkat-9.4.1.a -lpthread -lresolv -o usbupdater
clean:
rm -rf *.o *.cgh $(INSTALLDIR)/usbupdater
install:
mv usbupdater $(INSTALLDIR)/usbupdater
cp -rf 99-persistent-usb.rules /etc/udev/rules.d/99-persistent-usb.rules
postinstall:
#echo "$(OS)"
ifeq ($(findstring Debian,$(OS)),Debian) \
#echo "Estoy dentro del if"
$(shell '/etc/init.d/udev' restart) \
else \
#echo "Estoy dentro del else"
$(shell ls -l) \
endif
The problem is that when I type make postinstall is shows me this error:
#1 SMP Debian 3.2.46-1+deb7u1
ifeq (Debian,Debian) \
#echo "Estoy dentro del if"
/bin/sh: 1: Syntax error: word unexpected (expecting ")")
make: *** [postinstall] Error 2
I don't know where the problem is. I compare the result of uname -v with Debian to perform udev restart or udevcontrol reload_rules if it is an Opensuse OS.
Thanks in advance and sorry for my English.
ifeq is a make command. All lines in the makefile (in a recipe context) are passed to the shell. So make is passing ifeq to the shell and the shell is telling you that it has no idea what you're talking about.
You should write this using shell syntax, not make syntax.
Also it's rarely useful to use $(shell ...) functions inside a make recipe. The make recipe will be run by the shell, so when you use $(shell ...) you're just doubling the amount of shells that are running. Plus a command like $(shell ls -l) is not going to work, because $(shell ...) is like backtick and replaces the function with the stdout of the command. That'll be an error in this situation.
I would write it as:
postinstall:
#echo "$(OS)"
#case '$(OS)' in \
(*Debian*) \
echo "Estoy dentro del if"; \
/etc/init.d/udev restart ;; \
(*) \
echo "Estoy dentro del else"; \
ls -l ;; \
esac
You can't use make internal commands (like ifeq) within rule definition block. Use either shell's if or use ifeq outside of rule to generate some variables values, like
ifeq($(blah-blah), blah)
BUILD_CMD:=foo
endif
Also worth noting that else statement isn't exactly standard and may be missing in some versions of make.
Why do you want this, btw? I'd consider really bad practice if make install does something other then just installing (copying) files.

Resources