I have my project binary located at my repository's root, along with a Makefile used to build it.
This binary uses many of my self-made libraries, located in my lib/ folder
For the purpose of building (and cleaning) my repository's binary, I want to implement the following execution :
Instead of hardcoding the following lines,
clean_binaries:
make -C clean lib/folder1 -s
make -C clean lib/folder2 -s
make -C clean lib/another_folder -s
I created the BIN_PATH variable, containing the previous paths.
BIN_PATHS = lib/folder1 \
lib/folder2 \
lib/another_folder
And made a simple rule like this one :
clean_binaries: $(BIN_PATHS)
make -C clean $< -s
BUT it only executes the line for the first field of the variable (lib/folder1), which is not what I want to do.
I thought about using implicit rules(?), just like I compile the .c files, but I couldn't get it right.
In the end, I simply wonder how to execute a rule for every field of a given variable, and this inside a Makefile, if there is any way to do so.
Thank you for your answers :]
The way you get GNU make to generate a sequence of commands that vary by the
fields in a variable is to use the foreach function, e.g.
Makefile
BIN_PATHS := lib/folder1 lib/folder2 lib/another_folder
.PHONY: clean_binaries
clean_binaries:
$(foreach path,$(BIN_PATHS),make -C $(path) clean ;)
which runs like:
$ make
make -C lib/folder1 clean -s; make -C lib/folder2 clean -s; make -C lib/another_folder clean -s;
not requiring a shell-loop.
Note also that you need to correct:
make -C clean <directory>
to:
make -C <directory> clean
Related
I've tweak "a bit" Make so I can use it as a "kind of" cli for some tasks.
MAKEFLAGS += --no-builtin-rules
MAKEFLAGS += --no-builtin-variables
MAKEFLAGS += --no-print-directory
SHELL := /bin/bash
.ONESHELL:
.PHONY: project_list
project_list: all_projects_info.json
echo "Filtering project list with:" >&2
echo " PROJECT_FILTER: $(PROJECT_FILTER)" >&2
jq -r -S '.[] | select(
(.projectId | test("$(PROJECT_FILTER)"))
) | .projectId' $^ > $#
.PHONY: get_storage_info
get_storage_info: project_list
PROJECT_LIST=$$(cat $<)
$(MAKE) -f $(MKFILE) -j storage_info.json PROJECT_LIST="$$PROJECT_LIST"
all_projects_info.json:
curl -X GET https://toto/get_all_my_projects_info >$#
# here it's PHONY because we want to always rebuild it
.PHONY: storage_info.json
storage_info.json: $(STORAGE_INFO_JSON_FILES)
jq -s -S '[.[]?.items?[]?]' $(STORAGE_INFO_JSON_FILES) > $#
storage_info/:
mkdir -p $#
STORAGE_INFO_JSON_FILES=$(foreach project_name,$(PROJECT_LIST),storage_info/$(project_name).json)
$(STORAGE_INFO_JSON_FILES): storage_info/%.json: | storage_info/
curl \
-X GET \
"https://storage_api/list_s3?project=$*" \
2> /dev/null > $#
As you can see here, I've got 2 "command":
project_list witch list all project I can access too,
get_storage_info witch list all bucket in projects.
The trick here is because I've got a lot of projects and buckets, I may want to filter like this:
make get_storage_info PROJECT="foo"
And it will print ONLY bucket in project with foo in their name.
It's quit handy and fast (only the first time it may be slow, the time to get all informations).
What is bothering me, I've not found a better way than to call a sub make command (with the exact list of project to take into account).
Is it possible to express dynamic dependencies of a target ?
But something that can result from another target ?
Thanks.
I don't see anything wrong with invoking a submake. That's IMO the best way to do it, especially if you want to add -j to it.
It's not really possible to get rid of this easily. It's not the fact that you want to express a dynamic dependency: that can be done. The problem is you want the list of dependencies to be extracted from the results of running another rule. But that's not how make works: make always starts with the final target and works its way backwards. So, by the time you get around to building the prerequisite file, the target that depended on it has already been processed (not its recipe of course, but all the prerequisites).
Thanks for all your time and response -
Currently, we are using the nested build, multiple Makefiles, and individual subdirectories having their own Makefile, all are connected with a top-level Makefile. We are running
make xxxxx_yyyy_defconfig
make
this builds and creates an output file which is xxxxx.elf file. --- Till here everything works fine.
Now we are having multiple def-configs(around 50), I want to build all configurations using one "make all" command. is that possible?
This is not a simple case where we can put all "all: prog01 prog02 prog03" as every program needs to have a different configuration. Configuration can be achieved by using "make xxxxx_yyyy_defconfig". The output of "make config" is the .config file, which is used during the "make" command.
Based on .config file many variables are exported which is used at the subdirectory level.
So How can I build multiple configurations using a single "make all" command?
Environment - Ubuntu, Cross compile for ARM, output file xxxx.elf.
With the use of script and make file I am able to solve, But I have to solve only using Makefile.
in Makefile add one PHONY target
all:
./build_all.sh #shell script calling.
Created one shell script like this
#! /usr/bin/bash
echo "Make All"
for entry in `ls conf`; do
make $entry
wait
make
if [ $? -eq 0 ] ; then
for xxxfile in `ls xxx*_*` ; do
xxxdir=$(echo $xxxfile | cut -b yy-zz)
mkdir -p $xxxdir
mv $xxxfile $xxxdir/
done
else
break
fi
done
If you want to build several configurations you must do this out of tree in separate build directories (make O=/tmp/builds/foo foo_defconfig; make -C /tmp/builds/foo) to avoid conflicts. A shell script could do this as well as a Makefile but if you insist on using a Makefile you could try the following that assumes your source tree is in /src/kernel and you want to build configuration foo in /tmp/builds/foo; adapt to your needs:
$ pwd
/tmp/builds
$ cat Makefile
CONFIGS := uuuu_vvvv xxxx_yyyy ...
BUILD := /tmp/build
KERNEL := /src/kernel
.PHONY: $(CONFIGS) all
all: $(CONFIGS)
$(CONFIGS):
rm -rf $#
mkdir -p $(BUILD)/$#
$(MAKE) -C $(KERNEL) O=$(BUILD)/$# $#_defconfig
$(MAKE) -C $(BUILD)/$#
$ make
I need to do a makefile for run some programs. Every time I run that script all the file are processed also if the file are not changed. I'm sure there is a problem on my code but I don't understand where I made the mistakes.
RDIR=RAW
OUTDIR=Fusion_res/kallisto
RFILES:=$(wildcard $(RDIR)/*_R1_001.fastq.gz)
DATABASE=/home/sbsuser/databases/Kallsto_hg38_87
OUTFILE=$(patsubst %_R1_001.fastq.gz,%_R2_001.fastq.gz,$(RFILES))
OUTKAL=$(patsubst $(RDIR)/%_R1_001.fastq.gz,$(OUTDIR)/%,$(RFILES))
.PHONY: clean all
all: $(OUTFILE) $(RFILES) $(OUTDIR) $(OUTKAL)
#$(OUTKAL) $(OUTFILE): $(RDIR)/%._R1_001.fastq.gz
# echo "kallisto quant -i" $(DATABASE)/transcripts.idx -b 100 -o $# --fusion $< $(OUTFILE)
$(OUTDIR)/%: $(RDIR)/%_R1_001.fastq.gz $(OUTFILE)
kallisto quant -i $(DATABASE)/transcripts.idx -b 100 --fusion --rf-stranded -o $# $(RDIR)/$*_R1_00
1.fastq.gz $(RDIR)/$*_R2_001.fastq.gz
$(OUTDIR):
mkdir -p $(OUTDIR)
clean::
$(RM) -rf $(OUTDIR)
I suppose if the found some change on the input file and on the output execute the command. I don't know why every time force re-run. In some case Is that I want but I wan to also if there is some new input execute only that.
Thanks so much
A couple of things:
1) $(OUTDIR)/% is dependent on $(OUTFILE) (which is a list of all outfiles). Therefore if you change any one of the OUTFILEs, you make everything in $(OUTDIR)/% obsolete. I believe what you want is this:
$(OUTDIR)/%_R1_001.fastq.gz: $(RDIR)/%_R2_001.fastq.gz
.... (rules to make out/R1 from raw/R2
$(RDIR)/%_R2_001.fastq.gz: $(RDIR)/%_R1_001.fastq.gz
.... (rules to make R2 from R1
This makes each file dependent only on the files that effect it.
2) you have the target all dependent on $(OUTDIR) which is a directory. If you use parallel make, it may generate the $(OUTDIR) after it generates the other dependencies of all: (some of which would depend on $(OUTDIR) being created). What you want there is to remove all's dependency on $(OUTDIR), and add the line:
$(OUTFILE) : | $(OUTDIR)
Notice the |, which means order only (don't consider $(OUTFILE) out of date if $(OUTDIR) is newer. This is important, as a directory's timestamp is updated each time a file in the directory is changed, and so it tends to be newer than its contents.
I'm writing my first complex Makefile for a highly-modularized project.
I have various sub-directories, each one has its own Makefile which supports at least the all and the clean targets.
These sub-Makefiles work just fine, however I have a problem with the main Makefile, that should call all the sub-Makefiles automatically from the list contained in the variable COMPONENTS.
I tried with the following Makefile:
OUTFILE = diskimage.bin
export NASM = nasm
COMPONENTS = bootloader
.PHONY = all clean FORCE $(OUTFILE) $(COMPONENTS)
all: $(OUTFILE)
$(OUTFILE): $(COMPONENTS)
./make_image
$(COMPONENTS): FORCE
for component in $(COMPONENTS); do \
make -C $component; \
done
FORCE:
clean:
for component in $(COMPONENTS); do \
make -C $component clean; \
done
This results in the following error message:
for component in bootloader; do \
make -C omponent; \
done
make: *** omponent: No such file or directory. Stop.
make: *** [bootloader] Error 2
as if the $component expression was only parsed as $c. I don't understand why that happens and how to fix it.
Just double the dollar sign:
$(COMPONENTS): FORCE
for component in $(COMPONENTS); do \
make -C $$component; \
done
The trouble is that with your makefile, Make expands $component before executing the rule. And since $c has no value (there is no such variable), it expands to nothing, leaving "omponent", which it passes to she shell, which complains that there's no such directory. (If you had written $(component), Make would have expanded it to nothing, since Make knows of no such variable, and then the shell would have complained that you were not specifying a directory at all.)
With the double dollar sign, Make expands $$component to $component, which it then passes to the shell, which interprets it as the loop variable, and everything proceeds as planned.
You really should have played around with a simple loop in a command, before attempting to do actual work with one.
Several issues.
.PHONY should be written as a dependency, not a macro definition
Don't write shell loops, use make syntax instead
When you call make recursively, you must do it via the ${MAKE} macro invocation
Leading to
OUTFILE = diskimage.bin
export NASM = nasm
COMPONENTS = bootloader
.PHONY: all
all: ${OUTFILE}
.PHONY: ${OUTFILE}
${OUTFILE}: ${COMPONENTS}
./make_image
.PHONY: ${COMPONENTS}
${COMPONENTS}:
${MAKE} -C $#
The advantage of this formulation is that it is parallel make friendly.
Always a test of a good Makefile.
Here make -j5 all will cause make to keep 5 commands running at once,
across all invocations of make.
Nice if you have 4 CPUs.
What about clean?
(Personally I hate clean targets—it's a sign of dodgy dependencies,
and of unhygienic mixing of source and target folders.)
Just add -clean (say) to each of the component names,
and repeat the pattern above.
CLEANS := $(addsuxffix -clean,${COMPONENTS})
.PHONY: clean
clean: ${CLEANS} ; #echo Clean succesful
.PHONY: ${CLEANS}
${CLEANS}: %-clean:
${MAKE} -C $* clean
These two sections can tidied up and combined into one if you feel so inclined.
Tip
Always run make with --warn (or --warn-undefined-variables to give it its full name) to catch inadvertent expansion of $c in things like $component.
I'm trying to modify a makefile to cross-compile binaries. The command in question is below:
# GNU Make solution makefile autogenerated by Premake
# Type "make help" for usage help
ifndef config
config=debug
endif
export config
PROJECTS := json openjaus openjaus-core openjaus-environment openjaus-mobility openjaus-manipulator openjaus-ugv Base Managed PingTest LargeMessageTest PdDemo GposDemo GposClientDemo StillImageSensorDemo StillImageClientDemo
.PHONY: all clean help $(PROJECTS)
all: $(PROJECTS)
json:
#echo "==== Building json ($(config)) ===="
#${MAKE} --no-print-directory -C .build -f json.make
As can be seen the makefile has several targets. They all have the same structure as the 'json' target. The command in question in question is
#${MAKE} --no-print-directory -C .build -f json.make
The '${MAKE}' variable = make (I have verified this with echo)
What does the -C do?
What does the .build do?
I'm good with -f json.make
Also, when I run make the json.make file gets created compiles file and deletes it self, so I do not have access to that file.
The error I receive when I modify the command in question is
==== Building json (debug) ====
make[1]: Nothing to be done for `/home/botbear/openwrt/trunk/staging_dir/toolchain- arm_v6k_gcc-linaro_uClibc-0.9.32_eabi/bin/arm-openwrt-linux-c++'.
The command after modifications looks like:
#${MAKE} /home/botbear/openwrt/trunk/staging_dir/toolchain-arm_v6k_gcc-linaro_uClibc-0.9.32_eabi/bin/arm-openwrt-linux-c++ --no-print-directory -C .build -f json.make
Any help is appreciated!
you can use man make to understand the parameters for make:
-C dir, --directory=dir
Change to directory dir before reading the makefiles or doing any-
thing else. If multiple -C options are specified, each is inter-
preted relative to the previous one: -C / -C etc is equivalent to
-C /etc. This is typically used with recursive invocations of
make.
-f file, --file=file, --makefile=FILE
Use file as a makefile.
so -C .build changes to the directory .build.
and i don't understand the part of your question about modifying the command.
Try to find where json.make lives. It seems that it's that makefile which creates & deletes the directory you were talking about.
From the command line it seems that make changes directory to .build and executes the json.make. Not sure how json.make ends up there. Is .build the directory which is created and then deleted?