Doubts regarding the below MakeFile - makefile

I am new to nmake. I came across this Makefile for nmake. This is a legacy part of application that i need to support. This is working fine, its just that i had problems in understanding the syntax. I tried my best on google but still couldn't understand some intricacies.
I am curious about the concept of using 2 colons in the :INSTALL: rule below. Also not able to get the :B:S variable editing.
JAVA_FILES = \
Abc.java \
Def.java \
Ghi.java
/***********Define targets **********************/
.all : $(JAVA_FILES:B:S=.class)
:INSTALL: .all
$(JAVA_FILES:B:S=.class) : $(JAVA_FILES) .JOINT
for node in `echo $VPATH | tr ':' ' '`
do
nodecp=${nodecp}:${node}/gfp_scom:${node}/gfp_scom/lib/GfpScomWsdl.jar:${node}/gfp_scom/lib/jmxtools.jar
done
$(JAVAC) -cp ${nodecp}:$(CLASSPATH) -d ../../ $(*:M!=\.class)

Related

Bash code runs when directly called, but not when called and fed vars via a Makefile. But other bash files do work with the same structure, Error 126?

I had a makefile I used to create jobs in one step then submit them to a cluster in another, then finally merge them. This worked well and I had no issues with it, I was using a python script to break csv files down into smaller ones in the create-jobs step however, and it took longer than I'd like so I decided to add a step to do this with bash code instead, thinking it would be faster.
I laid the new step out in the Makefile in the exact same way as the others, and created a small bash script that would accept the name of the csv, how many parts to split it into and eventually I was going to add a variable for the location to place the output csves but I tried testing it with it just sticking them in the same directory as itself first. I first defined the variables that would later be passed on in the bash file so that I could check it worked by itself, it does. Those variables are commented out now but this is the bash file in question:
#!/bin/bash
#INPUT_DATA="/scratch/cb27g11/Looping_Bash/missing.csv"
#nJobs=2
tail -n +2 ${INPUT_DATA} | split -l ${nJobs} - split_
for file in split_*
do
head -n 1 ${INPUT_DATA} > tmp_file
cat $file >> tmp_file
mv -f tmp_file ../$file
done
Nothing super complicated.
The Makefile looks like this:
########################
### --- Split jobs - ###
########################
SPLIT_JOB_NAME = "Name_for_job"
SPLIT_JOB_INPUT_DATA = "/scratch/cb27g11/Looping_Bash/missing.csv"
SPLIT_JOB_HEADER = "header/default.header"
SPLIT_JOB_nJobs = 2
#######################
DECLARE_ROOT_DIR = ROOT_DIR="${THDM_T3PS_SCANNER_DIR}/job_submission/MadGraph/"
VAR_CREATE_JOB = $(shell echo '$(.VARIABLES)' | awk -v RS=' ' '/CREATE_JOB_/' | sed 's/CREATE_JOB_//g' )
EXPORT_CREATE_JOB = $(foreach v,$(VAR_CREATE_JOB),$(v)="$(CREATE_JOB_$(v))") $(DECLARE_ROOT_DIR)
VAR_SUBMIT_JOB = $(shell echo '$(.VARIABLES)' | awk -v RS=' ' '/SUBMIT_JOB_/' | sed 's/SUBMIT_JOB_//g' )
EXPORT_SUBMIT_JOB = $(foreach v,$(VAR_SUBMIT_JOB),$(v)="$(SUBMIT_JOB_$(v))") $(DECLARE_ROOT_DIR)
VAR_MERGE_JOB = $(shell echo '$(.VARIABLES)' | awk -v RS=' ' '/MERGE_JOB_/' | sed 's/MERGE_JOB_//g' )
EXPORT_MERGE_JOB = $(foreach v,$(VAR_MERGE_JOB),$(v)="$(MERGE_JOB_$(v))") $(DECLARE_ROOT_DIR)
VAR_SPLIT_JOB = $(shell echo '$(.VARIABLES)' | awk -v RS=' ' '/SPLIT_JOB_/' | sed 's/SPLIT_JOB_//g' )
EXPORT_SPLIT_JOB = $(foreach v,$(VAR_SPLIT_JOB),$(v)="$(SPLIT_JOB_$(v))") $(DECLARE_ROOT_DIR)
#########################################################
create-jobs:
#$(EXPORT_CREATE_JOB) ./utils/new_create-jobs.sh
submit-jobs:
#$(EXPORT_SUBMIT_JOB) ./utils/new_submit-jobs.sh
merge-jobs:
#$(EXPORT_MERGE_JOB) ./utils/merge-jobs.sh
split-jobs:
#$(EXPORT_SPLIT_JOB) ./utils/split-jobs.sh
I'm not currently using all of the variables set-up for split-jobs here but surely that doesn't matter? The other parts, create-jobs, submit-jobs and merge-jobs work just fine and I can't see that I've made a mistake in how I've set up split-jobs compared to them. So I guess there's something I'm missing about the bash file itself?
When I try to run make split-jobs I get this:
(THDM) [cb27g11#cyan53 MadGraph]$ make split-jobs
/bin/sh: ./utils/split-jobs.sh: Permission denied
make: *** [split-jobs] Error 126
Since this is on a shared cluster I don't have sudo rights or anything like that, could this be related to that somehow? Am I calling something wrong? Any help or suggestions would be much appreciated!
I can't really understand what all the shell commands are trying to do, but I'm completely confident that whatever they are doing there's a much simpler and more straightforward way to do it.
However, that doesn't have anything to do with your error, which is:
/bin/sh: ./utils/split-jobs.sh: Permission denied
Did you remember to give your shell script execute permissions:
chmod +x ./utils/split-jobs.sh
? Also, please remove the # from your makefile lines until AFTER your makefile is completely working, and show us the command line that make invoked (maybe once you see it, you will be able to figure out the problem for yourself). Using # is like trying to debug your program while sending all the output to /dev/null.

How to handle unknown dependencies in Makefile that are dynamically detected?

Let's say my Makefile is building a website. I write a index.html containing assets, like this:
<html>
<body><img src="asset/foo.img"/></body>
</html>
Now I want to write a rule to create a ZIP archive that contains the HTML as well as its assets.
I write a script to parse out the src= attributes, but what do I do with it?
If I put that in a recipe, but at that point the Makefile rules can't be changed.
If I put it in a $(shell ...) command, then it will have to run every time the Makefile is parsed, even if the user is doing a clean or something.
What is the proper way to handle such a scenario?
Actually I think I just figured it out myself. The solution is a sub-call to $(MAKE)β€”in fact, its primary difference compared to include seems to be the fact that it occurs at recipe execution time rather than at parse time.
Something like this, maybe:
assets := $(shell ./myscript index.html)
myzip.zip: index.html $(assets)
zip $# $^
I seem to miss your problem. Isn't it that a dependency tree of your situation looks like this:
my_page.html image1.bmp image2.bmp
| | | |
1 2 2 2
| | | |
| \ | /
| \ | /
| \ | /
| \ | /
| \ | /
| \ | /
depend.dep archive.zip
where "1" is the rule which scans my_page.html for all assets and writes them back to depend.dep (or the makefile itself, that doesn't change the solution) and "2" is updating the zip file with changed member files? As make is trying to re-make depend.dep (or makefile itself) prior to any other rule, you always have an up-to-date dependency list when starting a make-run which will do exactly as you command: add/update members to archive.zip if they are newer. I say it again: re-parsing my_page.html if it has changed relative to the dependencies is step zero in a make-run, if there exists a dependency towards a file which is included or makefile itself.

Print out all necessary for compilation files

I have bunch of Make files though directory hierarchy, some of them define C (CPP) and H files to use in compilations, some of them not, that are not adds to CPFLAFS -I../../SomeDir1 -I../SomeDir2/SomeDir2/Inc. etc.
Some of the Make files are called via make -C PATH Target .
The thing is hierarchy of sources grew for years and covers different projects and target platforms.
Now there are no people left in the company who remember "why that" and there is not enough documentation left.
The question: is there way to know particular sources and header files used to build the final target "all" ?
Thank you
There are ways, I know of two. Both require that you run make all, and before that remove all generated files so every file in question would actually be rebuilt (usually make clean).
Using strace
If strace is available, you can find the files by tracing and analyzing the system calls.
You can do it like this:
$ strace -f -o trace make all
$ grep 'open.*\.[ch]"' trace | grep -v 'open("/\(tmp\|usr\)' | sed -e 's/.*open("//' -e 's/".*//' | sort -u
Using Dependency Files (doesn't work with all compilers)
If your compiler can generate dependency files (i.e. gcc, armcc), you can analyze the dependency files. Make sure that dependency files are generated by putting the corresponding option in your CFLAGS or CPPFLAGS. For gcc, this would be -MMD. If Makefiles are written nicely, you can set CPPFLAGS on the command line, I will use this example here.
$ make all CPPFLAGS+=-MMD
$ find -name "*.d" -exec cat {} \; | sed -e 's/[:\\]//' -e 's/ /\n/g' | grep '\.[ch]$' | sort -u

hadoop file splitting using KeyFieldBasedPartitioner

I have a big file that is formatted as follows
sample name \t index \t score
And I'm trying to split this file based off of sample name using Hadoop Streaming.
I know ahead of time how many samples there are, so can specify how many reducers I need.
This post is doing something very similar, so I know that this is possible.
I tried using the following script to split this file into 16 files (there are 16 samples)
hadoop jar $STREAMING \
-D mapred.text.key.partitioner.options=-k1,1 \
-D stream.num.map.output.key.fields=2 \
-D mapred.reduce.tasks=16 \
-partitioner org.apache.hadoop.mapred.lib.KeyFieldBasedPartitioner \
-mapper cat \
-reducer org.apache.hadoop.mapred.lib.IdentityReducer \
-input input_dir/*part* -output output_dir
This somewhat works - some of the files contain only one sample name. However most of the part* files are blank and some of the part* files contain multiple sample names.
Is there a better way to make sure that every reducer gets only one sample name?
FYI, there is actually a much cleaner way to split up files using a custom OutputFormat
This link describes how to do this really well. I ended up tailoring this other link for my specific application. Altogether, its only a few extra lines of Java

How do you get the list of targets in a makefile?

I've used rake a bit (a Ruby make program), and it has an option to get a list of all the available targets, eg
> rake --tasks
rake db:charset # retrieve the charset for your data...
rake db:collation # retrieve the collation for your da...
rake db:create # Creates the databases defined in y...
rake db:drop # Drops the database for your curren...
...
but there seems to be no option to do this in GNU make.
Apparently the code is almost there for it, as of 2007 - http://www.mail-archive.com/help-make#gnu.org/msg06434.html.
Anyway, I made little hack to extract the targets from a makefile, which you can include in a makefile.
list:
#grep '^[^#[:space:]].*:' Makefile
It will give you a list of the defined targets. It's just a start - it doesn't filter out the dependencies, for instance.
> make list
list:
copy:
run:
plot:
turnin:
Under Bash (at least), this can be done automatically with tab completion:
make spacetabtab
Note: This answer has been updated to still work as of GNU make v4.3 - let us know if you come across something that breaks.
This is an attempt to improve on Brent Bradburn's great approach as follows:
uses a more robust command to extract the target names, which hopefully prevents any false positives (and also does away with the unnecessary sh -c)
does not invariably target the makefile in the current directory; respects makefiles explicitly specified with -f <file>
excludes hidden targets - by convention, these are targets whose name starts neither with a letter nor a digit
makes do with a single phony target
prefixes the command with # to prevent it from being echoed before execution
Curiously, GNU make has no feature for listing just the names of targets defined in a makefile. While the -p option produces output that includes all targets, it buries them in a lot of other information and also executes the default target (which could be suppressed with -f/dev/null).
Place the following rule in a makefile for GNU make to implement a target named list that simply lists all target names in alphabetical order - i.e.: invoke as make list:
.PHONY: list
list:
#LC_ALL=C $(MAKE) -pRrq -f $(lastword $(MAKEFILE_LIST)) : 2>/dev/null | awk -v RS= -F: '/(^|\n)# Files(\n|$$)/,/(^|\n)# Finished Make data base/ {if ($$1 !~ "^[#.]") {print $$1}}' | sort | egrep -v -e '^[^[:alnum:]]' -e '^$#$$'
Important: On pasting this, make sure that the last line is indented by exactly 1 actual tab char. (spaces do not work).
Note that sorting the resulting list of targets is the best option, since not sorting doesn't produce a helpful ordering in that the order in which the targets appear in the makefile is not preserved.
Also, the sub-targets of a rule comprising multiple targets are invariably output separately and will therefore, due to sorting, usually not appear next to one another; e.g., a rule starting with a z: will not have targets a and z listed next to each other in the output, if there are additional targets.
Explanation of the rule:
.PHONY: list
declares target list a phony target, i.e., one not referring to a file, which should therefore have its recipe invoked unconditionally
LC_ALL=C makes sure that make's output in in English, as parsing of the output relies on that.Tip of the hat to Bastian Bittorf
$(MAKE) -pRrq -f $(lastword $(MAKEFILE_LIST)) : 2>/dev/null
Invokes make again in order to print and parse the database derived from the makefile:
-p prints the database
-Rr suppresses inclusion of built-in rules and variables
-q only tests the up-to-date-status of a target (without remaking anything), but that by itself doesn't prevent execution of recipe commands in all cases; hence:
-f $(lastword $(MAKEFILE_LIST)) ensures that the same makefile is targeted as in the original invocation, regardless of whether it was targeted implicitly or explicitly with -f ....
Caveat: This will break if your makefile contains include directives; to address this, define variable THIS_FILE := $(lastword $(MAKEFILE_LIST)) before any include directives and use -f $(THIS_FILE) instead.
: is a deliberately invalid target that is meant to ensure that no commands are executed; 2>/dev/null suppresses the resulting error message. Note: This relies on -p printing the database nonetheless, which is the case as of GNU make 3.82. Sadly, GNU make offers no direct option to just print the database, without also executing the default (or given) task; if you don't need to target a specific Makefile, you may use make -p -f/dev/null, as recommended in the man page.
-v RS=
This is an awk idiom that breaks the input into blocks of contiguous non-empty lines.
/(^|\n)# Files(\n|$$)/,/(^|\n)# Finished Make data base/
Matches the range of lines in the output that contains all targets, across paragraphs - by limiting parsing to this range, there is no need to deal with false positives from other output sections.
Note: Between make versions 3.x and 4.3, paragraph structuring in make's output changed, so (^|\n) / (\n|$$) ensures that the lines that identify the start and the end of the cross-paragraph range of lines of interest are detected irrespective of whether they occur at the start or inside / at the end of a paragraph.
if ($$1 !~ "^[#.]")
Selectively ignores blocks:
# ... ignores non-targets, whose blocks start with # Not a target:
. ... ignores special targets
All other blocks should each start with a line containing only the name of an explicitly defined target followed by :
egrep -v -e '^[^[:alnum:]]' -e '^$#$$' removes unwanted targets from the output:
'^[^[:alnum:]]' ... excludes hidden targets, which - by convention - are targets that start neither with a letter nor a digit.
'^$#$$' ... excludes the list target itself
Running make list then prints all targets, each on its own line; you can pipe to xargs to create a space-separated list instead.
This obviously won't work in many cases, but if your Makefile was created by CMake you might be able to run make help.
$ make help
The following are some of the valid targets for this Makefile:
... all (the default if no target is provided)
... clean
... depend
... install
etc
I combined these two answers: https://stackoverflow.com/a/9524878/86967 and https://stackoverflow.com/a/7390874/86967
and did some escaping so that this could be used from inside a makefile.
.PHONY: no_targets__ list
no_targets__:
list:
sh -c "$(MAKE) -p no_targets__ | awk -F':' '/^[a-zA-Z0-9][^\$$#\/\\t=]*:([^=]|$$)/ {split(\$$1,A,/ /);for(i in A)print A[i]}' | grep -v '__\$$' | sort"
.
$ make -s list
build
clean
default
distclean
doc
fresh
install
list
makefile ## this is kind of extraneous, but whatever...
run
As mklement0 points out, a feature for listing all Makefile targets is missing from GNU-make, and his answer and others provides ways to do this.
However, the original post also mentions rake, whose tasks switch does something slightly different than just listing all tasks in the rakefile. Rake will only give you a list of tasks that have associated descriptions. Tasks without descriptions will not be listed. This gives the author the ability to both provide customized help descriptions and also omit help for certain targets.
If you want to emulate rake's behavior, where you provide descriptions for each target, there is a simple technique for doing this: embed descriptions in comments for each target you want listed.
You can either put the description next to the target or, as I often do, next to a PHONY specification above the target, like this:
.PHONY: target1 # Target 1 help text
target1: deps
[... target 1 build commands]
.PHONY: target2 # Target 2 help text
target2:
[... target 2 build commands]
...
.PHONY: help # Generate list of targets with descriptions
help:
#grep '^.PHONY: .* #' Makefile | sed 's/\.PHONY: \(.*\) # \(.*\)/\1 \2/' | expand -t20
Which will yield
$ make help
target1 Target 1 help text
target2 Target 2 help text
...
help Generate list of targets with descriptions
You can also find a short code example in this gist and here too.
Again, this does not solve the problem of listing all the targets in a Makefile. For example, if you have a big Makefile that was maybe generated or that someone else wrote, and you want a quick way to list its targets without digging through it, this won't help.
However, if you are writing a Makefile, and you want a way to generate help text in a consistent, self-documenting way, this technique may be useful.
My favorite answer to this was posted by Chris Down at Unix & Linux Stack Exchange. I'll quote.
This is how the bash completion module for make gets its list:
make -qp | awk -F':' '/^[a-zA-Z0-9][^$#\/\t=]*:([^=]|$)/ {split($1,A,/ /);for(i in A)print A[i]}'
It prints out a newline-delimited list of targets, without paging.
User Brainstone suggests piping to sort -u to remove duplicate entries:
make -qp | awk -F':' '/^[a-zA-Z0-9][^$#\/\t=]*:([^=]|$)/ {split($1,A,/ /);for(i in A)print A[i]}' | sort -u
Source: How to list all targets in make? (Unix&Linux SE)
If you have bash completion for make installed, the completion script will define a function _make_target_extract_script. This function is meant to create a sed script which can be used to obtain the targets as a list.
Use it like this:
# Make sure bash completion is enabled
source /etc/bash_completion
# List targets from Makefile
sed -nrf <(_make_target_extract_script --) Makefile
Focusing on an easy syntax for describing a make target, and having a clean output, I chose this approach:
help:
#grep -B1 -E "^[a-zA-Z0-9_-]+\:([^\=]|$$)" Makefile \
| grep -v -- -- \
| sed 'N;s/\n/###/' \
| sed -n 's/^#: \(.*\)###\(.*\):.*/\2###\1/p' \
| column -t -s '###'
#: Starts the container stack
up: a b
command
#: Pulls in new container images
pull: c d
another command
make-target-not-shown:
# this does not count as a description, so leaving
# your implementation comments alone, e.g TODOs
also-not-shown:
So treating the above as a Makefile and running it gives you something like
> make help
up Starts the container stack
pull Pulls in new container images
Explanation for the chain of commands:
First, grep all targets and their preceeding line, see https://unix.stackexchange.com/a/320709/223029.
Then, get rid of the group separator, see https://stackoverflow.com/a/2168139/1242922.
Then, we collapse each pair of lines to parse it later, see https://stackoverflow.com/a/9605559/1242922.
Then, we parse for valid lines and remove those which do not match, see https://stackoverflow.com/a/8255627/1242922, and also give the output our desired order: command, then description.
Lastly, we arrange the output like a table.
Add this target to your Makefile:
help:
#echo "\nTARGETS:\n"
#make -qpRr | egrep -e '^[a-z].*:$$' | sed -e 's~:~~g' | sort
#echo ""
make -qpRr = make --question --print-data-base --no-builtin-variables --no-builtin-rules
egrep -e '^[a-z].*:$$': searches for lines which start with lowercase and ends with ":"
sed -e 's~:~~g': deletes the ":"
Then just run:
make help
This works for me πŸ˜‰
PD: more info at...
make --help
I took a few answers mentioned above and compiled this one, which can also generate a nice description for each target and it works for targets with variables too.
Example Makefile:
APPS?=app1 app2
bin: $(APPS:%=%.bin)
## Help: A composite target that relies only on other targets
$(APPS:%=%.bin): %.bin:
## Help: A target with variable name, value = $*
test:
## Help: A normal target without variables
# A target without any help description
clean:
# A hidden target
.hidden:
help:
#printf "%-20s %s\n" "Target" "Description"
#printf "%-20s %s\n" "------" "-----------"
#make -pqR : 2>/dev/null \
| awk -v RS= -F: '/^# File/,/^# Finished Make data base/ {if ($$1 !~ "^[#.]") {print $$1}}' \
| sort \
| egrep -v -e '^[^[:alnum:]]' -e '^$#$$' \
| xargs -I _ sh -c 'printf "%-20s " _; make _ -nB | (grep -i "^# Help:" || echo "") | tail -1 | sed "s/^# Help: //g"'
Example output:
$ make help
Target Description
------ -----------
app1.bin A target with variable name, value = app1
app2.bin A target with variable name, value = app2
bin A composite target that relies only on other targets
clean
test A normal target without variables
How does it work:
The top part of the make help target works exactly as posted by mklement0 here - How do you get the list of targets in a makefile?.
After getting the list of targets, it runs make <target> -nB as a dry run for each target and parses the last line that starts with ## Help: for the description of the target. And that or an empty string is printed in a nicely formatted table.
As you can see, the variables are even expanded within the description as well, which is a huge bonus in my book :).
#nobar's answer helpfully shows how to use tab completion to list a makefile's targets.
This works great for platforms that provide this functionality by default (e.g., Debian, Fedora).
On other platforms (e.g., Ubuntu) you must explicitly load this functionality, as implied by #hek2mgl's answer:
. /etc/bash_completion installs several tab-completion functions, including the one for make
Alternatively, to install only tab completion for make:
. /usr/share/bash-completion/completions/make
For platforms that don't offer this functionality at all, such as OSX, you can source the following commands (adapated from here) to implement it:
_complete_make() { COMPREPLY=($(compgen -W "$(make -pRrq : 2>/dev/null | awk -v RS= -F: '/^# File/,/^# Finished Make data base/ {if ($1 !~ "^[#.]") {print $1}}' | egrep -v '^[^[:alnum:]]' | sort | xargs)" -- "${COMP_WORDS[$COMP_CWORD]}")); }
complete -F _complete_make make
Note: This is not as sophisticated as the tab-completion functionality that comes with Linux distributions: most notably, it invariably targets the makefile in the current directory, even if the command line targets a different makefile with -f <file>.
This help target will only print targets which have ## followed by a description. This allows for documenting both public and private targets. Using the .DEFAULT_GOAL makes the help more discoverable.
Only sed, xargs and printf used which are pretty common.
Using the < $(MAKEFILE_LIST) allows for the makefile to be called something other than Makefile for instance Makefile.github
You can customize the output to suit your preference in the printf. This example is set up to match the OP's request for rake style output
When cutting and pasting the below make file, don't forget to change the 4 spaces indentation to tabs.
# vim:ft=make
# Makefile
.DEFAULT_GOAL := help
.PHONY: test help
help: ## these help instructions
#sed -rn 's/^([a-zA-Z_-]+):.*?## (.*)$$/"\1" "\2"/p' < $(MAKEFILE_LIST) | xargs printf "make %-20s# %s\n"
lint: ## style, bug and quality checker
pylint src test
private: # for internal usage only
#true
test: private ## run pytest with coverage
pytest --cov test
Here is the output from the Makefile above. Notice the private target doesn't get output because it only has a single # for it's comment.
$ make
make help # these help instructions
make lint # style, bug and quality checker
make test # run pytest with coverage
This is far from clean, but did the job, for me.
make -p 2&>/dev/null | grep -A 100000 "# Files" | grep -v "^$" | grep -v "^\(\s\|#\|\.\)" | grep -v "Makefile:" | cut -d ":" -f 1
I use make -p that dumps the internal database, ditch stderr, use a quick and dirty grep -A 100000 to keep the bottom of the output. Then I clean the output with a couple of grep -v, and finally use cut to get what's before the colon, namely, the targets.
This is enough for my helper scripts on most of my Makefiles.
EDIT: added grep -v Makefile that is an internal rule
For a Bash Script
Here's a very simple way to do this in bash -- based on the comment by #cibercitizen1 above:
grep : Makefile | awk -F: '/^[^.]/ {print $1;}'
See also the more authoritative answer by #Marc.2377, too, which says how the Bash completion module for make does it.
To expand on the answer given by #jsp, you can even evaluate variables in your help text with the $(eval) function.
The proposed version below has these enhanced properties:
Will scan any makefiles (even included)
Will expand live variables referenced in the help comment
Adds documentation anchor for real targets (prefixed with # TARGETDOC:)
Adds column headers
So to document, use this form:
RANDOM_VARIABLE := this will be expanded in help text
.PHONY: target1 # Target 1 help with $(RANDOM_VARIABLE)
target1: deps
[... target 1 build commands]
# TARGETDOC: $(BUILDDIR)/real-file.txt # real-file.txt help text
$(BUILDDIR)/real-file.txt:
[... $(BUILDDIR)/real-file.txt build commands]
Then, somewhere in your makefile:
.PHONY: help # Generate list of targets with descriptions
help:
## find all help in targets and .PHONY and evaluate the embedded variables
$(eval doc_expanded := $(shell grep -E -h '^(.PHONY:|# TARGETDOC:) .* #' $(MAKEFILE_LIST) | sed -E -n 's/(\.PHONY|# TARGETDOC): (.*) # (.*)/\2 \3\\n/'p | expand -t40))
#echo
#echo ' TARGET HELP' | expand -t40
#echo ' ------ ----' | expand -t40
#echo -e ' $(doc_expanded)'
make doesn't support this by default and other answers have shown how to extract the list of possible targets automatically.
However, in case you want to have more control with the listing without any side-effects (such as using the .PHONY target to mark the documentation which prevents the logic of using the target names as actual files which Make uses to decide which targets needs to be rebuilt), you can invent your own syntax just for the documentation. I prefer to use ### like this:
CPUS ?= $(shell nproc)
MAKEFLAGS += -j $(CPUS) -l $(CPUS) -s
# Basic paths
PREFIX ?= usr
BINDIR ?= $(PREFIX)/bin
ETCDIR ?= etc
MANDIR ?= $(PREFIX)/share/man
# ...
### help: Show help message (default target)
# use "help" as the default target (first target in the Makefile)
.PHONY: help
help:
#printf "%s\n\n" "make: List of possible targets:"
#grep '^### .*:' $(lastword $(MAKEFILE_LIST)) | sed 's/^### \([^:]*\): \(.*\)/\1:\t\2/' | column -ts "$$(printf '\t')"
### install: Install all files in $PREFIX (used by debian binary package build scripts)
install:
install -D -o root -g root -m 755 ...
...
### release: Increase package version number
release:
debchange --release
(as usual, the indented files must start with exactly one tabulator but stackoverflow cannot reproduce that detail correctly.)
Output will look like this:
$ make
make: List of possible targets:
help: Show help message (default target)
install: Install all files in $PREFIX (used by debian binary package build scripts)
release: Increase package version number
This works because only lines starting with ### and having a : character are considered as the documentation to output. Note that this intentionally does not extract the actual target name but fully trusts the documentation lines only. This allows always emitting correct output for very complex Makefile tricks, too. Also note that this avoids needing to put the documentation line on any specific position relative to actual rule. I also intentionally avoid sorting the output because the order of output can be fully controlled from the Makefile itself simply by listing the documentation lines in preferred order.
You could obviously invent any other syntax you like and even do something like
### en: install: Install all files in $PREFIX
### fi: asennus: asenna kaikki tiedostot hakemistoon $PREFIX
and only print lines that match the current locale to support multiple languages and having aliases to localize the target names, too:
.PHONY: asennus
asennus: install
The most important question is why do you want to list the targets? Do you want actual documentation or some kind of debugging information?
This is a modification to jsp's very helpful answer (https://stackoverflow.com/a/45843594/814145). I like the idea of getting not only a list of targets but also their descriptions. jsp's Makefile puts the description as the comment, which I found often will be repeated in the target's description echo command. So instead, I extract the description from the echo command for each target.
Example Makefile:
.PHONY: all
all: build
: "same as 'make build'"
.PHONY: build
build:
#echo "Build the project"
.PHONY: clean
clean:
#echo "Clean the project"
.PHONY: help
help:
#echo -n "Common make targets"
#echo ":"
#cat Makefile | sed -n '/^\.PHONY: / h; /\(^\t#*echo\|^\t:\)/ {H; x; /PHONY/ s/.PHONY: \(.*\)\n.*"\(.*\)"/ make \1\t\2/p; d; x}'| sort -k2,2 |expand -t 20
Output of make help:
$ make help
Common make targets:
make all same as 'make build'
make build Build the project
make clean Clean the project
make help Common make targets
Notes:
Same as jsp's answer, only PHONY targets may be listed, which may or may not work for your case
In addition, it only lists those PHONY targets that have a echo or : command as the first command of the recipe. : means "do nothing". I use it here for those targets that no echo is needed, such as all target above.
There is an additional trick for the help target to add the ":" in the make help output.
Plenty of workable solutions here, but as I like saying, "if it's worth doing once, it's worth doing again."
I did upvote the sugestion to use (tab)(tab), but as some have noted, you may not have completion support, or, if you have many include files, you may want an easier way to know where a target is defined.
I have not tested the below with sub-makes...I think it wouldn't work. As we know, recursive makes considered harmful.
.PHONY: list ls
ls list :
## search all include files for targets.
## ... excluding special targets, and output dynamic rule definitions unresolved.
#for inc in $(MAKEFILE_LIST); do \
echo ' =' $$inc '= '; \
grep -Eo '^[^\.#[:blank:]]+.*:.*' $$inc | grep -v ':=' | \
cut -f 1 | sort | sed 's/.*/ &/' | sed -n 's/:.*$$//p' | \
tr $$ \\\ | tr $(open_paren) % | tr $(close_paren) % \
; done
# to get around escaping limitations:
open_paren := \(
close_paren := \)
Which I like because:
list targets by include file.
output raw dynamic target definitions (replaces variable delimiters with modulo)
output each target on a new line
seems clearer (subjective opinion)
Explanation:
foreach file in the MAKEFILE_LIST
output the name of the file
grep lines containing a colon, that are not indented, not comments, and don't start with a period
exclude immediate assignment expressions (:=)
cut, sort, indent, and chop rule-dependencies (after colon)
munge variable delimiters to prevent expansion
Sample Output:
= Makefile =
includes
ls list
= util/kiss/snapshots.mk =
rotate-db-snapshots
rotate-file-snapshots
snap-db
snap-files
snapshot
= util/kiss/main.mk =
dirs
install
%MK_DIR_PREFIX%env-config.php
%MK_DIR_PREFIX%../srdb
This one was helpful to me because I wanted to see the build targets required (and their dependencies) by the make target. I know that make targets cannot begin with a "." character. I don't know what languages are supported, so I went with egrep's bracket expressions.
cat Makefile | egrep "^[[:alnum:][:punct:]]{0,}:[[:space:]]{0,}[[:alnum:][:punct:][:space:]]{0,}$"
Yet another additional answer to above.
tested on MacOSX using only cat and awk on terminal
cat Makefile | awk '!/SHELL/ && /^[A-z]/ {print $1}' | awk '{print substr($0, 1, length($0)-1)}'
will output of the make file like below:
target1
target2
target3
in the Makefile, it should be the same statement, ensure that you escape the variables using $$variable rather than $variable.
Explanation
cat - spits out the contents
| - pipe parses output to next awk
awk - runs regex excluding "shell" and accepting only "A-z" lines then prints out the $1 first column
awk - yet again removes the last character ":" from the list
this is a rough output and you can do more funky stuff with just AWK. Try to avoid sed as its not as consistent in BSDs variants i.e. some works on *nix but fails on BSDs like MacOSX.
More
You should be able add this (with modifications) to a file for make, to the default bash-completion folder /usr/local/etc/bash-completion.d/
meaning when you "make tab tab" .. it will complete the targets based on the one liner script.
For AWK haters, and for simplicity, this contraption works for me:
help:
make -qpRr $(lastword $(MAKEFILE_LIST)) | egrep -v '(^(\.|:|#|\s|$)|=)' | cut -d: -f1
(for use outside a Makefile, just remove $(lastword ...) or replace it with the Makefile path).
This solution will not work if you have "interesting" rule names but will work well for most simple setups. The main downside of a make -qp based solution is (as in other answers here) that if the Makefile defines variable values using functions - they will still be executed regardless of -q, and if using $(shell ...) then the shell command will still be called and its side effects will happen. In my setup often the side effects of running shell functions is unwanted output to standard error, so I add 2>/dev/null after the make command.
I usually do:
grep install_targets Makefile
It would come back with something like:
install_targets = install-xxx1 install-xxx2 ... etc
I hope this helps
tl;dr I personally copy-paste the same help target for every Makefile I build.
.SILENT:
.PHONY: help
## This help screen
help:
printf "Available targets\n\n"
awk '/^[a-zA-Z\-\_0-9]+:/ { \
helpMessage = match(lastLine, /^## (.*)/); \
if (helpMessage) { \
helpCommand = substr($$1, 0, index($$1, ":")-1); \
helpMessage = substr(lastLine, RSTART + 3, RLENGTH); \
printf "%-30s %s\n", helpCommand, helpMessage; \
} \
} \
{ lastLine = $$0 }' $(MAKEFILE_LIST)
I also maintain a copy of it in this Github gist:
https://gist.github.com/Olshansk/689fc2dee28a44397c6e31a0776ede30
Very simple AWK solution:
all:
#awk -F'[ :]' '!/^all:/ && /^([A-z_-]+):/ {print "make " $$1}' Makefile
(Note: This doesn't cover all the corner-cases as the accepted answer, as explained here.)
Try this one:
make -qp | awk -F':' '/^[^ \t.%][-A-Za-z0-9_]*:/ {split($0,A,/ /);for(i in A)if(match(A[i],/^[^.%][-A-Za-z0-9_]*/))print substr(A[i],1,RLENGTH)}' | sort -u
This is a very simplified version of what the bash-completion script does.
make -npq : 2> /dev/null | \
awk -v RS= -F: '$1 ~ /^[^#%]+$/ { print $1 }'
Explanation:
make -npq: Print the database without executing anything
-v RS=: Separate records by whole paragraphs
-F:: Separate fields by : (so the rule name is $1)
$1 ~ /^[^#%]+$/: Match rules that don't contain # or % (comments or pattern rules)
{ print $1 }: Print the rule name
This is much simpler than mklement0's approach (which I fixed myself), and works better.
not sure why the previous answer was so complicated:
list:
cat Makefile | grep "^[A-z]" | awk '{print $$1}' | sed "s/://g"

Resources