I have the following Makefile WORKING fragment:
LOWER_PKG := skeleton-init-common toolchain-external-custom argp-standalone cpio dosfstools ncurses libzlib util-linux e2fsprogs iputils xz kmod lzo mtd busybox bzip2 dhcpcd d
LOWER_OVR := $(BR2_EXTERNAL_VOCORE2_PATH)/overlay
LOWER_ADD :=.
define ROOTFS_LOWER_REMOVE
echo "ooo $(LOWER_OVR)"
echo "ppp $(LOWER_PKG)"
echo "aaa $(LOWER_ADD)"
endef
If I split the first long line using normal "\newline" syntax it doesn't work anymore (all variables are printed empty, not just last one:
LOWER_PKG := skeleton-init-common toolchain-external-custom \
argp-standalone cpio dosfstools ncurses libzlib \
util-linux e2fsprogs iputils xz kmod lzo mtd busybox \
bzip2 dhcpcd
LOWER_OVR := $(BR2_EXTERNAL_VOCORE2_PATH)/overlay
LOWER_ADD :=.
define ROOTFS_LOWER_REMOVE
echo "ooo $(LOWER_OVR)"
echo "ppp $(LOWER_PKG)"
echo "aaa $(LOWER_ADD)"
endef
Note: I have TAB chars at start of line, but I don't know how to type them here, so I substituted 8 blanks.
What am I doing wrong?
That is absolutely the only change: I had a non working Makefile, joined the lines and it started doing its work just fine. I'm very baffled.
Note2: this fragment is a configuration part of a Buildroot compilation, which is a behemoth completely out of my control (if it matters).
Related
I'm trying to reproduce a situation which happens only in some machines. To reproduce it, I create a directory with 2000 files:
mkdir /tmp/test
cd /tmp/test
for f in $(seq 1 2000); do touch $f.txt; done
Then I use the following Makefile (simplified from the real use case):
FILES:=$(shell find . -name '*.txt')
%.done: %.txt
#echo "done $#"
toolong:
#$(foreach file,$(sort $(FILES)), \
if $(MAKE) $(file); \
then echo "did $(file)" >> $#; \
else echo "failed $(file)" >> $#; fi; )
Running make produces, unsurprisingly, an error:
make: execvp: /bin/sh: Argument list too long
This question presents a solution which does work. However, I need to understand exactly why this error does not happen on my colleagues' computers. I tried the following things:
Increasing stack limit (ulimit -s gives the same result on both machines, 8192, and increasing it does not change anything);
Checking getconf ARG_MAX (2097152 in both machines);
Checking MAX_ARG_STRLEN (131072 in both machines);
Using a different shell (zsh is being used in both machines; I also tried bash, dash and sh, via export SHELL=<shell> make, and also by replacing the symlink /bin/sh -> /bin/bash with a link to dash).
Finally, I tried recompiling Make from source, and realized that, even when I compile the same version of Make (4.1) in my Ubuntu test machine, I get the same behavior as I had in my Fedora, that is, the error "argument list too long".
make --version only shows a single difference between them:
Version from the apt package:
GNU Make 4.1
Built for x86_64-pc-linux-gnu
Version compiled from source (./configure && make):
GNU Make 4.1
Built for x86_64-unknown-linux-gnu
I even tried compiling make-dfsg, which results in an identical make --version, but the result is the same as with my other manually-compiled make.
By increasing the number of files on Ubuntu, I managed to identify that the actual limits in the size of the generated command line are:
Fedora or Arch Linux (both with Make 4.2.1), or Ubuntu with manually-compiled Make 4.1: 128 KB (~1200 files);
Debian Sid or Ubuntu, both with Make 4.1 installed from apt package: 2 MB (~19300 files).
I'd really like to understand (1) why this difference exists, and (2) how could I compile Make to obtain the higher limit, so that I can have the exact same behavior on both machines.
Your recipe expands as a one-line shell compound command (because of the line ending backslashes). And this line is probably way too long. Did you check what it looks like (the shell compound command) on all machines? I suggest that you wrap it in $(info...):
toolong:
#$(info $(foreach file,$(sort $(FILES)), \
if $(MAKE) $(file); \
then echo "did $(file)" >> $#; \
else echo "failed $(file)" >> $#; fi; ))
and see if the output is the same on your machine and on the others. If, for any reason, yours is longer, it could be the explanation. Else it must be a OS difference...
Note: you could have one rule per target, instead of one single rule with a huge recipe for all.
Note: your recipe (independently from its fantastic length) does nothing useful. As the files already exist and do not have pre-requisites, all your sub-make calls will just tell you that the files are up-to-date.
I have a makefile including the following lines:
buildrepo:
#$(call make-repo)
define make-repo
for dir in $(C_SRCS_DIR); \
do \
mkdir -p $(OBJDIR)/$$dir; \
done
endef
On the line with the commands for dir in $(C_SRCS_DIR); \ I get the following error message:
"dir not expected at this moment"
make: *** [buildrepo] Error 255
I am using GNU make.
Can anybody tell me what is going wrong?
Actually this for ... in ... ; do ... done statement is a Unix command not a GNU make command, therefore I guess you are using a Windows machine (or any other one). You have to find the equivalent for your system.
But GNU make has a foreach function which works like this :
$(foreach dir,$(C_SRCS_DIR),mkdir -p $(OBJDIR)/$(dir);)
Also note that in your very specific case (not related to GNU make but to Windows) you can create all the dirs without a for/foreach loop, just like this :
mkdir -p $(addprefix $(OBJDIR)/,$(C_SRCS_DIR))
Here's what I have so far. I'm not looking for RVM-specific answers – I need to be able to replace rvm with, say, evm. Just wanted to pick something everyone would be familiar with :)
install_rvm:
ifeq ("`which rvm > /dev/null; echo $?`", "0")
#echo "rvm already installed!"
else
#echo "installing rvm..."
install_rvm_cmd
endif
Edit
Sorry for not saying this up-front, but I would like install_rvm_cmd to be run as it would normally – I don't want to hide the install process from the user.
I would do it like this:
.PHONY: install_rvm
install_rvm:
#if which rvm > /dev/null; then \
echo "rvm already installed!"; \
else \
echo "installing rvm..."; \
install_rvm_cmd; \
fi
It would be possible to use ifeq but it would entail performing the which rvm check even if install_rvm is not an actual target in a specific run of make. (I'm assuming the general case in which install_rvm is just one target among many.)
Although the string install_rmv_cmd itself wont' be output by make, its output will be output as usual. If you really want to have install_rmv_cmd out on stdout, you can always add echo install_rmv_cmd just before the command itself. To avoid repetition you could have the else branch be:
echo "installing rvm..."; \
cmd=install_rmv_cmd; \
echo $$cmd; \
$$cmd; \
In addition to Louis's answer, you can use ifdef. For example:
EVM_LOCATION := $(shell which evm)
CASK_LOCATION := $(shell which cask)
install_cask:
ifdef CASK_LOCATION
$(info cask is already installed!)
else
curl -fsSkL https://raw.github.com/cask/cask/master/go | python
endif
install_evm:
ifdef EVM_LOCATION
$(info evm is already installed!)
else
curl -fsSkL https://raw.github.com/rejeep/evm/master/go | bash
endif
You could use something like this (untested) which uses a stamp file to avoid doing any work once it has been done.
install_rvm_stamp:
__rvm=$$(command -v rvm);\
if [ -z "$$__rvm" ] || ! "$$__rvm" --version >/dev/null 2>&1; then \
echo "installing rvm..."; \
install_rvm_cmd; \
else \
echo 'rvm already installed!'
fi
touch '$#'
.PHONY: install_rvm
install_rvm: install_rvm_stamp
Alternatively, if you don't mind not "function testing" the binary you could do something like this instead.
rvm_bin := $(or $(realpath /usr/local/bin/rvm),$(realpath /bin/rvm),$(realpath /usr/bin/rvm),do_install_rvm)
install_rvm: $(rvm_bin)
do_install_rvm:
#echo "installing rvm..."
install_rvm_cmd
One final comment, as an augmentation to the make-level ifdef option presented in your answer; if the targets in question are only intended to be run manually (and are not used as prerequisites for any other targets) then the extraneous which calls can be avoided with some extra make-level checking.
ifneq (,$(findstring install_evm,$(MAKECMDGOALS)))
EVM_LOCATION := $(shell which evm)
endif
Wrap that in the following and you can avoid undefined variable warnings about MAKECMDGOALS if --warn-undefined-variables is used.
ifneq (undefined,$(origin MAKECMDGOALS))
....
endif
Bash tab completion adds extra space after the first completion which stops further completion if the compeletion target is a file in multi-level folders.
For example, I have a file in the path ~/Documents/foo/bar.txt, and I want to list it.
I face the following problem, when input
a#b:~$ls Docu <TAB>
I get
a#b:~$ls Documents |(<-this is the cursor, so there is an extra space afer Documents)
So I cannot further tab complete. I have to backspace to delete the extra space.
Normally I want to get:
a#b:~$ls Docu <TAB>
a#b:~$ls Documents/ <TAB>
a#b:~$ls Documents/foo/ <TAB>
a#b:~$ls Documents/foo/bar.txt
Just for the record: There is also a bug in the adobereader-enu (acroread) package that breaks bash completion. In this case you can just delete the symlink:
rm /etc/bash_completion.d/acroread.sh
See also: https://bugs.launchpad.net/ubuntu/+source/acroread/+bug/769866
I have had this same problem with my bash completion in both Ubuntu 11.10 and 12.04. I found that I was able to get many commands to start working correctly by editing /etc/bash_completion. Specifically, I commented out the following section:
####
# makeinfo and texi2dvi are defined elsewhere.
#
#for i in a2ps awk bash bc bison cat colordiff cp csplit \
# curl cut date df diff dir du enscript env expand fmt fold gperf gprof \
# grep grub head indent irb ld ldd less ln ls m4 md5sum mkdir mkfifo mknod \
# mv netstat nl nm objcopy objdump od paste patch pr ptx readelf rm rmdir \
# sed seq sha{,1,224,256,384,512}sum shar sort split strip tac tail tee \
# texindex touch tr uname unexpand uniq units vdir wc wget who; do
# have $i && complete -F _longopt -o default $i
#done
Now ls works well again. I have not figured out yet why mv is still mis-behaving.
This has been answered here at askubuntu. It is related to the bug here
Relevant answer from the above thread:
edit /etc/bash_completion line 1587, change default to filenames (make a backup first).
i also got around the problem by changing
_filedir with _filedir_pdf
in /etc/bash_completion.d/acroread.sh
(Ubuntu 12.04)
acroread bash completion changes the _filedir function thereby altering the behaviour of a lot of other alsobash completion functions
How to get the name of the makefile in the makefile?
Thanks.
Note:
I would need that because I would like my makefile to call itself, but the makefile is not called Makefile, so I'd like to write something like this:
target:
($MAKE) -f ($MAKEFILENAME) other_target
location = $(CURDIR)/$(word $(words $(MAKEFILE_LIST)),$(MAKEFILE_LIST))
WHERE_ART_THOU := $(location)
$(warning $(WHERE_ART_THOU))
I also believe this is GNU make-specific, but I'm not too sure.
(Should you have any questions, refer to amazingly written GNU make manual. But remember, that, just like Makefile, this manual should be read completely before putting the concepts into practice).
I couldn't figure out how it is done easily. As far as I understand, you'll have to do some manual job.
Later I will describe how it could be done and show scripts that introduce current_makefile variable. But I would like to stress an important concept at the first place.
You should understand that if we had some kind of variable current_makefile, that expands to the current makefile name, then it will have to change during the process of reading makefiles. That means that it should be used withinin "immediate" expansion context -- i.e. within commands that are executed during reading the makefile. Most commands, however, are executed after makefiles are read. Therefore, some commands will print the correct value smoothly, while in certain places, where "deferred" expansion is used, it will always expand to the root makefile name.
If you would want to use this variable within rule text, for example, you'll have to do tricks, because rule text always has deferred expansion. So, if your have the rule
rule:
echo In makefile $(current_makefile):
echo Making target $#
it will always print the name of the root makefile. Instead, to force immediate expansion, you will have to create another variable with makefile-specific name (i.e. names of such variables should be different in each makefile):
this_makefile_unique_name := $(current_makefile)
rule:
echo In makefile $(current_makefile):
echo Making target $#
or use eval:.
define make_rule
rule:
echo In makefile $(1):
echo Making target $$#
$(eval $(call make_rule,$(current_makefile)))
If you want to use the name of current makefile for debug purpose only, consider special debugging functions, like warning or info:.
$(warning We're in makefile $(current_makefile))
These functions use "immediate" expansion and will print the correct value.
How to define such a $(current_makefile)?
You have to manually maintain stack of makefile inclusions. When you include a makefile, its name is placed to the top of the stack; when you return from included makefile to the outer one, the topmost name is popped out of stack. This is achieved by inserting special calls to the beginning and the end of makefile:
# Beginning of makefile
$(eval $(makefile_names_push))
#... makefile text
$(warning $(current_makefile))
#...
$(eval $(makefile_names_pop))
#End of file
Now define the functions at the beginning of your root makefile.
lastword=$(word $(words $(1)),$(1))
define makefile_names_push
current_makefile := $$(CURDIR)/$$(call lastword,$$(MAKEFILE_LIST))
makefile_stack :=$$(makefile_stack) $$(current_makefile)
endef
define makefile_names_pop
makefile_stack := $$(filter-out $$(current_makefile),$$(makefile_stack))
current_makefile := $$(call lastword,$$(makefile_stack))
endef
If you're sure your make is new enough (version 3.81+), replace lastword call with builtin function:.
#inctead of $$(call lastword,$$(MAKEFILE_LIST))
$$(lastword $$(MAKEFILE_LIST))
Is it useful?
Totally useless. An only use that might be useful here is to make 100 makefiles that are symlinks to one makefile, the rules in these makefiles depending on their names. But it can be achieved within one makefile and foreach-eval technique described in the manual. So my post was a complete waste of time, though I had some fun :-)
This returns the name of the first Makefile called, i.e. the one at the bottom of the call stack:
MAKEFILE_JUSTNAME := $(firstword $(MAKEFILE_LIST))
MAKEFILE_COMPLETE := $(CURDIR)/$(MAKEFILE_JUSTNAME)
When used in non-cross-recursive situations (e.g. for makedepend), it is just the name of the current makefile.
I wanted to do something similar (for echoing the contents of the Makefile) for when I use Make for managing simple repetitive tasks. I came across this page and found it was exactly what I was after and really useful for my limited understanding of make.
My result after reading this page:
# Makefile - 'make' and 'make help' now echo the makefile.
help:
cat $(lastword $(MAKEFILE_LIST))
start:
sudo -u www /path/to/webapp/myhttpd restart
stop:
sudo kill `cat /path/to/webapp/data/httpd.pid`
A quick excursion to Google suggests this site has the answer.
G'day,
If you make a copy of your original makefile, say makefile_test, and then enter the command:
make -np -f makefile_test 2>&1 | tee output
That will evaluate the makefile and your make environment but not execute any of the commands. Looking through the output file for references to makefile_test will show you what is set in make's environment and where that value is being set.
N.B. This can generate a lot of info! And don't add the -d (debug) switch which will generate tons of additional output about make's decision process but minimal additional info about make's env.
HTH
The solutions here addresses 1) POSIX make with 2) Invoked, non included, makefile in 3) A Unix alike platform.
What the OP asked for:
target:
#pid=$$$$; \
while test `ps -ocomm= $$pid` != make; do \
pid=`ps -oppid= $$pid`; \
done; \
MAKEFILENAME=`ps -oargs= $$pid|sed 's/^.* -f *\([^ ]*\).*$$/\1/'`; \
test -z "$$MAKEFILENAME" -a -f Makefile && MAKEFILENAME=Makefile; \
test -z "$$MAKEFILENAME" -a -f makefile && MAKEFILENAME=makefile; \
export MAKEFILENAME; \
$(MAKE) -e -f $$MAKEFILENAME other_target
The targets depends on the makefile, kind of bloated:
TARGET1_MAKEFILENAME = target1_preamble
all: target1 target2...
target1: $(TARGET1_MAKEFILENAME) other_dependencies...
#test $(TARGET1_MAKEFILENAME) == target1_preamble && exit 0; \
built_instructions_for_target1;
target1_preamble:
#pid=$$$$; \
while test `ps -ocomm= $$pid` != make; do \
pid=`ps -oppid= $$pid`; \
done; \
MAKEFILENAME=`ps -oargs= $$pid|sed 's/^.* -f *\([^ ]*\).*$$/\1/'`; \
test -z "$$MAKEFILENAME" -a -f Makefile && MAKEFILENAME=Makefile; \
test -z "$$MAKEFILENAME" -a -f makefile && MAKEFILENAME=makefile; \
export MAKEFILENAME; \
$(MAKE) -e -f $$MAKEFILENAME target1;
Can be a bit simplified if make is invoked only for all targets.
MAKEFILENAME = invoked_makefile_placeholder
all: target1 target2...
target1: $(MAKEFILENAME) other_dependencies...
#test $(MAKEFILENAME) == invoked_makefile_placeholder && exit 0; \
built_instructions_for_target1;
invoked_makefile_placeholder:
#pid=$$$$; \
while test `ps -ocomm= $$pid` != make; do \
pid=`ps -oppid= $$pid`; \
done; \
MAKEFILENAME=`ps -oargs= $$pid|sed 's/^.* -f *\([^ ]*\).*$$/\1/'`; \
test -z "$$MAKEFILENAME" -a -f Makefile && MAKEFILENAME=Makefile; \
test -z "$$MAKEFILENAME" -a -f makefile && MAKEFILENAME=makefile; \
export MAKEFILENAME; \
$(MAKE) -e -f $$MAKEFILENAME
With the previous approach is trivial to implement a solution for included makefiles based in grep and a unique pattern contained in the makefile.
I never answer when I feel the question got a proper solution.