how does this escaping work? - shell

Here is what it finally took to get my code in my makefile to work
Line 5 is the question area
BASE=50
INCREMENT=1
FORMATTED_NUMBER=${BASE}+${INCREMENT}
all:
echo $$((${FORMATTED_NUMBER}))
why do i have to add two $ and two (( )) ?
Formatted_Number if i echo it looks like "50+1" . What is the logic that make is following to know that seeing $$(("50+1")) is actually 51?
sorry if this is a basic question i'm new to make and dont fully understand it.

First, whenever asking questions please provide a complete example. You're missing the target and prerequisite here so this is not a valid makefile, and depending on where they are it could mean very different things. I'm assuming that your makefile is something like this:
BASE=50
INCREMENT=1
FORMATTED_NUMBER=${BASE}+${INCREMENT}
all:
echo $$((${FORMATTED_NUMBER}))
Makefiles are interesting in that they're a combination of two different formats. The main format is makefile format (the first five lines above), but inside a make recipe line (that's the last line above, which is indented with a TAB character) is shell script format.
Make doesn't know anything about math. It doesn't interpret the + in the FORMATTED_NUMBER value. Make variables are all strings. If you want to do math, you have to do it in the shell, in a shell script, using the shell's math facilities.
In bash and other modern shells, the syntax $(( ...expression... )) will perform math. So in the shell if you type echo $((50+1)) (go ahead and try it yourself) it will print 51.
That's why you need the double parentheses ((...)): because that's what the shell wants and you're writing a shell script.
So why the double $? Because before make starts the shell to run your recipe, it first replaces all make variable references with their values. That's why the shell sees 50+1 here: before make started the shell it expanded ${FORMATTED_NUMBER} into its value, which is ${BASE}+${INCREMENT}, then it expanded those variables so it ends up with 50+1.
But what if you actually want to use a $ in your shell script (as you do here)? Then you have to tell make to not treat the $ as introducing a make variable. You do this by doubling it, so if make sees $$ then it does not think that's a make variable, and sends a single $ to the shell.
So for the recipe line echo $$((${FORMATTED_NUMBER})) make actually invokes a shell script echo $((50+1)).

You can use this in BASH:
FORMATTED_NUMBER=$((BASE+INCREMENT))
Is using non BASH use:
FORMATTED_NUMBER=`echo "$BASE + $INCREMENT" | bc`

Related

Why does printf behave differently when called from a Makefile?

The printf program can be used to print binary data, e.g.:
$ printf '%b' '\xff\xff'
��
If I put this in a Makefile on its own, it works the same:
all:
printf '%b' '\xff\xff'
$ make
printf '%b' '\xff\xff'
��
However, if I want to do anything else on the same shell invocation in the Makefile, for example to redirect it to a file, or just printing something else afterwards, then although the command printed by Make doesn't change (suggesting it's not an escaping issue), but the output changes to a backslash followed by an "x" followed by a double "f", twice:
all:
printf '%b' '\xff\xff'; printf 'wtf?\n'
make
printf '%b' '\xff\xff'; printf 'wtf?\n'
\xff\xffwtf?
What is going on here? Why do the two printfs in one line behave differently than a single printf?
#chepner is on the right track in their comment but the details are not quite right:
This is wild speculation, but I suspect there is some sort of
optimization being applied by make that causes the first example, as a
simple command, to be executing a third option, the actual binary
printf (found in /usr/bin, perhaps), rather than a shell. In your
second example, the ; forces make to use a shell to execute the shell
command line.
Make always uses /bin/sh as its shell, regardless of what the user is using as their shell. On some systems, /bin/sh is bash (which has a builtin printf) and on some systems /bin/sh is something different (typically dash which is a lightweight, POSIX-conforming shell) which probably doesn't have a shell built-in.
On your system, /bin/sh is bash. But, when you have a "simple command" that doesn't require a shell (that is, make itself has enough trivial quoting smarts to understand your command) then to be more efficient make will invoke that command directly rather than running the shell.
That's what's happening here: when you run the simple command (no ;) make will invoke the command directly and run /usr/bin/printf. When you run the more complex command (including a ;) make will give up running the command directly and invoke your shell... which is bash, which uses bash's built-in printf.
Basically, your script is not POSIX-conforming (there is no %b in the POSIX standard) and so what it does is not well-defined. If you want the SAME behavior always you should use /usr/bin/printf to force that always to be used. Forcing make to always run a shell and never use its fast path is much trickier; you'll need to include a special character like a trailing ; in each command.

Invoking bash commands in makefile [duplicate]

This question already has an answer here:
command substitution doesn't work with echo in makefile [duplicate]
(1 answer)
Closed 6 years ago.
Inside of a makefile, I'm trying to check if fileA was modified more recently than fileB. Using a few prior posts (this and this) as references, I've come up with this as an attempt to store the time since last file modification as a variable:
(I'd rather this be in a function outside of a make recipe, but one step at a time.)
.PHONY: all clean
all : (stuff happens here)
radio :
BASE_MOD_TIME="$( expr $(date +%s) - $(date +%s -r src/radio_interface/profile_init.c) )"
#echo "$(BASE_MOD_TIME)"
I thought that I would be assigning the output of the expr command to a variable, BASE_MOD_TIME, but the output is:
bash-4.1$
BASE_MOD_TIME=""
echo ""
What am I doing wrong here? Simple attempts to save the output of ls -l also didn't work like this.
Make variables are normally global, and you don't normally set make variables in a recipe. A recipe is simply a list of commands to be executed by a shell, and so what looks like a variable assignment in a recipe is a shell variable assignment.
However, each line in a make recipe is run in its own shell subprocess. So a variable set in one line won't be visible in another line; they are not persistent. That makes setting shell variables in recipes less useful. [Note 1]
But you can combine multiple lines of a recipe into a single shell command using the backslash escape at the end of the line, and remembering to terminate the individual commands with semicolons (or, better, link them with &&), because the backslash-escaped newline will not be passed to the shell. Also, don't forget to escape the $ characters so they will be passed to the shell, rather than being interpreted by make.
So you could do the following:
radio:
#BASE_MOD_TIME="$$( expr $$(date +%s) - $$(date +%s -r src/radio_interface/profile_init.c) )"; \
echo "$$BASE_MOD_TIME"; \
# More commands in the same subprocess
But that gets quite awkward if there are more than a couple of commands, and a better solution is usually to write a shell script and invoke it from the recipe (although that means that the Makefile is no longer self-contained.)
Gnu make provides two ways to set make variables in a recipe:
1. Target-specific variables.
You can create a target-specific variable (which is not exactly local to the target) by adding a line like:
target: var := value
To set the variable from a shell command, use the shell function:
target: var := $(shell ....)
This variable will be available in the target recipe and all dependencies triggered by the target. Note that a dependency is only evaluated once, so if it could be triggered by a different target, the target-specific variable might or might not be available in the dependency, depending on the order in which make resolves dependencies.
2. Using the eval function
Since the expansion of recipes is always deferred, you can use the eval function inside a recipe to defer the assignment of a make variable. The eval function can be placed pretty well anywhere in a recipe because its value is the empty string. However, evaling a variable assignment makes the variable assignment global; it will be visible throughout the makefile, but its value in other recipes will depend, again, on the order in which make evaluates recipes, which is not necessarily predictable.
For example:
radio:
$(eval var = $(shell ...))
Notes:
You can change this behaviour using the .ONESHELL: pseudo-target, but that will apply to the entire Makefile; there is no way to mark a single recipe as being executed in a single subprocess. Since changing the behaviour can break recipes in unexpected ways, I don't usually recommend this feature.
What's wrong with this?
fileB: fileA
#echo $< was modified more recently than $#
Instead of forcing the makefile to do all of the heavy lifting via some bash commands, I just called a separate bash script. This provided a lot more clarity for a newbie to bash scripting like myself since I didn't have to worry about escaping the current shell being used by make.
Final solution:
.PHONY: all clean
all : (stuff happens here)
radio :
./radio_init_check.sh
$(MKDIR_P) $(OBJDIR)
make $(radio_10)
with radio_init_check.sh being my sew script.

Variable assignment with substitution

This won't work for me. I want to make some substitution and assign it to a variable in Makefile. An example is as follows but I prefer to do it with Perl since other substitutions can be more complex than this.
eval.%:
# make eval.exp-1.ans
# $* --> exp-1.ans
folder=`echo $* | sed -e 's/\..*//g'`
# OR
folder=`echo $* | perl -ne 'm/(.*)\.ans/; print $$1'
# I want that folder will be exp-1
echo $$folder ${folder}
Why this does not work? How can I do this kind of things in Makefile?
Your question is not clear. Are you trying to set a variable in your makefile, so that other recipes, etc. can see it? Or are you just trying to set a variable in one part of your recipe that can be used in other parts of the same recipe?
Make runs recipes in shells that it invokes. It's a fundamental feature of UNIX that no child process can modify the memory/environment/working directory/etc. of its parent process. So no variable that you assign in a recipe (subshell) of a makefile can ever set a make environment variable. There are ways to do this, but not from within a recipe. However that doesn't appear (from your example) to be what you want to do.
The next thing to note is that make runs each logical line of the recipe in a different shell. So shell variables set in one logical line will be lost when that shell exits, and the next logical line cannot see that value. The solution is to ensure that all the lines of your recipe that need access to the variable are on the same logical line, so they'll be sent to the same shell script, like this:
eval.%:
folder=`echo $* | perl -ne 'm/(.*)\.ans/; print $$1' || exit 1 ; \
echo $$folder
Have you tried the $(VARIABLE:suffix=replacement) syntax? In your case, $(*:.ans=). That will work for any suffix, even if it doesn't start with a dot.

ZSH/Shell variable assignment/usage

I use ZSH for my terminal shell, and whilst I've written several functions to automate specific tasks, I've never really attempted anything that requires the functionality I'm after at the moment.
I've recently re-written a blog using Jekyll and I want to automate the production of blog posts and finally the uploading of the newly produced files to my server using something like scp.
I'm slightly confused about the variable bindings/usage in ZSH; for example:
DATE= date +'20%y-%m-%d'
echo $DATE
correctly outputs 2011-08-23 as I'd expect.
But when I try:
DATE= date +'20%y-%m-%d'
FILE= "~/path/to/_posts/$DATE-$1.markdown"
echo $FILE
It outputs:
2011-08-23
blog.sh: line 4: ~/path/to/_posts/-.markdown: No such file or directory
And when run with what I'd be wanting the blog title to be (ignoring the fact the string needs to be manipulated to make it more url friendly and that the route path/to doesn't exist)
i.e. blog "blog title", outputs:
2011-08-23
blog.sh: line 4: ~/path/to/_posts/-blog title.markdown: No such file or directory
Why is $DATE printing above the call to print $FILE rather than the string being included in $FILE?
Two things are going wrong here.
Firstly, your first snippet is not doing what I think you think it is. Try removing the second line, the echo. It still prints the date, right? Because this:
DATE= date +'20%y-%m-%d'
Is not a variable assignment - it's an invocation of date with an auxiliary environment variable (the general syntax is VAR_NAME=VAR_VALUE COMMAND). You mean this:
DATE=$(date +'20%y-%m-%d')
Your second snippet will still fail, but differently. Again, you're using the invoke-with-environment syntax instead of assignment. You mean:
# note the lack of a space after the equals sign
FILE="~/path/to/_posts/$DATE-$1.markdown"
I think that should do the trick.
Disclaimer
While I know bash very well, I only started using zsh recently; there may be zshisms at work here that I'm not aware of.
Learn about what a shell calls 'expansion'. There are several kinds, performed in a particular order:
The order of word expansion is as follows:
tilde expansion
parameter expansion
command substitution
arithmetic expansion
pathname expansion, unless set -f is in effect
quote removal, always performed last
Note that tilde expansion is only performed when the tilde is not quoted; viz.:
$ FILE="~/.zshrc"
$ echo $FILE
~/.zshrc
$ FILE=~./zshrc
$ echo $FILE
/home/user42/.zshrc
And there must be no spaces around the = in variable assignments.
Since you asked in a comment where to learn shell programming, there are several options:
Read the shell's manual page man zsh
Read the specification of the POSIX shell, http://pubs.opengroup.org/onlinepubs/009695399/utilities/xcu_chap02.html, especially if you want to run your scripts on different operating systems (and you will find yourself in that situation one fine day!)
Read books about shell programming.
Hang out in the usenet newsgroup comp.unix.shell where a lot of shell wizards answer questions

Why the sh script cannot work

I write a sh script (test.sh) like this:
#!/bin/sh
echo $#
and then run it like this:
#./test.sh '["hello"]'
but the output is:
"
In fact I need
["hello"]
The bash version is:
#bash --version
GNU bash, version 3.2.25(1)-release (x86_64-redhat-linux-gnu)
And if I run
# echo '["hello"]'
["hello"]
I don't know why the script cannot work...
You probably mean "$#", though I don't think it should make much of a difference in this case. It's also worth making sure that the script is executable (chmod +x test.sh).
EDIT: Since you asked,
Bash has various levels of string expansion/manipulation/whatever. Variable expansion, such as $#, is one of them. Read man bash for more, but from what I can tell, Bash treats each command as one long string until the final stage of "word splitting" and "quote removal" where it's broken up into arguments.
This means, with your original definition, that ./test.sh "a b c d" is equivalent to echo "a" "b" "c" "d", not echo "a b c d". I'm not sure what other stages of expansion happen.
The whole word-splitting thing is common in UNIXland (and pretty much any command-line-backed build system), where you might define something like CFLAGS="-Os -std=c99 -Wall"; it's useful that $CFLAGS expands to "-Os" "-std=c99" "-Wall".
In other "$scripting" languages (Perl, and possibly PHP), $foo really means $foo.
The manual test you show works because echo gets the argument ["hello"]. The outermost quotes are stripped by the shell. When you put this in a shell script, each shell strips one layer of quotes: the one you type at and the one interpreting the script. Adding an extra layer of quotes makes that work out right.
Just a guess, but maybe try changing the first line to
#!/bin/bash
so that it actually runs with bash?

Resources