I wrote in Makefile
define deploy_meta
$(shell git log -n 2 --oneline | awk '{print "commit"NR ": " $0}')
commit: nogit-$(timestamp)
tag: nogit-$(timestamp)
deployed-from: $(shell hostname)
deployed-by: $(USER)
deploy-date: $(shell date -u '+%Y%m%d%H%M%S')
endef
but if gives me
$cat .deploy
commit1: commit2:
commit: nogit-1669806282
tag: nogit-1669806282
...
Command itself
git log -n 2 --oneline | awk '{print "commit"NR ": " $0}'
works fine and gives two lines. It is evident, that make feels it, since it prints two "commit#" words. But it doesn't print content. Why?
The main problem is simple enough: Make expands $0 here (to nothing). You need $$0.
If you want two lines, though, $(shell) is also part of your problem here:
The shell function provides for make the same facility that backquotes ('`') provide in most shells: it does command expansion. This means that it takes as an argument a shell command and expands to the output of the command. The only processing make does on the result is to convert each newline (or carriage-return / newline pair) to a single space. If there is a trailing (carriage-return and) newline it will simply be removed.
(emphasis mine). You'll need a completely different approach, e.g., use $(shell) to produce a file, then include that file, for instance.
Related
Makefile:
.PHONY: all
SHELL:=/usr/bin/env bash
all:
$(eval x=$(shell cat file))
#echo "$x"
File:
foo
bar
Output:
foo bar
How do I get the contents of the file into the make variable without losing the newlines?
You can't do this with shell, as described in its documentation.
If you have a sufficiently new version of GNU make, you can use the file function however.
Make converts newlines from shell outputs to spaces (see here):
The shell function performs the same function that backquotes (‘`’)
perform in most shells: it does command expansion. This means that it
takes as an argument a shell command and evaluates to the output of
the command. The only processing make does on the result is to convert
each newline (or carriage-return / newline pair) to a single space. If
there is a trailing (carriage-return and) newline it will simply be
removed.
So, you cannot preserve spaces from the $(shell) command directly. That being said, make does allow multiline variables using define -- but beware, attempting to use such variables is problematic. Consider:
define x
foo
bar
endef
all:
#echo "$x"
Make expands the $x in place, and you end up with:
all:
#echo " foo
bar"
(where the newline is considered the end of the recipe line..).
Depending on what you want this for, you may be able to get around this is using a bash variable:
all:
#x=$$(cat file); \
echo $$x
Or potentially storing your output in a file, and referencing that when necessary.
all:
eval (cat file >> output.txt)
cat output.txt
(and yes, the last one is convoluted as written, but I'm not sure what you're trying to do, and this allows the output of your command to be persistent across recipe lines).
If the file contents are ensured not to contain any binary data, and if you're willing to to extra processing each time you access the variable, then you could:
foo:=$(shell cat file | tr '\n' '\1')
all:
#echo "$(shell echo "$(foo)" | tr '\1' '\n')"
Note that you cannot use nulls \0, and I suspect that probably means there's a buffer overflow bug in my copy of Make.
Running this statement in OS X Terminal
for i in `ls -v *.mkv`; do echo $i; done
will successfully print out all the file names in the directory in name order with each file name on its own line.
Source: This StackOverFlow answer
However, if I run this statement in OS X Terminal
for i in 'ls -v *.mkv'; do echo $i; done
the output is "ls -v fileName1.mkv fileName2.mkv", etc. with all the file names concatenated into one long line (as opposed to each being printed on its own line).
My questions are:
What's the difference between ` and ' in bash?
Why is that difference responsible for the completely different output?
What keyboard combination produces `? (Keyboard combination)
1) Text between backticks is executed and replaced by the output of the enclosed command, so:
echo `echo 42`
Will expand to:
echo 42
This is called Command Substitution and can also be achieved using the syntax $(command). In your case, the following:
for i in `ls -v *.mkv`; do ...
Is replaced by something like (if your directory contains 3 files named a.mkv, b.mkv and c.mkv):
for i in a.mkv b.mkv c.mkv; do ...
Text between quotes or double quotes are just plain Bash strings with characters like space scaped inside them (there are other ways to quote strings in Bash and are described here):
echo "This is just a plain and simple String"
echo 'and this is another string'
A difference between using ' and " is that strings enclosed between " can interpolate variables, for example:
variable=42
echo "Your value is $variable"
Or:
variable=42
echo "Your value is ${variable}"
Prints:
Your value is 42
2) Wildcard expressions like *.mkv are replaced by the expanded filenames in a process known as Globbing. Globbing is activated using wildcards in most of the commands without enclosing the expression inside a string:
echo *.mkv
Will print:
a.mkv b.mkv c.mkv
Meanwhile:
echo "*.mkv"
prints:
*.mkv
The i variable in your for loop takes the value "ls -v *.mkv" but the echo command inside the loop body takes $i without quotes, so Bash applied globbing there, you end up with the following:
for i in 'ls -v *.mkv'; do
# echo $i
#
# which is expanded to:
# echo ls -v *.mkv (no quotes)
#
# and the globbing process transform the above into:
echo ls -v a.mkv b.mkv c.mkv
Which is just a one-line string with the file names after the globbing is applied.
3) It depends on your keyboard layout.
One trick to keep the character around is to use the program ascii, search for the character 96 (Hex 60), copy it and keep it on your clipboard (you can use parcellite or any other clipboard manager that suits your needs).
Update: As suggested by #triplee, you should check useless use of ls as this is considered a bash pitfall and there are better ways to achieve what you're trying to do.
'expression', will output the exact string in expression.
`expression`, will execute the content of the expression and echo outputs it.
For example:
x="ls"
echo "$x" --> $x
echo `$x` --> file1 file2 ... (the content of your current dir)
Backticks mean "run the thing between the backticks as a command, and then act as if I had typed the output of that command here instead". The single quotes mean, as others have said, just a literal string. So in the first case, what happens is this:
bash runs ls -v *.mkv as a command, which outputs something like:
fileName1.mkv
fileName2.mkv
bash then substitutes this back into where the backtick-surrounded command was, i.e. it effectively makes your for statement into this:
for i in fileName1.mkv fileName2.mkv; do echo $i; done
That has two "tokens": "fileName1.mkv" and "fileName2.mkv", so the loop runs its body (echo $i) twice, once for each:
echo fileName1.mkv
echo fileName2.mkv
By default, the echo command will output a newline after it finishes echoing what you told it to echo, so you'll get the output you expect, of each filename on its own line.
When you use single quotes instead of backticks, however, the stuff in between the single quotes doesn't get evaluated; i.e. bash doesn't see it as a command (or as anything special at all; the single quotes are telling bash, "this text is not special; do not try to evaluate it or do anything to it"). So that means what you're running is this:
for i in 'ls -v *.mkv'; do echo $i; done
Which has only one token, the literal string "ls -v *.mkv", so the loop body runs only once:
echo ls -v *.mkv
...but just before bash runs that echo, it expands the "*.mkv".
I glossed over this above, but when you do something like ls *.mkv, it's not actually ls doing the conversion of *.mkv into a list of all the .mkv filenames; it's bash that does that. ls never sees the *.mkv; by the time ls runs, bash has replaced it with "fileName1.mkv fileName2.mkv ...".
Similarly for echo: before running this line, bash expands the *.mkv, so what actually runs is:
echo ls -v fileName1.mkv fileName2.mkv
which outputs this text:
ls -v fileName1.mkv fileName2.mkv
(* Footnote: there's another thing I've glossed over, and that's spaces in filenames. The output of the ls between the backticks is a list of filenames, one per line. The trouble is, bash sees any whitespace -- both spaces and newlines -- as separators, so if your filenames are:
file 1.mkv
file 2.mkv
your loop will run four times ("file", "1.mkv", "file", "2.mkv"). The other form of the loop that someone mentioned, for i in *.mkv; do ... doesn't have this problem. Why? Because when bash is expanding the "*.mkv", it does a clever thing behind the scenes and treats each filename as a unit, as if you'd said "file 1.mkv" "file 2.mkv" in quotes. It can't do that in the case where you use ls because after it passes the expanded list of filenames to ls, bash has no way of knowing that what came back was a list of those same filenames. ls could have been any command.)
For the command ls, the option -1 is supposed to do run ls yielding an output with only one column (yielding one file per line). But if I put it in a script, it just shows every file jammed on one line, separated with spaces.
Script:
#!/bin/ksh
text=`ls -1`
echo $text
Folder contents:
test
|--bob
|--coolDir
|--file
|--notThisDirectoryAgain
|--script.sh
|--spaces are cool
|--thatFile
Script Output:
bob coolDir file notThisDirectoryAgain script.sh spaces are cool thatFile
Output if I run ls -1 in the terminal (not in a script)
bob
coolDir
file
notThisDirectoryAgain
script.sh
spaces are cool
thatFile
it just shows every file jammed on one line, separated with spaces.
You have to consider what it is.
When you do
text=`ls -1`
that runs the program ls and presents the output as if you typed it in. So the shell gets presented with:
ls=file1
file2
file3
etc.
The shell splits command-line tokens on whitespace, which by default includes a space, a tab, and a newline. So each filename is seen as a separate token by the shell.
These tokens are then passed into echo as separate parameters. The echo command is unaware that the newlines were ever there.
As I'm sure you know, all echo does is to write each parameter to stdout, with a space between each one.
This is why the suggestion given by #user5228826 works. Change IFS if you don't want a newline to separate tokens.
However, all you really had to do is to enclose the variable in quotes, so that it didn't get split:
echo "$text"
By the way, using `backticks` is deprecated and poor practice because it can be difficult to read, particularly when nested. If you run ksh -n on your script it will report this to you (assuming you are not using an ancient version). Use:
text=$(ls -1)
Having said all that, this is a terrible way to get a list of files. UNIX shells do globbing, this is an unnecessary use of the ls external program. Try:
text=(*) # Get all the files in current directory into an array
oldIFS="$IFS" # Save the current value of IFS
IFS=$'\n' # Set IFS to a newline
echo "${text[*]}" # Join the elements of the array by newlines, and display
IFS="$oldIFS" # Reset IFS to previous value
That's because you're capturing ls output into a variable. Bash does the same.
I'm trying to make a convenience function to fix issues when I accidentally have my caps locks on and am trying to run some case-sensitive tools.
e.g. I occasionally find myself typing MAKE DEBUG instead of make debug.
What I have now is pretty straightforward: alias MAKE="make" and editing the makefiles to duplicate the rules, e.g. DEBUG: debug.
I'd prefer a solution that works on the arguments, without having to modify the tools involved.
Using GNU sed
If you just want everything in your makefile to be in lowercase, you can use GNU sed to lowercase the whole thing:
sed -i 's/.*/\L&/' Makefile
You could also build a sed script that's a little more discriminating, but the \L replacement escape is your friend.
Using tr
Since you tagged your question with tr, you might also want a tr solution. It's a little more cumbersom, since tr won't do in-place translations, but you could shuffle temp files or use sponge from moreutils. For example:
tr '[[:upper:]]' '[[:lower:]]' < Makefile | sponge Makefile
This involves a script, but avoids the Ctrl-D issue of my earlier attempt:
For each command, an alias like
alias MAKE="casefixer make"
And then the following file, which I've created at /usr/local/bin/casefixer:
#!/bin/bash
command=`echo $1 | tr '[:upper:]' '[:lower:]'` # convert 1st arg to lowercase, it's the command to invoke
shift # remove 1st arg from $*
$command `echo "$*" | tr '[:upper:]' '[:lower:]'` # convert arguments to lowercase, and invoke the command with them
Playing on #Clayton Hughes' casefixer solution, here's a solution that'll handle funny things like spaces in arguments (which $* messes up):
casefixer() { eval "$(printf "%q " "$#" | tr '[:upper:]' '[:lower:]')"; }
alias MAKE='casefixer make'
Note: eval is a fairly dangerous thing, with a well-deserved reputation for causing really bizarre bugs. In this case, however, the combination of double-quoting and encoding the command and its arguments with %q should prevent trouble. At least, I couldn't find a case where it did anything unexpected.
Here's one solution, though it's not perfect:
alias MAKE="make `tr '[:upper:]' '[:lower:]`"
it works, but has the unfortunate problem that I need to press Ctrl-D to send an EOF before anything starts executing.
The readline command "downcase-word" (bound to M-u by default) is worth mentioning here. Suppose you typed "MAKE DEBUG". If you catch it before hitting return, you can move the cursor to the beginning of the line with C-a. (Otherwise, bring the command back first, using the up arrow). Then, each time you hit M-u, the word immediately after the cursor will be changed to lowercase, and the cursor will move to the beginning of the next word.
It's a little laborious, and I don't see a way to lowercase the entire line at once. Perhaps someone can improve on this.
The following command prints the absolute path of a particular C++ header, according to where g++ believes it to be.
echo \#include\<ham/hamsterdb.h\> | g++ -M -x c++-header - | grep hamsterdb.hpp | sed -e 's/-: //' -e 's/ \\//'
On my system, this outputs: /usr/local/include/ham/hamsterdb.hpp
I run into a problem when trying to run this inside a Makefile in order to set a variable:
FILE=$(shell echo \#include\<ham/hamsterdb.h\> | g++ -M -x c++-header - | grep hamsterdb.hpp | sed -e 's/-: //' -e 's/ \\//')
.PHONY spec
spec:
#echo $(FILE)
This outputs a new line. I think it's the hash ('#') character which is messing with make; if I rewrite the FILE=... line like this:
FILE=$(shell echo \#include\<ham/hamsterdb.h\>)
the output is still nothing.
You have to escape the hash twice in order to use it inside functions: once for Make and once again for the shell.
That is,
FILE = $(shell echo \\\#include\ \<ham/hamsterdb.h\>)
Notice three backslashes instead of two as one could expect. The third backslash is needed because otherwise the first one would escape the second and the hash would be still unescaped.
UPD.
Another possible solution is to only escape the hash for Make and use Bash single quotes to prevent interpreting the hash as a shell comment. This also eliminates the need of escaping spaces, < and >:
FILE = $(shell echo '\#include <ham/hamsterdb.h>')
You need just a bit more quoting:
FILE=$(shell echo \\\#include\<ham/hamsterdb.h\> ...
^^^
Quote once for make itself, a second time for the shell (the shell needs to 'see' \#).