I am writing a Bash script that creates a CMakeLists.txt file for a project.
The problem arises at this portion:
echo "file(GLOB projSRC src/*.cpp)" >> CMakeLists.txt
After that, I need the program to output ${SOURCES} into CMakeLists.txt
I don't mean a variable in the script named SOURCES, I mean it should actually write the plaintext ${SOURCES}.
What I mean is, the final file should look this like:
arbitrary_command(target PROJECT sources ${SOURCES})
and not like:
arbitrary_command(target PROJECT sources [insert however bash messes it up here])
How can I do this in my Bash script?
Use single quotes, not double quotes, for a literal string:
echo 'file(GLOB ${projSRC} src/*.cpp)' >> CMakeLists.txt
That said, you might consider using a heredoc (or even a quoted heredoc) in this case, to write the entire file as one command:
cat >CMakeLists.txt <<'EOF'
everything here will be emitted to the file exactly as written
${projSRC}, etc
even over multiple lines
EOF
...or, if you want some substitutions, an unquoted heredoc (that is, one where the sigil -- EOF in these examples -- isn't quoted at the start):
foo="this"
cat >CMakeLists.txt <<EOF
here, parameter expansions will be honored, like ${foo}
but can still be quoted: \${foo}
EOF
You can also have multiple commands writing output to a single redirection, to avoid paying the cost of opening your output file more than once:
foo=this
{
echo "Here's a line, which is expanded due to double quotes: ${foo}"
echo 'Here is another line, with no expansion due to single quotes: ${foo}'
} >CMakeLists.txt
May be I don't understand your question...
But
echo \${SOURCES}
will print
${SOURCES}
for you.
Related
new_contents = "\$$(cooly)"
all:
mkdir -p subdir
echo $(new_contents) | sed -e 's/^ //' > subdir/makefile
#echo "---MAKEFILE CONTENTS---"
#cd subdir && cat makefile
#echo "---END MAKEFILE CONTENTS---"
#cd subdir && $(MAKE)
# Note that variables and exports. They are set/affected globally.
cooly = "The subdirectory can see me!"
export cooly
# This would nullify the line above: unexport cooly
clean:
rm -rf subdir
What I want is a "$(cooly)" string, not the variable value.
I tried several combinations:
new_contents = "$(cooly)", gives the variable value, The subdirectory can see me!
new_contents = "$$", gives $
new_contents = "\$(cooly)", gives Syntax error: Unterminated quoted string error
Why new_contents = "$$(cooly)" doesn't give "$(cooly)" but result in nothing?
"$$" -> "$", so why isn't "$$(cooly)" ---> "$(cooly)"?
You have to understand both how make expansion works, and how shell expansion works, in order to write more complicated recipes in make. That's because, make recipes are passed to the shell after make is done expanding them.
Make passes recipe lines to the shell virtually verbatim: there is only one character that's special (not counting backslash/newlines at the end) and that's $. If make sees a $ it will try to expand it as a variable reference. To avoid that, you have to escape it as $$ to hide it from make.
So let's look at your makefile:
cooly = "The subdirectory can see me!"
echo $(new_contents) ...
If new_contents is "$(cooly)", make sees the $(cooly) as a variable reference and expands it before it even invokes the shell. So first make expands $(new_contents) to "$(cooly)", then it expands that to ""The subdirectory can see me!"" (because the quotes are in both variables, and quotes are not special to make: they're just like any other character like a or b). The result will be:
echo ""The subdirectory can see me!""
The shell will toss the quotes since they're no-ops and echo that value (into the pipe).
If new_contents is "\$(cooly)", that backslash doesn't mean anything to make. Just like quotes, backslashes (unless they are at the end of a line) are not special to make. So make expands just as before, but this time the command it passes to the shell is this:
echo "\"The subdirectory can see me!""
backslashes are not special to make, but they are special to the shell. Here you've escaped the second quote so the shell doesn't treat it as a quote character, which means you have an odd number of quotes in your command, which is why you get an error from the shell about non-terminated quotes.
If new_contents is "$$(cooly)", make doesn't expand the variable, it is passed along to the shell like this:
echo "$(cooly)"
However, $ is also special to the shell. Putting it in double quotes doesn't prevent the shell from trying to expand it. This tells the shell to run the command cooly and substitute the output. Almost certainly there is no command named cooly and so you'll get an error message to stderr (maybe you didn't notice it) and the shell will replace it with nothing because it didn't print anything to stdout.
If new_contents is "\$$(cooly)" then make will not expand, and run this shell command:
echo "\$(cooly)"
The shell sees the backslash and doesn't expand the $ but instead uses it literally, and you get the result you want.
Here are some hints:
First, do not include quotes in your make variables (unless the variable contains an entire shell command and you need quotes inside it). Make doesn't care about quotes and having them embedded in variables makes it very difficult to reason about what the shell will see.
Include the quotes only in the recipe.
Second, remember that since make doesn't care about quotes, it doesn't have the same behavior as the shell WRT single vs. double quotes. You can use single quotes around make variables to reduce the need to escape things from the shell, without hiding them from make.
So, I would write this:
new_contents = $$(cooly)
cooly = The subdirectory can see me!
all:
mkdir -p subdir
echo '$(new_contents)' | sed -e 's/^ //' > subdir/makefile
...
BTW, it's never a good idea to add # values to your makefile until it's completely done and working. Seeing the output make prints (which is what it's sending to the shell) is a great help in figuring out whether your recipes are right, and whether the problem is with your make constructs or shell constructs.
In my understanding of command substitution this should work, but it doesn't, can you explain me why and how to do something like this.
Why does this work:
cd ..; echo 123 # output is "123", after changing directories
...when this doesn't:
cd $(echo "..; echo 123") # error message is "cd: too many arguments"
Command substitution results (like expansion of variables) do not go through all parsing phases; they only go through word-splitting[1] and glob expansion[2], and even those happen only when the expansion itself is unquoted.
That means that your result is identical to:
cd "..;" "echo" "123"
...the semicolon is treated as literal text to be passed to the cd command, not shell syntax.
This is a feature, not a bug: If command substitution results were parsed as syntax, writing secure shell scripts handling untrusted data would be basically impossible.
[1] dividing the results into "words" on characters in IFS -- or, by default, whitespace.
[2] looking at whether each resulting word can be treated as a filename-matching pattern, and, if so, matching them against the local filesystem.
This is pretty basic, I guess I'm missing something really obvious...
The following sequence should explain it:
$ cat read_file_names.sh
#!/bin/bash
for i in $# ; do
echo "$i"
done
$ touch "filename has many spaces"
$ ./read_file_names.sh filename\ has\ many\ spaces
filename
has
many
spaces
ideally, the command line will have quotes around the filename as in:
$ ./read_file_names.sh "filename\ has\ many\ spaces"
The problem is that when allowing bash to auto-complete the filename (by hitting tab), the file name is left unquoted. Instead, it has a backslash-space "\ " to signal a space. I understand I can add quotes manually, but that would be tedious and a poor user experience.
I'm looking for a solution which assigns the entire file name to the for-loop variable, so that the output looks something like this:
$ ./read_file_names.sh filename\ has\ many\ spaces
filename has many spaces
The backslashes are working. It's your debugging printer that's wrong:
for i in $# ; do
That needs to be:
for i in "$#"; do
Otherwise, the argument string is inserted unquoted into the for expression and then word-split.
I encountered this code:
file=$(<filename)
This reads the file from the filename.
My question is how does this work?
I read from this post:
How to use double or single brackets, parentheses, curly braces
It tells me, single parentheses can function as:
- Sub bash execution
- Array construction
But in the case above, I do not know how this explains.
Besides this question, I want to know that why when I do echo $file, the file content concatenate into one line?
$(...) performs command substitution; the command inside is read and the output from stdout is returned to the script.
<... is redirection; the contents of the file are read and fed into stdin of the process.
Putting the two together results in an implicit cat, connecting the stdin of the redirection to the stdout of the command substitution, reading the contents of the file into the script.
You must surround your variable into double quotes, else it will be expanded into command line arguments that will be passed to echo.
If you surround it into double quotes variable will be passed as single argument and echo will display it correctly.
echo "$file"
echo $file
will give you a concatenated output.
Try
echo "$file"
this will give you an output in multiple lines.
Everybody says eval is evil, and you should use $() as a replacement. But I've run into a situation where the unquoting isn't handled the same inside $().
Background is that I've been burned too often by file paths with spaces in them, and so like to quote all such paths. More paranoia about wanting to know where all my executables are coming from. Even more paranoid, not trusting myself, and so like being able to display the created commands I'm about to run.
Below I try variations on using eval vs. $(), and whether the command name is quoted (cuz it could contain spaces)
BIN_LS="/bin/ls"
thefile="arf"
thecmd="\"${BIN_LS}\" -ld -- \"${thefile}\""
echo -e "\n Running command '${thecmd}'"
$($thecmd)
Running command '"/bin/ls" -ld -- "arf"'
./foo.sh: line 8: "/bin/ls": No such file or directory
echo -e "\n Eval'ing command '${thecmd}'"
eval $thecmd
Eval'ing command '"/bin/ls" -ld -- "arf"'
/bin/ls: cannot access arf: No such file or directory
thecmd="${BIN_LS} -ld -- \"${thefile}\""
echo -e "\n Running command '${thecmd}'"
$($thecmd)
Running command '/bin/ls -ld -- "arf"'
/bin/ls: cannot access "arf": No such file or directory
echo -e "\n Eval'ing command '${thecmd}'"
eval $thecmd
Eval'ing command '/bin/ls -ld -- "arf"'
/bin/ls: cannot access arf: No such file or directory
$("/bin/ls" -ld -- "${thefile}")
/bin/ls: cannot access arf: No such file or directory
So... this is confusing. A quoted command path is valid everywhere except inside a $() construct? A shorter, more direct example:
$ c="\"/bin/ls\" arf"
$ $($c)
-bash: "/bin/ls": No such file or directory
$ eval $c
/bin/ls: cannot access arf: No such file or directory
$ $("/bin/ls" arf)
/bin/ls: cannot access arf: No such file or directory
$ "/bin/ls" arf
/bin/ls: cannot access arf: No such file or directory
How does one explain the simple $($c) case?
The use of " to quote words is part of your interaction with Bash. When you type
$ "/bin/ls" arf
at the prompt, or in a script, you're telling Bash that the command consists of the words /bin/ls and arf, and the double-quotes are really emphasizing that /bin/ls is a single word.
When you type
$ eval '"/bin/ls" arf'
you're telling Bash that the command consists of the words eval and "/bin/ls" arf. Since the purpose of eval is to pretend that its argument is an actual human-input command, this is equivalent to running
$ "/bin/ls" arf
and the " gets processed just like at the prompt.
Note that this pretense is specific to eval; Bash doesn't usually go out of its way to pretend that something was an actual human-typed command.
When you type
$ c='"/bin/ls" arf'
$ $c
the $c gets substituted, and then undergoes word splitting (see ยง3.5.7 "Word Splitting" in the Bash Reference Manual), so the words of the command are "/bin/ls" (note the double-quotes!) and arf. Needless to say, this doesn't work. (It's also not very safe, since in addition to word-splitting, $c also undergoes filename-expansion and whatnot. Generally your parameter-expansions should always be in double-quotes, and if they can't be, then you should rewrite your code so they can be. Unquoted parameter-expansions are asking for trouble.)
When you type
$ c='"/bin/ls" arf'
$ $($c)
this is the same as before, except that now you're also trying to use the output of the nonworking command as a new command. Needless to say, that doesn't cause the nonworking command to suddenly work.
As Ignacio Vazquez-Abrams says in his answer, the right solution is to use an array, and handle the quoting properly:
$ c=("/bin/ls" arf)
$ "${c[#]}"
which sets c to an array with two elements, /bin/ls and arf, and uses those two elements as the word of a command.
With the fact that it doesn't make sense in the first place. Use an array instead.
$ c=("/bin/ls" arf)
$ "${c[#]}"
/bin/ls: cannot access arf: No such file or directory
From the man page for bash, regarding eval:
eval [arg ...]:
The args are read and concatenated together into a single command.
This command is then read and executed by the shell, and its exit
status is returned as the value of eval.
When c is defined as "\"/bin/ls\" arf", the outer quotes will cause the entire thing to be processed as the first argument to eval, which is expected to be a command or program. You need to pass your eval arguments in such a way that the target command and its arguments are listed separately.
The $(...) construct behaves differently than eval because it is not a command that takes arguments. It can process the entire command at once instead of processing arguments one at a time.
A note on your original premise: The main reason that people say that eval is evil was because it is commonly used by scripts to execute a user-provided string as a shell command. While handy at times, this is a major security problem (there's typically no practical way to safety-check the string before executing it). The security problem doesn't apply if you are using eval on hard-coded strings inside your script, as you are doing. However, it's typically easier and cleaner to use $(...) or `...` inside of scripts for command substitution, leaving no real use case left for eval.
Using set -vx helps us understand how bash process the command string.
As seen in the picture, "command" works cause quotes will be stripped when processing. However, when $c(quoted twice) is used, only the outside single quotes are removed. eval can process the string as the argument and outside quotes are removed step by step.
It is probably just related to how bash semanticallly process the string and quotes.
Bash does have many weird behaviours about quotes processing:
Bash inserting quotes into string before execution
How do you stop bash from stripping quotes when running a variable as a command?
Bash stripping quotes - how to preserve quotes