I have a BASH script called script.sh that takes 3 arguments and runs an executable file with them. The first two are just numbers, but the last is an argument giving the input file. I would like the script to run the executable with the input as an argument of the executable and using the "<" as a replacement for stdin. (i.e.
bash script.sh 5 1 input.txt
calls the BASH script, and the contents of script.sh are as follows:
#!/bin/bash
command1="./l${1}t${2} $3"
command2="./l${1}t${2} < $3"
echo + ${command1}
${command1}
echo + ${command2}
${command2}
When I echo command1 I get
./l5t1 input.txt
which is exactly what I want and it runs just fine.
When I echo command2 I get
./l5t1 < input.txt
which is again what I want. The problem is the actual command the script runs is
./l5t1 '<' input.txt
which of course causes a segmentation fault in my program.
I'd like to know if there is a way I can run command 2 so that it runs the string exactly as it is printed in the echo output. Honestly, I have no idea why the single quotes are even inserted around the < character.
If you want to store commands it's better to use functions than variables. As you've found out, redirections don't work when stored in variables (nor do |, ;, or &).
command1() {
"./l${1}t${2}" "$3"
}
command2() {
"./l${1}t${2}" < "$3"
}
command1 "$#"
command2 "$#"
Here I've defined two functions, which are called with the arguments from the array $#. "$#" forwards the script's arguments to the functions.
Notice also that I've put quotes around "./${1}t${2}" and "$3". Using double quotes allows these parameters to contain spaces. Liberal quoting is a good defensive scripting technique.
(I strongly recommend not doing eval "$command2". Using eval is a really dangerous habit to get into.)
Related
I am trying to write a bash function that takes command as the first variable and output file as the second variable:
my_func() {
$1 ${SOME_OTHER_PARAMS} |
tee $2
}
when I run my_func "python print_hello_to_a_file ~/out.txt" "second_file.txt", it seems to output "hello" to "~/out.txt", not "out.txt" in my home directory, meaning that "~" was not expanded correctly.
I am wondering if it is possible to correctly expand "~" inside this function?
Possible? Yes, but probably not a good idea.
The basic problem: When parsing the command line, Tilde Expansion happens before Parameter Expansion. This means you can't put a tilde inside a variable and have it be replaced by a path to your home directory in the simplest case.
Minimal demo:
[user#host]$ myvar="~"
[user#host]$ echo $myvar
~
[user#host]$ echo ~
/home/user
One possible solution is to use eval to force a second round of parsing before executing the command.
[user#host]$ eval echo $myvar
/home/user
But eval is VERY DANGEROUS and you should not use it without exhausting all other possibilities. Forcing a second parsing of the command line can result in unexpected, confusing, and potentially even unsafe results if you are not extremely familiar with the parsing rules and take sufficient steps to sanitize your inputs before running them through eval.
The more standard solution is to build up your command inside a bash array.
my_func() {
tee_output="${1}"
shift
# expand the inputs all together
# SOME_OTHER_PARAMS should be an array as well
"${#}" "${SOME_OTHER_PARAMS[#]}" | tee "${tee_output}"
}
# build the command up as an array with each word as its own element
# Tilde expansion will occur here and the result will be stored for later
my_command=( "python" "print_hello_to_a_file" ~/"out.txt" )
# expand the array and pass to my_func
# Important that the tee_location goes first here, as we
# shift it off to capture the remaining arguments as a complete command
my_func "${tee_loc}" "${my_command[#]}"
But my_func still only supports simple commands with this approach - no loops or if/case statements, no file redirections, etc. This might be okay if your goal is just to decorate a variety of commands with extra parameters and tee their output somewhere.
I used this command in my Bash Shell:
printf $VAR1 >> `printf $VAR2`
and it normally worked. But when I write this into the script file and run it in Shell, it does not work. File "script.sh" contains this:
#!/bin/bash
printf $VAR1 >> `printf $VAR2`
and the output in Shell is:
script.sh: line2: `printf $VAR2`: ambiguous redirect
I don´t know, how is this possible, because the command is absolutely the same. And of course, I run the script on the same system and in the same Shell window.
Thank you for your help.
There are 3 points worth addressing here:
Shell variables vs. environment variables:
Scripts (unless invoked with . / source) run in a child process that only sees the parent [shell]'s environment variables, not its regular shell variables.
This is what likely happened in the OP's case: $VAR1 and $VAR2 existed as regular shell variables, but not environment variables, so script script.sh didn't see them.
Therefore, for a child process to see a parent shell's shell variables, the parent must export them first, as a result of which they (also) become environment variables: export VAR1=... VAR2=...
Bash's error messages relating to output redirection (>, >>):
If the filename argument to a an output redirection is an - unquoted command substitution (`...`, or its modern equivalent, $(...)) - i.e., the output from a command - Bash reports error ambiguous redirect in the following cases:
The command output has embedded whitespace, i.e., contains more than one word.
The command output is empty, which is what likely happened in the OP's case.
As an aside: In this case, the error message's wording is unfortunate, because there's nothing ambiguous about a missing filename - it simply cannot work, because files need names.
It is generally advisable to double-quote command substitutions (e.g., >> "$(...)") and also variable references (e.g., "$VAR2"): this will allow you to return filenames with embedded whitespace, and, should the output be unexpectedly empty, you'll get the (slightly) more meaningful error message No such file or directory.
Not double-quoting a variable reference or command substitution subjects its value / to so-called shell expansions: further, often unintended interpretation by the shell.
The wisdom of using a command substitution to generate a filename:
Leaving aside that printf $VAR2 is a fragile way to print the value of variable $VAR2 in general (the robust form again involves double-quoting: printf "$VAR2", or, even more robustly, to rule out inadvertent interpretation of escape sequences in the variable value, printf %s "$VAR2"), there is no good reason to employ a command substitution to begin with if all that's needed is a variable's value:
>> "$VAR2" is enough to robustly specify the value of variable $VAR2 as the target filename.
I tried this on my Mac (10.11.1) in a terminal window and it worked fine.
Are you sure your default shell is bash?
echo $SHELL
Did you use EXPORT to set your shell vars?
$ export VAR1="UselessData"
$ export VAR2="FileHoldingUselessData"
$ ./script.sh
$ cat FileHoldingUselessData
UselessData$
However.... echo I think does a better job since with printf the output terminates with the first space so....
$ cat script.sh
#!/bin/bash
echo $VAR1 >> `printf $VAR2`
$ ./script.sh
$ cat FileHoldingUselessData
Some Useless Data
Which leads me to believe you might want to just use echo instead of printf all together..
#!/bin/bash
echo $VAR1 >> `echo $VAR2`
I currently have a bash script. It looks like this:
#!/bin/bash
case "$1" in
sendcommand)
do X with $2
exit
;;
esac
How would I send all of command this command with spaces into $2 without command acting as $3, with as $4 and so on? Is there something like PHP or Javascript's encodeURI for bash?
You also have to call your script with the second argument in quotes too
./scriptname sendcommand "command with spaces"
Your script should look like this
#!/bin/bash
case "$1" in
sendcommand)
something "$2"
exit
;;
esac
You can just use double quotes:
do X with "$2"
Enclose it in double quotes:
do_X_with "$2"
The double quotes preserve the internal spacing on the variable, including newlines if they are in the value in the first place. Indeed, it is important to understand the uses of double quotes with "$#" and "$*" too, not to mention when using bash arrays.
You can't easily have a command called do because the shell uses do as a keyword in its loop structure. To invoke it, you would have to specify a path to the command, such as ./do or $HOME/bin/do.
But $2 is "this" and the OP wants it to be "this command with spaces".
OK. We need to review command line invocations and desired behaviours. Let's assume that the script being executed is called script. Further, that command being executed is othercommand (can't use command; that is a standard command).
Possible invocations include:
script sendcommand "this command with spaces"
script sendcommand 'this command with spaces'
script sendcommand this command with spaces
The single-quote and double-quote invocations are equivalent in this example. They wouldn't be equivalent if there were variables to be expanded or commands to be invoked inside the argument lists.
It is possible to write script to handle all three cases:
#!/bin/bash
case "$1" in
sendcommand)
shift
othercommand "$*"
exit
;;
esac
Suppose that the invocation encloses the arguments in quotes. The shift command removes $1 from the argument list and renumbers the remaining (single) argument as $1. It then invokes othercommand with a single string argument consisting of the contents of the arguments concatenated together. If there were several arguments, the contents would be separated by a single 'space' (first character of $IFS).
Suppose that the invocation does not enclose the arguments in quotes. The shift command still removes $1 (the sendcommand) from the argument list, and then space separates the remaining arguments as a single argument.
In all three cases, the othercommand sees a single argument that consists of "this command with spaces" (where the program does not see the double quotes, of course).
I have a bash script that recieves a set of files from the user. These files are sometimes under directories with spaces in their names. Unfortunately unlike this question all the filenames are passed via the command line interface. Let's assume the paths are correctly quoted as they are passed in by the user, so spaces (save for quoted spaces) are delimiters between paths. How would I forward these parameters to a subroutine within my bash script in a way that preserves the quoted spaces?
#! /bin/bash
for fname in "$#"; do
process-one-file-at-a-time "$fname"
done
Note the excessive use of quotes. It's all necessary.
Passing all the arguments to another program is even simpler:
process-all-together "$#"
The tricky case is when you want to split the arguments in half. That requires a lot more code in a simple POSIX shell. But maybe the Bash has some special features.
You want "$#", which has the special syntax of expanding $# but preserving the white-space quoting of the caller (it does not create a single giant string with all the arguments in it). So someone can call your script like:
bash-script.sh AFile "Another File With Spaces"
Then in your script you can do things like:
for f in "$#"; do
echo "$f";
done
and get two lines of output (not 5).
Read the paragraph about the Special Parameter "#" here: http://www.gnu.org/s/bash/manual/bash.html#Special-Parameters
Bravo #Roland . Thans a lot for your solution
It has really worked!
I wrote a simple script function that opens a given path with nautilus.
And I've just nested a function with this "helper"-for-loop into the main function:
fmp () {
fmp2() {
nautilus "$#";
};
for fname in "$#";
do
fmp2 "$fname";
done;
}
Now I'm able to make all my scripts work handling with paths just by turning them into nested functions wrapped by a function with this helper-for-loop.
"$var"
For example,
$ var='foo bar'
$ perl -E'say "<<$_>>" for #ARGV' $var
<<foo>>
<<bar>>
$ perl -E'say "<<$_>>" for #ARGV' "$var"
<<foo bar>>
This question already has answers here:
How can I preserve quotes in printing a bash script's arguments
(7 answers)
Closed 3 years ago.
I have a Bash script where I want to keep quotes in the arguments passed.
Example:
./test.sh this is "some test"
then I want to use those arguments, and re-use them, including quotes and quotes around the whole argument list.
I tried using \"$#\", but that removes the quotes inside the list.
How do I accomplish this?
using "$#" will substitute the arguments as a list, without re-splitting them on whitespace (they were split once when the shell script was invoked), which is generally exactly what you want if you just want to re-pass the arguments to another program.
Note that this is a special form and is only recognized as such if it appears exactly this way. If you add anything else in the quotes the result will get combined into a single argument.
What are you trying to do and in what way is it not working?
There are two safe ways to do this:
1. Shell parameter expansion: ${variable#Q}:
When expanding a variable via ${variable#Q}:
The expansion is a string that is the value of parameter quoted in a format that can be reused as input.
Example:
$ expand-q() { for i; do echo ${i#Q}; done; } # Same as for `i in "$#"`...
$ expand-q word "two words" 'new
> line' "single'quote" 'double"quote'
word
'two words'
$'new\nline'
'single'\''quote'
'double"quote'
2. printf %q "$quote-me"
printf supports quoting internally. The manual's entry for printf says:
%q Causes printf to output the corresponding argument in a format that can be reused as shell input.
Example:
$ cat test.sh
#!/bin/bash
printf "%q\n" "$#"
$
$ ./test.sh this is "some test" 'new
>line' "single'quote" 'double"quote'
this
is
some\ test
$'new\nline'
single\'quote
double\"quote
$
Note the 2nd way is a bit cleaner if displaying the quoted text to a human.
Related: For bash, POSIX sh and zsh: Quote string with single quotes rather than backslashes
Yuku's answer only works if you're the only user of your script, while Dennis Williamson's is great if you're mainly interested in printing the strings, and expect them to have no quotes-in-quotes.
Here's a version that can be used if you want to pass all arguments as one big quoted-string argument to the -c parameter of bash or su:
#!/bin/bash
C=''
for i in "$#"; do
i="${i//\\/\\\\}"
C="$C \"${i//\"/\\\"}\""
done
bash -c "$C"
So, all the arguments get a quote around them (harmless if it wasn't there before, for this purpose), but we also escape any escapes and then escape any quotes that were already in an argument (the syntax ${var//from/to} does global substring substitution).
You could of course only quote stuff which already had whitespace in it, but it won't matter here. One utility of a script like this is to be able to have a certain predefined set of environment variables (or, with su, to run stuff as a certain user, without that mess of double-quoting everything).
Update: I recently had reason to do this in a POSIX way with minimal forking, which lead to this script (the last printf there outputs the command line used to invoke the script, which you should be able to copy-paste in order to invoke it with equivalent arguments):
#!/bin/sh
C=''
for i in "$#"; do
case "$i" in
*\'*)
i=`printf "%s" "$i" | sed "s/'/'\"'\"'/g"`
;;
*) : ;;
esac
C="$C '$i'"
done
printf "$0%s\n" "$C"
I switched to '' since shells also interpret things like $ and !! in ""-quotes.
If it's safe to make the assumption that an argument that contains white space must have been (and should be) quoted, then you can add them like this:
#!/bin/bash
whitespace="[[:space:]]"
for i in "$#"
do
if [[ $i =~ $whitespace ]]
then
i=\"$i\"
fi
echo "$i"
done
Here is a sample run:
$ ./argtest abc def "ghi jkl" $'mno\tpqr' $'stu\nvwx'
abc
def
"ghi jkl"
"mno pqr"
"stu
vwx"
You can also insert literal tabs and newlines using Ctrl-V Tab and Ctrl-V Ctrl-J within double or single quotes instead of using escapes within $'...'.
A note on inserting characters in Bash: If you're using Vi key bindings (set -o vi) in Bash (Emacs is the default - set -o emacs), you'll need to be in insert mode in order to insert characters. In Emacs mode, you're always in insert mode.
I needed this for forwarding all arguments to another interpreter.
What ended up right for me is:
bash -c "$(printf ' %q' "$#")"
Example (when named as forward.sh):
$ ./forward.sh echo "3 4"
3 4
$ ./forward.sh bash -c "bash -c 'echo 3'"
3
(Of course the actual script I use is more complex, involving in my case nohup and redirections etc., but this is the key part.)
Like Tom Hale said, one way to do this is with printf using %q to quote-escape.
For example:
send_all_args.sh
#!/bin/bash
if [ "$#" -lt 1 ]; then
quoted_args=""
else
quoted_args="$(printf " %q" "${#}")"
fi
bash -c "$( dirname "${BASH_SOURCE[0]}" )/receiver.sh${quoted_args}"
send_fewer_args.sh
#!/bin/bash
if [ "$#" -lt 2 ]; then
quoted_last_args=""
else
quoted_last_args="$(printf " %q" "${#:2}")"
fi
bash -c "$( dirname "${BASH_SOURCE[0]}" )/receiver.sh${quoted_last_args}"
receiver.sh
#!/bin/bash
for arg in "$#"; do
echo "$arg"
done
Example usage:
$ ./send_all_args.sh
$ ./send_all_args.sh a b
a
b
$ ./send_all_args.sh "a' b" 'c "e '
a' b
c "e
$ ./send_fewer_args.sh
$ ./send_fewer_args.sh a
$ ./send_fewer_args.sh a b
b
$ ./send_fewer_args.sh "a' b" 'c "e '
c "e
$ ./send_fewer_args.sh "a' b" 'c "e ' 'f " g'
c "e
f " g
Just use:
"${#}"
For example:
# cat t2.sh
for I in "${#}"
do
echo "Param: $I"
done
# cat t1.sh
./t2.sh "${#}"
# ./t1.sh "This is a test" "This is another line" a b "and also c"
Param: This is a test
Param: This is another line
Param: a
Param: b
Param: and also c
Changed unhammer's example to use array.
printargs() { printf "'%s' " "$#"; echo; }; # http://superuser.com/a/361133/126847
C=()
for i in "$#"; do
C+=("$i") # Need quotes here to append as a single array element.
done
printargs "${C[#]}" # Pass array to a program as a list of arguments.
My problem was similar and I used mixed ideas posted here.
We have a server with a PHP script that sends e-mails. And then we have a second server that connects to the 1st server via SSH and executes it.
The script name is the same on both servers and both are actually executed via a bash script.
On server 1 (local) bash script we have just:
/usr/bin/php /usr/local/myscript/myscript.php "$#"
This resides on /usr/local/bin/myscript and is called by the remote server. It works fine even for arguments with spaces.
But then at the remote server we can't use the same logic because the 1st server will not receive the quotes from "$#". I used the ideas from JohnMudd and Dennis Williamson to recreate the options and parameters array with the quotations. I like the idea of adding escaped quotations only when the item has spaces in it.
So the remote script runs with:
CSMOPTS=()
whitespace="[[:space:]]"
for i in "$#"
do
if [[ $i =~ $whitespace ]]
then
CSMOPTS+=(\"$i\")
else
CSMOPTS+=($i)
fi
done
/usr/bin/ssh "$USER#$SERVER" "/usr/local/bin/myscript ${CSMOPTS[#]}"
Note that I use "${CSMOPTS[#]}" to pass the options array to the remote server.
Thanks for eveyone that posted here! It really helped me! :)
Quotes are interpreted by bash and are not stored in command line arguments or variable values.
If you want to use quoted arguments, you have to quote them each time you use them:
val="$3"
echo "Hello World" > "$val"
As Gary S. Weaver shown in his source code tips, the trick is to call bash with parameter '-c' and then quote the next.
e.g.
bash -c "<your program> <parameters>"
or
docker exec -it <my docker> bash -c "$SCRIPT $quoted_args"
If you need to pass all arguments to bash from another programming language (for example, if you'd want to execute bash -c or emit_bash_code | bash), use this:
escape all single quote characters you have with '\''.
then, surround the result with singular quotes
The argument of abc'def will thus be converted to 'abc'\''def'. The characters '\'' are interpreted as following: the already existing quoting is terminated with the first first quote, then the escaped singular single quote \' comes, then the new quoting starts.
Yes, seems that it is not possible to ever preserve the quotes, but for the issue I was dealing with it wasn't necessary.
I have a bash function that will search down folder recursively and grep for a string, the problem is passing a string that has spaces, such as "find this string". Passing this to the bash script will then take the base argument $n and pass it to grep, this has grep believing these are different arguments. The way I solved this by using the fact that when you quote bash to call the function it groups the items in the quotes into a single argument. I just needed to decorate that argument with quotes and pass it to the grep command.
If you know what argument you are receiving in bash that needs quotes for its next step you can just decorate with with quotes.
Just use single quotes around the string with the double quotes:
./test.sh this is '"some test"'
So the double quotes of inside the single quotes were also interpreted as string.
But I would recommend to put the whole string between single quotes:
./test.sh 'this is "some test" '
In order to understand what the shell is doing or rather interpreting arguments in scripts, you can write a little script like this:
#!/bin/bash
echo $#
echo "$#"
Then you'll see and test, what's going on when calling a script with different strings