I am trying to understand what a parameter expansion does inside a bash script.
third_party_bash_script
#!/bin/sh
files="${*:--}"
# For my understanding I tried to print the contents of files
echo $files
pkill bb_stream
if [ "x$VERBOSE" != "" ]; then
ARGS=-v1
fi
while [ 1 ]; do cat $files; done | bb_stream $ARGS
When I run ./third_party_bash_script, all it prints is a hyphen - and nothing else. Since it did not make sense to me, I also tried to experiment with it in the terminal
$ set one="1" two="2" three="3"
$ files="${*:--}"
$ echo $files
one="1" two="2" three="3"
$ set four="4"
$ files="${*:--}"
four="4"
I can't seem to understand what it's doing. Could someone kindly help me with the interpretation of ${*:--} by the sh?
"$#" is an array of the arguments passed to your script, "$*" is a string of all of those arguments concatenated with blanks in between.
"${*:--}" is the string of arguments if any were provided (:-), or - otherwise which means "take input from stdin" otherwise.
"${#:--}" is the array of arguments if any were provided (:-), or - otherwise which means "take input from stdin" otherwise.
$ cat file
foo
bar
$ cat tst.sh
#!/usr/bin/env bash
awk '{ print FILENAME, $0 }' "${#:--}"
When an arg is provided to the script, "$#" contains "file" so that is the arg that awk is called with:
$ ./tst.sh file
file foo
file bar
When no arg is provided to the script, "$#" is empty so awk is called with - (meaning read from stdin) as it's arg:
$ cat file | ./tst.sh
- foo
- bar
You almost always want to use "${#:--}" rather than "${*:--}" in this context, see https://unix.stackexchange.com/questions/41571/what-is-the-difference-between-and for more info on "$#" vs "$*".
${param:-default} expands to the value of $param if $param is set and not empty, otherwise it expands to default.
$* is all the command-line arguments to the script.
In ${*:--}, param is * and default is -. If $* is not an empty string, it expands to the script arguments. If it's empty, it expands to the default value -.
This can be used in a script to implement the common behavior that a program reads from the files listed in its arguments, and reads from standard input if no filename arguments are given. Many commands treat an input filename argument - as standing for the standard input.
NOTE: addressing OP's original, pre-edited post ...
See shell parameter expansion for a brief review of different options.
While the other answers reference the use of ${*:--} (and ${#:--}) as a alternate means of reading from stdin, OP's sample script is a bit simpler ... if the variable $* (ie, script's command line args) is empty then replace with the literal string -.
We can see this with a few examples:
$ third_party_bash_script
-
$ third_party_bash_script a b c
a b c
$ echo 'a b c' | third_party_bash_script
-
If we replace ${*:--} with ${*:-REPLACEMENT}:
$ third_party_bash_script
REPLACEMENT
$ third_party_bash_script a b c
a b c
$ echo 'a b c' | third_party_bash_script
REPLACEMENT
I'm guessing in OP's actual script there's more going on with the $files variable so in order to know for sure how the ${*:--} is being processed we'd need to see the actual script and how it's referencing the $files variable.
As for OP's set|files=|echo code snippets:
$ set one="1" two="2" three="3"
$ files="${*:--}"
$ echo $files
one=1 two=2 three=3
We can see the same behavior from the script with:
$ third_party_bash_script one="1" two="2" three="3"
one=1 two=2 three=3
Related
I've got a program that I want to call by passing parameters from a shell variable. Throughout this question, I am going to assume that it is given by
#!/bin/sh
echo $#
i.e. that it prints out the number of arguments that are passed to it. Let's call it count-args.
I call my program like this:
X="arg1 arg2"
count-args $X
This works quite well. But now one of my arguments has a whitespace in it and I can't find a way to escape it, e.g. the following things do not work:
X="Hello\ World"
X="Hello\\ World"
X="'Hello World'"
In all of the cases, my program count-args prints out 2. I want to find a way so I can pass the string Hello World and that it returns 1 instead. How?
Just for clarification: I do not want to pass all parameters as a single string, e.g.
X="Hello World"
count-args $X
should print out 2. I want a way to pass parameters that contain whitespaces.
Use an array to store multiple, space-containing arguments.
$ args=("first one" "second one")
$ count-args "${args[#]}"
2
This can be solved with xargs. By replacing
count-args $X
with
echo $X | xargs count-args
I can use backslashes to escape whitespaces in $X, e.g.
X="Hello\\ World"
echo $X | xargs count-args
prints out 1 and
X="Hello World"
echo $X | xargs count-args
prints out 2.
count-args "$X"
The quotes ensure in bash, that the whole content of variable X is passed as a single parameter.
Your Counting script:
$ cat ./params.sh
#!/bin/sh
echo $#
For completeness here is what happens with various arguments:
$ ./params.sh
0
$ ./params.sh 1 2
2
$ ./params.sh
0
$ ./params.sh 1
1
$ ./params.sh 1 2
2
$ ./params.sh "1 2"
1
And here is what you get with variables:
$ XYZ="1 2" sh -c './params.sh $XYZ'
2
$ XYZ="1 2" sh -c './params.sh "$XYZ"'
1
Taking this a bit further:
$ cat params-printer.sh
#!/bin/sh
echo "Count: $#"
echo "1 : '$1'"
echo "2 : '$2'"
We get:
$ XYZ="1 2" sh -c './params-printer.sh "$XYZ"'
Count: 1
1 : '1 2'
2 : ''
This looks like what you wanted to do.
Now: If you have a script you cannot control and neither can you control the way the script is invoked. Then there is very little you can do to prevent a variable with spaces turning into multiple arguments.
There are quite a few questions around this on StackOverflow which indicate that you need the ability to control how the command is invoked else there is little you can do.
Passing arguments with spaces between (bash) script
Passing a string with spaces as a function argument in bash
Passing arguments to a command in Bash script with spaces
And wow! this has been asked so many times before:
How to pass argument with spaces to a shell script function
Im trying to create a bourne shell script that will take 0 or more arugments and print out the last arugment, I am use to writing Java and im so confused by this, slowly starting to learn C.
An alternative to #Michael's solution:
#!/usr/bin/env bash
echo "${#: -1}"
$# is the array with all the parameters.
: is used to split strings normally (you can try this with echo "${USER: -1}" to verify that it prints the last character of your user name), but can also be used for arrays to get the last element.
The curly brackets are needed for the array indexing to work
The quotes are simply good practice, to make the code more flexible in case you want to mix in other variables, or the value needs to be used in a statement which needs an empty string rather than nothing in case of no parameters (for example if [ "${#: -1}" = "--help" ])
lastArg=`echo $# | awk '{print $NF}'`
Here is a short Bash script that will do it:
#!/usr/bin/env bash
echo "${!#}"
This is not a Bourne shell script, though. Args are not read from the keyboard with the read command. Instead they are supplied on the command line when running your script. For example, if you put this text in script.sh and run ./script.sh a b c d e it will print:
e
In bash:
last_arg=`echo $* | rev | cut -d " " -f1 | rev`;
echo $last_arg
Your question mentions C. In C its easier:
int main (int argc, char *argv[]) {
char *last_arg = argv[argc - 1];
[...]
}
This should work:
eval lastarg='$'$#
echo last arg is $lastarg
This works with /bin/sh being bash or dash.
I am unable to test it with a "real" /bin/sh, whatever that would be.
I have a Perl script that gives me a defined list of random numbers that correspond to the lines of a file. Next I want to extract those lines from the file using sed.
#!/bin/bash
count=$(cat last_queries.txt | wc -l)
var=$(perl test.pl test2 $count)
The variable var returns an output like: cat last_queries.txt | sed -n '12p;500p;700p'. The problem is that I can't run this last command. I tried with $var, but the output is not correct (if I run manually the command it works fine, so no problem there). What is the correct way to do this?
P.S: Sure I could do all the work in Perl, but I'm trying to learn this way, because it could help me in other situations.
You just need to do:
#!/bin/bash
count=$(cat last_queries.txt | wc -l)
$(perl test.pl test2 $count)
However, if you want to call your Perl command later, and that's why you want to assign it to a variable, then:
#!/bin/bash
count=$(cat last_queries.txt | wc -l)
var="perl test.pl test2 $count" # You need double quotes to get your $count value substituted.
...stuff...
eval $var
As per Bash's help:
~$ help eval
eval: eval [arg ...]
Execute arguments as a shell command.
Combine ARGs into a single string, use the result as input to the shell,
and execute the resulting commands.
Exit Status:
Returns exit status of command or success if command is null.
You're are probably looking for eval $var.
line=$((${RANDOM} % $(wc -l < /etc/passwd)))
sed -n "${line}p" /etc/passwd
just with your file instead.
In this example I used the file /etc/password, using the special variable ${RANDOM} (about which I learned here), and the sed expression you had, only difference is that I am using double quotes instead of single to allow the variable expansion.
There're 2 basic ways of executing a string command in a shell script whether it's given as parameter or not here's.
COMMAND="ls -lah"
$(echo $COMMAND)
or
COMMAND="ls -lah"
bash -c $COMMAND
In the case where you have multiple variables containing the arguments for a command you're running, and not just a single string, you should not use eval directly, as it will fail in the following case:
function echo_arguments() {
echo "Argument 1: $1"
echo "Argument 2: $2"
echo "Argument 3: $3"
echo "Argument 4: $4"
}
# Note we are passing 3 arguments to `echo_arguments`, not 4
eval echo_arguments arg1 arg2 "Some arg"
Result:
Argument 1: arg1
Argument 2: arg2
Argument 3: Some
Argument 4: arg
Note that even though "Some arg" was passed as a single argument, eval read it as two.
Instead, you can just use the string as the command itself:
# The regular bash eval works by jamming all its arguments into a string then
# evaluating the string. This function treats its arguments as individual
# arguments to be passed to the command being run.
function eval_command() {
"$#";
}
Note the difference between the output of eval and the new eval_command function:
eval_command echo_arguments arg1 arg2 "Some arg"
Result:
Argument 1: arg1
Argument 2: arg2
Argument 3: Some arg
Argument 4:
Better ways to do it
Using a function:
# define it
myls() {
ls -l "/tmp/test/my dir"
}
# run it
myls
Using an array:
# define the array
mycmd=(ls -l "/tmp/test/my dir")
# run the command
"${mycmd[#]}"
cmd="ls -atr ${HOME} | tail -1" <br/>
echo "$cmd" <br/>
VAR_FIRST_FILE=$( eval "${cmd}" ) <br/>
or
cmd=("ls -atr ${HOME} | tail -1") <br/>
echo "$cmd" <br/>
VAR_FIRST_FILE=$( eval "${cmd[#]}" )
I have a bash script
fooA
#!/bin/bash
script_name=$1;
script_params=$( echo $# | awk '{ $1=""; print $0 }' );
bash /path/to/scripts/$script_name $script_params > /dev/stdout;
and another script fooB in the .../scripts/ directory.
#!/bin/bash
echo 1. $1
echo 2. $2
My plan is simple:
fooA fooB "some sentence 1" "some sentence 2"
should produce:
some sentence 1
some sentence 2
Using my current script, I would get
some
sentence
Because the double quotes are not preserved when calling fooB from fooA.
Keeping in mind that there many other scripts in the .../scripts directory, how would I change the script_params=$(...) line in fooA file to preserve variables when calling other scripts.
#jm666's answer will work fine if there are no additional constraints. For completeness, though, I'll give a version that doesn't mess with the first script's argument list:
#/bin/bash
script_name="$1"
script_params=( "${#:2}" )
bash /path/to/scripts/"$script_name" "${script_params[#]}" > /dev/stdout
Or you can skip the variables entirely:
#/bin/bash
bash /path/to/scripts/"$1" "${#:2}" > /dev/stdout
#!/bin/bash
name="$1"
shift
"/path/to/script/$name" "$#"
I'm trying to pass 2 arguments to a command and each argument contains spaces, I've tried escaping the spaces in the args, I've tried wrapping in single quotes, I've tried escaping \" but nothing will work.
Here's a simple example.
#!/bin/bash -xv
ARG="/tmp/a b/1.txt"
ARG2="/tmp/a b/2.txt"
ARG_BOTH="\"$ARG\" \"$ARG2\""
cat $ARG_BOTH
I'm getting the following when it runs:
ARG_BOTH="$ARG $ARG2"
+ ARG_BOTH='/tmp/a\ b/1.txt /tmp/a\ b/2.txt'
cat $ARG_BOTH
+ cat '/tmp/a\' b/1.txt '/tmp/a\' b/2.txt
cat: /tmp/a\: No such file or directory
cat: b/1.txt: No such file or directory
cat: /tmp/a\: No such file or directory
cat: b/2.txt: No such file or directory
See http://mywiki.wooledge.org/BashFAQ/050
TLDR
Put your args in an array and call your program as myutil "${arr[#]}"
#!/bin/bash -xv
file1="file with spaces 1"
file2="file with spaces 2"
echo "foo" > "$file1"
echo "bar" > "$file2"
arr=("$file1" "$file2")
cat "${arr[#]}"
Output
file1="file with spaces 1"
+ file1='file with spaces 1'
file2="file with spaces 2"
+ file2='file with spaces 2'
echo "foo" > "$file1"
+ echo foo
echo "bar" > "$file2"
+ echo bar
arr=("$file1" "$file2")
+ arr=("$file1" "$file2")
cat "${arr[#]}"
+ cat 'file with spaces 1' 'file with spaces 2'
foo
bar
This might be a good use-case for the generic "set" command, which sets the top-level shell parameters to a word list. That is, $1, $2, ... and so also $* and $# get reset.
This gives you some of the advantages of arrays while also staying all-Posix-shell-compatible.
So:
set "arg with spaces" "another thing with spaces"
cat "$#"
The most straightforward revision of your example shell script that will work correctly is:
#! /bin/sh
ARG="/tmp/a b/1.txt"
ARG2="/tmp/a b/2.txt"
cat "$ARG" "$ARG2"
However, if you need to wrap up a whole bunch of arguments in one shell variable, you're up a creek; there is no portable, reliable way to do it. (Arrays are Bash-specific; the only portable options are set and eval, both of which are asking for grief.) I would consider a need for this as an indication that it was time to rewrite in a more powerful scripting language, e.g. Perl or Python.