I'm interested to know what this snippet of RSH code does, and whether Bash has something similar:
if [ -z $ALPHA \
-z $BRAVO \
-z $CHARLIE \
-z $DELTA ]; then
var=$ZULU
fi
Those baskslashes are allowing for line continuation. It's as if the code were written like the following:
if [ -z $ALPHA -z $BRAVO -z $CHARLIE -z $DELTA ]; then
var=$ZULU
fi
From man bash
If a \<newline> pair
appears, and the backslash is not itself quoted, the \<newline> is
treated as a line continuation (that
is, it is removed from the input
stream and effectively ignored).
The \ is escaping the end of line.
It is a way to tell that the line is not yet complete and is being continued in the next line.
It just makes your code easier to read.
It is available in bash as well:
$ echo foo
foo
$ echo foo \
> bar
foo bar
$
Related
My understanding is that -n is the opposite of -z.
Testing -z...
#! /bin/bash
if [ -z $1 ]; then
echo bar
fi
$ ./test1.sh
bar
$ ./test1.sh foo
Works as expected!
Testing -n...
#! /bin/bash
if [ -n $1 ]; then
echo bar
fi
$ ./test1.sh
bar
$ ./test1.sh foo
bar
Does not work as expected unless I use quotes... Why?
Trying to figure out when to use quotes and when I can safely omit them.
Testing -n (with quotes)...
#! /bin/bash
if [ -n $1 ]; then
echo bar
fi
$ ./test1.sh
$ ./test1.sh foo
bar
Works as expected!
Does not work as expected unless I use quotes... Why?
From man test (and see also posix test):
-n STRING
the length of STRING is nonzero
STRING equivalent to -n STRING
-z STRING
the length of STRING is zero
When $1 is empty, then [ -n $1 ] executes [ -n ]. [ STRING ] is equivalent to [ -n STRING ], so [ -n ] is equivalent to [ -n -n ], and because -n is nonzero string, [ succeeds.
Check your scripts with http://shellcheck.net
Unquoted variable expansion undergo word splitting and filename expansion!
Research Difference between single and double quotes in Bash and When to wrap quotes around a shell variable? and https://mywiki.wooledge.org/Quotes
So I should ... always use double quotes?
As a rule of thumb you should always wrap expansions in double quotes.
So I should ... use double square brackets ... ?
The [[ is a bash extension not available in other shells, and from [[ documentation we know that: Word splitting and filename expansion are not performed on the words between the [[ and ]], so that's why there is no need to (but it does no harm) to double quote expansions between [[. [[ is specially handled by the shell, it's a special builtin with special syntax.
When working in bash-specific script that will never be ported, you may prefer [[ over [ because it should be tiny bit faster faster.
I got a bash script , for example
[root#test ~]# cat 1.sh
#!/bin/bash
var1=""
var2=""
touch "$var1"
touch $var2
it output as:
[root#test ~]# bash -x 1.sh
+ var1=
+ var2=
+ touch ''
touch: cannot touch ‘’: No such file or directory
+ touch
touch: missing file operand
so if I use "$var" over command , when it is empty , it will become '' and breaks my command
while if I use $var without double quotes , it works but shell check keep complaint to double quote to prevent word split
how can I workaround this ?
-------------edit -------
the touch here is example , please don't mind the command here , the goal is to make it output nothing
e.g.
my_command $var1 $var2 $var3 $var4
these var can be empty , so if I double quoted it
my_command "$var1" "$var2" "$var3" "$var4"
let's say if var1 and 2 is empty , then it will run as
var1=""
var2=""
var3="some"
var4="thing"
my_command "$var1" "$var2" "$var3" "$var4"
it output as
my_command '' '' some thing , and broken due to '' in argument.
if I do
var1=""
var2=""
var3="some"
var4="thing"
my_command $var1 $var2 $var3 $var4
it goes as my_command some thing which just works , but shell check keep complaints :(
You can omit empty variables from an argument list like this:
my_command ${var1:+"$var1"} ${var2:+"$var2"} ${var3:+"$var3"} ${var4:+"$var4"}
Explanation: ${var:+something} expands to something if var is defined and non-empty. So ${var:+"$var"} expands to a properly double-quoted reference to $var if the variable isn't empty. If it's empty or undefined, the whole thing expands to nothing, and since the outer reference isn't double-quoted it gets eliminated by word splitting.
When not quoted, empty string var=""; touch $var expands to nothing (as if you did not add it).
But when quoted var=""; touch "$var" then it expands to the empty string from var and this is an invalid filename argument to touch.
how can I workaround this ?
There is nothing to workaround as it works as expected. The presented command touch "$var" works as expected. When var is empty, one empty argument is being passed to touch command.
As creating a file with empty name is invalid, happily touch complains with a message.
the goal is to make it output nothing
Then check if the argument to touch is non-empty before running it.
if [ -n "$var" ]; then
somecommand "$var"
fi
If you want to omit an argument from a list if it's empty, use bash array to accumulate existing arguments and pass them properly quoted to the command:
args=()
if [[ -n "$var1" ]]; then
args+=("$var1")
fi
if [[ -n "$var2" ]]; then
args+=("$var2")
fi
somecommand "${args[#]}"
The POSIX compatible alternatives to bash arrays is to use set -- "$#" "$var" posititional arguments to accumulate the arguments or str+="$(printf " %q" "$var") properly quoted string to be evalulated.
If you want to "make it output nothing" literally, then silence stdout and stderr of a command, typically by redirecting to /dev/null:
{ somecommand ...; } >/dev/null 2>&1
You should check the validity of your variables before trying to use them. For example, you can use if [[ -z $VAR ]]:
if [[ -z $var1 || -z $var2 || -z $var3 || -z $var4 ]]; then
echo "Error: argument cannot be empty"
exit 1
fi
my_command "$var1" "$var2" "$var3" "$var4"
You can also split it into various ifs in case you want different messages for each one, for example.
I'm trying to run a for loop over a list of strings where some of them are quoted and others are not like so:
STRING='foo "bar_no_space" "baz with space"'
for item in $STRING; do
echo "$item"
done
Expected result:
foo
bar_no_space
baz with space
Actual result:
foo
"bar_no_space"
"baz
with
space"
I can achieve the expected result by running the following command:
bash -c 'for item in '"$STRING"'; do echo "$item"; done;'
I would like to do this without spawning a new bash process or using eval because I do not want to take the risk of having random commands executed.
Please note that I do not control the definition of the STRING variable, I receive it through an environment variable. So I can't write something like:
array=(foo "bar_no_space" "baz with space")
for item in "${array[#]}"; do
echo "$item"
done
If it helps, what I am actually trying to do is split the string as a list of arguments that I can pass to another command.
I have:
STRING='foo "bar_no_space" "baz with space"'
And I want to run:
my-command --arg foo --arg "bar_no_space" --arg "baz with space"
Use an array instead of a normal variable.
arr=(foo "bar_no_space" "baz with space")
To print the values:
print '%s\n' "${arr[#]}"
And to call your command:
my-command --arg "${arr[0]}" --arg "${arr[1]}" --arg "{$arr[2]}"
Can you try something like this:
sh-4.4$ echo $string
foo "bar_no_space" "baz with space"
sh-4.4$ echo $string|awk 'BEGIN{FS="\""}{for(i=1;i<NF;i++)print $i}'|sed '/^ $/d'
foo
bar_no_space
baz with space
Solved: xargs + subshell
A few years late to the party, but...
Malicious Input:
SSH_ORIGINAL_COMMAND='echo "hello world" foo '"'"'bar'"'"'; sudo ls -lah /; say -v Ting-Ting "evil cackle"'
Note: I originally had an rm -rf in there, but then I realized that would be a recipe for disaster when testing variations of the script.
Converted perfectly into safe args:
# DO NOT put IFS= on its own line
IFS=$'\r\n' GLOBIGNORE='*' args=($(echo "$SSH_ORIGINAL_COMMAND" \
| xargs bash -c 'for arg in "$#"; do echo "$arg"; done'))
echo "${args[#]}"
See that you can indeed pass these arguments just like $#:
for arg in "${args[#]}"
do
echo "$arg"
done
Output:
hello world
foo
bar;
sudo
rm
-rf
/;
say
-v
Ting-Ting
evil cackle
I'm too embarrassed to say how much time I spent researching this to figure it out, but once you get the itch... y'know?
Defeating xargs
It is possible to fool xargs by providing escaped quotes:
SSH_ORIGINAL_COMMAND='\"hello world\"'
This can make a literal quote part of the output:
"hello
world"
Or it can cause an error:
SSH_ORIGINAL_COMMAND='\"hello world"'
xargs: unmatched double quote; by default quotes are special to xargs unless you use the -0 option
In either case, it doesn't enable arbitrary execution of code - the parameters are still escaped.
Pure bash parser
Here's a quoted-string parser written in pure bash (what terrible fun)!
Caveat: just like the xargs example above, this errors in the case of an escaped quoted.
Usage
MY_ARGS="foo 'bar baz' qux * "'$(dangerous)'" sudo ls -lah"
# Create array from multi-line string
IFS=$'\r\n' GLOBIGNORE='*' args=($(parseargs "$MY_ARGS"))
# Show each of the arguments array
for arg in "${args[#]}"; do
echo "$arg"
done
Output:
$#: foo bar baz qux *
foo
bar baz
qux
*
Parse Argument Function
Literally going character-by-character and adding to the current string, or adding to the array.
set -u
set -e
# ParseArgs will parse a string that contains quoted strings the same as bash does
# (same as most other *nix shells do). This is secure in the sense that it doesn't do any
# executing or interpreting. However, it also doesn't do any escaping, so you shouldn't pass
# these strings to shells without escaping them.
parseargs() {
notquote="-"
str=$1
declare -a args=()
s=""
# Strip leading space, then trailing space, then end with space.
str="${str## }"
str="${str%% }"
str+=" "
last_quote="${notquote}"
is_space=""
n=$(( ${#str} - 1 ))
for ((i=0;i<=$n;i+=1)); do
c="${str:$i:1}"
# If we're ending a quote, break out and skip this character
if [ "$c" == "$last_quote" ]; then
last_quote=$notquote
continue
fi
# If we're in a quote, count this character
if [ "$last_quote" != "$notquote" ]; then
s+=$c
continue
fi
# If we encounter a quote, enter it and skip this character
if [ "$c" == "'" ] || [ "$c" == '"' ]; then
is_space=""
last_quote=$c
continue
fi
# If it's a space, store the string
re="[[:space:]]+" # must be used as a var, not a literal
if [[ $c =~ $re ]]; then
if [ "0" == "$i" ] || [ -n "$is_space" ]; then
echo continue $i $is_space
continue
fi
is_space="true"
args+=("$s")
s=""
continue
fi
is_space=""
s+="$c"
done
if [ "$last_quote" != "$notquote" ]; then
>&2 echo "error: quote not terminated"
return 1
fi
for arg in "${args[#]}"; do
echo "$arg"
done
return 0
}
I may or may not keep this updated at:
https://git.coolaj86.com/coolaj86/git-scripts/src/branch/master/git-proxy
Seems like a rather stupid thing to do... but I had the itch... oh well.
Here is a way without an array of strings or other difficulties (but with bash calling and eval):
STRING='foo "bar_no_space" "baz with space"'
eval "bash -c 'while [ -n \"\$1\" ]; do echo \$1; shift; done' -- $STRING"
Output:
foo
bar_no_space
baz with space
If You want to do with the strings something more difficult then just echo You can split Your script:
split_qstrings.sh
#!/bin/bash
while [ -n "$1" ]
do
echo "$1"
shift
done
Another part with more difficult processing (capitalizing of a characters for example):
STRING='foo "bar_no_space" "baz with space"'
eval "split_qstrings.sh $STRING" | while read line
do
echo "$line" | sed 's/a/A/g'
done
Output:
foo
bAr_no_spAce
bAz with spAce
I'm trying to get the first character of a variable, but I'm getting a Bad substitution error. Can anyone help me fix it?
code is:
while IFS=$'\n' read line
do
if [ ! ${line:0:1} == "#"] # Error on this line
then
eval echo "$line"
eval createSymlink $line
fi
done < /some/file.txt
Am I doing something wrong or is there a better way of doing this?
-- EDIT --
As requested - here's some sample input which is stored in /some/file.txt
$MOZ_HOME/mobile/android/chrome/content/browser.js
$MOZ_HOME/mobile/android/locales/en-US/chrome/browser.properties
$MOZ_HOME/mobile/android/components/ContentPermissionPrompt.js
To get the first character of a variable you need to say:
v="hello"
$ echo "${v:0:1}"
h
However, your code has a syntax error:
[ ! ${line:0:1} == "#"]
# ^-- missing space
So this can do the trick:
$ a="123456"
$ [ ! "${a:0:1}" == "#" ] && echo "doesnt start with #"
doesnt start with #
$ a="#123456"
$ [ ! "${a:0:1}" == "#" ] && echo "doesnt start with #"
$
Also it can be done like this:
$ a="#123456"
$ [ "$(expr substr $a 1 1)" != "#" ] && echo "does not start with #"
$
$ a="123456"
$ [ "$(expr substr $a 1 1)" != "#" ] && echo "does not start with #"
does not start with #
Update
Based on your update, this works to me:
while IFS=$'\n' read line
do
echo $line
if [ ! "${line:0:1}" == "#" ] # Error on this line
then
eval echo "$line"
eval createSymlink $line
fi
done < file
Adding the missing space (as suggested in fedorqui's answer ;) ) works for me.
An alternative method/syntax
Here's what I would do in Bash if I want to check the first character of a string
if [[ $line != "#"* ]]
On the right hand side of ==, the quoted part is treated literally whereas * is a wildcard for any sequence of character.
For more information, see the last part of Conditional Constructs of Bash reference manual:
When the ‘==’ and ‘!=’ operators are used, the string to the right of the operator is considered a pattern and matched according to the rules described below in Pattern Matching
Checking that you're using the right shell
If you are getting errors such as "Bad substitution error" and "[[: not found" (see comment) even though your syntax is fine (and works fine for others), it might indicate that you are using the wrong shell (i.e. not Bash).
So to make sure you are using Bash to run the script, either
make the script executable and use an appropriate shebang e.g. #!/bin/bash
or execute it via bash my_script
Also note that sh is not necessarily bash, sometimes it can be dash (e.g. in Ubuntu) or just plain ol' Bourne shell.
Try this:
while IFS=$'\n' read line
do
if ! [ "${line:0:1}" = "#" ]; then
eval echo "$line"
eval createSymlink $line
fi
done < /some/file.txt
or you can use the following for your if syntax:
if [[ ! ${line:0:1} == "#" ]]; then
TIMTOWTDI ^^
while IFS='' read -r line
do
case "${line}" in
"#"*) echo "${line}"
;;
*) createSymlink ${line}
;;
esac
done < /some/file.txt
Note: I dropped the eval, which could be needed in some (rare!) cases (and are dangerous usually).
Note2: I added a "safer" IFS & read (-r, raw) but you can revert to your own if it is better suited. Note that it still reads line by line.
Note3: I took the habit of using always ${var} instead of $var ... works for me (easy to find out vars in complex text, and easy to see where they begin and end at all times) but not necessary here.
Note4: you can also change the test to : *"#"*) if some of the (comments?) lines can have spaces or tabs before the '#' (and none of the symlink lines does contain a '#')
I am trying to run the below logic
if [-f $file] && [$variable -lt 1] ; then
some logic
else
print "This is wrong"
fi
It fails with the following error
MyScipt.ksh[10]: [-f: not found
Where 10th line is the if condition , I have put in .
I have also tried
if [-f $file && $variable -lt 1] ; then
which gives the same error.
I know this is a syntax mistake somehwere , but I am not sure , what is the correct syntax when I am using multiple conditions with && in a if block
[ is not an operator, it's the name of a program (or a builtin, sometimes). Use type [ to check. Regardless, you need to put a space after it so that the command line parser knows what to do:
if [ -f $file ]
The && operator might not do what you want in this case, either. You should probably read the bash(1) documentation. In this specific case, it seems like what you want is:
if [ -f $file -a $variable -lt 1 ]
Or in more modern bash syntax:
if [[ -f $file && $variable -lt 1 ]]
The [ syntax is secretly a program!
$ type [
[ is a shell builtin
$ ls -l $(which [)
-rwxr-xr-x 1 root root 35264 Nov 19 16:25 /usr/bin/[
Because of the way the shell parses (technically "lexes") your command line, it sees this:
if - keyword
[-f - the program [-f
$file] - A string argument to the [-f program, made by the value of $file and ]. If $file was "asdf", then this would be asdf]
And so forth, down your command. What you need to do is include spaces, which the shell uses to separate the different parts (tokens) of your command:
if [ -f "$file" ]; then
Now [ stands on its own, and can be recognized as a command/program. Also, ] stands on its own as an argument to [, otherwise [ will complain. A couple more notes about this:
You don't need to put a space before or after ;, because that is a special separator that the shell recognizes.
You should always "double quote" $variables because they get expanded before the shell does the lexing. This means that if an unquoted variable contains a space, the shell will see the value as separate tokens, instead of one string.
Using && in an if-test like that isn't the usual way to do it. [ (also known as test) understands -a to mean "and," so this does what you intended:
if [ -f "$file" -a "$variable" -lt 1 ]; then
Use -a in an if block to represent AND.
Note the space preceding the -f option.
if [ -f $file -a $variable -lt 1] ; then
some logic
else
print "This is wrong"
fi