bash variable substitution not working on Solaris - bash

I have this code snippet running on several Linux boxes, and a Solaris 10 box with bash 3.6 (iirc). However, on a Solaris 11 box, with GNU bash, version 4.4.11(1)-release (sparc-sun-solaris2.11) it gives the following error.
#!/bin/env bash
CLEAN_COUNT() {
local L_STRING=$(sed '/[^[:graph:][:space:]]/{
s/[^[:graph:][:space:]]//g; s/\[[0-9]*m//g; s/(B//g
}' <<<$*) || return 1
echo ${#L_STRING}
}
f() {
ARGS=($#)
echo $((${#ARGS[1]:-0} - $(CLEAN_COUNT ${ARGS[1]:-0}) ))
}
f one two three four
Error received: ./gather_data.bash: line 15: ${#ARGS[1]:-0} - $(CLEAN_COUNT ${ARGS[1]:-0}) : bad substitution
I've isolated the above code in it's own script, I've compared the shopt and set -o settings on that box with another one. I'm perplexed. If I can get the code to work without the substitution, even if ARGS has no element 1 and I'm running set -o nounset, then I will use another piece of code.

Changes affecting this happened in Bash 4.3 and Bash 4.4. Observe:
No error in Bash 4.2:
$ docker run --rm -it bash:4.2 bash -u
bash-4.2$ bash --version | head -n 1
GNU bash, version 4.2.53(2)-release (x86_64-pc-linux-musl)
bash-4.2$ declare -a var && echo "${#var[1]:-1}"
0
but this doesn't actually print my default value: var[1] is the empty string, hence 0. -u seems to ignore that var has no elements. There is no difference in behaviour between echo "${#var[1]:-1}", echo "${#var[1]}" and echo "${#var[1]}", they all print 0.
Bash 4.3 complains about unbound variable:
$ docker run --rm -it bash:4.3 bash -u
bash-4.3$ bash --version | head -n 1
GNU bash, version 4.3.48(1)-release (x86_64-pc-linux-musl)
bash-4.3$ declare -a var && echo "${#var[1]:-1}"
bash: var: unbound variable
Bash 4.4 complains about substitution:
$ docker run --rm -it bash:4.4 bash -u
bash-4.4$ bash --version | head -n 1
GNU bash, version 4.4.19(1)-release (x86_64-pc-linux-musl)
bash-4.4$ declare -a var && echo "${#var[1]:-1}"
bash: ${#var[1]:-1}: bad substitution
even without set -u:
bash-4.4# set +o nounset
bash-4.4# declare -a var && echo "${#var[1]:-1}"
bash: ${#var[1]:-1}: bad substitution
Also, ${#var:-1} is considered "bad substitution" in all versions, even without set -u:
$ for v in 3.2 4.{0..4}; do docker run --rm -it bash:$v; done
bash-3.2# echo "${#var:-1}"
bash: ${#var:-1}: bad substitution
bash-3.2# exit
exit
bash-4.0# echo "${#var:-1}"
bash: ${#var:-1}: bad substitution
bash-4.0# exit
exit
bash-4.1# echo "${#var:-1}"
bash: ${#var:-1}: bad substitution
bash-4.1# exit
exit
bash-4.2# echo "${#var:-1}"
bash: ${#var:-1}: bad substitution
bash-4.2# exit
exit
bash-4.3# echo "${#var:-1}"
bash: ${#var:-1}: bad substitution
bash-4.3# exit
exit
bash-4.4# echo "${#var:-1}"
bash: ${#var:-1}: bad substitution
bash-4.4# exit
exit
I can't see any mention of changes to this behaviour in NEWS, but it seems to make sense, as ${#var[0]:-1} doesn't default to 1 anyway, so now the behaviour is consistent across scalars and arrays.
This being said, I'd rewrite your function as follows:
f () {
local args=("$#")
if [[ -z ${args[1]:-} ]]; then
echo 0
else
echo $(( ${#args[1]} - $(clean_count "${args[1]}") ))
fi
}
Rename uppercase variable names to lowercase to avoid clash with shell and environment variables
Make args local to function
Quote "$#" in args to avoid splitting before assigning to array elements
Check if args[1] is the empty string, make sure no unset complaint is triggered with ${args[1]:-}
Treat cases for empty and non-empty string separately
Alternatively, if f () is not a simplification and you never access elements other than what you show, you could further simplify to
f () {
if [[ -z ${2:-} ]]; then
echo 0
else
echo $(( ${#2} - $(clean_count "$2") ))
fi
}

Related

Bash script working locally but returning syntax error in CI

On my gitlab CI I am running the following simple script (.gitlab-ci.yml):
STR=$(cat $FILE)
if grep -q "substring" <<< "$STR"; then echo "ok"; fi
Unfortunatley this gives me the error
/bin/sh: eval: line 100: syntax error: unexpected redirection
Running the same command locally as a script is working as expected:
#!/bin/sh
FILE="./file.txt"
STR=$(cat $FILE)
if grep -q "substring" <<< "$STR"; then
echo "ok"
fi
The file has the content:
This has a substring somewhere
/bin/sh is not bash and <<< is a bash extension not available on every shell. Install bash, change shebang to /bin/bash and make sure the script is run under bash or use posix compatible syntax printf "%s\n" "$str" | grep...
Note: UPPER CASE VARIABLES are by convention reserved for exported variables, like IFS COLUMNS PWD UID EUID LINES etc. Use lower case variables in your scripts.

Bash with few commands, whitelisting few built-in commands

Trying to open a bash shell with limited command capability.
Tried command line options like -r restriction but doesn't give intended result. Also tried shopt & unset commands.
bash --noprofile --noediting --verbose --version --init-file test.sh
unset ls
shopt -u -o history
Start a bash shell with only few built-in commands. For example cd, ls, cat only.
The usage of such a shell would be for read only purposes for Directory Navigation, Listing & File viewing purpose
You can take the list of all builtins and declare functions with the same name.
I did it like this:
File bash_limited.sh:
#!/bin/bash
export PATH=
eval "$(
echo '
:
.
[
alias
bg
bind
break
builtin
caller
cd
command
compgen
complete
compopt
continue
declare
dirs
disown
echo
enable
eval
exec
exit
export
fc
fg
getopts
hash
help
history
jobs
kill
let
local
logout
mapfile
popd
printf
pushd
pwd
read
readarray
readonly
return
set
shift
shopt
source
test
times
trap
type
typeset
ulimit
umask
unalias
unset
wait
' |
while IFS= read -r line; do
case "$line" in
''|ls|cat|cd|return|printf) continue; ;;
esac
printf "%s\n" "function $line () { /bin/printf -- 'bash: $line: Command not found.\n' >&2; return 127; }"
done
echo 'function ls() { /bin/ls "$#"; }'
echo 'function cat() { /bin/cat "$#"; }'
)" ## eval
Then I open a new shell and do:
$ source bash_limited.sh
after that it's just:
$ .
bash: .: Command not found.
$ :
bash: :: Command not found.
$ source
bash: source: Command not found.
$ declare
bash: declare: Command not found.
You can also use some chroot techniques with some other PATH restriction and it will be hard to get out.

How can I tell if I'm in a child shell

If I'm using bash and type bash I'm in the child shell and need to type exit to go back to the original parent shell. If I forget which one I'm in how do I check?
Use the SHLVL environment variable.
man bash:
SHLVL : Incremented by one each time an instance of bash is started.
$ echo "$SHLV"
1
$ bash
$ echo "$SHLV"
2
$ exit
$ echo "$SHLV"
1
This is an inferior answer but you can also use pstree:
$ pstree -s $BASHPID
systemd───systemd───gnome-terminal-───bash───pstree
$ bash
$ pstree -s $BASHPID
systemd───systemd───gnome-terminal-───bash───bash───pstree

bash on ubuntu 16: set -e not inheriting inside subshell

When I run this command
set -e; echo $(echo "$-");
I get himBH as the output. I was expecting the letter e to be included in the output. Whats going on?
I'm on Ubuntu 16.04.1 LTS with
GNU bash, version 4.3.46(1)-release (x86_64-pc-linux-gnu)
Command substitutions do not inherit the errexit option unless you are in POSIX mode or you use the inherit_errexit shell option (added to bash 4.4).
192% bash -ec 'echo "$(echo "$-")"'
hBc
192% bash --posix -ec 'echo "$(echo "$-")"'
ehBc
192% bash -O inherit_errexit -ec 'echo "$(echo "$-")"' # 4.4+
ehBc
This question!
Worked on this for a couple of hours until I found htis.
I was not able to have set -e inherited to the subshells.
This is my proof of concept:
#!/usr/bin/env bash
set -euo pipefail
# uncomment to handle failures properly
# shopt -s inherit_errexit
function callfail() {
echo "SHELLOPTS - callfail - $SHELLOPTS" >&2
local value
value=$(fail)
echo "echo will reset the result to 0"
}
function fail() {
echo "SHELLOPTS - fail - $SHELLOPTS" >&2
echo "failing" >&2
return 1
}
function root() {
local hello
hello=$(callfail)
echo "nothing went bad in callfail"
callfail
echo "nothing went bad in callfail"
}
root
execution without shopt -s inherit_errexit:
$ ./test.sh
SHELLOPTS - callfail - braceexpand:hashall:interactive-comments:nounset:pipefail
SHELLOPTS - fail - braceexpand:hashall:interactive-comments:nounset:pipefail
failing
nothing went bad in callfail
SHELLOPTS - callfail - braceexpand:errexit:hashall:interactive-comments:nounset:pipefail
SHELLOPTS - fail - braceexpand:hashall:interactive-comments:nounset:pipefail
failing
execution with shopt -s inherit_errexit:
$ ./test.sh
SHELLOPTS - callfail - braceexpand:errexit:hashall:interactive-comments:nounset:pipefail
SHELLOPTS - fail - braceexpand:errexit:hashall:interactive-comments:nounset:pipefail
failing

Unbound variable not causing exit from subshell when set -eu

$ cat test.sh
set -eu
echo "`wc -l < $DNE`"
echo should not get here
$ /bin/bash test.sh
test.sh: line 2: DNE: unbound variable
should not get here
I'm running bash version 4.1.2. Is there a way to make sure all such usage of unbound variables in subshells cause the script to exit without having to modify each call involving a subshell?
The better solution to ensure variable sanitisation
#!/usr/bin/env bash
set -eu
if [[ ${1-} ]]; then
DNE=$1
else
echo "ERROR: Please enter a valid filename" 1>&2
exit 1
fi
By putting a hyphen in to the variable name inside curly braces like this allows bash to sanely handle the variable being undefined. I also highly recommend looking at the Google shell style guide, it's a great reference https://google.github.io/styleguide/shell.xml
[[ -z ${variable-} ]] \
&& echo "ERROR: Unset variable \${variable}" \
&& exit 1 \
|| echo "INFO: Using variable (${variable})"
Use a temporary variable so as to let test.sh process know about the failure of wc. You can change it to:
#!/bin/bash
set -eu
out=$(wc -l < $DNE)
echo $out
echo should not get here
Now, you won't see the should not get here if wc fails.

Resources