I've looked over a couple other variable variable posts here but still seem stuck with what I'm trying to attempt.
I have an existing script that has a series of similar block like
set_name="something"
# Required values
var1=$(REQUIRED_ENV_VAR=/path/to/somewhere mybinary -p "key1_${set_name}")
if [[ -z "$var1" ]]; then
echo "Cannot find required var1"
exit 1
fi
var2=$(REQUIRED_ENV_VAR=/path/to/somewhere mybinary -p "key2_${set_name}")
if [[ -z "$var2" ]]; then
echo "Cannot find required var2"
exit 1
fi
# Optional, okay to be empty
var3=$(REQUIRED_ENV_VAR=/path/to/somewhere mybinary -p "key3_${set_name}")
var4=$(REQUIRED_ENV_VAR=/path/to/somewhere mybinary -p "key4_${set_name}")
I was trying to factor some of the boilerplate checks out to keep this set of lookups assignment easier to read (in my opinion anyway). The last iteration I had attempted (clearly not working) looks like
ZTest () {
var=$1
if [[ -z "${!var}" ]]; then
echo $2
exit 1
fi
}
VarRequire () {
var=$1
key=$2
errmsg=$3
VarLookup ${!var} $key
ZTest ${!var} $errmsg
}
VarLookup () {
var=$1
key=$2
${!var}=$(REQUIRED_ENV_VAR=/path/to/somewhere mybinary -p "$key")
}
# Required
VarRequire "var1" "key1_${set_name}" "Cannot find required var1"
VarRequire "var2" "key2_${set_name}" "Cannot find required var2"
# optional
VarLookup "var3" "key3_${set_name}"
VarLookup "var4" "key4_${set_name}"
The end result is I would be able to reference $var1, $var2, $var3, $var4 down the line in the script just the same as the original.
Is what I'm attempting possible in bash?
ZTest is already available as a parameter expansion operator:
: ${var1:?Cannot find required var1}
You are close with VarLookup, though; you need to use the declare command to create the variable. ${!var} is only for accessing the value once the variable exists. (Note that declare requires the -g option to avoid creating a local variable, and that option was only introduced in version 4.2.)
VarLookup () {
local var=$1
local key=$2
declare -g "${var}=$(REQUIRED_ENV_VAR=/path/to/somewhere mybinary -p "$key")"
}
Prior to version 4.2, you can use printf in place of declare:
printf -v "$var" '%s' "$(REQUIRED_ENV_VAR=/path/to/somewhere mybinary -p "$key")"
I would resist the urge to refactor too much in shell, as indirection can be fragile. I'd suggest something more direct like
var_lookup () {
REQUIRED_ENV_VAR=/path/to/somewhere mybinary -p "${1}_$set_name"
}
var1=$(var_lookup key1); : ${var1:?Cannot find required var1}
var2=$(var_lookup key2); : ${var2:?Cannot find required var2}
var3=$(var_lookup key3)
var4=$(var_lookup key4)
Too much indirection in the Var* functions
VarRequire () {
local var=$1
local key=$2
local errmsg=$3
VarLookup "$var" "$key"
ZTest "$var" "$errmsg"
}
VarLookup () {
local var=$1
local key=$2
declare -g "$var"="$(REQUIRED_ENV_VAR=/path/to/somewhere mybinary -p "$key")"
}
The declare command allows you to use a variable's value as the variable name. I use the -g option so the variable is global.
The ZTest function does require the indirection.
From the BASH manpage
local [option] [name[=value] ...]
For each argument, a local variable named name is created,
and assigned value. The option can be any of the options
accepted by declare. When local is used within a function, it
causes the variable name to have a visible scope restricted to
that function and its children. With no operands, local writes a
list of local variables to the standard output. It is an
error to use local when not within a function. The return status
is 0 unless local is used outside a function, an invalid name is
supplied, or name is a readonly variable.
So the answer is yes. If you declare a variable a local within a function, and it calls another function, that function has access to the variable as well.
Can you set REQUIRED_ENV_VAR=/path/to/somewhere once at the start?
And have your mybinary return 0 when a var is found?
You might like something like
function mylog {
errortype=$1
shift
echo "(maybe show ${errortype}) $*"
if [ "$errortype" = "error" ]; then
exit 1
fi
}
var1=$(mybinary -p "key1_${set_name}") || mylog error "Cannot find required var1"
var2=$(mybinary -p "key2_${set_name}") || mylog error "Cannot find required var2"
var3=$(mybinary -p "key3_${set_name}")
var4=$(mybinary -p "key4_${set_name}") || mylog debug "Just want to say something"
Related
I want to write some wrappers around the sha1sum function in bash. From the manpage:
SHA1SUM(1) User Commands SHA1SUM(1)
NAME
sha1sum - compute and check SHA1 message digest
SYNOPSIS
sha1sum [OPTION]... [FILE]...
DESCRIPTION
Print or check SHA1 (160-bit) checksums.
With no FILE, or when FILE is -, read standard input.
How can I set up my wrapper so that it works in the same way? I.e.:
my_wrapper(){
# some code here
}
that could work both as:
my_wrapper PATH_TO_FILE
and
echo -n "blabla" | my_wrapper
I think this is somehow related to Redirect standard input dynamically in a bash script but not sure how to make it 'nicely'.
Edit 1
I program in a quite defensive way, so I use in my whole script:
# exit if a command fails
set -o errexit
# make sure to show the error code of the first failing command
set -o pipefail
# do not overwrite files too easily
set -o noclobber
# exit if try to use undefined variable
set -o nounset
Anything that works with that?
You can use this simple wrapper:
args=("$#") # save arguments into an array
set -o noclobber nounset pipefail errexit
set -- "${args[#]}" # set positional arguments from array
my_wrapper() {
[[ -f $1 ]] && SHA1SUM "$1" || SHA1SUM
}
my_wrapper "$#"
Note that you can use:
my_wrapper PATH_TO_FILE
or:
echo -n "blabla" | my_wrapper
This code works for me, put it in a file named wrapper
#!/bin/bash
my_wrapper(){
if [[ -z "$1" ]];then
read PARAM
else
PARAM="$1"
fi
echo "PARAM:$PARAM"
}
Load the function in your environment
. ./wrapper
Test the function with input pipe
root#51ce582167d0:~# echo hello | my_wrapper
PARAM:hello
Test the function with parameter
root#51ce582167d0:~# my_wrapper bybye
PARAM:bybye
Ok, so the answers posted here are fine often, but in my case with defensive programming options:
# exit if a command fails
set -o errexit
# exit if try to use undefined variable
set -o nounset
things do not work as well. So I am now using something in this kind:
digest_function(){
# argument is either filename or read from std input
# similar to the sha*sum functions
if [[ "$#" = "1" ]]
then
# this needs to be a file that exists
if [ ! -f $1 ]
then
echo "File not found! Aborting..."
exit 1
else
local ARGTYPE="Filename"
local PARAM="$1"
fi
else
local ARGTYPE="StdInput"
local PARAM=$(cat)
fi
if [[ "${ARGTYPE}" = "Filename" ]]
then
local DIGEST=$(sha1sum ${PARAM})
else
local DIGEST=$(echo -n ${PARAM} | sha1sum)
fi
}
This is (or was, at least) a common pattern in Ruby, but I can't figure out how to do it in Zsh or Bash.
Let's suppose I have a shell function called "whoosiwhatsit", and I want to override it in a specific project, while still keeping the original available under a different name.
If I didn't know better, I might try creating an alias to point to whoosiwhatsit, and then create a new "whoosiwhatsit" function that uses the alias. Of course that work, since the alias will refer to the new function instead.
Is there any way to accomplish what I'm talking about?
Aliases are pretty weak. You can do this with functions though. Consider the following tools:
#!/usr/bin/env bash
PS4=':${#FUNCNAME[#]}:${BASH_SOURCE}:$LINENO+'
rename_function() {
local orig_definition new_definition new_name retval
retval=$1; shift
orig_definition=$(declare -f "$1") || return 1
new_name="${1}_"
while declare -f "$new_name" >/dev/null 2>&1; do
new_name+="_"
done
new_definition=${orig_definition/"$1"/"$new_name"}
eval "$new_definition" || return
unset -f "$orig_definition"
printf -v "$retval" %s "$new_name"
}
# usage: shadow_function target_name shadowing_func [...]
# ...replaces target_name with a function which will call:
# shadowing_func target_renamed_to_this number_of_args_in_[...] [...] "$#"
shadow_function() {
local shadowed_func eval_code shadowed_name shadowing_func shadowed_func_renamed
shadowed_name=$1; shift
shadowing_func=$1; shift
rename_function shadowed_func_renamed "$shadowed_name" || return
if (( $# )); then printf -v const_args '%q ' "$#"; else const_args=''; fi
printf -v eval_code '%q() { %q %q %s "$#"; }' \
"$shadowed_name" "$shadowing_func" "$shadowed_func_renamed" "$# $const_args"
eval "$eval_code"
}
...and the following example application of those tools:
whoosiwhatsit() { echo "This is the original implementation"; }
override_in_directory() {
local shadowed_func=$1; shift
local override_cmd_len=$1; shift
local override_dir=$1; shift
local -a override_cmd=( )
local i
for (( i=1; i<override_cmd_len; i++)); do : "$1"
override_cmd+=( "$1" ); shift
done
: PWD="$PWD" override_dir="$override_dir" shadowed_func="$shadowed_func"
: override_args "${override_args[#]}"
if [[ $PWD = $override_dir || $PWD = $override_dir/* ]]; then
[[ $- = *x* ]] && declare -f shadowed_func >&2 # if in debugging mode
"${override_cmd[#]}"
else
"$shadowed_func" "$#"
fi
}
ask_the_user_first() {
local shadowed_func=$1; shift;
shift # ignore static-argument-count parameter
if [[ -t 0 ]]; then
read -r -p "Press ctrl+c if you are unsure, or enter if you are"
fi
"$shadowed_func" "$#"
}
shadow_function whoosiwhatsit ask_the_user_first
shadow_function whoosiwhatsit \
override_in_directory /tmp echo "Not in the /tmp!!!"
shadow_function whoosiwhatsit \
override_in_directory /home echo "Don't try this at home"
The end result is a whoosiwhatsit function that asks the user before it does anything when its stdin is a TTY, and aborts (with different messages) when run under either /tmp or /home.
That said, I don't condone this practice. Consider the above provided as an intellectual exercise. :)
In bash, there is a built-in variable called BASH_ALIASES that is an associative array containing the current aliases. The semantics are a bit inconsistent when you update it (RTM!) but if you restrict yourself to reading BASH_ALIASES, you should be able to write yourself a shell function that implements alias chaining.
It's common and well supported to create a single level of overrides through functions that optionally invoke their overridden builtin or command:
# Make all cd commands auto-exit on failure
cd() { builtin cd "$#" || exit; }
# Make all ssh commands verbose
ssh() { command ssh -vv "$#"; }
It doesn't chain beyond the one link, but it's completely POSIX and often works better in practice than trying to write Ruby in Bash.
I have an if statement which is used many times in my script:
if [[ $? -ne 0 ]]; then
echo [$JOB_NAME] failed.
exit 1
fi
is it possible to define a variable and assign this statement to it, and then call it each time I need ti?
Example of use:
echo [$JOB_NAME] extracting manifests...
unzip -o ZIP_FILE "*.yml"
# Push the app to CF
cf push -f $MANIFEST_FILE -p ZIP_FILE $NEW_APP
if [[ $? -ne 0 ]]; then
echo [$JOB_NAME] failed.
exit 1
fi
As mentioned in the comments, this should be a function, which could be written like this:
ensure_success () {
if [[ $# -eq 0 ]]; then
echo "no command passed to ensure_success."
elif ! "$#"; then
echo "[$JOB_NAME] failed."
exit 1
fi
}
"$#" expands to the full list of arguments passed to the function. I added a check based on Inian's suggestion in the comments, to ensure that at least one argument is passed to the function.
This combines running the command and checking the error code, so you can use it like:
ensure_success command arg1 arg2 arg3
So, based on the example in your question it would be:
ensure_success cf push -f "$MANIFEST_FILE" -p ZIP_FILE "$NEW_APP"
The quotes are free.
I'm not sure where $JOB_NAME is defined but presumably it is a global.
In specifically Bash version 4.0.0, is there any way to work around the use of an empty $# raising an unbound variable error when set -u is enabled?
Consider the following:
#!/usr/bin/env bash-4.0.0-1
set -xvu
echo "$BASH_VERSION"
echo "${BASH_VERSINFO[#]}"
main () {
printf '%q\n' "${#:-}"
}
main "${#:-}"
Gives me the following output when I provide an empty set of arguments:
neech#nicolaw.uk:~ $ ./test.sh
echo "$BASH_VERSION"
+ echo '4.0.0(1)-release'
4.0.0(1)-release
echo "${BASH_VERSINFO[#]}"
+ echo 4 0 0 1 release x86_64-unknown-linux-gnu
4 0 0 1 release x86_64-unknown-linux-gnu
main () {
printf '%q\n' "${#:-}"
}
main "${#:-}"
./test.sh: line 12: $#: unbound variable
I only see this behaviour in Bash version 4.0.0.
I was hoping that using variable substitution ${#:-} would allow me to work around this, but it seems not.
Is there a way to work around this?
$#, $* are special variables so should always be defined it's a bug
https://unix.stackexchange.com/questions/16560/bash-su-unbound-variable-with-set-u
a workaround, maybe:
set +u
args=("$#")
set -u
main "${args[#]}"
or maybe also
main "${#:+$#}"
Why not do error handling on your own? This way you can control exactly what happens when an exception is encountered, for instance return a custom exit code and message for that error, rather than be confined to some predefined behavior.
function log_error
{
[[ $# -ne 1 ]] && return 1
typeset msg="$1"
typeset timestamp=$(date "+%F %T")
echo "[${timestamp}] [ERROR] - $msg " >&2
}
if [[ -z "$BASH_VERSION" ]]
then
log_error "BASH VERSION is not set"
exit 1
fi
I am currently learning bash programming and dont really understand why the passing argument for me is not working.
i have a script like this
#!/bin/bash
# the following environment variables must be set before running this script
# SIM_DIR name of directory containing armsim
# TEST_DIR name of the directory containing this script and the expected outputs
# LOG_DIR name of the directory that your output is written to by the run_test2 script
# ARMSIM_VERBOSE set to "-v" for verbose logging or leave unset
# First check the environment variables are set
giveup=0
if [[ ${#SIM_DIR} -eq 0 || ${#TEST_DIR} -eq 0 || ${#LOG_DIR} -eq 0 ]] ; then
echo One or more of the following environment variables must be set:
echo SIM_DIR, TEST_DIR, LOG_DIR
giveup=1
fi
# Now check the verbose flag
if [[ ${#ARMSIM_VERBOSE} != 0 && "x${ARMSIM_VERBOSE}" != "x-v" ]] ; then
echo ARMSIM_VERBOSE must be unset, empty or set to -v
giveup=1
fi
# Stop if environment is not set up
if [ ${giveup} -eq 1 ] ; then
exit 0
fi
cd ${TEST_DIR}
for i in test2-*.sh; do
echo "**** Running test ${i%.sh} *****"
./$i > ${LOG_DIR}/${i%.sh}.log
done
When I run the .sh file and pass in 3 example argument as below:-
$ ./run_test2 SIM_DIR TEST_DIR LOG_DIR
It still show: One or more of the following environment variables must be set:
SIM_DIR, TEST_DIR, LOG_DIR
Can anyone guide me on this? Thank you.
That's not how it's intended to work. The environment variables must be set beforehand either in the script or in the terminal like
export SIM_DIR=/home/someone/simulations
export TEST_DIR=/home/someone/tests
export LOG_DIR=/home/someone/logs
./run_test2
If you use these variables frequently, you might want to export them in ~/.bashrc. The syntax is identical to the exports in the above example.
Environment variables aren't really arguments in the sense I understand from your question/example. It sounds to me like you want to give arguments to a function/script, if you do that you can find your arguments in $1-9 (I think bash supports even more, unsure), the number of arguments are stored in $#
Example function that expects two arguments:
my_func() {
if [ $# -ne 2 ]; then
printf "You need to give 2 arguments\n"
return
fi
printf "Your first argument: %s\n" "$1"
printf "Your second argument: $s\n" "$2"
}
# Call the functionl like this
my_func arg1 arg2