Related, but not a duplicate of: How to define hash tables in Bash?
I can define and use a bash hash, but I am unable to export it, even with the -x flag. For example, the following works to export (and test exportation of) a normal string variable:
aschirma#graphics9-lnx:/$ export animal_cow="moo"
aschirma#graphics9-lnx:/$ bash -c "echo \$animal_cow"
moo
aschirma#graphics9-lnx:/$
However, if I try to export a hash:
aschirma#graphics9-lnx:/$ declare -A -x animals
aschirma#graphics9-lnx:/$ animals[duck]="quack"
aschirma#graphics9-lnx:/$ echo ${animals[duck]}
quack
aschirma#graphics9-lnx:/$ bash -c "echo \${animals[duck]}"
aschirma#graphics9-lnx:/$
It seems the nested bash shell does not have the hash in its scope. I did verify this also by manually entering the nested bash shell and attempting to use the hash interactively.
There isn't really a good way to encode an array variable into the environment. See
http://www.mail-archive.com/bug-bash#gnu.org/msg01774.html (Chet Ramey is the maintainer of bash)
As a workaround for this harsh Bash limitation I'm using "serialize to temporary file" method. You can export plain variables, so you can pass an array (associative) through filepath. Of course, this has limitations, but sometimes works and is good enough.
declare -A MAP # export associative array
MAP[bar]="baz"
declare -x serialized_array=$(mktemp) # create temporary variable
# declare -p can be used to dump the definition
# of a variable as shell code ready to be interpreted
declare -p MAP > "${serialized_array}" # serialize an array in temporary file
# perform cleanup after finishing script
cleanup() {
rm "${serialized_array}"
}
trap cleanup EXIT
# ... in place where you need this variable ...
source "${serialized_array}" # deserialize an array
echo "map: ${MAP[#]}"
This is a bit old but I answer anyway, you could use temp files. If you do it right you can wrapper it to use them like arrays. For example with this function:
var() { # set var or add comtent
case $1 in
*=|*=*)
local __var_part1=$( echo "$1" | sed -e 's/=.*//' -e 's/[+,-]//' ) # cut +=/=
local __var_part2=$( echo "$1" | sed -e 's/.*.=//' )
local __var12=$tmp_dir/$__var_part1
mkdir -p ${__var12%/*} #create all subdirs if its an array
case $1 in
*+=*)
# if its an array try to add new item
if [ -d $tmp_dir/$__var_part1 ] ; then
printf -- $__var_part2 > $tmp_dir/$__var_part1/\ $((
$( echo $tmp_dir/$__var_part2/* \
| tail | basename )\ + 1 ))
else
printf -- "$__var_part2" >> $tmp_dir/$__var_part1
fi
;;
*-=*) false ;;
# else just add content
*) printf -- "$__var_part2" > $tmp_dir/$__var_part1 ;;
esac
;;
*) # just print var
if [ -d $tmp_dir/$1 ] ; then
ls $tmp_dir/$1
elif [ -e $tmp_dir/$1 ] ; then
cat $tmp_dir/$1
else
return 1
fi
;;
esac
}
# you can use mostly like you set vars in bash/shell
var test='Hello Welt!'
# if you need arrays set it like this:
var fruits/0='Apple'
var fruits/1='Banana'
# or if you need a dict:
var contacts/1/name="Max"
var contacts/1/surname="Musterman"
This not the fastest way, but its very flexible, simple and works in nearly all shells.
short answer --> export animals after declaring it
full -->
Try this way as a script:
#!/usr/bin/env bash
declare -A -x animals
export animals
animals[duck]="quack"
echo ${animals[duck]}
bash -c "echo ${animals[duck]}"
Output on my side using Bash version: 5.1.16(1)
quack
quack
or in terminal:
$ declare -A -x animals
$ export animals
$ animals[duck]="quack"
$ echo ${animals[duck]}
quack
$ bash -c "echo ${animals[duck]}"
quack
$
Related
I've been working through creating a script to move some files from a local machine to a remote server. As part of that process I have a function that can either be called directly or wrapped with 'declare -fp' and sent along to an ssh command. The code I have so far looks like this:
export REMOTE_HOST=myserver
export TMP=eyerep-files
doTest()
{
echo "Test moving files from $TMP with arg $1"
declare -A files=(["abc"]="123" ["xyz"]="789")
echo "Files: ${!files[#]}"
for key in "${!files[#]}"
do
echo "$key => ${files[$key]}"
done
}
moveTest()
{
echo "attempting move with wrapped function"
ssh -t "$REMOTE_HOST" "$(declare -fp doTest|envsubst); doTest ${1#Q}"
}
moveTest $2
If I run the script with something like
./myscript.sh test dev
I get the output
attempting move with wrapped function
Test moving files from eyerep-files with arg dev
Files: abc xyz
bash: line 7: => ${files[]}: bad substitution
It seems like the string expansion for the for loop is not working correctly. Is this expected behaviour? If so, is there an alternative way to loop through an array that would avoid this issue?
If you're confident that your remote account's default shell is bash, this might look like:
moveTest() {
ssh -t "$REMOTE_HOST" "$(declare -f doTest; declare -p $(compgen -e)); doTest ${1#Q}"
}
If you aren't, it might instead be:
moveTest() {
ssh -t "$REMOTE_HOST" 'exec bash -s' <<EOF
set -- ${##Q}
$(declare -f doTest; declare -p $(compgen -e))
doTest \"\$#\"
EOF
}
I managed to find an answer here: https://unix.stackexchange.com/questions/294378/replacing-only-specific-variables-with-envsubst/294400
Since I'm exporting the global variables, I can get a list of them using compgen and use that list with envsubst to specify which variables I want to replace. My finished function ended up looking like:
moveTest()
{
echo "attempting move with wrapped function"
ssh -t "$REMOTE_HOST" "$(declare -fp doTest|envsubst "$(compgen -e | awk '$0="${"$0"}"') '${1}'"); doTest ${1#Q}"
}
I want to pass associate array as a single argument to shell script.
mainshellscript.sh :
#!/bin/bash
declare -A cmdOptions=( [branch]=testing [directory]=someDir [function1key]=function1value [function1key]=function1value )
./anothershellscript.sh cmdOptions
anothershellscript.sh :
#!/bin/bash
#--- first approach, didn't work, shows error "declare: : not found"
#opts=$(declare -p "$1")
#declare -A optsArr=${opts#*=}
#---
# second approach, didnt work, output - declare -A optsArr=([0]="" )
#declare -A newArr="${1#*=}"
#declare -p newArr
#---
I'm suspecting that they way I am collecting array in anothershellscript.sh is wrong, I'm finding a way to access map values simply by providing echo "${optsArr[branch]}" shall give testing.
I am using bash version 4.4.23(1)-release (x86_64-pc-msys).
Processes are separate - they do not share variables. A variable set in one process is not visible in the other process. Except: child processes inherit exported variables, but cmdOptions is not exported and you can't export associative arrays anyway.
So pass the string representation of the array to the child, either via arguments or with a file or with an exported variable:
#!/bin/bash
declare -A cmdOptions=( [branch]=testing [directory]=someDir [function1key]=function1value [function1key]=function1value )
export SOME_VAR="$(declare -p cmdOptions)" # env or
declare -p cmdOptions > some_file.txt # file or
./anothershellscript.sh "$(declare -p cmdOptions)" # arg
# Note - `declare -p` is called _on parent side_!
Then load in the child:
#!/bin/bash
declare -A newArr="${SOME_VAR#*=}" # env or
. some_file.txt # file or
declare -A newArr="${1#*=}" # arg
# How to load from file so that current environment is not affected:
tmp=$(
. some_file.txt
declare -p cmdOptions
)
declare -A newArr=${tmp#*=}"
Passing Associative array to sub script
Associative arrays and regular arrays are not well exported. But you could pass variables by using declare -p in some wrapper call.
First script:
#!/bin/bash
declare -A cmdOptions=( [branch]=testing [function1key]=function1value
[directory]=someDir [function2key]=function2value )
declare -a someArray=( foo "bar baz" )
declare -x someVariable="Foo bar baz" # Using *export* for simple variables
bash -c "$(declare -p cmdOptions someArray someVariable);. anothershellscript.sh"
Note syntax . anothershellscript.sh could be replaced by source anothershellscript.sh.
Doing so, prevent need of temporary files or temporary variable and keep STDIN/STDOUT free.
Then your anothershellscript.sh could use your variables as is.
#!/bin/bash
declare -p cmdOptions someArray someVariable
could work.
You'll have to rebuild that array in second script, here is an example script1:
#!/bin/bash
declare -A test=(
[v1]=1
[v2]=2
[v3]=3
)
for key in "${!test[#]}"; { data+=" $key=${test[$key]}"; }
echo "$data" | ./test2
script2:
#!/bin/bash
read -ra data
declare -A test2
for item in "${data[#]}"; {
key="${item%=*}"
val="${item#*=}"
test2[$key]="$val"
}
echo "${test2[#]}"
I want to write some wrappers around the sha1sum function in bash. From the manpage:
SHA1SUM(1) User Commands SHA1SUM(1)
NAME
sha1sum - compute and check SHA1 message digest
SYNOPSIS
sha1sum [OPTION]... [FILE]...
DESCRIPTION
Print or check SHA1 (160-bit) checksums.
With no FILE, or when FILE is -, read standard input.
How can I set up my wrapper so that it works in the same way? I.e.:
my_wrapper(){
# some code here
}
that could work both as:
my_wrapper PATH_TO_FILE
and
echo -n "blabla" | my_wrapper
I think this is somehow related to Redirect standard input dynamically in a bash script but not sure how to make it 'nicely'.
Edit 1
I program in a quite defensive way, so I use in my whole script:
# exit if a command fails
set -o errexit
# make sure to show the error code of the first failing command
set -o pipefail
# do not overwrite files too easily
set -o noclobber
# exit if try to use undefined variable
set -o nounset
Anything that works with that?
You can use this simple wrapper:
args=("$#") # save arguments into an array
set -o noclobber nounset pipefail errexit
set -- "${args[#]}" # set positional arguments from array
my_wrapper() {
[[ -f $1 ]] && SHA1SUM "$1" || SHA1SUM
}
my_wrapper "$#"
Note that you can use:
my_wrapper PATH_TO_FILE
or:
echo -n "blabla" | my_wrapper
This code works for me, put it in a file named wrapper
#!/bin/bash
my_wrapper(){
if [[ -z "$1" ]];then
read PARAM
else
PARAM="$1"
fi
echo "PARAM:$PARAM"
}
Load the function in your environment
. ./wrapper
Test the function with input pipe
root#51ce582167d0:~# echo hello | my_wrapper
PARAM:hello
Test the function with parameter
root#51ce582167d0:~# my_wrapper bybye
PARAM:bybye
Ok, so the answers posted here are fine often, but in my case with defensive programming options:
# exit if a command fails
set -o errexit
# exit if try to use undefined variable
set -o nounset
things do not work as well. So I am now using something in this kind:
digest_function(){
# argument is either filename or read from std input
# similar to the sha*sum functions
if [[ "$#" = "1" ]]
then
# this needs to be a file that exists
if [ ! -f $1 ]
then
echo "File not found! Aborting..."
exit 1
else
local ARGTYPE="Filename"
local PARAM="$1"
fi
else
local ARGTYPE="StdInput"
local PARAM=$(cat)
fi
if [[ "${ARGTYPE}" = "Filename" ]]
then
local DIGEST=$(sha1sum ${PARAM})
else
local DIGEST=$(echo -n ${PARAM} | sha1sum)
fi
}
This is (or was, at least) a common pattern in Ruby, but I can't figure out how to do it in Zsh or Bash.
Let's suppose I have a shell function called "whoosiwhatsit", and I want to override it in a specific project, while still keeping the original available under a different name.
If I didn't know better, I might try creating an alias to point to whoosiwhatsit, and then create a new "whoosiwhatsit" function that uses the alias. Of course that work, since the alias will refer to the new function instead.
Is there any way to accomplish what I'm talking about?
Aliases are pretty weak. You can do this with functions though. Consider the following tools:
#!/usr/bin/env bash
PS4=':${#FUNCNAME[#]}:${BASH_SOURCE}:$LINENO+'
rename_function() {
local orig_definition new_definition new_name retval
retval=$1; shift
orig_definition=$(declare -f "$1") || return 1
new_name="${1}_"
while declare -f "$new_name" >/dev/null 2>&1; do
new_name+="_"
done
new_definition=${orig_definition/"$1"/"$new_name"}
eval "$new_definition" || return
unset -f "$orig_definition"
printf -v "$retval" %s "$new_name"
}
# usage: shadow_function target_name shadowing_func [...]
# ...replaces target_name with a function which will call:
# shadowing_func target_renamed_to_this number_of_args_in_[...] [...] "$#"
shadow_function() {
local shadowed_func eval_code shadowed_name shadowing_func shadowed_func_renamed
shadowed_name=$1; shift
shadowing_func=$1; shift
rename_function shadowed_func_renamed "$shadowed_name" || return
if (( $# )); then printf -v const_args '%q ' "$#"; else const_args=''; fi
printf -v eval_code '%q() { %q %q %s "$#"; }' \
"$shadowed_name" "$shadowing_func" "$shadowed_func_renamed" "$# $const_args"
eval "$eval_code"
}
...and the following example application of those tools:
whoosiwhatsit() { echo "This is the original implementation"; }
override_in_directory() {
local shadowed_func=$1; shift
local override_cmd_len=$1; shift
local override_dir=$1; shift
local -a override_cmd=( )
local i
for (( i=1; i<override_cmd_len; i++)); do : "$1"
override_cmd+=( "$1" ); shift
done
: PWD="$PWD" override_dir="$override_dir" shadowed_func="$shadowed_func"
: override_args "${override_args[#]}"
if [[ $PWD = $override_dir || $PWD = $override_dir/* ]]; then
[[ $- = *x* ]] && declare -f shadowed_func >&2 # if in debugging mode
"${override_cmd[#]}"
else
"$shadowed_func" "$#"
fi
}
ask_the_user_first() {
local shadowed_func=$1; shift;
shift # ignore static-argument-count parameter
if [[ -t 0 ]]; then
read -r -p "Press ctrl+c if you are unsure, or enter if you are"
fi
"$shadowed_func" "$#"
}
shadow_function whoosiwhatsit ask_the_user_first
shadow_function whoosiwhatsit \
override_in_directory /tmp echo "Not in the /tmp!!!"
shadow_function whoosiwhatsit \
override_in_directory /home echo "Don't try this at home"
The end result is a whoosiwhatsit function that asks the user before it does anything when its stdin is a TTY, and aborts (with different messages) when run under either /tmp or /home.
That said, I don't condone this practice. Consider the above provided as an intellectual exercise. :)
In bash, there is a built-in variable called BASH_ALIASES that is an associative array containing the current aliases. The semantics are a bit inconsistent when you update it (RTM!) but if you restrict yourself to reading BASH_ALIASES, you should be able to write yourself a shell function that implements alias chaining.
It's common and well supported to create a single level of overrides through functions that optionally invoke their overridden builtin or command:
# Make all cd commands auto-exit on failure
cd() { builtin cd "$#" || exit; }
# Make all ssh commands verbose
ssh() { command ssh -vv "$#"; }
It doesn't chain beyond the one link, but it's completely POSIX and often works better in practice than trying to write Ruby in Bash.
I want to create a function locally, echo_a in the example, and pass it with to a remote shell through ssh, here with typeset -f. The problem is that function does not have access to the local variables.
export a=1
echo_a() {
echo a: $a
}
bash <<EOF
$(typeset -f echo_a)
echo local heredoc:
echo_a
echo
echo local raw heredoc:
echo a: $a
echo
EOF
ssh localhost bash <<EOF
$(typeset -f echo_a)
echo remote heredoc:
echo_a
echo
echo remote raw heredoc:
echo a: $a
echo
EOF
Assuming the ssh connection is automatic, running the above script gives me as output:
local heredoc:
a: 1
local raw heredoc:
a: 1
remote heredoc:
a:
remote raw heredoc:
a: 1
See how the "remote heredoc" a is empty? What can I do to get 1 there?
I tested adding quotes and backslashes everywhere without success.
What am I missing? Would something else than typeset make this work?
Thanks to #Guy for the hint, it indeed is because ssh disables by default sending the environment variables. In my case, changing the server's setting was not wanted.
Hopefully we can hack around by using compgen, eval and declare.
First we identify added variables generically. Works if variables are created inside a called function too. Using compgen is neat because we don't need to export variables explicitely.
The array diff code comes from https://stackoverflow.com/a/2315459/1013628 and the compgen trick from https://stackoverflow.com/a/16337687/1013628.
# Store in env_before all variables created at this point
IFS=$'\n' read -rd '' -a env_before <<<"$(compgen -v)"
a=1
# Store in env_after all variables created at this point
IFS=$'\n' read -rd '' -a env_after <<<"$(compgen -v)"
# Store in env_added the diff betwen env_after and env_before
env_added=()
for i in "${env_after[#]}"; do
skip=
for j in "${env_before[#]}"; do
[[ $i == $j ]] && { skip=1; break; }
done
if [[ $i == "env_before" || $i == "PIPESTATUS" ]]; then
skip=1
fi
[[ -n $skip ]] || env_added+=("$i")
done
echo_a() {
echo a: $a
}
env_added holds now an array of all names of added variables between the two calls to compgen.
$ echo "${env_added[#]}"
a
I filter out also the variables env_before and PIPESTATUS as they are added automatically by bash.
Then, inside the heredocs, we add eval $(declare -p "${env_added[#]}").
declare -p VAR [VAR ...] prints, for each VAR, the variable name followed by = followed by its value:
$ a = 1
$ b = 2
$ declare -p a b
declare -- a=1
declare -- b=2
And the eval is to actually evaluate the declare lines. The rest of the code looks like:
bash <<EOF
# Eval the variables computed earlier
eval $(declare -p "${env_added[#]}")
$(typeset -f echo_a)
echo local heredoc:
echo_a
echo
echo local raw heredoc:
echo a: $a
echo
EOF
ssh rpi_301 bash <<EOF
# Eval the variables computed earlier
eval $(declare -p "${env_added[#]}")
$(typeset -f echo_a)
echo remote heredoc:
echo_a
echo
echo remote raw heredoc:
echo a: $a
echo
EOF
Finally, running the modified script gives me the wanted behavior:
local heredoc:
a: 1
local raw heredoc:
a: 1
remote heredoc:
a: 1
remote raw heredoc:
a: 1