Iterating array in declared function of bash shell script - bash

I've been working through creating a script to move some files from a local machine to a remote server. As part of that process I have a function that can either be called directly or wrapped with 'declare -fp' and sent along to an ssh command. The code I have so far looks like this:
export REMOTE_HOST=myserver
export TMP=eyerep-files
doTest()
{
echo "Test moving files from $TMP with arg $1"
declare -A files=(["abc"]="123" ["xyz"]="789")
echo "Files: ${!files[#]}"
for key in "${!files[#]}"
do
echo "$key => ${files[$key]}"
done
}
moveTest()
{
echo "attempting move with wrapped function"
ssh -t "$REMOTE_HOST" "$(declare -fp doTest|envsubst); doTest ${1#Q}"
}
moveTest $2
If I run the script with something like
./myscript.sh test dev
I get the output
attempting move with wrapped function
Test moving files from eyerep-files with arg dev
Files: abc xyz
bash: line 7: => ${files[]}: bad substitution
It seems like the string expansion for the for loop is not working correctly. Is this expected behaviour? If so, is there an alternative way to loop through an array that would avoid this issue?

If you're confident that your remote account's default shell is bash, this might look like:
moveTest() {
ssh -t "$REMOTE_HOST" "$(declare -f doTest; declare -p $(compgen -e)); doTest ${1#Q}"
}
If you aren't, it might instead be:
moveTest() {
ssh -t "$REMOTE_HOST" 'exec bash -s' <<EOF
set -- ${##Q}
$(declare -f doTest; declare -p $(compgen -e))
doTest \"\$#\"
EOF
}

I managed to find an answer here: https://unix.stackexchange.com/questions/294378/replacing-only-specific-variables-with-envsubst/294400
Since I'm exporting the global variables, I can get a list of them using compgen and use that list with envsubst to specify which variables I want to replace. My finished function ended up looking like:
moveTest()
{
echo "attempting move with wrapped function"
ssh -t "$REMOTE_HOST" "$(declare -fp doTest|envsubst "$(compgen -e | awk '$0="${"$0"}"') '${1}'"); doTest ${1#Q}"
}

Related

How to create files in the shell where variables are not replaced

I need to create a file for subsequent nohup execution, the file contains some variables, I don't want the variables to be replaced when I generate through the file.
When I executed the code with the following code, all the variables were replaced in the generated script, which was not what I expected.
#!/bin/bash
gen_script() {
filepath=$1
if [ ! -f "$filepath" ]; then
cat >${filepath} <<EOF
#!/bin/bash
# Code generated by main.sh; DO NOT EDIT.
test(){
ip=$1
port=$2
restorefile=$3
redis-cli -h $ip -p $port --pipe < $restorefile
}
test "$#"
EOF
fi
}
main(){
gen_script exec.sh
nohup bash exec.sh $1 $2 > nohup.out 2>&1 &
}
main "$#"
How can I change my code please? I really appreciate any help with this.
To disable expansions in here document, quote the delimieter:
cat <<'EOF'
... $not_expanded
EOF
Instead, let bash serialize the function.
#!/bin/bash
work() {
ip=$1
port=$2
restorefile=$3
redis-cli -h $ip -p $port --pipe < $restorefile
}
gen_script() {
filepath=$1
if [ ! -f "$filepath" ]; then
cat >${filepath} <<EOF
#!/bin/bash
# Code generated by main.sh; DO NOT EDIT.
$(declare -f work)
work "\$#"
EOF
fi
}
main() {
gen_script exec.sh
nohup bash exec.sh "$1" "$2" > nohup.out 2>&1 &
}
main "$#"
Check your script with shellcheck. Do not define a function named test, there is already a super standard command test which is an alias for command [.

Saving a remote variable locally with heredoc through ssh

I am trying to run a local function remotely on a machine and retrieve the values of multiple variables of this function in a local function.
The first line of the heredoc section enables me to run the function remotely by defining it on the remote machine.
With the local machine named localmach and the remote machine named remotemach
#! /bin/bash
arr=()
scanHost()
{
arr+=("$(hostname)")
tmpResult=$(hostname)
}
scanHost
ssh user#remotemach "bash -s --" <<EOF
$(typeset -f scanHost)
tmpResult=$tmpResult
scanHost
EOF
echo "Local array -> ${arr[#]}"
echo "Local echo -> $tmpResult"
The snippet above returns
Local array -> localmach
Local echo -> localmach
But what I need is
Local array -> localmach remotemach
Local echo -> remotemach
In words, I need the value of the remote tmpResult AND the array arr stored locally.
Addendum :
Here I make the command hostname as an example, but in reality I am "scanning" a remote host and generate a JSON string that I store in the tmpResult variable. If I encounter problems during the scan, I append a string explaining the issue in arr. This way in my final json I can list all the problems encountered during the scan of multiple hosts.
You need to collect the result of ssh command :
#! /bin/bash
scanHost()
{
tmpResult=$(hostname)
}
scanHost
tmpResult=$(ssh user#remotemach "bash -s" <<EOF
$(typeset -f scanHost)
scanHost
echo "\$tmpResult"
EOF
)
echo "Local echo -> $tmpResult"
New Version
#! /bin/bash
arr=()
scanHost()
{
arr+=("$(hostname)")
tmpResult=$(hostname)
}
scanHost
eval "$(ssh user#remotemach "bash -s" <<EOF
$(typeset -f scanHost)
$(declare -p arr)
scanHost
declare -p arr
declare -p tmpResult
EOF
)"
echo "Local array -> ${arr[#]}"
echo "Local echo -> $tmpResult"
Use eval with caution due to side effects.

use of ssh variable in the shell script

I want to use the variables of ssh in shell script.
suppose I have some variable a whose value I got inside the ssh and now I want to use that variable outside the ssh in the shell itself, how can I do this ?
ssh my_pc2 <<EOF
<.. do some operations ..>
a=$(ls -lrt | wc -l)
echo \$a
EOF
echo $a
In the above example first echo print 10 inside ssh prints 10 but second echo $a prints nothing.
I would refine the last answer by defining some special syntax for passing the required settings back, e.g. "#SET var=value"
We could put the commands (that we want to run within the ssh session) in a cmdFile file like this:
a=`id`
b=`pwd`
echo "#SET a='$a'"
echo "#SET b='$b'"
And the main script would look like this:
#!/bin/bash
# SSH, run the remote commands, and filter anything they passed back to us
ssh user#host <cmdFile | grep "^#SET " | sed 's/#SET //' >vars.$$
# Source the variable settings that were passed back
. vars.$$
rm -f vars.$$
# Now we have the variables set
echo "a = $a"
echo "b = $b"
If you're doing this for lots of variables, you can add a function to cmdFile, to simplify/encapsulate your special syntax for passing data back:
passvar()
{
var=$1
val=$2
val=${val:-${!var}}
echo "#SET ${var}='${val}'"
}
a=`id`
passvar a
b=`pwd`
passvar b
You might need to play with quotes when the values include whitespace.
A script like this could be used to store all the output from SSH into a variable:
#!/bin/bash
VAR=$(ssh user#host << _EOF
id
_EOF)
echo "VAR=$VAR"
it produces the output:
VAR=uid=1000(user) gid=1000(user) groups=1000(user),4(adm),10(wheel)

How to export an associative array (hash) in bash?

Related, but not a duplicate of: How to define hash tables in Bash?
I can define and use a bash hash, but I am unable to export it, even with the -x flag. For example, the following works to export (and test exportation of) a normal string variable:
aschirma#graphics9-lnx:/$ export animal_cow="moo"
aschirma#graphics9-lnx:/$ bash -c "echo \$animal_cow"
moo
aschirma#graphics9-lnx:/$
However, if I try to export a hash:
aschirma#graphics9-lnx:/$ declare -A -x animals
aschirma#graphics9-lnx:/$ animals[duck]="quack"
aschirma#graphics9-lnx:/$ echo ${animals[duck]}
quack
aschirma#graphics9-lnx:/$ bash -c "echo \${animals[duck]}"
aschirma#graphics9-lnx:/$
It seems the nested bash shell does not have the hash in its scope. I did verify this also by manually entering the nested bash shell and attempting to use the hash interactively.
There isn't really a good way to encode an array variable into the environment. See
http://www.mail-archive.com/bug-bash#gnu.org/msg01774.html (Chet Ramey is the maintainer of bash)
As a workaround for this harsh Bash limitation I'm using "serialize to temporary file" method. You can export plain variables, so you can pass an array (associative) through filepath. Of course, this has limitations, but sometimes works and is good enough.
declare -A MAP # export associative array
MAP[bar]="baz"
declare -x serialized_array=$(mktemp) # create temporary variable
# declare -p can be used to dump the definition
# of a variable as shell code ready to be interpreted
declare -p MAP > "${serialized_array}" # serialize an array in temporary file
# perform cleanup after finishing script
cleanup() {
rm "${serialized_array}"
}
trap cleanup EXIT
# ... in place where you need this variable ...
source "${serialized_array}" # deserialize an array
echo "map: ${MAP[#]}"
This is a bit old but I answer anyway, you could use temp files. If you do it right you can wrapper it to use them like arrays. For example with this function:
var() { # set var or add comtent
case $1 in
*=|*=*)
local __var_part1=$( echo "$1" | sed -e 's/=.*//' -e 's/[+,-]//' ) # cut +=/=
local __var_part2=$( echo "$1" | sed -e 's/.*.=//' )
local __var12=$tmp_dir/$__var_part1
mkdir -p ${__var12%/*} #create all subdirs if its an array
case $1 in
*+=*)
# if its an array try to add new item
if [ -d $tmp_dir/$__var_part1 ] ; then
printf -- $__var_part2 > $tmp_dir/$__var_part1/\ $((
$( echo $tmp_dir/$__var_part2/* \
| tail | basename )\ + 1 ))
else
printf -- "$__var_part2" >> $tmp_dir/$__var_part1
fi
;;
*-=*) false ;;
# else just add content
*) printf -- "$__var_part2" > $tmp_dir/$__var_part1 ;;
esac
;;
*) # just print var
if [ -d $tmp_dir/$1 ] ; then
ls $tmp_dir/$1
elif [ -e $tmp_dir/$1 ] ; then
cat $tmp_dir/$1
else
return 1
fi
;;
esac
}
# you can use mostly like you set vars in bash/shell
var test='Hello Welt!'
# if you need arrays set it like this:
var fruits/0='Apple'
var fruits/1='Banana'
# or if you need a dict:
var contacts/1/name="Max"
var contacts/1/surname="Musterman"
This not the fastest way, but its very flexible, simple and works in nearly all shells.
short answer --> export animals after declaring it
full -->
Try this way as a script:
#!/usr/bin/env bash
declare -A -x animals
export animals
animals[duck]="quack"
echo ${animals[duck]}
bash -c "echo ${animals[duck]}"
Output on my side using Bash version: 5.1.16(1)
quack
quack
or in terminal:
$ declare -A -x animals
$ export animals
$ animals[duck]="quack"
$ echo ${animals[duck]}
quack
$ bash -c "echo ${animals[duck]}"
quack
$

SSH commands via bash script

I've been trying several fails to perform the following:
Basically, what I need is to execute several sequenced commands on a remote unix shell, such as setting environment variables with variables that I have on the script, move to a particular directory and run a script there and so on.
I've tried using a printf with the portion of the script and then piped the ssh command, but it didn't work quite well, also, I've read about the "ssh ... >> END" marker, which is great but since I'm using functions, it doesn't work well.
Do you have any thoughts?
Here's an excerpt of the code:
deployApp() {
inputLine=$1;
APP_SPECIFIC_DEPLOY_SCRIPT="$(echo $inputLine | cut -d ' ' -s -f1)";
BRANCH="$(echo $inputLine | cut -d ' ' -s -f2)";
JBOSS_HOME="$(echo $inputLine | cut -d ' ' -s -f3)";
BASE_PORT="$(echo $inputLine | cut -d ' ' -s -f4)";
JAVA_HOME_FOR_JBOSS="$(echo $inputLine | cut -d ' ' -s -f5)";
JAVA_HEAP="$(echo $inputLine | cut -d ' ' -s -f6)";
echo "DEPLOYING $APP_SPECIFIC_DEPLOY_SCRIPT"
echo "FROM BRANCH $BRANCH"
echo "IN JBOSS $JBOSS_HOME"
echo "WITH BASE PORT $BASE_PORT"
echo "USING $JAVA_HOME_FOR_JBOSS"
if [[ -n "$JAVA_HEAP" ]]; then
echo "WITH $JAVA_HEAP"
fi
echo
echo "Exporting jboss to $JBOSS_HOME"
ssh me#$SERVER <<END
cleanup() {
rm -f $JBOSS_SERVER/log/*.log
rm -Rf $JBOSS_SERVER/deploy/
rm -Rf $JBOSS_SERVER/tmp/
mkdir $JBOSS_SERVER/deploy
}
startJboss() {
cd $JBOSS_SERVER/bin
./jbossctl.sh start
return 0;
}
export JBOSS_HOME
export JBOSS_SERVER=$JBOSS_HOME/server/default
END
return 0;
}
With that "HERE" approach, I'm getting this error: "syntax error: unexpected end of file"
Thanks a lot in advance!
Just put the functions in your here document, too:
var="Hello World"
ssh user#host <<END
x() {
print "x function with args=$*"
}
x "$var"
END
[EDIT] Some comments:
You say "export JBOSS_HOME" but you never define a value for the variable in the here document. You should use export JBOSS_HOME="$JBOSS_HOME". BASH will take all text between the two END, replace all variables, and send the result to SSH for processing.
That also means the other side will see rm -f /path/to/jboss/server/*.log; the assignment to JBOSS_SERVER in the last line of the here document has no effect (at least not to the code in cleanup()).
If you want to pass $ unmodified to the remote server, you have to escape it with \: rm -f \$JBOSS_SERVER/log/*.log
You never call cleanup()
There is a } missing after return 0 to finish the definition of deployapp()
There may be other problems as well. Run the script with bash -x to see what the shell actually executes. You can also add echo commands in the here document to see what the values of the variables are or you can add set -x before cleanup() to get the same output as with bash -x but from the remote side.
I don't understand why you're using cut to split the arguments to your function. Just do
APP_SPECIFIC_DEPLOY_SCRIPT=$1
BRANCH=$2
JBOSS_HOME=$3
# etc.
If you don't quote your here document delimiter, the contents are expanded before they're sent to the server. That may be what you want. If you don't and you want all expansion to be done on the server side, then quote it like this:
ssh me#$SERVER <<'END'
# etc.
END
If you wan't a mixture, don't quote the delimiter, but do escape those things that you want delayed expansion for:
ssh me#$SERVER <<END
echo $EXPAND_ME_NOW \$EXPAND_ME_LATER
END
What are the export statements supposed to do? I can't see that they would have any effect at all.

Resources