Shell script calling ssh: how to interpret wildcard on remote server - shell

I work on a certain customer environment on a daily basis, comprised of 5 AIX servers, and sometimes I need to issue a same command on all 5 of them.
So I set up SSH key-based authentication between the servers, and whipped up a little ksh script that broadcasts the command to all of them:
#!/usr/bin/ksh
if [[ $# -eq 0 ]]; then
print "broadcast.ksh - broadcasts a command to the 5 XXXXXXXX environments and returns output for each"
print "usage: ./broadcast.ksh command_to_issue"
exit
fi
set -A CUST_HOSTS aaa bbb ccc ddd eee
for host in ${CUST_HOSTS[#]}; do
echo "============ $host ================"
if [[ `uname -n` = $host ]]; then
$*
continue
fi
ssh $host $*
done
echo "========================================="
echo "Finished"
Now, this works just fine, until I want to use a wildcard on the remote end, something like:
./broadcast.ksh ls -l java*
since the '*' is expanded on the local system as opposed to the remote.
Now, if using ssh remote commands, I can get around this by using single quotes:
ssh user#host ls -l java* <-- will _not_ work as expected, since asterisk will be interpreted locally
ssh user#host 'ls -l java*' <-- _will_ work as expected, since asterisk will be interpreted on the remote end
Now, I have tried to incorporate that into my script, and have tried to create a $command variable made up of the $* contents surrounded by single quotes, but have drowned in a sea of escaping backslashes and concatenation attempts in ksh, to no avail.
I'm sure there's a simple solution to this, but I'm not finding it so thought I would come out and ask.
Thanks,
James

As you found, passing an asterisk as an argument to your script doesn't work because the shell expands it before the arguments are processed. Try double-quoting $* and either escaping asterisks/semi-colons etc with backslashes in your script call, or single quoting the command.
for host in ${CUST_HOSTS[#]}; do
echo "============ $host ================"
if [[ `uname -n` = $host ]]; then
"$*"
continue
fi
ssh $host "$*"
done
$ ./broadcast.ksh ls -l java\*
$ ./broadcast.ksh 'ls -l java*; ls -l *log'

I wanted to comment but still too low on the totum, but Josh's single quote suggestion should work.
I spun up a couple of vms each with 2 files in /tmp : /tmp/foo1 and /tmp/foo2
then used a variation of your script
root#jdsdrop1:~# cat foo.sh
#!/usr/bin/ksh
if [[ $# -eq 0 ]]; then
print "broadcast.ksh - broadcasts a command to the 5 XXXXXXXX environments and returns output for each"
print "usage: ./broadcast.ksh command_to_issue"
exit
fi
set -A CUST_HOSTS jdsdropfed1 jdsdropfed2-2
for host in ${CUST_HOSTS[#]}; do
echo "============ $host ================"
if [[ `uname -n` = $host ]]; then
$*
continue
fi
ssh $host $*
done
echo "========================================="
echo "Finished"
root#jdsdrop1:~# ./foo.sh 'ls /tmp/foo*'
============ jdsdropfed1 ================
/tmp/foo1
/tmp/foo2
============ jdsdropfed2-2 ================
/tmp/foo1
/tmp/foo2
=========================================
Finished
root#jdsdrop1:~# ssh jdsdropfed1 "ls /tmp/foo*"
/tmp/foo1
/tmp/foo2
root#jdsdrop1:~# ssh jdsdropfed2-2. "ls /tmp/foo*"
/tmp/foo1
/tmp/foo2

Related

How to change name of file if already present on remote machine?

I want to change the name of a file if it is already present on a remote server via SSH.
I tried this from here (SuperUser)
bash
ssh user#localhost -p 2222 'test -f /absolute/path/to/file' && echo 'YES' || echo 'NO'
This works well with a prompt, echoes YES when the file exists and NO when it doesn't. But I want this to be launched from a crontab, then it must be in a script.
Let's assume the file is called data.csv, a condition is set in a loop such as if there already is a data.csv file on the server, the file will be renamed data_1.csv and then data_2.csv, ... until the name is unique.
The renaming part works, but the detection part doesn't :
while [[ $fileIsPresent!='false' ]]
do
((appended+=1))
newFileName=${fileName}_${appended}.csv
remoteFilePathname=${remoteFolder}${newFileName}
ssh pi#localhost -p 2222 'test -f $remoteFilePathname' && fileIsPresent='true' || fileIsPresent='false'
done
always returns fileIsPresent='true' for any data_X.csv. All the paths are absolute.
Do you have any idea to help me?
This works:
$ cat replace.sh
#!/usr/bin/env bash
if [[ "$1" == "" ]]
then
echo "No filename passed."
exit
fi
if [[ ! -e "$1" ]]
then
echo "no such file"
exit
fi
base=${1%%.*} # get basename
ext=${1#*.} # get extension
for i in $(seq 1 100)
do
new="${base}_${i}.${ext}"
if [[ -e "$new" ]]
then
continue
fi
mv $1 $new
exit
done
$ ./replace.sh sample.csv
no such file
$ touch sample.csv
$ ./replace.sh sample.csv
$ ls
replace.sh
sample_1.csv
$ touch sample.csv
$ ./replace.sh sample.csv
$ ls
replace.sh
sample_1.csv
sample_2.csv
However, personally I'd prefer to use a timestamp instead of a number. Note that this sample will run out of names after 100. Timestamps won't. Something like $(date +%Y%m%d_%H%M%S).
As you asked for ideas to help you, I thought it worth mentioning that you probably don't want to start up to 100 ssh processes each one logging into the remote machine, so you might do better with a construct like this that only establishes a single ssh session that runs till complete:
ssh USER#REMOTE <<'EOF'
for ((i=0;i<10;i++)) ; do
echo $i
done
EOF
Alternatively, you can create and test a bash script locally and then run it remotely like this:
ssh USER#REMOTE 'bash -s' < LocallyTestedScript.bash

ssh bash -c exit status does not propagate [duplicate]

This question already has an answer here:
How to have simple and double quotes in a scripted ssh command
(1 answer)
Closed 4 years ago.
According to man ssh and this previous answer, ssh should propagate the exit status of whatever process it ran on the remote server. I seem to have found a mystifying exception!
$ ssh myserver exit 34 ; echo $?
34
Good...
$ ssh myserver 'exit 34' ; echo $?
34
Good...
$ ssh myserver bash -c 'exit 34' ; echo $?
0
What?!?
$ ssh myserver
ubuntu#myserver $ bash -c 'exit 34' ; echo $?
34
So the problem does not appear to be either ssh or bash -c in isolation, but their combination does not behave as I would expect.
I'm designing a script to be run on a remote machine that needs to take an argument list that's computed on the client side. For the sake of argument, let's say it fails if any of the arguments is not a file on the remote server:
ssh myserver bash -c '
for arg ; do
if [[ ! -f "$arg" ]] ; then
exit 1
fi
done
' arg1 arg2 ...
How can I run something like this and effectively inspect its return status? The test above seems to suggest I cannot.
The problem is that the quoting is being lost. ssh simply concatenates the arguments, it doesn't requote them, so the command you're actually executing on the server is:
bash -c exit 34
The -c option only takes one argument, not all the remaining arguments, so it's just executing exit; 34 is being ignored.
You can see a similar effect if you do:
ssh myserver bash -c 'echo foo'
It will just echo a blank line, not foo.
You can fix it by giving a single argument to ssh:
ssh myserver "bash -c 'exit 34'"
or by doubling the quotes:
ssh myserver bash -c "'exit 34'"
Insofar as your question is how to run a command remotely while passing it on ssh's command line without it getting in a mangle that triggers the bug in question, printf '%q ' can be used to ask the shell to perform quoting on your behalf, to build a string which can then be passed to ssh:
printf -v cmd_str '%q ' bash -c '
for arg ; do
if [[ ! -f "$arg" ]] ; then
exit 1
fi
done
' arg1 arg2 ...
ssh "$host" "$cmd_str"
However, this is only guaranteed to work correctly if the default shell for the remote user is also bash (or, if you used ksh's printf %q locally, if the remote shell is ksh). It's much safer to pass your script text out-of-band, as on stdin:
printf -v arg_str '%q ' arg1 arg2 ...
ssh "$host" "bash -s $arg_str" <<'EOF'
for arg; do
if [[ ! -f "$arg" ]]; then
exit 1
fi
done
EOF
...wherein we still depend on printf %q to generate correct output, but only for the arguments, not for the script itself.
Try wrapping in quotes:
╰─➤ ssh server "bash -c 'exit 34' "; echo $?
34

Replacing 'source file' with its content, and expanding variables, in bash

In a script.sh,
source a.sh
source b.sh
CMD1
CMD2
CMD3
how can I replace the source *.sh with their content (without executing the commands)?
I would like to see what the bash interpreter executes after sourcing the files and expanding all variables.
I know I can use set -n -v or run bash -n -v script.sh 2>output.sh, but that would not replace the source commands (and even less if a.sh or b.sh contain variables).
I thought of using a subshell, but that still doesn't expand the source lines. I tried a combination of set +n +v and set -n -v before and after the source lines, but that still does not work.
I'm going to send that output to a remote machine using ssh.
I could use <<output.sh to pipe the content into the ssh command, but I can't log as root onto the remote machine, but I am however a sudoer.
Therefore, I thought I could create the script and send it as a base64-encoded string (using that clever trick )
base64 script | ssh remotehost 'base64 -d | sudo bash'
Is there a solution?
Or do you have a better idea?
You can do something like this:
inline.sh:
#!/usr/bin/env bash
while read line; do
if [[ "$line" =~ (\.|source)\s+.+ ]]; then
file="$(echo $line | cut -d' ' -f2)"
echo "$(cat $file)"
else
echo "$line"
fi
done < "$1"
Note this assumes the sourced files exist, and doesn't handle errors. You should also handle possible hashbangs. If the sourced files contain themselves source, you need to apply the script recursively, e.g. something like (not tested):
while egrep -q '^(source|\.)' main.sh; do
bash inline.sh main.sh > main.sh
done
Let's test it
main.sh:
source a.sh
. b.sh
echo cc
echo "$var_a $var_b"
a.sh:
echo aa
var_a="stack"
b.sh:
echo bb
var_b="overflow"
Result:
bash inline.sh main.sh
echo aa
var_a="stack"
echo bb
var_b="overflow"
echo cc
echo "$var_a $var_b"
bash inline.sh main.sh | bash
aa
bb
cc
stack overflow
BTW, if you just want to see what bash executes, you can run
bash -x [script]
or remotely
ssh user#host -t "bash -x [script]"

Pseudo-terminal will not be allocated because stdin is not a terminal ssh bash

okay heres part of my code when I ssh to my servers from my server.txt list.
while read server <&3; do #read server names into the while loop
serverName=$(uname -n)
if [[ ! $server =~ [^[:space:]] ]] ; then #empty line exception
continue
fi
echo server on list = "$server"
echo server signed on = "$serverName"
if [ $serverName == $server ] ; then #makes sure a server doesnt try to ssh to itself
continue
fi
echo "Connecting to - $server"
ssh "$server" #SSH login
echo Connected to "$serverName"
exec < filelist.txt
while read updatedfile oldfile; do
# echo updatedfile = $updatedfile #use for troubleshooting
# echo oldfile = $oldfile #use for troubleshooting
if [[ ! $updatedfile =~ [^[:space:]] ]] ; then #empty line exception
continue # empty line exception
fi
if [[ ! $oldfile =~ [^[:space:]] ]] ; then #empty line exception
continue # empty line exception
fi
echo Comparing $updatedfile with $oldfile
if diff "$updatedfile" "$oldfile" >/dev/null ; then
echo The files compared are the same. No changes were made.
else
echo The files compared are different.
cp -f -v $oldfile /infanass/dev/admin/backup/`uname -n`_${oldfile##*/}_$(date +%F-%T)
cp -f -v $updatedfile $oldfile
fi
done
done 3</infanass/dev/admin/servers.txt
I keep on getting this error and the ssh doesn't actually connect and perform the code on the server its suppose to be ssh'd on.
Pseudo-terminal will not be allocated because stdin is not a terminal
I feel like everything the guy above just said is so wrong.
Expect?
It's simple:
ssh -i ~/.ssh/bobskey bob#10.10.10.10 << EOF
echo I am creating a file called Apples in the /tmp folder
touch /tmp/apples
exit
EOF
Everything in between the 2 "EOF"s will be run in the remote server.
The tags need to be the same. If you decide to replace "EOF" with "WayneGretzky", you must change the 2nd EOF also.
You seem to assume that when you run ssh to connect to a server, the rest of the commands in the file are passed to the remote shell running in ssh. They are not; instead they will be processed by the local shell once ssh terminates and returns control to it.
To run remote commands through ssh there are a couple of things you can do:
Write the commands you want to execute to a file. Copy the file to the remote server using scp, and execute it with ssh user#remote command
Learn a bit of TCL and use expect
Write the commands in a heredoc, but be careful with variable substitution: substitution happens in the client, not on the server. For example this will output your local home directory, not the remote:
ssh remote <<EOF
echo $HOME
EOF
To make it print the remote home directory you have to use echo \$HOME.
Also, remember that data files such as filelist.txt have to be explicitly copied if you want to read them on the remote side.

how to create the option for printing out statements vs executing them in a shell script

I'm looking for a way to create a switch for this bash script so that I have the option of either printing (echo) it to stdout or executing the command for debugging purposes. As you can see below, I am just doing this manually by commenting out one statement over the other to achieve this.
Code:
#!/usr/local/bin/bash
if [ $# != 2 ]; then
echo "Usage: testcurl.sh <localfile> <projectname>" >&2
echo "sample:testcurl.sh /share1/data/20110818.dat projectZ" >&2
exit 1
fi
echo /usr/bin/curl -c $PROXY --certkey $CERT --header "Test:'${AUTH}'" -T $localfile $fsProxyURL
#/usr/bin/curl -c $PROXY --certkey $CERT --header "Test:'${AUTH}'" -T $localfile $fsProxyURL
I'm simply looking for an elegant/better way to create like a switch from the command line. Print or execute.
One possible trick, though it will only work for simple commands (e.g., no pipes or redirection (a)) is to use a prefix variable like:
pax> cat qq.sh
${PAXPREFIX} ls /tmp
${PAXPREFIX} printf "%05d\n" 72
${PAXPREFIX} echo 3
What this will do is to insert you specific variable (PAXPREFIX in this case) before the commands. If the variable is empty, it will not affect the command, as follows:
pax> ./qq.sh
my_porn.gz copy_of_the_internet.gz
00072
3
However, if it's set to echo, it will prefix each line with that echo string.
pax> PAXPREFIX=echo ./qq.sh
ls /tmp
printf %05d\n 72
echo 3
(a) The reason why it will only work for simple commands can be seen if you have something like:
${PAXPREFIX} ls -1 | tr '[a-z]' '[A-Z]'
When PAXPREFIX is empty, it will simply give you the list of your filenames in uppercase. When it's set to echo, it will result in:
echo ls -1 | tr '[a-z]' '[A-Z]'
giving:
LS -1
(not quite what you'd expect).
In fact, you can see a problem with even the simple case above, where %05d\n is no longer surrounded by quotes.
If you want a more robust solution, I'd opt for:
if [[ ${PAXDEBUG:-0} -eq 1 ]] ; then
echo /usr/bin/curl -c $PROXY --certkey $CERT --header ...
else
/usr/bin/curl -c $PROXY --certkey $CERT --header ...
fi
and use PAXDEBUG=1 myscript.sh to run it in debug mode. This is similar to what you have now but with the advantage that you don't need to edit the file to switch between normal and debug modes.
For debugging output from the shell itself, you can run it with bash -x or put set -x in your script to turn it on at a specific point (and, of course, turn it off with set +x).
#!/usr/local/bin/bash
if [[ "$1" == "--dryrun" ]]; then
echoquoted() {
printf "%q " "$#"
echo
}
maybeecho=echoquoted
shift
else
maybeecho=""
fi
if [ $# != 2 ]; then
echo "Usage: testcurl.sh <localfile> <projectname>" >&2
echo "sample:testcurl.sh /share1/data/20110818.dat projectZ" >&2
exit 1
fi
$maybeecho /usr/bin/curl "$1" -o "$2"
Try something like this:
show=echo
$show /usr/bin/curl ...
Then set/unset $show accordingly.
This does not directly answer your specific question, but I guess you're trying to see what command gets executed for debugging. If you replace #!/usr/local/bin/bash with #!/usr/local/bin/bash -x bash will run and echo the commands in your script.
I do not know of a way for "print vs execute" but I know of a way for "print and execute", and it is using "bash -x". See this link for example.

Resources