SSH Command in bash - bash

Need some help on these commands that we're not working during the ssh session but works fine when manually run on an ubuntu machine.
for w in $(echo "/var/asd/dsa/123" | tr "/" " ") ; do echo $w >&2; echo "123"; done
somehow the code above does not work, it expected to have an output like this:
var
123
asd
123
dsa
123
123
123
instead I get like this:
123
123
123
123
<blank space>
<blank space>
<blank space>
<blank space>
Here is the full code:
ssh -o "StrictHostKeyChecking no" -i $PEM_PATH $SERVER_USER#$SERVER_IP << EOF
for w in $(echo "/var/asd/dsa/123" | tr "/" " ") ; do echo $w >&2; echo "123"; done
EOF
Any help are really appreciated, thanks!

The problem is that the local shell is interpolating your $ expressions prior to sending them to the ssh session. You can prevent this by single quoting your heredoc tag: 'EOF'.
ssh -o "StrictHostKeyChecking no" -i $PEM_PATH $SERVER_USER#$SERVER_IP <<'EOF'
for w in $(echo "/var/asd/dsa/123" | tr "/" " ") ; do echo $w >&2; echo "123"; done
EOF
With the above code, $(echo "/var/asd/dsa/123" | tr "/" " ") ; is sent to the server instead of being replaced locally.

There are several issues with your approach:
Your here doc gets interpolated locally. The actual command you are running is: for w in var asd dsa 123 ; do echo >&2; echo "123"; done
You can see that by using set -x to enable tracing locally, and run ssh with cat instead of the login shell:
ssh pi cat << EOF
for w in $(echo "/var/asd/dsa/123" | tr "/" " ") ; do echo $w >&2; echo "123"; done
EOF
+ ssh pi cat
++ echo /var/asd/dsa/123
++ tr / ' '
for w in var asd dsa 123 ; do echo >&2; echo "123"; done
As you can see echo /var/asd/dsa/123 and tr / ' ' are executed locally, and $w is interpolated to the empty string.
You probably want to use single quotes around 'EOF':
ssh pi cat << 'EOF'
for w in $(echo "/var/asd/dsa/123" | tr "/" " ") ; do echo $w >&2; echo "123"; done
EOF
+ ssh pi cat
for w in $(echo "/var/asd/dsa/123" | tr "/" " ") ; do echo $w >&2; echo "123"; done
Now, the command gets there correctly.
Note: if you wish that $(echo "/var/asd/dsa/123" | tr "/" " ") is run locally, then you at least need to escape $w.
ssh pi << EOF
for w in $(echo "/var/asd/dsa/123" | tr "/" " ") ; do echo \$w >&2; echo "123"; done
EOF
Buffering will mess up the order. That's why your empty lines (the ones printed by echo >&2) are at the end of your output. You can use tee to "force" line buffering:
ssh -T pi << 'EOF'
for w in $(echo "/var/asd/dsa/123" | tr "/" " ") ; do echo $w >&2; echo "123"; done
EOF
+ ssh -T pi
Linux mserv 3.10.25+ #622 PREEMPT Fri Jan 3 18:41:00 GMT 2014 armv6l
...
123
123
123
123
var
asd
dsa
123
Versus:
ssh pi << 'EOF'
for w in $(echo "/var/asd/dsa/123" | tr "/" " ") ; do echo $w >&2; echo "123"; done 2>&1 | tee
EOF
+ ssh pi
Linux mserv 3.10.25+ #622 PREEMPT Fri Jan 3 18:41:00 GMT 2014 armv6l
...
var
123
asd
123
dsa
123
123
123
(in the first case STDIN is printed first, then STDERR, in the second, lines are alternated)
Piping commands to ssh and forcing a login shell
If you run commands by piping them to stdin of ssh, it will run a login shell and print the MOTD (depending on the server configuration) and run a login shell (which has several side effects).
In my example, Linux mserv 3.10.25+ #622 PREEMPT Fri Jan 3 18:41:00 GMT 2014 armv6l is part of the MOTD. However, there is no need for piping, ssh will run last argument via the user shell, without the need for piping it to STDIN. This might be limited to MAX_ARG_STRLEN (which is ~128K), but if you have a script that size, you probably should scp-it to the host first.
$ ssh pi '
for w in $(echo "/var/asd/dsa/123" | tr "/" " ")
do
echo $w >&2
echo "123"
done 2>&1 | tee
'
var
123
asd
123
dsa
123
123
123
Note: This might be limited to MAX_ARG_STRLEN (which is ~128K), but if you have a script that size, you probably should scp-it to the host first.
Note: pi is a test server I use, replace it with your appropriate connection info.

Related

bash: filling user read input automatically with param list

I'm trying to use Wireguard road-warrior script from here https://github.com/Nyr/wireguard-install
But i cant make it run in noninteractive mode with predefined parameters. I've read some other similar topics here about how to provide answers to bash "read", but suggestions from there doesn't work.
I've tried this:
# bash wireguard-install.sh < params
# printf '%s\n' 51822 clientcustom 2 y | bash wireguard-install.sh
# echo "51822 clientcustom 2 y" | bash wireguard-install.sh
in every case installer just uses default values. What am i doing wrong?
https://github.com/Nyr/wireguard-install/blob/master/wireguard-install.sh#L15
Indeed, this is a bit problematic. Typically, not caring, you would just:
( sleep 1; printf '%s\n' 51822 clientcustom 2 y ) | ...
A real robust solution, you would parse the output of the process to know when to write response, either with expect or Bash or something better.
coproc bash wireguard-install.sh
while IFS= read -r -u "${COPROC[0]}" line; do
case "$line" in
"IPv4 address [1]:"*) echo something >&"${COPROC[1]}"; ;;
"other prompt"*) echo other stuff >&"${COPROC[1]}"; ;;
"etc...") echo ... ;;
esac
done
Some alternatives:
1. you could simply drop problematic line while running.
bash <(sed /read\ -N\ 999999/d wireguard-install.sh) < <(
printf %s\\n 51822 clientcustom 2 y)
This will drop line used to
# Discard stdin. Needed when running from an one-liner which includes a newline
read -N 999999 -t 0.001
2. Instead of passing variables, you could edit script:
$ sed 's/^[ \o11]*read -p/ /p;d' wireguard-install.sh | uniq
"DNS server [1]: " dns
"IPv4 address [1]: " ip_number
"Public IPv4 address / hostname [$get_public_ip]: " public_ip
"Public IPv4 address / hostname: " public_ip
"IPv6 address [1]: " ip6_number
"Port [51820]: " port
"Name [client]: " unsanitized_client
"Should automatic updates be enabled for it? [Y/n]: " boringtun_updates
"Option: " option
"Name: " unsanitized_client
"Client: " client_number
"Confirm $client removal? [y/N]: " remove
"Confirm WireGuard removal? [y/N]: " remove
So you could prepare a sed string:
printf -v sedscr 's/read -p.* \(%s\) *$/\\1="%s"/;' port 51822 \
unsanitized_client clientcustom client_number 2 remove y
Then ensure all's ok:
sed -e "/read -N 99999/d;$sedscr" <wireguard-install.sh |
diff -u wireguard-install.sh -
you will see some lines like
# Discard stdin. Needed when running from an one-liner which includes a newline
-read -N 999999 -t 0.001
# Detect OpenVZ 6
if [[ $(uname -r | cut -d "." -f 1) -eq 2 ]]; then
## -235,15 +234,15 ##
fi
echo
echo "What port should WireGuard listen to?"
- read -p "Port [51820]: " port
+ port="51822"
until [[ -z "$port" || "$port" =~ ^[0-9]+$ && "$port" -le 65535 ]]; do
And finally
bash <(
printf -v sedscr 's/read -p.* \(%s\) *$/\\1="%s"/;' port 51822 \
unsanitized_client clientcustom client_number 2 remove y
sed -e "/read -N 99999/d;$sedscr" <wireguard-install.sh
)

using tail to follow a log and execute a command instantly? Only seems to work by starting a new line

I am trying to figure out a command which will enable me to read a log file in real time and execute a command when the string matches? I am using logkeys and trying to make it when I type a word it immediately triggers a command. This script works, but only when I press enter (start a newline) does it execute, and it seems anything I have found online also requires the press of the enter key to work. Is there a way to get around this somehow?
#/bin/bash
echo Waiting...
string='test'
tail /path/to/logfile -n0 -f | while read line; do
if [[ $line =~ $string ]]; then
echo "hello"
fi
done
I've played with buffering settings to no avail, so my conclusion is that read waits for a newline before it finishes. If you instead did read -n1, read would read exactly one character, which isn't quite what we want either, because then $line would always be just that one char.
Unfortunately, grep appears to have the same behavior (even with buffering options changed), even with grep -o:
$ tail logfile -f -n0 | grep -o test &
[1] 25524
$ echo -n test >> logfile
$ echo -n test >> logfile
$ echo test >> logfile
test
test
test
I think the general solution would be to roll our own "ring buffer grep" search tool that reads character per character into a ring buffer.
Here's my perl version of that, hope it helps. (Save as: ringgrep.pl)
#!/usr/bin/perl -w
use strict;
if (!$ARGV[0]) {
print "Usage: $0 needle\n";
exit 1;
}
my $needle = $ARGV[0];
my $buffer_len = length($needle);
my #buffer = (0) x $buffer_len;
my $i = 0;
my $input;
while(sysread(STDIN, $input, 1)) {
$buffer[$i] = $input;
my $string = join("", #buffer);
$string = (($i+1)%$buffer_len == 0 ? "" : substr($string, $i-$buffer_len+1)) . substr($string, 0, $i+1);
# print "string is: $string\n";
if ($string =~ /$needle/) {
print "got test!\n";
#buffer = (0) x $buffer_len;
$i = 0
} else {
$i = (++$i) % $buffer_len
}
}
Usage:
$ chmod +x ringgrep.pl
$ tail logfile -n0 -f | ./ringgrep.pl "this is a test" &
[1] 25756
$ echo -n "test" >> logfile
$ echo -n "test" >> logfile
$ echo -n "test" >> logfile
$ echo -n "test" >> logfile
$ echo -n "this is a test" >> logfile
got test!
$ echo -n "this is a test" >> logfile
got test!
$ echo -n "this is a test" >> logfile
got test!
$ (echo -n t; echo -n h; echo -n i; echo -n s; echo -n ' '; echo -n i; echo -n s; echo -n ' '; echo -n a; echo -n ' '; echo -n t; echo -n e; echo -n s; echo -n t) >> logfile
got test!

Shell script for searching different on multiple servers

I want to search different packages in multiple servers in Solaris 10. One file contains the package information, another file contains the server information.
I have tried this:
bash-3.00# cat pk
"VRTSvcs|VRTSvxfen"
"SUNWfmd|SUNWfsmgtr"
bash-3.00# cat ser
mokshi
niki
This is my script:
bash-3.00# cat tt
#!/usr/bin/bash
>output
for j in `cat ser`
do
for ip in `cat pk`
do
M=`ssh $j "pkginfo |egrep $ip |cut -d \" \" -f7 "`;
echo "$j " >>output
echo "$M" >>output
done
done
The expected output is
cat output
bash-3.00# cat output
moksha
VRTSvcs
VRTSvcsag
VRTSvcsea
VRTSvxfen
niki
SUNWfmd
SUNWfmdr
SUNWfsmgtr
but when I run the script it is running twice like this:
bash-3.00# bash -x tt
++ cat ser
+ for j in '`cat ser`
'
++ cat pk
+ for ip in '`cat pk`'
++ ssh mokshi 'pkginfo |egrep "VRTSvcs|VRTSvxfen" |cut -d " " -f7 '
Password:
+ M='VRTSvcs
VRTSvcsag
VRTSvcsea
VRTSvxfen'
+ echo 'mokshi '
+ echo 'VRTSvcs
VRTSvcsag
VRTSvcsea
VRTSvxfen'
+ for ip in '`cat pk`'
++ ssh mokshi 'pkginfo |egrep "SUNWfmd|SUNWfsmgtr" |cut -d " " -f7 '
Password:
+ M='SUNWfmd
SUNWfmdr
SUNWfsmgtr'
+ echo 'mokshi '
+ echo 'SUNWfmd
SUNWfmdr
SUNWfsmgtr'
+ for j in '`cat ser`'
++ cat pk
+ for ip in '`cat pk`'
++ ssh niki 'pkginfo |egrep "VRTSvcs|VRTSvxfen" |cut -d " " -f7 '
Password:
+ M='VRTSvcs
VRTSvcsag
VRTSvcsea
VRTSvxfen'
+ echo 'niki '
+ echo 'VRTSvcs
VRTSvcsag
VRTSvcsea
VRTSvxfen'
+ for ip in '`cat pk`'
++ ssh niki 'pkginfo |egrep "SUNWfmd|SUNWfsmgtr" |cut -d " " -f7 '
Password:
+ M='SUNWfmd
SUNWfmdr
SUNWfsmgtr'
+ echo 'niki '
+ echo 'SUNWfmd
SUNWfmdr
SUNWfsmgtr'
And I am getting output like this:
bash-3.00# cat output
moksha
VRTSvcs
VRTSvcsag
VRTSvcsea
VRTSvxfen
moksha
SUNWfmd
SUNWfmdr
SUNWfsmgtr
niki
VRTSvcs
VRTSvcsag
VRTSvcsea
VRTSvxfen
niki
SUNWfmd
SUNWfmdr
SUNWfsmgtr
The main goal of the script is that its take one server from server file and it has to search first line in package file. The it has to take second server name from server name and it searches for second line in package file.
please help me where I made a mistake.
You could do this - assuming you have an equal number of lines in your server and package files:
#!/usr/bin/env bash
while read -u 3 -r server && read -u 4 -r pkg; do
m=$(ssh -n "$server" "pkginfo | egrep '$pkg' | cut -d' ' -f7")
echo "$server"
echo "$m"
done 3<ser 4<pk >> output
3<ser connects fd 3 to file ser and 4<pk connects fd 4 to file pk
read -u 3 reads from file descriptor 3, read -u 4 reads from file descriptor 4
$() is better than back ticks (see the post below to find why)
cut -d' ' -> guess your delimiter is a space
ssh -n is needed so that ssh ignored stdin and doesn't interfere with read
it is better to put >> output at the end for more efficient I/O
See also:
Why you don't read lines with "for"
'read -r' doesn't read beyond first line in a loop that does ssh
What's the difference between $(command) and `command` in shell programming?

Letter Index script in bash

I am trying to write a script in bash to achieve a requirement described in the following example:
Eg: User inputs a sentence : "this is a book"
The Output must be "20 8 9 19 9 19 1 2 15 15 11" which is based on the alphabetical sequence.
Following is the code to achieve the following:
#!/bin/bash
echo "enter a sentence"
read sent
alpha=(" " {a..z})
for i in $(seq 27)
do
[ $i -eq 27 ] && break
eval ${alpha[$i]}=$i
done
eval echo $sent | tr -d [:punct:][:space:] | tr [:upper:] [:lower:] | fold -w1 | paste -sd " "| sed -e 's/\</\$/g'
Could someone guide me as to where i am going wrong or if there is a better way to approach the problem.
I know we should avoid doing full programs, but this was not a bad exercise:
#!/bin/bash
read -p "enter a sentence: "
sentence="$(echo $REPLY | tr '[A-Z]' '[a-z]' | tr -dc [:alpha:])"
for (( i=0; i<${#sentence}; i++ ));
do echo -n "$(($(printf "%d " "'${sentence:$i:1}") - 96 )) "
done
echo

unix (cygwin) fifo buffering

Looking for an intercepting proxy made with netcat I found this script:
#!/bin/sh -e
if [ $# != 3 ]
then
echo "usage: $0 <src-port> <dst-host> <dst-port>"
exit 0
fi
TMP=`mktemp -d`
BACK=$TMP/pipe.back
SENT=$TMP/pipe.sent
RCVD=$TMP/pipe.rcvd
trap 'rm -rf "$TMP"' EXIT
mkfifo -m 0600 "$BACK" "$SENT" "$RCVD"
sed 's/^/ => /' <"$SENT" &
sed 's/^/<= /' <"$RCVD" &
nc -l -p "$1" <"$BACK" | tee "$SENT" | nc "$2" "$3" | tee "$RCVD" >"$BACK"
Which work nicely, as expected.
Since I need to look closely to the encoding used, hence the actual bytes passing, I tried to change some lines to use hexdump -vC:
#!/bin/sh -e
if [ $# != 3 ]
then
echo "usage: $0 <src-port> <dst-host> <dst-port>"
exit 0
fi
TMP=`mktemp -d`
BACK=$TMP/pipe.back
SENT=$TMP/pipe.sent
RCVD=$TMP/pipe.rcvd
trap 'rm -rf "$TMP"' EXIT
mkfifo -m 0600 "$BACK" "$SENT" "$RCVD"
( hexdump -vC | sed 's/^/ => /' ) <"$SENT" &
( hexdump -vC | sed 's/^/<= /' ) <"$RCVD" &
nc -l -p "$1" <"$BACK" | tee "$SENT" | nc "$2" "$3" | tee "$RCVD" >"$BACK"
Now it's not working anymore. Actually, I've lost the "realtime" feature of the previous script. Every byte sent is dumped in a single batch; then every byte received in another batch; and this all only after the connection is closed.
I'm suspecting some sort of buffering occurs in the pipe (|), but I'm not sure how to:
test this hypotesis;
fix the script to make it work in realtime again.
PS1. I'm using cygwin.
PS2. sh --version outputs:
GNU bash, version 4.1.10(4)-release (i686-pc-cygwin)
Edit:
Removind the | sed ... part (that is, leaving only hexdump -vC <"$SENT" and hexdump -vC <"$RCVD") the realtime feature is back, increasing my suspicion over the pipeline operator. But the output turns out to be confusing since sent and received bytes are mixed.
Still I couldn't manage to resolve the buffering (?) issue, but I could change the hexdump invocation to render the sed unnecessary:
#!/bin/sh -e
if [ $# != 3 ]
then
echo "usage: $0 <src-port> <dst-host> <dst-port>"
exit 0
fi
TMP=`mktemp -d`
BACK=$TMP/pipe.back
SENT=$TMP/pipe.sent
RCVD=$TMP/pipe.rcvd
trap 'rm -rf "$TMP"' EXIT
mkfifo -m 0600 "$BACK" "$SENT" "$RCVD"
hexdump -v -e '" => %08.8_Ax\n"' -e '" => %08.8_ax " 8/1 "%02x " " " 8/1 "%02x "' -e '" |" 16/1 "%_p" "|\n"' <"$SENT" &
hexdump -v -e '"<= %08.8_Ax\n"' -e '"<= %08.8_ax " 8/1 "%02x " " " 8/1 "%02x "' -e '" |" 16/1 "%_p" "|\n"' <"$RCVD" &
nc -l "$1" <"$BACK" | tee "$SENT" | nc "$2" "$3" | tee "$RCVD" >"$BACK"
Yes, the new hexdump looks ugly, but works.
This question for me is now open just for the sake of curiosity. I'm still willing to give the "correct answer" points to the one who explains (and fixes) the buffering (?) behavior.

Resources