Delete first line/chars on Windows Pipe - windows

I'm trying to capture a live tcpdump from a custom designed linux system. The command I'm using so far is:
plink.exe -ssh user#IP -pw PW "shell nstcpdump.sh -s0 -U -w - not port 22 and not host 127.0.0.1" | "C:\Program Files\Wireshark\wireshark" -i -
This will fail since when executing the command on the remote system, this (custom) shell will output "Done" before sending data. I tried to find out a way to remove the "Done" message from the shell but doesn't appear to be any.
So I came up with this (added findstr -V):
plink.exe -ssh user#IP -pw PW "shell nstcpdump.sh -s0 -U -w - not port 22 and not host 127.0.0.1" | findstr -V "Done" | "C:\Program Files\Wireshark\wireshark" -i -
This works more or less fine since I will get some errors and the live capture will stop. I believe it might have something to do with the buffers, but I'm not sure.
Does anyone know of any other method of removing/bypassing the first bytes/chars of the output from plink/remote shell?
[edit]
as asked, nstcpdump.sh is a wrapper for tcpdump. as mentioned before, this system is highly customized. nstcpdump.sh code:
root#hostname# cat /netscaler/nstcpdump.sh
#!/bin/sh
# piping the packet trace to the tcpdump
#
# FILE: $Id: //depot/main/rs_120_56_14_RTM/usr.src/netscaler/scripts/nstcpdump.sh#1 $
# LAST CHECKIN: $Author: build $
# $DateTime: 2017/11/30 02:14:38 $
#
#
# Options: any TCPDUMP options
#
TCPDUMP_PIPE=/var/tmp/tcpdump_pipe
NETSCALER=${NETSCALER:-`cat /var/run/.NETSCALER`}
[ -r ${NETSCALER}/netscaler.conf ] && . ${NETSCALER}/netscaler.conf
TIME=${TIME:-3600}
MODE=${MODE:-6}
STARTCMD="start nstrace -size 0 -traceformat PCAP -merge ONTHEFLY -filetype PIPE -skipLocalSSH ENABLED"
STOPCMD="stop nstrace "
SHOWCMD="show nstrace "
NSCLI_FILE_EXEC=/netscaler/nscli
NSTRACE_OUT_FILE=/tmp/nstrace.out
NS_STARTTRACE_PIDFILE=/tmp/nstcpdump.pid
TRACESTATE=$(nsapimgr -d allvariables | grep tracestate | awk '{ print $2}')
trap nstcpdump_exit 1 2 15
nstcpdump_init()
{
echo "##### WARNING #####"
echo "This command has been deprecated."
echo "Please use 'start nstrace...' command from CLI to capture nstrace."
echo "trace will now start with all default options"
echo "###################"
if [ ! -d $NSTRACE_DIR ]
then
echo "$NSTRACE_DIR directory doesn't exist."
echo "Possible reason: partition is not mounted."
echo "Check partitions using mount program and try again."
exit 1
fi
if [ ! -x $NSCLI_FILE_EXEC ]
then
echo "$NSCLI_FILE_EXEC binary doesn't exist"
exit 1
fi
if [ -e $NSTRACE_OUT_FILE ]
then
rm $NSTRACE_OUT_FILE
echo "" >> $NSTRACE_OUT_FILE
fi
}
nstcpdump_start_petrace()
{
sleep 0.5;
$NSCLI_FILE_EXEC -U %%:.:. $STARTCMD >/tmp/nstcpdump.sh.out
rm -f ${NS_STARTTRACE_PIDFILE}
}
nstcpdump_start()
{
# exit if trace is already running
if [ $TRACESTATE -ne 0 ]
then
echo "Error: one instance of nstrace is already running"
exit 2
fi
nstcpdump_start_petrace &
echo $! > ${NS_STARTTRACE_PIDFILE}
tcpdump -n -r - $TCPDUMPOPTIONS < ${TCPDUMP_PIPE}
nstcpdump_exit
exit 1
}
nstcpdump_exit()
{
if [ -f ${NS_STARTTRACE_PIDFILE} ]
then
kill `cat ${NS_STARTTRACE_PIDFILE}`
rm ${NS_STARTTRACE_PIDFILE}
fi
$NSCLI_FILE_EXEC -U %%:.:. $STOPCMD >> /dev/null
exit 1
}
nstcpdump_usage()
{
echo `basename $0`: utility to view/save/sniff LIVE packet capture on NETSCALER box
tcpdump -h
echo
echo NOTE: tcpdump options -i, -r and -F are NOT SUPPORTED by this utility
exit 0
}
########################################################################
while [ $# -gt 0 ]
do
case "$1" in
-h )
nstcpdump_usage
;;
-i )
nstcpdump_usage
;;
-r )
nstcpdump_usage
;;
-F )
nstcpdump_usage
;;
esac
break;
done
TCPDUMPOPTIONS="$#"
check_ns nstcpdump
#nstcpdump_init
#set -e
if [ ! -e ${TCPDUMP_PIPE} ]
then
mkfifo $TCPDUMP_PIPE
if [ $? -ne 0 ]
then
echo "Failed creating pipe [$TCPDUMP_PIPE]"
exit 1;
fi
fi
nstcpdump_start
Regards

Related

using cronic to reduce email notifications in a wrapper script for successful backups

I've been using cronic to silence emails from cron jobs when the job is successful. I'm trying to customize it so when a response code is 0 and the error output matches a string of "mount: /VessRAID/RH: /dev/sde1 already mounted on /VessRAID/RH.", to not send an email. Below is the script, then the contents of the email then my attempt at trying to suppress the email which is not working. Any idea what I may be doing wrong?
#!/bin/bash
# Cronic v3 - cron job report wrapper
# Copyright 2007-2016 Chuck Houpt. No rights reserved, whatsoever.
# Public Domain CC0: http://creativecommons.org/publicdomain/zero/1.0/
set -eu
TMP=$(mktemp -d)
OUT=$TMP/cronic.out
ERR=$TMP/cronic.err
TRACE=$TMP/cronic.trace
set +e
"$#" >$OUT 2>$TRACE
RESULT=$?
set -e
PATTERN="^${PS4:0:1}\\+${PS4:1}"
if grep -aq "$PATTERN" $TRACE
then
! grep -av "$PATTERN" $TRACE > $ERR
else
ERR=$TRACE
fi
if [ $RESULT -ne 0 -o -s "$ERR" ]
then
echo "Cronic detected failure or error output for the command:"
echo "$#"
echo
echo "RESULT CODE: $RESULT"
echo
echo "ERROR OUTPUT:"
cat "$ERR"
echo
echo "STANDARD OUTPUT:"
cat "$OUT"
if [ $TRACE != $ERR ]
then
echo
echo "TRACE-ERROR OUTPUT:"
cat "$TRACE"
fi
fi
rm -rf "$TMP"
Here is what the email notification looks like:
Cronic detected failure or error output for the command:
/usr/local/sbin/reg-backup-cronic.sh daily
RESULT CODE: 0
ERROR OUTPUT:
mount: /VessRAID/RH: /dev/sde1 already mounted on /VessRAID/RH.
STANDARD OUTPUT:
/dev/sde1 on /VessRAID/RH type ext4 (rw,relatime)
Here is my attempt at a wrapper script:
#!/bin/bash
/usr/local/sbin/reg-backup.sh $1
CODE=$?
err=$TRACE
if [[ $CODE -eq 0 && $err = "mount: /VessRAID/RH: /dev/sde1 already mounted on /VessRAID/RH." ]]
then
exit $CODE
fi
Alas the emails continue.
Hat tip to the creator of cronic, Chuck Houpt, for cluing me in to an answer, which was to look at the original script and why the error is happening. Case-sensitivity got the best of me:
if mount | grep Vessraid; then
echo starting $1 backup >> /var/log/vessraid.log
Notice the case in VessRAID should have been:
if mount | grep VessRAID; then
echo starting $1 backup >> /var/log/vessraid.log
Now emails only happen when there really is an error.

how to run multiple commands on a remote linux server using bash script

I am currently writing the following script that logs into a remote server and runs couple of commands to verify the performance of the server and prints a message based on the output of those commands .But the ssh doesn't work and returns the stats of the server that hosts the script instead .
Script
#!/bin/bash
#######################
#Function to add hosts to the array
#the following function takes the ip addresses provided while the script is run and stores them in an array
#######################
Host_storing_func () {
HOST_array=()
for i in $# ;do
HOST_array+=(${i});
done
#echo ${HOST_array[*]}
}
#######################
#Calling above function
#######################
Host_storing_func "$#"
############################################################
#Collect Stats of Ping,memory,iowait time test function
############################################################
b=`expr ${#HOST_array[*]} - 1 `
for i in `seq 0 $b` ;do
sshpass -f /root/scripts/passwordFile.txt /usr/bin/ssh student35#${HOST_array[${i}]} << HERE
echo `hostname`
iowaittm=`sar 2 2|awk '/^Average/{print $5};'`
if [ $iowaittm > 10 ];then
echo "IO ==> BAD"
else
echo "IO ==> GOOD"
fi
memoryy=`free -m |grep Swap|awk '{if($2 == 0) print 0;else print (($4 / $2 ) * 100)}'`
if [ ${memoryy} < '10' ] ;then
echo "memory ==> good"
elif [[ "${memory}" -ge 0 ]] && [[ "${memory}" -le 10 ]];then
echo "No Swap"
else
echo "memory ==> bad"`enter code here`
fi
ping -w2 -c2 `hostname` | grep "packet loss"|awk -F, '{print $3}'|awk -F% '{print $1}'|sed 's/^ *//'|awk '{if ($1 == 0) print "Yes" ;else print "No"}'
HERE
done
Output : oc5610517603.XXX.com is the name of the source server
[root#oc5610517603 scripts]# ./big_exercise.sh 9.XXX.XXX.XXX 9.XXX.XXX.XXX
Pseudo-terminal will not be allocated because stdin is not a terminal.
oc5610517603.XXX.com
IO ==> GOOD
No Swap
ping: oc5610517603.ibm.com: Name or service not known
Pseudo-terminal will not be allocated because stdin is not a terminal.
oc5610517603.XXX.com
IO ==> GOOD
No Swap
ping: oc5610517603.XXX.com: Name or service not known
thanks for checking the script , I figured out a way to solve the problem
It is the sshpass command that is causing issue , you just have to put the opening HERE in single quotes if you want to use variables with in the HEREdoc but if the variables are calculated before ssh then you don't have to put opening HERE in single quotes
sshpass -f /root/scripts/passwordFile.txt /usr/bin/ssh -T student35#${i} << 'HERE'
after I changed the sshpass command as above my script worked
I have modified your script a bit.
As suggested by #chepner, I am not using the Host_storing_func.
Heredocs for sshpaas are somewhat tricky. You have to escape every back-tick and $ sign in the heredoc.
Notice the - before the heredoc start, it allows you to indent the heredoc body. Also, try to avoid back-ticks when you can. use $(command) instead.
Hope it helps.
#!/bin/bash
#######################
#Function to add hosts to the array
#the following function takes the ip addresses provided while the script is run and stores them in an array
#######################
array=( "$#" )
user="student35"
############################################################
#Collect Stats of Ping,memory,iowait time test function
############################################################
for host in ${array[#]}; do
sshpass -f /root/scripts/passwordFile.txt /usr/bin/ssh -l ${user} ${host} <<-HERE
thishost=\$(hostname)
echo "Current Host -> \$thishost";
iowaittm=\`sar 2 2|awk '/^Average/{print \$5}'\`
if [ \$iowaittm > 10 ]; then
echo "IO ==> BAD"
else
echo "IO ==> GOOD"
fi
memory=\$(free -m | grep Swap | awk '{if(\$2 == 0) print 0;else print ((\$4 / \$2 ) * 100)}')
if [ \${memory} < '10' ] ;then
echo "memory ==> good"
elif [[ "\${memory}" -ge 0 ]] && [[ "\${memory}" -le 10 ]]; then
echo "No Swap"
else
echo "memory ==> bad"\`enter code here\`
fi
ping -w2 -c2 \`hostname\` | grep "packet loss"|awk -F, '{print \$3}'|awk -F% '{print \$1}'|sed 's/^ *//'|awk '{if (\$1 == 0) print "Yes" ;else print "No"}'
HERE
done

trap ERR doesn't work with pipes

I am try to make a system backup script with trap "" ERR. I realized the trap doesn't get called when commands are part of pipes |.
Heres are some parts of my code that don't work with trap "" ERR ...
OpenFiles=$(lsof "$Source" | wc -l)
PackagesList=$(dpkg --get-selections | awk '!/deinstall|purge|hold/ {print $1}' | tee "$FilePackagesList")
How can I get this to work without using if [ "$?" -eq 0 ]; then, or similar coding ? Because this is the reason I declared a trap this way.
Here is the script ...
root#Lian-Li:~# cat /usr/local/bin/create_incremental_backup_of_system.sh
#!/bin/bash
# Create an incremental GNU-standard backup of important system-files.
# This script works with Debian Jessie and newer systems.
# Created for my lian-li NAS 2016-11-27.
MailTo="admin#example.com" # Mail Address of an admin
Source="boot etc root usr/local usr/lib/cgi-bin var/www"
BackupDirectory=/media/hdd1/backups/lian-li
SubDir="system.d"
FileTimeStamp=$(date "+%Y%m%d%H%M%S")
FileName=$(uname -n)
File="${BackupDirectory}/${SubDir}/${FileName}-${FileTimeStamp}.tgz"
FileIncremental="${BackupDirectory}/${SubDir}/${FileName}.gtar"
FilePackagesList="${BackupDirectory}/${SubDir}/installed_packages_on_${FileName}.txt"
# have2do ...
# Backup rotate
MailContent="None"
TimeStamp=$(date "+%F %T") # This format "2011-12-31 23:59:59" is needed to read the journal
exec 1> >(logger -i -s -t "$0" -p 3) 2>&1 # all error messages are redirected to syslog journal and after that to stdout
trap "BriefExit" ERR # Provide information for an admin (via sendmail) when an error occurred and exit the script
function BriefExit(){
rm -f "$File"
if [ "$MailContent" = "None" ]
then
case "$LANG" in
de_DE.UTF-8)
echo "Beende Skript, aufgrund vorherige Fehler." 1>&2
;;
*)
echo "Stopping script because of previous error(s)." 1>&2
;;
esac
MailContent=$(journalctl -p 3 -o "short" --since="$TimeStamp" --no-pager)
ScriptName="${0##*/}"
SystemName=$(uname -n)
MailSubject="${SystemName}: ${ScriptName}"
echo -e "Subject: ${MailSubject}\n\n${MailContent}\n" | sendmail "$MailTo"
fi
exit 1
}
if [ ! -d "${BackupDirectory}/${SubDir}" ]
then
mkdir -p "${BackupDirectory}/${SubDir}"
fi
LoopCount=0
OpenFiles=1
cd /
while [ "$OpenFiles" -ne 0 ]
do
if [ "$LoopCount" -le 180 ]
then
sleep 1
OpenFiles=$(lsof $Source | wc -l)
LoopCount=$(($LoopCount + 1))
else
echo "Closing Script. Reason: Can't create incremental backup, because some files are open." 1>&2
BriefExit
fi
done
tar -cpzf "$File" -g "$FileIncremental" $Source
chmod 0700 "$File"
PackagesList=$(dpkg --get-selections | awk '!/deinstall|purge|hold/ {print $1}' | tee "$FilePackagesList")
while read -r PackageName
do
case "$PackageName" in
minidlna)
# Code ...
;;
slapd)
# Code ...
;;
esac
done <<< "$PackagesList"
exit 0
This isn't a problem with ERR traps at all, or with command substitutions, but with pipelines.
false | true
returns true, unless the pipefail option is set.
Thus in OpenFiles=$(lsof "$Source" | wc -l), only a failure in wc will cause the pipeline to be considered a failure, or in PackagesList=$(dpkg --get-selections | awk '!/deinstall|purge|hold/ {print $1}' | tee "$FilePackagesList"), only a failure in tee will cause the command as a whole to be considered failed.
Put the command set -o pipefail at the top of your script if you want a failure from any pipeline component (as opposed to the last component alone) to cause the command as a whole to be considered failed -- and note the other caveats for ERR traps given in BashFAQ #105.
Another alternative is to look at the status for each stage in the pipeline:
# cat test_bash_return.bash
true | true | false | true
echo "${PIPESTATUS[#]}"
# ./test_bash_return.bash
0 0 1 0

Bash sub script redirects input to /dev/null mistakenly

I'm working on a script to automate the creation of a .gitconfig file.
This is my main script that calls a function which in turn execute another file.
dotfile.sh
COMMAND_NAME=$1
shift
ARG_NAME=$#
set +a
fail() {
echo "";
printf "\r[${RED}FAIL${RESET}] $1\n";
echo "";
exit 1;
}
set -a
sub_setup() {
info "This may overwrite existing files in your computer. Are you sure? (y/n)";
read -p "" -n 1;
echo "";
if [[ $REPLY =~ ^[Yy]$ ]]; then
for ARG in $ARG_NAME; do
local SCRIPT="~/dotfiles/setup/${ARG}.sh";
[ -f "$SCRIPT" ] && echo "Applying '$ARG'" && . "$SCRIPT" || fail "Unable to find script '$ARG'";
done;
fi;
}
case $COMMAND_NAME in
"" | "-h" | "--help")
sub_help;
;;
*)
CMD=${COMMAND_NAME/*-/}
sub_${CMD} $ARG_NAME 2> /dev/null;
if [ $? = 127 ]; then
fail "'$CMD' is not a known command or has errors.";
fi;
;;
esac;
git.sh
git_config() {
if [ ! -f "~/dotfiles/git/gitconfig_template" ]; then
fail "No gitconfig_template file found in ~/dotfiles/git/";
elif [ -f "~/dotfiles/.gitconfig" ]; then
fail ".gitconfig already exists. Delete the file and retry.";
else
echo "Setting up .gitconfig";
GIT_CREDENTIAL="cache"
[ "$(uname -s)" == "Darwin" ] && GIT_CREDENTIAL="osxkeychain";
user " - What is your GitHub author name?";
read -e GIT_AUTHORNAME;
user " - What is your GitHub author email?";
read -e GIT_AUTHOREMAIL;
user " - What is your GitHub username?";
read -e GIT_USERNAME;
if sed -e "s/AUTHORNAME/$GIT_AUTHORNAME/g" \
-e "s/AUTHOREMAIL/$GIT_AUTHOREMAIL/g" \
-e "s/USERNAME/$GIT_USERNAME/g" \
-e "s/GIT_CREDENTIAL_HELPER/$GIT_CREDENTIAL/g" \
"~/dotfiles/git/gitconfig_template" > "~/dotfiles/.gitconfig"; then
success ".gitconfig has been setup";
else
fail ".gitconfig has not been setup";
fi;
fi;
}
git_config
In the console
$ ./dotfile.sh --setup git
[ ?? ] This may overwrite existing files in your computer. Are you sure? (y/n)
y
Applying 'git'
Setting up .gitconfig
[ .. ] - What is your GitHub author name?
Then I cannot see what I'm typing...
At the bottom of dotfile.sh, I redirect any error that occurs during my function call to /dev/null. But I should normally see what I'm typing. If I remove 2> /dev/null from this line sub_${CMD} $ARG_NAME 2> /dev/null;, it works!! But I don't understand why.
I need this line to prevent my script to echo an error in case my command doesn't exists. I only want my own message.
e.g.
$ ./dotfile --blahblah
./dotfiles: line 153: sub_blahblah: command not found
[FAIL] 'blahblah' is not a known command or has errors
I really don't understand why the input in my sub script is redirected to /dev/null as I mentioned only stderr to be redirected to /dev/null.
Thanks
Do you need the -e option in your read statements?
I did a quick test in an interactive shell. The following command does not echo characters :
read -e TEST 2>/dev/null
The following does echo the characters
read TEST 2>/dev/null

UNIX script abruptly ending due to ssh command [duplicate]

This question already has an answer here:
Closed 11 years ago.
Possible Duplicate:
ssh invocation in script function
Below UNIX script abruptly ends while reading second line from file. When I comment 'ssh' command the script works as expected. I think I will have to run ssh command in a different process, but haven't got a handle yet as regards to how to do that. Any help in resolving this problem is highly appreciated.
*#!/usr/bin/ksh
exec 3<&0
exec 0<./bulkInput.dat
#cat ./bulkInput.dat | while read searchToken || (echo "reading failedi $?" && false)
index=0
while true
do
index=`expr $index + 1`
if [ $index -gt 450 ]
then
echo "Index limit reached. Now exiting"
exit 0
fi
read searchToken
if [ $? -ne "0" ]
then
echo "Read failed"
fi
echo "Search token is "${searchToken}
echo "************************************ **********************************"
echo "plsa0666 WSIP server " > WSIP.log
ssh zq742888#plsa0666 'grep -r '$searchToken' /logs/jas/was60/wsip/wsip*/wsip*' >> WSIP.log
echo "plsa0667 WSIP server " >> WSIP.log
#ssh zq742888#plsa0667 'grep -r '$searchToken' /logs/jas/was60/wsip/wsip*/wsip*' >> WSIP.log
echo "plsa0668 WSIP server " >> WSIP.log
#ssh zq742888#plsa0668 'grep -r '$searchToken' /logs/jas/was60/wsip/wsip*/wsip*' >> WSIP.log
echo "plsa4407 WSIP server " >> WSIP.log
#ssh zq742888#plsa4407 'grep -r '$searchToken' /logs/jas/was60/wsip/wsip*/wsip*' >> WSIP.log
echo "plsa0412 server " >> WSIP.log
cp WSIP.log bulk/WSIP.log_${searchToken}
echo $?
done
exec 0<&3
echo "Exiting script"*
ssh(1) is reading all of stdin and exhausting it, causing the next shell read to return false and break the loop. Try one of these:
ssh -n zq742888#plsa0666 ...
or
ssh < /dev/null zq742888#plsa0666 ...
to prevent this behavior.
Run the ssh command from the shell prompt and see what it does. If it is asking for input (e.g. password) then that may be problem.
There is also a flag to run in script mode(from memory -b but you should check) and that may also help you.
The -i flag allows you to specify the key to use if that is the problem.

Resources