Calling ssh in a loop is destroying the loop variable context (ksh93 on AIX 7.1) - ksh

I'm posting this in case anyone else runs into this issue. I will open a case with IBM, and I'll report back if there's an update from them.
I was having a problem on AIX 7.1 (but not on 6.1) when I'm using a file
to drive a loop (eg: cat myfile.txt | while read myvar). Calling ssh
from within the loop causes the loop to exit prematurely on the first iteration.
In my real-life example I wasn't using "ls" to populate the file, but doing so here made it easier to demonstrate.
I've got the following code:
#!/usr/bin/ksh93
ssh LESAUN01 -l root "ls -1 c*" > file.list
echo "------------------------------------------"
cat file.list
echo "------------------------------------------"
echo ""
echo "Loop 1"
echo ""
cat file.list | while read file1
do
echo " File: $file1"
done
echo ""
echo "Loop 2"
echo ""
cat file.list | while read file2
do
echo " File: $file2"
ssh LESAUN01 -l root "ls -l $file2"
done
echo ""
echo "Done"
exit
When I run it from an AIX 6.1 lpar I get these expected results
-----------------------------------------
client.txt
customer_handover.log
------------------------------------------
Loop 1
File: client.txt
File: customer_handover.log
Loop 2
File: client.txt
-rw-r--r-- 1 root root 91323 Feb 12 2015 client.txt
File: customer_handover.log
-rw------- 1 root root 27533 Aug 31 18:04 customer_handover.log
Done
When run on 7.1 I get this result:
------------------------------------------
client.txt
customer_handover.log
------------------------------------------
Loop 1
File: client.txt
File: customer_handover.log
Loop 2
File: client.txt
-rw-r--r-- 1 root root 91323 Feb 12 2015 client.txt
Done
My solution was to feed the loop from an array instead of a file (after loading the file into the array) which works as expected.
This works on both 6.1 and 7.1
#!/usr/bin/ksh93
ssh LESAUN01 -l root "ls -1 c*" > file.list
echo "------------------------------------------"
cat file.list
echo "------------------------------------------"
echo ""
echo "Loop"
echo ""
i=0
set -A file_array
cat file.list | while read line
do
file_array[ $i ]="$line"
(( i++ ))
done
for x in "${!file_array[#]}"
do
echo " File: ${file_array[$x]}"
ssh LESAUN01 -l root "ls -l ${file_array[$x]}"
done
echo ""
echo "Done"
exit
Which gives this expected result.
------------------------------------------
client.txt
customer_handover.log
------------------------------------------
Loop
File: client.txt
-rw-r--r-- 1 root root 91323 Feb 12 2015 client.txt
File: customer_handover.log
-rw------- 1 root root 27533 Aug 31 18:04 customer_handover.log
Done

cat file.list | while read file2
do
echo " File: $file2"
ssh LESAUN01 -l root "ls -l $file2"
done
The output of cat file.list is the standard input for every command inside the loop. ssh reads from its standard input in order to relay the stream to the remote process, so it's consuming the cat output.
A simple fix is to redirect ssh's standard input:
cat file.list | while read file2
do
echo " File: $file2"
ssh LESAUN01 -l root "ls -l $file2" < /dev/null
done

Related

Shell script to check if process is running everyday

Can someone please let me know how to complete the scenario below:
I have 3 processes (TC_ME, PS_ME, and NM_ME) running on a machine. When these services go down, they do not send out any alert or mail.
I would like to write a shell script based out of these processes, so whenever the processes go down, I would require an alert.
Will create an autosys job based on this.
you can try this;
#! /bin/bash
while :
do
RESULT=`ps -ef | grep "TC_ME\|PS_ME\|NM_ME" | grep -v 'grep' | grep -v $0 | wc -l`
if [ "$RESULT" != "3" ]; then
echo `date` ": some of your processes not running"
else
echo `date` ": processes runs"
fi
sleep 10
done
Eg;
user#host:/tmp$ ./processesCheck.sh
Wed Oct 5 15:54:01 EEST 2016 : some of your processes not running
Wed Oct 5 15:54:11 EEST 2016 : some of your processes not running
Wed Oct 5 15:54:21 EEST 2016 : some of your processes not running
Wed Oct 5 15:54:31 EEST 2016 : some of your processes not running
you can also redirect to a log file as below;
user#host:/tmp$ nohup ./processesCheck.sh >> processes.log
if i were to do with the command below, where Ownerserver is a common word for all the above processes can i edit the script as below?
while :
do
RESULT=ps -ef | grep "OwnerServer" | grep -v 'grep' | wc -l
if [ "$RESULT" -gt 1 ]; then
echo date ": processes runs"
else
echo date ": some of your processes not running"
fi
sleep 10
done

Why does not working bash files include?

I looked now more like a theme (e.g.: Bash: How _best_ to include other scripts?, it also tried none of them work.
I should also call "create_sample_html.sh" file in the same directory in the "test.sh" file.
But no matter what I tried, nothing happens when you run.
But in I tried to run script in Docker container return the following error message:
./create_sample_html.sh: No such file or directory
create_php_info: command not found
create_html: command not found
test.sh:
#!/bin/bash
source ./create_sample_html.sh
IP=$(ip route get 1 | awk '{print $NF;exit}')
create_php_info "<path-to-file>/index.php"
create_html ${IP} "test1" "<path-to-file>/index.html"
exit 0
create_sample_html.sh:
#!/bin/bash
create_html() {
touch $3
echo -e "<!Doctype html>" >> $3
echo -e "<html>" >> $3
echo -e "\t<head>" >> $3
echo -e "\t\t<meta charset=\"utf-8\">" >> $3
echo -e "\t\t<title>Server $1 VHost: $2</title>" >> $3
echo -e "\t</head>" >> $3
echo -e " " >> $3
echo -e "\t<body>" >> $3
echo -e "\t\t<h1>Server $1 VHost: $2</h1>" >> $3
echo -e "\t</body>" >> $3
echo -e "</html>" >> $3
}
create_php_info() {
touch $1
echo -e "<?php phpinfo(); ?>" >> $1
}
exit 0
file permissions:
-rwxrwxr-x 1 user user 480 aug 3 20:48 create_sample_html.sh
-rwxrwxr-x 1 user user 332 aug 3 20:37 test.sh
In create_sample_html.sh, remove exit 0.
Else, test.sh will execute exit 0 at the moment you are sourcing it. In the same step you can also remove #!/bin/bash from the sourced file because it's unnecessary since the functions will be executed with the shell of the parent script anyway.

Creating directories with increasing name using mkdir [duplicate]

This question already has answers here:
bash create dir with sequential numbers
(5 answers)
Closed 9 years ago.
I'd like to have a bash script that would create folder called "exam_folder" and if this folder already exists (I'd run the script again for example), the script would create "exam_folder_1" and if that folder exists I want the script to create "exam_folder_2" and so on and on. I'd like to have it smartly :)
I got something like this:
function create_folder {
if [ -d "$1" ]; then # if exists
mkdir $1_`i_max` # function i_max would find out the max number and add 1
else
mkdir $1
fi }
Thank you for your help.
You can do this:
max=`ls -1d exam_folder* | tr -dc '[0-9\n]' | sort -k 1,1n | tail -1`
mkdir exam_folder_$((max + 1))
Note that it won't fill 'holes' in the sequence, if your'e OK with that.
This should work under any POSIX-compliant shell (although I am not 100% certain about the [ "$foo" -eq "$foo" ] is-number-check). I gave it a quick test under dash and busybox' ash.
create_folder()
{
if [ -e "$1" ]; then
i=0
for d in "$1"_*; do
_i="${d##*_}"
[ "${_i}" -eq "${_i}" ] 2>/dev/null && i="${_i}"
done
i=$((i+1))
mkdir "$1_${i}"
else
mkdir "$1"
fi
}
Two ansers: The compatible shell and the pure bash:
Last edit: Make regexp more precise and correct some counting bugs.
Something like:
create_folder() {
if [ -e "$1" ] ;then
counter=1;
while [ -e "$1-$counter" ] ;do
counter=$((counter+1))
done
mkdir "$1-$counter";
else
mkdir "$1";
fi;
}
could do:
$ ls -ltr /tmp |grep foo
$ create_folder /tmp/foo\ bar
$ create_folder /tmp/foo\ bar
$ create_folder /tmp/foo\ bar
$ ls -ltr /tmp |grep foo
drwxr-xr-x 2 user user 4096 1 mai 10:53 foo bar
drwxr-xr-x 2 user user 4096 1 mai 10:53 foo bar-1
drwxr-xr-x 2 user user 4096 1 mai 10:53 foo bar-2
or with a better formatting and another way of counting (quicker if many directories exists as it use specialized count function, but longer if else because of forks.):
create_folder() {
if [ -e "$1" ] ;then
counter=`/bin/ls -1 "$1" "$1-"* 2>&1 | grep -E "^$1(-[0-9]*)?" | wc -l`;
nname="`printf "%s-%04d" "$1" $counter`";
mkdir "$nname";
else
mkdir "$1";
fi;
}
This was tested under different shell implementations: bash, dash, ash and ksh
$ create_folder /tmp/test
$ ls -ltr /tmp/| grep test
drwxr-xr-x 2 user user 4096 Apr 30 13:25 test
$ create_folder /tmp/test
$ create_folder /tmp/test
$ ls -ltr /tmp/| grep test
drwxr-xr-x 2 user user 4096 Apr 30 13:25 test
drwxr-xr-x 2 user user 4096 Apr 30 13:26 test-0001
drwxr-xr-x 2 user user 4096 Apr 30 13:26 test-0002
And with dirname containing whitespaces:
$ create_folder "foo bar"
$ create_folder "foo bar"
$ create_folder "foo bar"
$ create_folder "foo bar"
$ ls -ltr | grep foo
drwxr-xr-x 2 user user 4096 Apr 30 13:35 foo bar
drwxr-xr-x 2 user user 4096 Apr 30 13:35 foo bar-0001
drwxr-xr-x 2 user user 4096 Apr 30 13:35 foo bar-0002
drwxr-xr-x 2 user user 4096 Apr 30 13:35 foo bar-0003
And for bash fans:
create_folder() {
local -a list
local counter nname
if [ -e "$1" ] ;then
list=("$1-"*)
list=(${list[#]// /_})
list=(${list[#]//*\*})
counter=$((${#list[#]}+1))
printf -v nname "%s-%04d" "$1" $counter
mkdir "$nname";
else
mkdir "$1";
fi;
}
Very quick, no fork and nice formatting:
$ create_folder "/tmp/foo bar"
$ create_folder "/tmp/foo bar"
$ create_folder "/tmp/foo bar"
$ create_folder "/tmp/foo bar"
$ ls -ltr /tmp/ | tail -n 4
drwxr-xr-x 2 user user 4096 1 mai 11:17 foo bar
drwxr-xr-x 2 user user 4096 1 mai 11:17 foo bar-0001
drwxr-xr-x 2 user user 4096 1 mai 11:17 foo bar-0002
drwxr-xr-x 2 user user 4096 1 mai 11:17 foo bar-0003

if result from ping

I basically want the following:
#full .co.uk/.com domain names are kept in /var/www/vhosts
for each in /var/www/vhosts
do
echo "Attempting to ping $each"
ping -c 1 $each
if [ $? comes back with an IP ]; then
echo "The web server for $each is responding. $each points to $IPADDRESS"
domainlist=`echo "$each points to $IPADDRESS" >> domaintoiplist.txt`
echo ""
else
echo "Host $each is not reachable!"
fi
done
The bit I can't get working is the fact that ping returns exit status of 0 no matter what. Do I need the solution from: Ping function having undesirable results
walk through all files in /var/www/vhosts, filter directories, get the names only
ping the domain, get the IP address from the first line
exit code of ping is accessible via PIPESTATUS
write positives ( exit code 0 ) into the file
I am sure, this can be done simpler but I am not a bash guru :)
for domain in $(ls -lA /var/www/vhosts | grep "^d" | sed -E 's/.* ([^ ]+)?$/\1/g');
do
ip=$(ping -c 1 $domain|grep "PING" | sed -E 's/PING .* .([0-9.]+). .*/\1/g');
if [ ${PIPESTATUS[0]} -eq 0 ];
then
echo "${domain} translates to ${ip}";
echo -e "${domain}\t${ip}" >> translation.txt;
fi;
done
my test folder
# ls -al /var/www/vhosts
-rw-r--r-- 1 root root 0 29. Jan 15:45 other
-rw-r--r-- 1 root root 699 29. Jan 16:34 translation.txt
drwxr-xr-x 2 root root 4096 29. Jan 16:12 www.adasdasdadasdadsadadasdasdasd.com
drwxr-xr-x 2 root root 4096 29. Jan 15:44 www.doesntexist.com
drwxr-xr-x 2 root root 4096 29. Jan 15:44 www.google.com
my output
www.doesntexist.com translates to 204.13.248.119 // uh, actually it DOES exist :)
www.google.com translates to 173.194.35.145
EDIT
If you pipe a programs output to another program, $? will contain the exit code of the receiving program. To get the exit code of a program which is called earlier in the pipe chain, you can use the bash-internal variable $PIPESTATUS.
I think, this is the problem currently.
All credit to https://stackoverflow.com/users/1032504/michel-feldheim
Final script that worked is below:
#!/bin/bash
# Get the IP addresses assigned to the server. This can be used later to match the domains to the servers IP addresses.
IPs=`ifconfig | grep "inet addr" | awk '{print $2;}' | cut -c 6-19 | grep -v "127.0"`
echo "=== This server's IP addresses: ==="
ifconfig | grep "inet addr" | awk '{print $2;}' | cut -c 6-19 | grep -v "127.0"
echo "=== Done. They are listed above. ==="
echo ""
#change the directory where the sites are located below
for domain in $(ls -lA /var/www/vhosts | grep "^d" | sed -E 's/.* ([^ ]+)?$/\1/g' | grep -v '.skel\|chroot\|default\|fs\|fs-passwd\|httpsdocs');
do
echo "Attempting to ping $domain"
ip=$(ping -c 1 $domain|grep "PING" | sed -E 's/PING .* .([0-9.]+). .*/\1/g' | grep -v '.skel\|chroot\|default\|fs\|fs-passwd\|httpsdocs');
if [ ${PIPESTATUS[0]} -eq 0 ];
then
echo "${domain} points to ${ip}";
#the below line doesn't output to STDOUT on purpose, it goes into the file below.
echo -e "${domain} points to ${ip}" >> domaintoiplist.txt
echo ""
else
echo "Host ${domain} is not reachable!"
echo ""
fi;
done
echo "==== Done. Written a list of domains & its IP to domaintoiplist.txt in the current directory. ===="

Sort a list of files based on numeric value in name

I have this list of files and I want to sort them and increment their names by an integer value, my code works fine until the list hits 10. The linux 'sort' command then interprets the first '1' in '10' and thinks it is a smaller number than 9. Is there any way to make this work ?
This is the code I have written to loop over a folder and increment file names:
#!/bin/bash
#set -x
ROOT=~/testing/
FILE_COUNT=$(ls -1 $ROOT | wc -l | awk '{print $1}')
COUNT=5
if [[ ${FILE_COUNT} -eq $COUNT ]]; then
echo $COUNT backup files are there
FILE_LIST=$(ls -1 $ROOT | sort -n -r)
for file in $FILE_LIST; do
echo $file
file_new=`basename $file .zip`
if [[ -e $ROOT$file ]]; then
#mv $ROOT$file $ROOT${file_new%?}$COUNT.zip
FILENUM=${file_new:${#file_new}-1}
#echo "This is file # $FILENUM" next one is $(( FILENUM + 1 ))
echo mv $ROOT$file $ROOT${file_new%?}$(( FILENUM + 1 )).zip
mv $ROOT$file $ROOT${file_new%?}$(( FILENUM + 1 )).zip
fi
((COUNT--))
done
else
echo Not $COUNT files, there are $FILE_COUNT
COUNT=$FILE_COUNT
fi
And these are the results of the sort line:
macbookair:~ ilium007$ ls -l testing/ | sort -n -r -t "_"
total 40
-rw-r--r-- 1 ilium007 staff 15 16 Nov 21:24 backup_9.zip
-rw-r--r-- 1 ilium007 staff 15 16 Nov 21:24 backup_8.zip
-rw-r--r-- 1 ilium007 staff 15 16 Nov 21:24 backup_7.zip
-rw-r--r-- 1 ilium007 staff 15 16 Nov 21:24 backup_6.zip
-rw-r--r-- 1 ilium007 staff 15 16 Nov 21:24 backup_10.zip
How do I create this list of files:
backup_10.zip
backup_9.zip
backup_8.zip
backup_7.zip
backup_6.zip
Any help appreciated.
You need to specify the key to sort on, in this case -k2:
ls | sort -n -r -t "_" -k2
This code ended up working:
#!/bin/bash
set -x
ROOT=~/testing/
FILE_COUNT=$(ls -1 $ROOT | wc -l | awk '{print $1}')
COUNT=5
FILENAME=("backup_19.zip"
"backup_2.zip
"backup_29.zip
"backup_38.zip")
for i in ${FILENAME[#]}; do
BASE_FILE_NAME=`basename $i .zip`
FILENUM=${BASE_FILE_NAME##*_}
NEW_FILE_NUM=$(( FILENUM + 1 ))
NEW_FILE_SUFFIX=$(( FILENUM + 1 )).zip
TEST=${BASE_FILE_NAME%%_*}_${NEW_FILE_SUFFIX}
done
exit

Resources