I have a scritp that gathers information about a subdirectory of files. I am checking that the time between file creation is uniform.
last=0
LOGCHECK="YES"
ls -l /dir/*.log | gawk '{print $8}' | sed s/:/*60+/g | bc |
while read fname
do
current=$fname
if [ $last = 0 ]; then
last=$current
elif [ $((current - last)) -ne 1 ]; then
echo "Time difference discrepancy: $((current - last)) minute(s) is not expected"
LOGCHECK="NO"
last=$current
else
last=$current
fi
done
This outputs only if the time between .log file creation is not one minute. My problem is that the $LOGCHECK inside the while loop is in another subshell I believe from the pipe?
Is there a way to get this variable information?
This is a common situation with bash scripting. Restructure your loop like this:
while read fname
do
# stuff
done < <(ls -l /dir/*.log | gawk '{print $8}' | sed s/:/*60+/g | bc)
Then variables set within the loop will still be available afterwards.
Related
I want to run a command in a shell script if files in one directory have changed more recently than files in another directory.
I would like something like this
if [ dir1/* <have been modified more recently than> dir2/* ]; then
echo 'We need to do some stuff!'
fi
As described in BashFAQ #3, broken down here into reusable functions:
newestFile() {
local latest file
for file; do
[[ $file && $file -nt $latest ]] || latest=$file
done
}
directoryHasNewerFilesThan() {
[[ "$(newestFile "$1"/*)" -nt "$(newestFile "$2" "$2"/*)" ]]
}
if directoryHasNewerFilesThan dir1 dir2; then
echo "We need to do something!"
else
echo "All is well"
fi
If you want to count the directories themselves as files, you can do that too; just replace "$(newestFile "$1"/*)" with "$(newestFile "$1" "$1"/*)", and likewise for the call to newestFile for $2.
Using /bin/ls
#!/usr/bin/ksh
dir1=$1
dir2=$2
#get modified time of directories
integer dir1latest=$(ls -ltd --time-style=+"%s" ${dir1} | head -n 2 | tail -n 1 | awk '{print $6}')
integer dir2latest=$(ls -ltd --time-style=+"%s" ${dir2} | head -n 2 | tail -n 1 | awk '{print $6}')
#get modified time of the latest file in the directories
integer dir1latestfile=$(ls -lt --time-style=+"%s" ${dir1} | head -n 2 | tail -n 1 | awk '{print $6}')
integer dir2latestfile=$(ls -lt --time-style=+"%s" ${dir2} | head -n 2 | tail -n 1 | awk '{print $6}')
#sort the times numerically and get the highest time
val=$(/bin/echo -e "${dir1latest}\n${dir2latest}\n${dir1latestfile}\n${dir2latestfile}" | sort -n | tail -n 1)
#check to which file the highest time belongs to
case $val in
#(${dir1latest}|${dir1latestfile})) echo $dir1 is latest ;;
#(${dir2latest}|${dir2latestfile})) echo $dir2 is latest ;;
esac
It's simple, get times stamps of both the folders in machine format(epoch time) then do simple comparison. that's all
we were given a task to write a script in a course. We have to make the script find out which proccess is "deepest" in process hierarchy, something like "pstree" command, but the output will be "depth_of_process : processes_with_the_depth".
I have started something, but I can't make it work. Could you please look at it and help me ? I haven't even started producing the output, I am working on the algorithm now - trying to make it into something like reverse depth-first search. In case the code is not self-explanatory enough, please let me know, I will do my best to describe it.
#!/bin/bash
PROCS=$(ps -eo "%p %P" | tail -n +2 | sort -nr)
declare -a array
while read -r line; do
counter=1
read kid parent
while read -r otherline; do
read kid2 parent2
if [ "$parent" = "$kid2" ]; then
counter=$((counter+1))
parent="$parent2"
fi
done <<< "$PROCS"
test=2
array["$kid"]="$counter"
done <<< "$PROCS"
#for value in "${!array[#]}"; do
# echo "$value ${array[value]}"
#done
echo "$PROCS"
If pstree is allowed I could offer this (thanks #tripleee for optimizing):
for processid in $(ps -ax | awk 'NR>1 {print $1}' ); do
depth=$(pstree -sA $processid | head -n1 | sed -e 's#-+-.*#---foobar#' -e 's#---*#\n#g' -eq | wc -l)
echo "$depth: $processid"
done
It might have issues if your processes contain two or more dashes in a row.
Of course you can add " | sort" after "done" to get the deepest processes.
Code sample :
declare -i a=1
echo "The number of NMON instances running in Performance VM"
ps -ef | grep nmon | awk '{ print $2 }' | wc -l
echo "---------------------------------------------"
num2= ps -ef | grep nmon | awk '{ print $2 }' | wc -l
#num1=${num2%%.*}
#num2 = $(ps -ef | grep nmon | awk '{ print $2 }' | wc -l)
echo "---------------------------------------------"
echo "${num2}"
while [ "$a" -lt "$num2" ]
do
kill -USR2 $(ps -ef | grep nmon | awk '{ print $2 }' | head -1)
a=`expr $a + 1`
done
In the Output i am getting the following error
[: : integer expression expected
in the debug it shows
++ '[' 1 -lt '' ']'
that num2 is empty but when i echo the num2 value i am getting the value correctly.
Output:
The number of NMON instances running in Performance VM
1
1
thanks in advance
The 1 you see in the output is not from echo "${num2}". Like the diagnostics already tell you, this variable is empty.
The general syntax of shell scripts is
[ variable=value ...] command parameters ...
which will assign value to variable for the duration of command, then restore its original value. So the pipeline you are running temporarily sets num2 to the empty string (which apparently it already contained anyway), then runs the pipeline without storing the output anywhere (such as, I imagine you expected, in num2).
Here is a fixed version of your script, with the additional change that the Awk scripts handle stuff you used grep and head and wc for. Because the functionality of these commands is easily replaced within Awk, using external utilities is doubtful (especially so for grep which really is useless when you just run it as a preprocessor for a simple Awk script).
countnmon () {
ps -ef | awk '/[n]mon/ { ++n } END { print n }'
}
declare -i a=1
echo "The number of NMON instances running in Performance VM"
countnmon
echo "---------------------------------------------"
num2=$(countnmon)
#num1=${num2%%.*}
#num2 = $(countnmon)
echo "---------------------------------------------"
echo "${num2}"
while [ "$a" -lt "$num2" ]
do
kill -USR2 $(ps -ef | awk '/[n]mon/ { print $2; exit }')
a=`expr $a + 1`
done
The repeated code could be refactored even further to avoid all code duplication but that will somewhat hamper the readability of this simple script so I have not done that.
Whitespaces matter in bash.
The syntax for command execution is:
command arg1 arg2 ...
So,
var = value # command: var, arg1: =, arg2: value
There's are two exceptions to this rule
exporting variables to an executed command (the variables vanish after the command finishes):
var1=value1 var2=value2 .. command arg1 arg2 ...
assigning a variable (you want this one):
var=value
I have a problem creating a script that reads specific value from all the files of an entire folder
I have a number of email files in a directory and I need to extract from each file, 2 specific values.
After that I have to put them into a new file that looks like that:
--------------
To: value1
value2
--------------
This is what I want to do, but I don't know how to create the script:
# I am putting the name of the files into a temp file
`ls -l | awk '{print $9 }' >tmpfile`
# use for the name of a file
`date=`date +"%T"
# The first specific value from file (phone number)
var1=`cat tmpfile | grep "To: 0" | awk '{print $2 }' | cut -b -10 `
# The second specific value from file(subject)
var2=cat file | grep Subject | awk '{print $2$3$4$5$6$7$8$9$10 }'
# Put the first value in a new file on the first row
echo "To: 4"$var1"" > sms-$date
# Put the second value in the same file on the second row
echo ""$var2"" >>sms-$date
.......
and do the same for every file in the directory
I tried using while and for functions but I couldn't finalize the script
Thank You
I've made a few changes to your script, hopefully they will be useful to you:
#!/bin/bash
for file in *; do
var1=$(awk '/To: 0/ {print substr($2,0,10)}' "$file")
var2=$(awk '/Subject/ {for (i=2; i<=10; ++i) s=s$i; print s}' "$file")
outfile="sms-"$(date +"%T")
i=0
while [ -f "$outfile" ]; do outfile="sms-$date-"$((i++)); done
echo "To: 4$var1" > "$outfile"
echo "$var2" >> "$outfile"
done
The for loop just goes through every file in the folder that you run the script from.
I have added added an additional suffix $i to the end of the file name. If no file with the same date already exists, then the file will be created without the suffix. Otherwise the value of $i will keep increasing until there is no file with the same name.
I'm using $( ) rather than backticks, this is just a personal preference but it can be clearer in my opinion, especially when there are other quotes about.
There's not usually any need to pipe the output of grep to awk. You can do the search in awk using the / / syntax.
I have removed the cut -b -10 and replaced it with substr($2, 0, 10), which prints the first 10 characters from column 2.
It's not much shorter but I used a loop rather than the $2$3..., I think it looks a bit neater.
There's no need for all the extra " in the two output lines.
I sugest to try the following:
#!/bin/sh
RESULT_FILE=sms-`date +"%T"`
DIR=.
fgrep -l 'To: 0' "$DIR" | while read FILE; do
var1=`fgrep 'To: 0' "$FILE" | awk '{print $2 }' | cut -b -10`
var2=`fgrep 'Subject' "$FILE" | awk '{print $2$3$4$5$6$7$8$9$10 }'`
echo "To: 4$var1" >>"$RESULT_FIL"
echo "$var2" >>"$RESULT_FIL"
done
I have a bash script that processes all of the files in a directory using a loop like
for i in *.txt
do
ops.....
done
There are thousands of files and they are always processed in alphanumerical order because of '*.txt' expansion.
Is there a simple way to random the order and still insure that I process all of the files only once?
Assuming the filenames do not have spaces, just substitute the output of List::Util::shuffle.
for i in `perl -MList::Util=shuffle -e'$,=$";print shuffle<*.txt>'`; do
....
done
If filenames do have spaces but don't have embedded newlines or backslashes, read a line at a time.
perl -MList::Util=shuffle -le'$,=$\;print shuffle<*.txt>' | while read i; do
....
done
To be completely safe in Bash, use NUL-terminated strings.
perl -MList::Util=shuffle -0 -le'$,=$\;print shuffle<*.txt>' |
while read -r -d '' i; do
....
done
Not very efficient, but it is possible to do this in pure Bash if desired. sort -R does something like this, internally.
declare -a a # create an integer-indexed associative array
for i in *.txt; do
j=$RANDOM # find an unused slot
while [[ -n ${a[$j]} ]]; do
j=$RANDOM
done
a[$j]=$i # fill that slot
done
for i in "${a[#]}"; do # iterate in index order (which is random)
....
done
Or use a traditional Fisher-Yates shuffle.
a=(*.txt)
for ((i=${#a[*]}; i>1; i--)); do
j=$[RANDOM%i]
tmp=${a[$j]}
a[$j]=${a[$[i-1]]}
a[$[i-1]]=$tmp
done
for i in "${a[#]}"; do
....
done
You could pipe your filenames through the sort command:
ls | sort --random-sort | xargs ....
Here's an answer that relies on very basic functions within awk so it should be portable between unices.
ls -1 | awk '{print rand()*100, $0}' | sort -n | awk '{print $2}'
EDIT:
ephemient makes a good point that the above is not space-safe. Here's a version that is:
ls -1 | awk '{print rand()*100, $0}' | sort -n | sed 's/[0-9\.]* //'
If you have GNU coreutils, you can use shuf:
while read -d '' f
do
# some stuff with $f
done < <(shuf -ze *)
This will work with files with spaces or newlines in their names.
Off-topic Edit:
To illustrate SiegeX's point in the comment:
$ a=42; echo "Don't Panic" | while read line; do echo $line; echo $a; a=0; echo $a; done; echo $a
Don't Panic
42
0
42
$ a=42; while read line; do echo $line; echo $a; a=0; echo $a; done < <(echo "Don't Panic"); echo $a
Don't Panic
42
0
0
The pipe causes the while to be executed in a subshell and so changes to variables in the child don't flow back to the parent.
Here's a solution with standard unix commands:
for i in $(ls); do echo $RANDOM-$i; done | sort | cut -d- -f 2-
Here's a Python solution, if its available on your system
import glob
import random
files = glob.glob("*.txt")
if files:
for file in random.shuffle(files):
print file