Perhaps BASH differences? Worked fine in old server, not working in new.
It never echos "made it" in the get_running_palaces() function but instead outputs
comm: /dev/fd/63: No such file or directory
comm: /dev/fd/63: No such file or directory
#!/bin/bash
TYPE=$1
get_palaces(){
for PALACE in $(ls -trI shared /home | sort); do
if [ -d "/home/$PALACE/palace" ]; then
echo $PALACE
fi
done
}
# comm -12 file1 file2 Print only lines present in both file1 and file2.
# comm -3 file1 file2 Print lines in file1 not in file2, and vice vers
get_running_palaces(){
echo "made it";
PSFRONT_A=$(ps ax | grep '[p]sfront -p .* -r /home/.*/palace ' | sed 's| *\([0-9]*\).*/home/\(.*\)/palace.*$|\2|' | uniq | sort)
PSERVER_A=$(ps ax | grep '[p]server.* -f /home/.*/palace/psdata/pserver.conf ' | sed 's| *\([0-9]*\).*/home/\(.*\)/palace.*$|\2|' | sort)
ERRORS=$(comm -3 <(echo "${PSERVER_A[*]}") <(echo "${PSFRONT_A[*]}"))
if [ ! -z "$ERRORS" ]; then
comm -3 <(echo "${PSERVER_A[*]}") <(echo "${ERRORS[*]}")
else
echo "$PSERVER_A"
fi
}
case "$TYPE" in
online)
KNOWN_PALACES=$(get_palaces)
ERROR_LESS=$(get_running_palaces)
ONLINE=$(comm -12 <(echo "${KNOWN_PALACES[*]}") <(echo "${ERROR_LESS[*]}"))
[ ! -z "$ONLINE" ] && echo "$ONLINE"
;;
offline)
KNOWN_PALACES=$(get_palaces | sort)
ERROR_LESS=$(get_running_palaces)
OFFLINE=$(comm -3 <(echo "${KNOWN_PALACES[*]}") <(echo "${ERROR_LESS[*]}"))
[ ! -z "$OFFLINE" ] && echo "$OFFLINE"
;;
*)
get_palaces
;;
esac
exit 0;
Information:
New server:
uname -a
Linux www.ipalaces.org 2.6.32-274.7.1.el5.028stab095.1 #1 SMP Mon Oct 24 20:49:24 MSD 2011 x86_64 GNU/Linux
lsb_release -rd
-bash: lsb_release: command not found
bash --version
GNU bash, version 4.1.5(1)-release (x86_64-pc-linux-gnu)
Old server:
uname -a
Linux ipalaces.org 2.6.32-5-686 #1 SMP Mon Jan 16 16:04:25 UTC 2012 i686 GNU/Linux
lsb_release -rd
Description: Debian GNU/Linux 6.0.4 (squeeze)
Release: 6.0.4
bash --version
GNU bash, version 4.1.5(1)-release (i486-pc-linux-gnu)
Process substitution requires /dev/fd/* on Linux (how it's implemented varies on how Bash is built, I think). Maybe you have a screwed up /dev/ structure at the point where this script is running? Stuff like that happens.
I've seen boot-time bash scripts fail from trying to generate a here document, which required /tmp which wasn't mounted yet (and would come from tmpfs later, so there is no such directory in the root volume or anywhere else).
Does process substitution work at all on that system? I mean if you log in to a system that is up and running, can you do things like
diff <(echo "a") <(echo "b")
?
If that doesn't work, you either have to fix /dev, or change how Bash is built (get it to uses fifos for process substitution) or just change your script not to rely on process substitution.
If you cannot figure out how to enable process substitution in Bash on the new server, perhaps you should refactor the script to use a more traditional processing model. Basically, that boils down to using temporary files.
ps ax |
grep '[p]sfront -p .* -r /home/.*/palace ' |
sed 's| *\([0-9]*\).*/home/\(.*\)/palace.*$|\2|' |
uniq | sort >/tmp/PSFRONT_A
ps ax |
grep '[p]server.* -f /home/.*/palace/psdata/pserver.conf ' |
sed 's| *\([0-9]*\).*/home/\(.*\)/palace.*$|\2|' |
sort >/tmp/PSERVER_A
ERRORS=$(comm -3 /tmp/PSERVER_A /tmp/PSFRONT_A)
rm /tmp/PSERVER_A /tmp/PSFRONT_A
Incidentally, this is completely POSIX compatible, so you could change the shebang line to #!/bin/sh while you are at it.
You should simplify the grep | sed and refactor the recurring functionality; also, proper use of temporary files calls for the use of a trap to remove the temporary files even if the script is interrupted by a signal midway through.
t=`mktemp -t -d palaces.XXXXXXXX` || exit 127
trap 'rm -rf $t' 0
trap 'exit 126' 1 2 3 5 15
psg () {
local re
re=$1
ps ax |
sed -n "\\%$re%"'s| *\([0-9]*\).*/home/\(.*\)/palace.*$|\2|p'
}
psg '[p]sfront -p .* -r /home/.*/palace ' |
uniq | sort >$t/PSFRONT_A
psg '[p]server.* -f /home/.*/palace/psdata/pserver\.conf ' |
sort >$t/PSERVER_A
comm -3 $t/PSERVER_A $t/PSFRONT_A >$t/ERRORS
if [ -s $t/ERRORS ]; then
comm -3 $t/PSERVER_A $t/ERRORS
else
cat $t/PSERVER_A
fi
The rest of the script can be adapted accordingly.
Related
I have 3 different files that I want to compare
words_freq
words_freq_deduped
words_freq_alpha
For each file, I run a command like so, which I iterate on constantly to compare the results.
For example, I would do this:
$ cat words_freq | grep -v '[soe]'
$ cat words_freq_deduped | grep -v '[soe]'
$ cat words_freq_alpha | grep -v '[soe]'
and then review the results, and then do it again, with an additional filter
$ cat words_freq | grep -v '[soe]' | grep a | grep r | head -n20
a
$ cat words_freq_deduped | grep -v '[soe]' | grep a | grep r | head -n20
b
$ cat words_freq_alpha | grep -v '[soe]' | grep a | grep r | head -n20
c
This continues on until I've analyzed my data.
I would like to write a script that could take the piped portions, and pass it to each of these files, as I iterate on the grep/head portions of the command.
e.g. The following would dump the results of running the 3 commands above AND also compare the 3 results, and dump additional calculations on them
$ myScript | grep -v '[soe]' | grep a | grep r | head -n20
the letters were in all 3 runs, and it took 5 seconds
a
b
c
How can I do this using bash/python or zsh for the myScript part?
EDIT: After asking the question, it occurred to me that I could use eval to do it, like so, which I've added as an answer as well
The following approach allows me to process multiple files by using eval, which I know is frowned upon - any other suggestions are greatly appreciated!
$ myScript "grep -v '[soe]' | grep a | grep r | head -n20"
myScript
#!/usr/bin/env bash
function doIt(){
FILE=$1
CMD="cat $1 | $2"
echo processing file "$FILE"
eval "$CMD"
echo
}
doIt words_freq "$#"
doIt words_freq_deduped "$#"
doIt words_freq_alpha "$#"
You can't avoid your shell from running pipes itself, so using it like that isn't very practical - you'd need to either quote everything and then eval it, which would make it hard to pass arguments with spaces, or quote every pipe, which you can then eval, making it so you have to quote every pipe. But yeah, these solutions are kinda hacky.
I'd suggest doing one of these two:
Keep your editor open, and put whatever you want to run inside the doIt function itself before you run it. Then run it in your shell without any arguments:
#!/usr/bin/env bash
doIt() {
# grep -v '[soe]' < "$1"
grep -v '[soe]' < "$1" | grep a | grep r | head -n20
}
doIt words_freq
doIt words_freq_deduped
doIt words_freq_alpha
Or, you could always use a "for" in your shell, which you can use Ctrl+r to find in your history when you want to use:
$ for f in words_freq*; do grep -v '[soe]' < "$f" | grep a | grep r | head -n20; done
But if you really want your approach, I tried to make it accept spaces, but it ended up being even hackier:
#!/usr/bin/env bash
doIt() {
local FILE=$1
shift
echo processing file "$FILE"
local args=()
for n in $(seq 1 $#); do
arg=$1
shift
if [[ $arg == '|' ]]; then
args+=('|')
else
args+=("\"$arg\"")
fi
done
eval "cat '$FILE' | ${args[#]}"
}
doIt words_freq "$#"
doIt words_freq_deduped "$#"
doIt words_freq_alpha "$#"
With this version you can use it like this:
$ ./myScript grep "a a" "|" head -n1
Notice that it need you to quote the |, and that it now handles arguments with spaces.
Not fully understood problem correctly.
I understood you want to write a script without pipes, by including the filtering logic into the script.
And feeding the filtering patterns as arguments.
Here is a gawk script (standard Linux awk).
With one sweep on 3 input files, without piping.
script.awk
BEGIN {
RS="!#!#!#!#!#!#!#";
# set record separator to something unlikely matched, causing each file to be read entirely as a single record
}
$0 !~ excludeRegEx # if file does not match excludeRegEx
&& $0 ~ includeRegEx1 # and match includeRegEx1
&& $0 ~ includeRegEx2 { # and match includeRegEx2
system "head -n20 "FILENAME; # call shell command "head -n20 " on current filename
}
Running script.awk
awk -v excludeRegEx='[soe]' \
-v includeRegEx1='a' \
-v includeRegEx2='r' \
-f script.awk words_freq words_freq_deduped words_freq_alpha
The following approach allows me to process multiple files by using eval, which I know is frowned upon - any other suggestions are greatly appreciated!
$ myScript "grep -v '[soe]' | grep a | grep r | head -n20"
myScript
#!/usr/bin/env bash
function doIt(){
FILE=$1
CMD="cat $1 | $2"
echo processing file "$FILE"
eval "$CMD"
echo
}
doIt words_freq "$#"
doIt words_freq_deduped "$#"
doIt words_freq_alpha "$#"
Using bash 4.3 and root powers, I want to be able to detect one keystroke (any key) from another terminal.
In terminal 1, a background process that writes to a named pipe once one character was read from another tty, say pts/18
read -rsn1 < /dev/pts/18 && echo > ./myfifo &
In terminal 2, an attempt to read a character from the same fifo
read -rsn1 < ./myfifo
It works moderately well, but only after several (3 or 4) keystrokes in pts/18, not the first one.
The mechanism appears to be redundant but it allows to start several background processes with different ttys and redirect to the same named pipe.
Hope you can help me.
One day I've tried to do that.
Wrote script who getting that you want with strace:
#!/bin/bash
# Couple of special symbols
BACKSPACE='\10\33[K'
BACKSPACE_EMPTY='\7'
BACKSLASH='\\'
QUOTE="\'"
DOUBLE_QUOUTE='\"'
LARROW='\10'
RARROW='\33[C'
#APPOSTRO="'\`'"
BACKSPACE_='\\33\[1P'
#LOGDATE=`date '+%Y/%m/%d %H:%M:%S'`
BADBIN='\\33\]'
while read -r line
do
# Avoiding binary symbols
NOBINline=$(echo "${line}" | strings)
# Current `pwd` value for session
CURRENT_PWD_OF_PID=$(readlink -f /proc/${1}/cwd)
# Getting username
USER_OF_PID=$(cat /proc/${1}/environ | strings | tr '.' ' ' | grep USER | awk -F= '{print $2}')
# Not the best but working way to create prompt line
HOSTNAME_OF_PID=`hostname -a`
STR_TO_REMOVE=$(printf "${USER_OF_PID}""#""${HOSTNAME_OF_PID}"":""${CURRENT_PWD_OF_PID}")
# Cut useless symbols from strace output.
parsed_line=$(echo "${NOBINline}" | perl -nale 'print $1 if ~/\"(.*)\"/gi')
if [ "${parsed_line}" == "\n" ]
then
parsed_line="{{ENTER}}"
fi
output_line=''
inchar_line=''
postinchar_line=''
inchar_line=$(printf "${parsed_line}")
if [ "${inchar_line}" == "{{ENTER}}" ]
then
echo ""
else
output_line=$(printf "${output_line}""${inchar_line}")
fi
if [ "${output_line}" != "${CURRENT_PWD_OF_PID}" -a "${output_line}" != "${STR_TO_REMOVE}" -a `echo "${NOBINline}" | grep -c "${BADBIN}"` -eq 0 ]
then
printf "${output_line}"
fi
done < <(sudo strace -e trace=write -s1000 -p $1 2>/dev/stdout)
How to use:
./script.sh <PID_of_mointored_console>
Also I've written the second script (much more ugly than first):
#!/bin/bash
while [ 1 ]
do
# lets and log_start -- names of two my scripts.
for PIDs in `ps awwux | grep [p]ts | grep bash | grep -v lets | grep -v log_start | grep -v grep | awk '{print $2}'`
do
PTS=$(ps awwux | grep "${PIDs}" | grep -v grep | awk '{print $7}')
if [ -f /tmp/bash_log.${PIDs} ]
then
sleep 1
else
touch /tmp/bash_log.${PIDs}
# lets.sh -- first script.
/bin/bash /root/lets.sh $PIDs >> /tmp/bash_log.${PIDs} &
fi
# tr -cd '\11\12\15\40-\176' < /tmp/x_bash_log.${PIDs} | sed 's/\]0;//g' | sed 's/\[C//g' > /tmp/bash_log.${PIDs}
done
for IFEMPTY in `find /tmp/ -type f -name 'bash_log*' -mmin +600`
do
if [ `cat "${IFEMPTY}" | grep -v "\'" | wc -c` -lt 2 ]
then
rm -rf "${IFEMPTY}"
else
sleep 1
fi
done
done
Start it in the background, place near first script.
It will catching all new terminal sessions, and start firstscript.sh <PID> for each, and creating /tmp/bash_log.<PID> file.
How to use:
./monitor.sh &
# Start new terminal
[sahaquiel#sahaquiel-PC ~]$ echo $$
916
[sahaquiel#sahaquiel-PC ~]$ HELO, IM A NEW STRING
bash: HELO,: command not found
[sahaquiel#sahaquiel-PC ~]$ tail -n3 /tmp/bash_log.916
916
''[sahaquiel#sahaquiel-PC ~]$ HELO, IM A NEW STRING
'''[sahaquiel#sahaquiel-PC ~]$ tail -n3 /tmp/bash_log.916
NB. I'm still working with it, created just for fun.
/tmp/bash_log files have awful formatting. I've also tried to log in the file backslashes, Ctrl+R, Ctrl+C and the same, but at now it's looks like not readable enough:
'''[sahaquiel#sahaquiel-PC ~]$ I WILL TYPE BACKSLASH THREE TIMES IN THE END OF THIS STRING 123[BACKSPACE] [Backspace found. Rewritten string]: └────>
'''[sahaquiel#sahaquiel-PC ~]$ I WILL TYPE BACKSLASH THREE TIMES IN THE END OF THIS STRING 12[BACKSPACE] [Backspace found. Rewritten string]: └────>
'''[sahaquiel#sahaquiel-PC ~]$ I WILL TYPE BACKSLASH THREE TIMES IN THE END OF THIS STRING 1[BACKSPACE] [Backspace found. Rewritten string]: └────>
'''[sahaquiel#sahaquiel-PC ~]$ I WILL TYPE BACKSLASH THREE TIMES IN THE END OF THIS STRING
'''[sahaquiel#sahaquiel-PC ~]$ NOW I I[BACKSPACE] [Backspace found. Rewritten string]: └────>
'''[sahaquiel#sahaquiel-PC ~]$ NOW I WILL PRESS \"LEFT ARROW\" THREE TIMES[LARROW] [Left arrow found.]: └────>
'''[sahaquiel#sahaquiel-PC ~]$ NOW I WILL PRESS \"LEFT ARROW\" THREE TIME[LARROW]S[LARROW] [Left arrow found.]: └────>
'''[sahaquiel#sahaquiel-PC ~]$ NOW I WILL PRESS \"LEFT ARROW\" THREE TIM[LARROW]ES[LARROW] [Left arrow found.]: └────>
'''[sahaquiel#sahaquiel-PC ~]$ NOW I WILL PRESS \"LEFT ARROW\" THREE TI[LARROW]MES
You can change it somehow you want.
There are a lot of variables, I wanted to use them later for extended strings parse way, but it doesn't work now correctly.
P.S. If someone really interested in that, can check this project on my Github (check profile for link)
I want to search a number of files "in a bash loop".
Suppose I exec an ack search in a bash loop:
#!/bin/bash
seq 3 | while read i
do
test=`ack root /etc/passwd`
echo $test
done
prints one empty line.
Where as
#!/bin/bash
seq 3 | while read i
do
# test=`ack root /etc/passwd`
echo $test
done
prints 3 empty lines.
If I ran just that one command from the bash it works:
ack root /etc/passwd
This also works:
$ test=`ack root /etc/passwd`
$ echo $test
I think ack somehow breaks the loop.
Here's "the origin of the problem":
ls input/* | while read scan
do
temp=`basename "$scan"`
baseName=${temp%.*}
extension=${temp#*.}
# OCR:
abbyyocr11 -rl Russian -if "$scan" -f TextUnicodeDefaults -of "temp/$baseName.txt"
# get data:
firstName=` ack '^Имя\s+(.+)' --output='$1' "temp/$baseName.txt" | sed 's/ //g'`
middleName=`ack '^Отчество\s+(.+)' --output='$1' "temp/$baseName.txt" | sed 's/ //g'`
lastName=` ack '^Фамилия\s+(.+)' --output='$1' "temp/$baseName.txt" | sed 's/ //g'`
# copy the file with a meaningful name:
cp --backup=numbered "$scan" "output/$lastName$firstName$middleName.$extension"
done
Edit
Turns out --nofilter option solves it. According to --help message it forces ack treat stdin as tty as opposed to pipe. I wonder what this means.
It looks like it doesn't like a pipe on stdin for some reason.
I don't know why it does that, but a quick and easy fix would be:
seq 3 | while read i
do
test=`</dev/null ack root /etc/passwd`
echo $test
done
or in your case
# get data:
{
#....
} </dev/null
I have isolated the problem to the below code snippet:
Notice below that null string gets assigned to LATEST_FILE_NAME='' when the script is run using ksh; but the script assigns the value to variable $LATEST_FILE_NAME correctly when run using sh. This in turn affects the value of $FILE_LIST_COUNT.
But as the script is in KornShell (ksh), I am not sure what might be causing the issue.
When I comment out the tee command in the below line, the ksh script works fine and correctly assigns the value to variable $LATEST_FILE_NAME.
(cd $SOURCE_FILE_PATH; ls *.txt 2>/dev/null) | sort -r > ${SOURCE_FILE_PATH}/${FILE_LIST} | tee -a $LOG_FILE_PATH
Kindly consider:
1. Source Code: script.sh
#!/usr/bin/ksh
set -vx # Enable debugging
SCRIPTLOGSDIR=/some/path/Scripts/TEST/shell_issue
SOURCE_FILE_PATH=/some/path/Scripts/TEST/shell_issue
# Log file
Timestamp=`date +%Y%m%d%H%M`
LOG_FILENAME="TEST_LOGS_${Timestamp}.log"
LOG_FILE_PATH="${SCRIPTLOGSDIR}/${LOG_FILENAME}"
## Temporary files
FILE_LIST=FILE_LIST.temp #Will store all extract filenames
FILE_LIST_COUNT=0 # Stores total number of files
getFileListDetails(){
rm -f $SOURCE_FILE_PATH/$FILE_LIST 2>&1 | tee -a $LOG_FILE_PATH
# Get list of all files, Sort in reverse order, and store names of the files line-wise. If no files are found, error is muted.
(cd $SOURCE_FILE_PATH; ls *.txt 2>/dev/null) | sort -r > ${SOURCE_FILE_PATH}/${FILE_LIST} | tee -a $LOG_FILE_PATH
if [[ ! -f $SOURCE_FILE_PATH/$FILE_LIST ]]; then
echo "FATAL ERROR - Could not create a temp file for file list.";exit 1;
fi
LATEST_FILE_NAME="$(cd $SOURCE_FILE_PATH; head -1 $FILE_LIST)";
FILE_LIST_COUNT="$(cat $SOURCE_FILE_PATH/$FILE_LIST | wc -l)";
}
getFileListDetails;
exit 0;
2. Output when using shell sh script.sh:
+ getFileListDetails
+ rm -f /some/path/Scripts/TEST/shell_issue/FILE_LIST.temp
+ tee -a /some/path/Scripts/TEST/shell_issue/TEST_LOGS_201304300506.log
+ cd /some/path/Scripts/TEST/shell_issue
+ sort -r
+ tee -a /some/path/Scripts/TEST/shell_issue/TEST_LOGS_201304300506.log
+ ls 1.txt 2.txt 3.txt
+ [[ ! -f /some/path/Scripts/TEST/shell_issue/FILE_LIST.temp ]]
cd $SOURCE_FILE_PATH; head -1 $FILE_LIST
++ cd /some/path/Scripts/TEST/shell_issue
++ head -1 FILE_LIST.temp
+ LATEST_FILE_NAME=3.txt
cat $SOURCE_FILE_PATH/$FILE_LIST | wc -l
++ cat /some/path/Scripts/TEST/shell_issue/FILE_LIST.temp
++ wc -l
+ FILE_LIST_COUNT=3
exit 0;
+ exit 0
3. Output when using ksh ksh script.sh:
+ getFileListDetails
+ tee -a /some/path/Scripts/TEST/shell_issue/TEST_LOGS_201304300507.log
+ rm -f /some/path/Scripts/TEST/shell_issue/FILE_LIST.temp
+ 2>& 1
+ tee -a /some/path/Scripts/TEST/shell_issue/TEST_LOGS_201304300507.log
+ sort -r
+ 1> /some/path/Scripts/TEST/shell_issue/FILE_LIST.temp
+ cd /some/path/Scripts/TEST/shell_issue
+ ls 1.txt 2.txt 3.txt
+ 2> /dev/null
+ [[ ! -f /some/path/Scripts/TEST/shell_issue/FILE_LIST.temp ]]
+ cd /some/path/Scripts/TEST/shell_issue
+ head -1 FILE_LIST.temp
+ LATEST_FILE_NAME=''
+ wc -l
+ cat /some/path/Scripts/TEST/shell_issue/FILE_LIST.temp
+ FILE_LIST_COUNT=0
exit 0;+ exit 0
OK, here goes...this is a tricky and subtle one. The answer lies in how pipelines are implemented. POSIX states that
If the pipeline is not in the background (see Asynchronous Lists), the shell shall wait for the last command specified in the pipeline to complete, and may also wait for all commands to complete.)
Notice the keyword may. Many shells implement this in a way that all commands need to complete, e.g. see the bash manpage:
The shell waits for all commands in the pipeline to terminate before returning a value.
Notice the wording in the ksh manpage:
Each command, except possibly the last, is run as a separate process; the shell waits for the last command to terminate.
In your example, the last command is the tee command. Since there is no input to tee because you redirect stdout to ${SOURCE_FILE_PATH}/${FILE_LIST} in the command before, it immediately exits. Oversimplified speaking, the tee is faster than the earlier redirection, which means that your file is probably not finished writing to by the time you are reading from it. You can test this (this is not a fix!) by adding a sleep at the end of the whole command:
$ ksh -c 'ls /tmp/* | sort -r > /tmp/foo.txt | tee /tmp/bar.txt; echo "[$(head -n 1 /tmp/foo.txt)]"'
[]
$ ksh -c 'ls /tmp/* | sort -r > /tmp/foo.txt | tee /tmp/bar.txt; sleep 0.1; echo "[$(head -n 1 /tmp/foo.txt)]"'
[/tmp/sess_vo93c7h7jp2a49tvmo7lbn6r63]
$ bash -c 'ls /tmp/* | sort -r > /tmp/foo.txt | tee /tmp/bar.txt; echo "[$(head -n 1 /tmp/foo.txt)]"'
[/tmp/sess_vo93c7h7jp2a49tvmo7lbn6r63]
That being said, here are a few other things to consider:
Always quote your variables, especially when dealing with files, to avoid problems with globbing, word splitting (if your path contains spaces) etc.:
do_something "${this_is_my_file}"
head -1 is deprecated, use head -n 1
If you only have one command on a line, the ending semicolon ; is superfluous...just skip it
LATEST_FILE_NAME="$(cd $SOURCE_FILE_PATH; head -1 $FILE_LIST)"
No need to cd into the directory first, just specify the whole path as argument to head:
LATEST_FILE_NAME="$(head -n 1 "${SOURCE_FILE_PATH}/${FILE_LIST}")"
FILE_LIST_COUNT="$(cat $SOURCE_FILE_PATH/$FILE_LIST | wc -l)"
This is called Useless Use Of Cat because the cat is not needed - wc can deal with files. You probably used it because the output of wc -l myfile includes the filename, but you can use e.g. FILE_LIST_COUNT="$(wc -l < "${SOURCE_FILE_PATH}/${FILE_LIST}")" instead.
Furthermore, you will want to read Why you shouldn't parse the output of ls(1) and How can I get the newest (or oldest) file from a directory?.
First off, I'm new to this. I have some experience with windows scripting and apple script but not much with bash. What I'm trying to do is grab the PID and %CPU of a specific process. then compare the %CPU against a set number, and if it's higher, kill the process. I feel like I'm close, but now I'm getting the following error:
[[: 0.0: syntax error: invalid arithmetic operator (error token is ".0")
what am I doing wrong? here's my code so far:
#!/bin/bash
declare -i app_pid
declare -i app_cpu
declare -i cpu_limit
app_name="top"
cpu_limit="50"
app_pid=`ps aux | grep $app_name | grep -v grep | awk {'print $2'}`
app_cpu=`ps aux | grep $app_name | grep -v grep | awk {'print $3'}`
if [[ ! $app_cpu -gt $cpu_limit ]]; then
echo "crap"
else
echo "we're good"
fi
Obviously I'm going to replace the echos in the if/then statement but it's acting as if the statement is true regardless of what the cpu load actually is (I tested this by changing the -gt to -lt and it still echoed "crap"
Thank you for all the help. Oh, and this is on a OS X 10.7 if that is important.
I recommend taking a look at the facilities of ps to avoid multiple horrible things you do.
On my system (ps from procps on linux, GNU awk) I would do this:
ps -C "$app-name" -o pid=,pcpu= |
awk --assign maxcpu="$cpu_limit" '$2>maxcpu {print "crappy pid",$1}'
The problem is that bash can't handle decimals. You can just multiply them by 100 and work with plain integers instead:
#!/bin/bash
declare -i app_pid
declare -i app_cpu
declare -i cpu_limit
app_name="top"
cpu_limit="5000"
app_pid=`ps aux | grep $app_name | grep -v grep | awk {'print $2'}`
app_cpu=`ps aux | grep $app_name | grep -v grep | awk {'print $3*100'}`
if [[ $app_cpu -gt $cpu_limit ]]; then
echo "crap"
else
echo "we're good"
fi
Keep in mind that CPU percentage is a suboptimal measurement of application health. If you have two processes running infinite loops on a single core system, no other application of the same priority will ever go over 33%, even if they're trashing around.
#!/bin/sh
PROCESS="java"
PID=`pgrep $PROCESS | tail -n 1`
CPU=`top -b -p $PID -n 1 | tail -n 1 | awk '{print $9}'`
echo $CPU
I came up with this, using top and bc.
Use it by passing in ex: ./script apache2 50 # max 50%
If there are many PIDs matching your program argument, only one will be calculated, based on how top lists them. I could have extended the script by catching them all and avergaing the percentage or something, but this will have to do.
You can also pass in a number, ./script.sh 12345 50, which will force it to use an exact PID.
#!/bin/bash
# 1: ['command\ name' or PID number(,s)] 2: MAX_CPU_PERCENT
[[ $# -ne 2 ]] && exit 1
PID_NAMES=$1
# get all PIDS as nn,nn,nn
if [[ ! "$PID_NAMES" =~ ^[0-9,]+$ ]] ; then
PIDS=$(pgrep -d ',' -x $PID_NAMES)
else
PIDS=$PID_NAMES
fi
# echo "$PIDS $MAX_CPU"
MAX_CPU="$2"
MAX_CPU="$(echo "($MAX_CPU+0.5)/1" | bc)"
LOOP=1
while [[ $LOOP -eq 1 ]] ; do
sleep 0.3s
# Depending on your 'top' version and OS you might have
# to change head and tail line-numbers
LINE="$(top -b -d 0 -n 1 -p $PIDS | head -n 8 \
| tail -n 1 | sed -r 's/[ ]+/,/g' | \
sed -r 's/^\,|\,$//')"
# If multiple processes in $PIDS, $LINE will only match\
# the most active process
CURR_PID=$(echo "$LINE" | cut -d ',' -f 1)
# calculate cpu limits
CURR_CPU_FLOAT=$(echo "$LINE"| cut -d ',' -f 9)
CURR_CPU=$(echo "($CURR_CPU_FLOAT+0.5)/1" | bc)
echo "PID $CURR_PID: $CURR_CPU""%"
if [[ $CURR_CPU -ge $MAX_CPU ]] ; then
echo "PID $CURR_PID ($PID_NAMES) went over $MAX_CPU""%"
echo "[[ $CURR_CPU""% -ge $MAX_CPU""% ]]"
LOOP=0
break
fi
done
echo "Stopped"
Erik, I used a modified version of your code to create a new script that does something similar. Hope you don't mind it.
A bash script to get the CPU usage by process
usage:
nohup ./check_proc bwengine 70 &
bwegnine is the process name we want to monitor 70 is to log only when the process is using over 70% of the CPU.
Check the logs at: /var/log/check_procs.log
The output should be like:
DATE | TOTAL CPU | CPU USAGE | Process details
Example:
03/12/14 17:11 |20.99|98| ProdPROXY-ProdProxyPA.tra
03/12/14 17:11 |20.99|100| ProdPROXY-ProdProxyPA.tra
Link to the full blog:
http://felipeferreira.net/?p=1453
It is also useful to have app_user information available to test whether the current user has the rights to kill/modify the running process. This information can be obtained along with the needed app_pid and app_cpu by using read eliminating the need for awk or any other 3rd party parser:
read app_user app_pid tmp_cpu stuff <<< \
$( ps aux | grep "$app_name" | grep -v "grep\|defunct\|${0##*/}" )
You can then get your app_cpu * 100 with:
app_cpu=$((${tmp_cpu%.*} * 100))
Note: Including defunct and ${0##*/} in grep -v prevents against multiple processes matching $app_name.
I use top to check some details. It provides a few more details like CPU time.
On Linux this would be:
top -b -n 1 | grep $app_name
On Mac, with its BSD version of top:
top -l 1 | grep $app_name