I am having trouble greping qstat with a variable
Scenario: script-A my wrapper submits another script-B as a job, inside script-B I am submitting other jobs in a loop(job1,job2..jobx), each of these jobs have different job-names which are stored in a variable $New_OS. I am trying to make my script-B sleep until all (Job1,job2..jobx) are completed.
Script:
Job_status=`qstat | grep "${New_OS}" | wc -l`
echo -e "\n\nJob_status: $Job_status"
while [ $Job_status -ne "0" ];
do
echo -e "Running PostProcessing for $Job_status sampleR, sleeping for 3 minutes..."
sleep 3m
Job_status=`qstat | grep "${New_OS}" | wc -l`
done;
Problem: The variable $Job_status which is supposed to return the number of jobs with a name $New_OS only returns 0 even though the jobs are still running.
Question: Why the grep of $New_OS in qstat is not returning the jobs with $New_OS as their names? I couldn't figure out the problem here, please suggest me a solution or a work around. Thanks in advance.
Finally adding '*' in my grep worked.
Job_status1=`qstat | grep ${New_OS}* | wc -l`
Expected grep to work without that '*', since I am not looking to 'grep -w' for an exact match. I am sharing it just in case if someone ends up in a similar situation. Thanks.
How about using following.
Job_status1=$(qstat | grep -c ${New_OS}*)
Since I don't have qstat in my system so couldn't check, it should work but, let me know how it goes then.
Related
I have a script.
cat /root/test/ddd.sh
#!/bin/bash
s=/root/test/ddd.sh
ps -ef | grep $s | grep -v grep
result=$(ps -ef | grep $s | grep -v grep | wc -l)
echo $result
when i execute it, the result is weird, it shows that there is two line matched.
[root#l2 test]# /root/test/ddd.sh
root 15361 15032 0 09:52 pts/18 00:00:00 /bin/bash /root/test/ddd.sh
2
That's because you're running a subshell. That is, the $(...) piece causes bash to fork, thereby creating two (nearly) identical processes. By identical, I mean basically everything except for process ID, parent process ID, return code from the fork, and I can't think of anything else. But one thing that does remain the same for sure is the command line. Both of them will be "/bin/bash /root/test/ddd.sh". So, inside the result=$(...), there will be exactly one extra process that matches.
You can see this, for example, by removing the | wc -l piece at the end of your $(...), and, to make it more readable, enclose the echo's argument in quotes:
result="$(ps -ef | grep $s | grep -v grep)"
echo "$result"
Here you will see that there are two bashes, and the PPID of one is the PID of the other, showing the parent-child relationship.
Not sure how to explain this but, what I am trying to achieve is this:
- tailing a file and grepping for a patter A
- then I want to pipe into another customGrepFunction where it matches pattern B, and if B matches echo something out. Need the customGrepFunction in order to do some other custom stuff.
The sticky part here is how to make the grepCustomFunction work here.In other words when only patternA matches echo the whole line and when both patterA & patternB match printout something custom:
when I only run:
tail -f file.log | grep patternA
I can see the pattenA rows are being printed/tailed however when I add the customGrepFunction nothing happens.
tail -f file.log | grep patternA | customGrepFunction
And the customGrepFunction should be available globally in my bin folder:
customGrepFunction(){
if grep patternB
then
echo "True"
fi
}
I have this setup however it doesn't do what I need it to do, it only echos True whenever I do Ctrl+C and exit the tailing.
What am I missing here?
Thanks
What's Going Wrong
The code: if grep patternB; then echo "true"; fi
...waits for grep patternB to exit, which will happen only when the input from tail -f file.log | grep patternA hits EOF. Since tail -f waits for new content forever, there will never be an EOF, so your if statement will never complete.
How To Fix It
Don't use grep on the inside of your function. Instead, process content line-by-line and use bash's native regex support:
customGrepFunction() {
while IFS= read -r line; do
if [[ $line =~ patternB ]]; then
echo "True"
fi
done
}
Next, make sure that grep isn't buffering content (if it were, then it would be written to your code only in big chunks, delaying until such a chunk is available). The means to do this varies by implementation, but with GNU grep, it would look like:
tail -f file.log | grep --line-buffered patternA | customGrepFunction
Completely clueless with Bash, but trying to learn to accomplish this single task.
I need a script that can tail the last 50 lines of a logfile, looking for the string "ERROR", and send an email if that condition is true.
I can do bits and pieces manually, but am not able to build something that works completely to stick in a cron. Sendmail works by itself.
tail -n 50 /var/log/unifi-video/recording.log | grep ERROR
works to at least output that. Ideally I'd like an email only if ERROR is found in the last 50 lines, else no action needs to be taken. If anyone can assist with the if/else statement to make this happen, it would be greatly appreciated.
If you are looking to do this in a one liner you could use something like:
[ $(tail -n 50 /var/log/unifi-video/recording.log | grep ERROR | wc -l) -gt 0 ] && yourmailcommand
This just pipes the output of your grep to wc -l which returns how many lines are returned from grep. If that count is greater than 0 then it will execute that bit after the double ampersands. If you want to include an else in the event that no ERROR lines are found you could:
[ $(tail -n 50 /var/log/unifi-video/recording.log | grep ERROR | wc -l) -gt 0 ] && yourmailcommand || dosomethingelseinstead
grep exits with status 0 if a match is found, 1 if not, so something like
tail -n 50 /var/log/unifi-video/recording.log | grep -q ERROR && <sendmail command here>
would do what you want. (The -q option suppresses any output, because you don't care what line actually matched, just that a match was found.)
in my script i need to loop through lines in a file, once i find some specific line i need to save it to variable so later on i can use it outside the loop, i tried the following but it wont' work:
count=0
res=""
python my.py -p 12345 |
while IFS= read -r line
do
count=$((count+1))
if [ "$count" -eq 5 ]; then
res=`echo "$line" | xargs`
fi
done
echo "$res"
it output nothing, i also tried this,
res=""
... in the loop...
res=$res`echo "$line" | xargs`
still nothing. please help. thanks.
Update: Thanks for all the help. here is my final code:
res=python my.py -p 12345 | sed -n '5p' | xargs
for finding a specific line in a file, have you considered using grep?
grep "thing I'm looking for" /path/to/my.file
this will output the lines that match the thing you're looking for. Moreover this can be piped to xargs as in your question.
If you need to look at a particularly numbered line of a file, consider using the head and tail commands (which can also be piped to grep).
cat /path/to/my.file | head -n5 | tail -n1 | grep "thing I'm looking for"
These commands take the first lines specified (in this case, 5 and 1 respectively) and only prints those out. Hopefully this will help you accomplish your task.
Happy coding! Leave a comment if you have any questions.
I have this script:
#!/bin/bash
CLASSPATH="/blah/libs/*:/blah/more/libs"
CMD="java -cp $CLASSPATH MainClass"
ALREADY_RUNNING_PID=`ps -ef --no-headers | grep $CMD | grep -v grep | awk '{print $2}'`
if [ "$ALREADY_RUNNING_PID" ]; then
echo "Already running"
exit 1
fi
$CMD &
problem is it doesnt work due to the asterisk in the CMD variable. how can i tell grep to see the variable value as it is? Any solution? It is mandatory that grep is fed through the variable.
Thanks.
Since you are not using regular expressions you can use fgrep $CMD instead of grep
The problem is not grep, it's
CLASSPATH="/blah/libs/*:/blah/more/libs"
If you do
echo $CLASSPATH
you should see that your shell has expanded the * to all files in that directory. To remedy this, just use single quotes to prevent globbing:
CLASSPATH='/blah/libs/*:/blah/more/libs'
Totally unrelated to your specific grep problem, but jps will report on running Java processes and possibly make your grepping easier since you'd most likely have to just do:
jps | grep MainClass
(or something similar)