I've got a program (prog) that takes a lot of time to produce its output.
I need to check if the output contains string1 and not string2.
Right now I do the following. It invokes prog 2 times.
if prog | grep -q 'some string' &&
! prog | grep -q 'another string'; then
echo 'OK'
else
echo 'Not OK'
fi
Is there are way to do this with only 1 invocation of prog?
Any solution involving awk or sed will do as well.
UPDATE
What I was hoping for was a one liner—that my mind cannot see—to do the trick, having GNU coreutils at my disposal.
I don't think it is possible with grep without executing prog twice, or better, saving it's output into a temporary file.
I would recommend to use awk:
awk '/string1/{a=1}/string2/{b=1}END{exit !a && b}'
You can use it in the shell script like:
if prog | awk '/string1/{a=1}/string2/{b=1}END{exit !a && b}' ; then
echo "ok"
else
echo "not ok"
fi
Note: awk treats 0 as false and 1 as true. The shell treats 0 as true and 1 as false. To reflect that we return !a && b instead of the a && !b, which might look more reasonable in the first place.
With sed, to output line that meet the requirements :
prog | sed -n ':a;$!N;s/\n//;ta;/string1/{/string2/!p}'
Update :
To test the sed command (GNU sed) :
prog | sed -n ':a;$!N;s/\n//;ta;/string1/{/string2/!{q};/string2/{q 1}}' && echo "ok" || echo "nok"
For the problem by using grep command :
prog | grep -E 'string1|string2' | tr -d "\n" | grep -v string2 | grep string1 && echo "OK" || echo "NOT OK"
Related
I wrote what I thought was a quick script I could run on a bunch of machines. Instead it print what looks like might be directory contents in a recursive search:
version=$(mysql Varnish -B --skip-column-names -e "SELECT value FROM sys_param WHERE param='PatchLevel'" | sed -n 's/^.*\([0-9]\.[0-9]*\).*$/\1/p')
if [[ $(echo "if($version == 6.10) { print 1; } else { print 0; }" | bc) -eq 1 ]]; then
status=$(dpkg-query -l | awk '{print $2}' | grep 'sg-status-polling');
cons=$(dpkg-query -l | awk '{print $2}' | grep 'sg-consolidated-poller');
if [[ "$status" != "" && "$cons" != "" ]]; then
echo "about to change /var/www/Varnish/lib/Extra/SG/ObjectPoller2.pm"; echo;
cp /var/www/Varnish/lib/Extra/SG/ObjectPoller2.pm /var/www/Varnish/lib/Extra/SG/ObjectPoller2.pm.bkup;
sed -ir '184s!\x91\x93!\x91\x27--timeout=35\x27\x93!' /var/www/Varnish/lib/Extra/SG/ObjectPoller2.pm;
sed -n 183,185p /var/www/Varnish/lib/Extra/SG/ObjectPoller2.pm; echo;
else
echo "packages not found. Assumed to be not applicable";
fi
else
echo "This is 4.$version, skipping";
fi
The script is supposed to make sure Varnish is version 4.6.10 and has 2 custom .deb packages installed (not through apt-get). then makes a backup and edits a single line in a perl module from [] to ['--timeout=35']
it looks like its tripping up on the sed replace one liner.
There are two major problems (minor ones addressed in comments). The first is that you use the decimal code for [] instead of the hexa, so you should use \x5b\x5d instead of \x91\x93. The second problem is that if you do use the proper codes, sed will still interpret those syntactically as []. So you can't escape escaping. Here's what you should call:
sed -ri'.bkup' '184s!\[\]![\x27--timeout=35\x27]!' /var/www/Varnish/lib/Extra/SG/ObjectPoller2.pm
And this will create the backup for you (but you should double check).
I would like to grep for "String = " and when nothing is mentioned...when it's empty...echo with an if then else statement:
I tried something like that:
if grep "String" input.txt | sed -e 's/ String = //g' | egrep -q "^$"
then
<command> > output.txt
else
echo "--" > output.txt
fi
When I use the sed command the line is empty.
Thx in advance!
UPDATE
these are some possible lines in input.txt
String = TextA
String =
String = TextB
You can just use this grep to check for String = in the input file:
if grep -Eq 'String *= *$' input.txt; then
<command> > output.txt
else
echo "--" > output.txt
fi
You might consider using awk:
CMD=$(awk '$1=="String" {print $3}' input.txt)
if [ -n "$CMD" ]
then
CMD="--"
fi
echo "$CMD" > output.txt
some explanations:
awk prints the third column (since there is a space before and after the = sign; that is the 2nd column)
the line if [ -n "$CMD" ] just tests if the variable CMD is empty
From your question, I assume you are having trouble finding the specified lines in the file. The following grep command will find those lines:
egrep 'String[[:space:]]*=[[:space:]]*' input.txt
Adapt it to your if/else statements and it should work.
Or, if you want to specify that there is one space before the "=", do:
egrep 'String[[:space:]]{1}=[[:space:]]*;
I run my safe shell script to make sure a binary is running
to check a binary is running I do following command
pidof prog.bin | awk '{print NF}'
is some system it gives me 0 when binary not running
and
in some systems it gives me NULL(nothing)
I can check the NULL using -z option but why awk command acting this way ??
Instead of pidof you can use:
pgrep -qf prog.bin
And check its exit status.
As per man pgrep:
-f Match against full argument lists. The default is to match against process names.
-q Do not write anything to standard output.
You can use this,
if [ `pidof 'NetworkManager'` ]; then
echo "Running"
else
echo "Not Running"
fi
One way to handle this sort of thing (undefined variables) in awk is like this:
echo hi | awk '{print a}'
compared with:
echo hi | awk '{print a || 0}'
0
One Liner for If else
[[ $(pidof 'NetworkManager') ]] && echo "Running" || echo "Not Running"
Try this:
pidof prog.bin | awk '{ if (NF!=0) print NF }'
Here's some tests with awk and NF:
$ # regular line of input
$ echo foo | awk '{print NF}'
1
$ # empty line
$ echo | awk '{print NF}'
0
$ # a word on input with no newline
$ printf "%s" nonewline | awk '{print NF}'
1
$ # no input, not even a newline
$ printf %s | awk '{print NF}'
# no output from awk
I suspect the pidof case is the last: not even a newline. To force a newline:
echo $(pidof prog) | ...
printf "%s\n" "$(pidof prog)" | ...
I’m trying to validate input by using egrep and regex.Here is the line from script (c-shell):
echo $1 | egrep '^[0-9]+$'
if ($status == 0) then
set numvar = $1
else
echo "Invalid input"
exit 1
endif
If I pipe echo to egrep it works, but it also prints the variable on the screen, and this is something I don't need.
To simply suppress output you can redirect it to the null device.
echo $1 | egrep '^[0-9]+$' >/dev/null
if ($status == 0) then
set numvar = $1
else
echo "Invalid input"
exit 1
endif
You might also want to consider using the -c option to get the count of matches instead of using using the status.
Also, unless you are using csh, the status is stored in $? not in $status
grep has a -q option that suppresses output
So:
egrep -q '^[0-9]+$'
you can use awk
$ echo "1234" | awk '{print $1+0==$1?"ok":"not ok"}'
ok
$ echo "123d4" | awk '{print $1+0==$1?"ok":"not ok"}'
not ok
I have the following data in a Tab delimited file:
_ DATA _
Col1 Col2 Col3 Col4 Col5
blah1 blah2 blah3 4 someotherText
blahA blahZ blahJ 2 someotherText1
blahB blahT blahT 7 someotherText2
blahC blahQ blahL 10 someotherText3
I want to make sure that the data in 4th column of this file is always an integer. I know how to do this in perl
Read each line, Store value of 4th column in a variable
check if that variable is an integer
if above is true, continue the loop
else break out of the loop with message saying file data not correct
But how would I do this in a shell script using standard linux/unix filter? My guess would be to use grep, but I am not sure how?
cut -f4 data | LANG=C grep -q '[^0-9]' && echo invalid
LANG=C for speed
-q to quit at first error in possible long file
If you need to strip the first line then use tail -n+2 or you could get hacky and use:
cut -f4 data | LANG=C sed -n '1b;/[^0-9]/{s/.*/invalid/p;q}'
awk is the tool most naturally suited for parsing by columns:
awk '{if ($4 !~ /^[0-9]+$/) { print "Error! Column 4 is not an integer:"; print $0; exit 1}}' data.txt
As you get more complex with your error detection, you'll probably want to put the awk script in a file and invoke it with awk -f verify.awk data.txt.
Edit: in the form you'd put into verify.awk:
{
if ($4 !~/^[0-9]+$/) {
print "Error! Column 4 is not an integer:"
print $0
exit 1
}
}
Note that I've made awk exit with a non-zero code, so that you can easily check it in your calling script with something like this in bash:
if awk -f verify.awk data.txt; then
# action for success
else
# action for failure
fi
You could use grep, but it doesn't inherently recognize columns. You'd be stuck writing patterns to match the columns.
awk is what you need.
I can't upvote yet, but I would upvote Jefromi's answer if I could.
Sometimes you need it BASH only, because tr, cut & awk behave differently on Linux/Solaris/Aix/BSD/etc:
while read a b c d e ; do [[ "$d" =~ ^[0-9] ]] || echo "$a: $d not a numer" ; done < data
Edited....
#!/bin/bash
isdigit ()
{
[ $# -eq 1 ] || return 0
case $1 in
*[!0-9]*|"") return 0;;
*) return 1;;
esac
}
while read line
do
col=($line)
digit=${col[3]}
if isdigit "$digit"
then
echo "err, no digit $digit"
else
echo "hey, we got a digit $digit"
fi
done
Use this in a script foo.sh and run it like ./foo.sh < data.txt
See tldp.org for more info
Pure Bash:
linenum=1; while read line; do field=($line); if ((linenum>1)); then [[ ! ${field[3]} =~ ^[[:digit:]]+$ ]] && echo "FAIL: line number: ${linenum}, value: '${field[3]}' is not an integer"; fi; ((linenum++)); done < data.txt
To stop at the first error, add a break:
linenum=1; while read line; do field=($line); if ((linenum>1)); then [[ ! ${field[3]} =~ ^[[:digit:]]+$ ]] && echo "FAIL: line number: ${linenum}, value: '${field[3]}' is not an integer" && break; fi; ((linenum++)); done < data.txt
cut -f 4 filename
will return the fourth field of each line to stdout.
Hopefully that's a good start, because it's been a long time since I had to do any major shell scripting.
Mind, this may well not be the most efficient compared to iterating through the file with something like perl.
tail +2 x.x | sort -n -k 4 | head -1 | cut -f 4 | egrep "^[0-9]+$"
if [ "$?" == "0" ]
then
echo "file is ok";
fi
tail +2 gives you all but the first line (since your sample has a header)
sort -n -k 4 sorts the file numerically on the 4th column, letters will rise to the top.
head -1 gives you the first line of the file
cut -f 4 gives you the 4th column, of the first line
egrep "^[0-9]+$" checks if the value is a number (integers in this case).
If egrep finds nothing, $? is 1, otherwise it's 0.
There's also:
if [ `tail +2 x.x | wc -l` == `tail +2 x.x | cut -f 4 | egrep "^[0-9]+$" | wc -l` ] then
echo "file is ok";
fi
This will be faster, requiring two simple scans through the file, but it's not a single pipeline.
#OP, use awk
awk '$4+0<=0{print "not ok";exit}' file