I stumbled upon a shell instruction which looks odd:
ls -a | grep ".qmail-" | grep -v "mail" | grep ".mail" > t ; echo $?
I suspect that the returned value would represent an error. Could anyone confirm this or explain in which circumstances this instruction would be applied ?
The first grep only allows through lines that contain qmail (preceded by any character and followed by a dash, but that is largely immaterial). The second grep strips out lines that contain mail, which means every line passed by the first grep is deleted by the second. There's nothing left for the third one to process, so the file t will always be empty. The value for $? should be 1, failure, since the third grep failed to find any lines that matched its pattern (because it got no lines to process).
It is a mistake.
It is hard to know how to fix it without knowing what it is trying to do.
The bash shell (and most other shells) let users use the output of one command as the input of another. This is accomplished with the | operator which is called a pipe. So the output of of ls -a is fed to grep ".qmail-" and so on. The > operator sends the output of the command to a file, in this case t. So ls -a | grep ".qmail-" | grep -v "mail" | grep ".mail" > t lists the contents of a directory and passes the output through successive filters before finally saving the output to the file t.
The semicolon signals the end of a command and allows multiple bash commands to be entered on a single line.
echo $? prints out the return value of the last executed command, in this case, ls -a | grep ".qmail-" | grep -v "mail" | grep ".mail" > t. By convention, any value besides 0 indicates some sort of error occurred.The Linux Documentation Project gives a list of some common exit codes.
Related
I have a program that continuously runs and is piped into grep and every once in a while grep finds a match. I want my bash script to execute when grep finds the match.
Like this:
program | grep "pattern"
grep outputs -> script.sh executes
If I understand your question correct (the "every once in a while grep finds a match" part), your program produces the positive pattern several times during its longish runtime and you want script.sh run at those points in time .
When you pipe program to grep and watch on greps exit code with &&, then you execute script.sh only once and only at the end of programs runtime.
If my assumptions are correct (that you want script.sh run several times), then you need "if then logic" behind the | symbol. One way to do it is using awk instead of grep:
program | awk '/pattern/ {system("./script.sh")}'
/pattern/ {system("./script.sh")} in single quotes is the awk script
/pattern/ instructs awk to look for lines matching pattern and do the instructions inside the following curly braces
system("./script.sh") inside the curly braces behind the /.../ executes the script.sh in current directory via awks system command
The overall effect is: script.sh is executed as often as there are lines matching pattern in the output of program.
Another way to this is using sed:
program | sed -n '/pattern/ e ./script.sh'
-n prevents sed from printing every line in programs out put
/pattern/ is the filter criteria for lines in programs output: lines must match pattern
e ./script.sh is the action that sed does when a line matches the pattern: it executes your script.sh
Use -q grep parameter and && syntax, like this:
program | grep -q "pattern" && sh script.sh
where:
-q: Runs in quiet mode but exit immediately with zero status if any match is found
&&: is used to chain commands together, such that
the next command is run if and only if the preceding command exited
without errors (or, more accurately, exits with a return code of 0).
Explained here.
sh script.sh: Will run only if "pattern" is found
Equivalent to #masterguru's answer, but perhaps more understandable:
if program | grep -q "pattern"
then
# execute command here
fi
grep returns a zero exit status when it finds a match, so it can be used as a conditional in if.
I want to create a crontab entry for it.
So the solution has to be a one liner as I do not want to write a script and then invoke that script in crontab (with full if else inside a script-file, I can do this easily).
But I want to use some && or || to achieve something like:
cmd1 | cmd2 | cmd3 | wc -l != 0? mail -s "Found xyz" ...
How can this be done?
Try this:
test "$(unreadable | long | list | of | commands | in | one | line)" && mail -s ...
It is useless to count the lines, if you just want to know, if the commands produce any output.
You can use something like this:
cmd1 | cmd2 | cmd3 | grep '^' >/dev/null && mail -s "Found xyz"
With &&, it'll run the final command (mail) if there's at least one line of output. If you want to send mail if there isn't any output, use || instead.
Explanation: the grep command succeeds if it finds any lines matching the given pattern. The pattern ^ matches the beginning of a line... any line. So grep '^' succeeds if there's at least one line (even a blank one). The >/dev/null then discards the output.
(You could also use grep -q instead of the >/dev/null, but then grep will exit after the first match, which may cause SIGPIPE errors for the earlier commands, and possibly weird problems. I consider the >/dev/null method safer when used in a pipeline.)
I am creating a script that parses some files and greps out the necessary information.
I have set up many different variables in arrays that search for different aspects in the files.
e.g. dates, locations, types.
However I wanted to make each of these variables optional which is where I run into an issue.
the syntax of the command would have been simple
grep ${dates} filename | grep ${locations} | grep ${types}
However, due to variables being optional, the above won't work if a variable is unset.
I was trying to find a way to get grep to find anything (i.e. like egrep .* filename)
that way I could set the variables to the proper regex and have the command still run.
unfortunately, when I set the variable to equal '' it freezes, when I set it to '.' it just greps everything from every file in the current directory and when I leave the variable blank it takes the filename as the variable and waits for a filename.
is there anyway that I can set a variable so that grep $variable file outputs the same as cat would?
Many thanks in advance!
To get grep to act like cat use an empty string as a search pattern, i.e. grep "". Therefore to make some of those variables optional, but not have piped greps fail, just quote the variables:
grep "${dates}" filename | grep "${locations}" | grep "${types}"
Demonstration. Search {250,255,260...280} for the digits 5, 2, and 7:
x=5 y=2 z=7 ; seq 250 5 280 | grep "$x" | grep "$y" | grep "$z"
275
Now unset two of the variables, and it still works:
unset x y ; seq 250 5 280 | grep "$x" | grep "$y" | grep "$z"
270
275
If there aren't any $dates in filename then there is nothing to feed the rest of the pipeline.
I think the best way to do it is to grep for each string separately.
If you get a match, then grep for the next string.
Can you post the source file and a target output. From your question it sounds like you just need to use grep -E in conjunction with the | pipe delimiter.
grep -E "${dates}|${locations}|${types}" fileName
The above line should automatically get you every occurrence of the any of the patterns. This is not even a regex yet.
I am monitoring a log file through a shell script and once a string matches i want to exit. i am using a while statement to read the logs file. But the problem is my script never exits it prints the string which i expect but never exits. below is my piece of script
tail -fn0 $TOMCAT_HOME/logs/catalina.out | \
while read line ; do
echo "$line" | grep "Starting ProtocolHandler"
if [ $? = 0 ]
then
exit
fi
done
Tried using grep -q but doesn't work out
Any help will be appreciated
You can just use grep -q:
tail -fn0 $TOMCAT_HOME/logs/catalina.out | grep -q "Starting ProtocolHandler"
It will exit immediately after 1st occurrence of string "Starting ProtocolHandler"
Are you just wanting the line number? What about grep -n? Then you don't even need a script.
-n, --line-number
Prefix each line of output with the line number within its input file.
Couple that with -m to get it to exit after 1 match
-m NUM, --max-count=NUM
Stop reading a file after NUM matching lines. If the input is standard input from a regular file, and NUM matching lines are output, grep ensures that the standard input is positioned to just after the last matching line before exiting, regardless of the presence of trailing context lines. This enables a calling process to resume a search. When grep stops after NUM matching lines, it outputs any trailing context lines. When the -c or --count option is also used, grep does not output a count greater than NUM. When the -v or --invert-match option is also used, grep stops after outputting NUM non-matching lines.
So, you'd have something like
grep -n -m 1 "Starting ProtocolHandler" [filename]
If you want it to exit:
cat [filename] | grep -n -m 1 "Starting ProtocolHandler"
We use some package called Autosys and there are some specific commands of this package. I have a list of variables which i like to pass in one of the Autosys commands as variables one by one.
For example one such variable is var1, using this var1 i would like to launch a command something like this
autosys_showJobHistory.sh var1
Now when I launch the below written command, it gives me the desired output.
echo "var1" | while read line; do autosys_showJobHistory.sh $line | grep 1[1..6]:[0..9][0..9] | grep 24.12.2012 | tail -1 ; done
But if i put the var1 in a file say Test.txt and launch the same command using cat, it gives me nothing. I have the impression that command autosys_showJobHistory.sh does not work in that case.
cat Test.txt | while read line; do autosys_showJobHistory.sh $line | grep 1[1..6]:[0..9][0..9] | grep 24.12.2012 | tail -1 ; done
What I am doing wrong in the second command ?
Wrote all of below, and then noticed your grep statement.
Recall that ksh doesn't support .. as an indicator for 'expand this range of values'. (I assume that's your intent). It's also made ambiguous by your lack of quoting arguments to grep. If you were using syntax that the shell would convert, then you wouldn't really know what reg-exp is being sent to grep. Always better to quote argments, unless you know for sure that you need the unquoted values. Try rewriting as
grep '1[1-6]:[0-9][0-9]' | grep '24.12.2012'
Also, are you deliberately using the 'match any char' operator '.' OR do you want to only match a period char? If you want to only match a period, then you need to escape it like \..
Finally, if any of your files you're processing have been created on a windows machine and then transfered to Unix/Linux, very likely that the line endings (Ctrl-MCtrl-J) (\r\n) are causing you problems. Cleanup your PC based files (or anything that was sent via ftp) with dos2unix file [file2 ...].
If the above doesn't help, You'll have to "divide and conquer" to debug your problem.
When I did the following tests, I got the expected output
$ echo "var1" | while read line ; do print "line=${line}" ; done
line=var1
$ vi Test.txt
$ cat Test.txt
var1
$ cat Test.txt | while read line ; do print "line=${line}" ; done
line=var1
Unrelated to your question, but certain to cause comment is your use of the cat commnad in this context, which will bring you the UUOC award. That can be rewritten as
while read line ; do print "line=${line}" ; done < Test.txt
But to solve your problem, now turn on the shell debugging/trace options, either by changing the top line of the script (the shebang line) like
#!/bin/ksh -vx
Or by using a matched pair to track the status on just these lines, i.e.
set -vx
while read line; do
print -u2 -- "#dbg: Line=${line}XX"
autosys_showJobHistory.sh $line \
| grep 1[1..6]:[0..9][0..9] \
| grep 24.12.2012 \
| tail -1
done < Test.txt
set +vx
I've added an extra debug step, the print -u2 -- .... (u2=stderror, -- closes option processing for print)
Now you can make sure no extra space or tab chars are creeping in, by looking at that output.
They shouldn't matter, as you have left your $line unquoted. As part of your testing, I'd recommend quoting it like "${line}".
Then I'd comment out the tail and the grep lines. You want to see what step is causing this to break, right? So does the autosys_script by itself still produce the intermediate output you're expecting? Then does autosys + 1 grep produce out as expected, +2 greps, + tail? You should be able to easily see where you're loosing your output.
IHTH