I'm trying to get the following line to work
WERRORS=`echo $VALPG | grep -q -s -o -m 1 '\<[0-9]* Errors'`
What I want is that the result of grep go into WERRORS variable but not echo in the terminal.
So i use -q, but then WERRORS is empty
If grep sends any error messages, they go to the error output, which is not captured by the backticks. If you need this output in a variable (which is somewhat problematic, because it's often localized), redirect it using 2>&1:
WERRORS=`echo $VALPG | grep -s -o -m 1 '\<[0-9]* Errors' 2>&1`
WERRORS=`echo $VALPG | grep -s -o -m 1 '\<[0-9]* Errors'`
kent$ val=abcpc
kent$ a=$(echo $val|grep -o -m 1 -s 'pc')
kent$ echo $a
pc
Related
I have a bash/zsh command with multiple pipes | that fails when using set -o pipefail. For simplicity assume the command is
set -o pipefail; echo "123456" | head -c2 | grep 5 | cat
How do I quickly find out which command is the first to fail and why? I know I can check the exit code, but that doesn't show which part of the pipeline failed first.
Is there something simpler than the rather verbose check of building up the pipeline one by one and checking for the first failing exit code?
Edit: I removed the contrived code example I made up as it confused people about my purpose of asking. The actual command that prompted this question was:
zstdcat metadata.tsv.zst | \
tsv-summarize -H --group-by Nextclade_pango --count | \
tsv-filter -H --ge 'count:2' | \
tsv-select -H -f1 >open_lineages.txt
In bash, use echo "${PIPESTATUS[#]}" right after the command to get the exit status for each component in a space separated list:
#!/bin/bash
$ set -o pipefail; echo "123456" | head -c2 | grep 5 | cat
$ echo ${PIPESTATUS[#]}
0 0 1 0
Beware zsh users, you need to use the lower case pipestatus instead:
#!/bin/zsh
$ set -o pipefail; echo "123456" | head -c2 | grep 5 | cat
$ echo $pipestatus
0 0 1 0
In fish you can also simply use echo $pipestatus for the same output.
${PIPESTATUS[#]} right after is the answer you were looking for. However, I want to advise on the first example. It's a good habit to anticipate error, so instead of testing after you should have check the path prior everything.
if [ -d "/nonexistent_directory" ]; then
# here pipe shouldn't fail to grep
# ...unless there's something wrong with "foo"
# ...but "grep" may be a failure if the pattern isn't found
if ! ls -1 "/nonexistent_directory" | grep 'foo' ; then
echo "The command 'grep foo' failed."
# else
# echo "The pipeline succeeded."
fi
else
echo "The command 'ls /nonexistent_directory' failed."
fi
Whenever possible, avoid greping ls output in script, that' fragile...
For a half-finished script that already uses the output of a program I also need the name and the parameters of the program that was used to pipe to my script.
So I run it like this:
yay something | ./myscript
Now I need to store "yay something" into a variable.
There is a way to to get previous runned commands or the current one by using set -o history -o histexpand and echo !! or echo $0 but that doesn't include what I wrote right before the pipe.
Maybe you would suggest to pass the name of the program and it's parameter to my script as parameters and then run it there but I don't want this (pass a command as an argument to bash script).
UPDATED SOLUTION (old below):
#!/bin/bash -i
#get processes
processes=$(> >(ps -f))
echo beginning:
echo "$processes"
#filter bin/bash -i
pac=$(echo "$processes" | sed '1,/bin\/bash -i/!d')
pac=$(echo "$pac" | tail -2 | head -1)
#kill
delete=$(echo $pac | grep -oP "(?<=$USER\s)\w+")
pac=$(echo "$pac" | grep -o -P '(?<=00:00:00).*(?=)')
echo "$delete"
kill -9 "$delete"
#print
echo " "
echo end:
echo "${pac:1}"
Note: When you use echo, man or cat then $pac will be empty.
OLD Text:
Thanks to Charles for his enormous effort and his link that finally led me to processes=$(> >(ps -f)).
Here a working example. You can e.g. use it with vi test | ./testprocesses (or nano or package helpers like yay or trizen but it won't work with echo, man nor with cat):
#!/bin/bash -i
#get processes
processes=$(> >(ps -f))
echo beginning:
echo $processes
#filter
pac=$(echo $processes | grep -o -P '(?<=CM).*(?=testprocesses)' | grep -o -P '(?<=D).*(?=testprocesses)' | grep -o -P "(?<=00:00:00).*(?=$USER)")
#kill
delete=$(echo $pac | grep -oP "(?<=$USER\s)\w+")
pac=$(echo $pac | grep -o -P '(?<=00:00:00).*(?=)')
kill -9 $delete
#print
echo " "
echo end:
echo $pac
The kill part is necessary to kill the vi instance else it will still be running and eventually interfer with future executions of the script.
couldn't find this on SO. I ran the following command in the terminal:
>> grep -Rl "curl" ./
and this displays the list of files where the keyword curl occurs. I want to count the number of files. First way I can think of, is to count the number of lines in the output that came in the terminal. How can I do that?
Pipe the result to wc using the -l (line count) switch:
grep -Rl "curl" ./ | wc -l
Putting the comment of EaterOfCode here as an answer.
grep itself also has the -c flag which just returns the count
So the command and output could look like this.
$ grep -Rl "curl" ./ -c
24
EDIT:
Although this answer might be shorter and thus might seem better than the accepted answer (that is using wc). I do not agree with this anymore. I feel like remembering that you can count lines by piping to wc -l is much more useful as you can use it with other programs than grep as well.
Piping to 'wc' could be better IF the last line ends with a newline (I know that in this case, it will)
However, if the last line does not end with a newline 'wc -l' gives back a false result.
For example:
$ echo "asd" | wc -l
Will return 1 and
$ echo -n "asd" | wc -l
Will return 0
So what I often use is grep <anything> -c
$ echo "asd" | grep "^.*$" -c
1
$ echo -n "asd" | grep "^.*$" -c
1
This is closer to reality than what wc -l will return.
"abcd4yyyy" | grep 4 -c gives the count as 1
Should be fairly simple to answer:
Let's say I wanted to execute a command determined by the output of a previous one in Bash:
curl http://website.com 2> /dev/null | grep -i "test" --count | <MY-COMMAND>
What I need: <MY-COMMAND> should only execute if grep had some matches (at least 1).
How can I achieve that?
Also, please feel free to add matching tags, I couldn't come up with any
ifne utility ("run a program if the standard input is not empty") from Jeoy Hess's moreutils package will serve you.
A description of it:
a command that would run the following
command if and only if the standard
input is not empty. I often want this
in crontabs, as in:
find . -name core | ifne mail -s "Core files found" root
Do you need the output of grep to be piped to your command? The answer is simpler if you do not. In that case since grep's return code is success only if it finds a match, you can use && or if:
curl http://website.com 2> /dev/null | grep -q -i "test" && <MY-COMMAND>
if curl http://website.com 2> /dev/null | grep -q -i "test"; then
<MY-COMMAND>
fi
The && operator is a shorthand way of performing an if-else check. It is a short-circuiting operator, which means that the right hand side will only be executed if the left hand side fails.
If you need to pipe the output to your command then you'll need to save the output to a temporary file, test for a match, and then execute your command:
if curl http://website.com 2> /dev/null | grep -i "test" > /tmp/grep.txt; then
<MY-COMMAND> < /tmp/grep.txt
fi
curl http://website.com 2> /dev/null | grep -i "test" && <MY-COMMAND>
From the grep man page: "the exit status is 0 if selected lines are found and 1 otherwise"
The command after && is executed only if the previous command returned exit status 0.
curl http://www.google.com 2>/dev/null | grep window -i -c && echo "this is a success"
I want to count the number of lines output from a command in a bash script. i.e.
COUNT=ls | wc -l
But I also want the script to output the original output from ls. How to get this done? (My actual command is not ls and it has side effects. So I can't run it twice.)
The tee(1) utility may be helpful:
$ ls | tee /dev/tty | wc -l
CHANGES
qpi.doc
qpi.lib
qpi.s
4
info coreutils "tee invocation" includes this following example, which might be more instructive of tee(1)'s power:
wget -O - http://example.com/dvd.iso \
| tee >(sha1sum > dvd.sha1) \
>(md5sum > dvd.md5) \
> dvd.iso
That downloads the file once, sends output through two child processes (as started via bash(1) process substitution) and also tee(1)'s stdout, which is redirected to a file.
ls | tee tmpfile | first command
cat tmpfile | second command
Tee is a good way to do that, but you can make something simpler:
ls > __tmpfile
cat __tmpfile | wc -l
cat __tmpfile
rm __tmpfile