I want to parse the output of docker node ls -f name=manager.
On bash prompt this is how it looks
ID HOSTNAME STATUS AVAILABILITY
kdrdpvwlbwai6626u640sotnh * manager Ready Active
I want to print an error message if the STATUS is not Read i.e. if in grep the match is not found for the word "Ready".
I tried but the command docker node ls -f name=manager due to spaces and arguments is treated as more than one command in the script for some reason.
What is the right way to do this?
The conventional method of doing this would be
#docker command
if [ $? -ne 0 ]
then
echo "Docker error"
fi
Edit
Supposing you get two lines of output, below would do
count=$(docker node ls -f name=manager | tail -n 1 | grep -c 'Ready[[:blank:]]*[^[:blank:]]*$')
if [ $count -ne 1 ]
then
echo "Docker Error"
fi
By default, Bash recognizes array values as separated by blank characters.
It hold a variable called IFS with the separator, and in case you want to change it, you have to change this variable.
The regular process is: storing the old value of this var, change it, process the data and restore it:
OFS=$IFS
# for new line there is special var: $'\n'
IFS=$'\n'
#your code here
IFS=$OFS
SO you can change the IFS, read the whole lines as one value and than process it with substitution: https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html
Another option is using awk (this command will print host and status):
docker node ls -f name=manager | awk '{print $2, $3}'
OR refine you search and get the hosts that status is not ready:
docker node ls -f name=manager | awk -F' '{if ($3 != "Ready") print $2}'
In case you have only one host, you can use grep to print if it found or not:
docker node ls -f name=manager | grep -iq "not ready" && echo "Not found" || echo "Found"
or omit the ||echo "Found" if you want to print an error message only
Related
I have a text file which is in fact open and does logging activities performed by process P1 in the system. I was wondering how I can get the real time content of the last line of this file in a bash script and do "echo" a message, say "done was seen", if the line equals to "done".
You could use something like this :
tail -f log.txt | sed -n '/^done$/q' && echo done was seen
Explanation:
tail -f will output appended data as the file grows
sed -n '/^done$/q' will exit when a line containing only done is encountered, ending the command pipeline.
This should work for you:
tail -f log.txt | grep -q -m 1 done && echo done was seen
The -m flag to grep means "exit after N matches", and the && ensures that the echo statement will only be done on a successful exit from grep.
How do I grep without actually filtering, or highlighting?
The goal is to find out if a certain text is in the output, without affecting the output. I could tee to a file and then inspect the file offline, but, if the output is large, that is a waste of time, because it processes the output only after the process is finished:
command | tee file
file=`mktemp`
if grep -q pattern "$file"; then
echo Pattern found.
fi
rm "$file"
I thought I could also use grep's before (-B) and after (-A) flags to achieve live processing, but that won't output anything if there are no matches.
# Won't even work - DON'T USE.
if command | grep -A 1000000 -B 1000000 pattern; then
echo Pattern found.
fi
Is there a better way to achieve this? Something like a "pretend you're grepping and set the exit code, but don't grep anything".
(Really, what I will be doing is to pipe stderr, since I'm looking for a certain error, so instead of command | ... I will use command 2> >(... >&2; result=${PIPESTATUS[*]}), which achieves the same, only it works on stderr.)
If all you want to do is set the exit code if a pattern is found, then this should do the trick:
awk -v rc=1 '/pattern/ { rc=0 } 1; END {exit rc}'
The -v rc=1 creates a variable inside the Awk program called rc (short for "return code") and initializes it to the value 1. The stanza /pattern/ { rc=0 } causes that variable to be set to 0 whenever a line is encountered that matches the regular expression pattern. The 1; is an always-true condition with no action attached, meaning the default action will be taken on every line; that default action is printing the line out, so this filter will copy its input to its output unchanged. Finally, the END {exit rc} runs when there is no more input left to process, and ensures that awk terminates with the value of the rc variable as its process exit status: 0 if a match was found, 1 otherwise.
The shell interprets exit code 0 as true and nonzero as false, so this command is suitable for use as the condition of a shell if or while statement, possibly at the end of a pipeline.
To allow output with search result you can use awk:
command | awk '/pattern/{print "Pattern found"} 1'
This will print "Pattern found" when pattern is matched in any line. (Line will be printed later)
If you want Line to print before then use:
command | awk '{print} /pattern/{print "Pattern found"}'
EDIT: To execute any command on match use:
command | awk '/pattern/{system("some_command")} 1'
EDIT 2: To take care of special characters in keyword use this:
command | awk -v search="abc*foo?bar" 'index($0, search) {system("some_command"); exit} 1'
Try this script. It will not modify anything of output of your-command and sed exit with 0 when pattern is found, 1 otherwise. I think its what you want from my understand of your question and comment.:
if your-command | sed -nr -e '/pattern/h;p' -e '${x;/^.+$/ q0;/^.+$/ !q1}'; then
echo Pattern found.
fi
Below is some test case:
ubuntu-user:~$ if echo patt | sed -nr -e '/pattern/h;p' -e '${x;/^.+$/ q0;/^.+$/ !q1}'; then echo Pattern found.; fi
patt
ubuntu-user:~$ if echo pattern | sed -nr -e '/pattern/h;p' -e '${x;/^.+$/ q0;/^.+$/ !q1}'; then echo Pattern found.; fi
pattern
Pattern found.
Note previous script fails to work when there is no ouput from your-command because then sed will not run sed expression and exit with 0 all the time.
I take it you want to print out each line of your output, but at the same time, track whether or not a particular pattern is found. Simply passing the output to sed or grep would affect the output. You need to do something like this:
pattern=0
command | while read line
do
echo "$line"
if grep -q "$pattern" <<< "$lines"
then
((pattern+=1))
fi
done
if [[ $pattern -gt 0 ]]
then
echo "Pattern was found $pattern times in the output"
else
echo "Didn't find the pattern at all"
fi
ADDENDUM
If the original command has both stdout and stderr output, which come in a specific order, with the two possibly interleaved, then will your solution ensure that the outputs are interleaved as they normally would?
Okay, I think I understand what you're talking about. You want both STDERR and STDOUT to be grepped for this pattern.
STDERR and STDOUT are two different things. They both appear on the terminal window because that's where you put them. The pipe (|) only takes STDOUT. STDERR is left alone. In the above, only the output of STDOUT would be used. If you want both STDOUT and STDERR, you have to redirect STDERR into STDOUT:
pattern=0
command 2>&1 | while read line
do
echo "$line"
if grep -q "$pattern" <<< "$lines"
then
((pattern+=1))
fi
done
if [[ $pattern -gt 0 ]]
then
echo "Pattern was found $pattern times in the output"
else
echo "Didn't find the pattern at all"
fi
Note the 2>&1. This says to take STDERR (which is File Descriptor 2) and redirect it into STDOUT (File Descriptor 1). Now, both will be piped into that while read loop.
The grep -q will prevent grep from printing out its output to STDOUT. It will print to STDERR, but that shouldn't be an issue in this case. Grep only prints out STDERR if it cannot open a file requested, or the pattern is missing.
You can do this:
echo "'search string' appeared $(command |& tee /dev/stderr | grep 'search string' | wc -l) times"
This will print the entire output of command followed by the line:
'search string' appeared xxx times
The trick is, that the tee command is not used to push a copy into a file, but to copy everything in stdout to stderr. The stderr stream is immediately displayed on the screen as it is not connected to the pipe, while the copy on stdout is gobbled up by the grep/wc combination.
Since error messages are usually emitted to stderr, and you said that you want to grep for error messages, the |& operator is used for the first pipe to combine the stderr of command into its stdout, and push both into the tee command.
Consider the following on a debian based system:
VAR=$(dpkg --get-selections | awk '{print $1}' | grep linux-image)
This will print a list of installed packages with the string "linux-image" in them on my system this output looks like:
linux-image-3.11.0-17-generic
linux-image-extra-3.11.0-17-generic
linux-image-generic
Now as we all know
echo $VAR
results in
linux-image-3.11.0-17-generic linux-image-extra-3.11.0-17-generic linux-image-generic
and
echo "$VAR"
results in
linux-image-3.11.0-17-generic
linux-image-extra-3.11.0-17-generic
linux-image-generic
I do not want to use external commands in a if clause, it seems rather dirty and not very elegant, so I wanted to use bash built in regex matching:
if [[ "$VAR" =~ ^linux-image-g ]]; then
echo "yes"
fi
however that does not work, since it does not seem to consider multiple lines here. How can I match beginnings of lines in a variable?
There's nothing wrong with using an external command as part of the if statement; I would skip the VAR variable altogether and use
if dpkg --get-selections | awk '{print $1}' | grep -q linux-image;
The -q option to grep suppresses its output, and the if statement uses the exit status of grep directly. You could also drop the grep and test $1 directly in the awk script:
if dpkg --get-selections | awk '$1 =~ "^linux-image" { exit 0; } END {exit 1}'; then
or you can skip awk, since there doesn't seem to be a real need to drop the other fields before calling grep:
if dpkg --get-selections | grep -q '^linux-image'; then
I am working on a script that executes ssh to few systems (listed in lab.txt), run two commands, store the output of commands in two different variables and print them.
Here is the script used :
#!/bin/bash
while read host; do
ssh -n root#$host "$(STATUS=$(awk 'NR==1{print $1}' /etc/*release) \
OS=$(/opt/agent/bin/agent.sh status | awk 'NR==1{print $3 $4}'))"
echo $STATUS
echo $OS
done < lab.txt
The lab.txt file contains few Ips where I need to login, execute and print the command output.
~#] cat lab.txt
192.168.1.1
192.168.1.2
While executing the script, the ssh login prompt of 192.168.1.1 is shown and after entering the password, the output is shown blank. Same as for next ip 192.168.1.2
When I execute these command manually within 192.168.1.1, the following is returned.
~]# awk 'NR==1{print $1}' /etc/*release
CentOS
~]# /opt/agent/bin/agent.sh status | awk 'NR==1{print $3 $4}'
isrunning
What could be wrong with the script? Is there a better way of doing this?
As the comment says, you are setting the variables inside the bash session on the server side and trying to read them from the client side.
If you want to assign the variables in the client script you need to put the assignment in front of the ssh command, and separate the two assignments. Something like the following.
STATUS=`ssh -n root#$host 'awk \'NR==1{print $1}\' /etc/*release)`
OS=`ssh -n root#$host '/opt/agent/bin/agent.sh status | awk \'NR==1{print $3 $4}\''`
You need to do two ssh commands. It also simplifies things if you run awk on the client rather than the server, because quoting in the ssh command gets complicated.
while read host; do
STATUS=$(ssh -n root#$host 'cat /etc/*release' | awk 'NR==1{print $1}')
OS=$(ssh -n root#$host /opt/agent/bin/agent.sh status | awk 'NR==1{print $3 $4}')
echo $STATUS
echo $OS
done < lab.txt
with one ssh statement:
read STATUS OS < <(ssh -n root#$host "echo \
\$(awk 'NR==1{print \$1}' /etc/*release) \
\$(/opt/agent/bin/agent.sh status | awk 'NR==1{print \$3 \$4}')")
echo $STATUS
echo $OS
Explanation:
The <(command) syntax is called process substitution. You can use it anywhere where a file is expected.
Example:
sdiff <(echo -e "1\n2\n3") <(echo -e "1\n3")
The command sdiff expects two files as arguments. With the process substitution syntax you can use commands as arguments. ( e.g. fake files )
I am using GNU bash, version 4.2.20(1)-release (x86_64-pc-linux-gnu). I have a music file list I dumped into a variable: $pltemp.
Example:
/Music/New/2010s/2011;Ziggy Marley;Reggae In My Head
I wish to grep the 3rd field above, in the Master-Music-List.txt, then continue another grep for the 2nd field. If both matched, print else echo "Not Matched".
So the above will search for the Song Title (Reggae In My Head), then will make sure it has the artist "Shaggy" on the same line, for a success.
So far, success for a non-variable grep;
$ grep -i -w -E 'shaggy.*angel' Master-Music-MM-Playlist.m3u
$ if ! grep Shaggy Master-Music-MM-Playlist.m3u ; then echo "Not Found"; fi
$ grep -i -w Angel Master-Music-MM-Playlist.m3u | grep -i -w shaggy
I'm not sure how to best construct the 'entire' list to process.
I want to do this on a single line.
I used this to dump the list into the variable $pltemp...
Original: \Music\New\2010s\2011\Ziggy Marley - Reggae In My Head.mp3
$ pltemp="$(cat Reggae.m3u | sed -e 's/\(.*\)\\/\1;/' -e 's/\(.*\)\ -\ /\1;/' -e 's/\\/\//g' -e 's/\\/\//g' -e 's/.mp3//')"
If you realy want to "grep this, then grep that", you need something more complex than grep by itself. How about awk?
awk -F';' '$3~/title/ && $2~/artist/ {print;n=1;exit;} END {if(n=0)print "Not matched";}'
If you want to make this search accessible as a script, the same thing simply changes form. For example:
#!/bin/sh
awk -F';' -vartist="$1" -vtitle="$2" '$3~title && $2~artist {print;n=1;exit;} END {if(n=0)print "Not matched";}'
Write this to a file, make it executable, and pipe stuff to it, with the artist substring/regex you're looking for as the first command line option, and the title substring/regex as the second.
On the other hand, what you're looking for might just be a slightly more complex regular expression. Let's wrap it in bash for you:
if ! echo "$pltemp" | egrep '^[^;]+;[^;]*artist[^;]*;.*title'; then
echo "Not matched"
fi
You can compress this to a single line if you like. Or make it a stand-along shell script, or make it a function in your .bashrc file.
awk -F ';' -v title="$title" -v artist="$artist" '$3 ~ title && $2 ~ artist'
Well, none of the above worked, so I came up with this...
for i in *.m3u; do
cat "$i" | sed 's/.*\\//' | while read z; do
grep --color=never -i -w -m 1 "$z" Master-Music-Playlist.m3u \
| echo "#NotFound;"$z" "
done > "$i"-MM-Final.txt;
done
Each line is read (\Music\Lady Gaga - Paparazzi.mp3), the path is stripped, the song is searched in the Master Music List, if not found, it echos "Not Found", saved into a new playlist.
Works {Solved}
Thanks anyway.