I have a data looks like this
<some blah! blah!>|201451|<some blah! blah!>
<some blah! blah!>|201441|<some blah! blah!>
<some blah! blah!>|201431|<some blah! blah!>
<some blah! blah!>|201421|<some blah! blah!>
I have to sort it to
<some blah! blah!>|201421|<some blah! blah!>
<some blah! blah!>|201431|<some blah! blah!>
<some blah! blah!>|201441|<some blah! blah!>
<some blah! blah!>|201451|<some blah! blah!>
I have tried using both these:
sort -t"|" -k4.5,4.6 -b data
LC_ALL=c sort -t"|" -k4.5,4.6 -b data
but it always gives me
<FOUR BLANK SPACE>
<some blah! blah!>|201421|<some blah! blah!>
<some blah! blah!>|201431|<some blah! blah!>
<some blah! blah!>|201441|<some blah! blah!>
<some blah! blah!>|201451|<some blah! blah!>
I want to maintain the space with the sorted value. How do I do so??
Basically I have to ignore the line space and not remove them...
Here is a solution with sorting removing blank lines first and then inserting the blank lines back using sed:
grep -v '^$' <file> | sort -t'|' -k 4.5,4.6 | sed 's/$/\n/'
Try to execute:
sort -t"|" -k4.5,4.6 data | grep -v "^$" | awk '{print $0"\n"}'
You can accomplish what you need by leading with an awk that simply checks if there is content on the line - then follow it up with a sort that suits your particular case:
$ awk '$1' data_file | sort
some blah! blah!|201421|some blah! blah!
some blah! blah!|201431|some blah! blah!
some blah! blah!|201441|some blah! blah!
some blah! blah!|201451|some blah! blah!
Related
Looking for a way to pass the second column of output to geoiplookup, ideally on the same line, but not necessarily. This is the best I can muster. It's usable, but the geoiplookup results are unfortunately below the list of connections. I wanted more integrated results. If anyone can suggest improvements, they would be welcome.
ns () {
echo ""
while sleep 1; do
lsof -Pi |
grep ESTABLISHED |
sed "s/[^:]*$//g" |
sed "s/^[^:]*//g" |
sed "s/://g" |
sed "s/->/\t/g" |
grep -v localdomain$ |
tee >(for x in `grep -o "\S*$"`; do geoiplookup $x | sed "s/GeoIP.*: /\t/g"; done)
done
}
The results currently look something like this:
<Port> <URL or IP if no reverse available #1>
<Port> <URL or IP if no reverse available #2>
<geoiplookup trimmed result #1>
<geoiplookup trimmed result #2>
I received an excellent answer here.
Consider the following on a debian based system:
VAR=$(dpkg --get-selections | awk '{print $1}' | grep linux-image)
This will print a list of installed packages with the string "linux-image" in them on my system this output looks like:
linux-image-3.11.0-17-generic
linux-image-extra-3.11.0-17-generic
linux-image-generic
Now as we all know
echo $VAR
results in
linux-image-3.11.0-17-generic linux-image-extra-3.11.0-17-generic linux-image-generic
and
echo "$VAR"
results in
linux-image-3.11.0-17-generic
linux-image-extra-3.11.0-17-generic
linux-image-generic
I do not want to use external commands in a if clause, it seems rather dirty and not very elegant, so I wanted to use bash built in regex matching:
if [[ "$VAR" =~ ^linux-image-g ]]; then
echo "yes"
fi
however that does not work, since it does not seem to consider multiple lines here. How can I match beginnings of lines in a variable?
There's nothing wrong with using an external command as part of the if statement; I would skip the VAR variable altogether and use
if dpkg --get-selections | awk '{print $1}' | grep -q linux-image;
The -q option to grep suppresses its output, and the if statement uses the exit status of grep directly. You could also drop the grep and test $1 directly in the awk script:
if dpkg --get-selections | awk '$1 =~ "^linux-image" { exit 0; } END {exit 1}'; then
or you can skip awk, since there doesn't seem to be a real need to drop the other fields before calling grep:
if dpkg --get-selections | grep -q '^linux-image'; then
I am trying to search logs for an expression, then select the line below each match.
Example
I know I want the lines below CommonText, for example given the log data:
CommonTerm: something
This Should
random stuff
CommonTerm: something else
Be The
random stuff
more random stuff
CommonTerm: something else
Output Text
random fluff
Desired Output
This Should
Be The
Output Text
Current Attempt
Currently I can use grep log_file CommonTerm -B 0 -A 1 to get:
CommonTerm: something
This Should
--
CommonTerm: something else
Be The
--
CommonTerm: something else
Output Text
I can then pipe this through | grep "\-\-" -B 0 -A 1 to get
This Should
--
--
Be The
--
--
Output Text
--
And then through awk '{if (count++%3==0) print $0;}', giving:
This Should
Be The
Output Text
My question is: surely there's a good 'unix-y' way to do this? Multi greps and a hacky awk feels pretty silly... Is there?
Edit: I also tried:
(grep 'CommonTerm:' log_file -B 0 -A 2) | grep "\-\-" -B 1 -A 0 | grep -v "^--$"
but it seems much more clunky than the answers below which was expected ;)
Edit:
There are some great answers coming in, are there any which would let me easily select the nth line after the search term? I see a few might be more easy than others...
awk 'p { print; p=0 }
/CommonTerm/ { p=1 }' file
You can use sed:
sed -n "/^CommonTerm: /{n;p}" log_file
This searches for "CommonTerm: " at the start of the line (^), then skips to the next line (n) and prints it (p).
EDIT: As per the comment thread, if you're using BSD sed rather than GNU sed (likely to be the case on OS X), you need a couple of extra semicolons to get round a bug:
sed -n "/^CommonTerm: /{;n;p;}" log_file
How about:
grep -B 0 -A 1 "CommonTerm" log_file | grep -v "^CommonTerm:" | grep -v "^--$"
I'd do this with awk:
awk 'found{found=0;print;next}/CommonTerm/{found=1}'
For those that have pcregrep installed, this can be done at one shot. Notice the use of \K to reset the starting point of the match
pcregrep -Mo 'CommonTerm.*?\n\K.*?(?=\n)' file
I am trying to find an exact word in a file. The word is myServer and the file contents in which I am trying to find it are:
This is myServer and it is cool
This is myServer-test and it is cool
Here're the results of my various attempts to do this:
grep '^myServer$' test-file.txt
grep -Fx myServer test-file.txt
--> no results
grep -w myServer test-file.txt
grep -w myServer test-file.txt
grep '\<myServer\>' test-file.txt
grep '\bmyServer\b' test-file.txt
-->
This is myServer and it is cool
This is myServer-test and it is cool
Am I missing some switch here?
Many Thanks.
Try grep with word boundaries:
grep '\<myServer\>' test-file.txt
EDIT: Looks like you don't want to treat hyphen as word boundary. Use grep like this for that purpose:
grep -E '\<myServer\>([^-]|$)' test-file.txt
You can use \< and \> to identify beginning and end of words. So for example given \<myS to grep would search for lines with a word starting with myS.
Naturally, to search for an exact word, you would search like this:
grep '\<myServer\>' test-file.txt
In most of your attempts, you are doing completely irrelevant stuff:
^ and $ match the beginning and end of the line.
-F searches for multiple fixed strings
-x searches for whole lines matching the regex
Note: your -w would work correctly. However, it seems that your definition of "whole word" is different from what POSIX thinks.
You can use \b which means a word boundary:
rep '\bmyServer\b' test-file.txt
The following code gives output of a log containing the string "Type: Error"
grep -E "$Currentdate" *.log | grep "Type: Error" | sort -u
what is the code If I want to get output of all the lines in log except the lines with string "Type: Error"
Thanks in advance
Use -v option grep that will do the trick.
cat logFile | grep -v "Type: Error"
With this, first, you'll display to video your file, then you'll use -v option that invert the matches and the trick is done!
For multiple string, you could do something like this
cat logFile | egrep -v '(firstString)|(secondString)|(thirdString)|...|(nString)'