I have same log file, like this:
2016-04-01 11:16:30.745:[11878][TEST][test]
2016-04-01 11:16:30.745:[11878][TEST][wait|hold|name(0x03154246) 101ms]
....
at first, I use grep wait to found the log
2016-04-01 11:16:30.745:[11878][TEST][wait|hold|name(0x03154246) 101ms]
then, how can i get the field value
value1: 2016-04-01 11:16:30.745
value2: 0x03154246
value3: 101
It's hard to figure out from what you wrote but something like may be suitable for you:
#!/usr/bin/env sh
set -e
string=$(grep wait "$1")
value1=$(echo "$string" | rev | cut -d ":" -f 2- | rev)
value2=$(echo "$string" | grep -o -E "\(0x.+\)" | sed 's,),,' | sed 's,(,,')
value3=$(echo "$string" | grep -o -E "[0-9]+ms]$" | sed 's,ms],,')
echo vaule1: "$value1"
echo vaule2: "$value2"
echo vaule3: "$value3"
Usage:
$ ./get.sh LOG
value1 is everything up to last :.
value2 are digits between last (0x and ).
value3 are digits before ms] at the end of the string.
Related
I want to pipe the output of the command into two commands and paste the results together. I found this answer and similar ones suggesting using tee but I'm not sure how to make it work as I'd like it to.
My problem (simplified):
Say that I have a myfile.txt with keys and values, e.g.
key1 /path/to/file1
key2 /path/to/file2
What I am doing right now is
paste \
<( cat myfile.txt | cut -f1 ) \
<( cat myfile.txt | cut -f2 | xargs wc -l )
and it produces
key1 23
key2 42
The problem is that cat myfile.txt is repeated here (in the real problem it's a heavier operation). Instead, I'd like to do something like
cat myfile.txt | tee \
<( cut -f1 ) \
<( cut -f2 | xargs wc -l ) \
| paste
But it doesn't produce the expected output. Is it possible to do something similar to the above with pipes and standard command-line tools?
This doesn't answer your question about pipes, but you can use AWK to solve your problem:
$ printf %s\\n 1 2 3 > file1.txt
$ printf %s\\n 1 2 3 4 5 > file2.txt
$ cat > myfile.txt <<EOF
key1 file1.txt
key2 file2.txt
EOF
$ cat myfile.txt | awk '{ ("wc -l " $2) | getline size; sub(/ .+$/,"",size); print $1, size }'
key1 3
key2 5
On each line we first we run wc -l $2 and save the result into a variable. Not sure about yours, but on my system wc -l includes the filename in the output, so we strip it with sub() to match your example output. And finally, we print the $1 field (key) and the size we got from wc -l command.
Also, can be done with shell, now that I think about it:
cat myfile.txt | while read -r key value; do
printf '%s %s\n' "$key" "$(wc -l "$value" | cut -d' ' -f1)"
done
Or more generally, by piping to two commands and using paste, therefore answering the question:
cat myfile.txt | while read -r line; do
printf %s "$line" | cut -f1
printf %s "$line" | cut -f2 | xargs wc -l | cut -d' ' -f1
done | paste - -
P.S. The use of cat here is useless, I know. But it's just a placeholder for the real command.
From a file I'm retrieving the last line using the following cmd;
tail -n 1 build.log
The output looks like this:
1477101542,,ui,say,--> amazon-ebs: AMIs were created:\n\nus-east-1: ami-63237174\nus-west-1: ami-21236841\nus-west-2: ami-27872347
I'm trying to fetch the string after us-east-1:, us-west-1: & us-west-2 using the following grep commands:
echo | tail -n 1 build.log | egrep -m1 -oe 'us-east-1: ami-.{8}' | egrep -m1 -oe 'ami-.{8}'
I run this cmd three times for each condition. Is there a better way to do this?
If the order in which the regions appear is fixed, you can simply do:
$ echo | tail -n 1 build.log | egrep -o 'ami-.{8}'
ami-63237174
ami-21236841
ami-27872347
If you want to extract the region names and you have GNU grep, try:
$ echo | tail -n 1 build.log | grep -Po 'us-[^:]+(?=: ami-.{8})'
us-east-1
us-west-1
us-west-2
To get both region names and associated values:
$ echo | tail -n 1 build.log | egrep -o 'us-[^:]+: ami-.{8}'
us-east-1: ami-63237174
us-west-1: ami-21236841
us-west-2: ami-27872347
I want to parse the following string in shell script.
VERSION=2.6.32.54-0.11.def
Here I want to get two value.
first = 263254
second = 11
I am using following to get the first value:
first=`expr substr $VERSION 1 9| sed "s/\.//g" |sed "s/\-//g"`
to get the second:
second=`expr substr $VERSION 10 6| sed "s/\.//g" |sed "s/\-//g"`
Using above code the output is:
first=263254
second=11
The result wont be consistent if version is changed to:
VERSION=2.6.32.54-0.1.def
Here second value will become 1d, but I want it give output of 1 only.
How can I directly parse the number after '-' and before '.d'?
$ first=$(echo $VERSION | cut -d- -f1 | sed 's/\.//g')
$ second=$(echo $VERSION | cut -d- -f2 | cut -d. -f2)
$ first=$(echo $VERSION | cut -d- -f1 | tr -d '.')
$ second=$(echo $VERSION | cut -d- -f2 | cut -d. -f2)
$ echo $first
263254
$ echo $second
11
you don't need multiple processes (sed|sed|sed...). single process with awk should work.
if you have VERSION=xxxx as string:
to get the first:
awk -F'[-=]' '{gsub(/\./,"",$2)}$0=$2'
to get the second:
awk -F'-|\\.def' '{split($2,a,".")}$0=a[2]'
test:
first:
kent$ echo "VERSION=2.6.32.54-0.1.def"|awk -F'[-=]' '{gsub(/\./,"",$2)}$0=$2'
263254
second
kent$ echo "VERSION=2.6.32.54-0.1.def"|awk -F'-|\\.def' '{split($2,a,".")}$0=a[2]'
1
kent$ echo "VERSION=2.6.32.54-0.1234.def"|awk -F'-|\\.def' '{split($2,a,".")}$0=a[2]'
1234
if you have VERSION=xxx as variable $VERSION:
first:
awk -F'-' '{gsub(/\./,"",$1)}$0=$1'
second:
awk -F'-|\\.def' '{split($2,a,".")}$0=a[2]'
test:
VERSION=2.6.32.54-0.1234.def
kent$ echo $VERSION|awk -F'-' '{gsub(/\./,"",$1)}$0=$1'
263254
7pLaptop 11:18:22 /tmp/test
kent$ echo $VERSION|awk -F'-|\\.def' '{split($2,a,".")}$0=a[2]'
1234
You should use regular expressions instead of the number of characters.
first=`sed 's/.//g' | sed 's/\(.*\)-.*/\1/'`
second=`sed 's/.//g' | sed 's/.*-\([0-9]*\).*/\1/'`
\(...\) are used to create a capturing group, and \1 output this group.
first=$(echo ${VERSION} | sed -e 's/^\([^-]*\)-0\.\([0-9]*\)\.def/\1/' -e 's/\.//g')
second=$(echo ${VERSION} | sed -e 's/^\([^-]*\)-0\.\([0-9]*\)\.def/\2/' -e 's/\.//g')
$ first=$(echo $VERSION | awk -F"\." '{gsub(/-.*/,"",$4);print $1$2$3$4}')
$ second=$(echo $VERSION | awk -F"\." '{print $5}' )
I'm trying to make a small function that removes all the chars that are not digits.
123a45a ---> will become ---> 12345
I've came up with :
temp=$word | grep -o [[:digit:]]
echo $temp
But instead of 12345 I get 1 2 3 4 5. How to I get rid of the spaces?
Pure bash:
word=123a45a
number=${word//[^0-9]}
Here's a pure bash solution
var='123a45a'
echo ${var//[^0-9]/}
12345
is this what you are looking for?
kent$ echo "123a45a"|sed 's/[^0-9]//g'
12345
grep & tr
echo "123a45a"|grep -o '[0-9]'|tr -d '\n'
12345
I would recommend using sed or perl instead:
temp="$(sed -e 's/[^0-9]//g' <<< "$word")"
temp="$(perl -pe 's/\D//g' <<< "$word")"
Edited to add: If you really need to use grep, then this is the only way I can think of:
temp="$( grep -o '[0-9]' <<< "$word" \
| while IFS= read -r ; do echo -n "$REPLY" ; done
)"
. . . but there's probably a better way. (It uses grep -o, like your solution, then runs over the lines that it outputs and re-outputs them without line-breaks.)
Edited again to add: Now that you've mentioned that you use can use tr instead, this is much easier:
temp="$(tr -cd 0-9 <<< "$word")"
What about using sed?
$ echo "123a45a" | sed -r 's/[^0-9]//g'
12345
As I read you are just allowed to use grep and tr, this can make the trick:
$ echo "123a45a" | grep -o [[:digit:]] | tr -d '\n'
12345
In your case,
temp=$(echo $word | grep -o [[:digit:]] | tr -d '\n')
tr will also work:
echo "123a45a" | tr -cd '[:digit:]'
# output: 12345
Grep returns the result on different lines:
$ echo -e "$temp"
1
2
3
4
5
So you cannot remove those spaces during the filtering, but you can afterwards, since $temp can transform itself like this:
temp=`echo $temp | tr -d ' '`
$ echo "$temp"
12345
This can be so easy to a people who know. I am almost finishing this command
echo VERSION=1.0 | sed 's/^VERSION=\([0-9]\).\([0-9]\)/VERSION=\1.\2+1/'
I only want to write VERSION=1.1 . How can I evaluate \2 to integer and sum +1..
of couse sed can do that. that's what e for. you can pass matched/replaced string to shell command using "e"
see the example based on your sed line:
kent$ echo VERSION=1.0 | sed 's/^VERSION=\([0-9]\).\([0-9]\)/echo "VERSION=\1.$((\2+1))"/e'
VERSION=1.1
You can use the bc command:
echo VERSION=`echo "1.0 + 0.1" | bc`
Results in:
VERSION=1.1
man bc
echo "VERSION="`echo "v=1.0; v+=0.1; v" | bc` > myFile.txt
cat myFile.txt
VERSION=1.1
Crpytic answer - how to use the whole toolkit:
x='VERSION=1.0'
echo -n $x | sed 's/\..*/./'; expr `echo $x | grep -o '\..*' | cut -c 2-` + 1