shell command to truncate/cut a part of string - shell

I have a file with the below contents. I got the command to print version number out of it. But I need to truncate the last part in the version file
file.spec:
Version: 3.12.0.2
Command used:
VERSION=($(grep -r "Version:" /path/file.spec | awk '{print ($2)}'))
echo $VERSION
Current output : 3.12.0.2
Desired output : 3.12.0

There is absolutey no need for external tools like awk, sed etc. for this simple task if your shell is POSIX-compliant (which it should be) and supports parameter expansion:
$ cat file.spec
Version: 3.12.0.2
$ version=$(<file.spec)
$ version="${version#* }"
$ version="${version%.*}"
$ echo "${version}"
3.12.0

Try this:
VERSION=($(grep -r "Version:" /path/file.spec| awk '{print ($2)}' | cut -d. -f1-3))
Cut split string with field delimiter (-d) , then you select desired field with -f param.

You could use this single awk script awk -F'[ .]' '{print $2"."$3"."$4}':
$ VERSION=$(awk -F'[ .]' '{print $2"."$3"."$4}' /path/file.spec)
$ echo $VERSION
3.12.0
Or this single grep
$ VERSION=$(grep -Po 'Version: \K\d+[.]\d+[.]\d' /path/file.spec)
$ echo $VERSION
3.12.0
But you never need grep and awk together.

if you only grep single file, -r makes no sense.
also based on the output of your command line, this grep should work:
grep -Po '(?<=Version: )(\d+\.){2}\d+' /path/file.spec
gives you:
3.12.0
the \K is also nice. worked for fixed/non-fixed length look-behind. (since PCRE 7.2). There is another answer about it. but I feel look-behind is easier to read, if fixed length.

Related

Extract substring from a variables between two patterns in bash with special characters

I am trying to Extract substring from variables between two patterns in bash that as special characters inside the variable.
The variable:
MQ_URI=ssl://b-7dda5da6-59a5-4150-8e2f-16534985665-1.mq.us-east-1.amazonaws.com:61617?jms.prefetchPolicy.queuePrefetch=0
What I've tried so far:
echo "$MQ_URI" | sed -E 's/.*ssl:// (.*) :61617.*/\1/'
Got me this in response:
sed: -e expression #1, char 12: unknown option to `s'
Also tried with grep:
echo $MQ_URI | grep -o -P '(?<=ssl://).*(?=:61617jms.prefetchPolicy.queuePrefetch=0)
The output I need is everything between: "ssl://" and ":61617?jms.prefetchPolicy.queuePrefetch=0"
which is : "b-7dda5da6-59a5-4150-8e2f-16534985665-1.mq.us-east-1.amazonaws.com"
Using bash
$ mq_uri=${mq_uri##*/}
$ mq_uri=${mq_uri//:*}
$ echo "$mq_uri"
b-7dda5da6-59a5-4150-8e2f-16534985665-1.mq.us-east-1.amazonaws.com
sed
$ sed -E 's~[^-]*/([^?]*):.*~\1~' <<< "$mq_uri"
b-7dda5da6-59a5-4150-8e2f-16534985665-1.mq.us-east-1.amazonaws.com
grep
$ grep -Po '[^-]*/\K[^:]*' <<< "$mq_uri"
b-7dda5da6-59a5-4150-8e2f-16534985665-1.mq.us-east-1.amazonaws.com
awk
$ awk -F'[/:]' '{print $4}' <<< "$mq_uri"
b-7dda5da6-59a5-4150-8e2f-16534985665-1.mq.us-east-1.amazonaws.com
If this is what you expect
echo "$MQ_URI" | sed -E 's#.*ssl://(.*):61617.*#\1#'
b-7dda5da6-59a5-4150-8e2f-16534985665-1.mq.us-east-1.amazonaws.com
replace the delimiters by # or anything not found in the string.
With your shown samples and attempts please try following codes.
##Shell variable named `mq_uri` being created here.
##to be used in following all solutions.
mq_uri="ssl://b-7dda5da6-59a5-4150-8e2f-16534985665-1.mq.us-east-1.amazonaws.com:61617?jms.prefetchPolicy.queuePrefetch=0"
1st solution: Using awk's match function along with split` function here.
awk 'match($0,/^ssl:.*:61617\?/){split(substr($0,RSTART,RLENGTH),arr,"[/:]");print arr[4]}' <<<"$mq_uri"
2nd solution: Using GNU grep along with its -oP options and its \K option to get required output.
grep -oP '^ssl:\/\/\K[^:]*(?=:61617\?)' <<<"$mq_uri"
3rd solution: Using match function of awk along with using gsub to Globally substitute values to get required output.
awk 'match($0,/^ssl:.*:61617\?/){val=substr($0,RSTART,RLENGTH);gsub(/^ssl:\/\/|:.*\?/,"",val);print val}' <<<"$mq_uri"
4th solution: Using awk's match function along with its array creation capability in GNU awk.
awk 'match($0,/^ssl:\/\/(.*):61617\?/,arr){print arr[1]}' <<<"$mq_uri"
5th solution: With perl's One-liner solution please try following code.
perl -pe 's/ssl:\/\/(.*):61617\?.*/\1/' <<<"$mq_uri"

One-liner POSIX command to lowercase string in bash

Problem
I have this comand:
sed $((SS - default_scripts))!d customScripts.txt
and it gives me Foo Bar.
I want to convert this to lowercase.
Attempt
When I tried using the | awk '{print tolower($0)}' command on it it returned nothing:
$($(sed $((SS - default_scripts))!d customScripts.txt) | awk '{print tolower($0)}')
Final
Please enlighten me on my typo, or recommend me another POSIX way of converting a whole string to lowercase in a compact manner. Thank you!
The pipe to awk should be inside the same command substitution as sed, so that it processes the output of sed.
$(sed $((SS - default_scripts))!d customScripts.txt | awk '{print tolower($0)}')
You don't need another command substitution around both of them.
Your typo was wrapping everything in $(...) and so first trying to execute the output of just the sed part and then trying to execute the output of the sed ... | awk ... pipeline.
You don't need sed commands nor shell arithmetic operations when you're using awk. If I understand what you're trying to do with this:
$(sed $((SS - default_scripts))!d customScripts.txt) | awk '{print tolower($0)}'
correctly then it'd be just this awk command:
awk -v s="$SS" -v d="$default_scripts" 'BEGIN{n=s-d} NR==n{print tolower($0); exit}' customScripts.txt

How can I capture all numbers in a string using sed

I tried to use sed to capture numbers in a string with following script:
echo '["770001,德邦优化混合","750005,安信平稳增长混合发起A"]' | sed -n 's/.*"\(\d{6}\),/\1/p'
My expectation is echo
770001
750005
While nothing output. Why?
In case you are ok with awk then following awk may help you in same. Since I have old version of awk so I am using --re-interval if you have newer version of awk then you may not need it.
echo '["770001,德邦优化混合","750005,安信平稳增长混合发起A"]' |
awk --re-interval '{while(match($0,/[0-9]{6}/)){print substr($0,RSTART,RLENGTH);$0=substr($0,RSTART+RLENGTH+1)}}'
Output will be as follows.
770001
750005

Bash grep variable as an expression

I have a problem with bash. I have big log file and I must check only a part of all log. In this purpose I use those expressions:
cat 29.log | grep -A 999 "15/02/06-22:30"
or
awk '$1>="15/02/06-22:30" {print$0}' 29.log
I want to change "15/02/06-22:30" at "date +%y/%m/d-%H:M" but when I use command
awk '$1>="date +%y/%m/d-%H:M" {print$0}' 29.log
or
awk '$1>='date +%y/%m/d-%H:M' {print$0}' 29.log
nothing happens.
Any ideas?
I need this in one command, not a script
You can pass shell variables to AWK using the -v flag:
awk -v d="$(date '+%y/%m/%d-%H:%M')" '$1>=d' 29.log
grep -A 999 `date '+%y/%m/d-%H:M'` 29.log

need to parse between second underscore and first hyphen of the text using sed

I have an rpm file, e.g. abc_defg_hijd-3.29.0-2_el6_11h.txt.
I need to parse the words between the 2nd underscore _ and first hyphen - of the above text,
so the required output will be hijd.
I was able to parse the above with sed for the above, but it worked only for the above example and I have filenames which differ a little, hence I would like to explicitly parse between the second underscore and first hyphen.
Use this sed command (on Mac):
sed -E 's/^[^_]*_[^_]*_([^-]*)-.*$/\1/'
OR (on Linux):
sed -r 's/^[^_]*_[^_]*_([^-]*)-.*$/\1/'
Using awk:
awk -F '_' '{sub(/-.*$/, "", $3); print $3}'
$ foo='abc_defg_hijd-3.29.0-2_el6_11h.txt'
$ bar=${foo%%-*} # remove everything after the first -
$ bar=${bar#*_}; bar=${bar#*_} # remove everything before the second _
$ echo "${bar}"
hijd
grep was born to extract:
grep -oP '[^_-]*_\K[^_-]*(?=-)'
example
kent$ echo 'abc_defg_hijd-3.29.0-2_el6_11h.txt'|grep -oP '[^_-]*_\K[^_-]*(?=-)'
hijd
awk is nuclear bomb for text processing,but it can kill a fly for sure:
awk -F- 'split($1,a,"_")&&$0=a[3]'
or shorter(gawk):
awk -v FPAT="[^-_]*" '$0=$3'
example
kent$ echo 'abc_defg_hijd-3.29.0-2_el6_11h.txt'|awk -F- 'split($1,a,"_")&&$0=a[3]'
hijd
kent$ echo 'abc_defg_hijd-3.29.0-2_el6_11h.txt'|awk -v FPAT="[^-_]*" '$0=$3'
hijd
with GNU sed
echo 'abc_defg_hijd-3.29.0-2_el6_11h.txt' |
sed 's/\([^_]\+_\)\{2\}\([^-]\+\)-.*/\2/g'
hijd
windows batch:
for /f "tokens=3delims=_-" %%i in ("abc_defg_hijd-3.29.0-2_el6_11h.txt") do echo %%i
hijd

Resources