I have a string foo-0 that I want to convert to bar1baz, i.e., parse the trailing index and add a prefix/suffix. The part before the trailing index (in this case foo- can also contain numeric characters, but those should not be changed.
I tried the following:
echo foo-0 | cut -d'-' -f 2 | sed 's/.*/bar&baz/'
but that gives me only a partial solution (bar0baz). How can I increment the infix variable?
EDIT: the solutions below only work partially for what I am trying to achieve. This is my fault because I simplified the example above too much for the sake of clarity.
The final goal is to set an environmental variable (let's call it MY_ENV) to the output value using bash with the following syntax:
/bin/sh -c "echo $var | ... (some bash magic to replace the trailing index) | ... (some bash magic to set MY_ENV=the output of the pipe)"
Side note: The reason I am using /bin/sh -c "..." is because I want to use the command in a Kubernetes YAML.
Partial solution (using awk)
This works:
echo foo-0 | awk -F- '{print "bar" $2+1 "baz"}'
This doesn't (output is 1baz):
/bin/sh -c "echo foo-0 | awk -F- '{print \"bar\" $2+1 \"baz\"}'
Partial solution (using arithmetic context and parameter expansion)
$ var=foo-0
$ echo "bar$((${var//[![:digit:]]}+1))baz"
This does not work if var contains other numeric characters before the trailing index (e.g. for var foo=r2a-foo-0.
You may use awk:
awk -F- '{print "bar" $2+1 "baz"}' <<< 'foo-0'
bar1baz
You could use an arithmetic context and parameter expansion:
$ var=foo-0
$ echo "bar$((${var//[![:digit:]]}+1))baz"
bar1baz
Unrolled, from the inside:
${var//[![:digit:]]} removes all non-digits from var:
$ echo "${var//[![:digit:]]}"
0
$((blah+1)) adds 1 to the variable blah:
$ blah=0
$ echo "$((blah+1))"
1
or, instead of blah, we can use the result of the inner substitution:
$ echo "$(( ${var//[![:digit:]]} + 1 ))"
1
and finally, putting this between bar and baz, you get bar1baz.
Amending for the other case brought up: assuming there might be other digits and we want to increment only the trailing ones, e.g.,
var=2a-foo-21
To do this, we can use nested parameter expansion with extended globs (shopt -s extglob) and the +(pattern) pattern, which matches one or more of pattern. Observe:
$ echo "${var#"${var%%+([[:digit:]])}"}"
21
The outer expansion is ${var#pattern}, which removes the shortest match of pattern from the beginning of $var. For pattern, we use
"${var%%+([[:digit:]])}"
which is "remove the longest match of +([[:digit:]]) (one or more digits) from the end of $var". This leaves us with just the trailing digits, and incrementing them and adding string before and after looks something like this:
$ echo "bar$((${var#"${var%%+([[:digit:]])}"}+1))baz"
bar22baz
This is so unreadable that I'd suggest using regex instead:
$ re='([[:digit:]]+)$'
$ [[ $var =~ $re ]]
$ echo "bar$((${BASH_REMATCH[1]}+1))baz"
bar22baz
Related
for name in file*.txt
do
echo ${name%.txt} | grep -o -E '[0-9]+'
done
is there a better way then using grep? I have file1.txt, file2.txt ...., want to extract just the numbers.
If your shell is bash, you might also consider replacing all non-numeric characters with the empty string with a parameter expansion, as follows:
for name in file*.txt; do
echo "${name//[![:digit:]]/}"
done
By contrast, if you need to work with POSIX-compatible shells (and the file prefix is hardcoded), consider trimming prefix and suffix using the following POSIX-compliant PEs:
for name in file*.txt; do
num=${name%.txt}; num=${num#file}
echo "$num"
done
In bash:
$ for f in test* ; do f=${f#test} ; echo "${f%.txt}" ; done
1
2
3
Add some quotes if filenames have space in them.bb
Using GNU awk:
$ awk 'BEGINFILE {
gsub(/^test|\.txt/,"",FILENAME)
print FILENAME
nextfile
}' test*
1
2
3
I am creating this function to make multiple grep's over every line of a file. I run it as following:
cat file.txt | agrep string1 string2 ... stringN
function agrep () {
for a in $#; do
cmd+=" | grep '$a'";
done ;
while read line ; do
eval "echo "\'"$line"\'" $cmd";
done;
}
The idea is to print every line that contains all the strings: string1, string2, ..., stringN. This already works but I want to avoid the usage of the for to construct the expression:
| grep string1 | grep string2 ... | stringN
And if it's possible, also the usage of eval. I tried to make some expansion as follows:
echo "| grep $"{1..3}
And I get:
| grep $1 | grep $2 | grep $3
This is almost what I want but the problem is that when I try:
echo "| grep $"{1..$#}
The expansion doesn't occur because bash cant expand {1..$#} due to the $#. It just works with numbers. I would like to construct some expansion that works in order to avoid the usage of the for in the agrep function.
agrep () {
if [ $# = 0 ]; then
cat
else
pattern="$1"
shift
grep -e "$pattern" | agrep "$#"
fi
}
Instead of running each multiple greps on each line, just get all the lines that match string1, then pipe that to grep for string2, etc. One way to do this is make agrep recursive.
agrep () {
if (( $# == 0 )); then
cat # With no arguments, just output everything
else
grep "$1" | agrep "${#:2}"
fi
}
It's not the most efficient solution, but it's simple.
(Be sure to note Rob Mayoff's answer, which is the POSIX-compliant version of this.)
awk to the rescue!
you can avoid multiple grep calls and constructing the command by switching to awk
awk -v pat='string1 string2 string3' 'BEGIN{n=split(pat,p)}
{for(i=1;i<=n;i++) if($0!~p[i]) next}1 ' file
enter your space delimited strings as in the example above.
Not building a string for the command is definitely better (see chepner's and Rob Mayoff's answers). However, just as an example, you can avoid the for by using printf:
agrep () {
cmd=$(printf ' | grep %q' "$#")
sh -c "cat $cmd"
}
Using printf also helps somewhat with special characters in the patterns. From help printf:
In addition to the standard format specifications described in printf(1),
printf interprets:
%b expand backslash escape sequences in the corresponding argument
%q quote the argument in a way that can be reused as shell input
%(fmt)T output the date-time string resulting from using FMT as a format
string for strftime(3)
Since the aim of %q is providing output suitable for shell input, this should be safe.
Also: You almost always want to use "$#" with the quotes, not just plain $#.
How can we fetch a substring from a string in bash using scripting language?
Example:
fullstring="mnuLOCNMOD.URL = javascript:parent.doC...something"
The substring I want is everything before ".URL" in the full string.
With Parameter Expansion, you can do:
fullstring="mnuLOCNMOD.URL = javascript:parent.doC...something"
echo ${fullstring%\.URL*}
prints:
mnuLOCNMOD
$ fullstring="mnuLOCNMOD.URL = javascript:parent.doC...something"
$ sed -r 's/^(.*)\.URL.*$/\1/g' <<< "$fullstring"
mnuLOCNMOD
$
You can use grep:
echo "mnuLOCNMOD.URL = javas" | grep -oP '\w+(?=\.URL)'
and assign the result to a string. I used a positive lookahead (?=regex) because it's a zero length assertion, meaning that it'll be matched but won't be displayed.
Run grep --help to find out what o and P flags stand for.
Parameter Expansion is the way to go.
If you are interested in a simple grep:
% fullstring="mnuLOCNMOD.URL = javascript:parent.doC...something"
% grep -o '^[^.]*' <<<"$fullstring"
mnuLOCNMOD
fullstring="mnuLOCNMOD.URL = javascript:parent.doC...something"
menuID=`echo $fullstring | cut -f 1 -d '.'`
here I used dot as a separator
this works in .sh files
To offer yet another alternative: Bash's regular-expression matching operator, =~:
fullstring="mnuLOCNMOD.URL = javascript:parent.doC...something"
echo "$([[ $fullstring =~ ^(.*)'.URL' ]] && echo "${BASH_REMATCH[1]}")"
Note how the (one and only) capture group ((.*)) is reported through element 1 of the special "${BASH_REMATCH[#]}" array variable.
While in this case l3x's parameter expansion solution is simpler, =~ generally offers more flexibility.
awk offers an easy solution as well:
echo "$(awk -F'\\.URL' '{ print $1 }' <<<"$fullstring")"
I used many times [``] to capture output of command to a variable. but with following code i am not getting right output.
#!/bin/bash
export XLINE='($ZWP_SCRIP_NAME),$ZWP_LT_RSI_TRIGGER)R),$ZWP_RTIMER'
echo 'Original XLINE'
echo $XLINE
echo '------------------'
echo 'Extract all word with $ZWP'
#works fine
echo $XLINE | sed -e 's/\$/\n/g' | sed -e 's/.*\(ZWP[_A-Z]*\).*/\1/g' | grep ZWP
echo '------------------'
echo 'Assign all word with $ZWP to XVAR'
#XVAR doesn't get all the values
export XVAR=`echo $XLINE | sed -e 's/\$/\n/g' | sed -e 's/.*\(ZWP[_A-Z]*\).*/\1/g' | grep ZWP` #fails
echo "$XVAR"
and i get:
Original XLINE
($ZWP_SCRIP_NAME),$ZWP_LT_RSI_TRIGGER)R),$ZWP_RTIMER
------------------
Extract all word with $ZWP
ZWP_SCRIP_NAME
ZWP_LT_RSI_TRIGGER
ZWP_RTIMER
------------------
Assign all word with $ZWP to XVAR
ZWP_RTIMER
why XVAR doesn't get all the values?
however if i use $() to capture the out instead of ``, it works fine. but why `` is not working?
Having GNU grep you can use this command:
XVAR=$(grep -oP '\$\KZWP[A-Z_]+' <<< "$XLINE")
If you pass -P grep is using Perl compatible regular expressions. The key here is the \K escape sequence. Basically the regex matches $ZWP followed by one or more uppercase characters or underscores. The \K after the $ removes the $ itself from the match, while its presence is still required to match the whole pattern. Call it poor man's lookbehind if you want, I like it! :)
Btw, grep -o outputs every match on a single line instead of just printing the lines which match the pattern.
If you don't have GNU grep or you care about portability you can use awk, like this:
XVAR=$(awk -F'$' '{sub(/[^A-Z_].*/, "", $2); print $2}' RS=',' <<< "$XLINE")
First, the smallest change that makes your code "work":
echo "$XLINE" | tr '$' '\n' | sed -e 's/.*\(ZWP[_A-Z]*\).*/\1/g' | grep ZWP_
The use of tr replaces a sed expression that didn't actually do what you thought it did -- try looking at its output to see.
One sane alternative would be to rely on GNU grep's -o option. If you can't do that...
zwpvars=( ) # create a shell array
zwp_assignment_re='[$](ZWP_[[:alnum:]_]+)(.*)' # ...and a regex
content="$XLINE"
while [[ $content =~ $zwp_assignment_re ]]; do
zwpvars+=( "${BASH_REMATCH[1]}" ) # found a reference
content=${BASH_REMATCH[2]} # stuff the remaining content aside
done
printf 'Found variable: %s\n' "${zwpvars[#]}"
I am not quite sure how to understand this-
$ var=' '
$ echo "|${var}|"
| |
$ echo "|${var// /}|"
||
Vs.
$ set -- '' '' ''
$ echo "|${*}|"
| |
$ echo "|${*// /}|"
| |
However, when I add this after the above
$ IFS=
$echo "|${*// /}|"
||
What is going wrong in the second set of commands? Is this the expected outcome?
Example 1
$ var=' '
$ echo "|${var}|"
| |
$ echo "|${var// /}|"
||
Here you have a simple string consisting of two spaces. When you expand it between two pipe characters, you see two spaces between the pipes. When you use pattern substitution to remove all the spaces from the expansion of the variable, you see the empty string between two pipes.
Example 2
$ set -- '' '' ''
First, you've set each of the first three positional parameters to the empty string. You can observe this by comparing the results of the ${1-foo} with {$4-foo} (which displays the parameter if set, but 'foo' if it is unset).
$ echo ${1-foo}
$ echo ${4-foo}
foo
So we can see that $1 is set, but null, while $4 is unset.
$ echo "|${*}|"
| |
Next, we see the result of expanding the special parameter $* inside quotation marks, which is a single string consisting of the positional parameters that are set, separated by the first character of the IFS parameter. IFS by default has a space as its first parameter, so what we see is a string that consists of 3 empty strings, each separated by a space, which is just a single string of 2 spaces.
$ echo "|${*// /}|"
| |
When you apply pattern substitution to $*, the substitution is applied to each positional parameter separately before the resulting parameters are joined using IFS. Since the positional parameters are already empty, removing spaces from them leaves them unchanged. So you get the same result as when you just expanded $* by itself.
Example 3
$ IFS=
$ echo "|${*// /}|"
||
The procedure here is the same as in example 2, with the important difference that now IFS is the null string, rather than its default of ''. Once again, the pattern substitution doesn't really do anything, because there are no spaces to remove from any of the positional parameters. But now, expanding $* results in a string consisting of the positional parameters with no intervening characters. Instead of $1 $2 $3, you get $1$2$3. Since all three are themselves empty strings, the result is the empty string.