What is the correct syntax to combine multiple parameter expansions? - bash

My current code:
while read -r rbv_line || [[ -n "$rbv_line" ]]; do
if [[ "${rbv_line}" =~ ${rbv_reg} ]]; then
rbv_downcase="${BASH_REMATCH[0],,}" &&
ruby_version="${rbv_downcase//[^0-9a-z\.\-]/}" &&
((reg_matches="${reg_matches}"+1))
printf "\n"
printf "Setting Ruby version: %s\n" "${ruby_version}"
break
fi
done < "${1}"
It does what I want. But I would love to know if I can simplify this code even more, hoping someone can help me understand the syntax.
If you see these two lines:
rbv_downcase="${BASH_REMATCH[0],,}" &&
ruby_version="${rbv_downcase//[^0-9a-z\.\-]/}" &&
Initially I tried to combine those into one using something like this:
ruby_version="${BASH_REMATCH[0],,//[^0-9a-z\.\-]/}"
That does not work.
Is there a way to combine those two parameter expansions (,, and the //[^0-9a-z\.\-]/) or is passing it through an intermediary variable the right approach?
You can view the latest version of the code here:
https://github.com/octopusnz/scripts

You cannot combine multiple parameter expansions, but...
... you can simplify this code!
The biggest gain is by using already available tools.
Instead of looping, let's use grep. It's supposed to do something when RegEx pattern is occurred, so:
grep -E "$rbv_reg" "$1" # -E is for extended RegEx
I guess your pattern isn't case sensitive, so let's disable it with -i flag.
The loop breaks after match, so let's pass -m 1 to stop processing file after first match.
You want to convert uppercase to lowercase, so let's pipe it through tr:
grep -m 1 -E -i "$rbv_reg" "$1" | tr '[:upper:]' '[:lower:]'
You then replace some characters with //[^0-9a-z\.\-]/, piping it to sed will do the trick:
grep -m 1 -E -i "$rbv_reg" "$1" | tr '[:upper:]' '[:lower:]' | sed 's/[^0-9a-z\.\-]//g'
And at the very end, let's grab the output to variable:
ruby_version="$( grep -m 1 -E -i '$rbv_reg' '$1' | tr '[:upper:]' '[:lower:]' | sed 's/[^0-9a-z\.\-]//g' )"
Since you are printing new line anyway, let's use simple echo instead of printf
All what's left is if [ -n "$ruby_version" ] to increment reg_matches
At the end, we got:
ruby_version="$(
grep -m 1 -E -i '$rbv_reg' '$1' |
tr '[:upper:]' '[:lower:]' |
sed 's/[^0-9a-z\.\-]//g'
)"
if [ -n "$ruby_version" ]; then
reg_matches="$((reg_matches+1))"
echo
echo "Setting Ruby version: $ruby_version"
fi
The advantage of above code is the fact it isn't really Bash dependent and should work in any POSIX Bourne compatible shell.

Related

bash : change part of filename to lowercase

I need to rename a list of files changing any file extension to lowercase:
ie: from My_TEST.ONE.two.Three.fOuR.FIve to My_TEST.one.two.three.four.five
At the moment the way I've found is this one
#!/bin/bash
sourcefilename="My_TEST.ONE.two.Three.fOuR.FIve"
newfilename=""
for word in $(echo $sourcefilename | tr '.' '\n'); do
if [ -z "$newfilename" ]; then
newfilename="$word"
else
newfilename="$newfilename.$(echo $word | tr [:upper:] [:lower:])"
fi
done
Is there a better (and maybe elegant) approach?
Use bash Parameter Expansion features.
fileName='My_TEST.ONE.two.Three.fOuR.FIve'
first="${fileName%%.*}"
rest="${fileName#*.}"
echo mv -v "${fileName}" "${first}.${rest,,[A-Z]}"

Inline array substitution

I have file with a few lines:
x 1
y 2
z 3 t
I need to pass each line as paramater to some program:
$ program "x 1" "y 2" "z 3 t"
I know how to do it with two commands:
$ readarray -t a < file
$ program "${a[#]}"
How can i do it with one command? Something like that:
$ program ??? file ???
The (default) options of your readarray command indicate that your file items are separated by newlines.
So in order to achieve what you want in one command, you can take advantage of the special IFS variable to use word splitting w.r.t. newlines (see e.g. this doc) and call your program with a non-quoted command substitution:
IFS=$'\n'; program $(cat file)
As suggested by #CharlesDuffy:
you may want to disable globbing by running beforehand set -f, and if you want to keep these modifications local, you can enclose the whole in a subshell:
( set -f; IFS=$'\n'; program $(cat file) )
to avoid the performance penalty of the parens and of the /bin/cat process, you can write instead:
( set -f; IFS=$'\n'; exec program $(<file) )
where $(<file) is a Bash equivalent to to $(cat file) (faster as it doesn't require forking /bin/cat), and exec consumes the subshell created by the parens.
However, note that the exec trick won't work and should be removed if program is not a real program in the PATH (that is, you'll get exec: program: not found if program is just a function defined in your script).
Passing a set of params should be more organized :
In this example case I'm looking for a file containing chk_disk_issue=something etc.. so I set the values by reading a config file which I pass in as a param.
# -- read specific variables from the config file (if found) --
if [ -f "${file}" ] ;then
while IFS= read -r line ;do
if ! [[ $line = *"#"* ]]; then
var="$(echo $line | cut -d'=' -f1)"
case "$var" in
chk_disk_issue)
chk_disk_issue="$(echo $line | tr -d '[:space:]' | cut -d'=' -f2 | sed 's/[^0-9]*//g')"
;;
chk_mem_issue)
chk_mem_issue="$(echo $line | tr -d '[:space:]' | cut -d'=' -f2 | sed 's/[^0-9]*//g')"
;;
chk_cpu_issue)
chk_cpu_issue="$(echo $line | tr -d '[:space:]' | cut -d'=' -f2 | sed 's/[^0-9]*//g')"
;;
esac
fi
done < "${file}"
fi
if these are not params then find a way for your script to read them as data inside of the script and pass in the file name.

How to filter an ordered list stored into a string

Is it possible in bash to filter out a part of a string with another given string ?
I have a fixed list of motifs defined in a string. The order IS important and I want to keep only the parts that are passed as a parameter ?
myDefaultList="s,t,a,c,k" #order is important
toRetains="k,t,c,u" #provided by the user, order is not enforced
retained=filter $myDefaultList $toRetains # code to filter
echo $retained # will print t,c,k"
I can write an ugly method that will use IFS, arrays and loops, but I wonder if there's a 'clever' way to do that, using built-in commands ?
here is another approach
tolines() { echo $1 | tr ',' '\n'; }
grep -f <(tolines "$toRetains") <(tolines "$myDefaultList") | paste -sd,
will print
t,c,k
assign to a variable as usual.
Since you mention in your comments that you are open to sed/awk , check also this with GNU awk:
$ echo "$a"
s,t,a,c,k
$ echo "$b"
k,t,c,u
$ awk -v RS=",|\n" 'NR==FNR{a[$1];next}$1 in a{printf("%s%s",$1,RT)}' <(echo "$b") <(echo "$a")
t,c,k
#!/bin/bash
myDefaultList="s,t,a,c,k"
toRetains="s,t,c,u"
IFS=","
for i in $myDefaultList
do
echo $toRetains | grep $i > /dev/null
if [ "$?" -eq "0" ]
then
retained=$retained" "$i
fi
done
echo $retained | sed -e 's/ /,/g' -e 's/,//1'
I have checked it running for me. Kindly check.

bash: sed: unexpected behavior: displays everything

I wrote what I thought was a quick script I could run on a bunch of machines. Instead it print what looks like might be directory contents in a recursive search:
version=$(mysql Varnish -B --skip-column-names -e "SELECT value FROM sys_param WHERE param='PatchLevel'" | sed -n 's/^.*\([0-9]\.[0-9]*\).*$/\1/p')
if [[ $(echo "if($version == 6.10) { print 1; } else { print 0; }" | bc) -eq 1 ]]; then
status=$(dpkg-query -l | awk '{print $2}' | grep 'sg-status-polling');
cons=$(dpkg-query -l | awk '{print $2}' | grep 'sg-consolidated-poller');
if [[ "$status" != "" && "$cons" != "" ]]; then
echo "about to change /var/www/Varnish/lib/Extra/SG/ObjectPoller2.pm"; echo;
cp /var/www/Varnish/lib/Extra/SG/ObjectPoller2.pm /var/www/Varnish/lib/Extra/SG/ObjectPoller2.pm.bkup;
sed -ir '184s!\x91\x93!\x91\x27--timeout=35\x27\x93!' /var/www/Varnish/lib/Extra/SG/ObjectPoller2.pm;
sed -n 183,185p /var/www/Varnish/lib/Extra/SG/ObjectPoller2.pm; echo;
else
echo "packages not found. Assumed to be not applicable";
fi
else
echo "This is 4.$version, skipping";
fi
The script is supposed to make sure Varnish is version 4.6.10 and has 2 custom .deb packages installed (not through apt-get). then makes a backup and edits a single line in a perl module from [] to ['--timeout=35']
it looks like its tripping up on the sed replace one liner.
There are two major problems (minor ones addressed in comments). The first is that you use the decimal code for [] instead of the hexa, so you should use \x5b\x5d instead of \x91\x93. The second problem is that if you do use the proper codes, sed will still interpret those syntactically as []. So you can't escape escaping. Here's what you should call:
sed -ri'.bkup' '184s!\[\]![\x27--timeout=35\x27]!' /var/www/Varnish/lib/Extra/SG/ObjectPoller2.pm
And this will create the backup for you (but you should double check).

Speed up bash filter function to run commands consecutively instead of per line

I have written the following filter as a function in my ~/.bash_profile:
hilite() {
export REGEX_SED=$(echo $1 | sed "s/[|()]/\\\&/g")
while read line
do
echo $line | egrep "$1" | sed "s/$REGEX_SED/\x1b[7m&\x1b[0m/g"
done
exit 0
}
to find lines of anything piped into it matching a regular expression, and highlight matches using ANSI escape codes on a VT100-compatible terminal.
For example, the following finds and highlights the strings bin, U or 1 which are whole words in the last 10 lines of /etc/passwd:
tail /etc/passwd | hilite "\b(bin|[U1])\b"
However, the script runs very slowly as each line forks an echo, egrep and sed.
In this case, it would be more efficient to do egrep on the entire input, and then run sed on its output.
How can I modify my function to do this? I would prefer to not create any temporary files if possible.
P.S. Is there another way to find and highlight lines in a similar way?
sed can do a bit of grepping itself: if you give it the -n flag (or #n instruction in a script) it won't echo any output unless asked. So
while read line
do
echo $line | egrep "$1" | sed "s/$REGEX_SED/\x1b[7m&\x1b[0m/g"
done
could be simplified to
sed -n "s/$REGEX_SED/\x1b[7m&\x1b[0m/gp"
EDIT:
Here's the whole function:
hilite() {
REGEX_SED=$(echo $1 | sed "s/[|()]/\\\&/g");
sed -n "s/$REGEX_SED/\x1b[7m&\x1b[0m/gp"
}
That's all there is to it - no while loop, reading, grepping, etc.
If your egrep supports --color, just put this in .bash_profile:
hilite() { command egrep --color=auto "$#"; }
(Personally, I would name the function egrep; hence the usage of command).
I think you can replace the whole while loop with simply
sed -n "s/$REGEX_SED/\x1b[7m&\x1b[0m/gp"
because sed can read from stdin line-by-line so you don't need read
I'm not sure if running egrep and piping to sed is faster than using sed alone, but you can always compare using time.
Edit: added -n and p to sed to print only highlighted lines.
Well, you could simply do this:
egrep "$1" $line | sed "s/$REGEX_SED/\x1b[7m&\x1b[0m/g"
But I'm not sure that it'll be that much faster ; )
Just for the record, this is a method using a temporary file:
hilite() {
export REGEX_SED=$(echo $1 | sed "s/[|()]/\\\&/g")
export FILE=$2
if [ -z "$FILE" ]
then
export FILE=~/tmp
echo -n > $FILE
while read line
do
echo $line >> $FILE
done
fi
egrep "$1" $FILE | sed "s/$REGEX_SED/\x1b[7m&\x1b[0m/g"
return $?
}
which also takes a file/pathname as the second argument, for case like
cat /etc/passwd | hilite "\b(bin|[U1])\b"

Resources