Extract variables values from command output Bash Shell - bash

In bash shell I have a command that prints variable name and values such as:
Hello mate this is you variables:
MY_VAR1= this is the value of the first var
MY_VAR2= subvar1=27, subvar2=hello1
Have a good day!
The value of the variables in the output can contain characters as =,commas,;,:.., but I expect to find new line at the end of each variable value.
I need to create a short script which reads the values of MY_VAR1 and MY_VAR2.
So I need to end up with 2 variables as follows:
MY_VAR1 = this is the value of the first var
MY_VAR2 = subvar1=27, subvar2=hello1
I have a basic installation of CentOS 7, and I can't install additional stuff in that machine.
How can I achieve it?

I assume that the variable names are everything from the beginning of the lines, up to the = sign (excluded) with leading and trailing blanks removed (you cannot have blanks in your variable names).
If all you want is print the output you show, you can use something like:
while read line; do
if [[ $line =~ ^[[:blank:]]*([^[:blank:]]+)[[:blank:]]*=(.*)$ ]]; then
var="${BASH_REMATCH[1]}"
val="${BASH_REMATCH[2]}"
echo "$var = $val"
fi
done < <( my_command )
The =~ operator is a pattern matching with regular expressions. The regular expression ^[[:blank:]]*([^[:blank:]]+)[[:blank:]]*=(.*)$ models an output line of your command with variable assignment. It matches even if the variable name is surrounded with blanks. Two sub-expressions (enclosed in ()) are isolated: the variable name and the value. The BASH_REMATCH array contains the patterns that matched the sub-expressions in cells 1 and 2, respectively.
If you also want to assign the variables:
while read line; do
if [[ $line =~ ^[[:blank:]]*([^[:blank:]]+)[[:blank:]]*=(.*)$ ]]; then
var="${BASH_REMATCH[1]}"
val="${BASH_REMATCH[2]}"
echo "$var = $val"
declare $var="$val"
fi
done < <( my_command )

Related

Creating new shell variables in a loop

Through some here help I was to get only the text between quotation marks in a file like so:
text undefined but text
something something text something
shouldfindthis "dolphin"
butalsothis "elephant"
by doing:
i=0
regex='(".*?")' # Match all what is between "
while read line # read file line by line
do
if [[ $line =~ $regex ]]; then # If regex match
vars[$i]="${BASH_REMATCH[1]}" # store capturing group in a array
i=$((i + 1))
fi
done < txt # file "txt"
# Print what we found
if [ -v "vars" ]; then # if we found something , "vars" will exist
for var in "${vars[#]}"
do
echo "$var"
done
fi
Yielding an output:
$ ./script.sh
"dolphin"
"elephant"
Is there a way to mark these as variables? As such that dolphin is $text1 and elephant is $text2? in doing so that I can replace only the text within the paranthasis?
First: Using arrays, as your code already does, is the best-practice approach. Don't change it. However, if you really want to write to separate variables rather than array, then replace:
vars[$i]="${BASH_REMATCH[1]}" # this could also be vars+=( "${BASH_REMATCH[1]}" )
...with...
printf -v "text$i" '%s' "${BASH_REMATCH[1]}"
If you want to eliminate the quotes themselves, move them outside the grouping operator in your regex:
regex='"([^"]*)"'

substring extraction in bash

iamnewbie: this code is inefficient but it should extract the substring, the problem is with last echo statement,need some insight.
function regex {
#this function gives the regular expression needed
echo -n \'
for (( i = 1 ; i <= $1 ; i++ ))
do
echo -n .
done
echo -n '\('
for (( i = 1 ; i <= $2 ; i++ ))
do
echo -n .
done
echo -n '\)'
echo -n \'
}
# regex function ends
echo "Enter the string:"
read stg
#variable stg holds the string entered
if [ -z "$stg" ] ; then
echo "Null string"
exit
else
echo "Length of the $stg is:"
z=`expr "$stg" : '.*' `
#variable z holds the length of given string
echo $z
fi
echo "Enter the number of trailing characters to be extracted from $stg:"
read n
m=`expr $z - $n `
#variable m holds an integer value which is equal to total length - length of characters to be extracted
x=$(regex $m $n)
echo ` expr "$stg" : "$x" `
#the echo statement(above) is just printing a newline!! But not the result
What I intend to do with this code is, if I enter "racecar" and give "3" , it should display "car" which are the last three characters. Instead of displaying "car" its just printing a newline. Please correct this code rather than giving a better one.
Although you didn't ask for a better solution, it's worth mentioning:
$ n=3
$ stg=racecar
$ echo "${stg: -n}"
car
Note that the space after the : in ${stg: -n} is required. Without the space, the parameter expansion is a default-value expansion rather than a substring expansion. With the space, it's a substring expansion; -n is interpreted as an arithmetic expression (which means that n is interpreted as $n) and since the result is a negative number, it specifies the number of characters from the end to start the substring. See the Bash manual for details.
Your solution is based on evaluating the equivalent of:
expr "$stg" : '......\(...\)'
with an appropriate number of dots. It's important to understand what the above bash syntax actually means. It invokes the command expr, passing it three arguments:
arg 1: the contents of the variable stg
arg 2: :
arg 3: ......\(...\)
Note that there are no quotes visible. That's because the quotes are part of bash syntax, not part of the argument values.
If the value of stg had enough characters, the result of the above expr invocation would be to print out the 7th, 8th and 9th character of the value of stg`. Otherwise, it would print a blank line, and fail.
But that's not what you are doing. You're creating the regular expression:
'......\(...\)'
which has single quotes in it. Since single-quotes are not special characters in a regex, they match themselves; in other words, that pattern will match a string which starts with a single quote, followed by nine arbitrary characters, followed by another single quote. And if the string does match, it will print the three characters prior to the second single-quote.
Of course, since the regular expression you make has a . for every character in the target string, it won't match the target even if the target started and begun with a single-quote, since there would be too many dots in the regex to match that.
If you don't put single quotes into the regex, then your program will work, but I have to say that few times have I seen such an intensely circuitous implementation of the substring function. If you're not trying to win an obfuscated bash competition (a difficult challenge since most production bash code is obfuscated by nature), I'd suggest you use normal bash features instead of trying to do everything with regexen.
One of those is the syntax to determine the length of a string:
$ stg=racecar
$ echo ${#stg}
7
(although, as shown at the beginning, you don't actually even need that.)
What about:
$ n=3
$ string="racecar"
$ [[ "$string" =~ (.{$n})$ ]]
$ echo ${BASH_REMATCH[1]}
car
This looks for the last n characters at the end of the line. In a script:
#!/bin/bash
read -p "Enter a string: " string
read -p "Enter the number of characters you want from the end: " n
[[ "$string" =~ (.{$n})$ ]]
echo "These are the last $n characters: ${BASH_REMATCH[1]}"
You may want to add some more error handling, but this'll do it.
I'm not sure you need loops for this task. I wrote some example to get two parameters from user and cut the word according to it.
#!/bin/bash
read -p "Enter some word? " -e stg
#variable stg holds the string entered
if [ -z "$stg" ] ; then
echo "Null string"
exit 1
fi
read -p "Enter some number to set word length? " -e cutNumber
# check that cutNumber is a number
if ! [ "$cutNumber" -eq "$cutNumber" ]; then
echo "Not a number!"
exit 1
fi
echo "Cut first n characters:"
echo ${stg:$cutNumber}
echo
echo "Show first n characters:"
echo ${stg:0:$cutNumber}
echo "Alternative get last n characters:"
echo -n "$stg" | tail -c $cutNumber
echo
Example:
Enter some word? TheRaceCar
Enter some number to set word length? 7
Cut first n characters:
Car
Show first n characters:
TheRace
Alternative get last n characters:
RaceCar

Bash function to write value of variable into a text file with the name of the variable

I am currently working with a bash script that passes variables to some legacy code using text files.
The script sets the values of the variables some some legacy program (lp) by 1) creating variables with the prefix lp_ and 2) and then saving the values of these parameters as a text file in lp_run_directory. Here's a sample file:
#!/bin/bash
#Set the parameters here
lp_parameter_a=(0.2 0.4 1)
lp_parameter_b="TRUE"
lp_parameter_c=0
#Set the working directory here (put pwd for generality)
lp_run_directory=$(pwd)"/"
#Write parameter values in the directory
echo ${lp_parameter_a[#]} > $lp_run_directory"parameter_a.txt"
echo ${lp_parameter_b[#]} > $lp_run_directory"parameter_b.txt"
echo ${lp_parameter_c[#]} > $lp_run_directory"parameter_c.txt"
#Run program that depends on parameter values here...
lp_run_directory ="/users/ss/"
echo ${lp_parameter_a[#]} > lp_run_directory"parameter_a.txt"
echo ${lp_parameter_b[#]} > lp_run_directory"parameter_b.txt"
This works fine, but involves a lot of hardcoded text. I'm wondering if someone can help me define a function that can automate this in some way.
I'm not sure about what is possible in bash, but an ideal solution would be a function that takes the names of all of the variables in my workspace that begin with lp_ and then saves all of them to lp_run_directory.
Here's a bash function that takes an output dir. and a variable prefix as its arguments and outputs each matching variable's value into a file named for the variable (sans prefix) in the output dir.
varsToFiles() {
local outDir=$1 prefix=$2 name fname rest isArray
while IFS='=' read -r name rest; do
# Determine the output filename - the variable name w/o prefix.
fname=${name#$prefix}
# Determine if the variable is an array.
[[ $(declare -p "$name" | cut -d' ' -f2) == *a* ]] && isArray=1 || isArray=0
(( isArray )) && name="$name[#]"
# Output var. value and send to output file.
echo "${!name}" > "$outDir/$fname"
done < <(shopt -so posix; set | egrep "^${prefix}[^ =]*=")
}
# Invoke the function.
varsToFiles '/users/ss' 'lp_'
Note:
As in the question, elements of array variables are output simply with echo, so that the partitioning into individual elements is lost in case of elements with embedded whitespace (this could easily be fixed).
The input to the while loop:
shopt -so posix; set prints all variables and their values; shopt -so posix ensures that only variables are printed and not also functions.
egrep "^${prefix}[^ =]*=" filters the output down to variables whose names start with the specified prefix.
The while loop:
while IFS='=' read -r name rest loops over all input lines and parses them into the variable name (and the rest of the line, which is not otherwise used in this function).
fname=${name#$prefix} determines the output filename by stripping the prefix from the variable name using bash's parameter expansion.
[[ $(declare -p "$name" | cut -d' ' -f2) == *a* ]] determines if the variable at hand is an array variable or not - declare -p outputs a as part of the 2nd field for arrays.
(( isArray )) && name="$name[#]": array variables: an additional indirection step is required beforehand in order to output all array elements at once: the all-elements subscript [#] is appended to the name first, and then variable indirection is performed in the next step.
echo "${!name} echoes each matching variable's value using bash's variable indirection and send the output to the output file (> "$outDir/$fname")

associative array in bash

I found an implementation of the associative array here and would like to understand what the code actually does. Here are the piece of the code I don't understand, and would appreciate the explanation.
'
put() {
if [ "$#" != 3 ]; then exit 1; fi
mapName=$1; key=$2; value=`echo $3 | sed -e "s/ /:SP:/g"` #dont understand
eval map="\"\$$mapName\"" **#dont understand**
map="`echo "$map" | sed -e "s/--$key=[^ ]*//g"` --$key=$value" #dont understand
eval $mapName="\"$map\"" #dont understand
}
get() {
mapName=$1; key=$2
map=${!mapName}
#dont understand
value="$(echo $map |sed -e "s/.*--${key}=\([^ ]*\).*/\1/" -e 's/:SP:/ /g' )"
}
getKeySet() {
if [ "$#" != 1 ];
then
exit 1;
fi
mapName=$1;
eval map="\"\$$mapName\""
keySet=`
echo $map |
sed -e "s/=[^ ]*//g" -e "s/\([ ]*\)--/\1/g" #dont understand
`
}
Thanks.
So first in order of don't understands:
this simply checks you always have 3 arguments to the function and if different number is provided exits with 1(error)
This escapes the space chars by replacing them with :SP: so Hi how are you becomes Hi:SP:how:SP:are:SP:you
The map is stored in a var with name the first argument provided to put. So this line adds to the var the following text --$key=$value and here key is the second argument and value is the escaped third argument
get simply stores in value the value for the key provided as second argument(first one is the name of the map)
To understand better try printing the variable you decided to use for your map after each operation. It's easy you'll see ;)
Hope this makes it clear.
I'll try to explain every line you highlighted:
if [ "$#" != 3 ]; then exit 1; fi #dont understand
If there aren't exactly three arguments, exit with error.
mapName=$1; key=$2; value=`echo $3 | sed -e "s/ /:SP:/g"` #dont understand
Set the variable $mapName with the value of first argument
Set the variable $key with the value of second argument
Set the variable $value with the value of third argument, after replacing spaces with the string :SP:.
map="`echo "$map" | sed -e "s/--$key=[^ ]*//g"` --$key=$value" #dont understand
This will edit the $map variable, by removing the first occurrance of the value of $key followed by = and then by non-space characters, and then append the string -- followed by the value of $key, then by = and finally the value of $value.
eval $mapName="\"$map\"" #dont understand
This will evaluate a string that was generated by that line. Suppose $mapName is myMap and $map is value, the string that bash will evaluate is:
myMap="value"
So it will actually set a variable, for which its name will be passed by a parameter.
map=${!mapName}
This will set the variable $map with the value the variable that has the same name as the value in $mapName. Example: suppose $mapName has a, then $map would end up with the contents of a.
value="$(echo $map |sed -e "s/.*--${key}=\([^ ]*\).*/\1/" -e 's/:SP:/ /g' )"
Here we set the value of the $value variable as the value of the $map variable, after a couple of edits:
Extract only the contents that are specified in the expression between parenthesis in the sed expression, which matches characters that aren't spaces. The text before it specifies where the match should start, so in this case the spaces must start after the string -- followed by the value of the $key variable, followed by =. The `.*' at the start and the end matches the rest of the line, and is used in order to remove them afterwards.
Restore spaces, ie. replace :SP: with actual spaces.
eval map="\"\$$mapName\""
This creates a string with the value "$mapName", ie. a dollar, followed by the string contained in mapName, surrounded by double quotes. When evaluated, this gets the value of the variable whose name is the contents of $mapName.
Hope this helps a little =)
Bash, since version 4 and upwards, has builtin associative arrays.
To use them, you need to declare one with declare -A
#!/usr/bin/env bash
declare -A arr # 'arr' will be an associative array
# now fill it with: arr['key']='value'
arr=(
[foo]="bar"
["nyan"]="cat"
["a b"]="c d"
["cookie"]="yes please"
["$USER"]="$(id "$LOGNAME")"
)
# -- cmd -- -- output --
echo "${arr[foo]}" # bar
echo "${arr["a b"]}" # c d
user="$USER"
echo "${arr["$user"]}" # uid=1000(c00kiemon5ter) gid=100...
echo "${arr[cookie]}" # yes please
echo "${arr[cookies]}" # <no output - empty value> :(
# keys and values
printf '%s\n' "${arr[#]}" # print all elements, each on a new line
printf '%s\n' "${!arr[#]}" # print all keys, each on a new line
# loop over keys - print <key :: value>
for key in "${!arr[#]}"
do printf '%s :: %s\n' "$key" "${arr["$key"]}"
done
# you can combine those with bash parameter expansions, like
printf '%s\n' "${arr[#]:0:2}" # print only first two elements
echo "${arr[cookie]^^}" # YES PLEASE

Changing from string to array not working as expected in Bash

I want to put the result of the sort method into an array where each cell contains a word. I tried this code but only part of the $file is printed and its not sorted:
#!/bin/bash
for file in `ls ${1}` ; do
if [[ ! ($file = *.user) ]] ; then
continue
fi
arr=(`sort -nrk4 $file`)
echo ${arr[*]}
done
Why isnt this working? How can I do this?
Data file:
name1 01/01/1994 a 0
name2 01/01/1994 b 5
name3 01/01/1994 c 2
If I run the sort line only (sort -nrk4 $file), this is whats printed:
name2 01/01/1994 b 5
name3 01/01/1994 c 2
name1 01/01/1994 a 0
When I run the 2 lines above, this is what its printed:
name1 01/01/1994 a 0
In order for each line of the sort output to be put into its own array element, IFS needs to be set to a newline. To output the array, you need to iterate over it.
#!/bin/bash
for file in $1; do
if [[ ! $file = *.user ]]; then
continue
fi
saveIFS=$IFS
IFS=$'\n'
arr=($(sort -nrk4 "$file"))
IFS=$saveIFS
for i in "${arr[#]}"
do
echo "$i"
done
done
Alternatively, don't reset IFS, don't use a loop for output and the following command will output the elements separated by newlines since the first character of IFS is used for the output separator when * is used for the subscript in a quoted context:
echo "${arr[*]}"
Remember, quoting fixes everything.
Don't use ls in a for loop. If $1 may contain globbing, then it should be left unquoted. It not, then it should be quoted in case the directory or filename may contain whitespace characters. Ideally, the contents of $1 should be validated before they're used here.
The comparison operators have higher precedence than the negation operator so the parentheses are unnecessary.
For command substitution, $() is preferred over backticks. They're more readable and easier to use nested.
And finally, when variables are expanded, the should be quoted. Also, # should almost always be used to subscript arrays.

Resources