Storing Bash function parameters in variables - bash

I want to define and get all the required variables that are passed to a function as parameters:
function explain_vars() {
echo "Explaining vars '$#':" >&2
for _var in "$#"; do
printf " $_var: '${!_var}'\n" >&2
done
printf "\n" >&2
}
function read_params() (
## Define local variables
_vars=(v1 v2 v3)
local ${_vars[#]}
## Read variables from input:
read ${_vars[#]} <<< "$#"
explain_vars "${_vars[#]}"
)
The read takes puts all parameters in the specified variables, default delimiter here is space. So if I pass different strings as second parameter, read will store only the first string in the second parameter, and all the rest to the following parameters:
$ read_params one "two dot one" "three" "four"
Explaining vars 'v1 v2 v3':
v1: 'one'
v2: 'two'
v3: 'dot one three four'
As we can see, variable v2 is not synchronized with given parameters anymore. Moreover, it fails at reading empty strings:
$ read_params one "" " " '' ' ' "two dot one" "three" "four"
Explaining vars 'v1 v2 v3':
v1: 'one'
v2: 'two'
v3: 'dot one three four'
By looping through the all-parameters variable $# inside the function it is possible to distinguish variables:
function raw_params() (
echo "Explaining row parameters:"
for _v in "${#}"; do
printf " '$_v'\n"
done
)
$ raw_params one "" " " '' ' ' "two dot one" "three" "four"
Explaining row parameters:
'one'
''
' '
''
' '
'two dot one'
'three'
'four'
To me the read command offers benefit and quickness at defining, controlling and checking requested parameters that are passed to functions. However this works only for single and non-empty stringed parameters. Is it possible to read all different parameters in variables like the read command does, but respecting spaces and empty parameters? Or is there a better approach maybe?

From the original question seems read command is not correctly understood, read is a builtin which reads one line for standard input, IFS environment variable is used as field separator and -d option allows to change record delimiter (default is newline), for more information see read in bash manual.
The function arguments are retrieved using special variable "$#", bash syntax to assign an array is just
_vars=( "$#" ) # no space between variable name and = and no space between = and (
As a space is not valid in a variable name ${!_var} expansion will fail writing an error bash: ...: bad substitution, if _var contains an expression with a space.
function keyword is useless because of (), the use of parenthesis around the body of function instead of braces { ;} starts a new sub-shell.

I'm not sure what you are hoping to accomplish with this code, but this would appear to solve your problem for the case of three input parameters. Perhaps it shows you a way forward even if it doesn't completely do what you want.
read_params () (
## Define local variables
_vars=(v1 v2 v3)
local ${_vars[#]}
local i
for ((i=1; i<=$#; ++i)); do
## Read variables from input:
printf -v "${_vars[i-1]}" "${!i}"
done
explain_vars "${_vars[#]}"
)

I rewrote my function in a script according to tripleee's answer. printf works fine except for assigning empty values ('' and "") to arrays. For that, we need to format the string we pass to printf -v with '%s'. For the example I pass an array of parameters to my function twice: once per assigning them to my regular local parameters; a second time to pass the same parameters to my last local array variable. Here is the code:
$ cat parse-local-vars.sh
#!/usr/bin/env bash
function explain_vars() {
echo "Explaining vars '$#':" >&2
for _var in "$#"; do
printf " $_var: '${!_var}'\n" >&2
done
}
function parse_params() (
#
# Stores given parameters in defined local variables _vars.
# Last variable will be treated as an array and
# remaining parameters will be stored therein.
#
## Define local variables
local _vars=(v1 v_empty_double_quote v_spaced_double_quote v_empty_single_quote v_spaced_single_quote v2_1 v3 v4 args)
local ${_vars[#]}
## Make sure we assign parameters to variables
[ ${#_vars[#]} -gt 0 ] \
|| return 1
_args_pos=$(( ${#_vars[#]}-1 ))
_args_counter=0
local p
for ((p=1; p<=$#; ++p)); do
## Read variables from input:
if [ $p -le $_args_pos ]; then
#printf -v "${_vars[p-1]}" '%s' "${!p}"
printf -v "${_vars[p-1]}" "${!p}"
else
#printf -v "${_vars[_args_pos]}[$_args_counter]" '%s' "${!p}"
printf -v "${_vars[_args_pos]}[$_args_counter]" "${!p}" # Without the '%s' assigning empty variable to an array does not work
_args_counter=$(( _args_counter+1 ))
fi
done
explain_vars "${_vars[#]}"
echo "exlaining array args[#]: '${args[#]}'"
for _v in "${args[#]}"; do
echo " >'$_v'"
done
)
params_to_test=(one "" " " '' ' ' "two dot one" "three" "four")
parse_params "${params_to_test[#]}" "${params_to_test[#]}"
As one can see, here I use printf -v without formatting the parameter string (no use of '%s'):
$ bash parse-local-vars.sh
Explaining vars 'v1 v_empty_double_quote v_spaced_double_quote v_empty_single_quote v_spaced_single_quote v2_1 v3 v4 args':
v1: 'one'
v_empty_double_quote: ''
v_spaced_double_quote: ' '
v_empty_single_quote: ''
v_spaced_single_quote: ' '
v2_1: 'two dot one'
v3: 'three'
v4: 'four'
args: 'one'
exlaining array args[#]: 'one two dot one three four' (6 values)
>'one'
>' '
>' '
>'two dot one'
>'three'
>'four'
The empty parameters "" and '' are not passed to the array (6 values).
Allowing the string formatting of printf by toggling the following comments:
printf -v "${_vars[_args_pos]}[$_args_counter]" '%s' "${!p}"
#printf -v "${_vars[_args_pos]}[$_args_counter]" "${!p}" # Without the '%s' as,signing empty variable to an array does not work
Leads to this output:
$ bash parse-local-vars.sh
Explaining vars 'v1 v_empty_double_quote v_spaced_double_quote v_empty_single_quote v_spaced_single_quote v2_1 v3 v4 args':
v1: 'one'
v_empty_double_quote: ''
v_spaced_double_quote: ' '
v_empty_single_quote: ''
v_spaced_single_quote: ' '
v2_1: 'two dot one'
v3: 'three'
v4: 'four'
args: 'one'
exlaining array args[#]: 'one two dot one three four' (8 values)
>'one'
>''
>' '
>''
>' '
>'two dot one'
>'three'
>'four'
This is the expected results, as empty strings are correctly assigned to the array (8 values). I haven't figured out what is going on with passing empty strings to an array with printf -v, but using formatted strings with printf -v seems to be the safe way to go. Any correction, explanation and improvements are welcome.

Related

How to concatenate string to comma-separated element in bash

I am new to Bash coding. I would like to concatenate a string to each element of a comma-separated strings "array".
This is an example of what I have in mind:
s=a,b,c
# Here a function to concatenate the string "_string" to each of them.
# Expected result:
a_string,b_string,c_string
One way:
$ s=a,b,c
$ echo ${s//,/_string,}_string
a_string,b_string,c_string
Using a proper array is generally a much more robust solution. It allows the values to contain literal commas, whitespace, etc.
s=(a b c)
printf '%s\n' "${s[#]/%/_string}"
As suggested by chepner, you can use IFS="," to merge the result with commas.
(IFS=","; echo "${s[#]/%/_string}")
(The subshell is useful to keep the scope of the IFS reassignment from leaking to the current shell.)
Simply, you could use a for loop
main() {
local input='a,b,c'
local append='_string'
# create an 'output' variable that is empty
local output=
# convert the input into an array called 'items' (without the commas)
IFS=',' read -ra items <<< "$input"
# loop over each item in the array, and append whatever string we want, in this case, '_string'
for item in "${items[#]}"; do
output+="${item}${append},"
done
# in the loop, the comma was re-added back. now, we must remove the so there are only commas _in between_ elements
output=${output%,}
echo "$output"
}
main
I've split it up in three steps:
Make it into an actual array.
Append _string to each element in the array using Parameter expansion.
Turn it back into a scalar (for which I've made a function called turn_array_into_scalar).
#!/bin/bash
function turn_array_into_scalar() {
local -n arr=$1 # -n makes `arr` a reference the array `s`
local IFS=$2 # set the field separator to ,
arr="${arr[*]}" # "join" over IFS and assign it back to `arr`
}
s=a,b,c
# make it into an array by turning , into newline and reading into `s`
readarray -t s < <(tr , '\n' <<< "$s")
# append _string to each string in the array by using parameter expansion
s=( "${s[#]/%/_string}" )
# use the function to make it into a scalar again and join over ,
turn_array_into_scalar s ,
echo "$s"

How to iterate over multiple variables and echo them using Shell Script?

Consider the below variables which are dynamic and might change each time. Sometimes there might even be 5 variables, But the length of all the variables will be the same every time.
var1='a b c d e... upto z'
var2='1 2 3 4 5... upto 26'
var3='I II III IV V... upto XXVI'
I am looking for a generalized approach to iterate the variables in a for loop & My desired output should be like below.
a,1,I
b,2,II
c,3,III
d,4,IV
e,5,V
.
.
goes on upto
z,26,XXVI
If I use nested loops, then I get all possible combinations which is not the expected outcome.
Also, I know how to make this work for 2 variables using for loop and shift using below link
https://unix.stackexchange.com/questions/390283/how-to-iterate-two-variables-in-a-sh-script
With paste
paste -d , <(tr ' ' '\n' <<<"$var1") <(tr ' ' '\n' <<<"$var2") <(tr ' ' '\n' <<<"$var3")
a,1,I
b,2,II
c,3,III
d,4,IV
e...z,5...26,V...XXVI
But clearly having to add other parameter substitutions for more varN's is not scalable.
You need to "zip" two variables at a time.
var1='a b c d e...z'
var2='1 2 3 4 5...26'
var3='I II III IV V...XXVI'
zip_var1_var2 () {
set $var1
for v2 in $var2; do
echo "$1,$v2"
shift
done
}
zip_var12_var3 () {
set $(zip_var1_var2)
for v3 in $var3; do
echo "$1,$v3"
shift
done
}
for x in $(zip_var12_var3); do
echo "$x"
done
If you are willing to use eval and are sure it is safe to do so, you can write a single function like
zip () {
if [ $# -eq 1 ]; then
eval echo \$$1
return
fi
a1=$1
shift
x=$*
set $(eval echo \$$a1)
for v in $(zip $x); do
printf '=== %s\n' "$1,$v" >&2
echo "$1,$v"
shift
done
}
zip var1 var2 var3 # Note the arguments are the *names* of the variables to zip
If you can use arrays, then (for example, in bash)
var1=(a b c d e)
var2=(1 2 3 4 5)
var3=(I II III IV V)
for i in "${!var1[#]}"; do
printf '%s,%s,%s\n' "${var1[i]}" "${var2[i]}" "${var3[i]}"
done
Use this Perl one-liner:
perl -le '#in = map { [split] } #ARGV; for $i ( 0..$#{ $in[0] } ) { print join ",", map { $in[$_][$i] } 0..$#in; }' "$var1" "$var2" "$var3"
Prints:
a,1,I
b,2,II
c,3,III
d,4,IV
e,5,V
z,26,XXVI
The Perl one-liner uses these command line flags:
-e : Tells Perl to look for code in-line, instead of in a file.
-l : Strip the input line separator ("\n" on *NIX by default) before executing the code in-line, and append it when printing.
The input variables must be quoted with double quotes "like so", to keep the blank-separated words from being treated as separate arguments.
#ARGV is an array of the command line arguments, here $var1, $var2, $var3.
#in is an array of 3 elements, each element being a reference to an array obtained as a result of splitting the corresponding element of #ARGV on whitespace. Note that split splits the string on whitespace by default, but you can specify a different delimiter, it accepts regexes.
The subsequent for loop prints #in elements separated by comma.
SEE ALSO:
perldoc perlrun: how to execute the Perl interpreter: command line switches
perldoc perlvar: Perl predefined variables
The following is (almost) a copy of this answer with a few tweaks that make it fit this question.
The Original Question
First let’s assign a few variables to play with, 26 tokens in each of them:
var1="$(echo {a..z})"
var2="$(echo {1..26})"
var3="$(echo I II III IV \
V{,I,II,III} IX \
X{,I,II,III} XIV \
XV{,I,II,III} XIX \
XX{,I,II,III} XXIV \
XXV XXVI)"
var4="$(echo {A..Z})"
var5="$(echo {010101..262626..10101})"
Now we want a “magic” function that zips an arbitrary number of variables, ideally in pure Bash:
zip_vars var1 # a trivial test
zip_vars var{1..2} # a slightly less trivial test
zip_vars var{1..3} # the original question
zip_vars var{1..4} # more vars, becasuse we can
zip_vars var{1..5} # more vars, because why not
What could zip_vars look like? Here’s one in pure Bash, without any external commands:
zip_vars() {
local var
for var in "$#"; do
local -a "array_${var}"
local -n array_ref="array_${var}"
array_ref=(${!var})
local -ar "array_${var}"
done
local -n array_ref="array_${1}"
local -ir size="${#array_ref[#]}"
local -i i
local output
for ((i = 0; i < size; ++i)); do
output=
for var in "$#"; do
local -n array_ref="array_${var}"
output+=",${array_ref[i]}"
done
printf '%s\n' "${output:1}"
done
}
How it works:
It splits all variables (passed by reference (by variable name)) into arrays. For each variable varX it creates a local array array_varX.
It would be actually way easier if the input variables were already Bash arrays to start with (see below), but … we stick with the original question initially.
It determines the size of the first array and then blindly expects all arrays to be of that size.
For each index i from 0 to size - 1 it concatenates the ith elements of all arrays, separated by ,.
Arrays Make Things Easier
If you use Bash arrays from the very start, the script will be shorter and look simpler and there won’t be any string-to-array conversions.
zip_arrays() {
local -n array_ref="$1"
local -ir size="${#array_ref[#]}"
local -i i
local output
for ((i = 0; i < size; ++i)); do
output=
for arr in "$#"; do
local -n array_ref="$arr"
output+=",${array_ref[i]}"
done
printf '%s\n' "${output:1}"
done
}
arr1=({a..z})
arr2=({1..26})
arr3=( I II III IV
V{,I,II,III} IX
X{,I,II,III} XIV
XV{,I,II,III} XIX
XX{,I,II,III} XXIV
XXV
XXVI)
arr4=({A..Z})
arr5=({010101..262626..10101})
zip_arrays arr1 # a trivial test
zip_arrays arr{1..2} # a slightly less trivial test
zip_arrays arr{1..3} # (almost) the original question
zip_arrays arr{1..4} # more arrays, becasuse we can
zip_arrays arr{1..5} # more arrays, because why not

Parsing command output and build bash key-values

I have a command which gives output as following
2 physical slots found:
slot 1:
Cstatus: present
Sstatus: active
apps: 0
slot 2:
Cstatus: present
Sstatus: inactive
apps: 0
I want to build Key-value pair in bash script from this output like slot1[Cstatus] as "present" , slot2[Sstatus] as "inactive".
Any pointers will be very helpful.
I tried this which gives status for each slot entry but i want a better solution
slot1_status=$(awk '/slot 1:/ { for(i=1; i<3; i++) { getline; if(match($0, /Sstatus:/)) {print $3}}}' slot_status.txt)
Thanks in advance,
prap4learn
Here is a sample awk solution:
script.awk
BEGIN { # pre process initial variable
FS = ":"; # set field separator to "|"
}
/slot/ { # for each line having "slot"
currentSlot = $1;
}
!/slot/ { # for each line not having "slot"
sub("^[ ]+", "", $1); # trim spaces on 1st field
gsub("[ ]+", "", $2); # trim spaces on 2nd field
print currentSlot "[" $1 "] as \"" $2"\""; # output key/value pair
}
running the script.awk
awk -f script.awk input.txt
oneliner awk command
awk -F":" '/slot/{s=$1}!/slot/{sub("^[ ]+","",$1);gsub("[ ]+","",$2);print s"["$1"] as \""$2"\""}' input.txt
input.txt
2 physical slots found:
slot 1:
Cstatus: present
Sstatus: active
apps: 0
slot 2:
Cstatus: present
Sstatus: inactive
apps: 0
output
slot 1[Cstatus] as "present"
slot 1[Sstatus] as "active"
slot 1[apps] as "0"
slot 2[Cstatus] as "present"
slot 2[Sstatus] as "inactive"
slot 2[apps] as "0"
A solution that works with Bash version 4.3 or above with support for:
declare -n var nameref variables
declare -A var associative arrays
#!/usr/bin/env bash
# Require Bash version 4.3+ for nameref and associative arrays
# Declare associative arrays to capture key value pairs
declare -A slot1=() slot2=()
# Declare a nameref variable that will control in which array to insert
# captured key value pairs
declare -n array_ref=_
# Iterate reading lines of the file
while IFS= read -r line
do
# Regex match and capture groups or skip line if no match
[[ $line =~ ^([[:space:]]*)(.*):[[:space:]]*(.*)$ ]] || continue
# Uncomment to debug Regex
# typeset -p BASH_REMATCH
# If first captured group is empty (no space)
if [ -z "${BASH_REMATCH[1]}" ]; then
case ${BASH_REMATCH[2]} in
'slot 1')
# Switch the nameref variable to the slot1 associative array
declare -n array_ref=slot1
continue
;;
'slot 2')
# Switch the nameref variable to the slot2 associative array
declare -n array_ref=slot2
continue
;;
*)
# An unknown array name to ignore
declare -n array_ref=_
continue
;;
esac
else
# First captured group contains spaces, so capture key values
k=${BASH_REMATCH[2]}
v=${BASH_REMATCH[3]}
# Insert key value pair into the associative array pointed by array_ref
# shellcheck disable=SC2034 # array_ref is used in assignment
array_ref["$k"]=$v
fi
done
# shellcheck disable=SC2034 # debug print
typeset -p slot1 slot2
Example of result from the script above
declare -A slot1=([Cstatus]="present" [apps]="0" [Sstatus]="active" )
declare -A slot2=([Cstatus]="present" [apps]="0" [Sstatus]="inactive" )

values to array from variable names with pattern

I have an unknown number of variable names with the pattern rundate*. For example, rundate=180618 && rundate2=180820. I know from here that I can send multiple variable names to a third variable: alld=(`echo "${!rundate*}"`) and while attempting to solve my problem, I figured out how to send multiple variable indices to a third variable: alld_indices=(`echo "${!alld[#]}"`). But, how do I send multiple values to my third variable: alld_values such that echo ${alld_values[#]} gives 180618 180820. I know from here how I can get the first value: firstd_value=(`echo "${!alld}"`). I suspect, I've seen the answer already in my searching but did not realize it. Happy to delete my question if that is the case. Thanks!
#!/usr/bin/env bash
# set up some test data
rundate="180618"
rundate1="180820"
rundate2="Values With Spaces Work Too"
# If we know all values are numeric, we can use a regular indexed array
# otherwise, the below would need to be ''declare -A alld=( )''
alld=( ) # initialize an array
for v in "${!rundate#}"; do # using # instead of * avoids IFS-related bugs
alld[${v#rundate}]=${!v} # populate the array, using varname w/o prefix as key
done
# print our results
printf 'Full array definition:\n '
declare -p alld # emits code that, if run, will redefine the array
echo; echo "Indexes only:"
printf ' - %s\n' "${!alld[#]}" # "${!varname[#]}" expands to the list of keys
echo; echo "Values only:"
printf ' - %s\n' "${alld[#]}" # "${varname[#]}" expands to the list of values
...properly emits as output:
Full array definition:
declare -a alld=([0]="180618" [1]="180820" [2]="Values With Spaces Work Too")
Indexes only:
- 0
- 1
- 2
Values only:
- 180618
- 180820
- Values With Spaces Work Too
...as you can see running at https://ideone.com/yjSD1J
eval in a loop will do it.
$: for v in ${!rundate*}
> do eval "alld_values+=( \$$v )"
> done
$: echo "${alld_values[#]}"
180618 180820
or
$: eval "alld_values=( $( sed 's/ / $/g' <<< " ${!rundate*}" ) )"
or
$: echo "alld_values=( $( sed 's/ / $/g' <<< " ${!rundate*}" ) )" > tmp && . tmp

How to get latest words from shell-variable without looping through all words?

I want to get two latest words from a string variable.
Total number of words in the string variable is not constant.
This is what I try:
LIST=`some command`
LATEST1=""
LATEST2=""
for ITEM in $LIST
do
LATEST2="$LATEST1"
LATEST1="$ITEM"
done
echo "Latest: $LATEST1"
echo "2nd latest: $LATEST2"
But it is slow. Is there any better way to do it?
sh shell of busybox is used. Other scripting languages are not available.
You can just use parameter substitution:
words="one two three four"
last=${words##* }
echo $last # => four
tmp=${words% *}
last2=${tmp##* }
echo $last2 # => three
Regex which will match the last 2 words (including whitespaces in this case)
(\s\w+){2}$
$ echo -e 'hello world\nhow are you' |
tr '\n' ' ' |
awk '
END{if(NF>1)printf("Latest:\t\t%s\n2nd latest:\t%s\n", $NF, $(NF-1)); else print "ERROR"}'
Latest: you
2nd latest: are
You can do this in a purely shell way using something like this:
words="one two three four"
words=($words)
echo ${words[${#words}]} # prints 'four'
echo ${words[((${#words} - 1))]} # prints 'three'
This works by assigning the variable containing the strings into an array and then accessing it by key.

Resources