Pass variable by reference [duplicate] - bash

I want to ask if it is possible to pass arguments to a script function by reference:
i.e. to do something that would look like this in C++:
void boo(int &myint) { myint = 5; }
int main() {
int t = 4;
printf("%d\n", t); // t->4
boo(t);
printf("%d\n", t); // t->5
}
So then in BASH I want to do something like:
function boo ()
{
var1=$1 # now var1 is global to the script but using it outside
# this function makes me lose encapsulation
local var2=$1 # so i should use a local variable ... but how to pass it back?
var2='new' # only changes the local copy
#$1='new' this is wrong of course ...
# ${!1}='new' # can i somehow use indirect reference?
}
# call boo
SOME_VAR='old'
echo $SOME_VAR # -> old
boo "$SOME_VAR"
echo $SOME_VAR # -> new
Any thoughts would be appreciated.

It's 2018, and this question deserves an update. At least in Bash, as of Bash 4.3-alpha, you can use namerefs to pass function arguments by reference:
function boo()
{
local -n ref=$1
ref='new'
}
SOME_VAR='old'
echo $SOME_VAR # -> old
boo SOME_VAR
echo $SOME_VAR # -> new
The critical pieces here are:
Passing the variable's name to boo, not its value: boo SOME_VAR, not boo $SOME_VAR.
Inside the function, using local -n ref=$1 to declare a nameref to the variable named by $1, meaning it's not a reference to $1 itself, but rather to a variable whose name $1 holds, i.e. SOME_VAR in our case. The value on the right-hand side should just be a string naming an existing variable: it doesn't matter how you get the string, so things like local -n ref="my_var" or local -n ref=$(get_var_name) would work too. declare can also replace local in contexts that allow/require that. See chapter on Shell Parameters in Bash Reference Manual for more information.
The advantage of this approach is (arguably) better readability and, most importantly, avoiding eval, whose security pitfalls are many and well-documented.

From the Bash man-page (Parameter Expansion):
If the first character of parameter is an exclamation point (!), a
level of variable indirection is introduced. Bash uses the value of
the variable formed from the rest of parameter as the name of the
variable; this variable is then expanded and that value is used in
the rest of the substitution, rather than the value of parameter
itself. This is known as indirect expansion.
Therefore a reference is the variable's name. Here is a swap function using
variable indirection that does not require a temporary variable:
function swap()
{ #
# #param VARNAME1 VARNAME2
#
eval "$1=${!2} $2=${!1}"
}
$ a=1 b=2
$ swap a b
$ echo $a $b
2 1

Use a helper function upvar:
# Assign variable one scope above the caller.
# Usage: local "$1" && upvar $1 value [value ...]
# Param: $1 Variable name to assign value to
# Param: $* Value(s) to assign. If multiple values, an array is
# assigned, otherwise a single value is assigned.
# NOTE: For assigning multiple variables, use 'upvars'. Do NOT
# use multiple 'upvar' calls, since one 'upvar' call might
# reassign a variable to be used by another 'upvar' call.
# See: http://fvue.nl/wiki/Bash:_Passing_variables_by_reference
upvar() {
if unset -v "$1"; then # Unset & validate varname
if (( $# == 2 )); then
eval $1=\"\$2\" # Return single value
else
eval $1=\(\"\${#:2}\"\) # Return array
fi
fi
}
And use it like this from within Newfun():
local "$1" && upvar $1 new
For returning multiple variables, use another helper function upvars. This allows passing multiple variables within one call, thus avoiding possible conflicts if one upvar call changes a variable used in another subsequent upvar call.
See: http://www.fvue.nl/wiki/Bash:_Passing_variables_by_reference for helper function upvars and more information.
The problem with:
eval $1=new
is that it's not safe if $1 happens to contain a command:
set -- 'ls /;true'
eval $1=new # Oops
It would be better to use printf -v:
printf -v "$1" %s new
But printf -v cannot assign arrays.
Moreover, both eval and printf won't work if the variable happens to be declared local:
g() { local b; eval $1=bar; } # WRONG
g b # Conflicts with `local b'
echo $b # b is empty unexpected
The conflict stays there even if local b is unset:
g() { local b; unset b; eval $1=bar; } # WRONG
g b # Still conflicts with `local b'
echo $b # b is empty unexpected

I have found a way to do this but I am not sure how correct this is:
Newfun()
{
local var1="$1"
eval $var1=2
# or can do eval $1=2 if no local var
}
var=1
echo var is $var # $var = 1
newfun 'var' # pass the name of the variable…
echo now var is $var # $var = 2
So we pass the variable name as opposed to the value and then use eval ...

Bash doesn't have anything like references built into it, so basically the only way you would be able to do what you want is to pass the function the name of the global variable you want it to modify. And even then you'll need an eval statement:
boo() {
eval ${1}="new"
}
SOME_VAR="old"
echo $SOME_VAR # old
boo "SOME_VAR"
echo $SOME_VAR # new
I don't think you can use indirect references here because Bash automatically accesses the value of the variable whose name is stored in the indirect reference. It doesn't give you the chance to set it.

Ok, so this question has been waiting for a 'real' solution for some time now, and I am glad to say that we can now accomplish this without using eval at all.
The key to remember is to declare a reference in both the caller as the callee, at least in my example:
#!/bin/bash
# NOTE this does require a bash version >= 4.3
set -o errexit -o nounset -o posix -o pipefail
passedByRef() {
local -n theRef
if [ 0 -lt $# ]; then
theRef=$1
echo -e "${FUNCNAME}:\n\tthe value of my reference is:\n\t\t${theRef}"
# now that we have a reference, we can assign things to it
theRef="some other value"
echo -e "${FUNCNAME}:\n\tvalue of my reference set to:\n\t\t${theRef}"
else
echo "Error: missing argument"
exit 1
fi
}
referenceTester() {
local theVariable="I am a variable"
# note the absence of quoting and escaping etc.
local -n theReference=theVariable
echo -e "${FUNCNAME}:\n\tthe value of my reference is:\n\t\t${theReference}"
passedByRef theReference
echo -e "${FUNCNAME}:\n\tthe value of my reference is now:\n\t\t${theReference},\n\tand the pointed to variable:\n\t\t${theVariable}"
}
Output when run:
referenceTester:
the value of my reference is:
I am a variable
passedByRef:
the value of my reference is:
I am a variable
passedByRef:
value of my reference set to:
some other value
referenceTester:
the value of my reference is now:
some other value,
and the pointed to variable:
some other value

Eval should never be used on a string that a user can set because its dangerous. Something like "string; rm -rf ~" will be bad. So generally its best to find solutions where you don't have to worry about it.
However, eval will be needed to set the passed variables, as the comment noted.
$ y=four
$ four=4
$ echo ${!y}
4
$ foo() { x=$1; echo ${!x}; }
$ foo four
4

#!/bin/bash
append_string()
{
if [ -z "${!1}" ]; then
eval "${1}='$2'"
else
eval "${1}='${!1}''${!3}''$2'"
fi
}
PETS=''
SEP='|'
append_string "PETS" "cat" "SEP"
echo "$PETS"
append_string "PETS" "dog" "SEP"
echo "$PETS"
append_string "PETS" "hamster" "SEP"
echo "$PETS"
Output:
cat
cat|dog
cat|dog|hamster
Structure for calling that function is:
append_string name_of_var_to_update string_to_add name_of_var_containing_sep_char
Name of variable is passed to fuction about PETS and SEP while string to append is passed the usual way as value. "${!1}" refers to contents of global PETS variable. In the beginning that variable is empty and contens is added each time we call the function. Separator character can be selected as needed. "eval" starting lines update PETS variable.

This is what works for me on Ubuntu bash shell
#!/bin/sh
iteration=10
increment_count()
{
local i
i=$(($1+1))
eval ${1}=\$i
}
increment_count iteration
echo $iteration #prints 11
increment_count iteration
echo $iteration #prints 12

Related

On returning array list in shell script, getting 1 as value rather than actual array [duplicate]

I have a function that creates an array and I want to return the array to the caller:
create_array() {
local my_list=("a", "b", "c")
echo "${my_list[#]}"
}
my_algorithm() {
local result=$(create_array)
}
With this, I only get an expanded string. How can I "return" my_list without using anything global?
With Bash version 4.3 and above, you can make use of a nameref so that the caller can pass in the array name and the callee can use a nameref to populate the named array, indirectly.
#!/usr/bin/env bash
create_array() {
local -n arr=$1 # use nameref for indirection
arr=(one "two three" four)
}
use_array() {
local my_array
create_array my_array # call function to populate the array
echo "inside use_array"
declare -p my_array # test the array
}
use_array # call the main function
Produces the output:
inside use_array
declare -a my_array=([0]="one" [1]="two three" [2]="four")
You could make the function update an existing array as well:
update_array() {
local -n arr=$1 # use nameref for indirection
arr+=("two three" four) # update the array
}
use_array() {
local my_array=(one)
update_array my_array # call function to update the array
}
This is a more elegant and efficient approach since we don't need command substitution $() to grab the standard output of the function being called. It also helps if the function were to return more than one output - we can simply use as many namerefs as the number of outputs.
Here is what the Bash Manual says about nameref:
A variable can be assigned the nameref attribute using the -n option
to the declare or local builtin commands (see Bash Builtins) to create
a nameref, or a reference to another variable. This allows variables
to be manipulated indirectly. Whenever the nameref variable is
referenced, assigned to, unset, or has its attributes modified (other
than using or changing the nameref attribute itself), the operation is
actually performed on the variable specified by the nameref variable’s
value. A nameref is commonly used within shell functions to refer to a
variable whose name is passed as an argument to the function. For
instance, if a variable name is passed to a shell function as its
first argument, running
declare -n ref=$1 inside the function creates a nameref variable ref
whose value is the variable name passed as the first argument.
References and assignments to ref, and changes to its attributes, are
treated as references, assignments, and attribute modifications to the
variable whose name was passed as $1.
What's wrong with globals?
Returning arrays is really not practical. There are lots of pitfalls.
That said, here's one technique that works if it's OK that the variable have the same name:
$ f () { local a; a=(abc 'def ghi' jkl); declare -p a; }
$ g () { local a; eval $(f); declare -p a; }
$ f; declare -p a; echo; g; declare -p a
declare -a a='([0]="abc" [1]="def ghi" [2]="jkl")'
-bash: declare: a: not found
declare -a a='([0]="abc" [1]="def ghi" [2]="jkl")'
-bash: declare: a: not found
The declare -p commands (except for the one in f() are used to display the state of the array for demonstration purposes. In f() it's used as the mechanism to return the array.
If you need the array to have a different name, you can do something like this:
$ g () { local b r; r=$(f); r="declare -a b=${r#*=}"; eval "$r"; declare -p a; declare -p b; }
$ f; declare -p a; echo; g; declare -p a
declare -a a='([0]="abc" [1]="def ghi" [2]="jkl")'
-bash: declare: a: not found
-bash: declare: a: not found
declare -a b='([0]="abc" [1]="def ghi" [2]="jkl")'
-bash: declare: a: not found
Bash can't pass around data structures as return values. A return value must be a numeric exit status between 0-255. However, you can certainly use command or process substitution to pass commands to an eval statement if you're so inclined.
This is rarely worth the trouble, IMHO. If you must pass data structures around in Bash, use a global variable--that's what they're for. If you don't want to do that for some reason, though, think in terms of positional parameters.
Your example could easily be rewritten to use positional parameters instead of global variables:
use_array () {
for idx in "$#"; do
echo "$idx"
done
}
create_array () {
local array=("a" "b" "c")
use_array "${array[#]}"
}
This all creates a certain amount of unnecessary complexity, though. Bash functions generally work best when you treat them more like procedures with side effects, and call them in sequence.
# Gather values and store them in FOO.
get_values_for_array () { :; }
# Do something with the values in FOO.
process_global_array_variable () { :; }
# Call your functions.
get_values_for_array
process_global_array_variable
If all you're worried about is polluting your global namespace, you can also use the unset builtin to remove a global variable after you're done with it. Using your original example, let my_list be global (by removing the local keyword) and add unset my_list to the end of my_algorithm to clean up after yourself.
You were not so far out with your original solution. You had a couple of problems, you used a comma as a separator, and you failed to capture the returned items into a list, try this:
my_algorithm() {
local result=( $(create_array) )
}
create_array() {
local my_list=("a" "b" "c")
echo "${my_list[#]}"
}
Considering the comments about embedded spaces, a few tweaks using IFS can solve that:
my_algorithm() {
oldIFS="$IFS"
IFS=','
local result=( $(create_array) )
IFS="$oldIFS"
echo "Should be 'c d': ${result[1]}"
}
create_array() {
IFS=','
local my_list=("a b" "c d" "e f")
echo "${my_list[*]}"
}
Use the technique developed by Matt McClure:
http://notes-matthewlmcclure.blogspot.com/2009/12/return-array-from-bash-function-v-2.html
Avoiding global variables means you can use the function in a pipe. Here is an example:
#!/bin/bash
makeJunk()
{
echo 'this is junk'
echo '#more junk and "b#d" characters!'
echo '!#$^%^&(*)_^&% ^$##:"<>?/.,\\"'"'"
}
processJunk()
{
local -a arr=()
# read each input and add it to arr
while read -r line
do
arr+=('"'"$line"'" is junk')
done;
# output the array as a string in the "declare" representation
declare -p arr | sed -e 's/^declare -a [^=]*=//'
}
# processJunk returns the array in a flattened string ready for "declare"
# Note that because of the pipe processJunk cannot return anything using
# a global variable
returned_string="$(makeJunk | processJunk)"
# convert the returned string to an array named returned_array
# declare correctly manages spaces and bad characters
eval "declare -a returned_array=${returned_string}"
for junk in "${returned_array[#]}"
do
echo "$junk"
done
Output is:
"this is junk" is junk
"#more junk and "b#d" characters!" is junk
"!#$^%^&(*)_^&% ^$##:"<>?/.,\\"'" is junk
A pure bash, minimal and robust solution based on the 'declare -p' builtin — without insane global variables
This approach involves the following three steps:
Convert the array with 'declare -p' and save the output in a variable.
myVar="$( declare -p myArray )"
The output of the declare -p statement can be used to recreate the array.
For instance the output of declare -p myVar might look like this:
declare -a myVar='([0]="1st field" [1]="2nd field" [2]="3rd field")'
Use the echo builtin to pass the variable to a function or to pass it back from there.
In order to preserve whitspaces in array fields when echoing the variable, IFS is temporarly set to a control character (e.g. a vertical tab).
Only the right-hand-side of the declare statement in the variable is to be echoed - this can be achieved by parameter expansion of the form ${parameter#word}. As for the example above: ${myVar#*=}
Finally, recreate the array where it is passed to using the eval and the 'declare -a' builtins.
Example 1 - return an array from a function
#!/bin/bash
# Example 1 - return an array from a function
function my-fun () {
# set up a new array with 3 fields - note the whitespaces in the
# 2nd (2 spaces) and 3rd (2 tabs) field
local myFunArray=( "1st field" "2nd field" "3rd field" )
# show its contents on stderr (must not be output to stdout!)
echo "now in $FUNCNAME () - showing contents of myFunArray" >&2
echo "by the help of the 'declare -p' builtin:" >&2
declare -p myFunArray >&2
# return the array
local myVar="$( declare -p myFunArray )"
local IFS=$'\v';
echo "${myVar#*=}"
# if the function would continue at this point, then IFS should be
# restored to its default value: <space><tab><newline>
IFS=' '$'\t'$'\n';
}
# main
# call the function and recreate the array that was originally
# set up in the function
eval declare -a myMainArray="$( my-fun )"
# show the array contents
echo ""
echo "now in main part of the script - showing contents of myMainArray"
echo "by the help of the 'declare -p' builtin:"
declare -p myMainArray
# end-of-file
Output of Example 1:
now in my-fun () - showing contents of myFunArray
by the help of the 'declare -p' builtin:
declare -a myFunArray='([0]="1st field" [1]="2nd field" [2]="3rd field")'
now in main part of the script - showing contents of myMainArray
by the help of the 'declare -p' builtin:
declare -a myMainArray='([0]="1st field" [1]="2nd field" [2]="3rd field")'
Example 2 - pass an array to a function
#!/bin/bash
# Example 2 - pass an array to a function
function my-fun () {
# recreate the array that was originally set up in the main part of
# the script
eval declare -a myFunArray="$( echo "$1" )"
# note that myFunArray is local - from the bash(1) man page: when used
# in a function, declare makes each name local, as with the local
# command, unless the ‘-g’ option is used.
# IFS has been changed in the main part of this script - now that we
# have recreated the array it's better to restore it to the its (local)
# default value: <space><tab><newline>
local IFS=' '$'\t'$'\n';
# show contents of the array
echo ""
echo "now in $FUNCNAME () - showing contents of myFunArray"
echo "by the help of the 'declare -p' builtin:"
declare -p myFunArray
}
# main
# set up a new array with 3 fields - note the whitespaces in the
# 2nd (2 spaces) and 3rd (2 tabs) field
myMainArray=( "1st field" "2nd field" "3rd field" )
# show the array contents
echo "now in the main part of the script - showing contents of myMainArray"
echo "by the help of the 'declare -p' builtin:"
declare -p myMainArray
# call the function and pass the array to it
myVar="$( declare -p myMainArray )"
IFS=$'\v';
my-fun $( echo "${myVar#*=}" )
# if the script would continue at this point, then IFS should be restored
# to its default value: <space><tab><newline>
IFS=' '$'\t'$'\n';
# end-of-file
Output of Example 2:
now in the main part of the script - showing contents of myMainArray
by the help of the 'declare -p' builtin:
declare -a myMainArray='([0]="1st field" [1]="2nd field" [2]="3rd field")'
now in my-fun () - showing contents of myFunArray
by the help of the 'declare -p' builtin:
declare -a myFunArray='([0]="1st field" [1]="2nd field" [2]="3rd field")'
I recently discovered a quirk in BASH in that a function has direct access to the variables declared in the functions higher in the call stack. I've only just started to contemplate how to exploit this feature (it promises both benefits and dangers), but one obvious application is a solution to the spirit of this problem.
I would also prefer to get a return value rather than using a global variable when delegating the creation of an array. There are several reasons for my preference, among which are to avoid possibly disturbing an preexisting value and to avoid leaving a value that may be invalid when later accessed. While there are workarounds to these problems, the easiest is have the variable go out of scope when the code is finished with it.
My solution ensures that the array is available when needed and discarded when the function returns, and leaves undisturbed a global variable with the same name.
#!/bin/bash
myarr=(global array elements)
get_an_array()
{
myarr=( $( date +"%Y %m %d" ) )
}
request_array()
{
declare -a myarr
get_an_array "myarr"
echo "New contents of local variable myarr:"
printf "%s\n" "${myarr[#]}"
}
echo "Original contents of global variable myarr:"
printf "%s\n" "${myarr[#]}"
echo
request_array
echo
echo "Confirm the global myarr was not touched:"
printf "%s\n" "${myarr[#]}"
Here is the output of this code:
When function request_array calls get_an_array, get_an_array can directly set the myarr variable that is local to request_array. Since myarr is created with declare, it is local to request_array and thus goes out of scope when request_array returns.
Although this solution does not literally return a value, I suggest that taken as a whole, it satisfies the promises of a true function return value.
Useful example: return an array from function
function Query() {
local _tmp=`echo -n "$*" | mysql 2>> zz.err`;
echo -e "$_tmp";
}
function StrToArray() {
IFS=$'\t'; set $1; for item; do echo $item; done; IFS=$oIFS;
}
sql="SELECT codi, bloc, requisit FROM requisits ORDER BY codi";
qry=$(Query $sql0);
IFS=$'\n';
for row in $qry; do
r=( $(StrToArray $row) );
echo ${r[0]} - ${r[1]} - ${r[2]};
done
I tried various implementations, and none preserved arrays that had elements with spaces ... because they all had to use echo.
# These implementations only work if no array items contain spaces.
use_array() { eval echo '(' \"\${${1}\[\#\]}\" ')'; }
use_array() { local _array="${1}[#]"; echo '(' "${!_array}" ')'; }
Solution
Then I came across Dennis Williamson's answer. I incorporated his method into the following functions so they can a) accept an arbitrary array and b) be used to pass, duplicate and append arrays.
# Print array definition to use with assignments, for loops, etc.
# varname: the name of an array variable.
use_array() {
local r=$( declare -p $1 )
r=${r#declare\ -a\ *=}
# Strip keys so printed definition will be a simple list (like when using
# "${array[#]}"). One side effect of having keys in the definition is
# that when appending arrays (i.e. `a1+=$( use_array a2 )`), values at
# matching indices merge instead of pushing all items onto array.
echo ${r//\[[0-9]\]=}
}
# Same as use_array() but preserves keys.
use_array_assoc() {
local r=$( declare -p $1 )
echo ${r#declare\ -a\ *=}
}
Then, other functions can return an array using catchable output or indirect arguments.
# catchable output
return_array_by_printing() {
local returnme=( "one" "two" "two and a half" )
use_array returnme
}
eval test1=$( return_array_by_printing )
# indirect argument
return_array_to_referenced_variable() {
local returnme=( "one" "two" "two and a half" )
eval $1=$( use_array returnme )
}
return_array_to_referenced_variable test2
# Now both test1 and test2 are arrays with three elements
I needed a similar functionality recently, so the following is a mix of the suggestions made by RashaMatt and Steve Zobell.
echo each array/list element as separate line from within a function
use mapfile to read all array/list elements echoed by a function.
As far as I can see, strings are kept intact and whitespaces are preserved.
#!bin/bash
function create-array() {
local somearray=("aaa" "bbb ccc" "d" "e f g h")
for elem in "${somearray[#]}"
do
echo "${elem}"
done
}
mapfile -t resa <<< "$(create-array)"
# quick output check
declare -p resa
Some more variations…
#!/bin/bash
function create-array-from-ls() {
local somearray=("$(ls -1)")
for elem in "${somearray[#]}"
do
echo "${elem}"
done
}
function create-array-from-args() {
local somearray=("$#")
for elem in "${somearray[#]}"
do
echo "${elem}"
done
}
mapfile -t resb <<< "$(create-array-from-ls)"
mapfile -t resc <<< "$(create-array-from-args 'xxx' 'yy zz' 't s u' )"
sentenceA="create array from this sentence"
sentenceB="keep this sentence"
mapfile -t resd <<< "$(create-array-from-args ${sentenceA} )"
mapfile -t rese <<< "$(create-array-from-args "$sentenceB" )"
mapfile -t resf <<< "$(create-array-from-args "$sentenceB" "and" "this words" )"
# quick output check
declare -p resb
declare -p resc
declare -p resd
declare -p rese
declare -p resf
Here is a solution with no external array references and no IFS manipulation:
# add one level of single quotes to args, eval to remove
squote () {
local a=("$#")
a=("${a[#]//\'/\'\\\'\'}") # "'" => "'\''"
a=("${a[#]/#/\'}") # add "'" prefix to each word
a=("${a[#]/%/\'}") # add "'" suffix to each word
echo "${a[#]}"
}
create_array () {
local my_list=(a "b 'c'" "\\\"d
")
squote "${my_list[#]}"
}
my_algorithm () {
eval "local result=($(create_array))"
# result=([0]="a" [1]="b 'c'" [2]=$'\\"d\n')
}
[Note: the following was rejected as an edit of this answer for reasons that make no sense to me (since the edit was not intended to address the author of the post!), so I'm taking the suggestion to make it a separate answer.]
A simpler implementation of Steve Zobell's adaptation of Matt McClure's technique uses the bash built-in (since version == 4) readarray as suggested by RastaMatt to create a representation of an array that can be converted into an array at runtime. (Note that both readarray and mapfile name the same code.) It still avoids globals (allowing use of the function in a pipe), and still handles nasty characters.
For some more-fully-developed (e.g., more modularization) but still-kinda-toy examples, see bash_pass_arrays_between_functions. Following are a few easily-executable examples, provided here to avoid moderators b!tching about external links.
Cut the following block and paste it into a bash terminal to create /tmp/source.sh and /tmp/junk1.sh:
FP='/tmp/source.sh' # path to file to be created for `source`ing
cat << 'EOF' > "${FP}" # suppress interpretation of variables in heredoc
function make_junk {
echo 'this is junk'
echo '#more junk and "b#d" characters!'
echo '!#$^%^&(*)_^&% ^$##:"<>?/.,\\"'"'"
}
### Use 'readarray' (aka 'mapfile', bash built-in) to read lines into an array.
### Handles blank lines, whitespace and even nastier characters.
function lines_to_array_representation {
local -a arr=()
readarray -t arr
# output array as string using 'declare's representation (minus header)
declare -p arr | sed -e 's/^declare -a [^=]*=//'
}
EOF
FP1='/tmp/junk1.sh' # path to script to run
cat << 'EOF' > "${FP1}" # suppress interpretation of variables in heredoc
#!/usr/bin/env bash
source '/tmp/source.sh' # to reuse its functions
returned_string="$(make_junk | lines_to_array_representation)"
eval "declare -a returned_array=${returned_string}"
for elem in "${returned_array[#]}" ; do
echo "${elem}"
done
EOF
chmod u+x "${FP1}"
# newline here ... just hit Enter ...
Run /tmp/junk1.sh: output should be
this is junk
#more junk and "b#d" characters!
!#$^%^&(*)_^&% ^$##:"<>?/.,\\"'
Note lines_to_array_representation also handles blank lines. Try pasting the following block into your bash terminal:
FP2='/tmp/junk2.sh' # path to script to run
cat << 'EOF' > "${FP2}" # suppress interpretation of variables in heredoc
#!/usr/bin/env bash
source '/tmp/source.sh' # to reuse its functions
echo '`bash --version` the normal way:'
echo '--------------------------------'
bash --version
echo # newline
echo '`bash --version` via `lines_to_array_representation`:'
echo '-----------------------------------------------------'
bash_version="$(bash --version | lines_to_array_representation)"
eval "declare -a returned_array=${bash_version}"
for elem in "${returned_array[#]}" ; do
echo "${elem}"
done
echo # newline
echo 'But are they *really* the same? Ask `diff`:'
echo '-------------------------------------------'
echo 'You already know how to capture normal output (from `bash --version`):'
declare -r PATH_TO_NORMAL_OUTPUT="$(mktemp)"
bash --version > "${PATH_TO_NORMAL_OUTPUT}"
echo "normal output captured to file # ${PATH_TO_NORMAL_OUTPUT}"
ls -al "${PATH_TO_NORMAL_OUTPUT}"
echo # newline
echo 'Capturing L2AR takes a bit more work, but is not onerous.'
echo "Look # contents of the file you're about to run to see how it's done."
declare -r RAW_L2AR_OUTPUT="$(bash --version | lines_to_array_representation)"
declare -r PATH_TO_COOKED_L2AR_OUTPUT="$(mktemp)"
eval "declare -a returned_array=${RAW_L2AR_OUTPUT}"
for elem in "${returned_array[#]}" ; do
echo "${elem}" >> "${PATH_TO_COOKED_L2AR_OUTPUT}"
done
echo "output from lines_to_array_representation captured to file # ${PATH_TO_COOKED_L2AR_OUTPUT}"
ls -al "${PATH_TO_COOKED_L2AR_OUTPUT}"
echo # newline
echo 'So are they really the same? Per'
echo "\`diff -uwB "${PATH_TO_NORMAL_OUTPUT}" "${PATH_TO_COOKED_L2AR_OUTPUT}" | wc -l\`"
diff -uwB "${PATH_TO_NORMAL_OUTPUT}" "${PATH_TO_COOKED_L2AR_OUTPUT}" | wc -l
echo '... they are the same!'
EOF
chmod u+x "${FP2}"
# newline here ... just hit Enter ...
Run /tmp/junk2.sh # commandline. Your output should be similar to mine:
`bash --version` the normal way:
--------------------------------
GNU bash, version 4.3.30(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
`bash --version` via `lines_to_array_representation`:
-----------------------------------------------------
GNU bash, version 4.3.30(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
But are they *really* the same? Ask `diff`:
-------------------------------------------
You already know how to capture normal output (from `bash --version`):
normal output captured to file # /tmp/tmp.Ni1bgyPPEw
-rw------- 1 me me 308 Jun 18 16:27 /tmp/tmp.Ni1bgyPPEw
Capturing L2AR takes a bit more work, but is not onerous.
Look # contents of the file you're about to run to see how it's done.
output from lines_to_array_representation captured to file # /tmp/tmp.1D6O2vckGz
-rw------- 1 me me 308 Jun 18 16:27 /tmp/tmp.1D6O2vckGz
So are they really the same? Per
`diff -uwB /tmp/tmp.Ni1bgyPPEw /tmp/tmp.1D6O2vckGz | wc -l`
0
... they are the same!
There's no need to use eval or to change IFS to \n. There are at least 2 good ways to do this.
1) Using echo and mapfile
You can simply echo each item of the array in the function, then use mapfile to turn it into an array:
outputArray()
{
for i
{
echo "$i"
}
}
declare -a arr=( 'qq' 'www' 'ee rr' )
mapfile -t array < <(outputArray "${arr[#]}")
for i in "${array[#]}"
do
echo "i=$i"
done
To make it work using pipes, add (( $# == 0 )) && readarray -t temp && set "${temp[#]}" && unset temp to the top of output array. It converts stdin to parameters.
2) Using declare -p and sed
This can also be done using declare -p and sed instead of mapfile.
outputArray()
{
(( $# == 0 )) && readarray -t temp && set "${temp[#]}" && unset temp
for i; { echo "$i"; }
}
returnArray()
{
local -a arr=()
(( $# == 0 )) && readarray -t arr || for i; { arr+=("$i"); }
declare -p arr | sed -e 's/^declare -a [^=]*=//'
}
declare -a arr=( 'qq' 'www' 'ee rr' )
declare -a array=$(returnArray "${arr[#]}")
for i in "${array[#]}"
do
echo "i=$i"
done
declare -a array=$(outputArray "${arr[#]}" | returnArray)
echo
for i in "${array[#]}"
do
echo "i=$i"
done
declare -a array < <(outputArray "${arr[#]}" | returnArray)
echo
for i in "${array[#]}"
do
echo "i=$i"
done
This can also be done by simply passing array variable to the function and assign array values to this var then use this var outside of function. For example.
create_array() {
local __resultArgArray=$1
local my_list=("a" "b" "c")
eval $__resultArgArray="("${my_list[#]}")"
}
my_algorithm() {
create_array result
echo "Total elements in the array: ${#result[#]}"
for i in "${result[#]}"
do
echo $i
done
}
my_algorithm
The easest way y found
my_function()
{
array=(one two three)
echo ${array[#]}
}
result=($(my_function))
echo ${result[0]}
echo ${result[1]}
echo ${result[2]}
You can also use the declare -p method more easily by taking advantage of declare -a's double-evaluation when the value is a string (no true parens outside the string):
# return_array_value returns the value of array whose name is passed in.
# It turns the array into a declaration statement, then echos the value
# part of that statement with parentheses intact. You can use that
# result in a "declare -a" statement to create your own array with the
# same value. Also works for associative arrays with "declare -A".
return_array_value () {
declare Array_name=$1 # namespace locals with caps to prevent name collision
declare Result
Result=$(declare -p $Array_name) # dehydrate the array into a declaration
echo "${Result#*=}" # trim "declare -a ...=" from the front
}
# now use it. test for robustness by skipping an index and putting a
# space in an entry.
declare -a src=([0]=one [2]="two three")
declare -a dst="$(return_array_value src)" # rehydrate with double-eval
declare -p dst
> declare -a dst=([0]="one" [2]="two three") # result matches original
Verifying the result, declare -p dst yields declare -a dst=([0]="one" [2]="two three")", demonstrating that this method correctly deals with both sparse arrays as well as entries with an IFS character (space).
The first thing is to dehydrate the source array by using declare -p to generate a valid bash declaration of it. Because the declaration is a full statement, including "declare" and the variable name, we strip that part from the front with ${Result#*=}, leaving the parentheses with the indices and values inside: ([0]="one" [2]="two three").
It then rehydrates the array by feeding that value to your own declare statement, one where you choose the array name. It relies on the fact that the right side of the dst array declaration is a string with parentheses that are inside the string, rather than true parentheses in the declare itself, e.g. not declare -a dst=( "true parens outside string" ). This triggers declare to evaluate the string twice, once into a valid statement with parentheses (and quotes in the value preserved), and another for the actual assignment. I.e. it evaluates first to declare -a dst=([0]="one" [2]="two three"), then evaluates that as a statement.
Note that this double evaluation behavior is specific to the -a and -A options of declare.
Oh, and this method works with associative arrays as well, just change -a to -A.
Because this method relies on stdout, it works across subshell boundaries like pipelines, as others have noted.
I discuss this method in more detail in my blog post
If your source data is formatted with each list element on a separate line, then the mapfile builtin is a simple and elegant way to read a list into an array:
$ list=$(ls -1 /usr/local) # one item per line
$ mapfile -t arrayVar <<<"$list" # -t trims trailing newlines
$ declare -p arrayVar | sed 's#\[#\n[#g'
declare -a arrayVar='(
[0]="bin"
[1]="etc"
[2]="games"
[3]="include"
[4]="lib"
[5]="man"
[6]="sbin"
[7]="share"
[8]="src")'
Note that, as with the read builtin, you would not ordinarily* use mapfile in a pipeline (or subshell) because the assigned array variable would be unavailable to subsequent statements (* unless bash job control is disabled and shopt -s lastpipe is set).
$ help mapfile
mapfile: mapfile [-n count] [-O origin] [-s count] [-t] [-u fd] [-C callback] [-c quantum] [array]
Read lines from the standard input into an indexed array variable.
Read lines from the standard input into the indexed array variable ARRAY, or
from file descriptor FD if the -u option is supplied. The variable MAPFILE
is the default ARRAY.
Options:
-n count Copy at most COUNT lines. If COUNT is 0, all lines are copied.
-O origin Begin assigning to ARRAY at index ORIGIN. The default index is 0.
-s count Discard the first COUNT lines read.
-t Remove a trailing newline from each line read.
-u fd Read lines from file descriptor FD instead of the standard input.
-C callback Evaluate CALLBACK each time QUANTUM lines are read.
-c quantum Specify the number of lines read between each call to CALLBACK.
Arguments:
ARRAY Array variable name to use for file data.
If -C is supplied without -c, the default quantum is 5000. When
CALLBACK is evaluated, it is supplied the index of the next array
element to be assigned and the line to be assigned to that element
as additional arguments.
If not supplied with an explicit origin, mapfile will clear ARRAY before
assigning to it.
Exit Status:
Returns success unless an invalid option is given or ARRAY is readonly or
not an indexed array.
You can try this
my_algorithm() {
create_array list
for element in "${list[#]}"
do
echo "${element}"
done
}
create_array() {
local my_list=("1st one" "2nd two" "3rd three")
eval "${1}=()"
for element in "${my_list[#]}"
do
eval "${1}+=(\"${element}\")"
done
}
my_algorithm
The output is
1st one
2nd two
3rd three
I'd suggest piping to a code block to set values of an array. The strategy is POSIX compatible, so you get both Bash and Zsh, and doesn't run the risk of side effects like the posted solutions.
i=0 # index for our new array
declare -a arr # our new array
# pipe from a function that produces output by line
ls -l | { while read data; do i=$i+1; arr[$i]="$data"; done }
# example of reading that new array
for row in "${arr[#]}"; do echo "$row"; done
This will work for zsh and bash, and won't be affected by spaces or special characters. In the case of the OP, the output is transformed by echo, so it is not actually outputting an array, but printing it (as others mentioned shell functions return status not values). We can change it to a pipeline ready mechanism:
create_array() {
local my_list=("a", "b", "c")
for row in "${my_list[#]}"; do
echo "$row"
done
}
my_algorithm() {
i=0
declare -a result
create_array | { while read data; do i=$i+1; result[$i]="$data"; done }
}
If so inclined, one could remove the create_array pipeline process from my_algorithm and chain the two functions together
create_array | my_algorithm
A modern Bash implementation using #Q to safely output array elements:
#!/usr/bin/env bash
return_array_elements() {
local -a foo_array=('1st one' '2nd two' '3rd three')
printf '%s\n' "${foo_array[#]#Q}"
}
use_array_elements() {
local -a bar_array="($(return_array_elements))"
# Display declareation of bar_array
# which is local to this function, but whose elements
# hahaves been returned by the return_array_elements function
declare -p bar_array
}
use_array_elements
Output:
declare -a bar_array=([0]="1st one" [1]="2nd two" [2]="3rd three")
While the declare -p approach is elegant indeed, you can still create a global array using declare -g within a function and have it visible outside the scope of the function:
create_array() {
declare -ag result=("a", "b", "c")
}
my_algorithm() {
create_array
echo "${result[#]}"
}

Evaluate variable at time of function declaration in shell

I'm setting up my shell environments and I want to be able to use some of the same functions/aliases in zsh as in bash. One of these functions opens either .bashrc or .zshrc in an editor (whichever file is relevant), waits for the editor to close, then reloads the rc file.
# a very simplified version of this function
editrc() {
local rcfile=".$(basename $SHELL)rc"
code -w ~/$rcfile
. ~/$rcfile
}
I use the value of rcfile in a few other functions, so I've pulled it out of the function declaration.
_rc=".$(basename $SHELL)rc"
editrc() {
code -w ~/$_rc
. ~/$_rc
}
# ... other functions that use it ...
unset _rc
However, because I'm a neat freak, I want to unset _rc at the end of my script, but I still want my functions to run correctly. Is there a clever way to evaluate $_rc at the time the function is declared?
I know I could use eval and place everything except $_rc instances within single quotes, but that seems like a pain, since the full version of my function uses both single-quotes and double-quotes.
_rc=".$(basename $SHELL)rc"
eval 'editrc() {
echo Here'"'"'s a thing that uses single quotes. As you can see it'"'"'s a pain.
code -w ~/'$_rc'
. ~/'$_rc'
}'
# ... other functions using `_rc`
unset _rc
I'm guessing I could declare my functions, then do some magic with eval "$(declare -f editrc | awk)". It very well be more pain than it's worth, but I'm always interested in learning new things.
Note: I'd love to generalize this into a utility function that does this.
_myvar=foo
anothervar=bar
myfunc() {
echo $_myvar $anothervar
}
# redeclares myfunc with `$_myvar` expanded, but leaves `$anothervar` as-is
expandfunctionvars myfunc '$_myvar'
Is there a clever way to evaluate $_rc at the time the function is declared?
_rc=".$(basename "$SHELL")rc"
# while you could eval here, source lets you work with a stream
source <(
cat <<EOF
editrc() {
local _rc
# first safely trasfer context
$(declare -p _rc)
EOF
# use quoted here string to do anything inside without caring.
cat <<'EOF'
# do anything else
echo "Here's a thing that uses single quotes. As you can see it's not a pain, just choose proper quoting."
code -w "~/$_rc"
. "~/$_rc"
}
EOF
)
unset _rc
Generally first use declare -p to transfer variables as strings to be evaluated. Then after you "import" variables, use a quoted here document to do anything as in a normal script.
References to read:
<<EOF is a here document. Note the difference in parsing when the here delimiter is quoted vs unquoted.
<(..) is a process substitution
The source command reads a pipe created by process substitution. Inside the process subtitution I output the function to be sourced. With the first here document I output the function name definition, with a local of the variable so that it doesn't pollute global namespace. Then with declare -p I output the variable definition as a properly quoted string later to be sourced by source. Then with a quoted here document I output the rest of the function, so that I do not need to care about quoting.
The code is bash specific, I know nothing about zsh and don't use it.
You could do it with eval too:
eval '
editrc() {
local _rc
# first safely trasfer context
'"$(declare -p _rc)"'
# use quoted here string to do anything inside without caring.
# do anything else
echo "Here'\''s a thing that uses single quotes. As you can see it'\''s not a pain, just choose proper quoting."
code -w "~/$_rc"
. "~/$_rc"
}'
But for me using a quoted here document delimiter allows for easier writing.
While KamilCuck was working on their answer, I devised a function that will take in any function name and a set of variable names, expand just those variables, and redeclare the function.
expandFnVars() {
if [[ $# -lt 2 ]]; then
>&2 echo 'expandFnVars requires at least two arguments: the function name and the variable(s) to be expanded'
return 1
fi
local fn="$1"
shift
local vars=("$#")
if [[ -z "$(declare -F $fn 2> /dev/null)" ]]; then
>&2 echo $fn is not a function.
return 1
fi
foundAllVars=true
for v in $vars; do
if [[ -z "$(declare -p $v 2> /dev/null)" ]]; then
>&2 echo $v is not a declared value.
foundAllVars=false
fi
done
[[ $foundAllVars != true ]] && return 1
fn="$(declare -f $fn)"
for v in $vars; do
local val="$(eval 'echo $'$v)" # get the value of the varable represented by $v
val="${val//\"/\\\"}" # escape any double-quotes
val="${val//\\/\\\\\\}" # escape any backslashes
fn="$(echo "$fn" | sed -r 's/"?\$'$v'"?/"'"$val"'"/g')" # replace instances of "$$v" and $$v with $val
done
eval "$fn"
}
Usage:
foo="foo bar"
bar='$foo'
baz=baz
fn() {
echo $bar $baz
}
expandFnVars fn bar
declare -f fn
# prints:
# fn ()
# {
# echo "$foo" $baz
# }
expandFnVars fn foo
declare -f fn
# prints:
# fn ()
# {
# echo "foo bar" $baz
# }
Looking at it now, I see one flaw. Suppose $bar in the original function was in single-quotes. We probably would not want its value to be replaced. This could be fixed by some clever regex lookbehinds to count the number of unescaped 's, but I'm happy with it as-is.

Change global positional parameters inside a Bash function

Is there a way to set the positional parameters of a bash script from within a function?
In the global scope one can use set -- <arguments> to change the positional arguments, but it doesn't work inside a function because it changes the function's positional parameters.
A quick illustration:
# file name: script.sh
# called as: ./script.sh -opt1 -opt2 arg1 arg2
function change_args() {
set -- "a" "c" # this doesn't change the global args
}
echo "original args: $#" # original args: -opt1 -opt2 arg1 arg2
change_args
echo "changed args: $#" # changed args: -opt1 -opt2 arg1 arg2
# desired outcome: changed args: a c
As stated before, the answer is no, but if someone need this there's the option of setting an external array (_ARRAY), modifying it from within the function and then using set -- ${_ARRAY[#]} after the fact. For example:
#!/bin/bash
_ARGS=()
shift_globally () {
shift
_ARGS=$#
}
echo "before: " "$#"
shift_globally "$#"
set -- "${_ARGS[#]}"
echo "after: " "$#"
If you test it:
./test.sh a b c d
> before: a b c d
> after: b c d
It's not technically what you're asking for but it's a workaround that might help someone who needs a similar behaviour.
Not really. A function actually has its own scope. A parameter assigned in a function is global by default, but if you explicitly declare it it has local scope:
foo() {
x=3 # global
declare y # local
...
}
The positional parameters are always local to the function, so anything you do to manipulate them will only take effect within the function scope.
Technically, you can always use recursion to solve this issue. If you can confidently call the original script again, you can reorder the arguments that way. E.g.
declare -i depth
outer() {
if (( depth++ < 1 )); then
outer "${#:4:$#}" "${#:1:3}"
return
fi
main "$#"
}
main() {
printf '%s\n' "$#"
}
outer "$#"
This seems like an error prone and confusing solution to a problem I don't understand, but it essentially works.
I was searching for this myself and came up with the below, i.e. return a space separated string of the array (sorry, positional parameters) from the function by printing it to stdout and capture it with command substitution, then parse the string word for word and append it back into the positional arguments. Clear as mud!
Obviously won't work for args with spaces in them below but that's of course possible to workaround too.
#!/bin/sh
fn() {
set -- d e f
res=""
for i; do
res="${res:+${res} }${i}"
done
printf "%s\n" "$res"
}
set -- a b c
rv=$(fn)
set --
for word in $rv; do
set -- "$#" "$word"
done
for i; do
echo positional parms "$i"
done

Is there a way to avoid positional arguments in bash?

I have to write a function in bash. The function will take about 7 arguments. I know that I can call a function like this:
To call a function with parameters:
function_name $arg1 $arg2
And I can refer my parameters like this inside the function:
function_name () {
echo "Parameter #1 is $1"
}
My question is, is there a better way refer to the parameters inside the function? Can I avoid the $1, $2, $3, .... thing and simply use the $arg1, $arg2, ...?
Is there a proper method for this or do I need to re-assign these parameters to some other variables inside the function? E.g.:
function_name () {
$ARG1=$1
echo "Parameter #1 is $ARG1"
}
Any example would be much appreciated.
The common way of doing that is assigning the arguments to local variables in the function, i.e.:
copy() {
local from=${1}
local to=${2}
# ...
}
Another solution may be getopt-style option parsing.
copy() {
local arg from to
while getopts 'f:t:' arg
do
case ${arg} in
f) from=${OPTARG};;
t) to=${OPTARG};;
*) return 1 # illegal option
esac
done
}
copy -f /tmp/a -t /tmp/b
Sadly, bash can't handle long options which would be more readable, i.e.:
copy --from /tmp/a --to /tmp/b
For that, you either need to use the external getopt program (which I think has long option support only on GNU systems) or implement the long option parser by hand, i.e.:
copy() {
local from to
while [[ ${1} ]]; do
case "${1}" in
--from)
from=${2}
shift
;;
--to)
to=${2}
shift
;;
*)
echo "Unknown parameter: ${1}" >&2
return 1
esac
if ! shift; then
echo 'Missing parameter argument.' >&2
return 1
fi
done
}
copy --from /tmp/a --to /tmp/b
Also see: using getopts in bash shell script to get long and short command line options
You can also be lazy, and just pass the 'variables' as arguments to the function, i.e.:
copy() {
local "${#}"
# ...
}
copy from=/tmp/a to=/tmp/b
and you'll have ${from} and ${to} in the function as local variables.
Just note that the same issue as below applies — if a particular variable is not passed, it will be inherited from parent environment. You may want to add a 'safety line' like:
copy() {
local from to # reset first
local "${#}"
# ...
}
to ensure that ${from} and ${to} will be unset when not passed.
And if something very bad is of your interest, you could also assign the arguments as global variables when invoking the function, i.e.:
from=/tmp/a to=/tmp/b copy
Then you could just use ${from} and ${to} within the copy() function. Just note that you should then always pass all parameters. Otherwise, a random variable may leak into the function.
from= to=/tmp/b copy # safe
to=/tmp/b copy # unsafe: ${from} may be declared elsewhere
If you have bash 4.1 (I think), you can also try using associative arrays. It will allow you to pass named arguments but it will be ugly. Something like:
args=( [from]=/tmp/a [to]=/tmp/b )
copy args
And then in copy(), you'd need to grab the array.
You can always pass things through the environment:
#!/bin/sh
foo() {
echo arg1 = "$arg1"
echo arg2 = "$arg2"
}
arg1=banana arg2=apple foo
All you have to do is name variables on the way in to the function call.
function test() {
echo $a
}
a='hello world' test
#prove variable didnt leak
echo $a .
This isn't just a feature of functions, you could have that function in it's own script and call a='hello world' test.sh and it would work just the same
As an extra little bit of fun, you can combine this method with positional arguments (say you were making a script and some users mightn't know the variable names).
Heck, why not let it have defaults for those arguments too? Well sure, easy peasy!
function test2() {
[[ -n "$1" ]] && local a="$1"; [[ -z "$a" ]] && local a='hi'
[[ -n "$2" ]] && local b="$2"; [[ -z "$b" ]] && local b='bye'
echo $a $b
}
#see the defaults
test2
#use positional as usual
test2 '' there
#use named parameter
a=well test2
#mix it up
b=one test2 nice
#prove variables didnt leak
echo $a $b .
Note that if test was its own script, you don't need to use the local keyword.
Shell functions have full access to any variable available in their calling scope, except for those variable names that are used as local variables inside the function itself. In addition, any non-local variable set within a function is available on the outside after the function has been called. Consider the following example:
A=aaa
B=bbb
echo "A=$A B=$B C=$C"
example() {
echo "example(): A=$A B=$B C=$C"
A=AAA
local B=BBB
C=CCC
echo "example(): A=$A B=$B C=$C"
}
example
echo "A=$A B=$B C=$C"
This snippet has the following output:
A=aaa B=bbb C=
example(): A=aaa B=bbb C=
example(): A=AAA B=BBB C=CCC
A=AAA B=bbb C=CCC
The obvious disadvantage of this approach is that functions are not self-contained any more and that setting a variable outside a function may have unintended side-effects. It would also make things harder if you wanted to pass data to a function without assigning it to a variable first, since this function is not using positional parameters any more.
The most common way to handle this is to use local variables for arguments and any temporary variable within a function:
example() {
local A="$1" B="$2" C="$3" TMP="/tmp"
...
}
This avoids polluting the shell namespace with function-local variables.
I think I have a solution for you.
With a few tricks you can actually pass named parameters to functions, along with arrays.
The method I developed allows you to access parameters passed to a function like this:
testPassingParams() {
#var hello
l=4 #array anArrayWithFourElements
l=2 #array anotherArrayWithTwo
#var anotherSingle
#reference table # references only work in bash >=4.3
#params anArrayOfVariedSize
test "$hello" = "$1" && echo correct
#
test "${anArrayWithFourElements[0]}" = "$2" && echo correct
test "${anArrayWithFourElements[1]}" = "$3" && echo correct
test "${anArrayWithFourElements[2]}" = "$4" && echo correct
# etc...
#
test "${anotherArrayWithTwo[0]}" = "$6" && echo correct
test "${anotherArrayWithTwo[1]}" = "$7" && echo correct
#
test "$anotherSingle" = "$8" && echo correct
#
test "${table[test]}" = "works"
table[inside]="adding a new value"
#
# I'm using * just in this example:
test "${anArrayOfVariedSize[*]}" = "${*:10}" && echo correct
}
fourElements=( a1 a2 "a3 with spaces" a4 )
twoElements=( b1 b2 )
declare -A assocArray
assocArray[test]="works"
testPassingParams "first" "${fourElements[#]}" "${twoElements[#]}" "single with spaces" assocArray "and more... " "even more..."
test "${assocArray[inside]}" = "adding a new value"
In other words, not only you can call your parameters by their names (which makes up for a more readable core), you can actually pass arrays (and references to variables - this feature works only in bash 4.3 though)! Plus, the mapped variables are all in the local scope, just as $1 (and others).
The code that makes this work is pretty light and works both in bash 3 and bash 4 (these are the only versions I've tested it with). If you're interested in more tricks like this that make developing with bash much nicer and easier, you can take a look at my Bash Infinity Framework, the code below was developed for that purpose.
Function.AssignParamLocally() {
local commandWithArgs=( $1 )
local command="${commandWithArgs[0]}"
shift
if [[ "$command" == "trap" || "$command" == "l="* || "$command" == "_type="* ]]
then
paramNo+=-1
return 0
fi
if [[ "$command" != "local" ]]
then
assignNormalCodeStarted=true
fi
local varDeclaration="${commandWithArgs[1]}"
if [[ $varDeclaration == '-n' ]]
then
varDeclaration="${commandWithArgs[2]}"
fi
local varName="${varDeclaration%%=*}"
# var value is only important if making an object later on from it
local varValue="${varDeclaration#*=}"
if [[ ! -z $assignVarType ]]
then
local previousParamNo=$(expr $paramNo - 1)
if [[ "$assignVarType" == "array" ]]
then
# passing array:
execute="$assignVarName=( \"\${#:$previousParamNo:$assignArrLength}\" )"
eval "$execute"
paramNo+=$(expr $assignArrLength - 1)
unset assignArrLength
elif [[ "$assignVarType" == "params" ]]
then
execute="$assignVarName=( \"\${#:$previousParamNo}\" )"
eval "$execute"
elif [[ "$assignVarType" == "reference" ]]
then
execute="$assignVarName=\"\$$previousParamNo\""
eval "$execute"
elif [[ ! -z "${!previousParamNo}" ]]
then
execute="$assignVarName=\"\$$previousParamNo\""
eval "$execute"
fi
fi
assignVarType="$__capture_type"
assignVarName="$varName"
assignArrLength="$__capture_arrLength"
}
Function.CaptureParams() {
__capture_type="$_type"
__capture_arrLength="$l"
}
alias #trapAssign='Function.CaptureParams; trap "declare -i \"paramNo+=1\"; Function.AssignParamLocally \"\$BASH_COMMAND\" \"\$#\"; [[ \$assignNormalCodeStarted = true ]] && trap - DEBUG && unset assignVarType && unset assignVarName && unset assignNormalCodeStarted && unset paramNo" DEBUG; '
alias #param='#trapAssign local'
alias #reference='_type=reference #trapAssign local -n'
alias #var='_type=var #param'
alias #params='_type=params #param'
alias #array='_type=array #param'
I was personally hoping to see some sort of syntax like
func(a b){
echo $a
echo $b
}
But since that's not a thing, and a I see quite a few references to global variables (not without the caveat of scoping and naming conflicts), I'll share my approach.
Using the copy function from Michal's answer:
copy(){
cp $from $to
}
from=/tmp/a
to=/tmp/b
copy
This is bad, because from and to are such broad words that any number of functions could use this. You could quickly end up with a naming conflict or a "leak" on your hands.
letter(){
echo "From: $from"
echo "To: $to"
echo
echo "$1"
}
to=Emily
letter "Hello Emily, you're fired for missing two days of work."
# Result:
# From: /tmp/a
# To: Emily
# Hello Emily, you're fired for missing two days of work.
So my approach is to "namespace" them. I name the variable after the function and delete it after the function is done with it. Of course, I only use it for optional values that have default values. Otherwise, I just use positional args.
copy(){
if [[ $copy_from ]] && [[ $copy_to ]]; then
cp $copy_from $copy_to
unset copy_from copy_to
fi
}
copy_from=/tmp/a
copy_to=/tmp/b
copy # Copies /tmp/a to /tmp/b
copy # Does nothing, as it ought to
letter "Emily, you're 'not' re-hired for the 'not' bribe ;)"
# From: (no /tmp/a here!)
# To:
# Emily, you're 'not' re-hired for the 'not' bribe ;)
I would make a terrible boss...
In practice, my function names are more elaborate than "copy" or "letter".
The most recent example to my memory is get_input(), which has gi_no_sort and gi_prompt.
gi_no_sort is a true/false value that determines whether the completion suggestions are sorted or not. Defaults to true
gi_prompt is a string that is...well, that's self-explanatory. Defaults to "".
The actual arguments the function takes are the source of the aforementioned 'completion suggestions' for the input prompt, and as said list is taken from $# in the function, the "named args" are optional[1], and there's no obvious way to distinguish between a string meant as a completion and a boolean/prompt-message, or really anything space-separated in bash, for that matter[2]; the above solution ended up saving me a lot of trouble.
notes:
So a hard-coded shift and $1, $2, etc. are out of the question.
E.g. is "0 Enter a command: {1..9} $(ls)" a value of 0, "Enter a command:", and a set of 1 2 3 4 5 6 7 8 9 <directory contents>? Or are "0", "Enter", "a", and "command:" part of that set as well? Bash will assume the latter whether you like it or not.
Arguments get sent to functions as an tuple of individual items, so they have no names as such, just positions. this allows some interesting possibilities like below, but it does mean that you are stuck with $1. $2, etc. as to whether to map them to better names, the question comes down to how big the function is, and how much clearer it will make reading the code. if its complex, then mapping meaningful names ($BatchID, $FirstName, $SourceFilePath) is a good idea. for simple stuff though, it probably isn't necessary. I certianly wouldn't bother if you are using names like $arg1.
now, if you just want to echo back the parameters, you can iterate over them:
for $arg in "$#"
do
echo "$arg"
done
just a fun fact; unless you are processing a list, you are probably interested in somthing more useful
this is an older topic, but still i'd like to share the function below (requires bash 4). It parses named arguments and sets the variables in the scripts environment. Just make sure you have sane default values for all parameters you need. The export statement at the end could also just be an eval. It's great in combination with shift to extend existing scripts which already take a few positional parameters and you dont want to change the syntax, but still add some flexibility.
parseOptions()
{
args=("$#")
for opt in "${args[#]}"; do
if [[ ! "${opt}" =~ .*=.* ]]; then
echo "badly formatted option \"${opt}\" should be: option=value, stopping..."
return 1
fi
local var="${opt%%=*}"
local value="${opt#*=}"
export ${var}="${value}"
done
return 0
}

Passing arguments by reference

I want to ask if it is possible to pass arguments to a script function by reference:
i.e. to do something that would look like this in C++:
void boo(int &myint) { myint = 5; }
int main() {
int t = 4;
printf("%d\n", t); // t->4
boo(t);
printf("%d\n", t); // t->5
}
So then in BASH I want to do something like:
function boo ()
{
var1=$1 # now var1 is global to the script but using it outside
# this function makes me lose encapsulation
local var2=$1 # so i should use a local variable ... but how to pass it back?
var2='new' # only changes the local copy
#$1='new' this is wrong of course ...
# ${!1}='new' # can i somehow use indirect reference?
}
# call boo
SOME_VAR='old'
echo $SOME_VAR # -> old
boo "$SOME_VAR"
echo $SOME_VAR # -> new
Any thoughts would be appreciated.
It's 2018, and this question deserves an update. At least in Bash, as of Bash 4.3-alpha, you can use namerefs to pass function arguments by reference:
function boo()
{
local -n ref=$1
ref='new'
}
SOME_VAR='old'
echo $SOME_VAR # -> old
boo SOME_VAR
echo $SOME_VAR # -> new
The critical pieces here are:
Passing the variable's name to boo, not its value: boo SOME_VAR, not boo $SOME_VAR.
Inside the function, using local -n ref=$1 to declare a nameref to the variable named by $1, meaning it's not a reference to $1 itself, but rather to a variable whose name $1 holds, i.e. SOME_VAR in our case. The value on the right-hand side should just be a string naming an existing variable: it doesn't matter how you get the string, so things like local -n ref="my_var" or local -n ref=$(get_var_name) would work too. declare can also replace local in contexts that allow/require that. See chapter on Shell Parameters in Bash Reference Manual for more information.
The advantage of this approach is (arguably) better readability and, most importantly, avoiding eval, whose security pitfalls are many and well-documented.
From the Bash man-page (Parameter Expansion):
If the first character of parameter is an exclamation point (!), a
level of variable indirection is introduced. Bash uses the value of
the variable formed from the rest of parameter as the name of the
variable; this variable is then expanded and that value is used in
the rest of the substitution, rather than the value of parameter
itself. This is known as indirect expansion.
Therefore a reference is the variable's name. Here is a swap function using
variable indirection that does not require a temporary variable:
function swap()
{ #
# #param VARNAME1 VARNAME2
#
eval "$1=${!2} $2=${!1}"
}
$ a=1 b=2
$ swap a b
$ echo $a $b
2 1
Use a helper function upvar:
# Assign variable one scope above the caller.
# Usage: local "$1" && upvar $1 value [value ...]
# Param: $1 Variable name to assign value to
# Param: $* Value(s) to assign. If multiple values, an array is
# assigned, otherwise a single value is assigned.
# NOTE: For assigning multiple variables, use 'upvars'. Do NOT
# use multiple 'upvar' calls, since one 'upvar' call might
# reassign a variable to be used by another 'upvar' call.
# See: http://fvue.nl/wiki/Bash:_Passing_variables_by_reference
upvar() {
if unset -v "$1"; then # Unset & validate varname
if (( $# == 2 )); then
eval $1=\"\$2\" # Return single value
else
eval $1=\(\"\${#:2}\"\) # Return array
fi
fi
}
And use it like this from within Newfun():
local "$1" && upvar $1 new
For returning multiple variables, use another helper function upvars. This allows passing multiple variables within one call, thus avoiding possible conflicts if one upvar call changes a variable used in another subsequent upvar call.
See: http://www.fvue.nl/wiki/Bash:_Passing_variables_by_reference for helper function upvars and more information.
The problem with:
eval $1=new
is that it's not safe if $1 happens to contain a command:
set -- 'ls /;true'
eval $1=new # Oops
It would be better to use printf -v:
printf -v "$1" %s new
But printf -v cannot assign arrays.
Moreover, both eval and printf won't work if the variable happens to be declared local:
g() { local b; eval $1=bar; } # WRONG
g b # Conflicts with `local b'
echo $b # b is empty unexpected
The conflict stays there even if local b is unset:
g() { local b; unset b; eval $1=bar; } # WRONG
g b # Still conflicts with `local b'
echo $b # b is empty unexpected
I have found a way to do this but I am not sure how correct this is:
Newfun()
{
local var1="$1"
eval $var1=2
# or can do eval $1=2 if no local var
}
var=1
echo var is $var # $var = 1
newfun 'var' # pass the name of the variable…
echo now var is $var # $var = 2
So we pass the variable name as opposed to the value and then use eval ...
Bash doesn't have anything like references built into it, so basically the only way you would be able to do what you want is to pass the function the name of the global variable you want it to modify. And even then you'll need an eval statement:
boo() {
eval ${1}="new"
}
SOME_VAR="old"
echo $SOME_VAR # old
boo "SOME_VAR"
echo $SOME_VAR # new
I don't think you can use indirect references here because Bash automatically accesses the value of the variable whose name is stored in the indirect reference. It doesn't give you the chance to set it.
Ok, so this question has been waiting for a 'real' solution for some time now, and I am glad to say that we can now accomplish this without using eval at all.
The key to remember is to declare a reference in both the caller as the callee, at least in my example:
#!/bin/bash
# NOTE this does require a bash version >= 4.3
set -o errexit -o nounset -o posix -o pipefail
passedByRef() {
local -n theRef
if [ 0 -lt $# ]; then
theRef=$1
echo -e "${FUNCNAME}:\n\tthe value of my reference is:\n\t\t${theRef}"
# now that we have a reference, we can assign things to it
theRef="some other value"
echo -e "${FUNCNAME}:\n\tvalue of my reference set to:\n\t\t${theRef}"
else
echo "Error: missing argument"
exit 1
fi
}
referenceTester() {
local theVariable="I am a variable"
# note the absence of quoting and escaping etc.
local -n theReference=theVariable
echo -e "${FUNCNAME}:\n\tthe value of my reference is:\n\t\t${theReference}"
passedByRef theReference
echo -e "${FUNCNAME}:\n\tthe value of my reference is now:\n\t\t${theReference},\n\tand the pointed to variable:\n\t\t${theVariable}"
}
Output when run:
referenceTester:
the value of my reference is:
I am a variable
passedByRef:
the value of my reference is:
I am a variable
passedByRef:
value of my reference set to:
some other value
referenceTester:
the value of my reference is now:
some other value,
and the pointed to variable:
some other value
Eval should never be used on a string that a user can set because its dangerous. Something like "string; rm -rf ~" will be bad. So generally its best to find solutions where you don't have to worry about it.
However, eval will be needed to set the passed variables, as the comment noted.
$ y=four
$ four=4
$ echo ${!y}
4
$ foo() { x=$1; echo ${!x}; }
$ foo four
4
#!/bin/bash
append_string()
{
if [ -z "${!1}" ]; then
eval "${1}='$2'"
else
eval "${1}='${!1}''${!3}''$2'"
fi
}
PETS=''
SEP='|'
append_string "PETS" "cat" "SEP"
echo "$PETS"
append_string "PETS" "dog" "SEP"
echo "$PETS"
append_string "PETS" "hamster" "SEP"
echo "$PETS"
Output:
cat
cat|dog
cat|dog|hamster
Structure for calling that function is:
append_string name_of_var_to_update string_to_add name_of_var_containing_sep_char
Name of variable is passed to fuction about PETS and SEP while string to append is passed the usual way as value. "${!1}" refers to contents of global PETS variable. In the beginning that variable is empty and contens is added each time we call the function. Separator character can be selected as needed. "eval" starting lines update PETS variable.
This is what works for me on Ubuntu bash shell
#!/bin/sh
iteration=10
increment_count()
{
local i
i=$(($1+1))
eval ${1}=\$i
}
increment_count iteration
echo $iteration #prints 11
increment_count iteration
echo $iteration #prints 12

Resources