I'm trying to get back an array from perl to bash.
My perl scrip has an array and then I use return(#arr)
from my bash script I use
VAR = `perl....
when I echo VAR
I get the aray as 1 long string with all the array vars connected with no spaces.
Thanks
In the shell (and in Perl), backticks (``) capture the output of a command. However, Perl's return is normally for returning variables from subroutines - it does not produce output, so you probably want print instead. Also, in bash, array variables are declared with parentheses. So this works for me:
$ ARRAY=(`perl -wMstrict -le 'my #array = qw/foo bar baz/; print "#array"'`); \
echo "<${ARRAY[*]}> 0=${ARRAY[0]} 1=${ARRAY[1]} 2=${ARRAY[2]}"
<foo bar baz> 0=foo 1=bar 2=baz
In Perl, interpolating an array into a string (like "#array") will join the array with the special variable $" in between elements; that variable defaults to a single space. If you simply print #array, then the array elements will be joined by the variable $,, which is undef by default, meaning no space between the elements. This probably explains the behavior you mentioned ("the array vars connected with no spaces").
Note that the above will not work the way you expect if the elements of the array contain whitespace, because bash will split them into separate array elements. If your array does contain whitespace, then please provide an MCVE with sample data so we can perhaps make an alternative suggestion of how to return that back to bash. For example:
( # subshell so IFS is only affected locally
IFS=$'\n'
ARRAY=(`perl -wMstrict -e 'my #array = ("foo","bar","quz baz"); print join "\n", #array'`)
echo "0=<${ARRAY[0]}> 1=<${ARRAY[1]}> 2=<${ARRAY[2]}>"
)
Outputs: 0=<foo> 1=<bar> 2=<quz baz>
Here is one way using Bash word splitting, it will split the string on white space into the new array array:
array_str=$(perl -E '#a = 1..5; say "#a"')
array=( $array_str )
for item in ${array[#]} ; do
echo ": $item"
done
Output:
: 1
: 2
: 3
: 4
: 5
Related
I am new to Bash coding. I would like to concatenate a string to each element of a comma-separated strings "array".
This is an example of what I have in mind:
s=a,b,c
# Here a function to concatenate the string "_string" to each of them.
# Expected result:
a_string,b_string,c_string
One way:
$ s=a,b,c
$ echo ${s//,/_string,}_string
a_string,b_string,c_string
Using a proper array is generally a much more robust solution. It allows the values to contain literal commas, whitespace, etc.
s=(a b c)
printf '%s\n' "${s[#]/%/_string}"
As suggested by chepner, you can use IFS="," to merge the result with commas.
(IFS=","; echo "${s[#]/%/_string}")
(The subshell is useful to keep the scope of the IFS reassignment from leaking to the current shell.)
Simply, you could use a for loop
main() {
local input='a,b,c'
local append='_string'
# create an 'output' variable that is empty
local output=
# convert the input into an array called 'items' (without the commas)
IFS=',' read -ra items <<< "$input"
# loop over each item in the array, and append whatever string we want, in this case, '_string'
for item in "${items[#]}"; do
output+="${item}${append},"
done
# in the loop, the comma was re-added back. now, we must remove the so there are only commas _in between_ elements
output=${output%,}
echo "$output"
}
main
I've split it up in three steps:
Make it into an actual array.
Append _string to each element in the array using Parameter expansion.
Turn it back into a scalar (for which I've made a function called turn_array_into_scalar).
#!/bin/bash
function turn_array_into_scalar() {
local -n arr=$1 # -n makes `arr` a reference the array `s`
local IFS=$2 # set the field separator to ,
arr="${arr[*]}" # "join" over IFS and assign it back to `arr`
}
s=a,b,c
# make it into an array by turning , into newline and reading into `s`
readarray -t s < <(tr , '\n' <<< "$s")
# append _string to each string in the array by using parameter expansion
s=( "${s[#]/%/_string}" )
# use the function to make it into a scalar again and join over ,
turn_array_into_scalar s ,
echo "$s"
I have a filename as a string, say filname="a/b/c/d.png".
Is there a general method to extract the parent directory at a given level using ONLY shell parameter expansion?
I.e. I would like to extract "level 1" and return c or "level 2" and return b.
Explicitly, I DO NOT want to get the entire parent path (i.e. a/b/c/, which is the result of ${filename%/*}).
Using just shell parameter expansion, assuming bash, you can first transform the path into an array (splitting on /) and then ask for specific array indexes:
filename=a/b/c/d.png
IFS=/
filename_array=( $filename )
unset IFS
echo "0 = ${filename_array[0]}"
echo "1 = ${filename_array[1]}"
echo "2 = ${filename_array[2]}"
echo "3 = ${filename_array[3]}"
Running the above produces:
0 = a
1 = b
2 = c
3 = d.png
These indexes are the reverse of what you want, but a little
arithmetic should fix that.
Using zsh, the :h modifier trims the final component off a path in variable expansion.
The (s:...:) parameter expansion flag can be used to split the contents of a variable. Combine those with normal array indexing where a negative index goes from the end of the array, and...
$ filename=a/b/c/d.png
$ print $filename:h
a/b/c
$ level=1
$ print ${${(s:/:)filename:h}[-level]}
c
$ level=2
$ print ${${(s:/:)filename:h}[-level]}
b
You could also use array subscript flags instead to avoid the nested expansion:
$ level=1
$ print ${filename[(ws:/:)-level-1]}
c
$ level=2
$ print ${filename[(ws:/:)-level-1]}
b
w makes the index of a scalar split on words instead of by character, and s:...: has the same meaning, to say what to split on. Have to subtract one from the level to skip over the trailing d.png, since it's not stripped off already like the first way.
The :h (head) and :t (tail) expansion modifiers in zsh accept digits to specify a level; they can be combined to get a subset of the path:
> filname="a/b/c/d.png"
> print ${filname:t2}
c/d.png
> print ${filname:t2:h1}
c
> print ${filname:t3:h1}
b
If the level is in a variable, then the F modifier can be used to repeat the h modifier a specific number of times:
> for i in 1 2 3; printf '%s: %s\n' $i ${filname:F(i)h:t}
1: c
2: b
3: a
If using printf (a shell builtin) is allowed then this will do the trick in bash:
filename='a/b/c/d.png'
level=2
printf -v spaces '%*s' $level
pattern=${spaces//?/'/*'}
component=${filename%$pattern}
component=${component##*/}
echo $component
prints out
b
You can assign different values to the variable level.
I am stuck with a peculiar situation, where in from python I am printing two strings one by one and reading it in bash script (which calls the python code piece)
I am expecting array size to be 2, but somehow, bash considers spaces also as a element separator and return me size of 3.
Example scripts
multi_line_return.py file has following content
print("foo bar")
print(5)
multi_line_call.sh has following content
#!/bin/bash
PYTHON_EXE="ABSOLUTE_PATH TO PYTHON EXECUTABLE IN LINUX"
CURR_DIR=$(cd $(dirname ${BASH_SOURCE[0]}) && pwd)/
array=()
while read line ; do
array+=($line)
done < <(${PYTHON_EXE} ${CURR_DIR}multi_line_return.py)
echo "array length --> ${#array[#]}"
echo "each variable in new line"
for i in "${array[#]}"
do
printf $i
printf "\n"
done
Now keep both of the above file in same directory and make following call to see result.
bash multi_line_call.sh
As you can see in result,
I am getting
array length = 3
1.foo, 2.bar & 3. 5
The expectation is
One complete line of python output (stdout) as one element of bash array
array length = 2
1. foo bar & 2. 5
Put quotes around $line to prevent it from being split:
array+=("$line")
You can also do it without a loop using readarray:
readarray array < <(${PYTHON_EXE} ${CURR_DIR}multi_line_return.py)
I'm trying to understand a shell script written by a previous group member. there is this for loop. I can understand it's looping through a list ${!mylist[#]} but I've only seen ${mylist[#]} before, not ${!mylist[#]}.
What does the exclamation mark do here?
for i in ${!mylist[#]};
do
echo ${mylist[i]}
....
done
${!mylist[#]} returns the keys (or indices) to an an array. This differs from ${mylist[#]} which returns the values in the array.
As an example, let's consider this array:
$ arr=(abc def ghi)
In order to get its keys (or indices in this case):
$ echo "${!arr[#]}"
0 1 2
In order to get its values:
$ echo "${arr[#]}"
abc def ghi
From man bash:
It is possible to obtain the keys (indices) of an array as well
as the values. ${!name[#]} and ${!name[*]} expand to the indices
assigned in array variable name. The treatment when in double quotes
is similar to the expansion of the special parameters # and * within
double quotes.
Example using associative arrays
To show that the same applies to associative arrays:
$ declare -A Arr=([a]=one [b]=two)
$ echo "${!Arr[#]}"
a b
$ echo "${Arr[#]}"
one two
After thoroughly searching for a way to create an associative array in bash, I found that declare -A array will do the trick. But the problem is, it is only for bash version 4 and the bash version the server has in our system is 3.2.16.
How can I achieve some sort of associative array-like hack in bash 3? The values will be passed to a script like
ARG=array[key];
./script.sh ${ARG}
EDIT: I know that I can do this in awk, or other tools but strict bash is needed for the scenario I am trying to solve.
Bash 3 has no associative arrays, so you're going to have to use some other language feature(s) for your purpose. Note that even under bash 4, the code you wrote doesn't do what you claim it does: ./script.sh ${ARG} does not pass the associative array to the child script, because ${ARG} expands to nothing when ARG is an associative array. You cannot pass an associative array to a child process, you need to encode it anyway.
You need to define some argument passing protocol between the parent script and the child script. A common one is to pass arguments in the form key=value. This assumes that the character = does not appear in keys.
You also need to figure out how to represent the associative array in the parent script and in the child script. They need not use the same representation.
A common method to represent an associative array is to use separate variables for each element, with a common naming prefix. This requires that the key name only consists of ASCII letters (of either case), digits and underscores. For example, instead of ${myarray[key]}, write ${myarray__key}. If the key is determined at run time, you need a round of expansion first: instead of ${myarray[$key]}, write
n=myarray__${key}; echo ${!n}
For an assignment, use printf -v. Note the %s format to printf to use the specified value. Do not write printf -v "myarray__${key}" %s "$value" since that would treat $value as a format and perform printf % expansion on it.
printf -v "myarray__${key}" %s "$value"
If you need to pass an associative array represented like this to a child process with the key=value argument representation, you can use ${!myarray__*} to enumerate over all the variables whose name begins with myarray__.
args=()
for k in ${!myarray__*}; do
n=$k
args+=("$k=${!n}")
done
In the child process, to convert arguments of the form key=value to separate variables with a prefix:
for x; do
if [[ $x != *=* ]]; then echo 1>&2 "KEY=VALUE expected, but got $x"; exit 120; fi
printf -v "myarray__${x%%=*}" %s "${x#*=}"
done
By the way, are you sure that this is what you need? Instead of calling a bash script from another bash script, you might want to run the child script in a subshell instead. That way it would inherit from all the variables of the parent.
Here is another post/explanation on associative arrays in bash 3 and older using parameter expansion:
https://stackoverflow.com/a/4444841
Gilles' method has a nice if statement to catch delimiter issues, sanitize oddball input ...etc. Use that.
If you are somewhat familiar with parameter expansion:
http://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html
To use in your scenario [ as stated: sending to script ]:
Script 1:
sending_array.sh
# A pretend Python dictionary with bash 3
ARRAY=( "cow:moo"
"dinosaur:roar"
"bird:chirp"
"bash:rock" )
bash ./receive_arr.sh "${ARRAY[#]}"
Script 2: receive_arr.sh
argAry1=("$#")
function process_arr () {
declare -a hash=("${!1}")
for animal in "${hash[#]}"; do
echo "Key: ${animal%%:*}"
echo "Value: ${animal#*:}"
done
}
process_arr argAry1[#]
exit 0
Method 2, sourcing the second script:
Script 1:
sending_array.sh
source ./receive_arr.sh
# A pretend Python dictionary with bash 3
ARRAY=( "cow:moo"
"dinosaur:roar"
"bird:chirp"
"bash:rock" )
process_arr ARRAY[#]
Script 2: receive_arr.sh
function process_arr () {
declare -a hash=("${!1}")
for animal in "${hash[#]}"; do
echo "Key: ${animal%%:*}"
echo "Value: ${animal#*:}"
done
}
References:
Passing arrays as parameters in bash
If you don't want to handle a lot of variables, or keys are simply invalid variable identifiers, and your array is guaranteed to have less than 256 items, you can abuse function return values. This solution does not require any subshell as the value is readily available as a variable, nor any iteration so that performance screams. Also it's very readable, almost like the Bash 4 version.
Here's the most basic version:
hash_index() {
case $1 in
'foo') return 0;;
'bar') return 1;;
'baz') return 2;;
esac
}
hash_vals=("foo_val"
"bar_val"
"baz_val");
hash_index "foo"
echo ${hash_vals[$?]}
More details and variants in this answer
You can write the key-value pairs to a file and then grep by key. If you use a pattern like
key=value
then you can egrep for ^key= which makes this pretty safe.
To "overwrite" a value, just append the new value at the end of the file and use tail -1 to get just the last result of egrep
Alternatively, you can put this information into a normal array using key=value as value for the array and then iterator over the array to find the value.
This turns out to be ridiculously easy. I had to convert a bash 4 script that used a bunch of associative arrays to bash 3. These two helper functions did it all:
array_exp() {
exp=${#//[/__}
eval "${exp//]}"
}
array_clear() {
unset $(array_exp "echo \${!$1__*}")
}
I'm flabbergasted that this actually works, but that's the beauty of bash.
E.g.
((all[ping_lo] += counts[ping_lo]))
becomes
array_exp '((all[ping_lo] += counts[ping_lo]))'
Or this print statement:
printf "%3d" ${counts[ping_lo]} >> $return
becomes
array_exp 'printf "%3d" ${counts[ping_lo]}' >> $return
The only syntax that changes is clearing. This:
counts=()
becomes
array_clear counts
and you're set. You could easily tell array_exp to recognize expressions like "=()" and handle them by rewriting them as array_clear expressions, but I prefer the simplicity of the above two functions.