Set a shell array from an array in Perl script - bash

I have the following Perl script:
sub {
my $sequence="SEQUENCE1";
my $sequence2="SEQUENCE2";
my #Array = ($sequence, $sequence2);
return \#Array;
}
-1
I want to retrieve the values of the array via a bash script
#!/bin/bash
seq=$(perl vectTEST.pl)
# retrieve the column 1 of the Array
echo $seq[0]
My approach doesn't work.

You can't return an array. The concept makes no sense since print produces a stream of bytes, not variables.
One solution is to output a text representation of the array and have the shell parse it.
For example,
$ IFS=$'\n' array=( $(
perl -e'
my #array = ("a b", "c d", "e f");
print "$_\n" for #array;
'
) )
$ echo ${#array[#]}
3
$ echo "${array[1]}"
c d
This particular implementation assumes your array can't contain newlines.
The other alternative is to print out shell code that recreates the array and eval that code in the shell.
For example,
$ eval "array=( $(
perl -e'
use String::ShellQuote qw( shell_quote );
my #array = ("a b", "c d", "e f");
print join " ", map shell_quote($_), #array;
'
) )"
$ echo ${#array[#]}
3
$ echo "${array[1]}"
c d
This is a robust solution.

You can do this, but you need to change vectTEST.pl -- currently you have an anonymous sub that you're not assigning to anything. Change the perl script to:
$vect = sub {
my $sequence="SEQUENCE1";
my $sequence2="SEQUENCE2";
my #Array=();
push(#Array,$sequence,$sequence2);
return \#Array;
};
1;
Then, you can do this in bash:
mapfile -t seq < <(perl -E 'do "vectTEST.pl"; say join "\n", #{$vect->()}')
for idx in "${!seq[#]}"; do echo "$idx ${seq[idx]}"; done
0 SEQUENCE1
1 SEQUENCE2

Did you test your Perl script? In order to have that Perl script give you something to put into your shell script, you to make sure your Perl script works:
$ test.pl
No output at all.
First issue, you put the whole Perl script in sub. Subroutines in Perl don't execute unless you call them. You can't even do that since your subroutine doesn't even have a name. Let's get rid of the subroutine:
my $sequence="SEQUENCE1";
my $sequence2="SEQUENCE2";
my #Array = ($sequence, $sequence2);
print \#Array . "\n";
Okay, now let's try the program:
$ test.pl
ARRAY(0x7f8bab8303e0)
You're printing out an array reference with that \ in the front of #Array. Let's print out the array itself:
my $sequence="SEQUENCE1";
my $sequence2="SEQUENCE2";
my #Array = ($sequence, $sequence2);
print #Array, "\n";
That will now print #Array:
$ test.pl
SEQUENCE1SEQUENCE2
Not quite. There's no spaces between each element. Let's set $, which is the output field separator to be a single space:
my $sequence="SEQUENCE1";
my $sequence2="SEQUENCE2";
my #Array = ($sequence, $sequence2);
$,=' ';
print #Array, "\n";
Now:
$ test.pl
SEQUENCE1 SEQUENCE2
Now, we have a working Perl program that outputs what we need to put into your shell array:
seq=($(test.pl))
echo ${seq[*]}
SEQUENCE1 SEQUENCE2
When you have an issue, you need to break it down into pieces. Your first issue is that your Perl script wasn't working. Once that was fixed, you could now use it to initialize your array in your Bash shell.

Related

How to iterate over multiple variables and echo them using Shell Script?

Consider the below variables which are dynamic and might change each time. Sometimes there might even be 5 variables, But the length of all the variables will be the same every time.
var1='a b c d e... upto z'
var2='1 2 3 4 5... upto 26'
var3='I II III IV V... upto XXVI'
I am looking for a generalized approach to iterate the variables in a for loop & My desired output should be like below.
a,1,I
b,2,II
c,3,III
d,4,IV
e,5,V
.
.
goes on upto
z,26,XXVI
If I use nested loops, then I get all possible combinations which is not the expected outcome.
Also, I know how to make this work for 2 variables using for loop and shift using below link
https://unix.stackexchange.com/questions/390283/how-to-iterate-two-variables-in-a-sh-script
With paste
paste -d , <(tr ' ' '\n' <<<"$var1") <(tr ' ' '\n' <<<"$var2") <(tr ' ' '\n' <<<"$var3")
a,1,I
b,2,II
c,3,III
d,4,IV
e...z,5...26,V...XXVI
But clearly having to add other parameter substitutions for more varN's is not scalable.
You need to "zip" two variables at a time.
var1='a b c d e...z'
var2='1 2 3 4 5...26'
var3='I II III IV V...XXVI'
zip_var1_var2 () {
set $var1
for v2 in $var2; do
echo "$1,$v2"
shift
done
}
zip_var12_var3 () {
set $(zip_var1_var2)
for v3 in $var3; do
echo "$1,$v3"
shift
done
}
for x in $(zip_var12_var3); do
echo "$x"
done
If you are willing to use eval and are sure it is safe to do so, you can write a single function like
zip () {
if [ $# -eq 1 ]; then
eval echo \$$1
return
fi
a1=$1
shift
x=$*
set $(eval echo \$$a1)
for v in $(zip $x); do
printf '=== %s\n' "$1,$v" >&2
echo "$1,$v"
shift
done
}
zip var1 var2 var3 # Note the arguments are the *names* of the variables to zip
If you can use arrays, then (for example, in bash)
var1=(a b c d e)
var2=(1 2 3 4 5)
var3=(I II III IV V)
for i in "${!var1[#]}"; do
printf '%s,%s,%s\n' "${var1[i]}" "${var2[i]}" "${var3[i]}"
done
Use this Perl one-liner:
perl -le '#in = map { [split] } #ARGV; for $i ( 0..$#{ $in[0] } ) { print join ",", map { $in[$_][$i] } 0..$#in; }' "$var1" "$var2" "$var3"
Prints:
a,1,I
b,2,II
c,3,III
d,4,IV
e,5,V
z,26,XXVI
The Perl one-liner uses these command line flags:
-e : Tells Perl to look for code in-line, instead of in a file.
-l : Strip the input line separator ("\n" on *NIX by default) before executing the code in-line, and append it when printing.
The input variables must be quoted with double quotes "like so", to keep the blank-separated words from being treated as separate arguments.
#ARGV is an array of the command line arguments, here $var1, $var2, $var3.
#in is an array of 3 elements, each element being a reference to an array obtained as a result of splitting the corresponding element of #ARGV on whitespace. Note that split splits the string on whitespace by default, but you can specify a different delimiter, it accepts regexes.
The subsequent for loop prints #in elements separated by comma.
SEE ALSO:
perldoc perlrun: how to execute the Perl interpreter: command line switches
perldoc perlvar: Perl predefined variables
The following is (almost) a copy of this answer with a few tweaks that make it fit this question.
The Original Question
First let’s assign a few variables to play with, 26 tokens in each of them:
var1="$(echo {a..z})"
var2="$(echo {1..26})"
var3="$(echo I II III IV \
V{,I,II,III} IX \
X{,I,II,III} XIV \
XV{,I,II,III} XIX \
XX{,I,II,III} XXIV \
XXV XXVI)"
var4="$(echo {A..Z})"
var5="$(echo {010101..262626..10101})"
Now we want a “magic” function that zips an arbitrary number of variables, ideally in pure Bash:
zip_vars var1 # a trivial test
zip_vars var{1..2} # a slightly less trivial test
zip_vars var{1..3} # the original question
zip_vars var{1..4} # more vars, becasuse we can
zip_vars var{1..5} # more vars, because why not
What could zip_vars look like? Here’s one in pure Bash, without any external commands:
zip_vars() {
local var
for var in "$#"; do
local -a "array_${var}"
local -n array_ref="array_${var}"
array_ref=(${!var})
local -ar "array_${var}"
done
local -n array_ref="array_${1}"
local -ir size="${#array_ref[#]}"
local -i i
local output
for ((i = 0; i < size; ++i)); do
output=
for var in "$#"; do
local -n array_ref="array_${var}"
output+=",${array_ref[i]}"
done
printf '%s\n' "${output:1}"
done
}
How it works:
It splits all variables (passed by reference (by variable name)) into arrays. For each variable varX it creates a local array array_varX.
It would be actually way easier if the input variables were already Bash arrays to start with (see below), but … we stick with the original question initially.
It determines the size of the first array and then blindly expects all arrays to be of that size.
For each index i from 0 to size - 1 it concatenates the ith elements of all arrays, separated by ,.
Arrays Make Things Easier
If you use Bash arrays from the very start, the script will be shorter and look simpler and there won’t be any string-to-array conversions.
zip_arrays() {
local -n array_ref="$1"
local -ir size="${#array_ref[#]}"
local -i i
local output
for ((i = 0; i < size; ++i)); do
output=
for arr in "$#"; do
local -n array_ref="$arr"
output+=",${array_ref[i]}"
done
printf '%s\n' "${output:1}"
done
}
arr1=({a..z})
arr2=({1..26})
arr3=( I II III IV
V{,I,II,III} IX
X{,I,II,III} XIV
XV{,I,II,III} XIX
XX{,I,II,III} XXIV
XXV
XXVI)
arr4=({A..Z})
arr5=({010101..262626..10101})
zip_arrays arr1 # a trivial test
zip_arrays arr{1..2} # a slightly less trivial test
zip_arrays arr{1..3} # (almost) the original question
zip_arrays arr{1..4} # more arrays, becasuse we can
zip_arrays arr{1..5} # more arrays, because why not

Is it possible to save perl hash into bash array?

I have done some processing in perl, and got the result in perl's hash data structure. Usually in bash, when I try to retrieve result from other script like
output=$(perl -E '...')
I got the output in string. Is it possible to save the result in bash array?
Assuming a perl variable hash is an associative array, please try:
declare -A "output=($(perl -e '
$hash{"foo"} = "xx"; # just an example
$hash{"bar"} = "yy"; # ditto
for (keys %hash) {print "[\"$_\"]=\"$hash{$_}\"\n"}'))"
for i in "${!output[#]}"; do
echo "$i => ${output[$i]}" # see the result
done
The outermost double quotes around output=.. is required to tell declare
to evaluate the argument.
[Update]
Considering tripleee's comment, here is a robust version against special characters:
mapfile -d "" -t a < <(perl -e '
$hash{"baz"} = "boo"; # example
$hash{"foo"} = "x\"x"; # example with a double quote
$hash{"bar"} = "y\ny"; # example with a newline
print join("\0", %hash), "\0"') # use a nul byte as a delimiter
declare -A output # bash associative array
for ((i = 0; i < ${#a[#]}; i+=2 )); do
output[${a[i]}]=${a[i+1]} # key and value pair
done
for i in "${!output[#]}"; do
echo "$i => ${output[$i]}" # see the result
done
The conversion from perl variables to bash variables works only if they are free of null bytes (\n), as perl can store null bytes in strings, but bash cannot.
At least, we can use that limitation to print the hash in perl with null delimiters and safely parse it in bash again:
declare -A "array=($(
perl -e 'print join("\0", %hash), "\0"' |
xargs -0 printf '[%q]=%q '
))"
Please note that neither %q nor -0 are specified by posix. For a more portable solution see tshiono's answer.
If the hash is very big such that ARG_MAX might be exceeded you should ensure that xargs does not split a key value pair across two calls to printf. To do so, add the option -n2 (or any other number 2n where you are sure that n key value pairs never exceed ARG_MAX).

return array from perl to bash

I'm trying to get back an array from perl to bash.
My perl scrip has an array and then I use return(#arr)
from my bash script I use
VAR = `perl....
when I echo VAR
I get the aray as 1 long string with all the array vars connected with no spaces.
Thanks
In the shell (and in Perl), backticks (``) capture the output of a command. However, Perl's return is normally for returning variables from subroutines - it does not produce output, so you probably want print instead. Also, in bash, array variables are declared with parentheses. So this works for me:
$ ARRAY=(`perl -wMstrict -le 'my #array = qw/foo bar baz/; print "#array"'`); \
echo "<${ARRAY[*]}> 0=${ARRAY[0]} 1=${ARRAY[1]} 2=${ARRAY[2]}"
<foo bar baz> 0=foo 1=bar 2=baz
In Perl, interpolating an array into a string (like "#array") will join the array with the special variable $" in between elements; that variable defaults to a single space. If you simply print #array, then the array elements will be joined by the variable $,, which is undef by default, meaning no space between the elements. This probably explains the behavior you mentioned ("the array vars connected with no spaces").
Note that the above will not work the way you expect if the elements of the array contain whitespace, because bash will split them into separate array elements. If your array does contain whitespace, then please provide an MCVE with sample data so we can perhaps make an alternative suggestion of how to return that back to bash. For example:
( # subshell so IFS is only affected locally
IFS=$'\n'
ARRAY=(`perl -wMstrict -e 'my #array = ("foo","bar","quz baz"); print join "\n", #array'`)
echo "0=<${ARRAY[0]}> 1=<${ARRAY[1]}> 2=<${ARRAY[2]}>"
)
Outputs: 0=<foo> 1=<bar> 2=<quz baz>
Here is one way using Bash word splitting, it will split the string on white space into the new array array:
array_str=$(perl -E '#a = 1..5; say "#a"')
array=( $array_str )
for item in ${array[#]} ; do
echo ": $item"
done
Output:
: 1
: 2
: 3
: 4
: 5

change bash script code to perl

In bash :
#!/bin/bash
var=$(cat ps.txt)
for i in $var ; do
echo $i
done
and ps.txt is :
356735
535687
547568537
7345673
3653468
2376958764
12345678
12345
Now I want to do that with perl or i want to know how to save the output of a command in a variable in perl like var=$(cat ps.txt)
Instead of using cat to get file contents into a Perl variable, you should use open and <> in "slurp mode":
open my $fh, "<", "ps.txt" or die "Failed to open ps.txt: $!";
local $/;
my $file_contents = <$fh>;
Here are some ways to do it:
#!/usr/bin/perl
$ifile = "ps.txt";
# capture command output
# NOTE: this puts each line in a separate array element -- the newline is _not_
# stripped
#bycat = (`cat $ifile`);
# this strips the newline from all array elements:
chomp(#bycat);
# so would this:
# NOTE: for this type of foreach, if you modify $buf, it also modifies the
# corresponding array element
foreach $buf (#bycat) {
chomp($buf);
}
# read in all elements line-by-line
open($fin,"<$ifile") or die("unable to open '$ifile' -- $!\n");
while ($buf = <$fin>) {
chomp($buf);
push(#byread,$buf);
}
close($fin);
# print the arrays
# NOTE: we are passing the arrays "by-reference"
show("bycat",\#bycat);
show("byread",\#byread);
# show -- dump the array
sub show
# sym -- name of array
# ptr -- reference to array
{
my($sym,$ptr) = #_;
my($buf);
foreach $buf (#$ptr) {
printf("%s: %s\n",$sym,$buf);
}
}
I'm not sure what this is trying to achieve, but this is my answer:
my $var = `/bin/cat $0`; # the Perl program itself ;-)
print $var;
If you need the lines, $var can be split on $/.
#! /usr/bin/perl -w
my $var = `/bin/cat $0`;
print $var;
my $n = 1;
for my $line ( split( $/, $var ) ){
print "$n: $line\n";
$n++;
}

How to interpret a string as command to execute with parameters in perl

Trying to find an equivalent solution for below in perl. Suppose I have the following POSIX shell script:
#!/bin/sh
MY_CMD='WDT=$1; shift ; printf "%-${WDT}.${WDT}s\n" "$#"'
eval $MY_CMD
An suppose that above is being saved as my_script.sh
And if I execute it like: ./my_script.sh 4 Hello to thise nice girls then the output will be like:
Hell
to
this
nice
girl
How can I do the same using perl. Here the main problems is how do I take perl input parameters and pass them to my command saved in an variable so that those parameters will get evaluated accordingly to the ones from input (in shell eval does this properly).
Should it be that I asked horribly stupid questions, please excuse me as I just started learning perl .o)
EDIT: I think I need to clarify myself more with the question... I NEED to have the command in a variable. I have simplified the script example here to understand easily the problem. In all solutions so far you are giving me solution how that same task to be done in perl, but please don't focus on the command itself, that is just an example:
Other Examples would be:
MY_CMD='export AWKNUMF="%.2f"; exe 93 "$1" "$2" "$3" $(shift 3; echo "$#") | sort -k1,1 | exe 93 ":" 1 2'
and so on...
The idea that MY_CMD variable would be populated with some command retrieved from a repository which expects some parameters and I want those parameters to be provided in the input to the perl.
SYNOPSYS would be
./perl_script.pl my_command_name [param1 [param2 ... [paramN]]]
The point here is that you should not focus on the content of the MY_CMD variable. It is just a shell command(s) which gets parameters along.
the perl equivalent would be something like:
system ($my_cmd, "#ARGV"); but this of course does not work as expected.
No eval needed in Perl (I doubt it's needed in the shell, either).
#!/usr/bin/perl
use warnings;
use strict;
my $width = shift;
printf "%-$width.${width}s\n", $_ for #ARGV;
There's two elements to your request.
Firstly - perl uses #ARGV to hold command line parameters. You can access individual elements either with shift or $ARGV[1].
Secondly - eval works in perl too - but you'll probably find there's a better way of doing what you're trying to do.
So to take your example:
#!/usr/bin/perl
use strict;
use warnings;
my ( $WDT, #words ) = #ARGV;
foreach my $word ( #words ) {
printf ( "%-${WDT}.${WDT}s\n", $word );
}
How can I do the same using perl.
Perl too has the eval() function. It can take a string and evaluate it in the context it was called from.
Here the main problems is how do I take perl input parameters and pass them to my command saved in an variable so that those parameters will get evaluated accordingly to the ones from input (in shell eval does this properly).
You have to create the variables before the eval'ed Perl code can access. That also would, depending on the implementation, bypass or conflict directly with the use strict.
I'm away of two way to create the variables: using eval or by creating them by manipulating the symbol table. Consider:
my $value_of_a = 10;
my $value_of_b = 20;
# alt1 : create vars using `eval`
eval '$a = '.$value_of_a;
eval '$b = '.$value_of_b;
eval q{ print "$a + $b = ", $a+$b, "\n"; };
# alt2 : create the vars by manipulating the symbol table directly
$var_name = "aa";
${"::$var_name"} = $value_of_a;
$var_name = "bb";
${"::$var_name"} = $value_of_b;
eval q{ print "$aa + $bb = ", $aa+$bb, "\n"; };
To avoid calling the eval every time to interpret the code, one can also try to wrap the code into a nameless function and eval it once to create the subroutine which can be called at any time. Consider:
# the input data:
my $var_name_a = 'a';
my $var_name_b = 'b';
my $value_of_a = 10;
my $value_of_b = 20;
my $cmd = 'print "$a + $b = ", $a+$b, "\n";';
# the preparations:
eval '$'.$var_name_a.' = '.$value_of_a;
eval '$'.$var_name_b.' = '.$value_of_b;
my $sub = eval 'sub { '. $cmd .' }';
# the execution:
$sub->();

Resources