I am stuck with a peculiar situation, where in from python I am printing two strings one by one and reading it in bash script (which calls the python code piece)
I am expecting array size to be 2, but somehow, bash considers spaces also as a element separator and return me size of 3.
Example scripts
multi_line_return.py file has following content
print("foo bar")
print(5)
multi_line_call.sh has following content
#!/bin/bash
PYTHON_EXE="ABSOLUTE_PATH TO PYTHON EXECUTABLE IN LINUX"
CURR_DIR=$(cd $(dirname ${BASH_SOURCE[0]}) && pwd)/
array=()
while read line ; do
array+=($line)
done < <(${PYTHON_EXE} ${CURR_DIR}multi_line_return.py)
echo "array length --> ${#array[#]}"
echo "each variable in new line"
for i in "${array[#]}"
do
printf $i
printf "\n"
done
Now keep both of the above file in same directory and make following call to see result.
bash multi_line_call.sh
As you can see in result,
I am getting
array length = 3
1.foo, 2.bar & 3. 5
The expectation is
One complete line of python output (stdout) as one element of bash array
array length = 2
1. foo bar & 2. 5
Put quotes around $line to prevent it from being split:
array+=("$line")
You can also do it without a loop using readarray:
readarray array < <(${PYTHON_EXE} ${CURR_DIR}multi_line_return.py)
Related
this is probably a very simple question. I looked at other answers but couldn't come up with a solution. I have a 365 line date file. file as below,
01-01-2000
02-01-2000
I need to read this file line by line and assign each day to a separate variable. like this,
d001=01-01-2000
d002=02-01-2000
I tried while read commands but couldn't get them to work.It takes a lot of time to shoot one by one. How can I do it quickly?
Trying to create named variable out of an associative array, is time waste and not supported de-facto. Better use this, using an associative array:
#!/bin/bash
declare -A array
while read -r line; do
printf -v key 'd%03d' $((++c))
array[$key]=$line
done < file
Output
for i in "${!array[#]}"; do echo "key=$i value=${array[$i]}"; done
key=d001 value=01-01-2000
key=d002 value=02-01-2000
Assumptions:
an array is acceptable
array index should start with 1
Sample input:
$ cat sample.dat
01-01-2000
02-01-2000
03-01-2000
04-01-2000
05-01-2000
One bash/mapfile option:
unset d # make sure variable is not currently in use
mapfile -t -O1 d < sample.dat # load each line from file into separate array location
This generates:
$ typeset -p d
declare -a d=([1]="01-01-2000" [2]="02-01-2000" [3]="03-01-2000" [4]="04-01-2000" [5]="05-01-2000")
$ for i in "${!d[#]}"; do echo "d[$i] = ${d[i]}"; done
d[1] = 01-01-2000
d[2] = 02-01-2000
d[3] = 03-01-2000
d[4] = 04-01-2000
d[5] = 05-01-2000
In OP's code, references to $d001 now become ${d[1]}.
A quick one-liner would be:
eval $(awk 'BEGIN{cnt=0}{printf "d%3.3d=\"%s\"\n",cnt,$0; cnt++}' your_file)
eval makes the shell variables known inside your script or shell. Use echo $d000 to show the first one of the newly defined variables. There should be no shell special characters (like * and $) inside your_file. Remove eval $() to see the result of the awk command. The \" quoted %s is to allow spaces in the variable values. If you don't have any spaces in your_file you can remove the \" before and after %s.
I'm trying to get back an array from perl to bash.
My perl scrip has an array and then I use return(#arr)
from my bash script I use
VAR = `perl....
when I echo VAR
I get the aray as 1 long string with all the array vars connected with no spaces.
Thanks
In the shell (and in Perl), backticks (``) capture the output of a command. However, Perl's return is normally for returning variables from subroutines - it does not produce output, so you probably want print instead. Also, in bash, array variables are declared with parentheses. So this works for me:
$ ARRAY=(`perl -wMstrict -le 'my #array = qw/foo bar baz/; print "#array"'`); \
echo "<${ARRAY[*]}> 0=${ARRAY[0]} 1=${ARRAY[1]} 2=${ARRAY[2]}"
<foo bar baz> 0=foo 1=bar 2=baz
In Perl, interpolating an array into a string (like "#array") will join the array with the special variable $" in between elements; that variable defaults to a single space. If you simply print #array, then the array elements will be joined by the variable $,, which is undef by default, meaning no space between the elements. This probably explains the behavior you mentioned ("the array vars connected with no spaces").
Note that the above will not work the way you expect if the elements of the array contain whitespace, because bash will split them into separate array elements. If your array does contain whitespace, then please provide an MCVE with sample data so we can perhaps make an alternative suggestion of how to return that back to bash. For example:
( # subshell so IFS is only affected locally
IFS=$'\n'
ARRAY=(`perl -wMstrict -e 'my #array = ("foo","bar","quz baz"); print join "\n", #array'`)
echo "0=<${ARRAY[0]}> 1=<${ARRAY[1]}> 2=<${ARRAY[2]}>"
)
Outputs: 0=<foo> 1=<bar> 2=<quz baz>
Here is one way using Bash word splitting, it will split the string on white space into the new array array:
array_str=$(perl -E '#a = 1..5; say "#a"')
array=( $array_str )
for item in ${array[#]} ; do
echo ": $item"
done
Output:
: 1
: 2
: 3
: 4
: 5
Check out the following transcript. With all possible rigor and formality, what is going on at each step?
$> ls -1 #This command prints 3 items. no explanation required.
a
b
c
$> X=$(ls -1) #Capture the output (as what? a string?)
$> Y=($(ls -1)) #Capture it again (as an array now?)
$> echo ${#X[#]} #Why is the length 1?
1
$> echo ${#Y[#]} #This works because Y is an array of the 3 items?
3
$> echo $X #Why are the linefeeds now spaces?
a b c
$> echo $Y #Why does the array echo as its first element
a
$> for x in $X;do echo $x; done #iterate over $X
a
b
c
$> for y in $Y;do echo $y; done #iterating over y doesn't work
a
$> echo ${X[2]} #I can loop over $X but not index into it?
$> echo ${Y[2]} #Why does this work if I can't loop over $Y?
c
I assume bash has well established semantics about how arrays and text variables (if that's even what they're called) work, but the user manual is not organized in an optimal fashion for someone who wants to reason about scripts based on whatever small set of underlying principles the language designer intended.
Let me preface the following with the very strong suggestion that you never use ls to populate an array. The correct code would be
Z=( * )
to create an array with each (non-hidden) file in the current directory as a distinct array element.
$> ls -1 #This command prints 3 items. no explanation required.
a
b
c
Correct. Each file name is printed on a separate line (although, beware of file names containing newlines; the parts before and after each newline would appear as separate file names.)
$> X=$(ls -1) #Capture the output (as what? a string?)
Yes. The output of ls is concatenated by the command substitution into a single string using a single space to separate each line. (The command substitution would be subject to word-splitting if it weren't the right-hand side of an assignment; word-splitting will come up below.)
$> Y=($(ls -1)) #Capture it again (as an array now?)
Same as with X, but now each of the words in the result of the command substitution is treated as a separate array element. As long as none of the output lines contain any characters in the value of IFS, each file name is one word and will be treated as a separate array element.
$> echo ${#X[#]} #Why is the length 1?
1
X, not being a real array, is treated as an array with a single element, namely the value of $X.
$> echo ${#Y[#]} #This works because Y is an array of the 3 items?
3
Correct.
$> echo $X #Why are the linefeeds now spaces?
a b c
When $X is unquoted, the resulting expansion is subject to word-splitting. In this case, the newlines are simply treated the same as any other whitespace, separating the result into a sequence of words that are passed to echo as distinct arguments, which are then displayed separated by a single space each.
$> echo $Y #Why does the array echo as its first element
a
For a true array, $Y is equivalent to ${Y[0]}.
$> for x in $X;do echo $x; done #iterate over $X
a
b
c
This works, but has caveats.
$> for y in $Y;do echo $y; done #iterating over y doesn't work
a
See above; $Y only expands to the first element. You want for y in "${Y[#]}"; do to iterate over all the elements.
$> echo ${X[2]} #I can loop over $X but not index into it?
Correct. X is not an array, but $X expanded to a space-separated list which the for loop could iterate over.
$> echo ${Y[2]} #Why does this work if I can't loop over $Y?
c
Indexing and iteration are two completely different things in shell. You don't actually iterate over an array; you iterate over the resulting sequence of words of a properly expanded array.
I have a text file in the following format:
A Apple
A Ant
B Bat
B Ball
The number of definitions of each character can be any number.
I am writing a shell script which will receive inputs like "A B". The output of the shell script I am expecting is the possible string sequences which can be created.
For input "A B", the outputs will be:
Apple Bat
Apple Ball
Ant Bat
Ant Ball
I tried arrays, It is not working as expected. Can anyone help with some ideas on how to solve this issue?
Use associative arrays to accomplish this:
#!/usr/bin/env bash
first_letter=$1
second_letter=$2
declare -A words # declare associative array
while read -r alphabet word; do # read ignores blank lines in input file
words+=(["$word"]="$alphabet") # key = word, value = alphabet
done < words.txt
for word1 in "${!words[#]}"; do
alphabet1="${words[$word1]}"
[[ $alphabet1 != $first_letter ]] && continue
for word2 in "${!words[#]}"; do
alphabet2="${words[$word2]}"
[[ $alphabet2 != $second_letter ]] && continue
printf "$word1 $word2\n" # print matching word pairs
done
done
Output with A B passed in as arguments (with the content in your question):
Apple Ball
Apple Bat
Ant Ball
Ant Bat
You may want to refer to this post for more info on associative arrays:
Appending to a hash table in Bash
I want to run a shell script that can simultaneously loop through two variables.
So that I can have an input and output file name. I feel like this isn't too hard of a concept but any help is appreciated.
Files = "File1,
File2,
...
FileN
"
Output = OutFile1,
Outfile2,
...
OutfileN
"
and I would in theory my code would be:
for File in $Files
do
COMMAND --file $File --ouput $Output
done
Obviously, there needs to be another loop but I'm stuck, any help is appreciated.
You don't really need to loop 2 variables, just use 2 BASH arrays:
input=("File1" "File2" "File3")
output=("OutFile1" "OutFile2" "OutFile3")
for ((i=0; i<${#input[#]}; i++)); do
echo "Processing input=${input[$i]} and output=${output[$i]}"
done
zsh enables multiple loop variables before the list.
#!/bin/zsh
input2output=(
'File1' 'Outfile1'
'File2' 'Outfile2'
)
for input ouput in $input2output
do
echo "[$input] --> [$ouput]"
done
quotes from zsh(5.9) manual or man zshmisc
for name ... [ in word ... ] term do list done
More than one parameter name can appear before the list of words. If N names are given, then on each execution of the loop the next N words are assigned to the corresponding parameters. If there are more names than remaining words, the remaining parameters are each set to the empty string.