I hopefully would like to say I understand quotings used in BASH and their difference , " ", ' ', $' '.
I saw many shell scripts containing
IFS=$'\n'
but NO
IFS="\n"
It looks at least to me that there is no difference.
and in my environment both work correctly (for my understanding),
What difference is here? Is it just a custom?
They aren't the same.
IFS=$'\n' sets the value of IFS to a literal newline.
IFS="\n" sets the value of IFS to the string \n.
See?
$ IFS=$'\n'
$ declare -p IFS
declare -- IFS="
"
$ IFS="\n"
$ declare -p IFS
declare -- IFS="\\n"
$ IFS="\n" read a b c <<<$'anbncndn'
$ declare -p a b c
declare -- a="a"
declare -- b="b"
declare -- c="cndn"
$ IFS=$'\n' read a b c <<<$'anbncndn'
$ declare -p a b c
declare -- a="anbncndn"
declare -- b=""
declare -- c=""
Related
I have a variable like LINE=foo,bar,,baz
I tried to split this with delimiter , using 2 different techniques:
array=(${LINE//,/ })
array=($(echo "$LINE" | tr ',' '\n'))
echo ${array[2]} should return an empty value but it returns baz
(Both treat baz as the 3rd value when it should be the 4th instead.)
You can do this with read -a and an alternate delimiter:
IFS=, read -a array <<<"$LINE"
Note that since the assignment to IFS is a prefix to the read command, it only applies to that one command, and you don't have to set it back to normal afterward. Also, unlike the ones that depend on word-splitting of unquoted variables, it won't try to "expand" any entries that look like filename wildcards into lists of matching files.
Demo:
$ LINE=foo,bar,,baz
$ IFS=, read -a array <<<"$LINE"
$ declare -p array
declare -a array='([0]="foo" [1]="bar" [2]="" [3]="baz")'
You are relying on a sequence of whitespace between tokens as the separator, but of course, that will lose any empty strings.
As a somewhat crude workaround, temporarily override IFS:
oldIFS=$IFS
IFS=','
array=($LINE)
IFS=$oldIFS
Demo: https://ideone.com/Dd1gUV
By default array treat all 'blank' characters as delimiters. If you need to preserve empty values you have to take care of them first.
line='foo, ,bar,,baz'
line=${line//,,/,null,}
line=${line//,[[:blank:]],/,null,}
array=(${line//,/ })
$ echo "${array[#]}"
foo null bar null baz
$ echo "${array[#]//null/ }"
foo bar baz
You could use mapfile (or readarray, same thing):
$ LINE=foo,bar,,baz
$ declare -a PARTS
$ mapfile -t -d, PARTS <<<"$LINE"
$ declare -p PARTS
declare -a PARTS=([0]="foo" [1]="bar" [2]="" [3]=$'baz\n')
There's an extraneous newline at the end of the 3rd element, hence the $'baz\n' value, so you'd have to handle that (see discussion in comments, though). Not sure where it comes from.
So I know I can use a single IFS in a read statement, but is it possible to use two. For instance if I have the text
variable = 5 + 1;
print variable;
And I have the code to assign every word split to an array, but I also want to split at the ; as well as a space, if it comes up.
Here is the code so far
INPUT="$1"
declare -a raw_parse
while IFS=' ' read -r -a raw_input; do
for raw in "${raw_input[#]}"; do
raw_parse+=("$raw")
done
done < "$INPUT"
What comes out:
declare -a raw_parse=([0]="variable" [1]="=" [2]="5" [3]="+" [4]="1;" [5]="print" [6]="variable;")
What I want:
declare -a raw_parse=([0]="variable" [1]="=" [2]="5" [3]="+" [4]="1" [5]=";" [6]="print" [7]="variable" [8]=";")
A workaround with GNU sed. This inserts a space before every ; and replaces every newline with a space.
read -r -a raw_input < <(sed -z 's/;/ ;/g; s/\n/ /g' "$INPUT")
declare -p raw_input
Output:
declare -a raw_input=([0]="variable" [1]="=" [2]="5" [3]="+" [4]="1" [5]=";" [6]="print" [7]="variable" [8]=";")
I have a string in a variable Var.
And the value looks like this:
Var="Key1:Val1~Key2:Val2~"
I just simply need this split by "~" and assigned to an array in KSH only
When I try Var2=$(echo $Var | sed $'s/~/\\n/g')
and check the size of Var2 array as follows:
ArrSize=${#Var2[#]}
I always get 1. I would have imagined that would be 2. Please Help
Assuming you want to use the x=( list of array items ) method of populating the array then you need to wrap the right side of the assignment in a pair of parens, eg:
$ Var2=( $( echo $Var | sed $'s/~/\\n/g' ) )
$ typeset -p Var2
typeset -a Var2=(Key1:Val1 Key2:Val2)
$ echo "${#Var2[#]}"
2
Other options that accomplish the same thing but reduce the overhead of subprocess calls:
here string:
$ Var2=( $(sed 's/~/ /g' <<< "${Var}") )
$ typeset -p Var2
typeset -a Var2=(Key1:Val1 Key2:Val2)
$ echo "${#Var2[#]}"
2
parameter substitution:
$ Var2=( ${Var//\~/ } )
$ typeset -p Var2
typeset -a Var2=(Key1:Val1 Key2:Val2)
$ echo "${#Var2[#]}"
2
NOTE: while ${var//~/ } works in ksh, other shells (eg, bash) require the ~ to be escaped (ie, \~); ksh appears to work with both - ~ and \~ = so I've updated the answer to include the escape
$ cat file.txt
example1#domain1-username
I'd like to parse file.txt and export the contents to variables like:
var1=example1#domain1
var2=username
Using just Bash builtins:
$ IFS=- read var1 var2 <<< "$(< file.txt)"
$ declare -p var1 var2
declare -- var1="example1#domain1"
declare -- var2="username"
This sets the field separator IFS to -, then reads the file into the two variables.
<<< "$(< file.txt)" is a but unwieldy, as we're treating the file just like the single line of text that it is.
var1=$(cut -d "-" -f 1 file.txt)
var2=$(cut -d "-" -f 2 file.txt)
The following command will set var1 and var2 in a single pass over file.txt:
. file.txt
I'm trying to populate an associative array with the output of a command. I can do it without a command as:
$ declare -A x=( [first]=foo [second]=bar )
$ echo "${x[first]}, ${x[second]}"
foo, bar
and I can populate a non-associative array with command output as:
$ declare y=( $(echo 'foo bar') )
$ echo "${y[0]}, ${y[1]}"
foo, bar
but when I try to build on both of the above to create a statement that will populate an associative array from a command, I get the following error message:
$ declare -A z=( $(echo '[first]=foo [second]=bar') )
-bash: z: $(echo '[first]=foo [second]=bar'): must use subscript when assigning associative array
Why am I getting that error message and what is the correct syntax to populate an associative array with the output of a command? I am trying to avoid using eval for the usual reasons, do not want to use a temp file, and of course echo is just being used as an example of a command that produces the effect in question, the real command will be more complicated.
So, based on a couple of the answers below, it looks like it was just my quoting that was a problem:
$ declare -A z="( $(echo '[first]=foo [second]=bar') )"
$ echo "${z[first]}, ${z[second]}"
foo, bar
and with spaces in the indices and values:
$ declare -A z="( $(echo '[first field]="foo with space" [second]="space bar"') )"
$ echo "${z[first field]}, ${z[second]}"
foo with space, space bar
EDIT in response to a question in the comments about why the quotes are necessary (How do I populate a bash associative array with command output?) - I don't exactly know but maybe someone else can explain using the results of this script as reference (not expecting the specified indices to be used in the indexed arrays, they're just part of the strings being populated as the array values):
$ cat tst.sh
#!/bin/env bash
set -x
printf 'Indexed, no quotes\n'
declare -a w=( $(echo '[first]=foo [second]=bar') )
declare -p w
printf '\n---\n'
printf 'Indexed, with quotes\n'
declare -a x="( $(echo '[first]=foo [second]=bar') )"
declare -p x
printf '\n---\n'
printf 'Associative, no quotes\n'
declare -A y="( $(echo '[first]=foo [second]=bar') )"
declare -p y
printf '\n---\n'
printf 'Associative, with quotes\n'
declare -A z=( $(echo '[first]=foo [second]=bar') )
declare -p z
.
$ ./tst.sh
+ printf 'Indexed, no quotes\n'
Indexed, no quotes
+ w=($(echo '[first]=foo [second]=bar'))
++ echo '[first]=foo [second]=bar'
+ declare -a w
+ declare -p w
declare -a w=([0]="[first]=foo" [1]="[second]=bar")
+ printf '\n---\n'
---
+ printf 'Indexed, with quotes\n'
Indexed, with quotes
++ echo '[first]=foo [second]=bar'
+ declare -a 'x=( [first]=foo [second]=bar )'
+ declare -p x
declare -a x=([0]="bar")
+ printf '\n---\n'
---
+ printf 'Associative, no quotes\n'
Associative, no quotes
++ echo '[first]=foo [second]=bar'
+ declare -A 'y=( [first]=foo [second]=bar )'
+ declare -p y
declare -A y=([second]="bar" [first]="foo" )
+ printf '\n---\n'
---
+ printf 'Associative, with quotes\n'
Associative, with quotes
+ z=($(echo '[first]=foo [second]=bar'))
./tst.sh: line 24: z: $(echo '[first]=foo [second]=bar'): must use subscript when assigning associative array
+ declare -A z
+ declare -p z
declare -A z=()
Here is a traditional while loop approach to populate an associative array from a command's output:
while IFS= read -r; do
declare -A z+="( $REPLY )"
done < <(printf '[first]=foo [second]=bar\n[third]=baz\n')
# check output
$> echo "${z[first]}, ${z[second]}, ${z[third]}"
foo, bar, baz
# or declare -p
$> declare -p z
declare -A z='([third]="baz" [second]="bar" [first]="foo" )'
EDIT: Your original attempt will also work with proper quotes:
$> unset z
$> declare -A z="( $(echo '[first]=foo [second]=bar') )"
$> declare -p z
declare -A z='([second]="bar" [first]="foo" )'
I imagine this is somewhat brittle, but you can make the entire z=(...) assignment the result of a command substitution.
declare -A "$(echo z="($(echo '[first]=foo [second]=bar'))")"
Given that this works:
declare -A z=([first]=$(echo 'foo') [second]=$(echo 'bar'))
I'm guessing that Bash needs to see the associative array initialization list before doing any substitutions. So I don't see a way to avoid eval:
eval "declare -A z=($(echo '[first]=foo [second]=bar'))"
What is a "usual reason" to avoid eval?