The output is
/ ext4
/boot ext2
tank zfs
On each line the delimiter is a space. I need an associative array like:
"/" => "ext4", "/boot" => "ext2", "tank" => "zfs"
How is this done in bash?
If the command output is in file file, then:
$ declare -A arr=(); while read -r a b; do arr["$a"]="$b"; done <file
Or, you can read the data directly from a command cmd into an array as follows:
$ declare -A arr=(); while read -r a b; do arr["$a"]="$b"; done < <(cmd)
The construct <(...) is process substitution. It allows us to read from a command the same as if we were reading from a file. Note that the space between the two < is essential.
You can verify that the data was read correctly using declare -p:
$ declare -p arr
declare -A arr='([tank]="zfs" [/]="ext4" [/boot]="ext2" )'
Related
I am trying to store the output of this:
mdfind "kMDItemContentType == 'com.apple.application-bundle'"
Output is like this:
/Applications/Safari.app
/Applications/Xcode.app
/Applications/Xcode.app/Contents/Applications/Accessibility Inspector.app
/Applications/Xcode.app/Contents/Applications/RealityComposer.app
/Applications/Xcode.app/Contents/Applications/FileMerge.app
/Applications/Xcode.app/Contents/Applications/Instruments.app
/Applications/Xcode.app/Contents/Applications/Create ML.app
I try to store it as an array but the contents are split per spaces:
bash-5.1$ arr=( $(/usr/bin/mdfind "kMDItemContentType == 'com.apple.application-bundle'") )
bash-5.1$ echo ${arr[2]}
/Applications/Xcode.app/Contents/Applications/Accessibility
bash-5.1$ echo ${arr[3]}
Inspector.app
bash-5.1$
So how can I do the trick?
Use readarray and process substitution to read a null-delimited series of paths into an array.
readarray -d '' arr < <(mdfind -0 "...")
The -0 option tells mdfind to terminate each path with a null byte instead of a linefeed. (This guards agains rare, but legal, path names that include linefeeds. Null bytes are not a valid character for any path component.)
The -d '' option tells readarray to treat the null byte as the end of a "line".
readarray populates an array with one "line" of input per element. The input is the output of mdfind; the process substitution ensures that readarray executes in the current shell, not a subshell induced by a pipe like
mdfind -0 "..." | readarray -d '' arr
(Under some situations, you can make the last job of a pipeline execute in the current shell; that's beyond the scope of this answer, though.)
Example:
# Using printf to simulate mdfind -0
$ readarray -d '' arr < <(printf 'foo\nbar\000baz\000')
$ declare -p arr
declare -a arr=([0]=$'foo\nbar' [1]="baz")
So I know I can use a single IFS in a read statement, but is it possible to use two. For instance if I have the text
variable = 5 + 1;
print variable;
And I have the code to assign every word split to an array, but I also want to split at the ; as well as a space, if it comes up.
Here is the code so far
INPUT="$1"
declare -a raw_parse
while IFS=' ' read -r -a raw_input; do
for raw in "${raw_input[#]}"; do
raw_parse+=("$raw")
done
done < "$INPUT"
What comes out:
declare -a raw_parse=([0]="variable" [1]="=" [2]="5" [3]="+" [4]="1;" [5]="print" [6]="variable;")
What I want:
declare -a raw_parse=([0]="variable" [1]="=" [2]="5" [3]="+" [4]="1" [5]=";" [6]="print" [7]="variable" [8]=";")
A workaround with GNU sed. This inserts a space before every ; and replaces every newline with a space.
read -r -a raw_input < <(sed -z 's/;/ ;/g; s/\n/ /g' "$INPUT")
declare -p raw_input
Output:
declare -a raw_input=([0]="variable" [1]="=" [2]="5" [3]="+" [4]="1" [5]=";" [6]="print" [7]="variable" [8]=";")
I am using a bash for loop to cycle through a directory and print the file size. My issue though is that I need to assign a unique variable to each of the values so that they can be used later but the array is inputting all the data as one element in the array. I have tried both a double for loop and a if statement with a nested for loop but did not get the right results.
Question: How can I fix the below code to match my needs or is there a better method?
declare -a byte
for b in /home/usr/frames/*
do
byte+=$(wc -c < $b)
done
declare -p byte
With associative array (if available)
#!/usr/bin/env bash
for b in /home/usr/frames/*; do
declare -A byte["$b"]=$(wc -c < "$b")
done
Use Parameter Expansion to extract just the file name.
declare -A byte["${b##*/}"]=$(wc -c < "$b")
Now check the value of byte
declare -p byte
A variation on OPs original code and jetchisel's answer:
unset byte
declare -A byte # -A == associative array
for b in /home/usr/frames/*
do
byte[$(basename "${b}")]=$(wc -c < "${b}")
done
declare -p byte
Using some *.txt files on my host:
$ pwd
/c/temp/cygwin
$ wc -c *.txt
22 file.txt
405 somefile.txt
214 test.txt
With the modified for clause:
unset byte
declare -A byte
for b in /c/temp/cygwin/*.txt
do
byte[$(basename "${b}")]=$(wc -c < "${b}")
done
declare -p byte
Generates:
declare -A byte=([somefile.txt]="405" [test.txt]="214" [file.txt]="22" )
I'm trying to split key value pairs (around an = sign) which I then use to edit a config file, using bash. But I need an alternative to the <<< syntax for IFS.
The below works on my host system, but when i log in to my ubuntu virtual machine through ssh I have the wrong bash version. Whatever I try, <<< fails. (I am definitely calling the right version of bash at the top of the file, using #!/bin/bash (and I've tried #!/bin/sh etc too)).
I know I can use IFS as follows on my host mac os x system:
var="word=hello"
IFS='=' read -a array <<<"$var"
echo ${array[0]} ${array[1]]}
#alternative -for calling through e.g. sh file.sh param=value
for var in "$#"
do
IFS='=' read -a array <<<"$var"
echo ${array[0]} ${array[1]]}
done
#alternative
IFS='=' read -ra array <<< "a=b"
declare -p array
echo ${array[0]} ${array[1]}
But this doesn't work on my vm.
I also know that I can should be able to switch the <<< syntax through backticks, $() or echo "$var" | ... but I can't get it to work - as follows:
#Fails
IFS='=' read -ra myarray -d '' <"$var"
echo ${array[0]} ${array[1]]}
#Fails
echo "$var" | IFS='=' read -a array
echo ${array[0]} ${array[1]]}
#fails
echo "a=b" | IFS='=' read -a array
declare -p array
echo ${array[0]} ${array[1]}
Grateful for any pointers as I'm really new to bash.
Your first failed attempt is because < and <<< are different operators. < opens the named file.
The second fails because read only sets the value of array in the subshell started by the pipe; that shell exits after the completion of the pipe, and array disappears with it.
The third fails for the same reason as the second; the declare that follows doesn't make any difference.
Your attempts have been confounded because you have to use the variable in the same sub-shell as read.
$ echo 'foo=bar' | { IFS='=' read -a array; echo ${array[0]}; }
foo
And if you want your variable durable (ie, outside the sub-shell scope):
$ var=$(echo 'foo=bar' | { IFS='=' read -a array; echo ${array[0]}; })
$ echo $var
foo
Clearly, it isn't pretty.
Update: If -a is missing, that suggests you're out of the land of arrays. You can try parameter substitution:
str='foo=bar'
var=${str%=*}
val=${str#*=}
And if that doesn't work, fall back to good ole cut:
str='foo=bar'
var=$(echo $str | cut -f 1 -d =)
val=$(echo $str | cut -f 2 -d =)
Here is how to split a tab-split line into an array:
IFS=$'\t' read -a array < <(echo -e "a\tb\tc")
And here is how to read null-terminated lines into an array:
while IFS= read -r -d '' item
do array+=("$item")
done < <(echo -e "a\0b\0c\0")
Now, is it possible to combine both and have a one-liner suitable to read a null-split line into an array?
First, putting -r -d '' together with -a array, and setting IFS to empty, doesn't to work:
IFS= read -r -d '' -a array < <(echo -e "a\0b\0c")
as array will have 1 item: a
Secondly, such while construct reading a null-split line into an array is mildly unsatisfactory. It will drop the last item if the last item isn't followed by null. A workaround is to append the last item to the array after the loop, as suggested by chepner below.
while IFS= read -r -d '' item
do array+=("$item")
done < <(echo -e "a\0b\0c")
array+=("$item")
There isn't, if you don't consider a single while loop a "one"-liner.
while IFS= read -r -d '' item; do array+=("$item"); done < <(echo -e "a\0b\0c\0")
read only reads a line of input at a time, and you are using -d '' to define what a line is. That is, you aren't treating "a\0b\0c" as a single line of input, but as multiple lines separate by null byte. The readarray command, introduced in bash 4, doesn't provide a way to specify what to consider as the line separator.
readarray a.k.a. mapfile supports nul termination with the -d '' flag, at least in bash 5.2.2 where I tested, and 4.4.20 on a test Ubuntu 18.04 container, which is as far back as I cared to go.
TL;DR:
mapfile -d '' ARRAYNAME < <(command that emits null bytes here)
mapfile -d '' ARRAYNAME < file-with-null-bytes
Order is significant, arrayname must come after the -d ''. Using a pipe instead will not work how you expect, because the mapfile will run in a subshell, hence the use of command-redirection above.
To append to an existing array, use
mapfile -d '' -O "${#ARR[#]}" ARR
Demo and explanation
Set up a NUL-terminated stream with one plain entry, one entry with embedded space, and one entry with embedded newline. Use xargs -0 to show it's correct.
$ python3 -c 'print("\x00".join(["1", "2 3", "4\n5"]), end="\x00")'
12 34
5
$ python3 -c 'print("\x00".join(["1", "2 3", "4\n5"]), end="\x00")' \
| xargs -0 -I{} echo "[{}]"
[1]
[2 3]
[4
5]
Python is only used to generate the demo input; use whatever else you prefer.
Now we use mapfile -d '' to read the demo null terminated stream and verify that it was split solely on null bytes:
$ mapfile -d '' ARR < <(python3 -c 'print("\x00".join(["1", "2 3", "4\n5"]), end="\x00")')
$ echo "${#ARR[#]}"
3
$ printf "[%s]" "${ARR[#]}"
[1][2 3][4
5]$
Note:
readarray is just an alias for mapfile
You can write mapfile -d $'\0' ARRAYNAME instead if you want, to make your intent more explicit. $'xxxxx' is a bash escape-string. But bash strings are null-terminated, so $'\0' is the same as the empty-string anyway. mapfile special-cases the zero-length delimiter argument to mean null terminated.
Order of arguments to mapfile is significant. Array name must be the last argument. mapfile ARRAYNAME -d '' is not the same as mapfile -d '' ARRAYNAME. Redirection operators are removed by the calling shell so they don't count.
Invoking mapfile in a pipeline won't export the resulting array to the shell that invokes the pipeline, because the mapfile in the pipeline will run in its own separate shell process. Use process substitution redirection < <(command) instead of a pipe.
So this won't work
$ # DELIBERATELY WRONG #
$ python3 -c 'print("\x00".join(["1", "2 3", "4\n5"]), end="\x00")' | mapfile -d '' ARR2
$ echo "${#ARR2[#]}"
0
$ echo "${ARR2[#]}"
$ # DELIBERATELY WRONG #
I used Python for the demo output because working with null bytes in bash is too painful. Here's an alternative using only sh etc, using the creation of files and then find -print0 to get the desired output:
mkdir omg
cd omg
touch '1' '2 3' $'4\n5'
find . -print0