This question already has answers here:
How to sort an array in Bash
(20 answers)
Closed 2 years ago.
In bash, there is an array like this:
arr=(12345_34, 5_32134, 8_123, 13_1234)
And I'd like to sort (decreasing order) this array based on the numbers before the underscore.
So the desired result is the following:
(12345_34, 13_1234, 8_123, 5_32134)
I tried sort -t _-k 2 -g $arr
arr=(12345_34 5_32134 8_123 13_1234)
readarray -t arr_sorted < <(printf '%s\n' "${arr[#]}" | sort -r -t _ -g)
declare -p arr_sorted
...properly emits as output the ordering specified in the question:
declare -a arr_sorted=([0]="12345_34" [1]="13_1234" [2]="8_123" [3]="5_32134")
If you need to target versions of bash too old to have readarray, a while read loop can substitute, with considerable loss of terseness:
# define the input array
arr=(12345_34 5_32134 8_123 13_1234)
# generate a sorted version
arr_sorted=( )
while IFS= read -r item; do
arr_sorted+=( "$item" )
done < <(printf '%s\n' "${arr[#]}" | sort -r -t _ -g)
# print the sorted version to demonstrate that we built it correctly
declare -p arr_sorted
Try this:
sorted=($(printf '%s\n' "${arr[#]}" | sort -nr))
Related
This question already has answers here:
Read values into a shell variable from a pipe
(17 answers)
Closed 12 months ago.
I just discovered mapfile in bash when shellcheck recommended it to my code to capture two shell parameters from a space-delimited string of two words. But, mapfile does not seem to function consistently. For example, this works as expected:
mapfile arr < myfile
The array arr is populated with entries corresponding to the lines of myfile. However, this does not work:
echo -e "hello\nworld" | mapfile arr
The array arr is not populated at all. And, this doesn't work either:
echo "hello world" | mapfile -d ' ' arr
I am not sure why it would make a difference where the standard input for the command comes from. I didn't know it would be possible to distinguish what the input came from, a file or a pipeline.
Any clues?
Note to moderator: It was suggested my question was a duplicate to Read values into a shell variable from a pipe . I do not agree. Nowhere is mapfile mentioned in that question, nor was there any other useful Q/A found in a SO search. In addition, that referenced question did not deal with shell parameter assignments. Therefore, this question and answers are valuable.
Technically the array is being populated; the issue is that mapfile is called in a sub-process which, when it exits (back to the command line) the array assignment is lost, ie, you can't pass assignments 'up' from a sub-process to a parent/calling process.
Try these instead:
$ mapfile -d ' ' arr < <(echo -e "hello\nworld")
$ typeset -p arr
declare -a arr=([0]=$'hello\nworld\n')
$ mapfile -d ' ' arr < <(echo "hello world")
$ typeset -p arr
declare -a arr=([0]="hello " [1]=$'world\n')
While these will populate the array ... you'll have to decide if this is what you were expecting to show up in the array. Perhaps the following are a bit closer to what's desired?
$ mapfile -t arr < <(echo -e "hello\nworld")
$ typeset -p arr
declare -a arr=([0]="hello" [1]="world")
$ mapfile -t arr < <(echo "hello world")
$ typeset -p arr
declare -a arr=([0]="hello world")
On the 2nd command set, if the intention is to parse each word into an array then perhaps switch out mapfile with read?
$ read -ra arr < <(echo "hello world")
$ typeset -p arr
declare -a arr=([0]="hello" [1]="world")
This question already has an answer here:
How to store state between two consecutive runs of a bash script
(1 answer)
Closed 1 year ago.
I wrote this two simple functions to backup and restore the content of a bash dictionary:
declare -A dikv
declare -A dict
backup_dikv()
{
FILE=$1
rm -f $FILE
for k in "${!dikv[#]}"
do
echo "$k,${dikv[$k]}" >> $FILE
done
}
restore_dict()
{
FILE=$1
for i in $(cat $FILE)
do
key=$(echo $i | cut -f 1 -d ",")
val=$(echo $i | cut -f 2 -d ",")
dict[$key]=$val
done
}
# Initial values
dikv=( ["k1"]="v1" ["k2"]="v2" ["k3"]="v3" ["k4"]="v4")
backup_dikv /tmp/backup
restore_dict /tmp/backup
echo "${!dict[#]}"
echo "${dict[#]}"
My questions:
As you can see, these two funcions are very limited as the name of the backuped (dikv) and restored (dict) dictionaries is hardcoded. I would like to pass the dictionary as an input ($2) argument, but I don't know how to pass dictionaries as funcion arguments in bash.
Is this method to write keys and values into a file, using a string format ("key","value") and parse that string format to restore the dictionary, the unique / most eficient way to do that? Do you know some better mechanism to backup and restore a dictionary?
Thanks!
Use declare -p to reliably serialize variables regardless of their type
#!/usr/bin/env bash
if [ -f saved_vars.sh ]; then
# Restore saved variables
. saved_vars.sh
else
# No saved variables, so lets populate them
declare -A dikv=([foo]="foo bar from dikv" [bar]="bar baz from dikv")
declare -A dict=([baz]="baz qux from dict" [qux]="qux corge from dict")
fi
# Serialise backup dikv dict into the saved_vars.sh file
declare -p dikv dict >'saved_vars.sh'
printf %s\\n "${!dict[#]}"
printf %s\\n "${dict[#]}"
printf %s\\n "${!dikv[#]}"
printf %s\\n "${dikv[#]}"
Found a way to pass arrays to functions, using local -n in this way:
declare -A dikv
declare -A dict
backup_dictionary()
{
local -n dict_ref=$1
FILE=/tmp/backup
for k in "${!dict_ref[#]}"
do
echo "$k,${dict_ref[$k]}" >> $FILE
done
}
restore_dictionary()
{
local -n dict_ref=$1
FILE=/tmp/backup
for i in $(cat $FILE)
do
key=$(echo $i | cut -f 1 -d ",")
val=$(echo $i | cut -f 2 -d ",")
dict_ref[$key]=$val
done
}
dikv=( ["k1"]="v1" ["k2"]="v2" ["k3"]="v3" ["k4"]="v4")
backup_dictionary dikv
restore_dictionary dict
echo "${!dict[#]}"
echo "${dict[#]}"
Still trying to find the most convenient way to backup and restore the content.
This question already has answers here:
Convert a JSON array to a bash array of strings
(4 answers)
Closed 3 years ago.
I am getting following string and I am trying to convert that into array.
[ "aaa", "bbb" ]
Would someone let me know if there is a way to convert this into BASH array.
You can use jq to extract the individual strings, and then read them line by line:
myJsonArray='[ "aaa", "bbb", "more \"complex\"\u0020value" ]'
mapfile -t myBashArray < <(jq -r '.[]' <<< "$myJsonArray")
declare -p myBashArray
This outputs declare -a myBashArray=([0]="aaa" [1]="bbb" [2]="more \"complex\" value")
If you additionally want to support elements with linefeeds and such, you can have jq output NUL-separated values with a bit more work:
myJsonArray='[ "multi\nline\ndata", "hostile'\''\"\u0000\n$(rm foo)data" ]'
array=()
while IFS= read -r -d '' line
do
array+=("$line")
done < <(jq -j '(.[] | gsub("\u0000"; "")) + "\u0000"' <<< "$myJsonArray")
declare -p array
This outputs declare -a array=([0]=$'multi\nline\ndata' [1]=$'hostile\'"\n$(rm foo)data')
It makes sure NUL bytes in the data don't interfere, but they will be stripped from the output. This is unlikely to matter since Bash variables can't represent NUL bytes in the first place.
This question already has answers here:
How can I get unique values from an array in Bash?
(16 answers)
Closed 4 years ago.
How do I create a newArray containing only the unique elements present in Array?
Ex: ARRAY contains elements aa ab bb aa ab cc at ARRAY[0-5] respectively.
When I print newARRAY, I want only aa ab bb cc at newARRAY[0-3] respectively.
I've searched stack overflow for a while now and nothing is solving my problem. I tried to do newARRAY=$(ARRAY[#] | sort -u | uniq, but duplicated elements still exist.
Naive approach
To get the unique elements of arr and assuming that no element contains newlines:
$ printf "%s\n" "${arr[#]}" | sort -u
aa
ab
bb
cc
Better approach
To get a NUL-separated list that works even if there were newlines:
$ printf "%s\0" "${arr[#]}" | sort -uz
aaabbbcc
(This, of course, looks ugly on a terminal because it doesn't display NULs.)
Putting it all together
To capture the result in newArr:
$ newArr=(); while IFS= read -r -d '' x; do newArr+=("$x"); done < <(printf "%s\0" "${arr[#]}" | sort -uz)
After running the above, we can use declare to verify that newArr is the array that we want:
$ declare -p newArr
declare -a newArr=([0]="aa" [1]="ab" [2]="bb" [3]="cc")
For those who prefer their code spread over multiple lines, the above can be rewritten as:
newArr=()
while IFS= read -r -d '' x
do
newArr+=("$x")
done < <(printf "%s\0" "${arr[#]}" | sort -uz)
Additional comment
Don't use all caps for your variable names. The system and the shell use all caps for their names and you don't want to accidentally overwrite one of them.
You can use an associatve array to keep track of elements you've seen:
#!/bin/bash
ARRAY=(aa ab bb aa ab cc)
unset dupes # ensure it's empty
declare -A dupes
for i in "${ARRAY[#]}"; do
if [[ -z ${dupes[$i]} ]]; then
NEWARRAY+=("$i")
fi
dupes["$i"]=1
done
unset dupes # optional
printf "[%s]" "${ARRAY[#]}"
echo
printf "[%s]" "${NEWARRAY[#]}"
echo
I'm writing this script to count some variables from an input file. I can't figure out why it is not counting the elements in the array (should be 500) but only counts 1.
#initializing variables
timeout=5
headerFile="lab06.output"
dataFile="fortune500.tsv"
dataURL="http://www.tech.mtu.edu/~toarney/sat3310/lab09/"
dataPath="/home/pjvaglic/Documents/labs/lab06/data/"
curlOptions="--silent --fail --connect-timeout $timeout"
#creating the array
declare -a myWebsitearray #=('cut -d '\t' -f3 "dataPath$dataFile"')
#obtaining the data file
wget $dataURL$dataFile -O $dataPath$dataFile
#getting rid of the crap from dos
sed -e "s/^m//" $dataPath$dataFile | readarray -t $myWebsitesarray
readarray -t myWebsitesarray < <(cut -d, -f3 $dataPath$dataFile)
myWebsitesarray=("${#myWebsitesarray[#]:1}")
#printf '%s\n' "${myWebsitesarray2[#]}"
websitesCount=${#myWebsitesarray[*]}
echo $websitesCount
You are overwriting your array with the count of elements in this line
myWebsitesarray=("${#myWebsitesarray[#]:1}")
Remove the hash sign
myWebsitesarray=("${myWebsitesarray[#]:1}")
Also, #chepner suggestions are good to follow.