Shell split string and convert it to json objects format - bash

I have a string in this format:
string= "[one,two,three,four,five]"
I want to convert it to json format likely, without using any specific tool (jq or other) , I want to do it with pure shell:
[
{"id" : 0 , "word": "one"} ,
{"id" : 1 , "word": "two"} ,
{"id" : 2 , "word": "three"} ,
{"id" : 3 , "word": "four"} ,
{"id" : 4 , "word": "five"}
]
Suggestions ?

There is no legitimate reason not to use tr or jq for this. Portably, you cannot use read -d as below, but this is so absurdly fragile that it's not worth worrying about portability. This works in bash:
$ string="[one,two,three,four,five]"
$ c=0; echo '['; while read -d , i; do
echo "{\"id\" : $((c++)), \"word\": \"${i%]}\"},"
done <<< "${string#[}",; echo ']';
[
{"id" : 0 , "word": "one"},
{"id" : 1 , "word": "two"},
{"id" : 2 , "word": "three"},
{"id" : 3 , "word": "four"},
{"id" : 4 , "word": "five"},
]
At the moment, I can't think of a clean way to eliminate the trailing comma without tr (although I suppose you could use the rather banal if test "$c" -eq 5; then ...., but that's boring) so I'll just propose:
c=0; echo '['; while read -d , i; do
printf "{\"id\" : $((c++)), \"word\": \"${i%]}\"}";
expr $c = 5 | tr 10 ' ,'; done <<< "${string#[}",; echo ']'
I'll reiterate that this is absurdly fragile and only has value as an exercise. Use jq.

This may be still tedious but how about:
string="[one,two,three,four,five]"
out=$(while read -r -d, i; do
printf '{"id" : %d , "word": "%s"},\n' "$((c++))" "${i%]}"
done <<< "${string#[},")
printf "[\n%s\n]\n" "${out%,}"
Output:
[
{"id" : 0 , "word": "one"},
{"id" : 1 , "word": "two"},
{"id" : 2 , "word": "three"},
{"id" : 3 , "word": "four"},
{"id" : 4 , "word": "five"}
]

Related

Concatenate strings in Bash to call an array in a For loop [duplicate]

This question already has answers here:
How to iterate over an array using indirect reference?
(7 answers)
Closed 11 days ago.
I'm looking to nest a couple For loops in bash to first check one array and then based on that array, check a second array.
#!/bin/sh
domArr=( "ABC" "DEF" "GHI" )
ABCarr=( "1" "2" "3" )
DEFarr=( "4" "5" "6" )
GHIarr=( "7" "8" "9" )
for domain in "${domArr[#]}"
do
# This should be 'domain = "ABC"'
for group in "${domain+arr[#]}"
do
# This should be 'group = "1"'
done
done
You may use it like this:
domArr=( "ABC" "DEF" "GHI" )
ABCarr=( "1" "2" "3" )
DEFarr=( "4" "5" "6" )
GHIarr=( "7" "8" "9" )
for domain in "${domArr[#]}"
do
echo "iterating array ${domain}arr ..."
a="${domain}arr[#]"
for group in "${!a}"
do
echo "$group"
done
done
Output:
iterating array ABCarr ...
1
2
3
iterating array DEFarr ...
4
5
6
iterating array GHIarr ...
7
8
9
Simply:
#!/bin/bash
dom=( "ABC" "DEF" "GHI" )
ABC=( "1" "2" "3" )
DEF=( "4" "5" "6" )
GHI=( "7" "8" "9" )
for ((i=0; i<${#dom[#]}; i++)); do
# This should be 'domain = "ABC"' + "1"
echo "${dom[i]} ${ABC[i]}"
done
Output
ABC 1
DEF 2
GHI 3

Bash number comparision error: line 90: [[: 22: syntax error: operand expected (error token is "22")

Let's assume $commandGetEvents is an array of json objects. I use the following command to extract the event Id, which is a number from 1 - 65 and store it in currentEventId. Now let's assume I have another variable called startedEventId which holds the value I'm looking for, which is 22.
Here's an example of the data $commandGetEvents contains.
[
{
"eventId": 22,
"Name" : "Bob"
"Activity" : "Eat Food"
"startedEventId" : 15
},
{
"eventId": 21,
"Name" : "Smith"
"Activity" : "Ride a bike"
"startedEventId" : 13
},
{
"eventId": 20,
"Name" : "Tony"
"Activity": "Print paper"
"startedEventId" : 10
},
]
eventId is the unique identifier of the json object. & startedEventId is the identifier of json object that caused the current one to take place.
currentEventId=$(jq ".[$index].eventId" <<< ${commandGetEvents})
startedEventid=$(jq ".[${eventCounter}].startedEventId" <<< $commandGetEvents)
When i echo both statements in a while loop, I get the following output.
currentEventId = 1
startedEventId = 22
currentEventId = 2
startedEventId = 22
currentEventId = 3
startedEventId = 22
The while loop continues until all elements of currentEventId are exhausted.
My problem is when I compare both statements like this:
if [[ ${startedId} -eq ${currentEventId} ]] ;
then
echo "Equal"
fi
I get the following error message:
line 90: [[: 22: syntax error: operand expected (error token is "22")
The provided "JSON" is invalid as JSON. Please fix it.
When using jq at the bash command line, it's almost always best to enclose the jq program in single-quotation marks; bash shell variables can be passed in using --arg or --argjson. Consider for example the following snippet that assume `commandGetEvents' is valid JSON along the lines suggested in the Q:
index=0
currentEventId=$(jq --argjson index $index '.[$index].eventId' <<< ${commandGetEvents})
echo index=$index
echo currentEventId=$currentEventId
The part of the question involving eventCounter is somewhat obscure, but it looks like the example given immediately above will serve as a guide.
Rather than using bash constructs to iterate through the JSON array, it would almost certainly be better to use jq's support for iteration and selection. For example:
jq '.[] | select(.eventId == 22)' <<< ${commandGetEvents}
yields:
{
"eventId": 22,
"Name": "Bob",
"Activity": "Eat Food",
"startedEventId": 15
}
So if you just want the startedEventId value (or values) corresponding to .eventId == 22, you could write:
jq '.[] | select(.eventId == 22) | .startedEventId' <<< ${commandGetEvents}

print elements from multiple lists by index with jq

I have a json with 3 equal lists:
{
"value": {
"list1": [
"el11",
"el12",
"el13",
"el14"
],
"list2": [
"el21",
"el22",
"el23",
"el24"
],
"list3": [
"el31",
"el32",
"el33",
"el34"
]
}
}
I'm trying to extract from each list the elements with the same index:
el11 el21 el31
...
el13 el23 el33
Ideally those need to be exported as ENV values. But I just want to know if it's possible to do that with jq. And how.
Thank you
You can use the transpose function for this. For example,
$ jq -r '[.value[]] | transpose[] | join (" ")' tmp.json
el11 el21 el31
el12 el22 el32
el13 el23 el33
el14 el24 el34

Output the results of select operation in an array - jq

I have the following input:
[
{"id": "first", "val": 1},
{"id": "second", "val": 2},
{"id": "second", "val": 3}
]
Using the jq filter : .[] | select(.id == "second")
I get following as output :
{
"id": "second",
"val": 2
}
{
"id": "second",
"val": 3
}
I want to get the result in the form of an array. Is it possible to get the multiple result values of select operation in an array?
Yes; wrap the filter in an array :)
$ jq '[.[] | select(.id == "second")]' tmp.json
[
{
"id": "second",
"val": 2
},
{
"id": "second",
"val": 3
}
]
Or, use map/1, which is predefined as [.[] | ...].
$ jq 'map(select(.id == "second"))' tmp.json
[same result]
To wrap the results in a bash array, use the -c option to output each result on a single line, and read the result with readarray.
$ readarray -t arr < <(jq -c '.[] | select(.id == "second")' tmp.json)
$ for r in "${arr[#]}"; do echo "Result: $r"; done
Result: {"id":"second","val":2}
Result: {"id":"second","val":3}

Elasticsearch : Number of search operation per second

I am looking for a way to get the number of search operation per a second on a node (and / or on all nodes).
Is there a way to get this information without the Marvel plugin?
My ElasticSearch version is 0.90.11
Marvel does it by sampling. If you write a script to repeatedly run curl http://localhost:9200/_stats/search and parse the a result that looks like this:
...
"_all" : {
"primaries" : {
"search" : {
"open_contexts" : 0,
"query_total" : 51556,
"query_time_in_millis" : 2339958,
"query_current" : 0,
"fetch_total" : 8276,
"fetch_time_in_millis" : 34916,
"fetch_current" : 0
}
},
"total" : {
"search" : {
"open_contexts" : 0,
"query_total" : 73703,
"query_time_in_millis" : 2773745,
"query_current" : 0,
"fetch_total" : 10428,
"fetch_time_in_millis" : 45570,
"fetch_current" : 0
}
}
},
...
You can see the query_total values -- just repeatedly query those at some interval and then do the math.
Thanks Alcanzar.
Here is the script I created :
if [[ $1 == "" ]]
then
echo "Usage : $0 <refresh_interval_in second>"
echo "Example : ./$0 10"
exit 1
fi
refresh_interval=$1
while true; do
begin=$(curl --silent http://localhost:9200/_stats/search?pretty | grep '"query_total" :' | sed -n 2p | sed 's/,$//' | awk '{print $3}')
sleep $refresh_interval
end=$(curl --silent http://localhost:9200/_stats/search?pretty | grep '"query_total" :' | sed -n 2p | sed 's/,$//' | awk '{print $3}')
total=$(echo $(((( $end - $begin )) / $refresh_interval)))
echo $(date +"[%T] ")"Search ops/sec : "$total
done
For instance, refreshing every 10 seconds, execute : sh script.sh 10

Resources