Here is the bash function to parse a json file use 'jq' command:
jq_fullpath_endkey() {
PATHARRAY=$(jq -c 'paths | select(.[-1] == "'$keyword'")|map(strings |= ".\(.)" | numbers |= "[\(.)]") | join("")' **${news.json}**)
}
The news.json is the json file that contains all the content I'd like to parse with jq.
The function works once I replace ${news.json} with a variable named response which contains news.json content as string.
Below is the command which works:
PATHARRAY=$(jq -c 'paths | select(.[-1] == "'$keyword'")|map(strings |= ".\(.)" | numbers |= "[\(.)]") | join("")'**<<< "$response"**)
My question is how can I use 'json file' as part of the jq cmd ?
I suspect there is something wrong with double/single quote I am using.
I figure it out with "parameter substitution" in bash
myfile="news.json"
PATHARRAY=$(jq -c 'paths | select(.[-1] == "'$keyword'")|map(strings |= ".\(.)" | numbers |= "[\(.)]") | join("")' ${myfile})
Related concepts:
variable substitution
command substitution
Related
cat explorer/connection-profile/test-network.json | jq ".organizations.Org1MSP.adminPrivateKey.path |= 44ab"
jq: error: syntax error, unexpected IDENT, expecting $end (Unix shell quoting issues?) at <top-level>, line 1:
.organizations.Org1MSP.adminPrivateKey.path|=44ab
jq: 1 compile error
but it works fine with
cat explorer/connection-profile/test-network.json | jq ".organizations.Org1MSP.adminPrivateKey.path |= 44"
Why?
Actually I am trying to use
cat explorer/connection-profile/test-network.json | jq ".organizations.Org1MSP.adminPrivateKey.path |= ${PRIV_KEY}"
where the ${PRIV_KEY} is 44ab..._sk
You can assign a string to a variable that can be used inside a jq filter:
PRIV_KEY="44ab..._sk"
jq --arg path "$PRIV_KEY" '.organizations.Org1MSP.adminPrivateKey.path |= $path' explorer/connection-profile/test-network.json
This method is safer than trying to embed an expanded shell variable directly in the filter string because jq will properly handle arbitrary values instead of choking on things like quotes (Or their absence).
Note that jq takes filenames as arguments after the filter expression; no need for cat here (Unless that's standing in for curl or something, of course, and you're not using an existing file)
did you try this , i fixed the same error by setting the expression between '' and the values between ""
cat explorer/connection-profile/test-network.json | jq '.organizations.Org1MSP.adminPrivateKey.path |= "44ab"'
Given the following input:
J='{"a":1,"b":10,"c":100}
{"a":2,"b":20,"c":200}
{"a":3,"b":30,"c":300}'
The command
SELECT='a,b'; echo $J | jq -c -s --arg P1 $SELECT '.[]|{a,b}'
produces
{"a":1,"b":10}
{"a":2,"b":20}
{"a":3,"b":30}
but this command produces unexpected results:
SELECT='a,b'; echo $J | jq -c -s --arg P1 $SELECT '.[]|{$P1}'
{"P1":"a,b"}
{"P1":"a,b"}
{"P1":"a,b"}
How does one get jq to treat an arg string literally?
Using tostring gives an error
SELECT='a,b'; echo $J | jq -c -s --arg P1 $SELECT '.[]|{$P1|tostring}'
jq: error: syntax error, unexpected '|', expecting '}' (Unix shell quoting
issues?) at <top-level>, line 1:
.[]|{$SELECT|tostring}
jq: 1 compile error
SELECT needs to be a variable and not hardcoded in the script.
SELECT needs to be a variable and not hardcoded in the script.
Assuming you want to avoid the risks of "code injection" and that you want the shell variable SELECT to be a simple string such as "a,b", then consider this reduce-free solution along the lines you were attempting:
J='{"a":1,"b":10,"c":100}'
SELECT='a,b'
echo "$J" |
jq -c --arg P1 "$SELECT" '
. as $in | $P1 | split(",") | map( {(.): $in[.]} ) | add'
Output:
{"a":1,"b":10}
If you really want your data to be parsed as syntax...
This is not an appropriate use case for --arg. Instead, substitute into the code:
select='a,b'; jq -c -s '.[]|{'"$select"'}' <<<"$j"
Note that this has all the usual caveats of code injection: If the input is uncontrolled, the output (or other behavior of the script, particularly if jq gains more capable I/O features in the future) should be considered likewise.
If you want to split the literal string into a list of keys...
Here, we take your select_str (of the form a,b), and generate a map: {'a': 'a', 'b': 'b'}; then, we can break each data item into entries, select only the items in the map, and there's our output.
jq --arg select_str "$select" '
($select_str
| split(",")
| reduce .[] as $item ({}; .[$item]=$item)) as $select_map
| with_entries(select($select_map[.key]))' <<<"$j"
I have a program that triggers a script. The script works when I run it manually, but for some reason I get an error when the program runs it. It is supposed to trigger another command if the curl command returns true after parsing some JSON. Here is the relevant portion:
if
$(curl -s -b cookie "https://my.site.com/api" | /config/jq --arg var "test" '.Items[] | .item1[] | select( .Name== $var ) | .Valid') = "true"
then
It returns =: command not found. I tried removing the whitespaces from the = and it returns =true: command not found. I tried putting brackets around it like this:
if
[ $(curl -s -b cookie "https://my.site.com/api" | /config/jq --arg var "test" '.Items[] | .item1[] | select( .Name== $var ) | .Valid') = "true" ]
then
But that gives me an error related to the brackets. I have #!/bin/bash set at the beginning of the script and made sure there are no carriage return issues. What am I doing wrong?
I am executing a shell command in my ROOT code using gSystem which returns an int, as seen here gSystem->Exec(). But when I try to assign the output to a in code variable the assignment doesn't happen.
int low_edge = 0;
low_edge = gSystem->Exec("ls ./folder | egrep -o '[0-9]{3,3}' | head -1");
I have tried also gSystem->Exec("ls ./folder | egrep -o '[0-9]{3,3}' | head -1") >> low_edge, but it didn't work out.
Am I missing something obvious?
The return value of gSystem->Exec() is 0 or -1 depending on if the command was successful.
What you want is:
TString GetFromPipe(const char* command)
TString the_output=gSystem->GetFromPipe("ls ./folder | egrep -o '[0-9]{3,3}' | head -1");
should work, you just need to convert TString to int.
I want to assign the ouput of the following command to a variable in shell:
${arr2[0]} | rev | cut -c 9- | rev
For example:
mod=${arr2[0]} | rev | cut -c 9- | rev
echo $mod
The above method is not working: the output is blank.
I also tried:
mod=( "${arr2[0]}" | rev | cut -c 9- | rev )
But I get the error:
34: syntax error near unexpected token `|'
line 34: ` mod=( "${arr2[0]}" | rev | cut -c 9- | rev ) '
To add an explanation to your correct answer:
You had to combine your variable assignment with a command substitution (var=$(...)) to capture the (stdout) output of your command in a variable.
By contrast, your original command used just var=(...) - no $ before the ( - which is used to create arrays[1], with each token inside ( ... ) becoming its own array element - which was clearly not your intent.
As for why your original command broke:
The tokens inside (...) are subject to the usual shell expansions and therefore the usual quoting requirements.
Thus, in order to use $ and the so-called shell metacharacters (| & ; ( ) < > space tab) as literals in your array elements, you must quote them, e.g., by prepending \.
All these characters - except $, space, and tab - cause a syntax error when left unquoted, which is what happened in your case (you had unquoted | chars.)
[1] In bash, and also in ksh and zsh. The POSIX shell spec. doesn't support arrays at all, so this syntax will always break in POSIX-features-only shells.
mod=$(echo "${arr2[0]}" | rev | cut -c 9- | rev )
echo "****:"$mod
or
mod=`echo "${arr2[0]}" | rev | cut -c 9- | rev`
echo "****:"$mod