Get json field value with JQ from different directory - bash

Question

Title may be incorrect as I'm not actually sure where this is failing. I have a bash script running in one directory, and a JSON file I need a value from in a different directory. I want to copy the value from the external directory into an identical JSON file in the current directory.
I'm using jq to grab the value, but I can't figure out how to grab from a directory other than the one the script is running in.
The relevant bits of file structure are as follows;
cloudformation
- parameters_v13.json
environment_files
- prepare_stack_files.json (the script this is run from)
- directory, changes based on where the script is pointed
- created directory where created files are being output
- GREPNAME_parameters.json
The chunk of the JSON file I'm interested in looks like this;
[
{
"ParameterKey": "RTSMEMAIL",
"ParameterValue": "secretemail"
}
]
The script needs to get the "secretemail" from cloudformation/parameters_v13.json and paste it into the matching RTSMEMAIL field in the GREPNAME_parameters.json file.
I've been attempting the following with no luck - nothing is output. No error message either, just blank output. I know the GREPNAME path is correct because it's used elsewhere with no issues.
jq --arg email "$EMAIL" '(.[] | select(.ParameterKey == "RTSMEMAIL") | .ParameterValue) |= $email' ../cloudformation/parameters_v13.json | sponge ${GREPNAME}_parameters.json

Answer 1

This jq filter should help you get secretmail string
jq '.[] | select(.ParameterKey=="RTSMEMAIL") | .ParameterValue' json
"secretemail"
Add a -r file for raw output to remove quotes around the value
jq -r '.[] | select(.ParameterKey=="RTSMEMAIL") | .ParameterValue' json
secretemail
--raw-output / -r:
With this option, if the filter’s result is a string then it will be written directly to standard output rather than being formatted as a JSON string with quotes. This can be useful for making jq filters talk to non-JSON-based systems.
As I could see it you are trying to pass args to jq filter, for extraction you can do something first by setting the variable in bash
email="RTSMEMAIL"
and now pass it to the filter as
jq --arg email "$email" -r '.[] | select(.ParameterKey==$email) | .ParameterValue' json
secretemail
Now to replace the string obtained from parameters_v13.json file to your GREPNAME_parameters.json do the following steps:-
First storing the result from the first file in a variable to re-use later, I have used the file to extract as json, this actually points your parameters_v13.json file in another path.
replacementValue=$(jq --arg email "$email" -r '.[] | select(.ParameterKey==$email) | .ParameterValue' json)
now the $replacementValue will hold the secretmail which you want to update to another file. As you have indicated previously GREPNAME_parameters.json has a similar syntax as of the first file. Something like below,
$ cat GREPNAME_parameters.json
[
{
"ParameterKey": "SOMEJUNK",
"ParameterValue": "somejunkvalue"
}
]
Now I understand your intention is replace "ParameterValue" from the above file to the value obtained from the other file. To achieve that,
jq --arg replace "$replacementValue" '.[] | .ParameterValue = $replace' GREPNAME_parameters.json
{
"ParameterKey": "SOMEJUNK",
"ParameterValue": "secretemail"
}
You can then write this output to the a temp file and move it back as the GREPNAME_parameters.json. Hope this answers your question.

Answer 2

#Alex -
(1) sponge simply provides a convenient way to modify a file without having to manage a temporary file. You could use it like this:
jq ........ input.json | sponge input.json
Here, "input.json" is the file that you want to edit "in place". If you want to avoid overwriting the input file, you would not use sponge. In fact, I would recommend against doing so until you're absolutely sure that's what you want.
(2) There are several strategies for achieving what you have described using jq. They basically fall into two categories: (a) invoke jq twice; (b) invoke jq once.
Ignoring the sponge part:
the pattern for using jq twice would be as follows:
param=$(jq -r '.[]
| select(.ParameterKey == "RTSMEMAIL")|.ParameterValue
' cloudformation/parameters_v13.json )
jq --arg param "$param" -f edit.jq input.json
assuming you have jq 1.5, the pattern for doing everything with just one invocation of jq would be:
jq --argfile p cloudformation/parameters_v13.json -f manage.jq input.json
Here, edit.jq and manage.jq are files containing suitable jq programs.
Based on my understanding of your requirements, edit.jq might look like this:
(.[] | select(.ParameterKey == "RTSMEMAIL")|.ParameterValue) |= $param
And manage.jq might look like this:
($p[] | select(.ParameterKey == "RTSMEMAIL")|.ParameterValue) as $param
| (.[]| select(.ParameterKey == "RTSMEMAIL")|.ParameterValue) |= $param