I have a json file like this:
{
"name": "Job",
"type": "xdb",
"typeLogoUrl": "public/app/plugins/logo.svg",
"access": "proxy",
"url": "http://xdb:80",
"password": {},
"user": "xx",
"database": "Job",
"basicAuth": true,
"basicAuthUser": "xx",
"basicAuthPassword": {},
"withCredentials": true,
"isDefault": false,
"jsonData": {},
"secureJsonFields": null
}
Now all I want is to pass environment variables to key password and basicAuthPassword and to generate new json file as below:
{
"name": "Job",
"type": "xdb",
"typeLogoUrl": "public/app/plugins/logo.svg",
"access": "proxy",
"url": "http://xdb:80",
"password": "password" ,
"user": "xx",
"database": "Job",
"basicAuth": true,
"basicAuthUser": "xx",
"basicAuthPassword": "password",
"withCredentials": true,
"isDefault": false,
"jsonData": {},
"secureJsonFields": null
}
I have tried like below to replace at least one key and it's giving me null.
/usr/local/bin/jq -n --arg "password" '.password = $arg' < input.json
Can anyone suggest me how to achieve this?
Security wise, it's a bad idea to pass password in command line, but I hope you know what you're doing.
Assuming that, you need to modify your jq command like this:
jq --arg p "password" '.password = $p | .basicAuthPassword = $p' < input.json
If the password is already an environment variable (which may not be such a good idea though), then to avoid exposing it on the command-line, you could use the env builtin, along the lines of:
.password = env.password
etc.
An alternative to consider would be putting the password into a temporary file, and then using the --argfile command-line option, for example. Or if your shell supports it, --argfile pw <(echo "\"$password\"")
Related
I am trying to read a JSON file that contains an array of objects located in laravel storage and trying to iterate it. My attempt is as below.
$json = Storage::disk('local')->get('users.json');
json_decode($json, true);
This will return string type for the $json. So I cannot iterate. Anyone knows where I got wrong and how can I fix it?
Below is the part of my $json output.
[
{
"_id": 1,
"url": "http://initech.tokoin.io.com/api/v2/users/1.json",
"external_id": "74341f74-9c79-49d5-9611-87ef9b6eb75f",
"name": "Francisca Rasmussen",
"alias": "Miss Coffey",
"created_at": "2016-04-15T05:19:46 -10:00",
"active": true,
"verified": true,
"shared": false,
"locale": "en-AU",
"timezone": "Sri Lanka",
"last_login_at": "2013-08-04T01:03:27 -10:00",
"email": "coffeyrasmussen#flotonic.com",
"phone": "8335-422-718",
"signature": "Don't Worry Be Happy!",
"organization_id": 119,
"tags": [
"Springville",
"Sutton",
"Hartsville/Hartley",
"Diaperville"
],
"suspended": true,
"role": "admin"
},
{...}
]
Edit: json_decode_error function returns 5 which is unsupported utf8 characters included in the file is the issue.
Handle json_decode error as follow using json_last_error.
if ($json === null && json_last_error() !== JSON_ERROR_NONE) {
echo "Incorrect json data.";
}
If you've unsupported UTF-8 characters in your encoded json then convert them before decoding.
$input = Storage::disk('local')->get('users.json');
$json = iconv('UTF-8', 'UTF-8//IGNORE', utf8_encode($input));
$arr = json_decode($json, true);
If possible I suggest you to do the same while doing json_encode.
I am new to bash and jq. I want an array of all value of a key.
i have tried something like this but it doesn't work
jobName=$(echo "$jobResult" | jq '. | to_entries[] | .value.name')
but it gave a string to me like this string
"Basement Parking Construction Site 1" "Basement Parking Construction Site 2"
Input
{
"1": {
"_status_code": 202,
"_status_message": "Accepted",
"id": 4120489,
"parent_id": 0,
"assigned_to_all": false,
"billable": false,
"active": true,
"type": "regular",
"has_children": false,
"billable_rate": 0,
"short_code": "",
"name": "Basement Parking Construction Site 1",
"last_modified": "2019-03-12T10:02:54+00:00",
"created": "2019-03-12T10:02:54+00:00",
"filtered_customfielditems": "",
"required_customfields": [],
"locations": [],
"project_id": 0
},
"2": {
"_status_code": 202,
"_status_message": "Accepted",
"id": 4120491,
"parent_id": 0,
"assigned_to_all": false,
"billable": false,
"active": true,
"type": "regular",
"has_children": false,
"billable_rate": 0,
"short_code": "",
"name": "Basement Parking Construction Site 2",
"last_modified": "2019-03-12T10:02:54+00:00",
"created": "I019-03-12T10:02:54+00:00",
"filtered_customfielditems": "",
"required_customfields": [],
"locations": [],
"project_id": 0
}
I want an array of all the value with key name
result
["Basement Parking Construction Site 1","Basement Parking Construction Site 2"]
One of many possible solutions:
[.[] | .name]
If you want to ensure the values in the resultant array are distinct, you could add the unique filter.
Creating a bash array
If your bash has readarray, you can read the "name" values into a bash array, a, as follows:
readarray a <<< $(jq '.[] | .name' so-to-bash-array.json)
If not, simply use a bash loop.
Depending on your requirements, you might want to use the -r command-line option of jq.
This question already has answers here:
How do I use sed to change my configuration files, with flexible keys and values?
(8 answers)
Closed 4 years ago.
I have total 3 environment dev, stag and Prod all environment have config.json like this
{
"braintree": {
"merchantid": "MERCHANTID",
"publickey": "PUBLICKEY",
"privatekey": "PRIVATEKEY"
},
"karix": {
"url": "URL",
"pass": "PASS",
"user": "USER"
},
"fikarix": {
"source": "SOURCE"
},
"mailgun": {
"api_key": "API_KEY",
"domain": "DOMAIN",
"apikey": "APIKEY"
},
"paymentrails": {
"key": "KEY",
"environment": "ENVIRONMENT",
"secret": "SECRET"
}
}
Now I want to convert it into like this for all environment using shell script
dev environment config.json
{
"braintree": {
"merchantid": "dev_MERCHANTID",
"publickey": "dev_PUBLICKEY",
"privatekey": "dev_PRIVATEKEY"
},
"karix": {
"url": "dev_URL",
"pass": "dev_PASS",
"user": "dev_USER"
},
"fikarix": {
"source": "dev_SOURCE"
},
"mailgun": {
"api_key": "dev_API_KEY",
"domain": "dev_DOMAIN",
"apikey": "dev_APIKEY"
},
"paymentrails": {
"key": "dev_KEY",
"environment": "dev_ENVIRONMENT",
"secret": "dev_SECRET"
}
}
How I can get this using sed or any other solution?
sed 's/: "/: "dev_/g' config.json
using sed u can do
Edit:
To insert pass -i
sed -i 's/: "/: "dev_/g' config.json
I'm wondering if possible to use an argument to construct a field name in jq.
Example:
jq -rc \
--arg secret_name ${secret_name} \
--arg secret_value ${secret_value} \
'.data.$secret_name = "$secret_value"'
In above example, I want to use value of argument secret_name to create a key under .data. Is this possible using jq?
Example Data:
secret_name=abc
secret_value=xyz
JSON on which jq is run:
{
"apiVersion": "v1",
"data": {},
"kind": "Secret",
"metadata": {
"name": "kv-secrets",
"namespace": "default"
},
"type": "Opaque"
}
Expected output:
{
"apiVersion": "v1",
"data": {
"abc": "xyz"
},
"kind": "Secret",
"metadata": {
"name": "secrets"
},
"type": "Opaque"
}
Do mind that I intend to run the original command to fill .data will more key-value pairs.
With a variable, you need to use the long [...] form for the key. You don't need to quote the variables in a JSON filter; the variable is the string value.
jq -rc \
--arg secret_name "${secret_name}" \
--arg secret_value "${secret_value}" \
'.data[$secret_name] = $secret_value'
There's a file, code looks like that:
{
"name": {
"port": 4466,
"host": "localhost",
"appPort": 3555,
"abc": {
"defg": {
"host": "localhost",
"port": 5500,
"userName": "test",
"password": "test"
}
},
"name2": {
"port": 4321,
"host": "localhost",
"appPort": 1234,
"abc": {
"defg": {
"host": "localhost",
"port": 5500,
"userName": "test",
"password": "test"
}
},
etc.
I need to replace the following block of code in a file using sed (or maybe awk):
"name": {
"port": 4466,
"host": "localhost",
"appPort": 3555
Here's what I have, did not succeed:
confSearch='"name": {\n "port": 4466,\n "host": "localhost",\n "appPort": 3555'
confReplacement='"name": {\n "port": 4466,\n "host": "10.20.30.40",\n "appPort": 3555'
sed -i "s|$confSearch|$confReplacement|g" "$configFile"
Then I tried the following (to search from "name" to "3555" and then replace it):
sed -i "/name.*/ {N; s/name.*3555\./$confReplacement/g}" "$configFile"
I don't receive any error, the search text is simply not found. I suppose, that's because of indents. How to treat indents correctly? Or should I prefer something else? Taking into account only bash, not perl.
Would be grateful for the help.
I'd advise against using sed for this task; JSON is not a line-based format, and sed is ill-equipped to handle it unless you are very certain that the file will always be formatted just the right way. There are tools out there that parse JSON properly and work on the object it encodes, so indentation or swapped lines or having several nodes on the same line does not faze them.
In this particular case, I'd suggest jq:
jq 'if .name.port == 4466 and .name.appPort == 3555 and .name.host == "localhost" then .name.host="10.20.30.40" else . end' "$configFile"
sed -i '/"port": 4466,/,/"host": "localhost",/ s/localhost/110.20.30.40/' "$configFile"
assuming there is only 1 host with port 4466
it seems you just want to replace the host, won't you? use something like
sed -i 's/"host":.*$/"host": "10.20.30.40"\n/'.