Hi I have json data which I Unmarshal to machines slice . Now I am looking to copy/append each cluster information, hostname from machines slice struct to Cluster Struct []Cluster and try to populate different values in that struct.
Each machine record has an associated serviceName . I am Looking for the out json format in the desired output below where service_name ,required, vars, value are only passed once even though they are associated with each json record when passed from machine slice.
current Code :
https://go.dev/play/p/6zVRIaLIgdN
Desired output :
{
"cluster_name": "dev",
"services": [
{
"service_name": "serviceA",
"required" : true,
"vars": {
"common_vars": {
"user": "custom-user",
"group": "custom-group"
}
},
"hosts": [
{
"host_name": "host1",
"vars": {
"common_vars" :{
"id": 1
}
}
},
{
"host_name": "host2",
"vars": {
"common_vars":{
"id": 2
}
}
}
]
},
{
"service_name": "serviceB",
"required" : false
.....
}
}
]
}
Current OutPut:
where the ServiceName is repeated with every machine name , I want it to have service_name once in the slice as above output
"cluster_name": "dev",
"services": [
{
"service_name": "serviceA",
"required": true,
"hosts": [
{
"host_name": "Machine-1",
"vars": {
"common_vars": {
"id": "1"
},
"custom_listeners": {}
}
}
],
"vars": {
"custom_listeners": {}
}
},
{
**"service_name": "serviceA"**,
"required": true,
"hosts": [
{
"host_name": "Machine-2",
"vars": {
"common_vars": {
"id": "2"
},
"custom_listeners": {}
}
}
],
"vars": {
"custom_listeners": {}
}
}
]
}
You have to implement some logic for merging service records with same name.
map[<ServiceName>]<Service> could be used to avoid iterating through slice of services every time.
m := map[string]*Service{}
for i := range machines {
s, found := m[machines[i].Servicename]
if !found {
s = &Service{ServiceName: machines[i].Servicename, Required: true}
m[machines[i].Servicename] = s
}
s.Hosts = append(s.Hosts, Host{HostName: machines[i].Hostname, Vars: Var{CommonVars: map[string]interface{}{"id": machines[i].ID}}})
}
for _, s := range m {
cService.Services = append(cService.Services, *s)
}
Related
Hi pls help with this jolt transformation.
Note:
If Environment exists then add new field numid and value is 1
If costcenter exists then add new field numid and the value is 2
Input:
[
{
"Environment": "net",
"BillingProfileId": 863,
"ms-resource-usage": "azure-cloud-shell"
},
{
"CostCenter": "check",
"BillingAccountName": "HCL Technologies Ltd.",
"ServiceFamily": "Compute"
}
]
Expected output:
[
{
"Environment": "net",
"numid": "1"
"BillingProfileId": 863,
"ms-resource-usage": "azure-cloud-shell"
},
{
"CostCenter": "check",
"numid" : "2",
"BillingAccountName": "HCL Technologies Ltd.",
"ServiceFamily": "Compute"
}
]
Thanks
You can use this spec:
[
{
"operation": "shift",
"spec": {
"*": {
"#": "[&1]",
"Environment": {
"#1": "[&2].numid"
},
"CostCenter": {
"#2": "[&2].numid"
}
}
}
}
]
Using jq I am trying to convert the rawe json below into the desired json outcome.
Objectives:
name renamed to pathParameterName
type renamed to datasetParameter
Raw Json I'm trying to convert
{
"pathOptions": {
"parameters": {
"raw_date": {
"name": "raw_date",
"type": "Datetime",
"datetimeOptions": {
"localeCode": "en-GB"
},
"createColumn": true,
"filter": {
"expression": "(after :date1)",
"valuesMap": {
":date1": "2022-03-08T00:00:00.000Z"
}
}
}
}
}
}
Json desired outcome:
{
"pathOptions": {
"parameters": [
{
"pathParameterName": "raw_date",
"datasetParameter": {
"name": "raw_date",
"type": "Datetime",
"datetimeOptions": {
"localeCode": "en-GB"
},
"createColumn": true,
"filter": {
"expression": "(after :date1)",
"valuesMap": [
{
"valueReference": ":date1",
"value": "2022-03-08T00:00:00.000Z"
}
]
}
}
}
]
}
}
This is what I have so far:
map_values(if type == "object" then to_entries else . end)
This is what my code above currently produces. -I'm struggling with the key renaming.
{
"pathOptions": [
{
"key": "parameters",
"value": [
{
"pathParameterName": "raw_date",
"datasetParameter": {
"name": "raw_date",
"type": "Datetime",
"datetimeOptions": {
"localeCode": "en-GB"
},
"createColumn": true,
"filter": {
"expression": "(after :date1)",
"valuesMap": [
{
"valueReference": ":date1",
"value": "2022-03-08T00:00:00.000Z"
}
]
}
}
}
]
}
]
}
The function to_entries, "converts between an object and an array of key-value pairs" (see the manual). To rename the preset key and value fields, just reassign them to a new name with a new object as in {valueReference: .key, value}.
jq '
.pathOptions.parameters |= (
to_entries | map({
pathParameterName: .key,
datasetParameter: (
.value | .filter.valuesMap |= (
to_entries | map({valueReference: .key, value})
)
)
})
)
'
{
"pathOptions": {
"parameters": [
{
"pathParameterName": "raw_date",
"datasetParameter": {
"name": "raw_date",
"type": "Datetime",
"datetimeOptions": {
"localeCode": "en-GB"
},
"createColumn": true,
"filter": {
"expression": "(after :date1)",
"valuesMap": [
{
"valueReference": ":date1",
"value": "2022-03-08T00:00:00.000Z"
}
]
}
}
}
]
}
}
Demo
I cannot mutate a list of objects completely, because only the last element of the array will be mutated.
What already works perfectly is, if I put each element ({play_positions_id: ...}) in the array manually like here:
mutation CreateProfile {
__typename
create_profiles_item(data: {status: "draft", play_positions: [{play_positions_id: {id: "1"}}, {play_positions_id: {id: "2"}}]}) {
id
status
play_positions {
play_positions_id {
abbreviation
name
}
}
}
}
Output:
{
"data": {
"__typename": "Mutation",
"create_profiles_item": {
"id": "1337",
"status": "draft",
"play_positions": [
{
"play_positions_id": {
"id": "1",
"abbreviation": "RWB",
"name": "Right Wingback"
}
},
{
"play_positions_id": {
"id": "2",
"abbreviation": "CAM",
"name": "Central Attacking Midfielder"
}
}
],
}
}
}
Since you can add many of those elements, I defined a variable/argument like here
mutation CreateProfile2($cpppi: [create_profiles_play_positions_input]) {
__typename
create_profiles_item(data: {status: "draft", play_positions: $cpppi}) {
id
status
play_positions {
play_positions_id {
id
abbreviation
name
}
}
}
}
Variable object for above:
"cpppi": {
"play_positions_id": {
"id": "1"
},
"play_positions_id": {
"id": "2
}
}
Output:
{
"data": {
"__typename": "Mutation",
"create_profiles_item": {
"id": "1338",
"play_positions": [
{
"play_positions_id": {
"id": "2",
"abbreviation": "CAM",
"name": "Central Attacking Midfielder"
}
}
],
}
}
}
Schema:
input create_profiles_input {
id: ID
status: String!
play_positions: [create_profiles_play_positions_input]
}
input create_profiles_play_positions_input {
id: ID
play_positions_id: create_play_positions_input
}
input create_play_positions_input {
id: ID
abbreviation: String
name: String
}
At the last both snippets, only the last object with the id "2" will be mutated. I need these to use the defined input type from my backend.
I figured it out. I got it wrong with the brackets in the variable. Here the solution:
"cpppi": [
{
"play_positions_id": {
"id": "1"
}
},
{
"play_positions_id": {
"id": "2"
}
}
]
How query only those users whose Itemcount > 10 from the complex nested object(with dynamic key) from comosdb using sql api? UDF not preferred.
Something like,
Select c.username from c where c.Data[*].Order.ItemCount > 10;
{
{
"Username": "User1",
"Data": {
"RandomGUID123": {
"Order": {
"Item": "ItemName123",
"ItemCount" : "40"
},
"ShipmentNumber": "7657575"
},
"RandomGUID976": {
"Order": {
"Item": "ItemName7686"
"ItemCount" : "7"
},
"ShipmentNumber": "876876"
}
}
},
{
"Username": "User2",
"Data": {
"RandomGUID654": {
"Order": {
"Item": "ItemName654",
"ItemCount" : "9"
},
"ShipmentNumber": "7612575"
},
"RandomGUID908": {
"Order": {
"Item": "ItemName545"
"ItemCount" : "6"
},
"ShipmentNumber": "6454"
}
}
}
}
I'm not sure about how to handle unknown keys, but if you're willing to model the key as a value instead (simpler and cleaner I'd argue), you could have:
{
"Username": "User1",
"Data": [
{
"Id": "RandomGUID123",
"Order": {
"Item": "ItemName123",
"ItemCount": 40
},
"ShipmentNumber": "7657575"
},
{
"Id": "RandomGUID976",
"Order": {
"Item": "ItemName7686",
"ItemCount": 7
},
"ShipmentNumber": "876876"
}
]
}
With a query like:
SELECT DISTINCT VALUE(c.Username)
FROM c
JOIN (SELECT VALUE d from d IN c.Data where d["Order"].ItemCount > 10)
Result:
[
"User1"
]
"Order" is a reserved keyword and requires the bracket syntax to reference.
As Noah answers,model the key as a value is a way to achieve.
Additionally,there is another way to achieve without changing your schema of your document .Create UDF like this:
function getResult(data){
for(var key in data){
const itemCount = data[key].Order.ItemCount;
if (parseFloat(itemCount).toString() != "NaN" && parseFloat(itemCount) > 10 ) {
return true;
}
}
return false;
}
Then run this sql:
SELECT c.Username FROM c where udf.getResult(c.Data)
Result:
[
{
"Username": "User1"
}
]
I am trying to learn JSON parsing using jmesquery in Ansible.
Please consider the following play:
---
- name: GET ALL THE INTERFACES
junos_command:
commands: show configuration interfaces | display json
register: A
- name: DISPLAY VARIABLE A CONTENTS
debug:
var: A.stdout_lines
- name: JOSN QUERY TO STORE PORTS IN NEW VARIABLE ALL_PORTS
set_fact:
ALL_PORTS: "{{ A.stdout_lines | json_query(jmesquery) }}"
vars:
jmesquery: 'configuration.interfaces.interface[*].name'
- name: DISPLAY VARIABLE ALL_PORTS CONTENTS
debug:
var: ALL_PORTS
#
Based on the JSON query, the port ge-0/0/0 will be stored in ALL_PORT, but we do not see that when we run the playbook, debug on ALL_PORT shows it is empty.
PLAY [ROUTER-STIG-PLAYBOOK] ************************************************************************************************
TASK [STIG_ROUTER : GET ALL THE INTERFACES] ********************************************************************************
ok: [192.168.22.9]
TASK [STIG_ROUTER : DISPLAY VARIABLE A CONTENTS] ***************************************************************************
ok: [192.168.22.9] => {
"A.stdout_lines": [
{
"configuration": {
"#": {
"junos:changed-localtime": "2020-05-10 11:14:49 UTC",
"junos:changed-seconds": "1589109289",
"xmlns": "http://xml.juniper.net/xnm/1.1/xnm"
},
"interfaces": {
"interface": [
{
"name": "ge-0/0/0",
"unit": [
{
"family": {
"inet": {
"address": [
{
"name": "192.168.22.9/24"
}
]
}
},
"name": 0
}
]
}
]
},
"security": {
"policies": {
"global": {
"policy": [
{
"match": {
"application": [
"any"
],
"destination-address": [
"any"
],
"source-address": [
"any"
]
},
"name": "TEST",
"then": {
"permit": [
null
]
}
}
]
}
},
"screen": {
"ids-option": [
{
"icmp": {
"ping-death": [
null
]
},
"ip": {
"source-route-option": [
null
],
"tear-drop": [
null
]
},
"name": "untrust-screen",
"tcp": {
"land": [
null
],
"syn-flood": {
"alarm-threshold": 1024,
"attack-threshold": 200,
"destination-threshold": 2048,
"queue-size": 2000,
"source-threshold": 1024,
"timeout": 20
}
}
}
]
},
"zones": {
"security-zone": [
{
"name": "trust",
"tcp-rst": [
null
]
},
{
"name": "untrust",
"screen": "untrust-screen"
},
{
"host-inbound-traffic": {
"protocols": [
{
"name": "all"
}
],
"system-services": [
{
"name": "all"
}
]
},
"interfaces": [
{
"name": "ge-0/0/0.0"
}
],
"name": "A"
}
]
}
},
"system": {
"license": {
"autoupdate": {
"url": [
{
"name": "https://ae1.juniper.net/junos/key_retrieval"
}
]
}
},
"root-authentication": {
"encrypted-password": "$6$thcHCjAV$e3o5ZRNWv7WtysOxuKpBP2X0cA3QDNtWYyCSBAUkImSEsulEGTgfwEQBa12Wll0fegpwvZfTHLvCbDUIW1n211"
},
"services": {
"ftp": [
null
],
"netconf": {
"ssh": [
null
]
},
"ssh": {
"root-login": "allow"
}
},
"syslog": {
"file": [
{
"contents": [
{
"any": [
null
],
"name": "any"
},
{
"info": [
null
],
"name": "authorization"
}
],
"name": "messages"
},
{
"contents": [
{
"any": [
null
],
"name": "interactive-commands"
}
],
"name": "interactive-commands"
}
],
"user": [
{
"contents": [
{
"emergency": [
null
],
"name": "any"
}
],
"name": "*"
}
]
}
},
"version": "20191026.124700_builder.r1063854"
}
}
]
}
TASK [STIG_ROUTER : JOSN QUERY TO STORE PORTS IN NEW VARIABLE ALL_PORTS] ***************************************************
ok: [192.168.22.9]
TASK [STIG_ROUTER : DISPLAY VARIABLE ALL_PORTS CONTENTS] *******************************************************************
ok: [192.168.22.9] => {
"ALL_PORTS": ""
}
#
I used online JSON query tester.
https://jsonpath.com/
When I checked my my query "configuration.interfaces.interface[*].name" against A.stdout_lines, I found the expected response i.e
[
ge-0/0/0
]
Any feedback/guidance is appreciated!!
Have a good weekend!!
Your JMESPath may very well be correct for one object but as its name implies, and as your debug: var=A.stdout_lines shows, stdout_lines is a list
So, you can do one of several things:
recognize that your stdout_lines only contains one object and just feed that into json_query as {{ A.stdout_lines[0] | json_query(jmesquery) }}
use the map filter to apply that same filter to every list item, with something like {{ A.stdout_lines | map("json_query", jmesquery) | list }}
rewrite your JMESPath to apply that filter to the input list, akin to json_query("[*].configuration.AND THE REST HERE")
Those last two will naturally produce a different output shape, since they are lists of lists, and so it will look like [['ge-0/0/0']] when output, but they do have the advantage that if your stdout_lines ever does mysteriously start to contain more objects, they will be json_query-ied as expected