I am trying to give as input a JSON that contains a fixed variable (in my example group) that has as value an imbrication of objects and my goal is to transform it to sub objects.
My Input :
[
{
"key": "name",
"value": "marc",
"group": "office.people"
}
]
My Spec :
[
{
"operation": "shift",
"spec": {
"*": {
"value": "#(1,key).#(1,group)"
}
}
}
]
Expected:
{
"name": {
"office": {
"people": "marc"
}
}
}
Actual:
{
"name" : {
"office.people" : "marc"
}
}
You could split the group first:
[
{
"operation": "modify-overwrite-beta",
"spec": {
"*": {
"group": "=split('\\.',#(1,group))"
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"value": "#(1,key).#(1,group[0]).#(1,group[1])"
}
}
}
]
Related
I am trying to achieve the following transformation. However, my solution adds undesired null values into the final array.
The transformation needs to shift names in child array for all root elements. I have created 3 cases to illustrate the problem.
Case 1
Input
{
"root": [
{
"child": [
{
"name": "John"
},
{
"name": "Frazer"
}
]
},
{
"child": [
{
"name": "Brandon"
},
{
"name": "Josef"
}
]
}
]
}
Desired Output
{
"NAMES": ["John,Frazer","Brandon,Josef"]
}
Case 2: One child is empty
Input
{
"root": [
{
"child": []
},
{
"child": [
{
"name": "Brandon"
},
{
"name": "Josef"
}
]
}
]
}
Desired Output
{
"NAMES": ["","Brandon,Josef"]
}
Case 3: All childs are empty
Input
{
"root": [
{
"child": []
},
{
"child": []
}
]
}
Desired Output
{
"NAMES": ["",""]
}
EDIT: root array will always have at least 1 element.
Current JOLT spec works fine except for cases where child is an empty array. It generates null values and I'm trying to specify an empty string instead (or any hardcoded string value such as "NO_NAMES")
[
{
"operation": "shift",
"spec": {
"root": {
"*": {
"child": {
"*": {
"name": "NAMES[&3]"
}
}
}
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"NAMES": {
"*": "=trim"
}
}
},
{
"operation": "cardinality",
"spec": {
"NAMES": "MANY"
}
},
{
"operation": "default",
"spec": {
"NAMES": []
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"NAMES": {
"*": "=join(',',#0)"
}
}
}
]
You can apply consecutive transformations modify-overwrite-beta and then shift in order to determine comma-seperated elements of the list(unless they have zero size,this case only double quotes will appear), and then concatenate them within a list such as
[
{
"operation": "modify-overwrite-beta",
"spec": {
"root": {
"*": {
"child": { "*": "#(0,name)" },
"NAMES": "=join(',',#(1,child))"
}
}
}
},
{
"operation": "shift",
"spec": {
"root": {
"*": {
"NAMES": "&"
}
}
}
}
]
In nifi, I am trying to transform a JSON with a variable amount of keys, but will always have a "date" key. I would like to transform the Json and change the string value of the date into a json object. However I am not getting what I need. Which operation/spec can I use in order to accomplish the expected output.
Input:
{
"name": "val1",
"date": "2021-05-19T00:53:20+00:00"
}
Spec:
[
{
"operation": "shift",
"spec": {
"#0": "wrapper"
}
}, {
"operation": "default",
"spec": {
"wrapper": {
"date": { "$date": "${date_attr}" }
}
}
}
]
expected output
{
"wrapper": {
"name": "val1",
"date": {"$date": "2021-05-19T00:53:20+00:00"}
}
}
what I am getting
{
"wrapper": {
"name": "val1",
"date": "2021-05-19T00:53:20+00:00"
}
}
Only single step of shift transformation along with the escaping characters (\\) for $ operator would suffice to use such as
[
{
"operation": "shift",
"spec": {
"name": "wrapper.name",
"date": { "#(1,date)": "wrapper.&.\\$date" }
}
}
]
where we're going one level up by using the first argument 1 in #(1,date) as been staying in the nested object
Edit : Considering that you only need to override the attribute date, and leave the others as they're without individually adding, use the following which again has only single step of shift transformation
[
{
"operation": "shift",
"spec": {
"*": "wrapper.&",
"date": { "#(1,date)": "wrapper.&.\\$date" }
}
}
]
In case of default, if the key mentioned in the spec is missing in the input json, then its added or else no changes may happen.
Here you are trying to push a node to a more level, which can be achieved by creating and assigning an temporary node.
\\ is an escape character.
[
{
"operation": "shift",
"spec": {
"date": "date1",
"#0": "."
}
},
{
"operation": "remove",
"spec": {
"date": ""
}
}, {
"operation": "shift",
"spec": {
"date1": "wrapper.date.\\$date",
"#0": "wrapper"
}
}, {
"operation": "remove",
"spec": {
"wrapper": {
"date1": ""
}
}
}
]
I am not able to find a solution for this transformation. My goal is to sum all parts and storing it into an array with the same length as operations.
Input data:
{
"operations": [
{
"partOne": 10,
"partTwo": 12.5,
"partThree": 30.5
},
{
"partOne": 25.5,
"partTwo": 2,
"partThree": 7.5
}
]
}
Output data:
{
"costs": [
53, // operations[0].partOne + operations[0].partTwo + operations[0].partThree
35 // operations[1].partOne + operations[1].partTwo + operations[1].partThree
]
}
EDIT: I was able to find this intermediate step, still missing the sum part.
[
{
"operation": "shift",
"spec": {
"operations": {
"*": {
"partOne": "costs[&1].[]",
"partTwo": "costs[&1].[]",
"partThree": "costs[&1].[]"
}
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"*": {
"costs": "=doubleSum(#(1,costs[&1]))"
}
}
}
]
Also, if you could share some documentation about JOLT specification I would highly appreciate that. Could not find anything to get me started properly.
You can add one more shift operation between the current ones as rearranging by getting rid of square brackets like this :
[
{
"operation": "shift",
"spec": {
"operations": {
"*": {
"partOne": "costs&1",
"partTwo": "costs&1",
"partThree": "costs&1"
}
}
}
},
{
"operation": "shift",
"spec": {
"*": "&"
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"tot_costs0": "=doubleSum(#(1,costs0))",
"tot_costs1": "=doubleSum(#(1,costs1))"
}
}
]
P.S. this source might help.
I was able to find the solution.
Spec:
[
{
"operation": "shift",
"spec": {
"operations": {
"*": {
"partOne": "costs[&1].[]",
"partTwo": "costs[&1].[]",
"partThree": "costs[&1].[]"
}
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"costs": {
"*": "=doubleSum"
}
}
}
]
In JOLT Based on the jsonarray values the output key and value should be added.
In the output CO2_VAL,CO_VAL,O3_VAL shoould come based on param value in env_values. So, how to apply the above filter.
The input payload is:
{
"id":"abcd",
"env_values":[
{
"param":"CO2",
"values":"20.0"
},
{
"param":"CO",
"values":"21.0"
},
{
"param":"O3",
"values":"22.0"
}
]
}
The output is :
{
"sl":"abcd",
"CO2_VAL":"20.0",
"CO_VAL":"21.0",
"O3_VAL":"22.0"
}
Hope it is what you ment. First of all, we are adding '_VAL' to the key. In the second spec we are putting keys to the values. And at the end we are pairing each key with value.
[
{
"operation": "shift",
"spec": {
"id": "s1",
"env_values": {
"*": {
"param": {
"*": "param[&2].t.&_VAL"
},
"values": "param[&1].values"
}
}
}
},
{
"operation": "shift",
"spec": {
"s1": "s1",
"param": {
"*": {
"t": {
"*": {
"$": "param[&3].key"
}
},
"values": "param[&1].value"
}
}
}
},
{
"operation": "shift",
"spec": {
"s1": "s1",
"param": {
"*": {
"value": "#(1,key)"
}
}
}
}
]
I am new to nifi. I was trying to create a jolt spec but not getting it. Could anyone please help me.
Details as below:
The Attributes in flow file : details, id, name , address, status
Flow file looks like :
[{"to": "xxx1"},{"to": "xxx2"},{"to": "xxx3"},{"to": "xxxn"}]
Expecting below output:
{ "details": "personal",
"home":[
{"mobileno": "xxx1",
"id": "1",
"name" :"bbb",
"address": "Address1" },
{ "mobileno": "xxx2",
"id": "2",
"name": "aaa",
"address": "address2" }
],
"status": "enabled" }
Am able to develop till this. But am not getting how to get "details" field
[{
"operation": "shift",
"spec": {
"*": "home",
"mobileno": "home[0].mobileno"
}
}, {
"operation": "default",
"spec": {
"status": "${status}",
"home[]": {
"*": {
"name": "${name}",
"id" : "${id},
"address": "${address}"
}
}
}
}]
In addition to 7632695's answer, your shift spec won't match "mobileno" on the input, try the following:
[{
"operation": "shift",
"spec": {
"*": {
"to": "home[&1].mobileno"
}
}
}, {
"operation": "default",
"spec": {
"status": "${status}",
"details": "${details}",
"home[]": {
"*": {
"name": "${name}",
"id": "${id}",
"address": "${address}"
}
}
}
}]
Also, note that for a single flow file, the attributes are constant, so for each entry in the home array, each id, name, and address field will be the same. From your attributes, how would JOLT know to use id=1 for the first element and id=2 for the second, and so on?
If you want to use the index of the input array as the id, you can add this spec to your chain:
{
"operation": "shift",
"spec": {
"home": {
"*": {
"$": "home[&1].id",
"*": "home[&1].&"
}
},
"*": "&"
}
}
And if you want them to start at 1 instead of 0, you can add 1 to each of them by adding the following spec to your chain:
{
"operation": "modify-overwrite-beta",
"spec": {
"home": {
"*": {
"id": "=intSum(#0, 1)"
}
}
}
}
In default operation you need to add details attribute.
try with below jolt spec
[{
"operation": "shift",
"spec": {
"*": "home",
"mobileno": "home[0].mobileno"
}
}, {
"operation": "default",
"spec": {
"status": "${status}",
"details":"${details}",
"home[]": {
"*": {
"name": "${name}",
"id": "${id}",
"address": "${address}"
}
}
}
}]