I have an Array of Jsons and I want to transform the keys of individual Jsons while still maintaining the Array.
[
{
"a": "1",
"b": "2"
},
{
"a": "one",
"b": "two"
}
]
Desired output:
[
{
"my_a": "1",
"my_b": "2"
},
{
"my_a": "one",
"my_b": "two"
}
]
JOLT Spec:
[
{
"operation": "shift",
"spec": {
"*": {
"a": "my_a",
"b": "my_b"
}
}
}
]
However, I see this:
{
"my_a" : [ "1", "one" ],
"my_b" : [ "2", "two" ]
}
I see that the transformation is applied but the output is not what I expect.
Anyone who has faced similar problems?
You need to include array index [&1] while changing the name
Try with below Jolt Spec:
[
{
"operation": "shift",
"spec": {
"*": {
"a": "[&1].my_a",
"b": "[&1].my_b"
}
}
}
]
Validating the Spec:
Related
I'm receving mistake with #timestamp parameter in my jolt transformation:
[
{
"operation": "shift",
"spec": {
"*": {
"#(1,timestamp)": "values.[#2].timestamp",
"$": [
"values.[#2].id",
"values.[#2].doc_id"
],
"#": "values.[#2].value"
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"values": {
"*": {
"doc_id": "=concat(#(1,doc_id),#(1,timestamp))",
"#timestamp": "=concat(#(1,timestamp),'000')",
"value": "=concat(#(1,value),'')"
}
}
}
}
]
Input ex:
{
"timestamp": 1559350080,
"param": 12
}
Is it possible to use the character # in Jolt Transformation?
# is a special character. Need to prefix with escape characters(\\) in order to make it a literal by replacing "#timestamp" key with "\\#timestamp".
This way, you'll have "#timestamp" : "1559350080000" as key-value pair in the output.
I have the following input json, which has an array of json:
{
"ArrayList": [
{
"a": "value1",
"b": "value2",
"c": "value3"
},
{
"a": "value4",
"b": "value5",
"c": "value6"
},
{
"a": "value7",
"b": "value8",
"c": "value9"
}
]
}
And the desired output is :
{
"ArrayList": [
{
"a": "value1",
"b": "value2"
},
{
"a": "value4",
"b": "value5"
},
{
"a": "value7",
"b": "value8"
}
]
}
What will be the Jolt transform spec expression for this?
Seems you want to remove the attribute c of objects of the ArrayList array, then a remove transformation might be used such as
[
{
"operation": "remove",
"spec": {
"ArrayList": {
"*": {
"c": ""
}
}
}
}
]
In nifi, I am trying to transform a JSON with a variable amount of keys, but will always have a "date" key. I would like to transform the Json and change the string value of the date into a json object. However I am not getting what I need. Which operation/spec can I use in order to accomplish the expected output.
Input:
{
"name": "val1",
"date": "2021-05-19T00:53:20+00:00"
}
Spec:
[
{
"operation": "shift",
"spec": {
"#0": "wrapper"
}
}, {
"operation": "default",
"spec": {
"wrapper": {
"date": { "$date": "${date_attr}" }
}
}
}
]
expected output
{
"wrapper": {
"name": "val1",
"date": {"$date": "2021-05-19T00:53:20+00:00"}
}
}
what I am getting
{
"wrapper": {
"name": "val1",
"date": "2021-05-19T00:53:20+00:00"
}
}
Only single step of shift transformation along with the escaping characters (\\) for $ operator would suffice to use such as
[
{
"operation": "shift",
"spec": {
"name": "wrapper.name",
"date": { "#(1,date)": "wrapper.&.\\$date" }
}
}
]
where we're going one level up by using the first argument 1 in #(1,date) as been staying in the nested object
Edit : Considering that you only need to override the attribute date, and leave the others as they're without individually adding, use the following which again has only single step of shift transformation
[
{
"operation": "shift",
"spec": {
"*": "wrapper.&",
"date": { "#(1,date)": "wrapper.&.\\$date" }
}
}
]
In case of default, if the key mentioned in the spec is missing in the input json, then its added or else no changes may happen.
Here you are trying to push a node to a more level, which can be achieved by creating and assigning an temporary node.
\\ is an escape character.
[
{
"operation": "shift",
"spec": {
"date": "date1",
"#0": "."
}
},
{
"operation": "remove",
"spec": {
"date": ""
}
}, {
"operation": "shift",
"spec": {
"date1": "wrapper.date.\\$date",
"#0": "wrapper"
}
}, {
"operation": "remove",
"spec": {
"wrapper": {
"date1": ""
}
}
}
]
I'm new to Nifi. I have below json structure to be implemented using Nifi..
{
"jobs": [
{
"jid": "1",
"name": "job1",
"state": "FAILED",
"start-time": 12243
},
{
"jid": "2",
"name": "job2",
"state": "FAILED",
"start-time": 1233
},
{
"jid": "3",
"name": "job2",
"state": "RUNNING",
"start-time": 1223213
}
]
}
I need the output to be like below filtered only FAILED job details
{
[
job1,
job2
]
}
Please help and thanks in advance.
This spec will output a JSON array [job1,job2]:
[
{
"operation": "shift",
"spec": {
"jobs": {
"*": {
"state": {
"FAILED": {
"#(2,name)": "[]"
}
}
}
}
}
}
]
Your desired output of an array inside braces is not valid JSON, as objects need key/value pairs. If instead you want the array in an object, let's call the key failed_jobs and you can use the following spec:
[
{
"operation": "shift",
"spec": {
"jobs": {
"*": {
"state": {
"FAILED": {
"#(2,name)": "failed_jobs[]"
}
}
}
}
}
}
]
could anyone help to convert below mention input json to desired output json using jolt ?
Here in treefield list i want to fetch "paramid" and "paramvalue" to first level for only matched "paramid", rest of the list items should be intact in that treefield list.
e.g i want to take paramid "k1" with its value to first level as mentioned in the output.
Input
{
"A": "value1",
"B": "value2",
"C": {
"D": "x1",
"E": {
"treefield": [
{
"paramid": "k1",
"paramvalue": {
"string": "value1"
}
},
{
"paramid": "k2",
"paramvalue": {
"string": "value2"
}
},
{
"paramid": "k3",
"paramvalue": {
"string": "value3"
}
}
]
},
"F": {
"a": "x1",
"x": {
"y": 1
}
},
"H": "x4"
}
}
]```
**Output**
```[
{
"A": "value1",
"B": "value2",
"C": {
"D": "x1",
"E": {
"treefield": [
{
"paramid": "k1",
"paramvalue": {
"string": "value1"
}
},
{
"paramid": "k3",
"paramvalue": {
"string": "value3"
}
}
]
},
"F": {
"a": "x1",
"x": {
"y": 1
}
},
"H": "x4"
},
"k2": "value2"
}
]```
If I understand what you're trying to do (pull out key/value pairs and put them at the top level), this spec should do it:
[
{
"operation": "shift",
"spec": {
"C": {
"E": {
"treefield": {
"*": {
"paramvalue": {
"string": "#(2,paramid)"
}
}
}
},
"*": "&"
},
"*": "&"
}
}
]
That works on individual JSON objects. In your sample input you don't have a leading array bracket but you do have a trailing one, so if you expect multiple objects in an array and want them output in an array, this spec should work:
[
{
"operation": "shift",
"spec": {
"*": {
"C": {
"E": {
"treefield": {
"*": {
"paramvalue": {
"string": "[&6].#(2,paramid)"
}
}
}
},
"*": "[&2].&"
},
"*": "[&1].&"
}
}
}
]