How do I access json fields with Jolt transform?
For example I have this json:
{
"a": 110,
"b": 10
}
I would like to have:
{
"a": 110,
"b": 10,
"c": 100 // 110 - 10 (substraction)
}
The following transformation will add a c variable which is set to a - b:
[
{
"operation": "shift",
"spec": {
"a": "a",
"b": "b"
}
},
{
"operation": "modify-default-beta",
"spec": {
"c": "=intSubtract(#(1,a), #(1,b))"
}
}
]
If you wish to test it, the Jolt demo website is an excellent resource. Put your original JSON into the "JSON Input" box:
{
"a": 110,
"b": 10
}
Then place the transformation spec from the top of this answer into the "JOLT Spec" box and hit the Transform button. The result should be as you desired:
{
"a" : 110,
"b" : 10,
"c" : 100
}
You just can use a single modify-overwrite-beta transformation along with a intSubtract function in order to add add an extra element to the current json value such as
[
{
"operation": "modify-overwrite-beta",
"spec": {
"c": "=intSubtract(#(1,a),#(1,b))"
}
}
]
Related
I have the following input in Nifi Jolt Specification processor:
[
{
"values": [
{
"id": "paramA",
"value": 1
}
]
},
{
"values": [
{
"id": "paramB",
"value": 3
}
]
}
]
Expected output:
[
{
"id": "paramA",
"value": 1
},
{
"id": "paramB",
"value": 2
}
]
Can you explain how I have to do?
thanks in advance
You want to reach the objects of the values array which are nested within seperate object signs ({}). A "*" notation is needed in order to cross them over per each individual values array, and then use another "*" notation for indexes of those arrays while choosing "" as the counterpart values in order to grab nothing but the sub-objects such as
[
{
"operation": "shift",
"spec": {
"*": {
"values": {
"*": ""
}
}
}
}
]
Can we use string manipulation functions like below in JoltTransformJSON
${BIG:padRight(5, '#')}
Link:
https://jolt-demo.appspot.com/#modify-stringFunctions
Expected Output: BIG##
Like we use
"small_toUpper": "=toUpper(#(1,small))",
"BIG_toLower": "=toLower(#(1,BIG))",
I am trying but it's not giving any error, but not giving the desired result as well. What would be other alternative for this.
Input JSON:
{
"x": [
3,
2,
1,
"go"
],
"small": "small",
"BIG": "BIG",
"people": [
{
"firstName": "Bob",
"lastName": "Smith",
"address": {
"state": null
}
},
{
"firstName": "Sterling",
"lastName": "Archer"
}
]
}
Spec:
[
{
"operation": "modify-default-beta",
"spec": {
"y": "=join(',',#(1,x))",
"z": "=join(' ',#(1,x))",
"small_toUpper": "=toUpper(#(1,small))",
"BIG_toLower": "=toLower(#(1,BIG))",
"BIG_padding": "=padRight(#(5,BIG))"
}
}
]
You can use rightPad function along with modify-xXx-beta (xXx:default or overwrite) transformation such as
[
{
"operation": "modify-default-beta",
"spec": {
"BIG_padding": "=rightPad(#(1,BIG),5,'#')"
}
}
]
I need to transform a JSON response from one time-series DB and output it to as a response in a new format.
Having an input JSON array, I need to transform it with JoltTransformJson (NiFi) and add a key-value for each item from a FlowFile attribute.
This is my input JSON:
{
"Items": [
{
"Timestamp": "2020-04-29T07:46:20.558731Z",
"Value": 66.0303
},
{
"Timestamp": "2020-04-29T07:46:35.558731Z",
"Value": 69.11584
}
]
}
The desired output should be:
[{
"sensor_id": "xyz",
"sample_time": "2020-04-29T07:46:20.558731Z",
"sample_value": 66.0303
}, {
"sensor_id": "xyz",
"sample_time": "2020-04-29T07:46:35.558731Z",
"sample_value": 69.11584
}]
where sensor_id is a FlowFile attribute...
I came across to this spec:
[{
"operation": "shift",
"spec": {
"Items": {
"*": {
"#Timestamp": "[#2].sample_time",
"#Value": "[#2].sample_value",
"${sensor_id}": "[#2].sensor_id"
}
}
}
}]
But I cannot get the sensor_id in the output json.... instead, this is what I get:
[{
"sample_time": "2020-04-29T07:46:20.558731Z",
"sample_value": 66.0303
}, {
"sample_time": "2020-04-29T07:46:35.558731Z",
"sample_value": 69.11584
}]
Finally I've managed to project the ${sensor_id} into the output json, what was missing is #, so the final spec will be:
[{
"operation": "shift",
"spec": {
"Items": {
"*": {
"#Timestamp": "[#2].sample_time",
"#Value": "[#2].sample_value",
"#${sensor_id}": "[#2].sensor_id"
}
}
}
}]
Now, I've found that the Timestamp returned from my time-series DB is not exact what I need it to be. Instead I would like to convert it to integer timestamp (epoch sinse 1970)... Any idea how this can be achieved?
I have JSON payload like this;
{
"id": "",
"name": "",
"A": {...},
"B": {...},
"C": {...}
}
And I want to extract A, B and C fields with id and name field as different record. Like this;
{
"id": "",
"name": "",
"A": {...}
}
{
"id": "",
"name": "",
"B": {...}
}
{
"id": "",
"name": "",
"C": {...}
}
I'm using record based processors. But I don't know that how can I do this in Nifi using record based processors.
The "EvaluateJsonPath" is probably what you're looking for. You can add JSONPath expressions, that will be converted to attributes, or written to the flowfile.
http://jsonpath.com/ is a handy web tool to test your expressions.
If you want to use record based processors, then JoltTransformRecord would do the trick. Just set Jolt Transformation DSL as Chain and Jolt Specification as:
[
{
"operation": "shift",
"spec": {
"id": "id",
"name": "name",
"*": {
"#": "array.&"
}
}
},
{
"operation": "shift",
"spec": {
"array": {
"*": {
"#(2,id)": "[#2].id",
"#(2,name)": "[#2].name",
"#": "[#2].&"
}
}
}
}
]
This will first put your unique elements in an array and separate the common keys from them, then it will put the common keys in all of the elements while extracting the array to a top array.
Then, if you want them as different FlowFiles too, you can SplitRecord the array and you got it!
I have the input as below,
INPUT:
{
"value": 265.3
}
I want to remove the decimal values.
OUTPUT:
{
"value": 265
}
JOLT:
[{
"operation": "modify-overwrite-beta",
"spec": {
"value": "=toLong"
}
}]
This works fine for removing decimals but not for rounding up like,
268.1 ---> 268
268.9 ---> 269
Can anyone please suggest me a solution for the aforementioned scenario?
Thanks.
You can do it by rounding and then converting to int:
Input Json:
{
"value1": 268.1,
"value2": 268.9
}
Jolt spec
[
{
"operation": "modify-overwrite-beta",
"spec": {
"*": "=divideAndRound(0,#(0),1)"
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"*": "=toInteger"
}
}
]
Result:
{
"value1" : 268,
"value2" : 269
}
There is not OOTB support for that in Jolt.