I have incoming JSON rows of data like,
{"signalName": "IU_BATT_ParkAssist", "msgId": 2268, "epoch": 1582322746, "usec": 376360, "vlan": "-1", "msgName": "EBS_Frame12", "vin": "000004", "value": 14.171869, "timestamp": 1582322746376}
I want the output to be modified to produce,
{"IU_BATT_ParkAssist":14.171869, "msgId": 2268, "epoch": 1582322746, "usec": 376360, "vlan": "-1", "msgName": "EBS_Frame12", "vin": "000004", "timestamp": 1582322746376}
The signalName and value keys were combined to make one new key:value pair with the key being the signalName and the value being the value field, "IU_BATT_ParkAssist":14.171869 along with the other original key, value pairs.
How can I achieve this in Nifi given that the signalName field will be dynamically changing in each row?
Try with below spec:
[
{
"operation": "shift",
"spec": {
"#(1,value)": "#(2,signalName)",
"*": "&"
}
},
{
"operation": "remove",
"spec": {
"signalName": "",
"value": ""
}
}
]
In shift operation we are combining signalName and value.
In remove operation we are removing signalName and value from our json data.
Output:
{
"IU_BATT_ParkAssist" : 14.171869,
"msgId" : 2268,
"epoch" : 1582322746,
"usec" : 376360,
"vlan" : "-1",
"msgName" : "EBS_Frame12",
"vin" : "000004",
"timestamp" : 1582322746376
}
Related
how to convert the following data
{
"name" : "abcd",
"middleName" : "srivastav"
}
into
{
"name" : "abcd",
"middleName" : "srivastav"
"id" : "abcd"
}
by using a jolt transform such that the id and the name are within the same object as the expected output.
modify-default-beta or default operations will work for this :
[
{
"operation": "modify-default-beta",
"spec": {
"id": "abcd"
}
}
]
One option is to replicate the value of name through use of "#(0,name)": while keeping the current attributes by "*": "&" notation within a shift transformation such as
[
{
"operation": "shift",
"spec": {
"*": "&",
"#(0,name)": "id"
}
}
]
the demo on the site http://jolt-demo.appspot.com/ is
or another option is to use a modify(overwrite/default) transformation to dynamically replicate the value of name attribute for id such as
[
{
"operation": "modify-overwrite-beta",
"spec": {
"id": "#(1,name)"
}
}
]
the second demo is
I need to create a JSON array in order to split it into several jobs with Nifi.
The array needs to be created based on an existing array inside the JSON.
Can't figure out how to dynamically create a reference to another object in the JSON. I want the reference "#(2,#)" to work, but this is not supported.
INPUT
{
"name": "Loki",
"id": "1234",
"loc": "Utgard",
"age": "unknown",
"listitems": [
"name",
"id"
]
}
SPEC (that doesn't work):
[
{
"operation": "shift",
"spec": {
// Loop all listitems
"listitems": {
"*": {
// Get the value of the current item and push to processlist.type array
"#": "processlist[#2].type",
// Here is the problem, I need to get the "top level" value for the current value/key
"#(2,#)": "processlist[#2].value"
}
}
}
}
]
Expected output:
{
"processlist" : [
{
"type" : "name",
"value" : "Loki"
}, {
"type" : "id",
"value" : "1234"
}
]
}
SPEC (that will run but is not correct)
[
{
"operation": "shift",
"spec": {
// Loop all listitems
"listitems": {
"*": {
// Get the value of the current item and push to processlist.type array
"#": "processlist[#2].type",
// Here is the problem, I need to get the top level value for the current value/key
// Forcing this to "name" will at least execute the code
"#(2,name)": "processlist[#2].value"
}
}
}
}
]
Any ideas?
You can proceed one more step by adding "*" key to nest the current spec more while roaming by #(3,&) dynamically as this ampersand represents the incurred key values name and id such as
[
{
"operation": "shift",
"spec": {
"listitems": {
"*": {
"*": {
"#1": "processlist[#3].type",
"#(3,&)": "processlist[#3].value"
}
}
}
}
}
]
I need to transform a JSON response from one time-series DB and output it to as a response in a new format.
Having an input JSON array, I need to transform it with JoltTransformJson (NiFi) and add a key-value for each item from a FlowFile attribute.
This is my input JSON:
{
"Items": [
{
"Timestamp": "2020-04-29T07:46:20.558731Z",
"Value": 66.0303
},
{
"Timestamp": "2020-04-29T07:46:35.558731Z",
"Value": 69.11584
}
]
}
The desired output should be:
[{
"sensor_id": "xyz",
"sample_time": "2020-04-29T07:46:20.558731Z",
"sample_value": 66.0303
}, {
"sensor_id": "xyz",
"sample_time": "2020-04-29T07:46:35.558731Z",
"sample_value": 69.11584
}]
where sensor_id is a FlowFile attribute...
I came across to this spec:
[{
"operation": "shift",
"spec": {
"Items": {
"*": {
"#Timestamp": "[#2].sample_time",
"#Value": "[#2].sample_value",
"${sensor_id}": "[#2].sensor_id"
}
}
}
}]
But I cannot get the sensor_id in the output json.... instead, this is what I get:
[{
"sample_time": "2020-04-29T07:46:20.558731Z",
"sample_value": 66.0303
}, {
"sample_time": "2020-04-29T07:46:35.558731Z",
"sample_value": 69.11584
}]
Finally I've managed to project the ${sensor_id} into the output json, what was missing is #, so the final spec will be:
[{
"operation": "shift",
"spec": {
"Items": {
"*": {
"#Timestamp": "[#2].sample_time",
"#Value": "[#2].sample_value",
"#${sensor_id}": "[#2].sensor_id"
}
}
}
}]
Now, I've found that the Timestamp returned from my time-series DB is not exact what I need it to be. Instead I would like to convert it to integer timestamp (epoch sinse 1970)... Any idea how this can be achieved?
I have Json array with each of its object having dynamic attributes. I have to merge all the object into one Json object in nifi with common attribute's value concatenated and other attributes as it is.
I have already tried following spec, and it is giving common attributes as array list. But I want that common attribute's value to be concatenated and I don't know how to do that. Any suggestion would be really helpful. Thanks in advance.
[
{
"operation": "shift",
"spec": {
"*": {
"*": "&"
}
}
}
]
The input is like( NOTE: The attributes are dynamic so not sure always going to have these attributes)
[
{
"firstName": "Sam1",
"lastName": "Achom1",
"place": "Silchar1"
},
{
"firstName": "Saya2",
"lastName": "Singh2",
"place": "Macherial2",
"Second1stAttribute": "SomeValue"
},
{
"firstName": "Sam3",
"lastName": "Achom3",
"place": "Silchar3",
"Third2ndAttribute": "SomeValue2"
}
]
The output I am getting with before mentioned spec
{
"firstName" : [ "Sam1", "Saya2", "Sam3" ],
"lastName" : [ "Achom1", "Singh2", "Achom3" ],
"place" : [ "Silchar1", "Macherial2", "Silchar3" ],
"Second1stAttribute" : "SomeValue",
"Third2ndAttribute" : "SomeValue2"
}
And Expecting the output as
{ "firstName" : "Sam1 Saya2 Sam3",
"lastName" : "Achom1 Singh2 Achom3",
"place" : "Silchar1 Mancherial2 Silchar3",
"Second1stAttribute" : "SomeValue",
"Third2ndAttribute" : "SomeValue2"
}
You can "join" the items from your arrays. This spec will do the trick:
[
{
"operation": "shift",
"spec": {
"*": {
"*": "&"
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"*": "=join(' ',#(1,&))"
}
}
]
I'm new with Jolt and I need to have two field having different name but same value coming from the input JSON.
INPUT:
{
"date": 15746555589,
"sensorid": "23r098hd20c8jd02hd0h02300000000000"
}
DESIDERED OUTPUT:
{
"lastseen": 15746555589,
"firstseen": 15746555589,
"sensorid": "23r098hd20c8jd02hd0h02300000000000"
}
This spec should work:
[
{
"operation": "shift",
"spec": {
"date": "lastseen",
"#(1,date)": "firstseen",
"*": "&"
}
}
]