I had broken my mind with JOLT.
I need to group by data[].key then transform array in each group to object.
I had source JSON
{
"status": "OK",
"data": [
{
"value": 1,
"key": "John",
"subKey": "1W"
},
{
"value": 1,
"key": "John",
"subKey": "3W"
},
{
"value": 2,
"key": "John",
"subKey": "2W"
},
{
"value": 4,
"key": "John",
"subKey": "older"
},
{
"value": 5,
"key": "Tharanya",
"subKey": "2W"
},
{
"value": 5,
"key": "Tharanya",
"subKey": "1W"
}
]
}
Desired result is:
{
"status": "OK",
"data": [
{
"user": "John",
"1W": 1,
"2W": 2,
"3W": 1,
"older": 4
},
{
"user": "Tharanya",
"2W": 5,
"1W": 5
}
]
}
This transformation aggregates flat-list to map
[
{
"operation": "shift",
"spec": {
"data": {
"*": {
"#": "data.#(1,key)"
}
}
}
}
]
This one transform array to object
[
{
"operation": "shift",
"spec": {
"*": {
"value": "#(1,subKey)"
}
}
}
]
I need help to combine this transformation to chain.
Thank you.
IMAGE: Group transformation
IMAGE: array to object
You can mix up those current methods within the first shift transformation such as
[
{
"operation": "shift",
"spec": {
"*": "&",
"data": {
"*": {
"key": "&2.#(1,key).user",
"#value": "&2.#(1,key).#subKey"
}
}
}
},
{
// get rid of key names of the objects within the "data" array
"operation": "shift",
"spec": {
"*": "&",
"data": {
"*": "&1"
}
}
},
{
// get rid of repeating components of "user" attribute
"operation": "cardinality",
"spec": {
"*": {
"*": {
"*": "ONE"
}
}
}
}
]
Related
My input JSON:
[
{
"label": [
{
"name": "abc"
}
]
},
{
"label": [
{
"name": "xyz"
}
]
}
]
My spec:
[{
"operation": "shift",
"spec": {
"*": {
"label": {
"0": {
"name": "name"
}
}
}
}
}]
Expected output:
[{
"name": "abc"
}, {
"name": "xyz"
}]
Generated output:
{
"name" : [ "abc", "xyz" ]
}
How do I not combine the array in the spec?
Try this spec,
[
{
"operation": "shift",
"spec": {
"*": {
"label": {
"*": {
"name": "[&3].name"
}
}
}
}
}
]
I am using NiFi Jolt Processor to transform some JSON data.
My JSON field genre contains shared attributes and an array that contains the list of actual genre names.
I needed to transform the "genre" attribute to become an array of "genres" containing the list of the common attributes and the different genre names.
I have the following input JSON:
{
"programIdentifier": "9184663",
"programInstance": {
"genre": {
"source": "GN",
"locked": false,
"lastModifiedDate": 1527505462094,
"lastModifiedBy": "Some Service",
"genres": [
"Miniseries",
"Drama"
]
}
}
}
I have tried the following spec:
[{
"operation": "shift",
"spec": {
"programIdentifier": ".&",,
"genre": {
"source": "genres[].source.value",
"locked": "genres[].locked",
"lastModifiedDate": "genres[].lastModifiedDate",
"lastModifiedBy": "genres[].lastModifiedBy",
"genres": {
"*": "genres[&0].name"
}
}
}]
This my expected output:
{
"programIdentifier": "9184663",
"programInstance": {
"genres": [
{
"source": {
value: "GN"
}
"locked": false,
"lastModifiedDate": 1527505462094,
"lastModifiedBy": "Some Service",
"name": "Miniseries"
},
{
"source": {
value: "GN"
}
"locked": false,
"lastModifiedDate": 1527505462094,
"lastModifiedBy": "Some Service",
"name": "Drama"
}
]
}
}
But it's coming out as:
{
"programIdentifier": "9184663",
"programInstance": {
"genres": [
{
"source": {
"value": "GN"
},
"name": "Miniseries"
}, {
"locked": false,
"name": "Drama"
}, {
"lastModifiedDate": 1527505462094
}, {
"lastModifiedBy": "Some Service"
}],
}
}
Is it what you want to achieve?
[
{
"operation": "shift",
"spec": {
"programIdentifier": ".&",
"programInstance": {
"genre": {
"genres": {
"*": {
"#2": {
"source": "programInstance.genres[&2].source[]",
"locked": "programInstance.genres[&2].locked",
"lastModifiedDate": "programInstance.genres[&2].lastModifiedDate",
"lastModifiedBy": "programInstance.genres[&2].lastModifiedBy"
},
"#": "programInstance.genres[&1].name"
}
}
}
}
}
}
]
I need to transform below input to exact output using JOLT transformation.
Input:
{
"First_Name": "Test",
"Last_Name": "User",
"allpointofInterest": "someText",
"campaign_id": "123456789",
"lead_id": "123456789"
}
Output:
"input" : {
"tokens" : [ {
"name" : "{{my.First_Name}}",
"value" : "Test"
}, {
"name" : "{{my.Last_Name}}",
"value" : "User"
}, {
"name" : "{{my.allpointofInterest}}",
"value" : "someText"
} ]
},
"leads": [{
"id": "123456789"
}]
}
I tried with below spec but not driven to exact output what i needed.
spec:
[
{
"operation": "remove",
"spec": {
"campaign_id": ""
}
}, {
"operation": "shift",
"spec": {
"lead_id": {
"#": "input.leads[#1].id"
},
"*": "temptoken.&"
}
}, {
"operation": "shift",
"spec": {
"*": "&",
"temptoken": {
"*": {
"$": "input.tokens[#2].tmpname",
"#": "input.tokens[#2].value"
}
}
}
}, {
"operation": "modify-overwrite-beta",
"spec": {
"input": {
"leads": {
"*": {
"id": ["=toInteger", 0]
}
}
}
}
}, {
"operation": "modify-default-beta",
"spec": {
"input": {
"tokens": {
"*": {
"name": "=concat('{{my.',#(1,tmpname), '}}')"
}
}
}
}
}, {
"operation": "remove",
"spec": {
"input": {
"tokens": {
"*": {
"tmpname": ""
}
}
}
}
}
]
Any help would be much appreciated. Thanks
Does sth like that suits you?
[
{
"operation": "shift",
"spec": {
"#{{my.First_Name}}": "input.tokens[0].name",
"First_Name": "input.tokens[0].value",
"#{{my.Last_Name}}": "input.tokens[1].name",
"Last_Name": "input.tokens[1].value",
"#{{my.allpointofInterest}}": "input.tokens[2].name",
"allpointofInterest": "input.tokens[2].value",
"lead_id": "leads[#].id"
}
}
]
I am trying to modify my input and add an id for each object with '-indexNum' but this seems not working.Here I m posting my code.
My input,
{
"employees": [{
"f_name": "tom",
"l_name": "smith"
},
{
"f_name": "don",
"l_name": "jones"
}
]
}
Expected o/p:
{
"employees": [{
"firstName": "tom",
"lastName": "smith",
"test": "emp-"
}, {
"firstName": "don",
"lastName": "jones",
"test": "emp-"
}]
}
My spec,
[{
"operation": "shift",
"spec": {
"employees": {
"*": {
"f_name": "employees[&1].firstName",
"l_name": "employees[&1].lastName"
}
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"employees": {
"*": {
"test": "=concat('emp-',)"
}
}
}
}]
I am trying to modify my input and add an id for each object with '-indexNum'.Can anyone please suggest me help.Thanks.
Spec
[
{
"operation": "shift",
"spec": {
"employees": {
"*": {
"$": "employees[&1].employeeNum",
"f_name": "employees[&1].firstName",
"l_name": "employees[&1].lastName"
}
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"employees": {
"*": {
"employeeNum": "=concat('emp-',#(1,employeeNum))"
}
}
}
}
]
I need to transform a JSON data using the JoltTransformJSON inside Nifi, here's my spec which I am using for the transformation:
[{
"operation": "shift",
"spec": {
"*": {
"Header": {
"readOn": "created_date_time",
"fileName": "readFile"
},
"Data": {
"id": "Id",
"first_name": "First_Name",
"last_name": "Last_Name",
}
}
}}]
My input data:
[{
"Header": {
"readOn": "2017/04/18 10:55:05",
"fileName": "sample1.csv",
"recordNum": 1
},
"Data": {
"last_name": "Martin",
"id": 21,
"first_name": "Clarence"
}
}, {
"Header": {
"readOn": "2017/04/18 10:55:05",
"fileName": "sample.csv",
"recordNum": 2
},
"Data": {
"last_name": "Graham",
"id": 22,
"first_name": "Walter"
}
}]
Output what I m getting:
{
"created_date_time": ["2017/04/18 10:55:05", "2017/04/18 10:55:05"],
"readFile": ["sample1.csv", "sample2.csv"],
"Id": [21, 22],
"First_Name": ["Clarence", "Walter"],
"Last_Name": ["Martin", "Graham"]
}
The required output:
[{
"recordNum": 1,
"Header": {
"created_date_time": "2017/04/18 10:55:05",
"readFile": "getusroi.csv"
},
"Data": {
"Last_Name": "Martin",
"Id": 21,
"First_Name": "Clarence"
}
}, {
"recordNum": 2,
"Header": {
"created_date_time": "2017/04/18 10:55:05",
"readFile": "getusroi.csv"
},
"Data": {
"Last_Name": "Graham",
"Id": 22,
"First_Name": "Walter"
}
}]
Question: Can someone guide me where all I need to change my jolt spec to achieve the desired transformed output.
The trick is to walk back up the tree until you get to the element in the array, then you can reference that index in the target. Try this spec:
[
{
"operation": "shift",
"spec": {
"*": {
"Header": {
"recordNum": "[&2].recordNum",
"readOn": "[&2].&1.created_date_time",
"fileName": "[&2].&1.readFile"
},
"Data": {
"id": "[&2].&1.Id",
"first_name": "[&2].&1.First_Name",
"last_name": "[&2].&1.Last_Name"
}
}
}
}
]