I am very new to Nifi. In Nifi using JoltTransformJson I need to convert following input to below output: Can someone please provide the Jolt specification. I have tried several ways to do this but could not get the same output.
Input
{
"myOps": {
"Ops1": {
"type": "software",
"url": "url-software"
},
"Ops2": {
"type": "hardware",
"url": "url-hardware"
}
}
}
Output
{
"type": "software",
"url": "url-software"
},
{
"type": "hardware",
"url": "url-hardware"
}
You don't need Jolt for this. You can use the EvaluateJsonPath processor with a JsonPath expression of $.myOps.* which will return a valid JSON array containing the two objects you're looking for. JsonPath.com is a good resource for evaluating & debugging JsonPath expressions.
If you have a requirement to avoid the wrapping [] (although that's valid JSON), you can use ReplaceText to remove them.
Check this spec,
[
{
"operation": "shift",
"spec": {
"myOps": {
"Ops*": {
"#": "[]"
}
}
}
}
]
Related
In an Apache NIFI dataflow I try to transform a date from MM-dd-yyyy to yyyy-MM-dd (the default format which wil be accepted by MySQL as a date).
In the NIFI advanced editor of a JoltTransformJSON 1.15.0 Processor with DSL "Chain" I have entered:
Input Json:
{
"Name": "Jan",
"Birthday": "12-31-1994"
}
Transformation Jolt script:
[{
"operation": "modify-overwrite-beta",
"spec": {
"Birthday": "=${Birthday:toDate('MM-dd-yyyy'):format('yyyy-MM-dd')}"
}
}]
Result:
{
"Name": "Jan",
"Birthday": "12-31-1994"
}
I do not get any syntax errors. It seems the Jolt transformation does not change anything.
Why is Birthday not transformed?
You can use the following consecutive specs within a JoltTransformJSON processor's specification part
[
{
"operation": "modify-overwrite-beta",
"spec": {
"b": "=split('-', #(1,Birthday))",
"Birthday": "=concat(#(1,b[2]),'-',#(1,b[0]),'-',#(1,b[1]))"
}
},
{
"operation": "remove",
"spec": {
"b": ""
}
}
]
where split the current date value by dashes and reorder them by use of concat function in the first step. Then remove the auxiliary b attribute in the last step
Input JSON
{
"Uid": "00d44cd2-38c9-4d96-b0e8-e34a476b2be8", // to be removed
"eventId": "00d44cd2-38c9-4d96-b0e8-e34a476b2be8",
"clientId": "**asd",
"customerIdHash": "1e18411a-63b2-eb11-812d-asdasd",
"eventTimestamp": "2021-05-"
}
I want to remove the key Uid and its value then send it to the next nifi processor as it is.
How I can achieve this in NIFI?
OutPut JSON:
{
"eventId": "00d44cd2-38c9-4d96-b0e8-e34a476b2be8",
"clientId": "**asd",
"customerIdHash": "1e18411a-63b2-eb11-812d-005056a43026",
"eventTimestamp": "2021-05-"
}
You can use remove spec,
[
{
"operation": "remove",
"spec": {
"Uid": ""
}
}
]
I need to transform a JSON response from one time-series DB and output it to as a response in a new format.
Having an input JSON array, I need to transform it with JoltTransformJson (NiFi) and add a key-value for each item from a FlowFile attribute.
This is my input JSON:
{
"Items": [
{
"Timestamp": "2020-04-29T07:46:20.558731Z",
"Value": 66.0303
},
{
"Timestamp": "2020-04-29T07:46:35.558731Z",
"Value": 69.11584
}
]
}
The desired output should be:
[{
"sensor_id": "xyz",
"sample_time": "2020-04-29T07:46:20.558731Z",
"sample_value": 66.0303
}, {
"sensor_id": "xyz",
"sample_time": "2020-04-29T07:46:35.558731Z",
"sample_value": 69.11584
}]
where sensor_id is a FlowFile attribute...
I came across to this spec:
[{
"operation": "shift",
"spec": {
"Items": {
"*": {
"#Timestamp": "[#2].sample_time",
"#Value": "[#2].sample_value",
"${sensor_id}": "[#2].sensor_id"
}
}
}
}]
But I cannot get the sensor_id in the output json.... instead, this is what I get:
[{
"sample_time": "2020-04-29T07:46:20.558731Z",
"sample_value": 66.0303
}, {
"sample_time": "2020-04-29T07:46:35.558731Z",
"sample_value": 69.11584
}]
Finally I've managed to project the ${sensor_id} into the output json, what was missing is #, so the final spec will be:
[{
"operation": "shift",
"spec": {
"Items": {
"*": {
"#Timestamp": "[#2].sample_time",
"#Value": "[#2].sample_value",
"#${sensor_id}": "[#2].sensor_id"
}
}
}
}]
Now, I've found that the Timestamp returned from my time-series DB is not exact what I need it to be. Instead I would like to convert it to integer timestamp (epoch sinse 1970)... Any idea how this can be achieved?
I have JSON payload like this;
{
"id": "",
"name": "",
"A": {...},
"B": {...},
"C": {...}
}
And I want to extract A, B and C fields with id and name field as different record. Like this;
{
"id": "",
"name": "",
"A": {...}
}
{
"id": "",
"name": "",
"B": {...}
}
{
"id": "",
"name": "",
"C": {...}
}
I'm using record based processors. But I don't know that how can I do this in Nifi using record based processors.
The "EvaluateJsonPath" is probably what you're looking for. You can add JSONPath expressions, that will be converted to attributes, or written to the flowfile.
http://jsonpath.com/ is a handy web tool to test your expressions.
If you want to use record based processors, then JoltTransformRecord would do the trick. Just set Jolt Transformation DSL as Chain and Jolt Specification as:
[
{
"operation": "shift",
"spec": {
"id": "id",
"name": "name",
"*": {
"#": "array.&"
}
}
},
{
"operation": "shift",
"spec": {
"array": {
"*": {
"#(2,id)": "[#2].id",
"#(2,name)": "[#2].name",
"#": "[#2].&"
}
}
}
}
]
This will first put your unique elements in an array and separate the common keys from them, then it will put the common keys in all of the elements while extracting the array to a top array.
Then, if you want them as different FlowFiles too, you can SplitRecord the array and you got it!
I am new to nifi. I need help to make new json template using
jolttransform processor. could anyone please help to make joltspec for this requirement.
From convertavrotojson processor flow file
am getting the following attributes
name, address, id,status
And from its content file, am getting mobileno.
Expecting the following output
{
"id": "1",
"details":[
{
"mobileno": "xxxxx",
"name ": "AAAA",
"address": "addressline1"
}
],
"status" :"true"
}
As you are having name, address, id,status attributes to the flowfile,In new versions of NiFi-1.2+ we can add attributes into json message using jolt.
Try with below jolt spec:-
[
{
"operation": "shift",
"spec": {
"mobileno": "details[0].mobileno"
}
},
{
"operation": "default",
"spec": {
"id":"${id}",
"status":"${status}",
"details[]": {
"*": {
"name":"${name}",
"address":"${address}"
}
}
}
}
]
Jolt Config screenshot:-
For more reference
https://community.hortonworks.com/questions/152046/nifi-how-to-use-jolt-to-add-json-keyvalue-dynamica.html