I'm new to Nifi. I have below json structure to be implemented using Nifi..
{
"jobs": [
{
"jid": "1",
"name": "job1",
"state": "FAILED",
"start-time": 12243
},
{
"jid": "2",
"name": "job2",
"state": "FAILED",
"start-time": 1233
},
{
"jid": "3",
"name": "job2",
"state": "RUNNING",
"start-time": 1223213
}
]
}
I need the output to be like below filtered only FAILED job details
{
[
job1,
job2
]
}
Please help and thanks in advance.
This spec will output a JSON array [job1,job2]:
[
{
"operation": "shift",
"spec": {
"jobs": {
"*": {
"state": {
"FAILED": {
"#(2,name)": "[]"
}
}
}
}
}
}
]
Your desired output of an array inside braces is not valid JSON, as objects need key/value pairs. If instead you want the array in an object, let's call the key failed_jobs and you can use the following spec:
[
{
"operation": "shift",
"spec": {
"jobs": {
"*": {
"state": {
"FAILED": {
"#(2,name)": "failed_jobs[]"
}
}
}
}
}
}
]
Related
I am quite new to Nifi and I already run into an issue:
I have a list of items in an array:
{
"locations": [
1,
2,
3
]
}
I want to transform this to:
{
"locations": [
{
"locationid": 1
},
{
"locationid": 2
},
{
"locationid": 3
}
]
}
Any ideas?
(After this I also want to add another element from a nifi attribute, but I think I will be able to manage that myself.)
Never mind:
[
{
"operation": "shift",
"spec": {
"locations": {
"*": {
"#": "[#2].locationID"
}
}
}
},
{
"operation": "modify-default-beta",
"spec": {
"*": {
"ReportTimestamp": "${ReportTimestamp:toDate('yyyy-MM-dd HH:mm:ss'):toNumber()}"
}
}
}
]
I am trying to achieve the following transformation. However, my solution adds undesired null values into the final array.
The transformation needs to shift names in child array for all root elements. I have created 3 cases to illustrate the problem.
Case 1
Input
{
"root": [
{
"child": [
{
"name": "John"
},
{
"name": "Frazer"
}
]
},
{
"child": [
{
"name": "Brandon"
},
{
"name": "Josef"
}
]
}
]
}
Desired Output
{
"NAMES": ["John,Frazer","Brandon,Josef"]
}
Case 2: One child is empty
Input
{
"root": [
{
"child": []
},
{
"child": [
{
"name": "Brandon"
},
{
"name": "Josef"
}
]
}
]
}
Desired Output
{
"NAMES": ["","Brandon,Josef"]
}
Case 3: All childs are empty
Input
{
"root": [
{
"child": []
},
{
"child": []
}
]
}
Desired Output
{
"NAMES": ["",""]
}
EDIT: root array will always have at least 1 element.
Current JOLT spec works fine except for cases where child is an empty array. It generates null values and I'm trying to specify an empty string instead (or any hardcoded string value such as "NO_NAMES")
[
{
"operation": "shift",
"spec": {
"root": {
"*": {
"child": {
"*": {
"name": "NAMES[&3]"
}
}
}
}
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"NAMES": {
"*": "=trim"
}
}
},
{
"operation": "cardinality",
"spec": {
"NAMES": "MANY"
}
},
{
"operation": "default",
"spec": {
"NAMES": []
}
},
{
"operation": "modify-overwrite-beta",
"spec": {
"NAMES": {
"*": "=join(',',#0)"
}
}
}
]
You can apply consecutive transformations modify-overwrite-beta and then shift in order to determine comma-seperated elements of the list(unless they have zero size,this case only double quotes will appear), and then concatenate them within a list such as
[
{
"operation": "modify-overwrite-beta",
"spec": {
"root": {
"*": {
"child": { "*": "#(0,name)" },
"NAMES": "=join(',',#(1,child))"
}
}
}
},
{
"operation": "shift",
"spec": {
"root": {
"*": {
"NAMES": "&"
}
}
}
}
]
I am trying to give as input a JSON that contains a fixed variable (in my example group) that has as value an imbrication of objects and my goal is to transform it to sub objects.
My Input :
[
{
"key": "name",
"value": "marc",
"group": "office.people"
}
]
My Spec :
[
{
"operation": "shift",
"spec": {
"*": {
"value": "#(1,key).#(1,group)"
}
}
}
]
Expected:
{
"name": {
"office": {
"people": "marc"
}
}
}
Actual:
{
"name" : {
"office.people" : "marc"
}
}
You could split the group first:
[
{
"operation": "modify-overwrite-beta",
"spec": {
"*": {
"group": "=split('\\.',#(1,group))"
}
}
},
{
"operation": "shift",
"spec": {
"*": {
"value": "#(1,key).#(1,group[0]).#(1,group[1])"
}
}
}
]
I am new to nifi. I was trying to create a jolt spec but not getting it. Could anyone please help me.
Details as below:
The Attributes in flow file : details, id, name , address, status
Flow file looks like :
[{"to": "xxx1"},{"to": "xxx2"},{"to": "xxx3"},{"to": "xxxn"}]
Expecting below output:
{ "details": "personal",
"home":[
{"mobileno": "xxx1",
"id": "1",
"name" :"bbb",
"address": "Address1" },
{ "mobileno": "xxx2",
"id": "2",
"name": "aaa",
"address": "address2" }
],
"status": "enabled" }
Am able to develop till this. But am not getting how to get "details" field
[{
"operation": "shift",
"spec": {
"*": "home",
"mobileno": "home[0].mobileno"
}
}, {
"operation": "default",
"spec": {
"status": "${status}",
"home[]": {
"*": {
"name": "${name}",
"id" : "${id},
"address": "${address}"
}
}
}
}]
In addition to 7632695's answer, your shift spec won't match "mobileno" on the input, try the following:
[{
"operation": "shift",
"spec": {
"*": {
"to": "home[&1].mobileno"
}
}
}, {
"operation": "default",
"spec": {
"status": "${status}",
"details": "${details}",
"home[]": {
"*": {
"name": "${name}",
"id": "${id}",
"address": "${address}"
}
}
}
}]
Also, note that for a single flow file, the attributes are constant, so for each entry in the home array, each id, name, and address field will be the same. From your attributes, how would JOLT know to use id=1 for the first element and id=2 for the second, and so on?
If you want to use the index of the input array as the id, you can add this spec to your chain:
{
"operation": "shift",
"spec": {
"home": {
"*": {
"$": "home[&1].id",
"*": "home[&1].&"
}
},
"*": "&"
}
}
And if you want them to start at 1 instead of 0, you can add 1 to each of them by adding the following spec to your chain:
{
"operation": "modify-overwrite-beta",
"spec": {
"home": {
"*": {
"id": "=intSum(#0, 1)"
}
}
}
}
In default operation you need to add details attribute.
try with below jolt spec
[{
"operation": "shift",
"spec": {
"*": "home",
"mobileno": "home[0].mobileno"
}
}, {
"operation": "default",
"spec": {
"status": "${status}",
"details":"${details}",
"home[]": {
"*": {
"name": "${name}",
"id": "${id}",
"address": "${address}"
}
}
}
}]
I'm struggling with a very simple jolt transformation. I want to copy the whole input json (structure not known) into an array in the output json.
So for example the input could be:
{
"rating": {
"quality": {
"value": 3,
"max": 5
}
}
}
The output should look like this:
{
"items": [{
"item": {
"rating": {
"quality": {
"value": 3,
"max": 5
}
}
}
}]
}
Basically put the whole input json into an object called item into an array called items.
Could you please help me?
thx
I found the solution by myself. The required spec is
[
{
"operation": "shift",
"spec": {
"#1": {
"#": "items[].item"
}
}
}
]
[
{
"operation": "shift",
"spec": {
"#": "items[0].item"
}
}
]