I am testing a flow in NIFI that checks for specific value of a counter using REST API. I am able to extract it correct value from REST response. However, when I check the condition in Route on Attribute processor, the expected matched condition's results are routing to unmatched processor.
Attached is the :
Flow and configuration
I have already checked my response to be "1". But its routing to unmatched branch.
Is there something wrong with NIFI expression language I am using?
Jasim,
Initial setup check counter attribute in which value is 1.
And modify the expression language like ${counter:equals('1')} or ${counter:matches('1')} instead of contains.
because contains not suitable for your scanerio.
Hope this helpful for you.
Related
In a project, I am using spring cloud gateway. I need to discard some requests based on a header value as below:
-If a specific header(say myval1) is not present, discard the request
-If a specific header(say myval1) is present but the value does not match one of a set of values discard the request
-If a specific header(say myval1) is present and the value matches one of a set of values route the request
Normally I would implement a predicate but in my case, the value set to be checked is dynamic means each time I have to check against changing values. In this condition, how can I achieve this feature?
Thanks!
I want to send a dataflow to a webservice which will then respond based on the content of the dataflow. So it might respond with a JSON that says {'Errors':'None'} or {'Errors':'5'}. I would like to then use this information to continue processing the original dataflow if no errors are found, orlog the information if any errors are found. My question is, how can I route based on the values in the JSON response?
You can use EvaluateJsonPath to get the value of Errors into an attribute, then RouteOnAttribute to send the different values down different paths.
Alternatively you could use QueryRecord with queries like SELECT * FROM FLOWFILE WHERE Errors = 'None'. Each query gets its own downstream relationship, so it has the effect of routing in this case.
How to route a flow on the basis of URL(if it has file path route to one processor and if it contains username and password it routes to some different processor ). Please specify the flow of processor to be used. Thanks.
With the help of NiFi Expression Language and RouteOnAttribute processor, you can route the incoming FlowFiles to different processors.
For ex, lets assume that the FlowFiles generated by the source processor has an attribute assigned to it named url and this attribute can have either of the following value formats:
file:///path/to/some/file
http://somesite.xyz/path/to/some/resource
For the above case, you can add two dynamic properties to RouteOnAttribute as follows:
file : ${url:startsWith("file:///")}
websource : ${url:startsWith("http://")}
Then connect file and websource relationships to the different processors. The following links point to a detailed usage guide(s) and examples:
https://community.hortonworks.com/questions/54811/redirecting-flow-based-on-certain-condition-nifi.html
https://docs.hortonworks.com/HDPDocuments/HDF3/HDF-3.1.1/bk_getting-started-with-apache-nifi/content/routing-on-attributes.html
Our requirement is split the flow data based on condition.
We thought to use "ExecuteStreamCommand" processor for that (intern it will use java class) but it is giving single flow data file only. We would like to have two flow data files, one is for matched and another is for unmatched criteria.
I looked at "RouteText" processor but it has no feature to use java class as part of it.
Let me know if anyone has any suggestion.
I think you could use GetMongo to read those definition values and store them in a map accessed by DistributedMapCacheClientService, then use RouteOnContent to route the incoming flowfiles based on the absence/presence of the retrieved values.
If that doesn't work, you could instead route the query result from GetMongo to PutFile and then use ScanContent, which reads from a dictionary file on the file system and routes flowfiles based on the absence/presence of those keywords in the content.
Finally, if all else fails, you can use ExecuteScript to combine those steps into a single processor and route to matched/unmatched relationships. It processes Groovy code easily, so you can directly invoke your existing Java class if necessary.
I need to know how to update the values in nifi processors using Rest API.
https://nifi.apache.org/docs/nifi-docs/rest-api/index.html
For example: I have used below processor structure
GetFile>SplitText>ExtractText>ReplaceText>ConvertJSONToSQL>PUTSQL.
I have passed following inputs for above processors.,
FileLocation(GetFile).
validation(ExtractText).
ReplacementValue(ReplaceText).
DBCP ConnectionPool,username and pwd for SQL.
I just need to use nifi rest api client to write above inputs into processors.
For example : If I give Processor name and input file in Rest API Client then it will write into processor.
Please stop me if anything i'm doing wrong.
Help Appreciated and Tell me any other ways is possible?
Mahen,
You can issue a PUT request to /processors/{id} and provide the new value of the "Replacement Value" property. You'll need to provide JSON body in the request to do this, and you can see the structure by expanding the endpoint noted above on the documentation link you provided, then clicking ProcessorEntity > ProcessorDTO > ProcessorConfigDTO to see the pop-up dialogs with the element listing and examples. You can also quickly get the current values of the processor by issuing a GET request to /processors/{id}.