I need some help in UpdateAttribute processor:
I have a CSV file which contains hostnames. I need to separate each hostname in the FlowFile and pass it as a variable to the REST API.
My REST API part is working fine when passing data manually. However, I didn't get how to pass a variable value as hostname in it.
Sharing sample file:
SRHAPP001,SRHWEBAPP002,SRHDB006,SRHUATAPP4,ARHUATDB98
I don't quite understand your goal, but I assume that you try to pass the hostname to your REST API module by using FlowFile variables.
You can achieve this by using the ExtractText-Processor. You simply use RegEx for separating your hostnames from the CSV file.
For more information, see
https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.12.1/org.apache.nifi.processors.standard.ExtractText/
How can I extract a substring from a flowfile data in Nifi?
If needed, you can split incoming FlowFiles on every hostname by using SplitContent-Processor
Related
In my JMeter test plan, I have two CSV Data Config elements.
CSV Data Config-element-requestBodies: This CSV Data Config element has a variable named "requestBody". In the corresponding CSV file, each line has an XML request with a placeholder string.
CSV Data Config-element-subject: variables. This CSV Data Config element has a variable name "subjectDn". In the corresponding CSV file, each line has a distinguished name (DN) for a user.
The "requestBody" variable will contain an XML request from the CSV file that has the request bodies, with a placeholder string and I want to replace that placeholder with the contents of the "subjectDn" variable, and then use the modified "requestBody" variable in the HTTP Request element.
In other words:
Replace the placeholder string in the ${requestBody} with the value in the ${subjectDn}, then
Use the modified ${requestBody} in the HTTP Request
How can I do that? What is the easiest/least overhead way of doing that?
I've tried using the "__evalVar" function (see https://am.net/lib/tools/NetworkManagement/apache-jmeter-2.8/printable_docs/usermanual/functions.html), but it looks like that is not able to do the replacement when the string is from a CSV file?
Thanks in advance!
Jim
Wrap your ${requestBody} into __eval() function, it will allow JMeter to resolve JMeter Functions and Variables which are present as "placeholders" in the CSV file to their respective values.
${__eval(${requestBody})}
Also if you consider scripting to accomplish your requirements be informed that starting from JMeter 3.1 it's recommended to use JSR223 Test Elements and Groovy language. More information: Apache Groovy: What Is Groovy Used For?
I was able to get this woring by using a Beanshell pre-processor and some Java code that does var.get() and then Java String replaceall() and then var.put().
while using RouteOnAttribute nifi processor , i have input of json data
[{"dev":"xyz","detail":"abc"}] which i got from convertRecord processor
Routing Strategy :Route to Property name
ifmatch: ${dev:equals( "xyz" )}
I tried ${dev:matches( "xyz")} in both single quotes and double quotes still am not getting flowfile redirecting towards "ifmatch" . its redirecting to unmatched
is there anyway to resolve this i tried many other option
The flowfile content is different from attributes. Content is arbitrary -- could be empty, text, KB of XML, GB of video or binary. Each flowfile also has attributes which are key/value pairs of Strings kept in memory.
If you want to route on this piece of data, you have multiple options:
Use RouteOnText or RouteOnContent to use the actual flowfile content directly.
Extract it to an attribute using EvaluateJsonPath and then route on that attribute.
The Apache NiFi User Guide and In-Depth provide more information around this distinction.
I am getting some numerical data with API from URL and I am looking for a way to make some mathematical operations in apache nifi before putting data to file directory. Thanks already now.
By the way, I am using InvokeHTTP processor to get data and to put file in somewhere I am using PutFile processor. I searched some related websites but I could not find out a working way.
Try using QueryRecord processor and Define Record Reader/Writer controller services to read/write the flowfile.
Add new property to the QueryRecord processor by using Apache calcite SQL query with your mathematical operations on flowfile.
Results of the SQL query will be added to the outgoing flowfile in your desired format.
Ultimately the answer depends on whether the data you're working with is in the content of the FlowFile or in the attributes. If the data is small enough and it's only a couple operations, the suggested approach would be to work with the data as attributes and use NiFi's expression language to do the transformations.
There is a section of mathematical operations[1] in the Apache documentation[2]. The operations range from simple operand like plus/minus to exposing the java.lang.Math static methods.
[1] https://nifi.apache.org/docs/nifi-docs/html/expression-language-guide.html#numbers
[2] https://nifi.apache.org/docs.html
You can try ExecuteStreamCommand if you want to intake the whole file and then run operations. Alternatively, you can fiddle around with the variables on the flowfile - depending on how large your operation is.
For example if you have some initial variables you can include them in the name of your file and then extract them, run the operations within the variables of the flowfile, then add to the bottom of the original file
New to NiFi!
I've split a flowfile into a single line of text using splitJSON processor.
The NiFi flowfile contents are as follows:
abcdefg
I'd like to be able to take the text in the flowfile and either add it to a url to make a subsequent call using InvokeHTTP or add the contents of the flowfile as an attribute so I can make the subsequent call using InvokeHTTP like so
http://localhost/${my.newly.added.attribute}
How do i do this?
Any help would be appreciated!
Thanks in advance!
ExtractText will allow you to find sections of content and place in an attribute on the FlowFile. For your example, you could capture the entirety of the content and assign to an attribute my.newly.added.attribute. InvokeHTTP would then access it using Expression Language 2 as in your example.
Our requirement is split the flow data based on condition.
We thought to use "ExecuteStreamCommand" processor for that (intern it will use java class) but it is giving single flow data file only. We would like to have two flow data files, one is for matched and another is for unmatched criteria.
I looked at "RouteText" processor but it has no feature to use java class as part of it.
Let me know if anyone has any suggestion.
I think you could use GetMongo to read those definition values and store them in a map accessed by DistributedMapCacheClientService, then use RouteOnContent to route the incoming flowfiles based on the absence/presence of the retrieved values.
If that doesn't work, you could instead route the query result from GetMongo to PutFile and then use ScanContent, which reads from a dictionary file on the file system and routes flowfiles based on the absence/presence of those keywords in the content.
Finally, if all else fails, you can use ExecuteScript to combine those steps into a single processor and route to matched/unmatched relationships. It processes Groovy code easily, so you can directly invoke your existing Java class if necessary.