New to NiFi!
I've split a flowfile into a single line of text using splitJSON processor.
The NiFi flowfile contents are as follows:
abcdefg
I'd like to be able to take the text in the flowfile and either add it to a url to make a subsequent call using InvokeHTTP or add the contents of the flowfile as an attribute so I can make the subsequent call using InvokeHTTP like so
http://localhost/${my.newly.added.attribute}
How do i do this?
Any help would be appreciated!
Thanks in advance!
ExtractText will allow you to find sections of content and place in an attribute on the FlowFile. For your example, you could capture the entirety of the content and assign to an attribute my.newly.added.attribute. InvokeHTTP would then access it using Expression Language 2 as in your example.
Related
I need some help in UpdateAttribute processor:
I have a CSV file which contains hostnames. I need to separate each hostname in the FlowFile and pass it as a variable to the REST API.
My REST API part is working fine when passing data manually. However, I didn't get how to pass a variable value as hostname in it.
Sharing sample file:
SRHAPP001,SRHWEBAPP002,SRHDB006,SRHUATAPP4,ARHUATDB98
I don't quite understand your goal, but I assume that you try to pass the hostname to your REST API module by using FlowFile variables.
You can achieve this by using the ExtractText-Processor. You simply use RegEx for separating your hostnames from the CSV file.
For more information, see
https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.12.1/org.apache.nifi.processors.standard.ExtractText/
How can I extract a substring from a flowfile data in Nifi?
If needed, you can split incoming FlowFiles on every hostname by using SplitContent-Processor
while using RouteOnAttribute nifi processor , i have input of json data
[{"dev":"xyz","detail":"abc"}] which i got from convertRecord processor
Routing Strategy :Route to Property name
ifmatch: ${dev:equals( "xyz" )}
I tried ${dev:matches( "xyz")} in both single quotes and double quotes still am not getting flowfile redirecting towards "ifmatch" . its redirecting to unmatched
is there anyway to resolve this i tried many other option
The flowfile content is different from attributes. Content is arbitrary -- could be empty, text, KB of XML, GB of video or binary. Each flowfile also has attributes which are key/value pairs of Strings kept in memory.
If you want to route on this piece of data, you have multiple options:
Use RouteOnText or RouteOnContent to use the actual flowfile content directly.
Extract it to an attribute using EvaluateJsonPath and then route on that attribute.
The Apache NiFi User Guide and In-Depth provide more information around this distinction.
Before posting this question about Apache NiFi InvokeHTTP I have gone through all other questions and their answersbut I am still unsure the best flow I should have. My situation is as below:
1) From Apache Kakfa, I get raw metadata.
2) Using EvaluateJSONPath I get attribute I want.
3) Using RouteOnAttribute I created 3 routes based on the attribute value I got from step-2 above.
4) Now based on the attribute value I want to decide whether I should go for GET or for POST or for Delete.
5) My question is where/how to set POST message? GET message? Delete Message body?
6) I am able to set the URL in configuration part provided by InvokeHTTP. But message body I don't know which is that property? or its in flow file using ReplaceText?
I read somewhere that before you divert your Restful POST HTTP request to InvokeHTTP you must have another processor before which changes the content of flow file.
Ref: Configuring HTTP POST request from Nifi
Please help. Thanks.
regards,
Yeshwant
Adding on to what Bryan had explained, POST will use the FlowFile content as the message body so if you have some other data which you want to wipe/transform into something and then sent as the message body, you can leverage the following processors :
ExtractText to read data from the existing FlowFile content
ReplaceText to erase the existing content of the FlowFile and replace it with different one
To set the headers for the REST calls, InvokeHTTP has the property Attributes to Send property which takes a regex which will scanned against the incoming FlowFiles' attributes and whichever attributes are matched are taken and sent as HTTP header.
To add new attribute to your existing FlowFile, you can use UpdateAttribute
For a POST, the body will be whatever is in the flow file content.
a GET and DELETE typically wouldn't have a body since the information would typically be provided in the URL or query params.
I am trying to insert the whole content of a row of an XML file into a JSON attribute (I am a newbie).
I am doing it this way (tell me if there is an easier way, it's good to now):
I have configured Extract text this way:
And to finish, I configure the Replace Text, giving a JSON format:
But he result appears to be wrong (doesn't work like a normal JSON file, for example if I a try to do a httpPost):
How can I fix this problem?
cheers
If you are concern regards to new lines and json key/values then use NiFi expression language functions on the extracted attribute(data).
ReplaceText Configs:
Replacement value:
{"name" : "user1","time" : "${now()}","data" : "${data:replaceAll('\s',''):escapeJson()}"}
Use escapeJson and replaceAll function to replace all spaces,newlines with ''
Replacement Strategy as Always Replace
(or)
Another way of preparing json message is by using AttributesToJson processor.
if we are using this processor then we need to prepare attributes/values before AttributesToJson processor by using UpdateAttribute processor
Flow:
1.SplitXml
2.ExtractText //add data property to extract content to flowfile attribute
3.UpdateAttribute //add name property -> user1
add time property -> ${now()}
add data property -> ${data:replaceAll('\s',''):escapeJson()}}
4.AttributeToJson //Attributes List -> name,time,data
Destination -> flowfile content
include core attributes -> false
I need to perform an HTTP Post from NiFi, but I don't want/need the request to carry all of the FlowFile's content.
Is there some way to pass attributes of a FlowFile but not the full content?
If the request body of your Http Post is JSON, you can use the AttributesToJSON Processor which allows you to pick which attributes you want to include in the resulting JSON. You can then configure the processor so the resulting JSON overwrites the existing flowfile content.
Keep in mind that the resulting JSON will be flat so you may need to transform it to the expected format. For that, you can use the JoltTransformJSON Processor.
Below is an example of what your dataflow might look like. I hope this helps!