I would like to set some properties before my InvokeHTTP processor in NiFi. In fact, I try to use an API and I would like to give HTTP headers with key API.
I can set all properties with a lot of UpdadeAttribute but there is two problems :
I can't start my work on a UpdadeAttribute
All my properties have to be set one time, no more
These properties are in assembly of two keys.
First, I merge the two keys in merged_keys
${api_key}:${api_secret_key}
Second, I encode merged_keys in encoded64_key.
${merged_key:base64Encode()}
Then, I write my HTTP header Authorization using encoded64_key
'Authorization': 'Basic ${encoded64_key}'
edit : I tried to set this configuration in UpdateAttribute but I know that isn't a good way so I tried to set it directly in my InvokeHTTP and nothing happen.
The part of building HTTP header works perfectly but I don't know how can I do for give this header to my InvokeHTTP one time.
Set a property on the InvokeHTTP processor with name Authorization and value Based ${encoded64_key}. This is the way to attach the header. See the documentatinos.
An HTTP client processor which can interact with a configurable HTTP Endpoint. The destination URL and HTTP Method are configurable. FlowFile attributes are converted to HTTP headers and the FlowFile contents are included as the body of the request (if the HTTP Method is PUT, POST or PATCH).
FYI with the contents, you should give the contents as a flowfile by using replaceText processor.
Related
I have a endpoint would return response as follow.
{
"result": [
{},
....
{}]
}
I am trying to use invokeHTTP and enable “Put Response Body In Attribute” to keep origin flowfile and response from api.
but it seems add a attribute named $.result as follow
Is there any way to set a proper name for result attribute ? Thanks.
You try to extract results using a JSON path. However this is not possible from within InvokeHttp. You may want to use EvaluateJsonPath processor.
Documentation for Put Response Body In Attribute:
If set, the response body received back will be put into an attribute
of the original FlowFile instead of a separate FlowFile. The attribute
key to put to is determined by evaluating value of this property.
To keep the original flow file after calling InvokeHttp, you can move it to an attribute (if the content is small enough) just before calling InvokeHttp or you can use MergeContent with original flow file and response flow file from InvokeHttp.
Before posting this question about Apache NiFi InvokeHTTP I have gone through all other questions and their answersbut I am still unsure the best flow I should have. My situation is as below:
1) From Apache Kakfa, I get raw metadata.
2) Using EvaluateJSONPath I get attribute I want.
3) Using RouteOnAttribute I created 3 routes based on the attribute value I got from step-2 above.
4) Now based on the attribute value I want to decide whether I should go for GET or for POST or for Delete.
5) My question is where/how to set POST message? GET message? Delete Message body?
6) I am able to set the URL in configuration part provided by InvokeHTTP. But message body I don't know which is that property? or its in flow file using ReplaceText?
I read somewhere that before you divert your Restful POST HTTP request to InvokeHTTP you must have another processor before which changes the content of flow file.
Ref: Configuring HTTP POST request from Nifi
Please help. Thanks.
regards,
Yeshwant
Adding on to what Bryan had explained, POST will use the FlowFile content as the message body so if you have some other data which you want to wipe/transform into something and then sent as the message body, you can leverage the following processors :
ExtractText to read data from the existing FlowFile content
ReplaceText to erase the existing content of the FlowFile and replace it with different one
To set the headers for the REST calls, InvokeHTTP has the property Attributes to Send property which takes a regex which will scanned against the incoming FlowFiles' attributes and whichever attributes are matched are taken and sent as HTTP header.
To add new attribute to your existing FlowFile, you can use UpdateAttribute
For a POST, the body will be whatever is in the flow file content.
a GET and DELETE typically wouldn't have a body since the information would typically be provided in the URL or query params.
I'm trying to receive a json payload on a hapijs server, that json may not be valid, since is some custom format that i need to manually proxy the request to an elasticsearch cluster, cannot use the proxy option for hapijs because i need to do multiple requests to different clusters, for that i use nodejs http library.
Elasticsearch doesn't receive a valid json when doing bulk actions, it receives new lines instead of commas, to separate json objects:
{"index":[".kibana-devnull"],"ignore_unavailable":true}
{"size":500,"sort":[]}
Hapijs tries to validate the json payload when it gets application/json header in the request and it responds "Invalid request payload JSON format", as i cannot remove that header i need to look for another method to allow that invalid json in the route, even if the header is present.
I would look at the docs, in particular http://hapijs.com/api#route-configuration. If you set payload.output.parse to false you will receive the raw buffer inside handler which can then be parsed by yourself as opposed to by the framework.
Jmeter: I tried to Parameterize the headers(pass about 5 dynamic headers) in "HTTP header manager" using "CSV data set config". But i could not, can anybody help me??
Just use the CSV data setup variables in the HTTP header manager.
Can you be more specific on where you're stuck?
YouTube tutorial on using CSV data with JMeter:
http://www.youtube.com/watch?v=aEJNc3TW-g8
Blog that explains exactly what to do:
http://community.blazemeter.com/knowledgebase/articles/65138-using-csv-data-set-config
If you need to add 5 dynamic headers one by one, i.e. one header per request just change the value on each consecutive request - it's pretty easy doable, just place your CSV Data Set Config as a child of required request, the same for HTTP Header Manager.
If you need to add all 5 headers before any testing starts - you'll need to consider do it in a separate thread group or use combination of Loop Controller with Once Only Controller
If none above is applicable to your test scenario try updating it and provide as much information as you can.
I know OData supports responding in JSON format when it's given the appropriate Accept header:
Accept: application/json
Some articles say you'll need to specify odata verbosity otherwise you'll get the default xml format, but I have not seen this to be actually true. But let me mention it anyway:
Accept: application/json;odata=verbose
But (how) can I make my request using JSON instead of a querystring?
OData doesn't provide a way to specify the query in a request body, it only supports the query in the URL. So the answer is that there's no way to do that in JSON. Note that it applies to GET requests. Modification requests (POST/PUT/...) do accept JSON as the payload (typically representing an entity for example), in which case simply specify the content type of the request in its Content-Type header.
There are java script libraries which let you build the query string using more structured code (as compared to just strings). For example the datajs http://datajs.codeplex.com/.