I am trying to build an automated flow to export profiles from a SaaS cloud platform where I have all the REST APIs. Here is what I am doing
Login to login API to get token (I have been able to successfully do that using InvokeHttp processor). I get the token in the response JSON as follows
{ "access_token" : "<some_token_value>", "links" : [ { "rel" : "self", "href" : "https://ccadmin-xxxx.oracleoutsourcing.com/ccadmin/v1/login" } ], "token_type" : "bearer", "expires_in" : 300 }
I extract the token from the JSON using EvaluateJSONPath and add dynamic property "token" to map to access_token of the JSON response I got.
The output of evaluateJsonPath is an attribute "token" which has the value of access_token from step 1.
Now i am preparing a BODY for the next POST call, so i use UpdateAttribute processor to define my attributes.
Next i convert the attributes that i define in the previous step to JSON using AttributesToJSON processor. This adds the following to the attributes path in its output
JSONAttributes
{"mode":"standalone","filename":"profiles.json","format":"json","id":"Profiles"}
Also the token from step 1 still flows as an attribute
Next i pass all this to my 2nd InvokeHttp processor to call the bulk export.
Now, I want to pass this token as the Header and also pass the JSON body to the Invoke HTTP processor. So i just make the following change. Add a dynamic attribute to the InvokeHTTP called "Authorization" and pass value as Bearer ${token} and i believe the body will automatically be taken from the attributes from previous processor.
But the call fails. Is there anything i am missing. I have tried running this through POSTMAN and everything runs fine there.
Nifi Flow Diagram
Input to 2nd InvokeHttp Processor
Second InvokeHttp POST processor
I think i got the issue. i see all the things are flowing as attributes in the flow. The body needs to flow as content. so i cut short the UpdateAttribute and AttributeToJSON and instead put a ReplaceText processor and put the JSON body there. And i am able to make a successful call now.
Related
we have certain token mechanism recently implemented into our project. What it does is, say it generates 50 tokens while triggering the corresponding API. Now we have to capture this tokens and pass it sequentially one by one to the next 50 API’s request body. One way is by writing multiple json path extractors I have to capture this values individually and pass it to all the requests. But readability is not proper this way. Is there any other way to capture this tokens and pass it to all the next requests one each. Say capturing those into a file and passing it sequentially or something.
Sample Response from where tokens generated:
{
"data" : {
"requestTokens" : [ "9FDE794DD00E4A09122343BDCAF214E9", "616C5DFFC1234516A925824AEA6EFE9F", "7A8C507EC1DF4AD88E0912345E1DB409", "763C32CF67899946B6BC946949BD1344", "3C143F2FC25E495012345500E0F343DF", "3FD78335C763420B1234574061D9417F", "C43C368A1E612345AB17D2BA2693BEAF", "56E8FA9036D3486123451DE3237004DC", "5867B8E399FB4E12345626337D0E143C", "D06B30BDEAFC4A7D8618BF67712345DB", "F795258D390D4812345EB62C83BEFC58", "F0829D861234560392DE432E121B3CD4", "C8B9D5E6BE6A465FB91B0123459FBA9A", "4169D93D97204123457BA5A98C914D27", "784809E5BCBF4F123459D7D848AD67CE", "D0BFDCDC13994C0123455B2B110C35F9", "4F59619BBEDE4FE812345EA14C5E785", "E3942AE182214605BE91234595D95C18", "2005C506659C425EAD2022446123459B", "FE716E2A13A74C759C12345AED5AD54C" ]
}
}
Add JSON JMESPath Extractor as a child of the request which returns this JSON
Configure it as follows:
Add ForEach Controller after the request which returns this JSON
Configure it as follows:
Put your "next request" as a child of the ForEach Controller
Refer the token as ${token} in this request
That's it, ForEach Controller will loop as many times as there are tokens in the first response and each iteration the ${token} JMeter Variable will be updated with the new value
Demo:
I would like to set some properties before my InvokeHTTP processor in NiFi. In fact, I try to use an API and I would like to give HTTP headers with key API.
I can set all properties with a lot of UpdadeAttribute but there is two problems :
I can't start my work on a UpdadeAttribute
All my properties have to be set one time, no more
These properties are in assembly of two keys.
First, I merge the two keys in merged_keys
${api_key}:${api_secret_key}
Second, I encode merged_keys in encoded64_key.
${merged_key:base64Encode()}
Then, I write my HTTP header Authorization using encoded64_key
'Authorization': 'Basic ${encoded64_key}'
edit : I tried to set this configuration in UpdateAttribute but I know that isn't a good way so I tried to set it directly in my InvokeHTTP and nothing happen.
The part of building HTTP header works perfectly but I don't know how can I do for give this header to my InvokeHTTP one time.
Set a property on the InvokeHTTP processor with name Authorization and value Based ${encoded64_key}. This is the way to attach the header. See the documentatinos.
An HTTP client processor which can interact with a configurable HTTP Endpoint. The destination URL and HTTP Method are configurable. FlowFile attributes are converted to HTTP headers and the FlowFile contents are included as the body of the request (if the HTTP Method is PUT, POST or PATCH).
FYI with the contents, you should give the contents as a flowfile by using replaceText processor.
I have a endpoint would return response as follow.
{
"result": [
{},
....
{}]
}
I am trying to use invokeHTTP and enable “Put Response Body In Attribute” to keep origin flowfile and response from api.
but it seems add a attribute named $.result as follow
Is there any way to set a proper name for result attribute ? Thanks.
You try to extract results using a JSON path. However this is not possible from within InvokeHttp. You may want to use EvaluateJsonPath processor.
Documentation for Put Response Body In Attribute:
If set, the response body received back will be put into an attribute
of the original FlowFile instead of a separate FlowFile. The attribute
key to put to is determined by evaluating value of this property.
To keep the original flow file after calling InvokeHttp, you can move it to an attribute (if the content is small enough) just before calling InvokeHttp or you can use MergeContent with original flow file and response flow file from InvokeHttp.
Before posting this question about Apache NiFi InvokeHTTP I have gone through all other questions and their answersbut I am still unsure the best flow I should have. My situation is as below:
1) From Apache Kakfa, I get raw metadata.
2) Using EvaluateJSONPath I get attribute I want.
3) Using RouteOnAttribute I created 3 routes based on the attribute value I got from step-2 above.
4) Now based on the attribute value I want to decide whether I should go for GET or for POST or for Delete.
5) My question is where/how to set POST message? GET message? Delete Message body?
6) I am able to set the URL in configuration part provided by InvokeHTTP. But message body I don't know which is that property? or its in flow file using ReplaceText?
I read somewhere that before you divert your Restful POST HTTP request to InvokeHTTP you must have another processor before which changes the content of flow file.
Ref: Configuring HTTP POST request from Nifi
Please help. Thanks.
regards,
Yeshwant
Adding on to what Bryan had explained, POST will use the FlowFile content as the message body so if you have some other data which you want to wipe/transform into something and then sent as the message body, you can leverage the following processors :
ExtractText to read data from the existing FlowFile content
ReplaceText to erase the existing content of the FlowFile and replace it with different one
To set the headers for the REST calls, InvokeHTTP has the property Attributes to Send property which takes a regex which will scanned against the incoming FlowFiles' attributes and whichever attributes are matched are taken and sent as HTTP header.
To add new attribute to your existing FlowFile, you can use UpdateAttribute
For a POST, the body will be whatever is in the flow file content.
a GET and DELETE typically wouldn't have a body since the information would typically be provided in the URL or query params.
The following flow takes a username/password combination and authenticates against an API via InvokeHTTP:
The result of InvokeHTTP is an authentication token:
I want to use this token in any preceding API calls. What flow do I need to provide the auth token as an authorization header for InvokeHTTP? The header has to look like this:
Authorization: Token ...
You can use an EvaluateJsonPath processor to extract a JSON value using the JsonPath expression $.token and Destination flowfile-attribute, which will place it in an attribute. So using a dynamic property named Authorization with the value $.token will result in InvokeHTTP sending a header with the value Authorization: abcdef..... In order to format the header correctly, you'll want to use an UpdateAttribute processor between EvaluateJsonPath and InvokeHTTP to update the attribute value using the Expression Language expression Token ${Authorization} to prepend the literal string.