I want to create a simple flow. Firstly, I need to take accountIds from REST service and then use received values to create new HTTP request to get token, and then use this token to create some requests with Oauth2.
AccountIds flow:
From InvokeHttp I'll receive some ids in json (REST written in Java and returns List with Integers). 99% chance that there will be only one number. My response looks like: [40]. Now I need to replace square brackets and get this number (using SplitJson). This number I should put to the next getToken as one of the GET param (on a screenshot i hardcode this):
This will return a token. Token is a text/plain;charset=UTF-8. And then, i want to use InvokeHttp again, add atribute Authorization and add to this attribute Bearer + received token. I do not really understand how to use the received data from processors in the following processors. Can someone explain how to reach it with my flow?
The Rest Api provides you with payload body - in nifi terms a flowfile content, you need to parse that incoming content using an evaluatejson (if payload is json - in most cases) and store it in flow attributes.
Then this attributes will be used in the down stream processors.
Also to pass the Authorization to your InvokeHTTP you need to decalre it in the InvokeHTTP porcesor. The ${access_token} is from the upstream attributes extraction.
Related
I am using spring boot 2 and apache camel 2.24 to build an api gateway which exposes REST end points to receive JSON/XML request and do the following
Validate incoming JSON/XML
Convert incoming request to downstream expected format.
Push the request to a camel route which invokes the downstream REST end point and returns back the response.
Convert response to expected format and return back to client.
Currently I have route configured as below
from("direct::camel").process(preprocessor).process(httpClientProcessor).process(postprocessor);
httpClientProcessor - This does the main job of invoking downstream end point and return the response.
preprocessor - splits the request for configured requests, puts them in a list before passing to httpClientProcessor.
postprocessor - does the following based on content type
XML - Remove "xml" tag from individual responses and combine to form one single response under one root element
JSON - Combine json response under one parent array element.
There can be cases where the multiple requests need to be sent to the same end point or each to be sent to a unique end point. Currently I have that logic in httpClientProcessor. One issue with this approach is that I can invoke the downstream end points only one after another rather than in parallel (unless I add thread pool executor in httpClientProcessor)
I am new to apache camel and hence started with this basis route configuration. Based on going through the documentation, I came across camel components like split(), parallelProcessing(), multicast and aggregator components of camel but I am not sure how to plug these all together to achieve my requirement -
Split the incoming request using a pre configured delimiter to create multiple requests. The incoming request may or may not have multiple requests.
Based on endpoint url configuration, either post all split requests to same end point or each request to a unique endpoint.
Combine responses from all to form one master response (XML or JSON) as output from the route.
Please advise.
This sounds like you should have a look a the Camel Split EIP.
Especially
How to split a request into parts and re-aggregate the parts later
How to process each part in parallel
For the dynamic downstream endpoints, you can use a dynamic To (.toD(...)) or the Recipient List EIP. The latter can even send a message to multiple or no endpoint.
If you are in a middle-ware that both receives the context and maybe append some data to context to send it to the next interceptor, then which of the two methods i.e. metadata.FromOutgoingContext and metadata.FromIncomingContext shall be called?
If you are writing that middle-ware in the server, then you are receiving that metadata in the incoming request.
You should then use metadata.FromIncomingContext to get the metadata at that point.
The metadata in the "outgoing context" is the one generated by the client when sending an outgoing request to the server.
See here for examples of both:
https://github.com/grpc/grpc-go/blob/master/Documentation/grpc-metadata.md
I need a comprehensive list of what is valid and what is invalid when sending a bulk payload to ElasticSearch.
Bulk endpoint is a indexing endpoint. So at a highlevel you can only send indexing requests to that endpoint.
Since its bulk, a valid request is designed around multiple doc and how they are delimited. e.g. if you can not using a ES client then you will need to have payload formatted as ndjson (new delimited json) and last doc should end in a new line as well. Better to use a client since a client will do this all for you.
Apart from syntax of payload data you can target a index , a type etc in URL.
You can also send some other param like "wait_for_completion", "retry_on_conflict" etc. These are parameter which will control how each requests will behave.
Needless to say but best would be to read up the doc:
https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html
In our Java app we need to accept a (large) Grpc message, extract a field, and then based on the value of that field forward the message on to another server.
I'm trying to avoid the overhead of completely deserializing the message before passing it on.
One way to do this would be to send the field as a separate query or header parameter, but Grpc doesn't support them.
Another way would be to extract just the field of interest from the payload, but Protobuf doesn't support partial or selective deserialization.
How else can I do this?
One way you can do this is by doing it on the server side. When the server is about to send a response, it can extract the field and set it as part of the initial headers sent. You can do this by using a ServerInterceptor to extract the field that you want from the response and add it to the Metadata.
Aside from that, Protocol buffers currently require that you parse the message before accessing the internal fields.
I was wondering if here is a way to send multiple 'data' elements in a single Parse REST API push request. I know we can send the same message to multiple devices or channels as required. But i have a different message for each channel. So what i need is a way to call Parse REST API once with the JSON of different channels but each with a different data and alert message.
is is possible? or do i need to make such requests one by one.