I am currently using the ListenHTTP processor to accept flow files from a different NiFi instance. This works fine but for some reason GET requests do not work. Does ListenHTTP only allow POST requests ?
This is the error I receive:
HTTP method GET is not supported by this URL
P. S I am aware of a more advanced HandleHTTPRequest processor.
Yes, ListenHTTP only accepts POST and HEAD requests. GET, PUT, and DELETE are not accepted by the processor and will return a 405 HTTP Status Code. The documentation of the processor could be improved to document this.
You are correct that to handle GET requests, you should use the HandleHTTPRequest processor.
However, if your use case is transmitting flowfiles between two NiFi instances, you will get much better behavior and performance by using the Site to Site capability. This can be routed over HTTP(S) or raw sockets, provides security, integrity, load balancing, and many additional features.
Related
I am wondering how Quarkus is handling simultaneous requests to for example a REST API with json-rest.
Example:
REST API is called by lots of clients simultaneously
REST API call calls other APIs
REST API processes the response of the other called APIs and returns the processed response
Questions:
Are the requests queued and processed one by one?
Are requests rejected if the API is busy?
Is handling parallelism offloaded to the infrastructure using tools like Istio?
Can someone please point me to some documentation about that or give an explanation? Thank you.
Quarkus uses Vert.x under the hood which implements an event loop. This means that it can handle thousands of the requests because its threads are not blocked.
You may read more about it in the Vert.x's documentation: https://vertx.io/docs/vertx-core/java/
When I connect Retry from invokeHTTP to HandleHttpRespose (or any other process) in NiFi, task count will become high (about 1,000,000 tasks/time) and response slow, what must be the reason?
I can't explain the task count on HandleHttpResponse, that usually would only happen when the processor has the # TriggerWhenEmpty annotation, which means its running all the time when no flow files are available and just doing nothing.
In general, using HandleHttpResponse with InvokeHttp is not going to work, it was made to work with HandleHttpRequest which places accepts a request, creates an entry in the HTTP Context Map, allows the flow to proceed, and then can respond to the original request with HandleHttpRequest.
InvokeHttp is a client making a connection to a server, where as HandleHttpRequest is a server that needs to send a response to a client using HandleHttpResponse. InvokeHttp does not put anything into the HTTP Context Map so there is nothing for HandleHttpRequest to do in that case.
You would typically connect the "retry" relationship of InvokeHttp in a self-loop back to InvokeHttp so it can keep retrying.
I have following two scenario and for each one I need recommendation as to which NiFi processor to use:
I have Restful web services running outside NiFi. NiFi would like to get/post/delete/update some data by calling specific restful API. Once the Restful API receives request from NiFi it sends back the response to NiFi. Which NiFi processor to use here?
In 2nd scenario, I have an application running outside NiFi. This application has its own GUI. The user need some information so he want to send request to NiFi. In NiFi, is there any processor which accepts request from application, process the request, and sends response back?
I actually read all the question with getHTTP and invokeHTTP.
I have initially tried with invokeHTTP processor. I tried both get and post call using invokeHTTP. But I don't see any response from Restful API running outside NiFi.
I did not try getHTTP.
I am using NiFi. NiFi do not have code.
I expect NiFi should be able to call Restful API running outside. I expect NiFi should accept request coming from application running outside and process that request.
Yep, NiFi comes bundled with processors that satisfy both of your requirements.
For scenario #1, you can use either a combination of GetHTTP/PostHTTP which as their name implies are HTTP clients that make GET and POST calls respectively. However, later the community came up with InvokeHTTP that offers more features like support for NiFi Expression Language, support for incoming flowfiles, etc.,
For scenario #2, you can either use ListenHTTP or the combination of HandleHttpRequest/HandleHttpResponse. The later literally offers you have a more robust web-service implementation while the former is a simple web-hook kind. I haven't worked much with ListenHTTP so probably can't comment more on that.
Having said that, for your second scenario, if your objective is to consume NiFi statistics, you can directly hit NiFi's rest api, rather than having a separate NiFi flow with web service capability.
Useful Links
https://pierrevillard.com/2016/03/13/get-data-from-dropbox-using-apache-nifi/
https://dzone.com/articles/using-websockets-with-apache-nifi
https://ddewaele.github.io/http-communication-with-apache-nifi/
I would like to just HTTP POST events into a spout. Do I need to set up a web server myself, or would that be redundant? All of the tutorials that I have seen so far assume that an application will be fetching (or even just generating) the data itself and passing it to emit-spout!.
Storm used a pull based model in Spouts.nextTuple(). Thus, it might be best to have a buffer in between -- a WebServer takes HTTP POST requests and writes into that buffer. A Spout can pull the date from the buffer.
I need to test an application that processes SIP requests. For now, I want to test the performance of the application, so I need a way to generate a big number of SIP requests.
I know there are tools for this (like SipP), but I don't know what is the maximum number of requests that a single computer can really send in a particular time interval.
I never done this type of test, i need help.
Thanks
Well sipp can generate requests pretty quickly and if you're testing call set up and tear down, i.e. INVITE requests an d associated transaction processing, it's almost certainly the tool for the job.
If you're not concerned about SIP transaction processing and instead just want to bombard your server with SIP requests you could just whip up a console application with a UDP socket and send dummy requests by using a template request and modifying the following:
The branchid parameter on the Via header,
The tag parameter on the From header,
The Call-ID header.
Since your app will only be doing a few string search and replaces and a UDP send it will be able to generate requests probably a 100 to 1000 times faster than a server on the same hardware, that needs to parse and understand the requests, will be able to process them.