A fasthttp based server is up to 10 times faster than net/http.
Which implementation details make fasthttp so much faster? Moreover, how does it manage incoming requests better than net/http?
The article "http implementation fasthttp in golang" from husobee mentions:
Well, this is a much better implementation for several reasons:
The worker pool model is a zero allocation model, as the workers are already initialized and are ready to serve, whereas in the stdlib implementation the go c.serve() has to allocate memory for the goroutine.
The worker pool model is easier to tune, as you can increase/decrease the buffer size of the number of work units you are able to accept, versus the fire and and forget model in the stdlib
The worker pool model allows for handlers to be more connected with the server through channel communications, if the server needs to shutdown for example, it would be able to more easily communicate with the workers than in the stdlib implementation
The handler function definition signature is better, as it takes in only a context which includes both the request and writer needed by the handler. this is HUGELY better than the standard library, as all you get from the stdlib is a request and response writer… The work in go1.7 to include context within the request is pretty much a hack to give people what they really want (context) without breaking anyone.
Overall it is just better to write a server with a worker pool model for serving requests, as opposed to just spawning a “thread” per request, with no way of throttling out of the box.
Related
TL;DR: which is more pattern? Using Mutiny + Imperative resteasy, or just use Reactive resteasy?
My understanding is Mutiny allows for me to pass Quarkus a longer running action, and have it handle the specifics of how that code gets run in the context. Does using Reactive provide equal or more benefit than Mutiny+imperative? If from a functional point of view from a thread handling perspective it's equal or better, then Reactive would be great as it requires less code to maintain (Creating Unis, etc). However, if passing a Uni back is significantly better, then it might make sense to use that.
https://quarkus.io/guides/getting-started-reactive#imperative-vs-reactive-a-question-of-threads
Mutiny + imperative:
#GET
#Path("/getState")
#Produces(MediaType.APPLICATION_JSON)
public Uni<State> getState() throws InterruptedException {
return this.serialService.getStateUni();
}
Reactive:
#GET
#Path("/getState")
#Produces(MediaType.APPLICATION_JSON)
public State getState() throws InterruptedException {
return this.serialService.getState();
}
As always, it depends.
First, I recommend you to read https://quarkus.io/blog/resteasy-reactive-smart-dispatch/, which explains the difference between the two approaches.
It's not about longer action (async != longer); it's about dispatching strategies.
When using RESTEasy Reactive, it uses the I/O thread (event loop) to process the request and switches to a worker thread only if the signature of the endpoint requires it. Using the I/O thread allows better concurrency (as you do not use worker threads), reduces memory usage (because you do not need to create the worker thread), and also tends to make the response time lower (as you save a few context switches).
Quarkus detects if your method can be called on the I/O thread or not. The heuristics are based on the signature of the methods (including annotations). To reuse the example from the question:
a method returning a State is considered blocking and so will be called on a worker thread
a method returning a Uni<State> is considered as non-blocking and so will be called on the I/O thread
a method returning a State but explicitly annotated with #NonBlocking is considered as non-blocking and so will be called on the I/O thread
So, the question is, which dispatching strategy should you use?
It really depends on your application and context. If you do not expect many concurrent requests (it's hard to give a general threshold, but it's often between 200 and 500 req/sec), it is perfectly fine to use a blocking/imperative approach. If your application acts as an API Gateway with potential peaks of requests, non-blocking will provide better results.
Remember that even if you choose the imperative/blocking approach, RESTEasy Reactive provides many benefits. As most of the heavy-lifting request/response processing is done on the I/O thread, you get faster and use less memory for... free.
I have an array of objects that i need to send to an endpoint. I am currently looping through the array and sending the requests one by one. The issue is that i now have over 35,000 requests to be made, and i need to update the database with the response.In my limited knowledge of springboot , i am not aware of any method i can use to send the 35,000 requests at once (without looping through one by one).
Is the best method to use still employing looping but utilize asynchronous calls, or is there a method that i can use to send the 35,000 http requests at once?..i just need a pointer because i am not aware how threads can be used, since this is already an array and each element needs to be sent.
Thank you
Well, first off 35,000 at a time of, well, anything, is a bad idea.
However, if you look in to the Java ExecutorService, this gives you the ability to fill a queue with tasks, and then each task will be performed by a thread taken from a thread pool. As the threads complete, the service pulls another request from the queue and handles that. So, you simply provide a Runnable that performs your web requests, create an Adequately Sized Thread Pool (which is basically sized through experimentation to give the best throughput), and then let the threads crunch away on the queue of tasks.
You will need a queue large enough to absorb all of your tasks, or you can look at something like the NotifyingBlockingThreadPoolExecutor. This will allow you to just gorge a queue and block when the queue gets to full, until all of your tasks are complete.
Addenda:
I don't know enough about Spring Boot to comment about whether a "batch job" would do what you want or not.
However, on that note, an alternative to creating 35,000 individual entries for the ExecutorService, you could, indeed, send a subset. For example 3,500 entries representing 10 items each, or 350 with 100 each. The idea there is to leverage any potential gains from reusing HTTP connections and what not, so there's less stand up and tear down for each request. Standing up 350 connections if far cheaper than standing up 35,000.
Most example apps I come across for receiving data are using async calls. For instance, c++ examples use boost asio services to bind message handlers to callbacks. But what about an app that only needs to listen to data from a single socket and process the messages in order? Would it be faster to have a loop that polls/recv's from the socket and calls the handler without using a callback (assume main and logging threads are separate)? Or is there no performance difference (assume messages are coming in as fast as the network card and kernel can handle them)?
There are many intricacies I don't know such as the impact of callbacks to performance due to things like branch prediction. Or if there will be a performance penalty of the callbacks call a different thread to do the processing. Curious to hear some thoughts, experiences, dialog on this subject to save myself from attempting both implementations to discover the answer.
Just starting out with Go and hoping to create a simple Web API. I'm looking into using Gorilla mux (http://www.gorillatoolkit.org/pkg/mux) to handle web requests.
I'm not sure how to best use Go's concurrency options to handle the requests. Did I read somewhere that the main function is actually a goroutine or should I dispatch each request to a goroutine as they are received? Apologies if I'm "way off".
Assuming you're using the Go's http.ListenAndServe to serve your http requests, the documentation clearly states that each incoming connection is handled by a separate goroutine for you. http://golang.org/pkg/net/http/#Server.Serve
You would usually call ListenAndServe from your main function.
Gorilla mux is simply a package for more flexible routing of requests to your handlers than the http.DefaultServeMux. It doesn't actually handle the incoming connection or request just simply relays it to your handler.
I highly suggest you read a bit of the documentation, specifically this guide https://golang.org/doc/articles/wiki/#tmp_3 on writing web applications.
I'm providing an answer even though I voted to close for being too broad.
Anyway, none of that is really necessary. You're over thinking it. If you haven't read this it looks like a decent tutorial; http://thenewstack.io/make-a-restful-json-api-go/
You can really just set up routes like you would with most typical rest frameworks and let the webserver/framework worry about concurrency at the request handling level. You would only employ goroutines to generate the response of a request, say if you needed to aggregate data from 10 files that are all in a folder. Contrived example, but this is where you would spin off 1 goroutine per file, aggregate all the information by reading off a channel in a non-blocking select and then return the result. You can expect all points of entry to your code are called in an asynchronous, non-blocking fashion if that makes sense...
Is MPI_Bsend_init/MPI_Start best asynchronous buffered communication. Can you guys think of better way to communicate data between processors. Pseudo-Code for N Processing nodes
MPI_Recv(request[i]) -- Recv data
for(i=0;i<N;i++) MPI_Bsend_init(request[i]) -- Setup request
MPI_Start(request[i]) -- Send data
Bsend is the wrong function here from a performance standpoint. There is little to no advantage of Bsend, as the eager protocol used by essentially all implementations today, is buffered on the receiver side automatically for small messages, which is where Bsend would be viable.
In any case, persistent send - as you are using - is already nonblocking, so there is no such thing as Isent_init. See e.g. http://www.mpi-forum.org/docs/mpi-1.1/mpi-11-html/node51.html:
The call is local, with similar semantics to the nonblocking
communication operations described in section Nonblocking
communication . That is, a call to MPI_START with a request created by
MPI_SEND_INIT starts a communication in the same manner as a call to
MPI_ISEND...
And my colleagues have surprised me with confirmation that persistent Send-Recv does provide efficiency gains on modern InfiniBand clusters. I can only assume this is because IB page registration is done up-front.