I'm having a bit of trouble understanding the "lifespan" of context.
I'm receiving RPC requests, and storing the request ID using context.WithValue so that it can be used for logging. There is no deadline or timeout for context in my service.
One of the methods called by RPC pulls some data from the database, calls a goroutine to do some processing (not relevant to the client), sends a response, and the method returns. The goroutine could run for ~15 seconds after the method returns.
I log the request ID in the goroutine a few times, and so far it works fine, but is there a situation where context could be garbage collected, and unavailable when I try to use it? Or will go know to keep context around until my goroutine has completed?
This leads me to another question - I'm not using context.WithCancel, does this mean it will remain in memory indefinitely? I'd imagine after a while this could cause some performance issues.
A context is a glorified done channel. Depending on where the context comes from, determines how it should be reclaimed:
If you create a context (context.WithCancel etc.) ensure you reclaim it when the sub-task it represents completes (defer cancelfn() etc. ensure this happens on API return).
If you are using a context from an external source (e.g.a gRPC request) - it's the responsibility of the API framework to close out the context.
go will reclaim any memory during GC as long as there are no active references (function closures etc.)
Related
We started a new project with Quarkus and Mutiny, and created a bunch of endpoints with Quarkus #Funq, everything has been working fine so far. Now we want to process something very time-consuming in one of the endpoints, and what we are expecting is, once user clicks a button to send the http request from frontend and hits this specific endpoint, we are going to return 202 Accepted immediately, leaving the time-consuming operation processing in another thread from backend, then send notification email accordingly to user once it completes.
I understand this can be done with #Async or CompletableFuture, but now we want to do this with Mutiny. Based on how I read Mutiny documentation here https://smallrye.io/smallrye-mutiny/guides/imperative-to-reactive, runSubscriptionOn will avoid blocking the caller thread by running the time-consuming method on another thread, and my testing showed the time-consuming codes did get executed on a different thread. However, the http request does not return immediately, it is still pending until the time-consuming method finishes executing (as I observe in the browser's developer tool). Did I misunderstand how runSubscriptionOn works? How do I implement this feature with Mutiny?
My #Funq endpoint looks like this
#Inject
MyService myService;
#Funq("api/report")
public Uni<String> sendReport(MyRequest request) {
ExecutorService executor = Executors.newFixedThreadPool(10, r -> new Thread(r, "CUSTOM_THREAD"));
return Uni.createFrom()
.item(() -> myService.timeConsumingMethod(request))
.runSubscriptionOn(executor);
}
Edit: I found the solution using Uni based on #Ladicek's answer. After digging deeper into Quarkus and Uni I have a follow-up question:
Currently most of our blocking methods are not returning Uni on Service level, instead we create Uni object from what they return (i.e. object or list), and return the Uni on Controller level in their endpoints like this
return Uni.createFrom().item(() -> myService.myIOBlockingMethod(request)).
As #Ladicek explained, I do not have to use .runSubscriptionOn explicitly as the IO blocking method will automatically run on a worker thread (as my method on Service level does not return Uni). Is there any downside for this? My understanding is, this will lead to longer response time because it has to jump between the I/O thread and worker thread, am I correct?
What is the best practice for this? Should I always return Uni for those blocking methods on Service level so that they can run on the I/O threads as well? If so I guess I will always need to call .runSubscriptionOn to run it on a different worker thread so that the I/O thread is not blocked, correct?
By returning a Uni, you're basically saying that the response is complete when the Uni completes. What you want is to run the action on a thread pool and return a complete response (Uni or not, that doesn't matter).
By the way, you're creating an extra thread pool in the method, for each request, and don't shut it down. That's wrong. You want to create one thread pool for all requests (e.g. in a #PostConstruct method) and ideally also shut it down when the application ends (in a #PreDestroy method).
I don't understand part of the latest Windows threadpool API. I need help with that.
From the documentation, the recipe to use it for I/O (in my case, for SOCKET) can be summarized as follows:
Call CreateThreadpoolIo.
Call StartThreadpoolIo. You can find this warning there:
You must call this function before initiating each asynchronous I/O operation on the file handle bound to the I/O completion object. Failure to do so will cause the thread pool to ignore an I/O operation when it completes and will cause memory corruption.
Call the operation on the file handle (e.g., WSARecvFrom). If it fails, call CancelThreadpoolIo. Otherwise, process the result when it is available. WSARecvFrom, when used asynchronously, asks for a WSAOVERLAPPED (that you have to create beforehand) but not for any information that links it to the previous call to StartThreadpoolIo. CancelThreadpoolIo only asks for the PTP_IO, but not for any additional information to derive a specific asynchronous operation.
Repeat steps 2 and 3.
Call CloseThreadpoolIo to finish. You can find this warning there:
It may be necessary to cancel threadpool I/O notifications to prevent memory leaks. For more information, see CancelThreadpoolIo.
I usually need it for UDP, so I strive to have several reception operations queued (asynchronous WSARecvFrom operations started) at any given time. That way I don't have to rush to start another reception operation at the beginning of the callback function nor synchronize access to the reception buffers (I can have a pool of them, each one able to contain a datagram, and reissue the reception operation when I finish processing each message; in the interim, other queued operations will keep the receiver busy). Datagrams are independent and self contained. I'm aware that this approach may not be valid for TCP.
StartThreadpoolIo/CancelThreadpoolIo seem to me the source of the problem: StartThreadpoolIo and WSARecvFrom are not directly bound (they don't share any arguments). So:
How can the framework know which operation to cancel when you call CancelThreadpoolIo? How does it cancel just the operation that failed and not any of the pending ones?
You can say, "don't call StartThreadpoolIo concurrently". I can live without several concurrent WSARecvFrom's, but I can't live without concurrent WSARecvFrom and WSASendTo. So I think being unable to have several asynchronous operations at the same time can't be the way the API was designed.
You can say, "call StartThreadpoolIo only once, that will suffice to register the callback; it is an on/off process". But the documentation says:
You must call this function before initiating each asynchronous I/O operation on the file handle...
You can say, "it cancels the operation started by the same thread that just called StartThreadpoolIo". But then the advice of calling CancelThreadpoolIo in the context of calling CloseThreadpoolIo doesn't make sense (I will call CloseThreadpoolIo from the thread that triggers stopping, which will be completely independent from the threads issuing the asynchronous operations; and a single call to CancelThreadpoolIo may not be enough to cancel several operations). Being unable to trigger cancellation from a different thread is a serious limitation, anyway. I'm aware of the existence of CreateThreadpoolCleanupGroup, but my question is more fundamental. I want to understand how this API can be fundamentally right and useful.
You can say "call CreateThreadpoolIo several times, so that you have independent PTP_IO's to work with". It doesn't work. When I call CreateThreadpoolIo a second time, nullptr is returned.
Am I wrong, or is this API awkward? Normally, other asynchronous APIs work with one of these patterns:
Create an operation and receive a handle => call methods passing the handle.
Create a reusable handle => call methods (including starting operations) passing the handle.
The latest Windows threadpool API, in which the handle seems to be implicit, or there are several handles for the same operation (TP_IO, WSAOVERLAPPED, StartThreadpoolIo) and they aren't all explicitly linked together, uses neither of them.
Thank you very much for your help.
How can the framework know which operation to cancel when you call CancelThreadpoolIo? How does it cancel just the operation that failed
and not any of the pending ones?
CancelThreadpoolIo() doesn't cancel IO. It is reciprocal to StartThreadpoolIo(). StartThreadpoolIo() prepares threadpool to accept a completion. If threadpool doesn't expect a completion, it won't wait for it, thus you may miss it. If threadpool expects a completion but completion doesn't happen, threadpool may waste resources.
CancelThreadpoolIo() undoes whatever StartThreadpoolIo() did.
Is it more idiomatic to have an async api, with a blocking function as the synchronous api that simply calls the async api and waits for an answer before returning, rather than using a non-concurrent api and let the caller run it in their own goroutine if they want it async?
In my current case I have a worker goroutine that reads from a request channel and sends the return value down the response channel (that it got in a request struct from the request channel).
This seems to differ from the linked question since I need the return values, or to synchronize so that I can be sure the api call finishes before I do something else, to avoid race conditions.
For golang, I recommend Effective Go-concurrency. Especially I think everyone using golang need to known the basics of goroutine and parallelization:
Goroutines are multiplexed onto multiple OS threads so if one should block, such as while waiting for I/O, others continue to run. Their design hides many of the complexities of thread creation and management.
The current implementation of the Go runtime dedicates only a single core to user-level processing. An arbitrary number of goroutines can be blocked in system calls, but by default only one can be executing user-level code at any time.
I'm making a little server for a project, I have a log handler class which contains a log implemented as a map and some methods to act on it (add entry, flush to disk, commit etc..)
This object is instantiated in the server Class, and I'm passing the address to the session so each session can add entries to it.
The sessions are async, the log writes will happen in the async_read callback. I'm wondering if this will be an issue and if i need to use locks?
The map format is map<transactionId map<sequenceNum, pair<head, body>>, each session will access a different transactionId, so there should be no clashes as far as i can figure. Also hypothetically, if they were all writing to the same place in memory -- something large enough that the operation would not be atomic; would i need locks? As far as I understand each async method dispatches a thread to handle the operation, which would make me assume yes. At the same time I read that one of the great uses of async functions is the fact that synchronization primitives are not needed. So I'm a bit confused.
First time using ASIO or any type of asynchronous functions altogether, and i'm not a very experienced coder. I hope the question makes sense! The code seems to run fine so far, but i'm curios if it's correct.
Thank you!
Asynchronous handlers will only be invoked in application threads processing the io_service event loop via run(), run_one(), poll(), or poll_one(). The documentation states:
Asynchronous completion handlers will only be called from threads that are currently calling io_service::run().
Hence, for a non-thread safe shared resource:
If the application code only has one thread, then there is neither concurrency nor race conditions. Thus, no additional form of synchronization is required. Boost.Asio refers to this as an implicit strand.
If the application code has multiple threads processing the event-loop and the shared resource is only accessed within handlers, then synchronization needs to occur, as multiple threads may attempt to concurrently access the shared resource. To resolve this, one can either:
Protect the calls to the shared resource via a synchronization primitive, such as a mutex. This question covers using mutexes within handlers.
Use the same strand to wrap() the ReadHandlers. A strand will prevent concurrent invocation of handlers dispatched through it. For more details on the usage of strands, particularly for composed operations, such as async_read(), consider reading this answer.
Rather than posting the entire ReadHandler into the strand, one could limit interacting with the shared resource to a specific set of functions, and these functions are posted as CompletionHandlers to the same strand. This subtle difference between this and the previous solution is the granularity of synchronization.
If the application code has multiple threads and the shared resource is accessed from threads processing the event loop and from threads not processing the event loop, then synchronization primitives, such as a mutex, needs to be used.
Also, even if a shared resource is small enough that writes and reads are always atomic, one should prefer using explicit and proper synchronization. For example, although the write and read may be atomic, without proper memory fencing to guarantee memory visibility, a thread may not observe a chance in memory even though the actual memory has chanced. Boost.Asio's will perform the proper memory barriers to guarantee visibility. For more details, on Boost.Asio and memory barriers, consider reading this answer.
(This probably duplicates the question ASP.NET MVC4 Async controller - Why to use?, but about webapi, and I do not agree with answers in there)
Suppose I have a long running SQL request. Its data should be than serialized to JSON and sent to browser (as a response for xhr request). Sample code:
public class DataController : ApiController
{
public Task<Data> Get()
{
return LoadDataAsync(); // Load data asynchronously?
}
}
What actually happens when I do $.getJson('api/data', ...) (see this poster http://www.asp.net/posters/web-api/ASP.NET-Web-API-Poster.pdf):
[IIS] Request is accepted by IIS.
[IIS] IIS waits for one thread [THREAD] from the managed pool (http://msdn.microsoft.com/en-us/library/0ka9477y(v=vs.110).aspx) and starts work in it.
[THREAD] Webapi Creates new DataController object in that thread, and other classes.
[THREAD] Uses task-parallel lib to start a sql-query in [THREAD2]
[THREAD] goes back to managed pool, ready for other processing
[THREAD2] works with sql driver, reads data as it ready and invokes [THREAD3] to reply for xhr request
[THREAD3] sends response.
Please, feel free to correct me, if there's something wrong.
In the question above, they say, the point and profit is, that [THREAD2] is not from The Managed Pool, however MSDN article (link above) says that
By default, parallel library types like Task and Task<TResult> use thread pool threads to run tasks.
So I make a conclusion, that all THREE THREADS are from managed pool.
Furthermore, if I used synchronous method, I would still keep my server responsive, using only one thread (from the precious thread pool).
So, what's the actual point of swapping from 1 thread to 3 threads? Why not just maximize threads in thread pool?
Are there any clearly useful ways of using async controllers?
I think the key misunderstanding is around how async tasks work. I have an async intro on my blog that may help.
In particular, a Task returned by an async method does not run any code. Rather, it is just a convenient way to notify callers of the result of that method. The MSDN docs you quoted only apply to tasks that actually run code, e.g., Task.Run.
BTW, the poster you referenced has nothing to do with threads. Here's what happens in an async database request (slightly simplified):
Request is accepted by IIS and passed to ASP.NET.
ASP.NET takes one of its thread pool threads and assigns it to that request.
WebApi creates DataController etc.
The controller action starts an asynchronous SQL query.
The request thread returns to the thread pool. There are now no threads processing the request.
When the result arrives from the SQL server, a thread pool thread reads the response.
That thread pool thread notifies the request that it is ready to continue processing.
Since ASP.NET knows that no other threads are handling that request, it just assigns that same thread the request so it can finish it off directly.
If you want some proof-of-concept code, I have an old Gist that artificially restricts the ASP.NET thread pool to the number of cores (which is its minimum setting) and then does N+1 synchronous and asynchronous requests. That code just does a delay for a second instead of contacting a SQL server, but the general principle is the same.
The profit of asynchronous actions is that while the controller is waiting for the sql query to finish no threads are allocated for this request, while if you used a synchronous method a thread would be locked up in the execution of this method from the start to end of that method. While SQL server is doing its job the thread isn't doing anything but waiting.
If you use asynchronous methods this same thread can respond to other requests while SQL server is doing its thing.
I believe your steps are wrong at step 4, I don't think it will create a new thread to do the SQL query. At 6 there isn't created a new thread, it is just one of the available threads that will be used to continue from where the first thread left off. The thread at 6 could be the same as started the async operation.
The point of async is not to make an application multi threaded but rather to let a single threaded application carry on with something different instead of waiting for a response from an external call that is executing on a different thread or process.
Consider a desk top application that shows stock prices from different exchanges. The application needs to make a couple of REST / http calls to get some data from each of the remote stock exchange servers.
A single threaded application would make the first call, wait doing nothing until it got the first set of prices, update it's window, then make a call to the next external stock price server, again wait doing nothing until it got the prices, update it's window... etc..
We could go all multi threaded kick off the requests in parallel and update the screen in parallel, but since most of the time is spent waiting for a response from the remote server this seems overkill.
It might be better for the thread to:
Make a request for the first server but instead of waiting for the answer leave a marker, a place to come back to when the prices arrive and move on to issuing the second request, again leaving a marker of a place to come back to...etc.
When all the requests have been issued the application execution thread can go on to dealing with user input or what ever is required.
Now when a response from one of the servers is received the thread can be directed to continue from the marker it laid down previously and update the window.
All of the above could have been coded long hand, single threaded, but was so horrendous going multi threaded was often easier. Now the process of leaving the marker and coming back is done by the compiler when we write async/await. All single threaded.
There are two key points here:
1) Multi threaded does still happen! The processing of our request for stock prices happens on a different thread (on a different machine). If we were doing db access the same would be true. In examples where the wait is for a timer, the timer runs on a different thread. Our application though is single threaded, the execution point just jumps around (in a controlled manner) while external threads execute
2) We lose the benefit of async execution as soon as the application requires an async operation to complete. Consider an application showing the price of coffee from two exchanges, the application could initiate the requests and update it's windows asynchronously on a single thread, but now if the application also calculated the difference in price between two exchanges it would have to wait for the async calls to complete. This is forced on us because an async method (such as one we might write to call an exchange for a stock price) does not return the stock price but a Task, which can be thought of as a way to get back to the marker that was laid down so the function can complete and return the stock price.
This means that every function that calls an async function needs to be async or to wait for the "other thread/process/machine" call at the bottom of the call stack to complete, and if we are waiting for the bottom call to complete well why bother with async at all?
When writing a web api, IIS or other host is the desktop application, we write our controller methods async so that the host can execute other methods on our thread to service other requests while our code is waiting for a response from work on a different thread/process/machine.
I my opinion the following describes a clear advantage of async controllers over synchronous ones.
A web application using synchronous methods to service high latency
calls where the thread pool grows to the .NET 4.5 default maximum of
5, 000 threads would consume approximately 5 GB more memory than an
application able the service the same requests using asynchronous
methods and only 50 threads. When you’re doing asynchronous work,
you’re not always using a thread. For example, when you make an
asynchronous web service request, ASP.NET will not be using any
threads between the async method call and the await. Using the thread
pool to service requests with high latency can lead to a large memory
footprint and poor utilization of the server hardware.
from Using Asynchronous Methods in ASP.NET MVC 4