Most example apps I come across for receiving data are using async calls. For instance, c++ examples use boost asio services to bind message handlers to callbacks. But what about an app that only needs to listen to data from a single socket and process the messages in order? Would it be faster to have a loop that polls/recv's from the socket and calls the handler without using a callback (assume main and logging threads are separate)? Or is there no performance difference (assume messages are coming in as fast as the network card and kernel can handle them)?
There are many intricacies I don't know such as the impact of callbacks to performance due to things like branch prediction. Or if there will be a performance penalty of the callbacks call a different thread to do the processing. Curious to hear some thoughts, experiences, dialog on this subject to save myself from attempting both implementations to discover the answer.
Related
In concurrent systems, domain events are typically handled asynchronously. In Go, a simple approach for asynchronous event handling can be implemented via channels, but the issue is that if something bad happens for handling an event, or worst, for the whole program, the event will be lost.
How asynchronous domain events can be handled properly in a Go program, i.e.:
When an event handler fails, the event should not be purged from the event queue, in order to be handled properly in a later time.
If the whole program goes down, the events have to be recovered and processed accordingly.
The first is relatively easy; you can have an error handler within the worker that re-queues the work in the event of an error.
The second is much harder; your options are a) roll your own bulletproof mechanism for writing events to disk and purging them when they're completed in a thread-safe way or b) use one of the many, many popular systems available that's already proven reliable, e.g. RabbitMQ or Kafka, with the appropriate replication and redundancy to ensure the level of reliability you require. I would strongly recommend the latter.
Is MPI_Bsend_init/MPI_Start best asynchronous buffered communication. Can you guys think of better way to communicate data between processors. Pseudo-Code for N Processing nodes
MPI_Recv(request[i]) -- Recv data
for(i=0;i<N;i++) MPI_Bsend_init(request[i]) -- Setup request
MPI_Start(request[i]) -- Send data
Bsend is the wrong function here from a performance standpoint. There is little to no advantage of Bsend, as the eager protocol used by essentially all implementations today, is buffered on the receiver side automatically for small messages, which is where Bsend would be viable.
In any case, persistent send - as you are using - is already nonblocking, so there is no such thing as Isent_init. See e.g. http://www.mpi-forum.org/docs/mpi-1.1/mpi-11-html/node51.html:
The call is local, with similar semantics to the nonblocking
communication operations described in section Nonblocking
communication . That is, a call to MPI_START with a request created by
MPI_SEND_INIT starts a communication in the same manner as a call to
MPI_ISEND...
And my colleagues have surprised me with confirmation that persistent Send-Recv does provide efficiency gains on modern InfiniBand clusters. I can only assume this is because IB page registration is done up-front.
Closed. This question needs to be more focused. It is not currently accepting answers.
Closed 4 years ago.
Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
From the wikipedia Reactor Pattern article:
The reactor design pattern is an event handling pattern for handling service requests delivered concurrently to a service handler by one or more inputs.
It named a few examples, e.g. nodejs, twisted, eventmachine
But what I understand that above is popular event driven framework, so make them also a reactor pattern framework?
How to differentiate between these two? Or they are the same?
The reactor pattern is more specific than "event driven programming". It is a specific implementation technique used when doing event driven programming. However, the term is not used with much accuracy in typical conversation, so you should be careful about using it and expecting your audience to understand you, and you should be careful in how you interpret the term when you encounter its use.
One way to look at the reactor pattern is to consider it closely related to the idea of "non-blocking" operations. The reactor sends out notifications when certain operations can be completed without blocking. For example, select(2) can be used to implement the reactor pattern for reading from and writing to sockets using the standard BSD socket APIs (recv(2), send(2), etc). select will tell you when you can receive bytes from a socket instantly - because the bytes are present in the kernel receiver buffer for that socket, for example.
Another pattern you might want to consider while thinking about these ideas is the proactor pattern. In contrast to the reactor pattern, the proactor pattern has operations start regardless of whether they can finish immediately or not, has them performed asynchronously, and then arranges to deliver notification about their completion.
The Windows I/O Completion Ports (IOCP) API is one example where the proactor pattern can be seen. When performing a send on a socket with IOCP, the send operation is started regardless of whether there is any room in the kernel send buffer for that socket. The send operation continues (in another thread, perhaps a thread in the kernel) while the WSASend call completes immediately. When the send actually completes (meaning only that the bytes being sent have been copied into the kernel send buffer for that socket), a callback function supplied to the WSASend call is invoked (in a new thread in the application).
This approach of starting operations and then being notified when they are complete is central to the idea of asynchronous operations. Compare it to non-blocking operations where you wait until an operation can complete immediately before attempting to perform it.
Either approach can be used for event driven programming. Using the reactor pattern, a program waits for the event of (for example) a socket being readable and then reads from it. Using the proactor pattern, the program instead waits for the event of a socket read completing.
Strictly speaking, Twisted misuses the term reactor. The Twisted reactor which is based on select(2) (twisted.internet.selectreactor) is implemented using non-blocking I/O, which is very reactor-like. However, the interface it exposes to application code is asynchronous, making it more proactor-like. Twisted also has a reactor based on IOCP. This reactor exposes the same asynchronous application-facing API and uses the proactor-like IOCP APIs. This hybrid approach, varying from platform to platform in its details, makes neither the term "reactor" nor "proactor" particularly accurate, but since the API exposed by twisted.internet.reactor is basically entirely asynchronous instead of non-blocking, proactor would probably have been a better choice of name.
I think that this separation "non-blocking" and "asynchronous" is wrong, as the main implication of "asynchronous" is "non-blocking". Reactor pattern is about asynchronous (so non-blocking) calls, but synchronous (blocking) processing of those calls. Proactor is about asynchronous (non-blocking) calls and asynchronous (non-blocking) processing of those calls.
To handle TCP connections, there are two competing web architectures, namely thread-based architecture and event-driven architecture.
Thread-Based Architecture
The oldest way of implementing a multi-threaded server is following the “thread per connection” approach. In order to control and limit the number of running threads, a single dispatcher thread can be used along with a bounded blocking queue and a thread pool.
The dispatcher blocks on a TCP socket for new connections and offers them to the bounded blocking queue. TCP connections exceeding the bound of the queue will be dropped allowing the accepted connections to operate with a desirable and predictable latency.
Event-Driven Architecture
Separating threads from connections, event-driven architecture only allows threads to be used for events on specific handlers.
This creative concept allows Reactor Pattern to come out of the shelf and show off. A system built on this architecture consists of event creators and event consumers.
The Reactor Pattern
The reactor pattern is the most popular implementation technique of event-driven architecture for TCP connection handling. In simple terms, it uses a single-threaded event loop, blocking on events and dispatches those events to corresponding handlers.
There is no need for other threads to block on I/O, as long as handlers for events are registered to take care of them. Considering a TCP connection, we can easily refer events to these instances: connected, input-ready, output-ready, timeout, and disconnected.
Reactor pattern decouples the modular application-level code from reusable reactor implementation. To achieve that, the architecture of the reactor pattern consists of two important participants — Reactor and Handlers.
Reactor
A Reactor runs in a separate thread, and it is reacting to the I/O events such as connected, input-ready, output-ready, timeout and disconnected, by dispatching the work to the appropriate registered handler.
Handlers
A Handler performs the actual work or the response that needs to be done with an I/O event. A Reactor responds to I/O events by dispatching the appropriate handler.
“Pattern Languages of Program Design” by Jim Coplien and Douglas C. Schmidt which was published way back in 1995, is one of the books that has explained the Reactor Pattern in detail.
I am designing an simulator application where the application launches multiple socket connection(around 1000 connections) to a server. I don't want to launch as many as threads to handle those connections, since the system cant handle that much clients. Using Select doesnt make sense, since i need to loop through 1000 connections which may be slow. Please suggest me how to handle this scenario.
You want to be using asynchronous I/O with an I/O Completion Port (IOCP).
It's too much to explain shortly, but any Windows application that needs to support a large number of concurrent sockets should be using an IOCP.
An IOCP is essentially an Windows-provided thread safe work queue. You queue a 'completion packet' to an IOCP and then another thread dequeues it and does work with it.
You can also associate many types of handles that support overlapped operations, such as sockets, to an IOCP. When you associate a handle with an IOCP, overlapped operations such as WSARecv will automatically post a completion packet to the associated IOCP.
So, essentially, you could have one thread handling all 1000 connections. Each socket will be created as an overlapped socket and then associated with your IOCP. You can then call WSARecv on all 1000 sockets and wait for a completion packet to become available. When data is received, the operating system will post a completion packet to the associated IOCP. This will contain relevant information, such as how much data was read and the buffer containing the data.
Looping through 1000 handles is still significantly faster than sending 1000 packets, so I wouldn't worry about performance here. select() is still the way to go.
Is there a difference in performance when multi-threading developed with win32 event objects (CreateEvent) or with thread window message queue. Both should use some kind of WaitFor... calls.
My own code almost completely based on event, but maybe I lose something when don't use messages.
If you are worried about the performance difference between thread messages, and kernel events, then you should probably not use thread messages.
Win32 thread message queues are a mechanism developed originally for Windows 16 - when there were no threads. They've grown to handle Win32's threading model, but under the covers they are rather complex beasts.
This has pro's and con's. The cons are, quite simply, that they are slower, and have a lot more limits, than other forms of inter thread synchronization and comms. For starters, because lots of pieces of windows code (MessageBox, DoDragDrop etc) implement modal message loops - there are a lot of times when thread messages can be lost. Its important not to use those APIs from threads that are meant to be receiving thread messages.**1* There are also limits to the size of a message queue before it starts to drop messages, and the thread messaging APIs (GetMessage) do not cause the thread to enter an alertable state (so you cannot use QueueUserAPC).
The pro' for thread messages are - as long as their limits are respected - they are a very reliable pre-made wheel that serializes calls to a thread. If you find yourself implementing a queueing mechanism for a non UI worker thread, why re-invent a well tested wheel - use the pre-built message queue.
**1* This includes most implementations of debugging macro's like ASSERT that will pop up a message box.