According to apple's docs it says:
Because your MIDIReadProc callback is invoked from a separate thread,
be aware of the synchronization issues when using data provided by
this callback.
Does this mean, use #synchronize to do thread blocking for safety?
Or does this literally mean synchronization timing issues may happen?
I am currently trying to read a midi file, and use a MIDIReadProc to trigger the note-on / note-off of a software synth based off of midi events. I need this to be extremely reliable and perfectly in-time. Right now, I am noticing that when I consume these midi events and write the audio to a buffer (all done from the MIDIReadProc), the timing is extremely sloppy and not sounding right at all. So I would like to know, what is the "proper" way to consume midi events from a MIDIReadProc?
Also, is a MIDIReadProc the only option for consuming midi events from a midi file?
Is there another option as far as setting up a virtual endpoint that could be directly consumed by my synthesizer? If so, how does that work exactly?
If you presume a function of this format to be the midiReadProc,
void midiReadProc(const MIDIPacketList *packetList,
void* readProcRefCon,
void* srcConnRefCon)
{
MIDIPacket *packet = (MIDIPacket*)packetList->packet;
int count = packetList->numPackets;
for (int k=0; k<count; k++) {
Byte midiStatus = packet->data[0];
Byte midiChannel= midiStatus & 0x0F;
Byte midiCommand = midiStatus >> 4;
//parse MIDI messages, extract relevant information and pass it to the controller
//controller must be visible from the midiReadProc
}
packet = MIDIPacketNext(packet);
}
the MIDI client has to be declared in the controller, interpreted MIDI events get stored into the controller from MIDI callback and read by the audioRenderCallback() on each audio render cycle. This way you can minimize timing imprecisions to the
length of the audio buffer, which you can negotiate during AudioUnit setup to be as short as the system allows for.
A controller can be a #interface myMidiSynthController : NSViewController you define, consisting of a matrix of MIDI channels and a pre-determined maximum-polyphony-per-channel, among other relevant data such as interface elements, phase accumulators for each active voice, AudioComponentInstance, etc... It would be wrong to resize the controller based on the midiReadProc() input. RAM is cheap nowadays.
I'm using such MIDI callbacks for processing live input from MIDI devices. Concerning playback of MIDI files, if you
want to process streams or files of arbitrary complexity, you may also run into surprises. MIDI standard itself
has timing features, which work as good as MIDI hardware allows for. Once you read an entire file into the memory, you can
translate your data into whatever you want and use your own code for controlling sound synthesis.
Please, observe not to use any code which would block the audio render thread (i.e. inside audioRenderCallback()), or would do memory management on it.
You could use AVAudioEngine.musicSequence and prepare your audio unit graph. Then use the MusicSequence API to load your GM file. Like this you don’t need to do the timing by yourself. Note I have not done this myself so far but I understand in theory it should work like this.
After you instantiate your synthesizer audio unit, you attach and connect it to the AVAudioEngine graph.
Does this mean, use #synchronize to do thread blocking for safety?
The opposite of what you’ve said is true: You should certainly not lock in a realtime thread. The #synchronized directive will lock if the resource is already locked. You may consider to use lock-free queues for realtime threads. See also Four common mistakes in audio development.
If you have to use CoreMIDI and MIDIReadProc, you can send MIDI commands to the synthesizer audio unit by calling MusicDeviceMIDIEvent right from your callback.
As is known, some blocking calls like read and write would return -1 and set errno to EINTR, and we need handle this.
My question is: Does this apply for non-blocking calls, e.g, set socket to O_NONBLOCK?
Since some articles and sources I have read said non-blocking calls don't need bother with this, but I have found no authoritative reference about it. If so, does it apply cross different implementations?
I cannot give you a definitive answer to this question, and the answer may further vary from system to system, but I would expect a non-blocking socket to never fail with EINTR. If you take a look at the man pages of various systems for the following socket functions bind(), connect(), send(), and receive(), or look those up in the POSIX standard, you'll notice something interesting: All these functions except one may return -1 and set errno to EINTR. The one function that is not documented to ever fail with EINTR is bind(). And bind() is also the only function of that list that will never block by default. So it seems that only blocking functions may fail because of EINTR, including read() and write(), yet if these functions never block, they also will never fail with EINTR and if you use O_NONBLOCK, those functions will never block.
It would also make no sense from a logical perspective. E.g. consider you are using blocking I/O and you call read() and this call has to block, but while it was blocking, a signal is sent to your process and thus the read request is unblocked. How should the system handle this situation? Claiming that read() did succeed? That would be a lie, it did not succeed because no data was read. Claiming it did succeed, but zero bytes data were read? This wouldn't be correct either, since a "zero read result" is used to indicate end-of-stream (or end-of-file), so your process would to assume that no data was read, because the end of a file has been reached (or a socket/pipe has been closed at other end), which simply isn't the case. The end-of-file (or end-of-stream) has not been reached, if you call read() again, it will be able to return more data. So that would also be a lie. You expectation is that this read call either succeeds and reads data or fails with an error. Thus the read call has to fail and return -1 in that case, but what errno value shall the system set? All the other error values indicate a critical error with the file descriptor, yet there was no critical error and indicating such an error would also be a lie. That's why errno is set to EINTR, which means: "There was nothing wrong with the stream. Your read call just failed, because it was interrupted by a signal. If it wasn't interrupted, it may still have succeeded, so if you still care for the data, please try again."
If you now switch to non-blocking I/O, the situation of above never arises. The read call will never block and if it cannot read data immediately, it will fail with an error EAGAIN (POSIX) or EWOULDBLOCK (unofficial, on Linux both are the same error, just alternative names for it), which means: "There is no data available right now and thus your read call would have to block and wait for data arriving, but blocking is not allowed, so it failed instead." So there is an error for every situation that may arise.
Of course, even with non-blocking I/O, the read call may have temporarily interrupted by a signal but why would the system have to indicate that? Every function call, whether this is a system function or one written by the user, may be temporarily interrupted by a signal, really every single one, no exception. If the system would have to inform the user whenever that happens, all system functions could possibly fail because of EINTR. However, even if there was a signal interruption, the functions usually perform their task all the way to the end, that's why this interruption is irrelevant. The error EINTR is used to tell the caller that the action he has requested was not performed because of a signal interruption, but in case of non-blocking I/O, there is no reason why the function should not perform the read or the write request, unless it cannot be performed right now, but then this can be indicated by an appropriate error.
To confirm my theory, I took a look at the kernel of MacOS (10.8), which is still largely based on the FreeBSD kernel and it seems to confirm the suspicion. If a read call is currently not possible, as no data are available, the kernel checks for the O_NONBLOCK flag in the file descriptor flags. If this flag is set, it fails immediately with EAGAIN. If it is not set, it puts the current thread to sleep by calling a function named msleep(). The function is documented here (as I said, OS X uses plenty of FreeBSD code in its kernel). This function causes the current thread to sleep until it is explicitly woken up (which is the case if data becomes ready for reading) or a timeout has been hit (e.g. you can set a receive timeout on sockets). Yet the thread is also woken up, if a signal is delivered, in which case msleep() itself returns EINTR and the next higher layer just passes this error through. So it is msleep() that produces the EINTR error, but if the O_NONBLOCK flag is set, msleep() is never called in the first place, hence this error cannot be returned.
Of course that was MacOS/FreeBSD, other systems may be different, but since most systems try to keep at least a certain level of consistency among these APIs, if a system breaks the assumption, that non-blocking I/O calls can never fail because of EINTR, this is probably not by intention and may even get fixed if your report it.
#Mecki Great explanation. To add to the accepted answer, the book "Unix Network Programming - Volume 1, Third Edition" (Stevens) makes a distinction between slow system call and others in chapter/section 5.9 - "Handling Interrupted System Calls". I am quoting from the book -
We used the term "slow system call" to describe accept, and we use
this term for any system call that can block forever. That is, the
system call need never return.
In the next para of the same section -
The basic rule that applies here is that when a process is blocked in
a slow system call and the process catches a signal and the signal
handler returns, the system call can return an error of EINTR.
Going by this explanation, a read / write on a non-blocking socket is not a slow system call and hence should not return an error of EINTR.
Just to add some evidence to #Mecki's answer, I found this discussion about fixing a bug in Linux where a patch caused non-blocking recvmsg to return EINTR. It was stated:
EINTR always means that you asked for a blocking operation, and a
signal arrived meanwhile.
Once you invert the "blocking" part of that set of conditions, EINTR
becomes an impossible event.
Also:
Look at what we do for AF_INET. We handle this the proper way.
If we are 'interrupted' by a signal while sleeping in lock_sock(),
recvmsg() on a non blocking socket, we return -EAGAIN properly, not
-EINTR.
Fact that we potentially sleep to get the socket lock is hidden for
the user, its an implementation detail of the kernel.
We never return -EINTR, as stated in manpage for non blocking sockets.
Source here: https://patchwork.ozlabs.org/project/netdev/patch/1395798147.12610.196.camel#edumazet-glaptop2.roam.corp.google.com/#741015
Remark from MSDN about CompletionKey in CreateIoCompletionPort function:
Use the CompletionKey parameter to help your application track which
I/O operations have completed. This value is not used by
CreateIoCompletionPort for functional control; rather, it is attached
to the file handle specified in the FileHandle parameter at the time
of association with an I/O completion port. This completion key should
be unique for each file handle, and it accompanies the file handle
throughout the internal completion queuing process. It is returned in
the GetQueuedCompletionStatus function call when a completion packet
arrives. The CompletionKey parameter is also used by the
PostQueuedCompletionStatus function to queue your own special-purpose
completion packets.
The above remarks leave me a question. Why use the CompletionKey given that we can associate user context with the file handle in an extended overlapped structure like this:
typedef struct s_overlappedplus
{
OVERLAPPED ol;
int op_code;
/*we can alternatively put user context over here instead of CompletionKey*/
LPVOID user_context;
} t_overlappedplus;
and retreive through CONTAINING_RECORD macro after completion?
Cool, I'm only convinced that CompletionKey is per-handle context while the extended overlapped structure is per-I/O one. But what's the philosophy behind such design and in what circumstance can it be necessary to use CompletionKey instead of an extended overlapped structure in term of user context?
I'm using I/O completion ports on Windows for serial port communication (we will potentially have lots and lots of serial port usage). I've done the usual, creating the IOCP, spinning up the I/O threads, and associating my CreateFile() handle with the IOCP (CreateFile() was called with FILE_FLAG_OVERLAPPED). That's all working fine. I've set the COMMTIMEOUTS all to 0 except ReadIntervalTimeout which is set to MAXDWORD in order to be completely async.
In my I/O thread, I've noticed that GetQueuedCompletionStatus() blocks indefinitely. I'm using an INFINITE timeout. So I put a ReadFile() call right after I associate my handle with the IOCP. Now that causes GetQueuedCompletionStatus() to release immediately for some reason with 0 bytes transferred, but there's no errors (it returns true, GetLastError() reports 0). I obviously want it to block if there's nothing for it to do. If I put another ReadFile() after GetQueuedCompletionStatus(), then another thread in the pool will pick it up with 0 bytes transferred and no errors.
In the examples I've seen and followed, I don't see anyone setting the hEvent on the OVERLAPPED structure when using IOCP. Is that necessary? I don't care to ever block IOCP threads -- so I'll never be interested in CreateEvent(...) | 1.
If it's not necessary, what could be causing the problem? GetQueuedCompletionStatus() needs to block until data arrives on the serial port.
Are there any good IOCP serial port examples out there? I haven't found a complete serial port + IOCP example out there. Most of them are for sockets. In theory, it should work for serial ports, files, sockets, etc.
I figured it out -- I wasn't calling SetCommMask() with EV_RXCHAR | EV_TXEMPTY and then WaitCommEvent() with the OVERLAPPED struct. After I did that, my IOCP threads behaved as expected. GetQueuedCompletionStatus() returned when a new character appeared on the port. I could then call ReadFile().
So to answer the original question: "no, you don't need to set hEvent for IOCP with serial ports."
Is it possible to use IO Completion Ports for Serial I/O? According to Windows via C/C++ it is alluded to that it is possible, and does give an example of using IOCP with physical files showing work with CreateFile, ReadFile, WriteFile, etc. However can this actually work with serial comms - has anyone got it working?
I can't find any examples of this on the web, but I cannot be the first to attempt it?
Yes, using I/O Completion Ports for Serial I/O works fine. There is some setup work needed to create a file handle for a serial port that is appropriate for IOCP. But once the setup is done, you can do asynchronous ReadFile() and WriteFile() operations just like with regular file handles and socket handles.
The setup is basically:
Open serial port with CreateFile() passing in the FILE_FLAG_OVERLAPPED value as the dwFlagsAndAttributes parameter.
Modify the serial port state as desired using GetCommState() and SetCommState(). Do this just like you would do when not using IOCP.
Use GetCommTimeouts() and SetCommTimeouts() to turn off total-timeouts for read operations, since it typically doesn't make sense to have timeouts for asynchronous operations. (You would instead explicitly call CancelIO() to cancel a read operation instead.) Turning off total-timeouts is done by setting the ReadTotalTimeoutMultiplier and ReadTotalTimeoutConstant fields of the COMMTIMEOUTS structure to zero.
Now you can use the handle with IOCP just like you would do with regular file handles and socket handles. I.e. attach the handle to a completion port using CreateIoCompletionPort(), initiate I/O operations with ReadFile() or WriteFile() using an OVERLAPPED structure, dequeue completed, failed or canceled operations from the completion port using the GetQueuedCompletionStatus() function.
Additional serial port specific events can also be retrieved asynchronously using the WaitCommEvent() function.