MIDI MusicDevice AudioUnit: Playing two notes of same pitch, stop one? - core-audio

I am quite a novice when it comes to AudioUnits, so please forgive me if my question is very basic.
I am using the MusicDevice AudioUnit to playback some notes. I am using MusicDeviceMIDIEvent to send the note-on and note-off messages. It works well. Sometimes more than one note should sound simultaneously, so I may send two note-on messages in a row. Sometimes these notes happen to have the same pitch. Then when I want to turn off one of the notes, I send a note-off event for this pitch. But this message turns off all notes of the pitch. Of course this behavior makes a lot of sense, but I wanted to ask how people normally deal with this issue.
Should I use different channels for simultaneous notes? Or manage the notes manually, say with a counted set which holds the pitches currently playing, and only send the note-off event after the last instance of a pitch should stop playing? Or something else entirely?
EDIT:
Since this is on the iOS, I have to use kAudioUnitSubType_Sampler as the AudioUnit subType. While the documentation only mentions that this type is mono-timbral, I now suspect it is also monophonic. This of course would explain the behavior. Still I wonder how I would go about this is if I actually had an polyphonic instrument.
EDIT 2:
I did some more tests, and it seems to me now that sending a note-off message on any channel stops all notes with the same pitch on all channels. I took the apple example code at http://developer.apple.com/library/ios/#samplecode/LoadPresetDemo/Introduction/Intro.html and modified the stopPlay[low/mid/high]Note methods to send the note-off messages on some random channel (if you must know, on channels 7,8, and 9, respectively). It still stops the notes, despite the fact that the note-on messages are being sent on channel 0. Is this expected behavior?
Just to make sure I am not making a stupid mistake, these are the methods that send note-on and note-off messages:
- (IBAction) startPlayLowNote:(id)sender {
UInt32 noteNum = kLowNote;
UInt32 onVelocity = 127;
UInt32 noteCommand = kMIDIMessage_NoteOn << 4 | 0;
OSStatus result = noErr;
require_noerr (result = MusicDeviceMIDIEvent (self.samplerUnit, noteCommand, noteNum, onVelocity, 0), logTheError);
logTheError:
if (result != noErr) NSLog (#"Unable to start playing the low note. Error code: %d '%.4s'\n", (int) result, (const char *)&result);
}
- (IBAction) stopPlayLowNote:(id)sender {
//note the channel!
UInt32 noteNum = kLowNote;
UInt32 noteCommand = kMIDIMessage_NoteOff << 4 | 7;
OSStatus result = noErr;
require_noerr (result = MusicDeviceMIDIEvent (self.samplerUnit, noteCommand, noteNum, 0, 0), logTheError);
logTheError:
if (result != noErr) NSLog (#"Unable to stop playing the low note. Error code: %d '%.4s'\n", (int) result, (const char *)&result);
}

I'm pretty sure that the behavior for a note off after two note on events of the same pitch on the same channel is undefined. Some instruments might turn off both notes and some might turn off one and require a second note off to turn off the other.
If you really need to have two simultaneous notes of the same pitch, they should be on different channels.
Edit regarding posted code
I tried the sample project in your link and changed the channel the same way you did in your posted code. It turns out that kAudioUnitSubType_Sampler is indeed mono-timbral, so it ignores the MIDI channel parameter. So if you want to have two simultaneous notes of the same pitch with kAudioUnitSubType_Sampler, you'll have to create two separate instances.
Note that kAudioUnitSubType_Sampler is not monophonic. It is polyphonic since it can play multiple pitches simultaneously.

What about using MusicDevice.h's MusicDeviceStartNote() and MusicDeviceStopNote()? It uses a unique token for the note so you should be able to distinguish between two of the same pitch.

Related

Display message "Connect to customer care" if bot isn't able to answer on 3 consecutive attempts...dotnet core 3.1...............MS Bot Framework

I have a bot which displays "No answers found" if the question that user has asked isn't present in the knowledge base.
I want to display a different message (e.g Contact support desk on some phone number 123456789) if the bot isn't able to answer on 3 consecutive attempts.
Something like as shown in the below image:
This is hard to answer without knowing exactly how you have QnA Maker set up, but what #JJ_Wailes suggested is the way to go. You need to keep track of the number of wrong answers, and then provide your alternative default message. To give myself more flexibility, I have always created my own default response logic instead of relying on QnA Maker. I'm going to make some assumptions that you are already good making the QnA Maker calls and dealing with user and conversation state. Instead of relying on the QnA Maker default answer, I check for the confidence score returned and create my own message activities for those cases. For simplicity I'm going to ignore prompts. Here is a sample of my typical QnA Maker flow. Unfortunately I have developed in nodejs only, but I think this should be similar enough that you can adapt for dotnet.
var MINIMUM_SCORE = 50;
const conversationData = await this.dialogState.get(context, {});
// Will use conversationData.qnaFailCount to track consecutive wrong answers
// Make the initial call
var qnaResult = await QnAServiceHelper.queryQnAService(activity.text);
var qnaAnswer = qnaResult[0].answer;
// Apply a confidence filter
if (qnaResult[0].score > MINIMUM_SCORE) {
outputActivity = MessageFactory.text(qnaAnswer);
conversationData.qnaFailCount = 0; // Reset counter when answer found
} else {
// If low confidence, increment counter
conversationData.qnaFailCount += 1;
if (conversationData.qnaFailCount < 3) {
outputActivity = MessageFactory.text(defaultAnswer);
} else {
// Send the escalation message for every consecutive "no answer" starting at 3
outputActivity = MessageFactory.text(escalation);
}
}
await this.conversationState.saveChanges(context); // Don't forget to save state!
return outputActivity;
So now you will increment the counter for each answer that is not found and reset it whenever an answer is found. In this way it will continue to give the escalation mesasge for the 4th, 5th, etc. query with no answer. Since you are using conversation state, when the user leaves the site and comes back, they will have a new conversation and the process will start over.
Note that in the event you are using this within Microsoft Teams, that is treated as one long conversation, so the message wouldn't reset even after multiple days, only with a correct answer. It seems like this is not an MS Teams use case but I wanted to mention that.

Geolocation.GetLastKnownLocationAsync() sometimes returns null

This is on iOS 12.1.4 on an iPhone 6s. Usually Geolocation.GetLastKnownLocationAsync() works but sometimes it doesn't. It's the exact same code but if I sit here and press my "get latitude and longitude" button over and over eventually Geolocation.GetLastKnownLocationAsync() spits out a null.
Do you know why this happens and how I might handle it? Perhaps put it in a loop that tries ten times, waiting a second between each try?
var location = await Essentials.Geolocation.GetLastKnownLocationAsync(); // works most of the time but sometimes it doesn't work.
This is my proposed work around:
Essentials.Location location = null;
for(var i = 0; i < 10; i++)
{
location = await Essentials.Geolocation.GetLastKnownLocationAsync();
if(location == null)
{
Thread.Sleep(1000);
}
else
{
break;
}
}
First, it is really bad practice to use Thread.Sleep (unless you are not on the main/UI loop) as you are hanging the run loop, if you really need a delay, use await Task.Delay(.... Also CLLocationManager on iOS is running on the main loop and if you are blocking it, the message pump is hung and the location manager manager can not report back to the app.
"Spamming" CLLocationManager.Location (which Essentials is using on iOS) can (and will) result in null returns due to OS rate limiting updates (mainly a battery conservation measure) and if the OS is powering up the GPS radio to update its location, this method will timeout on from the OS, thus report nil back to GetLastKnownLocationAsync and thus you get a return of null.
CLLocationManager.Location on iOS is meant for a quick low-power return from the OS to app as is updated upon app launch, device reboot, etc... not every time you call it.
You can get the last known location of the device by calling the GetLastKnownLocationAsync method. This is often faster then doing a full query, but can be less accurate.
Otherwise you should be using GetLocationAsync in order to do a full GPS power up to obtain an updated accurate location.
To query the current device's location coordinates, the GetLocationAsync can be used. It is best to pass in a full GeolocationRequest and CancellationToken since it may take some time to get the device's location.
Typically I recommend using GetLastKnownLocationAsync as a quick way to get the general area of the user knowing that this might also return null. Then proceed to do a GetLocationAsync (passing both a GeolocationRequest and CancellationToken instance) in the background and update the app accordingly upon the more accurate and recent position.
re: https://learn.microsoft.com/en-us/xamarin/essentials/geolocation?tabs=ios

Stop MMC queue from fetching new requests when communication with card times out

We are using a custom board running version 3.2 of the kernel, but we've come across some issues when testing the robustness of the MMC layer.
First of all, our MMC slot does not have a card detection pin. The issue in question consists of the following:
Load the module (omap_hsmmc). The card is detected on power-up,
and is mounted appropriately.
Read something from the SD card (i.e. cat foo.txt)
While the file is being read, remove the card.
After many failed attempts, the system hangs.
Now, I've tracked the issue to the following code section, in drivers/mmc/card/queue.c:
static int mmc_queue_thread(void *d)
{
struct mmc_queue *mq = d;
struct request_queue *q = mq->queue;
current->flags |= PF_MEMALLOC;
down(&mq->thread_sem);
do {
struct request *req = NULL;
struct mmc_queue_req *tmp;
spin_lock_irq(q->queue_lock);
set_current_state(TASK_INTERRUPTIBLE);
req = blk_fetch_request(q);
mq->mqrq_cur->req = req;
spin_unlock_irq(q->queue_lock);
if (req || mq->mqrq_prev->req) {
set_current_state(TASK_RUNNING);
mq->issue_fn(mq, req);
} else {
if (kthread_should_stop()) {
set_current_state(TASK_RUNNING);
break;
}
up(&mq->thread_sem);
schedule();
down(&mq->thread_sem);
}
/* Current request becomes previous request and vice versa. */
mq->mqrq_prev->brq.mrq.data = NULL;
mq->mqrq_prev->req = NULL;
tmp = mq->mqrq_prev;
mq->mqrq_prev = mq->mqrq_cur;
mq->mqrq_cur = tmp;
} while (1);
up(&mq->thread_sem);
return 0;
}
Investigating this code, I've found that it hangs inside the mq->issue_fn(mq, req) call. This function prepares and issues the appropriate commands to fufill the request passed into it, and it is aware of any errors that happens when it can't communicate with the card. The error is handled in an awkward (in my opinion) manner, and does not 'bubble up' to the mmc_queue_thread. However, I've checked my version's code against that of the most recent kernel release (4.9), and I did not see any difference in the treatment of such errors, apart from a better separation of each error case (the treatment is very similar).
I suspect that the issue is caused by the inability ofthe upper layers to stop making new requests to read from the MMC card.
What I've Tried:
Rewrote the code so that the error is passed up so that I can do ret = mq->issue_fn(mq, req).
Being able to identify the specific errors, I tried to cancel the thread in different ways: calling kthread_stop, calling mmc_queue_suspend, __blk_end_request, and others. With these, the most I've been able to accomplish was keep the thread in a "harmless" state, where it still existed, but did not consume any resources. However, the user space program that triggered the call does not return, locking up in an uninterruptible state.
My questions:
Is the best way to solve this by stopping the upper layers from making new requests? Or should the thread itself be 'killed off'?
In a case such as mine, should it be assumed that because there is no card detection pin, the card should not be removed?
Update: I've found out that you can tell the driver to use polling to detect card insertion/removal. It can be done by adding the MMC_CAP_NEEDS_POLL flag to the mmc's .caps on driver initialization (on newer kernels, you can use the broken-cd property on your DT). However, the problem still happens after this modification.
Figured out the solution!
The issue was, as I suspected, that the block requests kept being issued even after the card was removed. The error was fixed in commit a8ad82cc1b22d04916d9cdb1dc75052e80ac803c from Texas' kernel repository:
commit a8ad82cc1b22d04916d9cdb1dc75052e80ac803c
Author: Sujit Reddy Thumma <sthumma#codeaurora.org>
Date: Thu Dec 8 14:05:50 2011 +0530
mmc: card: Kill block requests if card is removed
Kill block requests when the host realizes that the card is
removed from the slot and is sure that subsequent requests
are bound to fail. Do this silently so that the block
layer doesn't output unnecessary error messages.
Signed-off-by: Sujit Reddy Thumma <sthumma#codeaurora.org>
Acked-by: Adrian Hunter <adrian.hunter#intel.com>
Signed-off-by: Chris Ball <cjb#laptop.org>
The key in the commit is that a return BLKPREP_KILL was added to the mmc_prep_request(), besides some minor tweaks that added some 'granularity' to the error treatment, adding a specific code for the cases where the card was removed.

Supercollider: automatic sustain in events with envelope

(
SynthDef(\testEvt,{
arg out, gate = 1;
var sint = Blip.ar(440) * Linen.kr(gate,doneAction:2,releaseTime:0.8);
Out.ar(out, Pan2.ar(sint, 0));
}).add();
Synth(\testEvt)
(instrument: \testEvt, freq:220, sustain: inf).play;
(instrument: \testEvt,freq:220).play;
)
Executing the first and the second line after the SynthDef would create a synth which playes forever, whereas the third line's synth plays for 0.8 seconds as per default value the generated event.
The problem is that I don't use 'sustain' anywhere in my SynthDef and it uses automatically just because there is a Linen.
The same this doesn't happen for freq: both the events play at 440 and not at 220, and that's just because the SynthDef doesn't use 'freq' as an argument. So why sustain doesn't follow the same rule ?
Also, is there a way to reference synths created by an event ? So that, when they have sustain: inf as argument, I can free them on a later time.
(instrument: \testEvt, freq:220, sustain: inf).play;
and
(instrument: \testEvt,freq:220).play;
are events. Events handle a lot of things for you. One thing they do is calculate when to set a gate to 0. (Remember that gate is one of the arguments in your SynthDef.) In the first example, because you sustain for infinite duration, the gate never goes to zero. In the second example, it uses the default duration and sets the gate to zero after that duration has passed. You can find out what key words are used in event environment variables by looking at the Event.sc source file. If you search for sustain, you'll find out the other keywords it uses for timing. One of these is dur. Try this:
(instrument: \testEvt, dur:3).play
Freq is also a keyword for events, but since you have no freq argument, it can't effect your synthDef. If you want to set the freq, you'll need to make a change:
SynthDef(\testEvt,{
arg out, gate = 1, freq = 440;
var sint = Blip.ar(freq) * Linen.kr(gate,doneAction:2,releaseTime:0.8);
Out.ar(out, Pan2.ar(sint, 0));
}).add();
For contrast between events and controlling a synth directly, try:
a = Synth.new(\testEvt, [\out, 0, \gate, 1])
You can add in any other arguments you want, like freq or sustain, but they have no effect because those aren't arguments to your synthdef. And, unlike Event, synth doesn't do any calculations on your behalf. When you want the note to end, set the gate to 0 yourself:
a.set(\gate, 0)
It's good to be aware of event environment variables because they're also used by Pbinds and by using them, you can effect other things. If you had a synthdef that used sustain as an argument for something else, you could be surprised by it changing your durations.
Regarding your last sub-question,
Also, is there a way to reference synths created by an event ? So that, when they have sustain: inf as argument, I can free them on a later time.
Yes, by "indexing" the Event by \id key. This actually returns an array of node ids because an Event with \strum can fire up more than one node/synth. Also, the \id value is nil while the event is not playing. But this indexing method is fairly unnecessary for what you want, because...
You can end the (associated) synth by ending the Event early with release, just like for the Synth itself. What this does is basically gate-out its internal synth. (In your example, this release call transitions to the release point of the ASR envelope generated by Linen, by lowering gate to 0.). And, of course, use a variable to save the "reference" to the synth and/or event, if don't plan to release it right away in a program (which would produce no sound with a gated envelope).
Basically
fork { var x = Synth(\testEvt); 2.wait; x.release }
does the same as
fork { var e = (instrument: \testEvt, sustain: inf).play; 2.wait; e.release }
except there's one layer of indirection in the latter case for the release. The first example is also equivalent to
fork { var x = Synth(\testEvt); 2.wait; x.set(\gate, 0); }
which does the work of release explicitly. Event also supports set and it passes the value to the corresponding Synth control (if the latter was properly added on the server.)
Now the complicated method you asked about (retrieving node ids for the event and sending them messages) is possible too... although hardly necessary:
fork { var e = (instrument: \testEvt, sustain: inf).play; 2.wait;
e[\id].do({ arg n; s.sendMsg("/n_set", n, "gate", 0); }) }
By the way, you can't use wait outside of a Routine, that's why fork was needed in the above examples. Interactively, in the editor, you can "wait manually", of course, before calling release on either the Synth or the Event.
As a somewhat subtle point of how envelope gating works, it doesn't actually start playing (technically begin transitioning to the endpoint of the first [attack] envelope segment) until you set gate to 1. I.e. you can delay the (envelope) start as in:
fork { x = Synth(\testEvt, [\gate, 0]); 3.wait; x.set(\gate, 1); 2.wait; x.release }
Beware that the default Event.play doesn't generate this 0 to 1 gate transition though, i.e. you can't rely on it to fire your synth's envelope if you set the initial gate value to zero in your SynthDef.
Also, I'm assuming that by "free" you mean "stop playing" rather than "free their memory on the server". There's no need to manually free those (event) synths in the latter sense since they have doneAction:2 in the envelope, which does that for you once they are released and the final segment of the envelope finishes playing. If you somehow want to kill the synth right away (like Ctrl+. does) instead of triggering its fade-out you can replace the message sent in the inner function of the "complicated" example (above) with s.sendMsg("/n_free", n). Or much more simply
fork { var e = (instrument: \testEvt, sustain: inf).play; 2.wait; e.free }
Also, if you wonder about \strum, an example is:
e = (instrument: \testEvt, sustain: inf, strum: 1, out: #[0, 0]).play
Now e[\id] is an array of two nodes. Event is a bit cheeky in that it will only create multiple nodes for arrays passed to actual Synth controls rather than random fields, so "strumming" \freq (or its precursors like \degree etc.) only creates multiple nodes if your SynthDesc has a freq control.
Alas the "complicated" method is almost useless when it comes to playing Pbinds (patterns). This is because Pbind.play returns and EventStreamPlayer... that alas makes a private copy of the prototype event being played and plays that private copy, which is inaccessible to the caller context (unless you hack EventStreamPlayer.prNext). Confusingly EventStreamPlayer has an accessible event variable, but that's only the "prototype", not the private copy event being played... So if p is an instance of an EventStreamPlayer then p.event[\id] is always nil (or whatever you set it to beforehand) even while playing. Since one seldom plays Events individually and much more often patterns...
Simply as a hacking exercise tough, it turns out there is an even more convoluted way to access the ids of nodes that EventStreamPlayer fires... This relies on overriding the default Event play which thankfully can be extended outside class inheritance because the default is conveniently saved in a class dictionary...
(p = Pbind(\instrument, \testEvt, \sustain, Pseq([1, 2]), \play, {
arg tempo, srv;
var rv;
"playhack".postln;
rv = Event.parentEvents[\default][\play].value(tempo, srv);
~id.postln;
rv;
}).play)
In general however, patterns are clearly not designed to be used this way, i.e. by hacking "a layer below" to get to the node ids. As "proof", while the above works well enough with Pbind (which uses the default Event type \note) it doesn't work reliably with Pmono which doesn't set the Event \id on its first note (Event type \monoNote) but only on subsequent notes (which generate a different Event type, \monoSet). Pmono keeps an internal copy of the node id, but this is completely inaccessible on the first mono note; it only copies it to the Events on the subsequent notes for some reason (bug perhaps, but could be "by design"). Also, if you use Pdef which extends Event with type \phrase... the above hack doesn't work all, i.e. \id is never set by type \phrase; perhaps you can get to the underlying sub-events generated somehow... I haven't bothered to investigate further.
The SC documentation (in the pattern guide) even says at one point
Remember that streams made from patterns don't expose their internals. That means you can't adjust the parameters of an effect synth directly, because you have no way to find out what its node ID is.
That's not entirely correct given the above hack, but it is true in some contexts.

sendto() dgrams do not block for ENOBUFS on OSX

This is more of a observation and also a suggestion for whats the best way to handle this scenario.
I have two threads one just pumps in data and another receives the data and does lot of work before sending it another socket. Both the threads are connected via a Domain socket. The protocol used here is UDP. I did not want to use TCP as it is stream based, which means if there is little space in the queue my data is split and sent. This is bad as Iam sending data that should not be split. Hence I used DGRAM. Interestingly when the send thread overwhelms the recv thread by pumping so much data, at some point the Domain socket buffer gets filled up and sendto() returns ENOBUFS. I was of the opinion that should this happen, sendto() would block until the buffer is available. This would be my desired behaviour. However this does not seem to be the case. I solve this problem in a rather weird way.
CPU Yield method
If I get ENOBUFS, I do a sched_yield(); as there is no pthread_yield() in OSX. After that I try to resend again. If that fails I keep doing the same until it is taken. This is bad as Iam wasting cpu cycles just doing something useless. I would love if sendto() blocked.
Sleep method
I tried to solve the same issue using sleep(1) instead of sched_yield() but this of no use as sleep() would put my process to sleep instead of just that send thread.
Both of them does not seem to work for me and Iam running out of options. Can someone suggest what is the best way to handle this issue? Is there some clever tricks Iam not aware of that can reduce unnecessary cpu cycles? btw, what the man page says about sentto() is wrong, based on this discussion http://lists.freebsd.org/pipermail/freebsd-hackers/2004-January/005385.html
The Upd code in kernel:
The udp_output function in /sys/netinet/udp_usrreq.c, seems clear:
/*
* Calculate data length and get a mbuf
* for UDP and IP headers.
*/
M_PREPEND(m, sizeof(struct udpiphdr), M_DONTWAIT);
if (m == 0) {
error = ENOBUFS;
if (addr)
splx(s);
goto release;
}
I'm not sure why sendto() isn't blocking for you... but you might try calling this function before you each call to sendto():
#include <stdio.h>
#include <sys/select.h>
// Won't return until there is space available on the socket for writing
void WaitUntilSocketIsReadyForWrite(int socketFD)
{
fd_set writeSet;
FD_ZERO(&writeSet);
FD_SET(socketFD, &writeSet);
if (select(socketFD+1, NULL, &writeSet, NULL, NULL) < 0) perror("select");
}
Btw how big are the packets that you are trying to send?
sendto() on OS X is really nonblocking (that is M_DONTWAIT flag for).
I suggest you to use stream based connection and just receive the whole data on the other side by using MSG_WAITALL flag of the recv function. If your data has strict structure than it would be simple, just pass the correct size to the recv. If not than just send some fixed-size control packet first with the size of the next chunk of data and then the data itself. On the receiver side you would be wait for control packet of fixed size and than the data of size from control packet.

Resources