controlling an AudioUnit with MIDI keyboard OSX - macos

I'd like to learn if a simple CoreAudio component (of subtype kAudioUnitSubType_HALOutput, e.g.), can be parametrically controlled by a MIDI keyboard, let's say MIDI note number be translated into interpollating oscillator frequency? On the other hand, controlling such a parameter by means of a GUI element works like a dream.
I found no single example of such a code on the entire web.
I don't need SinSynth, Sampler, MusicDevice, SoundFonts, Midi files, GM, ADSRs, plug-in level of functionality, etc.
Just need a plain piece of information or hint on how can data from a MIDI packet read by means of a midiReadProc get passed to a audio render callback, much like values of a slider can. With MIDI there seems to be a threading issue, I found no documentation about.
I'd prefer to do it in CoreAudio API, if possible, I'm sure it must be.
On the other hand, using Apple pre-built music instrument devices would lead me into a completely wrong direction.
Thanks in advance,
CA

It seems you want to controls some of parameters or properties of AudioUnit using MIDI-keyboard. In this case all that you need is take MIDIPacket's data field.
What every byte mean you can look here.
After that, depending on value of needed byte, you need to set property or parameter value.

Here's a minimalistic answer to the question that I've learned and made work meanwhile. It's a matter of making a midiReadProc generate values which an audioRenderProc can accept as parameters. Please note that this works in stand-alone apps. For writing AU-plug-ins I recommend understanding and using CoreAudioUtilityClasses, as provided by Apple.
A simplest example of createMidi in C:
//these have to be declared somewhere
MIDIClientRef midiclient;
MIDIPortRef midiin;
void createMIDI (void)
{
//create MIDI input and client - - - - - - - - - - -
midiclient = 0;
CheckError(MIDIClientCreate(CFSTR("MIDI_Client"),
NULL,
/*midiClientNotifyRefCon*/NULL,
&midiclient),
"MIDI Client Create Error\n");
CheckError(MIDIInputPortCreate(midiclient,
CFSTR("MIDI_Input"),
midiReadProc,
NULL,
&midiin),
"MIDI Port Create Error\n");
//connect MIDI - - - - - - - - - - - - - - - - - - -
ItemCount mSrcs = MIDIGetNumberOfSources();
printf("MIDI Sources: %ld\n", (long)mSrcs);
ItemCount iSrc;
for (iSrc=0; iSrc<mSrcs; iSrc++) {
MIDIEndpointRef src = MIDIGetSource(iSrc);
MIDIPortConnectSource(midiin, src, NULL);
}
}
CheckError( ) is a generic utility function modeled after "Learning Core Audio", by C.Adamson & K.Avila, ISBN 0-321-63684-8...
...and a plain-C midiReadProc template. Please note that many manufacturers of MIDI hardware don't implement the standardized noteOff event, but rather a "hacked" version consisting of a zero-velocity-noteOn, due to alleged improving MIDI-latency issues, but they hardly document it. So, one has to check against both scenarios:
void midiReadProc(const MIDIPacketList *packetList,
void* readProcRefCon,
void* srcConnRefCon)
{
Byte note;
Byte velocity;
MIDIPacket *packet = (MIDIPacket*)packetList->packet;
int count = packetList->numPackets;
for (int k=0; k < count; k++) {
Byte midiStatus = packet->data[0];
Byte midiChannel = midiStatus & 0x0F;
Byte midiCommand = midiStatus >> 4;
if ((midiCommand == 0x08)||(midiCommand == 0x09)){
if(midiCommand == 0x09){
note = packet->data[1] & 0x7F;
velocity = packet->data[2] & 0x7F;
if (velocity == 0x0){ //"hacked" note-off
; //do something
}else{//note on
; //do something
}
}
if(midiCommand == 0x08){ //proper note-off
;//do something
}
}else{
;//do something else
}
packet = MIDIPacketNext(packet);
}//end for (k = 0; ...;...)
}
Everything else is a matter of common good programing practice.

Related

MultiRoute Audio Input in iOS

We've been working with AudioUnits in Core Audio. It is simultaniously a very powerful audio framework, and one of the worst documented which makes it both a joy and a frustration to work with.
We want to accomplish something we know iPads had been able to do since iOS 6.0 - Multiple audio inputs.
So far - from the 2012 Developer Talk - It appears you have to set the audio session to MultiRoute. We've done this. If I plug in an a soundcard from a keyboard. I can see that there are two inputs. Great. We're then told that we need to set a ChannelMap on a Remote I/O unit.
To what? Well... here's where it gets vague. We need to set all the channels we don't want to -1 and the channels we want to 0 and 1 (for stereo input or for mono?).
We attempt this and... nothing. Sound still plays through on the 'last in wins' principle. Microphone if everything plugged out, soundcard if that's the one plugged in. But we can't switch between them.
This setup code is always run before the other function listed
func setupAudioSession() {
self.audioSession = AVAudioSession.sharedInstance()
do {
try audioSession.setCategory(AVAudioSessionCategoryMultiRoute, with: [.mixWithOthers])
try audioSession.setActive(true)
audioSessionWasSetup = true
} catch let error {
//TODO: Implement something here
print(error)
audioSessionWasSetup = false
}
}
We then have a remote I/O with an associated audiograph set up. This has been tested and works beautifully. But we need to be able to set where it's pulling sound from.
I've attempted to do it with this, but not only doesn't it have any effect... nothing happens.
Am I missing something?
private func setChannelMap(onAudioUnit audioUnit: AudioUnit?, toChannel channelIndex: Int = 0) {
var channelMap: [Int32] = []
if audioUnit == nil {
return
}
var numberOfInputChannels: UInt32 = 4 // Two stereo inputs? - I'm just guessing here
let mapSize: UInt32 = numberOfInputChannels * UInt32(MemoryLayout<Int32>.size);
for _ in 0...(numberOfInputChannels) {
channelMap.append(-1)
}
channelMap[2 * channelIndex] = 0;
channelMap[2 * channelIndex + 1] = 1;
let status = AudioUnitSetProperty(audioUnit!,
kAudioOutputUnitProperty_ChannelMap,
kAudioUnitScope_Input,
0,
&channelMap,
mapSize);
self.checkError(status, "Failed to set Channel Map on input unit")
}
There isn't any documentation on this at all as far as I've been able to find. Nor any code examples.
I hope you can help us.

FMOD Ex dropping sounds, eventually going silent

I'm attempting to port an old open-source FMOD 3 game (Candy Crisis) to the latest version of FMOD Ex 4 on OS X. Its sound needs are very simpleā€”it plays WAVs, sometimes changing their frequency or speaker mix, and also plays MOD tracker music, sometimes changing the speed. I'm finding that the game works fine at first, but over the course of a few minutes, it starts truncating sounds early, then the music loses channels and eventually stops, then over time all sound ceases. I can cause the problem to reproduce more quickly if I lower the number of channels available to FMOD.
I can get the truncated/missing sounds issue to occur even if I never play a music file, but music definitely seems to make things worse. I have also tried commenting out the code which adjusts the sound frequency and speaker mix, and that was not the issue.
I am calling update() every frame.
Here's the entirety of my interactions with FMOD to play WAVs:
void InitSound( void )
{
FMOD_RESULT result = FMOD::System_Create(&g_fmod);
FMOD_ERRCHECK(result);
unsigned int version;
result = g_fmod->getVersion(&version);
FMOD_ERRCHECK(result);
if (version < FMOD_VERSION)
{
printf("Error! You are using an old version of FMOD %08x. This program requires %08x\n", version, FMOD_VERSION);
abort();
}
result = g_fmod->init(8 /* was originally 64, but 8 repros the issue faster */, FMOD_INIT_NORMAL, 0);
FMOD_ERRCHECK(result);
for (int index=0; index<kNumSounds; index++)
{
result = g_fmod->createSound(QuickResourceName("snd", index+128, ".wav"), FMOD_DEFAULT, 0, &s_sound[index]);
FMOD_ERRCHECK(result);
}
}
void PlayMono( short which )
{
if (soundOn)
{
FMOD_RESULT result = g_fmod->playSound(FMOD_CHANNEL_FREE, s_sound[which], false, NULL);
FMOD_ERRCHECK(result);
}
}
void PlayStereoFrequency( short player, short which, short freq )
{
if (soundOn)
{
FMOD::Channel* channel = NULL;
FMOD_RESULT result = g_fmod->playSound(FMOD_CHANNEL_FREE, s_sound[which], true, &channel);
FMOD_ERRCHECK(result);
result = channel->setSpeakerMix(player, 1.0f - player, 0, 0, 0, 0, 0, 0);
FMOD_ERRCHECK(result);
float channelFrequency;
result = s_sound[which]->getDefaults(&channelFrequency, NULL, NULL, NULL);
FMOD_ERRCHECK(result);
result = channel->setFrequency((channelFrequency * (16 + freq)) / 16);
FMOD_ERRCHECK(result);
result = channel->setPaused(false);
FMOD_ERRCHECK(result);
}
}
void UpdateSound()
{
g_fmod->update();
}
And here's how I play MODs.
void ChooseMusic( short which )
{
if( musicSelection >= 0 && musicSelection <= k_songs )
{
s_musicChannel->stop();
s_musicChannel = NULL;
s_musicModule->release();
s_musicModule = NULL;
musicSelection = -1;
}
if (which >= 0 && which <= k_songs)
{
FMOD_RESULT result = g_fmod->createSound(QuickResourceName("mod", which+128, ""), FMOD_DEFAULT, 0, &s_musicModule);
FMOD_ERRCHECK(result);
result = g_fmod->playSound(FMOD_CHANNEL_FREE, s_musicModule, true, &s_musicChannel);
FMOD_ERRCHECK(result);
EnableMusic(musicOn);
s_musicModule->setLoopCount(-1);
s_musicChannel->setPaused(false);
musicSelection = which;
s_musicPaused = 0;
}
}
If someone wants to experiment with this, let me know and I'll upload the project somewhere. My gut feeling is that FMOD is busted but I'd love to be proven wrong.
Sounds like your music needs to be set as higher priority than your other sounds. Remember, lower numbers are more important. I think you can just set the priority on the channel.
Every time I play the following WAV, FMOD loses one channel permanently. I am able to reproduce this channel-losing behavior in the "playsound" example if I replace the existing jaguar.wav with my file.
https://drive.google.com/file/d/0B1eDRY8sV_a9SXMyNktXbWZOYWs/view?usp=sharing
I contacted Firelight and got this response. Apparently WAVs can include a looping command! I had no idea.
Hello John,
I've taken a look at the two files you have provided. Both files end
with a 2 sample infinite loop region.
FMOD 4 (and FMOD 5 for that matter) will see the loop region in the
file and automatically enable FMOD_LOOP_NORMAL if you haven't
specified any loop mode. Assuming you want one-shot behavior just pass
in FMOD_LOOP_OFF when you create the sound.
Kind regards, Mathew Block | Senior Platform Engineer
Technically this behavior contradicts the documented behavior of FMOD_DEFAULT (which is specified to imply FMOD_LOOP_OFF) so they are planning to improve the documentation here.
Based on the wave sample you supplied, FMOD is behaving correctly as it appears you've figured out. The sample has a loop that is honored by FMOD and the last samples are simply repeated forever. While useless, this is correct and the variance in the samples is so slight as to not be audible. While not part of the original spec for wave format, extended information was added later to support meta data such as author, title, comments and multiple loop points.
Your best bet is to examine all your source assets for those that contain loop information. Simply playing all sounds without loop information is probably not the best workaround. Some loops may be intentional. Those that are will have code that stops them. Typically, in a game, the entire waveform is looped when looping is desired. You can then write or use a tool that will strip the loop information. If you do write your own tool, I'd recommend resampling the audio to the native output sampling rate of the hardware. You'd need to insure your resampler was sample accurate (no time shift) and did not introduce noise.
Historically, some game systems had a section at the end of the sound with silence and a loop point set on this region. The short reason for this was to reduce popping that might occur at the end of a sound in a hardware audio channel.
Curiosly, the last 16 samples of your .wav look like garbage and I'm wondering if the .wav assets you're using were converted from a source meant for a game console and that's where the bogus loop information came from as well.
This would have been a comment but my lowly rep does not allow it.

How to change the user's input method (eg. U.S. vs. Arabic) on OS X?

Writing a vocabulary trainer for Arabic, my program shall automatically switch the input method between the user's default language and arabic.
On Windows, this can be done with LoadKeyboardLayout and ActivateKeyboardLayout.
Is there an API call to accomplish the same on OS X?
It's okay to require the user to enable the Arabic kbd layout once himself via the System prefs, but him having to switch back and forth manually every time is a nuisance I want to avoid.
Or, alternatively, can I assign the preferred input method to a Carbon text input field so that I could use two input fields, one for U.S., one for Arabic, that way?
You need the Carbon api, look at Text Input Source Services Reference
Here is a sample code without error handling (Mac OS X v10.5 and later)
CFStringRef tID = CFSTR("com.apple.keylayout.Arabic"); // or "com.apple.keylayout.US"
TISInputSourceRef inputSource = NULL;
CFArrayRef allInputs = TISCreateInputSourceList(NULL, true);
NSUInteger count = CFArrayGetCount(allInputs);
for (int i = 0; i < count; i++) {
inputSource = (TISInputSourceRef)CFArrayGetValueAtIndex(allInputs, i);
if (!CFStringCompare(tID, TISGetInputSourceProperty(inputSource, kTISPropertyInputSourceID), 0)) {
TISEnableInputSource(inputSource);
TISSelectInputSource(inputSource);
break;
}
}
CFRelease(allInputs);

How to get correct hDevMode values from CPrintDialogEx (PrintDlgEx)?

I'm displaying a CPrintDialogEx dialog to choose a printer and modify the settings. I set the hDevNames member so that a default printer will be selected, but I leave hDevMode set to NULL. On successful return I pull some values such as paper size out of the returned DEVMODE structure from hDevMode.
I'm having a problem because hDevMode appears to be initialized with the values from the default printer that I passed in, not the printer that was finally selected. How do I get the parameters from the actual selected printer?
As requested here's the relevant part of the code. I've deleted some of it in the interest of space. TOwnedHandle is a smart pointer I wrote for holding a memory handle and locking it automatically.
CPrintDialogEx dlg(PD_ALLPAGES | PD_NOCURRENTPAGE | PD_NOPAGENUMS | PD_NOSELECTION, this);
ASSERT(dlg.m_pdex.hDevMode == NULL);
ASSERT(dlg.m_pdex.hDevNames == NULL);
dlg.m_pdex.hDevNames = GlobalAlloc(GHND, sizeof(DEVNAMES) + iSizeName);
DEVNAMES * pDevNames = (DEVNAMES *) GlobalLock(dlg.m_pdex.hDevNames);
// ...
GlobalUnlock(dlg.m_pdex.hDevNames);
if ((dlg.DoModal() == S_OK) && (dlg.m_pdex.dwResultAction == PD_RESULT_PRINT))
{
TOwnedHandle<DEVMODE> pDevMode = dlg.m_pdex.hDevMode;
TRACE("Printer config = %dx%d %d\n", (int)pDevMode->dmPaperWidth, (int)pDevMode->dmPaperLength, (int)pDevMode->dmOrientation);
// ...
}
Edit: I've determined that I don't get the problem if I don't set the hDevNames parameter. I wonder if I've discovered a Windows bug? This is in XP, I don't have a more recent version of Windows handy to test with.
I've distilled the code into a test that doesn't use MFC, this is strictly a Windows API problem. This is the whole thing, nothing left out except the definition of pDefaultPrinter - but of course it doesn't do anything useful anymore.
PRINTDLGEX ex = {sizeof(PRINTDLGEX)};
ex.hwndOwner = m_hWnd;
ex.Flags = PD_ALLPAGES | PD_NOCURRENTPAGE | PD_NOPAGENUMS | PD_NOSELECTION;
ex.nStartPage = START_PAGE_GENERAL;
#if 1
int iSizeName = (strlen(pDefaultPrinter) + 1) * sizeof(char);
ex.hDevNames = GlobalAlloc(GHND, sizeof(DEVNAMES) + iSizeName);
DEVNAMES * pDevNames = (DEVNAMES *) GlobalLock(ex.hDevNames);
ASSERT(pDevNames != NULL);
pDevNames->wDeviceOffset = sizeof(DEVNAMES);
strcpy((char *)pDevNames + pDevNames->wDeviceOffset, pDefaultPrinter);
GlobalUnlock(ex.hDevNames);
#endif
HRESULT hr = PrintDlgEx(&ex);
if ((hr == S_OK) && (ex.dwResultAction == PD_RESULT_PRINT))
{
DEVMODE * pdm = (DEVMODE *) GlobalLock(ex.hDevMode);
ASSERT(pdm != NULL);
TRACE("Printer config = %dx%d %d\n", (int)pdm->dmPaperWidth, (int)pdm->dmPaperLength, (int)pdm->dmOrientation);
GlobalUnlock(ex.hDevMode);
DEVNAMES * pdn = (DEVNAMES *) GlobalLock(ex.hDevNames);
ASSERT(pdn != NULL);
TRACE(_T("Printer device = %s\n"), (char *)pdn + pdn->wDeviceOffset);
GlobalUnlock(ex.hDevNames);
}
If I can't get a fix, I'd love to hear of a work-around.
After much head scratching I think I've figured it out.
When the dialog comes up initially, the hDevMode member gets filled with the defaults for the printer that is initially selected. If you select a different printer before closing the dialog, that DEVMODE structure is presented to the new printer driver; if the paper size doesn't make sense to the driver it may change it, and the drivers are not consistent.
The reason this tripped me up is that I was switching between three printers: two label
printers with very different characteristics, and a laser printer with US Letter paper.
The laser printer always responds with the proper dimensions but may indicate a wrong paper size code.
The first label printer will override the size provided by the laser printer but not the other label printer.
The second label printer will accept the size provided by the first label printer, because it's capable of using that size even though it's not loaded and not configured. It modifies the size provided by the laser printer by returning the maximum width and the Letter size length of 11 inches.
I determined two ways to work around the problem. The first is to implement IPrintDialogCallback and respond to SelectionChange calls by reloading the default DEVMODE for the newly selected printer. EDIT: I tried this and it does not work. CPrintDialogEx already implements an IPrintDialogCallback interface, making this easy. It appears that PrintDlgEx has its own internal handle that it uses to track the current DEVMODE structure and only uses the one in the PRINTDLGEX structure for input/output. There's no way to affect the DEVMODE while the dialog is up, and by the time it returns it's too late.
The second solution is to ignore the returned results entirely and work from the default paper configuration for the printer. Any changes made from the printer defaults within the dialog are lost completely, but for my application this is acceptable.
bool MyDialog::GetPaperSize(const TCHAR * pPrinterName, double & dPaperWidth, double & dPaperLength)
{
// you need to open the printer before you can get its properties
HANDLE hPrinter;
if (OpenPrinter((TCHAR *)pPrinterName, &hPrinter, NULL))
{
// determine how much space is needed for the DEVMODE structure by the printer driver
int iDevModeSize = DocumentProperties(m_hWnd, hPrinter, (TCHAR *)pPrinterName, NULL, NULL, 0);
ASSERT(iDevModeSize >= sizeof(DEVMODE);
// allocate a DEVMODE structure and initialize it to a clean state
std::vector<char> buffer(iDevModeSize, 0);
DEVMODE * pdm = (DEVMODE *) &buffer[0];
pdm->dmSpecVersion = DM_SPECVERSION;
DocumentProperties(m_hWnd, hPrinter, (TCHAR *)pPrinterName, pdm, NULL, DM_OUT_BUFFER);
ClosePrinter(hPrinter);
// convert paper size from tenths of a mm to inches
dPaperWidth = pdm->dmPaperWidth / 254.;
dPaperLength = pdm->dmPaperLength / 254.;
return true;
}
return false;
}

Cocoa NSOutputStream send to a connection

I am new to Cocoa, but managed to get a connection (to a FTP) up and running, and I've set up an eventhandler for the NSInputStream iStream to alert every response (which also works).
What I manage to get is simply the hello message and a connection timeout 60 sec, closing control connection.
EDIT: I guess my question is "without closing->opening what would be an non-terminating way of flushing the outputStream?
After searching stackoverflow and finding a lot of NSOutputStream write problems (e.g. How to use NSOutputStream's write message?) and a lot of confusion in my google hits, I figured I'd try to ask my own question:
I've tried reading the developer.apple.com doc on OutputStream, but it seems almost impossible for me to send some data (in this case just a string) to the "connection" via the NSOutputStream oStream.
- (IBAction) send_something: sender
{
const char *send_command_char = [#"USER foo" UTF8String];
send_command_buffer = [NSMutableData dataWithBytes:send_command_char length:strlen(send_command_char) + 1];
uint8_t *readBytes = (uint8_t *)[send_command_buffer mutableBytes];
NSInteger byteIndex = 0;
readBytes += byteIndex;
int data_len = [send_command_buffer length];
unsigned int len = ((data_len - byteIndex >= 1024) ?
1024 : (data_len-byteIndex));
uint8_t buf[len];
(void)memcpy(buf, readBytes, len);
len = [oStream write:(const uint8_t *)buf maxLength:len];
byteIndex += len;
}
the above seems not to result in any useable events. typing it under NSStreamEventHasSpaceAvailable sometimes give a response if I spam the ftp by keep creating new connection instances and keep sending some command whenever oStream has free space. In other words, nothing "right" and so I'm still unclear how to properly send a command to the connection. Should I open -> write -> close every time i want to write to oStream (and thus to the ftp) and can I then expect a reply (hasBytesAvailable event on iStream)?
EDIT: Doesn't look like it no.
For some reason I find it very difficult to find any clear tutorials on this matter. Seems like there are more than a few in the same position as me: unclear how to use oStream write?
Please! Any little bit that can help clear this up is greatly appreciated!
If needed I can write the rest of the code.
Chuck
Okay, so 10 hours 28 views and no answers/comments, but that's OK, because I just solved it with some good help from a very very friendly irssi coder (no butt licking intended ;)).
He proposed that I tried to std::endl'e it (newline + flush), and so I tried simply adding the newline char (\n, 0x0A) and it worked perfectly!

Resources