This question is not about AU plugins, but about integrating audio units as building blocks of standalone application programs. After much trying I can't figure out what would be the simplest "graphless" connection of two AudioUnits, which would function as a "playthrough".
I understand how powerful and sufficient a single audio unit of subtype kAudioUnitSubType_HALOutput can be in capturing, rendering, live-processing and forwarding of any audio input data. However, play through seems functional as long as working with either full-duplex audio hardware or creating aggregate i/o device from built-in devices on user level.
However, built-in devices are not full duplex and "aggregating" them also has certain disadvantage. Therefore I've decided to study a hard-coded two-unit connection possibility (without plunging into Graph API), and test its behavior with non-full-duplex hardware.
Unfortunately, I have found neither comprehensive documentation nor example code for creating a simplest two-unit play through using only the straightforward connecting paradigm, as suggested in Apple Technical Note TN2091:
AudioUnitElement halUnitOutputBus = 1; //1 suggested by TN2091 (else 0)
AudioUnitElement outUnitInputElement = 1; //1 suggested by TN2091 (else 0)
AudioUnitConnection halOutToOutUnitIn;
halOutToOutUnitIn.sourceAudioUnit = halAudioUnit;
halOutToOutUnitIn.sourceOutputNumber = halUnitOutputBus;
halOutToOutUnitIn.destInputNumber = outUnitInputElement;
AudioUnitSetProperty (outAudioUnit, // connection destination
kAudioUnitProperty_MakeConnection, // property key
kAudioUnitScope_Input, // destination scope
outUnitInputElement, // destination element
&halOutToOutUnitIn, // connection definition
sizeof (halOutToOutUnitIn)
);
My task is to keep off involving Graphs if possible, or even worse, CARingBuffers from so-called PublicUtility, which used to be plagued by bugs and latency issues for years and involve some ambitious assumptions, such as:
#if TARGET_OS_WIN32
#include <windows.h>
#include <intrin.h>
#pragma intrinsic(_InterlockedOr)
#pragma intrinsic(_InterlockedAnd)
#else
#include <CoreFoundation/CFBase.h>
#include <libkern/OSAtomic.h>
#endif
Thanks in advance for any hint which may point me in the right direction.
Related
It is well known that ios_base::sync_with_stdio(false) will help the performance of cin and cout in <iostream> by preventing sync b/w C and C++ I/O. However, I am curious as to whether it makes any difference at all in <fstream>.
I ran some tests with GNU C++11 and the following code (with and without the ios_base::sync_with_stdio(false) snippet):
#include <fstream>
#include <iostream>
#include <chrono>
using namespace std;
ofstream out("out.txt");
int main() {
auto start = chrono::high_resolution_clock::now();
long long val = 2;
long long x=1<<22;
ios_base::sync_with_stdio(false);
while (x--) {
val += x%666;
out << val << "\n";
}
auto end = chrono::high_resolution_clock::now();
chrono::duration<double> diff = end-start;
cout<<diff.count()<<" seconds\n";
return 0;
}
The results are as follows:
With sync_with_stdio(false): 0.677863 seconds (average 3 trials)
Without sync_with_stdio(false): 0.653789 seconds (average 3 trials)
Is this to be expected? Is there a reason for a nearly identical, if not slower speed, with sync_with_stdio(false)?
Thank you for your help.
The idea of sync_with_stdio() is to allow mixing input and output to standard stream objects (stdin, stdout, and stderr in C and std::cin, std::cout, std::cerr, and std::clog as well as their wide character stream counterparts in C++) without any need to worry about characters being buffered in any of the buffers of the involved objects. Effectively, with std::ios_base::sync_with_stdio(true) the C++ IOStreams can't use their own buffers. In practice that normally means that buffering on std::streambuf level is entirely disabled. Without a buffer IOStreams are rather expensive, though, as they process individual character involving potentially multiple virtual function calls. Essentially, the speed-up you get from std::ios_base::sync_with_stdio(false) is allowing both the C and C++ library to user their own buffers.
An alternative approach could be to share the buffer between the C and C++ library facilities, e.g., by building the C library facilities on top of the more powerful C++ library facilities (before people complain that this would be a terrible idea, making C I/O slower: that is actually not true at all with a proper implementation of the standard C++ library IOStreams). I'm not aware of any non-experimental implementation which does use that. With this setup std::ios_base::sync_with_stdio(value) wouldn't have any effect at all.
Typical implementations of IOStreams use different stream buffers for the standard stream objects from those used for file streams. Part of the reason is probably that the standard stream objects are normally not opened using a name but some other entity identifying them, e.g., a file descriptor on UNIX systems and it would require a "back door" interface to allow using a std::filebuf for the standard stream objects. However, at least early implementations of Dinkumware's standard C++ library which shipped (ships?), e.g., with MSVC++, used std::filebuf for the standard stream objects. This std::filebuf implementation was just a wrapper around FILE*, i.e., literally implementing what the C++ standard says rather than semantically implementing it. That was already a terrible idea to start with but it was made worse by inhibiting std::streambuf level buffering for all file streams with std::ios_base::sync_with_stdio(true) as that setting also affected file streams. I do not know whether this [performance] problem was fixed since. Old issue in the C/C++ User Journal and/or P.J.Plauger's "The [draft] Standard C++ Library" should show a discussion of this implementation.
tl;dr: According to the standard std::ios_base::sync_with_stdio(false) only changes the constraints for the standard stream objects to make their use faster. Whether it has other effects depends on the IOStream implementation and there was at least one (Dinkumware) where it made a difference.
According to apple's docs it says:
Because your MIDIReadProc callback is invoked from a separate thread,
be aware of the synchronization issues when using data provided by
this callback.
Does this mean, use #synchronize to do thread blocking for safety?
Or does this literally mean synchronization timing issues may happen?
I am currently trying to read a midi file, and use a MIDIReadProc to trigger the note-on / note-off of a software synth based off of midi events. I need this to be extremely reliable and perfectly in-time. Right now, I am noticing that when I consume these midi events and write the audio to a buffer (all done from the MIDIReadProc), the timing is extremely sloppy and not sounding right at all. So I would like to know, what is the "proper" way to consume midi events from a MIDIReadProc?
Also, is a MIDIReadProc the only option for consuming midi events from a midi file?
Is there another option as far as setting up a virtual endpoint that could be directly consumed by my synthesizer? If so, how does that work exactly?
If you presume a function of this format to be the midiReadProc,
void midiReadProc(const MIDIPacketList *packetList,
void* readProcRefCon,
void* srcConnRefCon)
{
MIDIPacket *packet = (MIDIPacket*)packetList->packet;
int count = packetList->numPackets;
for (int k=0; k<count; k++) {
Byte midiStatus = packet->data[0];
Byte midiChannel= midiStatus & 0x0F;
Byte midiCommand = midiStatus >> 4;
//parse MIDI messages, extract relevant information and pass it to the controller
//controller must be visible from the midiReadProc
}
packet = MIDIPacketNext(packet);
}
the MIDI client has to be declared in the controller, interpreted MIDI events get stored into the controller from MIDI callback and read by the audioRenderCallback() on each audio render cycle. This way you can minimize timing imprecisions to the
length of the audio buffer, which you can negotiate during AudioUnit setup to be as short as the system allows for.
A controller can be a #interface myMidiSynthController : NSViewController you define, consisting of a matrix of MIDI channels and a pre-determined maximum-polyphony-per-channel, among other relevant data such as interface elements, phase accumulators for each active voice, AudioComponentInstance, etc... It would be wrong to resize the controller based on the midiReadProc() input. RAM is cheap nowadays.
I'm using such MIDI callbacks for processing live input from MIDI devices. Concerning playback of MIDI files, if you
want to process streams or files of arbitrary complexity, you may also run into surprises. MIDI standard itself
has timing features, which work as good as MIDI hardware allows for. Once you read an entire file into the memory, you can
translate your data into whatever you want and use your own code for controlling sound synthesis.
Please, observe not to use any code which would block the audio render thread (i.e. inside audioRenderCallback()), or would do memory management on it.
You could use AVAudioEngine.musicSequence and prepare your audio unit graph. Then use the MusicSequence API to load your GM file. Like this you don’t need to do the timing by yourself. Note I have not done this myself so far but I understand in theory it should work like this.
After you instantiate your synthesizer audio unit, you attach and connect it to the AVAudioEngine graph.
Does this mean, use #synchronize to do thread blocking for safety?
The opposite of what you’ve said is true: You should certainly not lock in a realtime thread. The #synchronized directive will lock if the resource is already locked. You may consider to use lock-free queues for realtime threads. See also Four common mistakes in audio development.
If you have to use CoreMIDI and MIDIReadProc, you can send MIDI commands to the synthesizer audio unit by calling MusicDeviceMIDIEvent right from your callback.
i have a program that generates both midi and wav audio. i need to be able to control the volume and balance of midi and audio separately and in theory, its seems like all i need to do is call
unsigned short left = (unsigned short)(wavvol*wavbal/100.0);
unsigned short right = (unsigned short)(wavvol*(100-wavbal)/100.0);
MMRESULT err = waveOutSetVolume(hWaveOut, left | (right<<16)); // for audio
and
unsigned short left = (unsigned short)(midivol*midibal/100.0);
unsigned short right = (unsigned short)(midivol*(100-midibal)/100.0);
MMRESULT err = midiOutSetVolume(s_hmidiout, left | (right<<16)); // for midi
for midi
the problem it, controlling midi volume sets wave volume and visa-verse, its like they are glues together inside windows
does anyone know if there is a way to unglue them?
BTW, i'm on windows 7, i know Microsoft messed up audio in win7. on XP i had an audio control panel with separate controls for midi and wave, that seems to have gone. i guess they just mix it down internally now and don't expose that even at the API level so maybe i've answered my own question.
still interested to know if there is a better answer through.
thanks, steve
I don't think they can be separated. You could move to the newer IAudioClient interface and use two WASAPI sessions to control the volume separately - one for wav and one for midi. This won't work on anything below Vista tho.
Alternatively you could track the volume levels in-code and as long as you don't play back both wav and midi at the same time reset them before playback.
Is there a "modern" replacement for the old Windows sndPlaySound() function, which was a very convenient way of playing a .wav file in the background while you focused on other matters? I now find myself needing to play an .mp3 file in the background and am wondering how to accomplish the same thing in a relatively easy way that the system supports inherently. Perhaps there's a COM component to acccomplish basic .mp3 playback?
Over years there have been a few audio and media related APIs and there are a few ways to achieve the goal.
The best in terms of absence of third party libs, best OS version coverage, feature set and simplicity is DirectShow API. 15 years old and still beats the hell out of rivals, supported in all versions of Windows that current and a few of previous versions of Visual Studio could target, except WinRT.
The code snippet below plays MP3 and WMA files. It is C++ however since it is all COM it is well portable across languages.
#include "stdafx.h"
#include <dshow.h>
#include <dshowasf.h>
#include <atlcom.h>
#pragma comment(lib, "strmiids.lib")
#define V(x) ATLVERIFY(SUCCEEDED(x))
int _tmain(int argc, _TCHAR* argv[])
{
static LPCTSTR g_pszPath = _T("F:\\Music\\Cher - Walking In Memphis.mp3");
V(CoInitialize(NULL));
{
CComPtr<IGraphBuilder> pGraphBuilder;
V(pGraphBuilder.CoCreateInstance(CLSID_FilterGraph));
CComPtr<IBaseFilter> pBaseFilter;
V(pBaseFilter.CoCreateInstance(CLSID_WMAsfReader));
CComQIPtr<IFileSourceFilter> pFileSourceFilter = pBaseFilter;
ATLASSERT(pFileSourceFilter);
V(pFileSourceFilter->Load(CT2COLE(g_pszPath), NULL));
V(pGraphBuilder->AddFilter(pBaseFilter, NULL));
CComPtr<IEnumPins> pEnumPins;
V(pBaseFilter->EnumPins(&pEnumPins));
CComPtr<IPin> pPin;
ATLVERIFY(pEnumPins->Next(1, &pPin, NULL) == S_OK);
V(pGraphBuilder->Render(pPin));
CComQIPtr<IMediaControl> pMediaControl = pGraphBuilder;
CComQIPtr<IMediaEvent> pMediaEvent = pGraphBuilder;
ATLASSERT(pMediaControl && pMediaEvent);
V(pMediaControl->Run());
LONG nEventCode = 0;
V(pMediaEvent->WaitForCompletion(INFINITE, &nEventCode));
}
CoUninitialize();
return 0;
}
If you are playing your own files you are sure to not contain large ID3 tag sections, the code might be twice as short.
A simple answer to a lot of problems like this is to simply call out to a command line program with system("play.exe soundfile.mp3") or equivalent. Just treat the command line as another API, an API that is has extensive functionality and is standard, portable, flexible, easy to debug and easy to modify. It may not be as efficient as calling a library function but that often doesn't matter, particularly if the program being called is already in the disk cache. Incidentally, avoid software complexity just because it's "modern"; often that's evidence of an architecture astronaut and poor programming practice.
When you say "Modern", do you mean a Windows 8 WinRT API? Or do you mean, "an API slightly newer than the ones invented for Windows 3.1"?
A survey of audio and video apis can be found here
For classic Windows desktop applications, there's PlaySound, which can play any WAV file.
For MP3, my team invented a solution using DirectSound and the Windows Media Format SDK. The latter can decode any WMA and MP3 file. We fed the audio stream directly into a DSound buffer. This is not for the faint of heart.
You could likely use the higher level alternative, the Windows Media Player API.
DirectShow is a very legacy alternative, but is easy to get something up and working. Media Foundation is the replacement for DirectShow.
Is there a way to programmatically detect whether the microphone is on on Windows?
No, microphones don't tell you whether they're ‘on’ or that a particular sound channel is connected to a microphone device. The best you can do is to read audio data from the input channel you suspect to be a microphone (eg. the Windows default input device/channel), and see if there's any signal on it.
To do that you'd have to remove any DC offset and look for any signal above a reasonable noise floor. (Be generous: many cheap audio input devices are quite noisy even when there is no signal coming in. A mid-band filter/FFT would also be useful to detect only signals in the mid-range of a voice and not low-frequency hum and transient clicks.)
This is not tested in any way, but I would try to read some samples and see if there is any variation. If the mike is on then you should get different values from the ambient sounds. If the mike is off you should get a 0. Again this is just how I imagine things should work - I don't know if they actually work that way.
Due to a happy accident, I may have discovered that yes there is a way to detect the presence of a connected microphone.
If your windows "recording devices" shows "no microphone", then this approach (using the Microsoft Speech API) will work and confirm you have no mic. If windows however thinks you have a mic, this won't disagree.
#include <sapi.h>
#include <sapiddk.h>
#include <sphelper.h>
CComPtr<ISpRecognizer> m_cpEngine;
m_cpEngine.CoCreateInstance(CLSID_SpInprocRecognizer);
CComPtr<ISpObjectToken> pAudioToken;
HRESULT hr = SpGetDefaultTokenFromCategoryId(SPCAT_AUDIOIN, &pAudioToken);
if (FAILED(hr)) ::OutputDebugString("no input, aka microphone, detected");
more specifically, hr will return this result:
SPERR_NOT_FOUND 0x8004503a -2147200966
The requested data item (data key, value, etc.) was not found.