I'm using WebRTC (Native Mac & Native Windows -- not JS) and am trying to change the default playout and recording devices and am having a lot of trouble. This is starting to drive me nuts since it should be very simple.
Question: What's the recommended way to change audio playout and recording devices while a call is ongoing on Mac & Windows, natively?
Here's what I've tried:
Method 1
Mac
I noticed the audio device module listens to Core Audio API notifications and adjusts playout and recording devices properly. This works, but I'm not sure if this is the recommended way to change devices.
Windows
I was not able to find a system-wide way of setting the default audio playout/recording device. The only way I could tell MIGHT work is by getting a reference to the audio device module and calling SetPlayoutDevice / SetRecordingDevice on it manually...which leads to Method 2 below:
Method 2
Mac
If possible, I'd rather use SetPlayoutDevice (link) / SetRecordingDevice (link) to change the audio input/output (so Mac & Windows work the same way).
The unit tests to test real audio IO devices shows we should be able to call StartPlayout and StopPlayout after a call to SetPlayoutDevice -- but this makes my app freeze. I've tried it without the call to StopPlayout and StartPlayout however it doesn't seem to do anything. This makes sense since it looks like only internal state is modified but nothing is modified.
Q: How can I change the default audio playout device and recording device on Mac?
Windows
I haven't had a chance to try this out on Windows yet, but Mac not working makes me think there's something I'm missing here.
Answering this myself.
VoEHardwareImpl (https://chromium.googlesource.com/external/webrtc/stable/webrtc/+/refs/heads/master/voice_engine/voe_hardware_impl.cc) seems to have some relevant code.
For playout:
StopPlayout
...set device index...
StereoPlayoutIsAvailable
SetStereoPlayout
InitPlayout
StartPlayout
For recording:
StopRecording
...set device index...
StereoRecordingIsAvailable
SetStereoRecording
InitRecording
StartRecording
Depending on the commit you're on, you might need to ensure you won't get into a deadlock situation. Some of these methods acquire locks so make sure the lock isn't acquired yet if you're calling a method that requires locks. Better yet -- do this a level above or wrap the adm if possible.
Related
I would like to know whether SoX/LibSoX offers the possibility to access a sound device in exclusive/hog mode. The idea is to prevent other applications from accessing the sound card / DAC that is being used by the focal app.
My main target is OSX CoreAudio output, but I am also eager to know about Linux (OSS/Alsa).
I know this is possible in CoreAudio, because I have seen it implemented in several apps, including this open source one.
On Mac OS X at least, the answer appears to be no. In http://sourceforge.net/p/sox/code/ci/master/tree/src/coreaudio.c SoX uses the default input or output device but there is no provision for hog mode.
I want to create a virtual audio device that gets audio data from the default output (which is an output IOAudioStream) and converts it to an input IOAudioStream.
I went through most of the examples I could find, however they only implement a feature to copy the output IOAudioStream to the input one at most. That means it only converts the audio to an input stream if the audio device is selected as output.
This should be possible, since ScreenFlow allows recording of computer audio by installing a kext that creates a virtual driver.
How can I access the audio data from the default output and send it to my virtual driver?
Take a look at the open-source WavTap, which is a simplified fork of the open-source SoundFlower virtual sound card driver. It is a .kext that I believe does substantially what you want.
For reference, here is how some popular commercial closed-source options work:
Rogue Amoeba's Audio Hijack Pro
-Captures system audio via code based off of the open-source SoundFlower .kext
-Captures an application's audio by substituting a "patch" framework for the normal CoreAudio.framework when launching the application
-Captures an already-running application's audio with the help of the haxie "Application Enhancer" (APE) from Unsanity
These features are branded as their "Instant On" feature (InstantOn.kext).
Ambrosia Software's WireTap Studio
-Captures system audio and application audio via an in-house developed .kext
Telestream's ScreenFlow
-Captures system audio via an in-house developed .kext. (Version 2.x uses varaudio.kext; Version 3.x uses TelestreamAudio.kext)
Macsome's Audio Recorder
-Unknown method
Araelium Group's Screenflick
-Captures system audio using the SoundFlower .kext
UPDATE #1
After reading the author's comments, it appears the underlying goal is to be able to capture the system sound without publishing the virtual audio driver as a device (that would appear in the System Preference's list) and without changing the current default output device (or at least the appearance that the device has changed).
SoundFlower: Adds a sound device to the list upon installation
WavTap: Adds a sound device to the list upon installation; auto-selects the device when the WavTap application is started; auto-deselects the device when the application is shutdown and reselects the previous device
Audio Hijack Pro: Adds a sound device only when audio capture of the default system sound is selected; removes the sound device when audio capture is no longer selected and reselects the previous device
WireTap Studio: Unknown
ScreenFlow: Captures the system sound without changing the current default output device and without publishing the virtual audio driver as a device
UPDATE #2
A quote from Jeff Moore, a CoreAudio Apple engineer, in reference to applications such as WireTap and Audio Hijack Pro:
"There are no APIs on the system that will give you the output of any specific app or the whole mix going to the hardware...[Capturing System Sound] isn't supported by the System and those folks had to be clever. There's nothing stopping you from doing the same thing except how willing you are to get your hands dirty.
The fact is, Mac OS X's audio system was designed first and foremost for performance. This lead us to a design where it is not easy to support the functionality you want without imposing performance penalties. So, we have opted for better performance at the cost of not being able to provide this feature."
If you want to read more on the subject, check out these threads on the CoreAudio API mailing list:
"WireTap, CoreAudio's API, and system capture, and kexts..."
"Another question on capturing audio played back by a software"
"Capturing currently played audio using CoreAudio on Mac"
"'audio hijack'"
"monitoring system audio output like wire tap"
"Capturing audio output to a file"
"Mirroring Audio Output"
"Recording system audio"
Relevant SO Questions:
Hide Audio device using codeless kext
So long story short, you're not likely to find examples from Apple that accomplish this, and you're not likely to find open source code that accomplishes this either, unless someone is feeling very generous. It appears to be too valuable of information.
After additional research, here are some theoretical techniques I came up with that might allow you to accomplish your goal:
Similar to Prosoft Engineering's Hear product, you could create a HAL plugin (user-mode virtual driver) rather than a .kext (kernel-mode virtual driver). Apple has a sample HAL plugin called "SampleHardwarePlugIn" and PulseAudio has one as well. However, with his method I don't think you get access to a pre-mixed system sound stream. You would have to gather up all streams from the various applications (which must use CoreAudio to play sound) and mix them together for pseudo system sound capture.
Create a virtual audio device that is hidden [1][2] from user interaction. When the user wishes to capture the default sound, programmatically create an aggregate device that includes your hidden virtual device and the current default sound device. Temporarily set this aggregate device as the default output. In this manner, you are able to both capture the default sound and hear it.
Side Note: If Mac OS X allows for a hidden device to also be set as the default output device, what would System Preferences show as the selected device? If it instead shows the secondary output device as selected, then you have the added allusion that nothing has changed.
A newer open-source virtual audio device that works with the latest versions of MacOS is BlackHole - it supports multiple audio channels and sampling rates.
It can be used as an audio sink and/or source. It's also handy as part of an aggregate audio device so audio can be heard and re-routed - e.g. using the MacOS Audio MIDI Setup app
I have been searching through how to do gamepad and joystick support on Mac for some days and all resources that I found seems to suggest a pre-installed driver along with using Apple's HID API, which works.
The drawback about this approach is that each joystick and gamepad will require another kernel extension to be loaded, so it can be recognized by HID manager, or at least a code less Info.plist saying it conforms to the earlier installed driver. For instance, when I have an 360 Xbox driver KEXT in house, the Xbox controller from Microsoft will work, but not the Logitech one (I tried F710).
As Apple suggests the application that uses a gamepad or joystick should be able to do themselves at user space without introducing any KEXT stuff. Is there a way to do it?
The thing I had in mind was something like using IORegistry or IOUSB API to get the device when they get plugged in (USB Prober shows it at least). Then somehow get the description of the device, then use that description to register the device as a HID one. Then the whole HID manager can be used.
Am I on the right track? Or is there any other way to do this?
Since IOKit API actually provided keywords like kHIDUsage_GD_Joystick, and there's an ForceFeedback.h library, I suppose Apple designed their HID API with joystick and force feedback in mind. That's the slim hope I had that this might work.
Some reference documentation and open source project:
Colin Munro's 360 driver
HID API Documents
DDHID Project
After revisiting this, I found out the solution to be operating directly on the file descriptors of the device. libusb is an excellent library which greatly simplify your life on this and they have Mac supported.
xboxdrv link is a great example on how to operate on file socket using libusb.
In pseudo code, it should look like this:
enumerate device
detect kernel driver and detach it if possible
open device
file off a initial transfer
wait on the callback function to handle msg and error properly
start run loop or select on fd to call libusb_event_handle
Check libusb for more info link.
I wonder if anyone can help at all, a bit of a specialist problem this.
I have an application that needs to read and analyse a number of USB devices (not simultaneously, they are each run in seperate tests and could in theory be run on different machines).
Each of the USB devices is based on the USB HID class, and are manufactured by different companies, none of these USB devices are designed to be run on PC, but are meant for a different platform, however for the purposes of testing the devices the client has requested that the test application is run from a PC.
Some of the devices will start up, be recognised by windows which will initialise and start them correctly using the generic HID class driver built into windows, the devices will then start sending correct data packets of the data to be tested.
Some of the devices will start up, be recognised by windows which will try to start them but fail to fully to initialise them leaving them in a half initialised state. This is fine, as I can use my beagle protocol analyser to capture the initialisation packets from the genuine platform and then use the LibUSBDotNet library to replicate the remaining packets in the initialisation sequence and get them to start sending the packets correctly.
The problem I have is with one particular device (though there are some more I haven't tested yet so it's quite possible one of those may also exhibit the same problem). The issue is the the Windows HID class driver recognises the device and trys to initialise and start it, this works after a fashion and the device starts sending data.
The problem is that the data being sent is different to that which is sent to the genuine platform (containing only a subset of the full data). It's as though windows has initialised the device into a different mode.
When I capture the initialisation packets from both the PC and the genuine platform using my USB protocol analyser I see that Windows is sending some slightly different initialisation packets. Using LibUSBDotNet to resend the correct packets once Windows has already started the device seems to have no effect.
My problem is that I need to stop windows from trying to initialise the device using the standard HID class driver, I've tried removing the driver in Device Manager but it still initialises it (and the driver is magically reassigned in device manager). I've done some investigation and there are possible alternatives:
Create a specific driver which windows will assign to the particular VID/PID of the device but that does nothing, then I can use LibUSBDotNet to send the correct initialisation sequence to the device from within my own code.
Use something like WinUSB to create a proper driver for the device (or possibly to create a "dead" driver like 1.
Will a driver with a specific VID/PID defined be used by windows in preference to it's inbuilt USB HID class driver? If not then I would be wasting my time going down this route?
Note, my mac initialises the problem device correctly, and I've asked the question of the client whether the application can be developed for Mac and their answer was frustrating Windows only.
I've no experience in writing proper Windows drivers, though I have experience in talking to USB at a relatively low level (so that part doesn't worry too much). Can anyone suggest a good course of action (before I potentially waste weeks investigating how to write drivers for the PC only to find my selected course of action can't deliver what I required).
Any help or suggest much appreciated.
Thanks,
Rich
Added after trying suggestions below:
I tried using the LibUsbDotNet inf wizard to create the necessary files and install them and this appeared to work - certainly the device was now appearing in Device Manager as a libusb-win32 device - not HID device and the associated driver was libusb driver. Even after doing this the device still seems to become initialised and start sending the wrong type of data packets although now those packets are no longer handled by the class driver and are just lost.
I also came across Zadig which has a similar inf creation wizard for WinUSB and this had exactly the same result.
A colleague has suggested that it might not be windows itself that is switching the device into this mode, rather the device identifying that it is connected to a windows machine and switching itself into this mode. I suspect this is the case, in which case I am stuck - time to have another conversation with the client.
Many thanks for the help.
You're using libusb-win32 as a filter driver; that is, the HidUsb device driver is assigned and loaded for your device, but then the libusb-win32 driver is loaded on top and gives you unobstructed access to the hardware.
If you don't want a HidUsb (or any other class driver) to perform any communication "on your behalf", simply associate libusb-win32 as a device driver with your hardware. For this, you'd have to create an .INF file associating it with the VID/PID/Revision of each USB device. If I recall correctly, libusb-win32 even comes with a utility to generate such .INF files.
If you install this .INF file e.g. with PnpUtil.exe (available on Vista or higher), you might still run into issues where, although you're a better match than the generic HID driver, the HID driver is still selected.
The generic HID driver matches devices by their Compatible IDs (i.e. by a USB interface class) while you'd be matching by Hardware IDs (which have higher priority). However, Windows might give priority to other aspects, such as your driver being unsigned. Read: How Windows Selects Drivers
Luckily, even in that scenario, signing drivers with a self-generated certificate (use CertUtil.exe, MakeCat.exe and SignTool.exe) is not too difficult.
A friend of mine and I got into a conversation and realized Windows 7 is missing a key component to its per-application audio settings. You can set volume but you can't stipulate which device each application should use.
Some applications such as ventrilo or skype allow you to select which device to use however, MOST applications simply rely on the current 'Default Audio Device.'
Is there a way to access this? What language would be best used to expose these kinds of functions? Thanks!
Ventrilo and Skype are able to choose which audio device to use because they are coded to directly specify audio output devices instead of just getting the default from the OS. For applications which are coded to use the default Windows device, you can of course change which device is the default device using the sound settings, but this will change the default for the whole system.
Setting different audio devices for separate applications which all use the default audio device isn't something that is necessarily supported by Windows, and many applications use the DirectSound API which complicates the situation further. However, some applications check which device is the default when they initialize and then output exclusively to that device. In this case, you could change the default sound device to one audio device, start a program, then change the default to another audio device, and the first program would continue to use the device which was default when it started up.
However, this is a pretty weak workaround and will only work for specific applications which have been coded in the way described above.