Is video data that comes from Webcam to low level USB driver already encoded?
What does the low level webcam driver actually do? What is its main and only minimum mission?
Where can I read the principles or the protocol between the webcam hardware and standard Windows driver?
Ok, based on 5 webcams, that have bought to understand the difference between them, I can make some conclusions.
> Is video data that comes from Webcam to low level USB driver already encoded?
Related
I've been working on some WinAPI code to try to get audio latency as low as possible in Windows 10 natively. I eventually found this documentation which indicates the WASAPI in Windows is the lowest level you can get to, and there are even some improvements in Windows 10 to make the latency even lower. Most notably, support for <10ms audio buffer sizes with newer drivers.
I used the WASAPI but the latency still isn't quite low enough. I'm measuring about 80ms round trip using USB microphone and audio devices. I tested the new drivers mentioned and successfully got lower latency and smaller buffer sizes, but the only drivers mentioned seem to be the "High Definition Audio Device" driver, which I traced to hdaudio.sys. However, typical USB audio devices seem to use "USB Audio Device" drivers, implemented in usbaudio.sys instead I believe, which isn't allowing smaller buffer sizes.
It seems like the changes to allow smaller buffer sizes are relatively minor, but for some reason the stock HD Audio drivers contain these updates but the stock USB Audio drivers do not. What is happening here? Do these updated USB Audio drivers even exist in Windows?
For those looking to operate generic USB audio hardware on Windows at low latency, the KoordASIO driver provides a convenient ASIO-WASAPI bridge, enables either WASAPI Shared or Exclusive mode, and configurable buffer size.
https://github.com/koord-live/KoordASIO
I'm investigating an issue with an in-house developed app using WMF to capture UVC data from a Cypress FX3 device. The stream is generated from a test pattern generator fed from an FPGA to the FX3. For a frame size of 1920x1080 (#30FPS), the frame capture works fine. For a frame size of (say) 3264x2448 (at <8FPS to meet throughput restrictions) the app is getting stuck during the ReadSample(). It does appear data is being received as the data image pattern can be seen in memory. Device enumeration looks ok i.e. the reported descriptors look to be correct and SelectFormat() is set accordingly.
Are there any restrictions on frame size?
I see that problem with your hardware is very specific and can be related with hardware part. The frame size - 3264x2448 is very huge. I have experience with Logitecn HD Pro Webcam C920 and I can get max frame size 2304x1536 at 2 FPS and RGB24. If you have connected via USB then it can be problem to transmit it via USB bus (especially for USB2). You must know that Windows Media Foundation is engine of the media part of Windows - for example WindowsStore can work only with Media Foundation. More over, Windows 10 Includes encoder and decoder for HEVC (H265) video and supports 4K and 8K in native, but playing video and working with live video via USB has some difference.
When I connect several USB audio devices using the default drivers I usually end up with some kind of friendly device description like "nn- USB Audio codec" or something else, so I have currently no unique property which refers to a specific hardware the e.g. manufacturer ID. I would like to ensure that in case of re-enumerating or replugging the hardware to a different port that my software will automatically identify the changed windows audio device which maps to this hardware. I there a possibility to gather further windows audio device information which can be used to determine the physical hardware ?
Maybe a little clarification is needed:
I have two or more USB audio devices, not necessarily same product or manufacturer. The audio connections have different purposes, so I want to ensure that I my software uses constantly the same physical audio devices for different tasks. This sometimes fails when a re-enumeration occurs ( sometimes without changing the physical USB port connections, it's a windows thing...)
Ideally I would distinguish them by an individual serial number, which usually is only available with storage devices. But what I can retrieve is the "physical" USB topology when I'm looking at the device with a tool such as USBDeview from Nir Sofer. There I have a property like "Hub 3, Port1". If I could map this to a sound device I get from audio audio api like "nn- USB Audio codec" as seen in the sound control panel I would be perfectly happy.
Edit2:
May be this[1] post helps here, but I still have to figure out how to get a USB Port <-> Windwos Sounddevice mapping.
[1]: Can the physical USB port be identified programatically for a device in Windows? "
I would like to experiment with real time audio and was wondering what options are there for accessing audio via a usb port?
I would like to get data from this hardware in an audio stream.
Is this sort of thing best handled by audio frameworks or should the data be fetched using IOKit and passed to e.g. an audio component using a framework?
Also, how is this access effected if e.g. A DAW is accessing the audio from the port? Is dual access handled by the OS?
The specs page for that hardware (Moog Minitaur) doesn't claim that it can transmit audio over USB, only that it has USB MIDI- essentially it has a built in MIDI interface.
If you wanted to get audio from it into your computer you would have to connect it to your computer's line or mic input, or an external soundcard - which may itself connect via USB. One way to handle audio in OSX would be Core Audio. You shouldn't need to write any drivers or anything to deal with the low-level I/O with the soundcard, as the soundcard should come with a driver if there isn't one already in the OS.
I am trying to design a Windows KMDF driver for an old USB Webcam.
I think I have enough informations on the driver development / USB protocol side, but I am wondering how I should expose the webcam functionalities from the driver so that Windows program can take advantage of it.
This webcam has 3 functionalities:
a video stream,
an audio stream,
a "take picture" hardware button.
For each of those, what is the more natural way to expose it to Windows, so that my webcam can be used in relevant programs (gTalk, Skype, etc.)? Or, to revert the question: what is expected by those programs, and how do they detect webcams?
When you write a 'WebCam' driver, you should use kernel streaming as your driver model. this provides you with the correct interfaces, so that other programs can communicate with your device through DirectShow/VfW/etc. .
Currently kernel streaming is implemented in the AVStream mini driver base. The WDK contains a AVStream mini driver sample to base your own driver on. The backend to the device can then be implemented using KMDF.