I would like to take video from a webcam, render some text on the frames and do some motion tracking and pass it on to a virtual webcam so it can be streamed easily.
I found some answers on stackoverflow suggesting that I should use DirectShow. According to information in DirectShow documentation, the DirectShow SDK is part of Windows SDK. So I installed the latest Windows SDK but it seems that it doesn't include DirectShow because there are no DirectShow samples under C:\Program Files (x86)\Microsoft SDKs\Windows. (The stackoverflow answers are also pretty old - dated around 2010)
Can you suggest a way to make DirectShow work (including samples working on Visual Studio 2015) or some other alternative to DirectShow, that would help me create a virtual webcam?
Virtual webcam is typically a software only implementation that application discover as if it is a device with physical representation. The mentioned applications use APIs to work with web cameras and ability to extend the APIs and add your own video source is the way to create a virtual web camera.
In Windows there are a few APIs to consume video sources: Video for Windows, DirectShow, Media Foundation (in chronological order).
Video for Windows is not really extensible and limited in capabilities overall. It will see a virtual device if you provide a kernel mode driver for a virtual camera.
DirectShow is the API used by most video capture enabled Windows applications and it is present in all Windows versions including Windows 10 (except just Windows RT). Then it's perfectly extensible and in most cases the term "virtual webcam" refers to DirectShow virtual webcam. Methods to create DirectShow virtual webcam discussed in many StackOverflow questions remain perfectly valid for Windows 10, for applications that implement video capture using DirectShow:
Virtual webcam input as byte stream
Simulate a DirectShow Webcam
DirectShow samples were removed from Windows SDK but you can still find them in older releases:
Getting DirectShow Samples on Windows 8
If you provide a kernel mode driver for video camera device (your virtual webcam through custom kernel driver), DirectShow would also see it just like other video APIs.
Media Foundation is a supposed successor of DirectShow but its video capture capabilities in the part of extensibility simply do not exist1. Microsoft decided to not allow custom video sources application would be able to discover the same way as web cameras. Due to Media Foundation complexity, and overhead and overall unfriendliness it is used by modest amount of applications. To implement a virtual webcam for Media Foundation application you again, like in case of Video for Windows, have to implement a kernel mode driver.
1 Starting with Windows Build 22000 (Windows 11), there is new API MFCreateVirtualCamera which offers virtual camera creation. A developer can implement a video source which the API connects to so called Windows Camera Frame Server service, which in turn distributes the generated video as a source along with regular cameras. Applications see this software implementation the same way as if it was, for example, a webcam.
This is an ancient question internet-wise but I thought I could contribute:
I was looking into this about a year ago and almost abandoned my project altogether until I found Microsoft's SimpleMediaSource driver sample on their Github. It is documented here but it is a tough read if you haven't written drivers before - which was the case for me. Fortunately, documentation seems to have been updated and improved since I used it.
To get it working, I had to manually delete and copy-paste the DLL into C:\System32 after each compilation with Visual Studio. I also had to side-download and install the now removed (from what I can tell) devcon utility to add & remove drivers with devcon dp_add/dp_remove commands. You also need the Windows Driver Kit (WDK).
You need to enable unsigned driver loading within Windows so it may not be a great route if you want to distribute it. Anticheat and DRM software may also not appreciate it :)
There are two projects being compiled:
MediaSource - COM DLL project for the custom media source
SimpleMediaSourceDriver - UMDF driver install package
Just install obs studio
In newer versions it automatiaclly installs an easy to use virtual webcam that mirrors the OBS scene.
Related
I want to detect buffer underrun situation in direct sound environment.
I use two sound buffers (primary and secondary). Sometimes (when server gives data) I call lock method of IDirectSoundBuffer interface for posting data to secondary sound buffer. If data from server do not have time to come, the sound buffer starts play again from start of the buffer (and repeat it until I send new data to buffer). Maybe it's flag DSBPLAY_LOOPING, but as I read (and test it), primary buffer cannot be created without this flag (there was error at Play method).
I try to get status of playing, but GetStatus method always returns the same status, even there is no new data and player repeat old data.
So, how can I detect buffer underrun situation (when there is no new data for playing and all old data is played)?
Thanks in advance.
IDirectSoundBuffer8::GetCurrentPosition is really the only way you can determine where it is playing from, but it's also only reliable on Windows Vista or later systems that report DSBCAPS_TRUEPLAYPOSITION.
A few things to note:
DirectSound is 'legacy', meaning it hasn't been actively worked on, promoted, or tested directly in ages. The last samples were shipped in the also now end-of-life DirectX SDK in November 2007. Versions of DirectSound prior to 8 are not supported for x64 native applications, and the DirectSound 8 headers/libs are in the Windows SDK as of the Windows SDK 7. It's not supported for Windows on ARM, or Windows Store apps, or Universal Windows apps. The documentation for DirectSound can only be found offline in the legacy DirectX SDK and is not on Microsoft Docs--the only DirectSound content still online is for driver writers.
DirectSound is also 'emulated' on modern versions of Windows so there's nothing actually 'direct' about it. The primary buffer is not actually connected directly to the audio hardware or used for mixing at all, so it's just another software buffer like a secondary buffer. It does emulate the legacy restrictions of primary buffers that applied back in Windows 9x/ME, but it doesn't' do much at all otherwise.
Starting with Windows Vista, LOC_HARDWARE buffers are no longer supported at all. Windows Vista did add support for multi-channel LOC_SOFTWARE buffers, which on Windows XP were only available as LOC_HARDWARE buffers.
Starting with Windows Vista, Effects (ID3L, EAX, etc.) are not supported through DirectSound.
TL;DR: Don't use DirectSound in new applications. It is only still supported at all for old software & games.
So, what is a developer supposed to use if not DirectSound?
(1) Windows Core Audio (WASAPI) is a good option if you can provide the sound data at a known data rate and format. If you need any real-time mixing or source-rate conversion, you have to do it yourself -or- you can use one of the many existing 3rd party audio libraries to do it that also send the final result to WASAPI. See Microsoft docs.
(2) XAudio version 2 is a good choice if you want to do real-time mixing, source-rate conversion, and software-based DSP effects. It is included in the operating system as of Windows 8, but to support Windows 7 you have to use some legacy distribution and SDKs. See Microoft docs and this blog.
Both WASPI and XAudio use a 'audio packet' model instead of a looping buffer for data submission. As long as a packet is pending processing, you won't have under-run.
I'm playing with the "Capture Source Filter" from http://tmhare.mvps.org/downloads.htm.
After registering the ax driver, I'm trying to understand its compatibility across applications that use video sources.
For example, Skype recognize it while browsers (Edge, Chrome) don't.
I wonder if it's a limitation of the used approach (DirectShow filter) or it's just a matter of configuration.
The purpose of the question is to understand if that approach is still useful or it's better to move on Media Foundation.
I described this here: Applicability of Virtual DirectShow Sources
Your virtual camera and the applications capable to recognize and pick it up are highlighted with green on the figure below.
... if that approach is still useful or it's better to move on Media Foundation.
Media Foundation does not even have a concept of virtual video source. It does not have a compatibility layer to connect to DirectShow video sources. Obviously, in other direction DirectShow applications won't be able to see virtual Media Foundation streams (well, again, because they do not exist in compatible concept in first place).
If you want to expose your video source to all applications, you need a driver for this (see red box on the figure above). Applciations exist out there that implement such concept, even though writing a new one from the ground up is not something compatibly easy with the DirectShow virtual source you referenced in your question.
Further reading on MSDN on Media Foundation: How to register a live media source - media foundation
Though it is technically possible to write a virtual driver that shows up as a capture device, policies will probably prevent this. In Media Foundation, a device must have a certificate to appear as a capture device, and so far only actual hardware devices through the USB video class driver have been certified. Supporting a scheme through a scheme handler, or a file type with a byte stream handler, is the way to expose a new source to applications.
I would like to write an app that
accesses the audio by being played by the user in another app
analyzes that audio
displays the analysis to the user
however, I have not found anything that makes this seem possible. Can apps access access the audio being played through the user's speakers as a byte stream or is this not built into the API?
Can apps access access the audio being played through the user's
speakers as a byte stream or is this not built into the API?
You can do that(Loopback recording), but you need to program with low level APIs - WASAPI. For available APIs for UWP, check the Core Audio part in Win32 and COM for Windows Runtime apps and Universal Windows Platform (UWP) apps (multimedia)
Here is an old article for 8.1 app, but it still applies to UWP on concept level. And Microsoft has provided a sample on GitHub about WASAPI and Core Audio API, which may help you to familiar with the APIs and understand how to make it work in UWP app.
Use the MediaCapture class to capture audio. link to docs
I have googled and searched a number of forums and developer websites without any success; I believe it is a specific question that needs direct expertise or knowledge, so please read on!
BACKGROUND:
I have an audio enhancement algorithm that is implemented as a system Audio Processing Object (sAPO) that was developed and tested successfully in Windows 7. As an APO, it applies processing to all audio stream through an end point device, including audio originating from Skype.
QUESTION:
Is it true that this is not applicable to Windows 8.x ( 8.1 or greater)? More specifically, does sAPO processing still work for Skype? Does Skype disable any and all APO processing on its stream?
WHAT HAS BEEN TRIED SO FAR:
(1) I have succeeded in following the standard procedure of loading an unsigned APO from Windows 7 in Windows 8.
(2) I have tested this with Skype audio stream and that works as well.
HOWEVER:
(1) above, fails in Windows 8.1 developer preview. As a result I have not been able to test (2).
Please note that I am specifically asking about Windows 8.1, in a laptop or desktop. This is not for mobile phones or tablets. Any information or links regarding this is much appreciated!
I am also trying to update an APO which have been developed for W7/8 to the new format introduced by W8.1, however it doesn't seem like much documentation has been released yet.
What I have found so far, is that Windows 8.1 requires some new methods for discovery and control of effects to be implemented into the APO. This means that Skype would be able to discover certain APO effects and disable them if it needs to.
The new interface IAudioSystemEffects2:
link
Some updated code can be found in the new SwapAPO example:
link
Not much, but hopefully it can help you get going in the right direction.
I've been experimenting with two cameras, one is a webcam and another is an evaluation-kit camera that comes with its own drivers, I can run AMCAP.exe (provided from Vimicro) and it will display live streaming from the connected camera (any of the two) although each one uses a different driver.
My question is: Is it safe to assume that AMCAP.exe is only running as a video-stream display program?
In which case, I assume that most camera vendors follow a common standard interface for camera drivers. Could anyone comment on such assumption or explain how AMCAP.exe is able to do that.
I've been able to use both cameras in my c++ based OpenCv applications, but I'm asking because a 3rd company is going to provide me with a usb board-based camera (for evaluation) and they are asking about driver specs. so I suggested that it should work with windows default driver similar to how a webcam would so as to avoid compatibility issues but I'm wondering if there is a better option that I might be missing.
Note I: I've been able to verify that different drivers are being used for each camera by: Start->Devices and Printers->USB 2.0 Camera->Hardware tabe->USB 2.0 camera->Driver->Driver Details. When the webcam is connected, it is using the microsoft provided driver files: ksthunk.sys and usbvideo.sys. When I disconnect the webcam and connect the evaluation camera, I am able to verify that it is using it's own (non-windows) driver.
Note II: Vimicro's AMCAP.exe can be downloaded from: VIMICRO USB PC Camera (VC0303) - CNET Download.com
Note III: Computer is core i7. OS is win7 64bit.
Any help or input on this is truly appreciated and immensely needed.
Best,
Hasan.
Camera drivers must implement the OS-defined interface which isn't different for various devices of the same category (in your case it's USB camera). Default drivers for some devices are bundled with Windows and it's ok to use them. However, they aren't necessary optimal for each and every device. E.g., for cameras they might support only a limited set of supported resolutions or might not be optimal with power saving (just an example, not necessary what really happens!). This is because MS implements only the basic and necessary functionality that must be present in every device and doesn't depend on proprietary HW of various vendors. Dedicated camera driver should provide all the additional functionality - you're the one to decide if it's important for you or not.