I'm playing with the "Capture Source Filter" from http://tmhare.mvps.org/downloads.htm.
After registering the ax driver, I'm trying to understand its compatibility across applications that use video sources.
For example, Skype recognize it while browsers (Edge, Chrome) don't.
I wonder if it's a limitation of the used approach (DirectShow filter) or it's just a matter of configuration.
The purpose of the question is to understand if that approach is still useful or it's better to move on Media Foundation.
I described this here: Applicability of Virtual DirectShow Sources
Your virtual camera and the applications capable to recognize and pick it up are highlighted with green on the figure below.
... if that approach is still useful or it's better to move on Media Foundation.
Media Foundation does not even have a concept of virtual video source. It does not have a compatibility layer to connect to DirectShow video sources. Obviously, in other direction DirectShow applications won't be able to see virtual Media Foundation streams (well, again, because they do not exist in compatible concept in first place).
If you want to expose your video source to all applications, you need a driver for this (see red box on the figure above). Applciations exist out there that implement such concept, even though writing a new one from the ground up is not something compatibly easy with the DirectShow virtual source you referenced in your question.
Further reading on MSDN on Media Foundation: How to register a live media source - media foundation
Though it is technically possible to write a virtual driver that shows up as a capture device, policies will probably prevent this. In Media Foundation, a device must have a certificate to appear as a capture device, and so far only actual hardware devices through the USB video class driver have been certified. Supporting a scheme through a scheme handler, or a file type with a byte stream handler, is the way to expose a new source to applications.
Related
I'm looking for a way to programmatically create multiple "virtual" desktops, and stream their contents to whatever.
Essentially, what macOS apis are these guys https://cindori.org/vrdesktop/ using to create those virtual desktops and additionally how do they get a video stream of it?
Just looking for guidance to the right apis/docs. No code required :)
You'll need to create a kernel extension that simulates a graphics adapter, essentially just rendering everything into a framebuffer. The framebuffer can then be exported as a video stream or whatever you choose.
You can find example source codes to look at here:
https://github.com/tSoniq/displayx
https://github.com/andreacremaschi/Syphon-virtual-screen/tree/develop
https://github.com/mkernel/EWProxyFramebuffer
https://code.google.com/archive/p/ioproxyvideofamily/source/default/source
Please note that these projects are not up to date in terms of supporting the latest macOS versions. Later versions of macOS have introduced things such as mandatory kernel extension signing that makes it harder for "hobby-developers" to produce something that can be shared for free on the internet in terms of kernel extensions. It also makes it harder for malware authors, which is the upside.
So take a look at these source repositories and you'll find your guidance - but don't expect them to be complete solutions.
I would like to take video from a webcam, render some text on the frames and do some motion tracking and pass it on to a virtual webcam so it can be streamed easily.
I found some answers on stackoverflow suggesting that I should use DirectShow. According to information in DirectShow documentation, the DirectShow SDK is part of Windows SDK. So I installed the latest Windows SDK but it seems that it doesn't include DirectShow because there are no DirectShow samples under C:\Program Files (x86)\Microsoft SDKs\Windows. (The stackoverflow answers are also pretty old - dated around 2010)
Can you suggest a way to make DirectShow work (including samples working on Visual Studio 2015) or some other alternative to DirectShow, that would help me create a virtual webcam?
Virtual webcam is typically a software only implementation that application discover as if it is a device with physical representation. The mentioned applications use APIs to work with web cameras and ability to extend the APIs and add your own video source is the way to create a virtual web camera.
In Windows there are a few APIs to consume video sources: Video for Windows, DirectShow, Media Foundation (in chronological order).
Video for Windows is not really extensible and limited in capabilities overall. It will see a virtual device if you provide a kernel mode driver for a virtual camera.
DirectShow is the API used by most video capture enabled Windows applications and it is present in all Windows versions including Windows 10 (except just Windows RT). Then it's perfectly extensible and in most cases the term "virtual webcam" refers to DirectShow virtual webcam. Methods to create DirectShow virtual webcam discussed in many StackOverflow questions remain perfectly valid for Windows 10, for applications that implement video capture using DirectShow:
Virtual webcam input as byte stream
Simulate a DirectShow Webcam
DirectShow samples were removed from Windows SDK but you can still find them in older releases:
Getting DirectShow Samples on Windows 8
If you provide a kernel mode driver for video camera device (your virtual webcam through custom kernel driver), DirectShow would also see it just like other video APIs.
Media Foundation is a supposed successor of DirectShow but its video capture capabilities in the part of extensibility simply do not exist1. Microsoft decided to not allow custom video sources application would be able to discover the same way as web cameras. Due to Media Foundation complexity, and overhead and overall unfriendliness it is used by modest amount of applications. To implement a virtual webcam for Media Foundation application you again, like in case of Video for Windows, have to implement a kernel mode driver.
1 Starting with Windows Build 22000 (Windows 11), there is new API MFCreateVirtualCamera which offers virtual camera creation. A developer can implement a video source which the API connects to so called Windows Camera Frame Server service, which in turn distributes the generated video as a source along with regular cameras. Applications see this software implementation the same way as if it was, for example, a webcam.
This is an ancient question internet-wise but I thought I could contribute:
I was looking into this about a year ago and almost abandoned my project altogether until I found Microsoft's SimpleMediaSource driver sample on their Github. It is documented here but it is a tough read if you haven't written drivers before - which was the case for me. Fortunately, documentation seems to have been updated and improved since I used it.
To get it working, I had to manually delete and copy-paste the DLL into C:\System32 after each compilation with Visual Studio. I also had to side-download and install the now removed (from what I can tell) devcon utility to add & remove drivers with devcon dp_add/dp_remove commands. You also need the Windows Driver Kit (WDK).
You need to enable unsigned driver loading within Windows so it may not be a great route if you want to distribute it. Anticheat and DRM software may also not appreciate it :)
There are two projects being compiled:
MediaSource - COM DLL project for the custom media source
SimpleMediaSourceDriver - UMDF driver install package
Just install obs studio
In newer versions it automatiaclly installs an easy to use virtual webcam that mirrors the OBS scene.
I've been experimenting with two cameras, one is a webcam and another is an evaluation-kit camera that comes with its own drivers, I can run AMCAP.exe (provided from Vimicro) and it will display live streaming from the connected camera (any of the two) although each one uses a different driver.
My question is: Is it safe to assume that AMCAP.exe is only running as a video-stream display program?
In which case, I assume that most camera vendors follow a common standard interface for camera drivers. Could anyone comment on such assumption or explain how AMCAP.exe is able to do that.
I've been able to use both cameras in my c++ based OpenCv applications, but I'm asking because a 3rd company is going to provide me with a usb board-based camera (for evaluation) and they are asking about driver specs. so I suggested that it should work with windows default driver similar to how a webcam would so as to avoid compatibility issues but I'm wondering if there is a better option that I might be missing.
Note I: I've been able to verify that different drivers are being used for each camera by: Start->Devices and Printers->USB 2.0 Camera->Hardware tabe->USB 2.0 camera->Driver->Driver Details. When the webcam is connected, it is using the microsoft provided driver files: ksthunk.sys and usbvideo.sys. When I disconnect the webcam and connect the evaluation camera, I am able to verify that it is using it's own (non-windows) driver.
Note II: Vimicro's AMCAP.exe can be downloaded from: VIMICRO USB PC Camera (VC0303) - CNET Download.com
Note III: Computer is core i7. OS is win7 64bit.
Any help or input on this is truly appreciated and immensely needed.
Best,
Hasan.
Camera drivers must implement the OS-defined interface which isn't different for various devices of the same category (in your case it's USB camera). Default drivers for some devices are bundled with Windows and it's ok to use them. However, they aren't necessary optimal for each and every device. E.g., for cameras they might support only a limited set of supported resolutions or might not be optimal with power saving (just an example, not necessary what really happens!). This is because MS implements only the basic and necessary functionality that must be present in every device and doesn't depend on proprietary HW of various vendors. Dedicated camera driver should provide all the additional functionality - you're the one to decide if it's important for you or not.
i need to get the streaming from 2 webcams on the same computer, and mix it as a fake webcam (so then i can use the fake webcam on any software).
I have seen that camcamx is for mac, webcamstudio is for linux, but i need a solution for windows and i can't find it, so i was thinking to write my own small app.
I can program with C#, Java and lazarus, but examples or library or whatever in any language will help anyway.
i will need to make a fake webcam that can be used as a webcam (detected on my computer as a usb webcam), and some code to grasp the stream from two real webcam and mix everything together (there will be like a primary webcam that will be bigger and a secondary webcam that will be smaller, on a corner of the big image)
Anyone can help me on that?
This is not a trivial exercise but it can be done. I know because I've done it before. :)
I implemented this in C++.
What you need to do is to create what's known as a shared memory server. A shared memory server is a region of ram that more than one process can access. Here's how to create one using Named Shared Memory under Windows:
http://msdn.microsoft.com/en-us/library/windows/desktop/aa366551(v=vs.85).aspx
In your application that mixes the video from the two cameras, you need to create a DirectShow rendering filter (CBaseRenderer) that writes the mixed video frame into this shared memory.
On the other end, you need to create a separate Visual Studio DLL project that will implement a DirectShow capture filter (CSource and CSourceStream) that will read the video bitmaps your main application writes into this buffer. This VS project needs to be a registerable DLL that can be called to register it as a DirectShow capture device for windows.
Your main application will create and maintain this shared memory buffer when it is operating. If another application (like a video conferencing program) accesses the capture device, all that will come from the device will be a blank buffer until you main application stars feeding real video frames into it.
Tip #1: Since this is a multi-threaded operation, you will need an event handle to signal the capture filter that a frame is ready. You will also need a mutex to control access to the buffer by the "rendering" thread in your application and the "capture" thread in the capture device.
Tip #2: You won't need to call UnmapViewOfFile or CloseHandle on the memory pointers until the rendering or capture filters are disposed.
There is a lot of code you will need to grind out, so any useful examples will be beyond the scope of this discussion. This should get you going in the right direction. Good luck!
I think your question is too far out of scope for what this site is all about. You're talking about thousands and thousands of lines of code and intimate knowledge of drivers, video decoding, mixing, etc., etc. if you're going to write this software on your own.
With that said, there probably is software for this for Windows. I'd start here:
http://alternativeto.net/software/webcamstudio/
Capture video from real webcam: Video Capture on MSDN
Fake webcam: the well known starting point is Vivek's sample/project available at http://tmhare.mvps.org/downloads.htm, see also this post "Fake" DirectShow video capture device
Getting all together is doable, though not trivial.
I have this garden variety USB video camera, and it came with two mini-apps, one that just lets you see what the camera sees, and one that records to an .avi file.
But what's the API if I want to grab images from the camera in my own C program? I am making the assumptions that it's (1) possible and (2) desirable to make some call and have a 2D array of pixel information filled in.
What I really want to do is tinker with image processing algorithms, and for that I'd really like to get my code around some live data.
EDIT -
Having had a healthy exposure to Linux, I can grasp how (ideally/in theory) you could open() the device, use ioctl() to configure it, and read() the data. And I'm virtually certain that that's not how Windows is going to present the API. Not knowing what function names Windows might use for a video device API, or even if it has one, makes it difficult to look up, at least with the win32 api search capabilities that I have at my disposal.
You'll probably need the DirectShow API, provided that's how the camera operates. If the manufacturer created their own code path, you'll need their API.
Your first step, as pointed out by ChrisBD, is to check if Windows supports your device.
If that is the case you have three possible Windows APIs for capture:
DirectShow
VFW. Has more or less been replaced by DirectShow
MediaFoundation. Is the newest API that is intended to replace DirectShow. AFAIK not fully implemented yet and only available in Vista.
From the three DirectShow is the best choice. However, learning and using DirectShow is not a trivial task. An excellent example can be found here.
Another possibility is to use OpenCV. OpenCV is an image processing library, that you can also use for processing the images. OpenCV has an image capture API that provides a simpler abstraction and is easier to use than the Windows APIs.
The API is the way to go.
A good indication of whether the camera requires a bespoke one or not is to see if it is recognised by a PC without the manufacturer's applications installed. If windows has the drivers built in the you should be able to use the windows APIs to capture the images.
Alternatively if you know what compression codec has been used for the AVI file you could unpack it.
Ideally it would be good if you could capture the video in native (YUV, RGB15 or similar) format as then you can work on compression as well as manipulation.