I am using Windows Media Format SDK to capture the desktop in real time and save it in a WMV file (actually this is an oversimplification of my project, but this is the relevant part). For encoding, I am using the Windows Media Video 9 Screen codec because it is very efficient for screen captures and because it is available to practically everybody without the need to install anything, as the codec is included with Windows Media Player 9 runtime (included in Windows XP SP1).
I am making BITMAP screen shots using the GDI functions and feed those BITMAPs to the encoder. As you can guess, taking screen shots with GDI is slow, and I don't get the screen cursor, which I have to add manually to the BITMAPs. The BITMAPs I get initially are DDBs, and I need to convert those to DIBs for the encoder to understand (RGB input), and this takes more time.
Firing a profiler shows that about 50% of the time is spent in WMVCORE.DLL, the encoder. This is to be expected, of course as the encoding is CPU intensive.
The thing is, there is something called Windows Media Encoder that comes with a SDK, and can do screen capture using the desired codec in a simpler, and more CPU friendly way.
The WME is based on WMF. It's a higher lever library and also has .NET bindings. I can't use it in my project because this brings unwanted dependencies that I have to avoid.
I am asking about the method WME uses for feeding sample data to the WMV encoder. The encoding takes place with WME exactly like it takes place with my application that uses WMF. WME is more efficient than my application because it has a much more efficient way of feeding video data to the encoder. It doesn't rely on slow GDI functions and DDB->DIB conversions.
How is it done?
The source to CamStudio, a GPL'd screencasting app that's been around for years (commercially and then open-srcd later) might be useful?
http://sourceforge.net/project/showfiles.php?group_id=131922
I'd suggest looking at the guts of VNC clients too, though they're probably very simplistic (I think just grabbing screenshots then jpg'ing the tiles that have changed since the last capture).
You might want to consider not using WMV9 as the encoder for on-the-fly encoding if it is too cpu-heavy? Maybe use an older, less efficient compressor (like MS RLE) as used by HyperCam and then compress to WMV afterwards? MS RLE has been a default install since at least Win2000 I believe:
http://wiki.multimedia.cx/index.php?title=Microsoft_RLE
CamStudio's Lossless codec is GPL (same link as above), that offers pretty good compression (though you'd need to bundle the dll in your installer) and could be used on the fly, it works well with high compression on all modern systems.
It's been ages since I've done any Win32 coding, but AFAIK, WMF as a format is basically a list of GDI commands and their parameters which would explain why it is much more efficient to encode...
You'd probably need to hook into the top level GDI context (just as Remote Desktop does, I guess) and capture the GDI commands as they are called. I seem to remember there being some way of creating a WMF output GDI context which means you may be able to just delegate calls to it in some way.
I'm guessing here, but you may be able to find example code for the above in the TightVNC/QuickVNC for Windows projects as they would have to do something like that to capture changes on screen in an efficient way.
Have you checked out the BB FlashBack library?
I am on a similar hunt, and I have just started evaluating the BB FlashBack library.
I am not sure about the external dependencies or install footprint. It appears to have a proprietary codec that has to be installed, but the installation of the codec can be handled by the exposed BB FlashBack API.
Beware, there are licensing restrictions (Runtime setting of license keys, ...)
I can send you the CHM from the SDK via e-mail if you want to evaluate the API before committing to a licensed download.
Things I am in the midst of evaluating:
Proper captures of WPF views
mouse cursor tracking
Size of stored movie
How to display stored movie without proprietary codec (i.e. SWF export)
--Batgar
Related
This question arises out of a combination of this being my first time working with video and unfamiliarity with Macs. Basically I'm finding it difficult to figure out how to play a video (within a QWidget, or otherwise) using any standard format, e.g. avi, mpeg, mov, etc. In particular,
QMovie::supportedFormats() gives me only .gif and .mng, but I need to use standard formats. Is there a way to increase the number of supported formats?
Phonon requires the presence of a 'backend' which the user has to implement himself. I looked to see if I could somehow do this with Quicktime, but I couldn't get the application to launch--and anyway I didn't really see how to do that. Also, Phonon looks pretty heavyweight, I'd like to avoid it if I could.
While there are plenty of avi (et al.) players floating around on the web, I think it's probably unlikely I'd be able to use them--I need to start, stop, and change the playback speed of videos programmatically i.e. through my C++ program.
I'm not sure why this should be so hard--working with images in Qt is a snap by comparison. So: What's a good way to play videos from within a C++/Qt program?
Stop what you are doing right now: Phonon is the past, Qt Mobility is the future.
After you download, compile and install Qt Mobility, check the examples: videowidget and videographicsitem, located at: qt-mobility-opensource-src-1.2.0/examples/
They pretty much answer all your questions.
i need to get the streaming from 2 webcams on the same computer, and mix it as a fake webcam (so then i can use the fake webcam on any software).
I have seen that camcamx is for mac, webcamstudio is for linux, but i need a solution for windows and i can't find it, so i was thinking to write my own small app.
I can program with C#, Java and lazarus, but examples or library or whatever in any language will help anyway.
i will need to make a fake webcam that can be used as a webcam (detected on my computer as a usb webcam), and some code to grasp the stream from two real webcam and mix everything together (there will be like a primary webcam that will be bigger and a secondary webcam that will be smaller, on a corner of the big image)
Anyone can help me on that?
This is not a trivial exercise but it can be done. I know because I've done it before. :)
I implemented this in C++.
What you need to do is to create what's known as a shared memory server. A shared memory server is a region of ram that more than one process can access. Here's how to create one using Named Shared Memory under Windows:
http://msdn.microsoft.com/en-us/library/windows/desktop/aa366551(v=vs.85).aspx
In your application that mixes the video from the two cameras, you need to create a DirectShow rendering filter (CBaseRenderer) that writes the mixed video frame into this shared memory.
On the other end, you need to create a separate Visual Studio DLL project that will implement a DirectShow capture filter (CSource and CSourceStream) that will read the video bitmaps your main application writes into this buffer. This VS project needs to be a registerable DLL that can be called to register it as a DirectShow capture device for windows.
Your main application will create and maintain this shared memory buffer when it is operating. If another application (like a video conferencing program) accesses the capture device, all that will come from the device will be a blank buffer until you main application stars feeding real video frames into it.
Tip #1: Since this is a multi-threaded operation, you will need an event handle to signal the capture filter that a frame is ready. You will also need a mutex to control access to the buffer by the "rendering" thread in your application and the "capture" thread in the capture device.
Tip #2: You won't need to call UnmapViewOfFile or CloseHandle on the memory pointers until the rendering or capture filters are disposed.
There is a lot of code you will need to grind out, so any useful examples will be beyond the scope of this discussion. This should get you going in the right direction. Good luck!
I think your question is too far out of scope for what this site is all about. You're talking about thousands and thousands of lines of code and intimate knowledge of drivers, video decoding, mixing, etc., etc. if you're going to write this software on your own.
With that said, there probably is software for this for Windows. I'd start here:
http://alternativeto.net/software/webcamstudio/
Capture video from real webcam: Video Capture on MSDN
Fake webcam: the well known starting point is Vivek's sample/project available at http://tmhare.mvps.org/downloads.htm, see also this post "Fake" DirectShow video capture device
Getting all together is doable, though not trivial.
I have an old computer vision experiment that uses Video for Windows to grab frames from a camera connected to the PC. It's a hack, it uses VfW to create a preview window, then it does a GetDIBits from the window DC.
I'm finally ready to port this to DirectShow. My understanding was that I could grab frames from a video capture graph by using ISampleGrabber, but now I read that ISampleGrabber is deprecated.
What's the non-deprecated way to grab frames from a video feed? Do I have to implement my own DirectShow filter that does essentially what ISampleGrabber does?
DirectShow is not deprecated; just the DirectShow Editing Services. I would strongly recommend using DirectShow because of the much wider level of support, unless there are specific features of MF that are needed.
There's been no development of DES for some years, but the sample grabber is a widely-used filter that is somewhat independent of DES. I would be happy to recommend that you use it. If there is an issue in future versions of windows, it would not be more than a day or two's work to replace the filter.
G
I think Windows Media Foundation would be your best bet if you are only targeting Vista/Win7, otherwise you can still use DirectShow/SampleGrabber approach, I doubt it will be removed any time soon. Related question here.
In Windows XP, you can configure your sound card properties via the preloaded windows software. In the recording properties, if "stereo mix" or "wave out" (or something similar) is selected as the recording device, programs that can record audio ("Sound Recorder" in windows for example) record a decent quality wave file of the audio stream. I usually use Goldwave from download.com to do this as an example of a third-party application that functions the same.
Well, I've had trouble getting this scenario to happen on Windows Vista or later in a direct no-bullsh*t manner as described above. It's more than just Vista+, it's also that some sound cards don't have that option at all.
I was just wondering if there is a way to run a windows-friendly program (VB?) that takes your audio output stream and converts it (in realtime, obviously) to a WAV file with the default sampling rate as other WAV files have.
Ideally, it would cool if it worked on any operating system, so is it possible to write a web service that "listens" to your audio card like that without making the computer think it's getting a virus attack or something?
Possibly related question:
How to save web audio streaming to file ( c++ / java )
I'm only aware of one manufacturer of sound cards that enabled that option (Creative). However Vista and beyond support a "loopback" mode which gives you effectively the same functionality. You need to use the low level WASAPI rendering stack but it should work just fine.
https://github.com/rdp/virtual-audio-output-sniffer provides a directshow input device to capture the sum of wave out for vista+
You could use low level waveOut API injection and capture what it receives.
I have SkypeMXrecorder, a software that does just that - inject into any exe and 'sniffs' what's going out from it and into the sound hardware. But, it seems rather complicated to implement...
I have this garden variety USB video camera, and it came with two mini-apps, one that just lets you see what the camera sees, and one that records to an .avi file.
But what's the API if I want to grab images from the camera in my own C program? I am making the assumptions that it's (1) possible and (2) desirable to make some call and have a 2D array of pixel information filled in.
What I really want to do is tinker with image processing algorithms, and for that I'd really like to get my code around some live data.
EDIT -
Having had a healthy exposure to Linux, I can grasp how (ideally/in theory) you could open() the device, use ioctl() to configure it, and read() the data. And I'm virtually certain that that's not how Windows is going to present the API. Not knowing what function names Windows might use for a video device API, or even if it has one, makes it difficult to look up, at least with the win32 api search capabilities that I have at my disposal.
You'll probably need the DirectShow API, provided that's how the camera operates. If the manufacturer created their own code path, you'll need their API.
Your first step, as pointed out by ChrisBD, is to check if Windows supports your device.
If that is the case you have three possible Windows APIs for capture:
DirectShow
VFW. Has more or less been replaced by DirectShow
MediaFoundation. Is the newest API that is intended to replace DirectShow. AFAIK not fully implemented yet and only available in Vista.
From the three DirectShow is the best choice. However, learning and using DirectShow is not a trivial task. An excellent example can be found here.
Another possibility is to use OpenCV. OpenCV is an image processing library, that you can also use for processing the images. OpenCV has an image capture API that provides a simpler abstraction and is easier to use than the Windows APIs.
The API is the way to go.
A good indication of whether the camera requires a bespoke one or not is to see if it is recognised by a PC without the manufacturer's applications installed. If windows has the drivers built in the you should be able to use the windows APIs to capture the images.
Alternatively if you know what compression codec has been used for the AVI file you could unpack it.
Ideally it would be good if you could capture the video in native (YUV, RGB15 or similar) format as then you can work on compression as well as manipulation.