I'm recording from a webcam. The camera looks great in PhotoBooth. However, when I preview it in my program with a QTCaptureView, or record it to a file, it is very, very slow. The reason is that QuickTime is giving me the maximum possible resolution of 1600x1200. How can I force a more reasonable size for both my QTCaptureView and my recording to file?
As described here, you can set the pixel buffer attributes within the output from your QTCaptureSession to change the resolution of the video being captured. For example:
[[[myCaptureSession outputs] objectAtIndex:0] setPixelBufferAttributes: [NSDictionary dictionaryWithObjectsAndKeys:
[NSNumber numberWithInt:480], kCVPixelBufferHeightKey,
[NSNumber numberWithInt:640], kCVPixelBufferWidthKey, nil]];
will set the video resolution to be 640x480 for the first output in your capture session. This should also adjust the camera settings themselves to have it return image frames of that size (if supported by the camera hardware).
You may also wish to use base MPEG4 encoding, instead of h.264, to do your realtime video recording. This can be set using code similar to the following:
NSArray *outputConnections = [mCaptureMovieFileOutput connections];
QTCaptureConnection *connection;
for (connection in outputConnections)
{
if ([[connection mediaType] isEqualToString:QTMediaTypeVideo])
[mCaptureMovieFileOutput setCompressionOptions:[QTCompressionOptions compressionOptionsWithIdentifier:#"QTCompressionOptionsSD480SizeMPEG4Video"] forConnection:connection];
}
h.264 encoding, particularly the Quicktime implementation, uses a lot more CPU power to encode than the base MPEG4.
The solution above (setPixelBufferAttributes:) does set the preview size correctly, but once movie recording starts, the preview image will get set back to it's original value (1280 x 1024 on my MBP) if you've set (almost) any compression options.
If that was just during movie recording that would be one thing, but once recording is complete, further calls to setPixelBufferAttributes will have no effect.
So, you can change the preview image size, as long as you don't plan on doing any actual compressed movie recording.
This is on 10.5.8/9L30, MBP with a GeForce 8600M. Any compression option except for no compression or QTCompressionOptionsSD240SizeH264Video breaks as described above.
rdar://7447812
To add more information about the topic:
you can't specifiy directly the definition on the capture side. Rather, this is the output of the capture session that defines the definition. e.g.
if you capture into a QtCaptureDecompressedVideoOutput, you shall specify the definition on this object.
Related
I am getting a list of small rentangle images with contain the parts of the image that have changed from the previous image. This results from the desktop image capture with directx11 which provides what parts of the desktop image have changed and the rectangles from them.
I am trying to figure out if I can pass them to ffmpeg libavcodecs encoder for h.264. I looked into AVFrame and didn't see a way to specify the actual parts that have changed from the previous image.
Is there a way to actually do this, when passing an image to the ffmpeg codecContext to encode it in the video, to just pass the changed parts from the previous frame? Maybe doing this will reduce the amount of CPU usage because this is for a live stream.
I use the standard avcodec_send_frame to send a frame to the codec for encoding, it only has an AVframe and a codec context as parameters.
I need to be able take a video from Photos and re-rendering, both clipping it in time, changing the width and height, and frame rate. Certainly I need to start with:
PHContentEditingInputRequestOptions *options = [[PHContentEditingInputRequestOptions alloc] init];
[self.asset requestContentEditingInputWithOptions:options completionHandler:^(PHContentEditingInput *contentEditingInput, NSDictionary *info) {
// Get full image
NSURL *url = [contentEditingInput fullSizeImageURL];
}];
And I should be able to adjust width, height and duration. Grab an NSData from that, write that out to the file syset.m
But the url is nil, which implies to me that I can't edit videos with the new Photos framework. (ALAsset didn't have a problem with this using AVAssetExportSession.) This makes sense since the Apple Dev sample code can't edit videos either.
Now, to make life easier I could just pass that url to an AVAssetExportSession but I can't, because it is nil. If I just modified width, height and duration I'd still need to grab an NSData from it, write that out to the file system.
I do not need to write the modified video back to Photos, I actually need the video on the file system since I'll be uploading it to our servers.
fullSizeImageURL is for working with Photo assets. You want the avAsset property when working with a video. Modify the actual video, not the metadata, by writing a new video file.
To do that, you could use that avAsset in an AVMutableComposition:
Insert the appropriate time range of the avAsset's video track (AVAssetTrack) into an AVMutableCompositionTrack. That'll do your trimming.
Place/size it appropriately using layer instructions. (AVMutableVideoCompositionLayerInstruction) to do your cropping and scaling.
I'm capturing images from my webcam with some code that mainly bases on this: Using the Sample Grabber.
Here I only get the default resolution of 640x480 while the connected camera is able to show more (other capture applications show a bigger resolution).
So, how can I:
retrieve the list of available resolutions
set one of these resolutions so that the captured image comes with it?
IAMStreamConfig interface lists capabilities and lets you select resolution of interest. enumerating media types on an unconnected yet pin will list you specific media types (amd resolutions) the camera advertises as supported.
More on this (an links from there):
Video recording resolution using DirectShow
Video Capture output always in 320x240 despite changing resolution
We are using a directshow interface to capture images from a video stream. These images are presented in a fixed size window.
Once we have captured an image we store it as a bitmap. Downstream we have the ability to add annotation to the image, for example letters in a fixed size font.
In one of our desktop environments, the annotation has started appearing at half the size that it normally appears at. This implies that the image we are merging the text onto has dimensions that are maybe twice as large.
The system that this happens on is a shared resource as in some unknown individual has installed software on the system that differs from our baseline.
We have two approaches - the 1st is to reimage the system to get our default text size behaviour back. The 2nd is to figure out how directshow manages image dimensions so that we can set the scaling on the image correctly.
A survey of the directshow literature indicates that the above is not a trivial task. The original work was done by another team that did not document what they did. Can anybody point us in the direction of what directshow object we want to deal with to properly size the sampled image?
DirectShow - as a framework - does not deal with resolutions directly. Your video source (such as capture hardware) is capable of providing video feed in certain resolution which you possibly can change. You normally use IAMStreamConfig as described in Configure the Video Output Format in order to choose capture resolution.
Sometimes you cannot affect capture resolution and you need to resample the image in whatever dimensions you captured it. There is no stock filter for this, however Media Foundation provides a suitable Video Resizer DSP which does most of the task. Unfortunately it does not fit DirectShow pipeline smoothly, so you need fitting and/or custom filter for resizing.
When filters connect in DirectShow, they have an AM_MEDIA_TYPE. Here you will find a VIDEOINFOHEADER with a BITMAPINFOHEADER and this header has a biWidth and biHeight.
Try to build the FilterGraph manually (with GraphEdit or GraphStudioNext) and inspect these fields.
Given a sample buffer of H.264, is there a way to extract the frame it represents as an image?
I'm using QTKit to capture video from a camera and using a QTCaptureMovieFileOutput as the output object.
I want something similar to the CVImageBufferRef that is passed as a parameter to the QTCaptureVideoPreviewOutput delegate method. For some reason, the file output doesn't contain the CVImageBufferRef.
What I do get is a QTSampleBuffer which, since I've set it in the compression options, contains an H.264 sample.
I have seen that on the iPhone, CoreMedia and AVFoundation can be used to create a CVImageBufferRef from the given CMSampleBufferRef (Which, I imagine is as close to the QTSampleBuffer as I'll be able to get) - but this is the Mac, not the iPhone.
Neither CoreMedia or AVFoundation are on the Mac, and I can't see any way to accomplish the same task.
What I need is an image (whether it be a CVImageBufferRef, CIImage or NSImage doesn't matter) from the current frame of the H.264 sample that is given to me by the Output object's call back.
Extended info (from the comments below)
I have posted a related question that focusses on the original issue - attempting to simply play a stream of video samples using QTKit: Playing a stream of video data using QTKit on Mac OS X
It appears not to be possible which is why I've moved onto trying to obtain frames as images and creating an appearance of video, by scaling, compressing and converting the image data from CVImageBufferRef to NSImage and sending it to a peer over the network.
I can use the QTCapturePreviewVideoOutput (or decompressed) to get uncompressed frame images in the form of CVImageBufferRef.
However, these images references need compressing, scaling and converting into NSImages before they're any use to me, hence the attempt to get an already scaled and compressed frame from the framework using the QTCaptureMovieFileOutput (which allows a compression and image size to be set before starting the capture), saving me from having to do the expensive compression, scale and conversion operations, which kill CPU.
Does the Creating a Single-Frame Grabbing Application section of the QTKit Application Programming Guide not work for you in this instance?