I'm trying to use a smaller resolution when getting access a webcam video feed, I need to do fast editing when it comes down to previewing. Currently when the image is being outputted from the sample buffer the resolution comes out as 1600x1200 which is too high for what I want to do with it
When setting up the session I use this which it accepts, however regardless of this is does not seem that the changes are being made
_session = [[AVCaptureSession alloc] init];
if ([_session canSetSessionPreset:AVCaptureSessionPreset320x240])
{
[_session setSessionPreset:AVCaptureSessionPreset320x240];
}
One other thing I will also need the webcam to take full size images using captureStillImageAsynchronouslyFromConnection:, this is currently fine
Related
Very weird behavior, but I have narrowed the problem down as far as I can go I think
I have a NSImage, let's call it inputImage. It is represented by a NSBitmapImageRep in a CGColorSpaceCreateDeviceGray, if that matters
I want to create a CGImageRef from it, but with inverted colors.
NSData *data = [inputImage TIFFRepresentation];
CGDataProviderRef provider = CGDataProviderCreateWithCFData((__bridge CFDataRef)data);
NSBitmapImageRep *maskedRep = (NSBitmapImageRep*)[inputImage representations][0];
CGFloat decode[] = {0.0, 1.0};
maskRef = CGImageMaskCreate(maskedRep.pixelsWide,
maskedRep.pixelsHigh,
8,
maskedRep.bitsPerPixel,
maskedRep.bytesPerRow,
provider,
decode,
false);
NSImage *testImage = [[NSImage alloc] initWithCGImage:maskRef size:NSMakeSize(1280,1185)]; //the size of the image, hard-coded for testing
NSLog(#"testimage: %#", testImage);
The problem is when I look at testImage, all of the pixels are slightly offset to the right from the original image.
inputImage:
testImage:
It's much easier to see if you save the pictures off, but you'll notice that everything in testImage is offset to the right by about 5 pixels or so. You'll see a white gap to the left of the black content in testImage
Somewhere in my 5 lines of code I am somehow moving my image over. Does anybody have any idea how this could be happening? I'm currently suspecting TIFFRepresentation
The data provider you pass to CGImageMaskCreate() is supposed to be raw pixels in the form specified by the other parameters, not an image file format. You shouldn't be passing TIFF data. Frankly, I'm surprised you got anything remotely resembling your original image, rather than noise/garbage. I'm guessing the TIFF data wasn't compressed, by sheer chance. You can actually see a bit of noise, which is the TIFF header interpreted as pixels, at the upper-left of your mask.
Probably, your best bet is to create a CGImage from your inputImage (or, depending on how inputImage was created, skip NSImage and create the CGImage directly from a file using the CGImageSource API or CGImageCreateWith{JPEG,PNG}DataProvider()). To get a CGImage from an NSImage, use -CGImageForProposedRect:context:hints:.
Then, get the data provider from that CGImage and create the mask CGImage from that, using the various properties (width, height, bpp, etc.) queried from the first CGImage using the various CGImageGet... functions.
I need to be able take a video from Photos and re-rendering, both clipping it in time, changing the width and height, and frame rate. Certainly I need to start with:
PHContentEditingInputRequestOptions *options = [[PHContentEditingInputRequestOptions alloc] init];
[self.asset requestContentEditingInputWithOptions:options completionHandler:^(PHContentEditingInput *contentEditingInput, NSDictionary *info) {
// Get full image
NSURL *url = [contentEditingInput fullSizeImageURL];
}];
And I should be able to adjust width, height and duration. Grab an NSData from that, write that out to the file syset.m
But the url is nil, which implies to me that I can't edit videos with the new Photos framework. (ALAsset didn't have a problem with this using AVAssetExportSession.) This makes sense since the Apple Dev sample code can't edit videos either.
Now, to make life easier I could just pass that url to an AVAssetExportSession but I can't, because it is nil. If I just modified width, height and duration I'd still need to grab an NSData from it, write that out to the file system.
I do not need to write the modified video back to Photos, I actually need the video on the file system since I'll be uploading it to our servers.
fullSizeImageURL is for working with Photo assets. You want the avAsset property when working with a video. Modify the actual video, not the metadata, by writing a new video file.
To do that, you could use that avAsset in an AVMutableComposition:
Insert the appropriate time range of the avAsset's video track (AVAssetTrack) into an AVMutableCompositionTrack. That'll do your trimming.
Place/size it appropriately using layer instructions. (AVMutableVideoCompositionLayerInstruction) to do your cropping and scaling.
I'm using CameraCaptureTask chooser and always is returning a smaller photo that the original. Camera has a resolution of 2592x1944, but the returned photo is always 1296x972. I'm using a LG E-900.
Any ideas ?
Any JPG or PNG image file that is loaded into a BitmapImage object on Windows Phone will automatically be re-sized to be less than 2000x2000 to reduce the internal memory used to display the image to the screen. If you must access the image at a higher resolution you will need access to the original byte stream (e.ChosenPhoto from the completed event for example) and then load that into a System.Windows.Media.Imaging.WriteableBitmap object.
The issue then becomes that you may need to know the original size of the image, as the WriteableBitmap takes a size and width as part of it's constructor and will automatically re-size the image that you try and load into it. I believe the only way to load a high resolution image at it's original size is to use ExifLib on the byte stream to detect the original height and width, then create a WriteableBitmap at that size and use the System.Windows.Media.Imaging - Extensions.LoadJpeg method to load the image into the object. For more image loading samples see this question regarding re-sizing an image on Windows Phone.
Sorry - but there's currently no way to do this - see Is it possible to configure the resolution of `CameraCaptureTask` in WP7?
if you are lucky, then the new Mango SDK might help - it's now 2 weeks away.
Given a sample buffer of H.264, is there a way to extract the frame it represents as an image?
I'm using QTKit to capture video from a camera and using a QTCaptureMovieFileOutput as the output object.
I want something similar to the CVImageBufferRef that is passed as a parameter to the QTCaptureVideoPreviewOutput delegate method. For some reason, the file output doesn't contain the CVImageBufferRef.
What I do get is a QTSampleBuffer which, since I've set it in the compression options, contains an H.264 sample.
I have seen that on the iPhone, CoreMedia and AVFoundation can be used to create a CVImageBufferRef from the given CMSampleBufferRef (Which, I imagine is as close to the QTSampleBuffer as I'll be able to get) - but this is the Mac, not the iPhone.
Neither CoreMedia or AVFoundation are on the Mac, and I can't see any way to accomplish the same task.
What I need is an image (whether it be a CVImageBufferRef, CIImage or NSImage doesn't matter) from the current frame of the H.264 sample that is given to me by the Output object's call back.
Extended info (from the comments below)
I have posted a related question that focusses on the original issue - attempting to simply play a stream of video samples using QTKit: Playing a stream of video data using QTKit on Mac OS X
It appears not to be possible which is why I've moved onto trying to obtain frames as images and creating an appearance of video, by scaling, compressing and converting the image data from CVImageBufferRef to NSImage and sending it to a peer over the network.
I can use the QTCapturePreviewVideoOutput (or decompressed) to get uncompressed frame images in the form of CVImageBufferRef.
However, these images references need compressing, scaling and converting into NSImages before they're any use to me, hence the attempt to get an already scaled and compressed frame from the framework using the QTCaptureMovieFileOutput (which allows a compression and image size to be set before starting the capture), saving me from having to do the expensive compression, scale and conversion operations, which kill CPU.
Does the Creating a Single-Frame Grabbing Application section of the QTKit Application Programming Guide not work for you in this instance?
I'm recording from a webcam. The camera looks great in PhotoBooth. However, when I preview it in my program with a QTCaptureView, or record it to a file, it is very, very slow. The reason is that QuickTime is giving me the maximum possible resolution of 1600x1200. How can I force a more reasonable size for both my QTCaptureView and my recording to file?
As described here, you can set the pixel buffer attributes within the output from your QTCaptureSession to change the resolution of the video being captured. For example:
[[[myCaptureSession outputs] objectAtIndex:0] setPixelBufferAttributes: [NSDictionary dictionaryWithObjectsAndKeys:
[NSNumber numberWithInt:480], kCVPixelBufferHeightKey,
[NSNumber numberWithInt:640], kCVPixelBufferWidthKey, nil]];
will set the video resolution to be 640x480 for the first output in your capture session. This should also adjust the camera settings themselves to have it return image frames of that size (if supported by the camera hardware).
You may also wish to use base MPEG4 encoding, instead of h.264, to do your realtime video recording. This can be set using code similar to the following:
NSArray *outputConnections = [mCaptureMovieFileOutput connections];
QTCaptureConnection *connection;
for (connection in outputConnections)
{
if ([[connection mediaType] isEqualToString:QTMediaTypeVideo])
[mCaptureMovieFileOutput setCompressionOptions:[QTCompressionOptions compressionOptionsWithIdentifier:#"QTCompressionOptionsSD480SizeMPEG4Video"] forConnection:connection];
}
h.264 encoding, particularly the Quicktime implementation, uses a lot more CPU power to encode than the base MPEG4.
The solution above (setPixelBufferAttributes:) does set the preview size correctly, but once movie recording starts, the preview image will get set back to it's original value (1280 x 1024 on my MBP) if you've set (almost) any compression options.
If that was just during movie recording that would be one thing, but once recording is complete, further calls to setPixelBufferAttributes will have no effect.
So, you can change the preview image size, as long as you don't plan on doing any actual compressed movie recording.
This is on 10.5.8/9L30, MBP with a GeForce 8600M. Any compression option except for no compression or QTCompressionOptionsSD240SizeH264Video breaks as described above.
rdar://7447812
To add more information about the topic:
you can't specifiy directly the definition on the capture side. Rather, this is the output of the capture session that defines the definition. e.g.
if you capture into a QtCaptureDecompressedVideoOutput, you shall specify the definition on this object.