This has been driving me mad for months: I have a little app to preview camera raw images. As the files in question can be quite big and stored on a slow network drive I wanted to offer the user a chance to stop the loading of the image.
Handily I found this thread:
Cancel NSData initWithContentsOfURL in NSOperation
and am using Nick's great convenience method to cache the data and be able to issue a cancel request halfway through.
Anyway, once I have the data I use:
NSImage *sourceImage = [[NSImage alloc]initWithData:data];
The problem comes when looking at Nikon .NEF files; sourceImage returns only a thumbnail and not the full size. Displaying Canon .CR2 files and in fact, any other .TIFF's and .JPEG's seems fine and sourceImage is the expected size. I've checked the amount of data that is being loaded (with NSLog and [data length]) and it does seem that all of the Nikon files' 12mb is there for the -initWithData:
If I use
NSImage *sourceImage = [[NSImage alloc]initWithContentsOfURL:myNEFURL];
then I get the full sized image of the Nikon files but of course the app blocks.
So after poking around for what is beginning to feel like my entire life I think I know that the problem is related to the Nikon's metadata stating that the file's DPI is 300 whereas Canon et al is 72.
I hoped a solution would be to lazily access the file with:
NSImage*tempImg = [[NSImage alloc] initByReferencingURL:myNEFURL];
and having seen similar postings here and elsewhere I found a common possible answer of simply
[sourceImage setSize:tempImg.size];
but of course this just resizes the tiny thumbnail up to 3000x2000 or thereabouts.
I've been messing with the following hoping that they would provide a way to get the big picture from the .NEF:
CGImageSourceRef isr = CGImageSourceCreateWithData((__bridge CFDataRef)data, NULL);
CGImageRef isrRef = CGImageSourceCreateImageAtIndex(isr, 0, NULL);
and
NSBitmapImageRep *bitMapIR = [[NSBitmapImageRep alloc] initWithData:data];
But checking the sizes on these show similar thumbnail widths and heights. In fact, isrRef returns an even smaller thumbnail, one that is 4.2 times smaller. Perhaps worth noting that 300 / 72 == 4.2, so isrRef is taking account of the DPI on an image where the DPI (possibly) has already been observed.
Please! Can someone [nicely] put me out of my misery and help me get the full-sized image from the loaded data?!?! Currently, I'm special-case'ing the NEF files with a case insensitive search on the file extension and then loading the URL with the blocking methods. I have to take a hit on the app blocking and the search can't be fool-proof in the long run.
As an aside: is this actually a bug in the OS? It does seem like NSImage's -initWithData: and -initWithContentsOfURL: methods use different engines to actually render the image. Would it not be reasonable to have assumed that -initWithURL: simply loads the data which then gets rendered just as though it had been presented to the class with -initWithData: ?
It's a bug - confirmed when I did a DTS. Apparently I need to file a bug report. Currently the only way is to use the NSURL methods. Instead of checking the file extension I should probably traverse the meta dictionaries and check the manufacturer's entry for "Nikon", though...
Related
I am creating a simple photo catalogue application for macOS to see whether the latest APIs can significantly improve performance of loading directories with large numbers of images.
So far it looks pretty promising and loading around 600 45MB RAW image thumbnails using QLThumbnailGenerator and CGImageSourceCreateWithURL is super fast allowing thumbnail images and image metadata to be displayed almost instantly.
Displaying these images in a NSCollectionView using a CALayer in the NSCollectionViewItem's view also appears to be extremely fast and scrolling is very smooth.
I did find that QLThumbnailGeneratorseems to start failing after a few hundred images and starts returning error code 108 if I call the api in a continuous loop - I fixed that by calling CGImageSourceCopyPropertiesAtIndex immediately after the thumbnail generator api call - so maybe there is a timing issue or not enough file handles or something if the api is called to quickly and for too long.
However I am still having trouble rendering a full sized image to the display - here I am using a NSScrollView with a layer backed NSView documentView. Everything is super fast until the following call:
view.layer.contents = cgImage
And at this point the entire main thread hangs until the image has loaded - and this may take a few seconds.
Once it has loaded it's fine and zooming in and out by changing the documentView frame size is very fast - scrolling around the full size image is also super smooth without any of the typical hiccups.
Is there a way of loading these images without causing the UI to freeze ?
I've seen the recent WWDC2020 session where they demonstrate similar scrolling of large numbers of images but I haven't been able to find anything useful on loading large images other than CATiledLayer - but it's not really clear if that is the right answer for this problem.
The old Apple sample RawExpose seemed to be an option but most of that code is deprecated and it seems one has to use MetalKit not instead of GLKit - unfortunately there is no example of using MetaKit with Core Image that I can find.
FYI - I tried using some the new SwiftUI CollectionView and List but they seem to be significantly slower than AppKit and I found some of the collection view items never render - of course these could just be bugs in the macOS 11 beta.
OK - well I finally figured it out and it's complicated but simple. It's complicated because there are so many options to choose from and so many outdated sample apps to look at. In any event I think I have solved most if not all the issues related to using metal backed CALayers and rendering realtime updates of the images as CIFilter adjustments are applied. There are many pieces to the puzzle and happy to share if anyone is looking for help.
Some key pointers:
I am using CAMetalLayer and NSView
I override the CAMetalLayer.display(layer:) method and call the layer.setNeedsDisplay() when the user slides an adjustment slider.
I chain together all the CIFilters, including the RAW filter created with CIFilter(imageUrl:)
Most importantly I use the RAW filters scaleFactor parameter to size the image - encountered major performance issues using any other method to resize the image for the views size
Don't expect high performance if the image is zoomed right in - 50% is seems to be the limit for 45megapixel RAW imaged from Nikon D850.
A short video of the result is here https://youtu.be/5wp0CIWAoIM
I am downloading cover images uploaded by App.net users. App.net requires these cover images to be at least 960 pixels wide. I fetch them with a simple AFImageRequestOperation:
NSURLRequest *urlRequest = [NSURLRequest requestWithURL:URL];
AFImageRequestOperation *imageRequestOperation = [AFImageRequestOperation imageRequestOperationWithRequest:urlRequest success:^(UIImage *image) {
if (completionHandler) {
completionHandler(image); // Load image into UI...
}
}];
[self.fetchQueue addOperation:imageRequestOperation];
This is working, no memory spikes.
I want to cache the authenticated users' images so users don't have to download them each time the app opens. As soon as I archive the downloaded image to disk, I get huge spikes in memory. For example, my cover image is currently 3264 x 2448 pixels. When downloaded on my Mac it comes to around 1,3 MB. However, as soon as I create a NSData object with either UIImagePNGRepresentation(image) or via TMCache's setObject:forKey: method, the app's used memory spikes to around 60,0 MB.
For clarity, This is all I'm doing to write the file to disk:
dispatch_async(dispatch_get_global_queue(DISPATCH_QUEUE_PRIORITY_BACKGROUND, 0), ^{
NSURL *fileURL = ... // URL of file in "/Application Support"
NSData *imageData = UIImagePNGRepresentation(imageToSave);
[imageData writeToURL:fileURL atomically:YES];
});
Can anyone tell me what is going on? Why is a 1,3 MB being extrapolated into almost sixty times that. How can I avoid this massive and potentially crippling inflation. This is one image, what if the user opens several profiles, each with a cached image?
The image dimensions are what have the greatest bearing on memory usage. For a given image size (regardless of PNG, JPG), the memory usage is pretty much the same and is given by: width x height x 4 bytes.
A cover image of 3264x2448 would decode to roughly 32MB. Perhaps the atomic write explains the doubling you see.
Spikes like this may be unavoidable if that's the size of the image you need to work with. The important thing is to make sure the memory usage isn't growing without bound. When you run the app and look at the memory instrument gauge, does it eventually go down as your app does its work? You can also try wrapping the image-writing code in an #autoreleasepool.
I'm trying to figure an easy way to create screenshots at 1200x800 from nearly one hundred HTML files using QuickLook. This line pretty much sums up with I'm doing:
CGImageRef imageRef = QLThumbnailImageCreate(NULL, (__bridge CFURLRef) [NSURL fileURLWithPath:layoutHtml isDirectory:NO], CGSizeMake(1280 , 800), (__bridge CFDictionaryRef) #{ (NSString*) kQLThumbnailOptionIconModeKey:#(NO)});
Unfortunately the create image does not contain any images used in the HTML file, I only get the question mark "image not found" placeholder. When I use QuickLook from the Finder, images get loaded.
Any ideas on how I could convince QLThumbnailImageCreate to include images?
Thanks,
Ilja
I don't think this is possible unless there is hidden Preview Property.
The Generating Enriched HTML also does not mention anything about loading external resources, only kQLPreviewPropertyAttachmentsKey which requires the HTML to use the cid:identifier URL scheme.
There is webkit2png which is a python script that does what you need. Searching for WebKit screenshot solution also brings up some Cocoa code snippets.
I'm looking for the best way to quickly and repeatedly "blit" RGB bitmap data to a specific area within a Mac OS X window, for the purpose of displaying video frames coming from a custom video engine in real time. The data is in a simple C-style array containing a 32-BPP bitmap.
On Win32, I'd setup HWND and HDC's, copy the raw data into its memory space, and then use BitBlt(). On iOS, I've done it via UIImageView, although I didn't fully assess the performance of that approach (really didn't need to in that particular limited case). I have neither available to me on Mac OS X with Cocoa, so what should I do?
I know there are several bad or convoluted ways for me to accomplish this, but I'm hoping someone with experience can point me to something that's actually meant for this use and/or is performance efficient while being reasonably straightforward and reliable.
Thanks!
I would recommend either creating NSImages or CGImages with your data and then drawing them to the current context.
If you use NSImage, you'll need to create an NSBitmapImageRep with the data of your image. You don't need to copy the data, just pass the pointer to it as one of the parameters to the initializer.
If you use CGImage, you can create a CGBitmapContextRef using CGBitmapContextCreate(), and as above, just pass a pointer to the existing image data. Then you can create a CGImage from it by calling CGBitmapContextCreateImage().
This did the trick... (32-BPP RGBA bitmap data)
int RowBytes = Width * 4;
NSBitmapImageRep * ImageRep = [[NSBitmapImageRep alloc] initWithBitmapDataPlanes:&Data pixelsWide:Width pixelsHigh:Height bitsPerSample:8 samplesPerPixel:4 hasAlpha:YES isPlanar:NO colorSpaceName:NSCalibratedRGBColorSpace bytesPerRow:RowBytes bitsPerPixel:32];
NSImage * Image = [[NSImage alloc] init];
[Image addRepresentation:ImageRep];
[ImageView setImage:Image];
Compared to a Windows bitmap, the Red and Blue channels are swapped (RGBA vs BGRA), and of course the Y rows are in opposite order (ie upside-down), but that's all easy enough to accommodate by manipulating the source data.
If I create an NSImage via something like:
NSImage *icon = [NSImage imageNamed:NSImageNameUser];
it only has one representation, a NSCoreUIImageRep which seems to be a private class.
I'd like to archive this image as an NSData but if I ask for the TIFFRepresentation I get a
small icon when the real NSImage I originally created seemed to be vector and would scale up to fill my image views nicely.
I was kinda hoping images made this way would have a NSPDFImageRep I could use.
Any ideas how can I get an NSData (pref the vector version or at worse a large scale bitmap version) of this NSImage?
UPDATE
Spoke with some people on Twitter and they suggested that the real source of these images are multi resolution icns files (probably not vector at all). I couldn't find the location of these on disk but interesting to hear none-the-less.
Additionally they suggested I create the system NSImage and manually render it into a high res NSImage of my own. I'm doing this now and it's working for my needs. My code:
+ (NSImage *)pt_businessDefaultIcon
{
// Draws NSImageNameUser into a rendered bitmap.
// We do this because trying to create an NSData from
// [NSImage imageNamed:NSImageNameUser] directly results in a 32x32 image.
NSImage *icon = [NSImage imageNamed:NSImageNameUser];
NSImage *renderedIcon = [[NSImage alloc] initWithSize:CGSizeMake(PTAdditionsBusinessDefaultIconSize, PTAdditionsBusinessDefaultIconSize)];
[renderedIcon lockFocus];
NSRect inRect = NSMakeRect(0, 0, PTAdditionsBusinessDefaultIconSize, PTAdditionsBusinessDefaultIconSize);
NSRect fromRect = NSMakeRect(0, 0, icon.size.width, icon.size.width);;
[icon drawInRect:inRect fromRect:fromRect operation:NSCompositeCopy fraction:1.0];
[renderedIcon unlockFocus];
return renderedIcon;
}
(Tried to post this as my answer but I don't have enough reputation?)
You seem to be ignoring the documentation. Both of your major questions are answered there. The Cocoa Drawing Guide (companion guide linked from the NSImage API reference) has an Images section you really need to read thoroughly and refer to any time you have rep/caching/sizing/quality issues.
...if I ask for the TIFFRepresentation I get a small icon when the
real NSImage I originally created seemed to be vector and would scale
up to fill my image views nicely.
Relevant subsections of the Images section for this question are: How an Image Representation is Chosen, Images and Caching, and Image Size and Resolution. By default, the -cacheMode for a TIFF image "Behaves as if the NSImageCacheBySize setting were in effect." Also, for in-memory scaling/sizing operations, -imageInterpolation is important: "Table 6-4 lists the available interpolation settings." and "NSImageInterpolationHigh - Slower, higher-quality interpolation."
I'm fairly certain this applies to a named system image as well as any other.
I was kinda hoping images made [ by loading an image from disk ] would
have a NSPDFImageRep I could use.
Relevant subsection: Image Representations. "...with file-based images, most of the images you create need only a single image representation." and "You might create multiple representations in the following situations, however: For printing, you might want to create a PDF representation or high-resolution bitmap of your image."
You get the representation that suits the loaded image. You must create a PDF representation for a TIFF image, for example. To do so at high resolution, you'll need to refer back to the caching mode so you can get higher-res items.
There are a lot of fine details too numerous to list because of the high number of permutations of images/creation mechanisms/settings/ and what you want to do with it all. My post is meant to be a general guide toward finding the specific information you need for your situation.
For more detail, add specific details: the code you attempted to use, the type of image you're loading or creating -- you seemed to mention two different possibilities in your fourth paragraph -- and what went wrong.
I would guess that the image is "hard wired" into the graphics system somehow, and the NSImage representation of it is merely a number indicating which hard-wired graphic it is. So likely what you need to do is to draw it and then capture the drawing.
Very generally, create a view controller that will render the image, reference the VC's view property to cause the view to be rendered, extract the contentView of the VC, get the contentView.layer, render the layer into a UIGraphics context, get the UIImage from the context, extract whatever representation you want from the UIImage.
(There may be a simpler way, but this is the one I ended up using in one case.)
(And, sigh, I suppose this scheme doesn't preserve scaling either.)