I understand the use of thumbnail in network applications but assuming all the image are in the application itself (photo application), in this case do we still need thumbnail images for performance reasons or is it just fine for the device to resize the actual image on run time?
Since the question is too opinion based I am going to ask more quantitively.
The images are 500x500, about 200-300kb size jpg.
There will be about 200 images.
It is targeted for iphone4 and higher, so that would be the minimum hardware specs users will have.
The maximum memory used should not pass 20% of the devices capacity.
Will the application in this case need separate thumbnail images?
It depends on your application. Just test performance and memory usage on device.
If you show a lot of images and/or they change very quickly (like when you are scrolling UITableView with a lot of images) you will probably have to use thumbnails.
UPDATE:
When image is shown it takes width * height * 3 (width * height * 4 for images with ALPHA channel) bytes of memory. 10 photos 2592 x 1936 stored in memory will require 200Mb of RAM. It is too much. You definitely have to use thumbnails.
Your question is a bit lacking on detail but I assume you're asking if, for say a photo album app, can you just throw around full size UIImages and let a UIImageView resize them to fit on the screen, or do you need to resize?
You absolutely need to resize.
An image taken by an iPhone camera will be several megabytes in compressed file size, more in actual bytes used to represent pixels. The dimensions of the image will be far greater than the screen dimensions of the device. The memory use is very high, particularly if you're thinking of showing multiple "thumbnails". It's not so much a CPU issue (once the image has been rendered it doesn't need re-rendering) but a memory one, and you're severely memory constrained on a mobile device.
Each doubling in size of an image (e.g. from a 100x100 to a 200x200) represents a four-fold increase in the memory needed to hold it.
Related
I have a page with many large fluid images. http://altarjewelry.com/gallery
I want to get a smooth 60fps webapp feel while scrolling. The Chrome DevTools tell me my paint times are the biggest problem (which you can check for yourself while scrolling). I'm assuming this is due to my many large fluid images.
I've read every article on HTML5Rocks about performance. I found many good tips on JS performance but no help optimizing large image paint times other then using small fixed size images, which is not an option for me as I'm building a responsive site.
I'm already serving up responsive images depending on the client.
Thank you for your help.
Not really sure about how your gallery looks because it never loaded from the URL in your post, and I don't know if that's a javascript issue or what--but I'll take a stab at helping you come up with a solution. Image optimization is image optimization, regardless of whether or not you're building a responsive site.
Approach and Design Considerations
Do you really need one large, high resolution image for each item, at the same DPI/PPI and compression, that should be responsive?
Or, should you serve appropriately sized images at differing DPI/PPI and compression, to different displays, all of which are still used in a responsive application?
Popular Convention
You're showing a gallery, and typically, you want smaller representations of the actual image--thumbnails or placeholders, generally of lower resolution, which link to the actual image at a higher resolution. This is an accepted design approach, and if you're going to vary from it, be sure it's with good reason.
The Lowest Common Denominator
If you're building a responsive site, some users will obviously be on mobile devices which may have resolutions as small as 320 pixels wide. Consider things like that, and this: even if someone shows up on a desktop, are you going to have huge, full width images loading? They will take forever to load, and visitors will never see your gallery. How is your gallery to look on a wide screen desktop? If your intention is to have one image full width across the entire page, and load the same image regardless of the device accessing your site, you may be using responsive design, but you'll find that's far away from best or even good practice.
The Flip-Side, Large/Wide Screens
Why not have four gallery images going across a desktop? Or more? And if that's the case, they're likely to have a maximum size in any case. I honestly don't know because I've tried to load your site a few times and get nothing. But consider that if there's a maximum size practically for your gallery images in an initial display, say 6 images at 200 pixels each across a 1200 pixel max layout width (Or, are you using a % based framework and using 100% of the display width? Even responsive sites often limit the max width of the content area, and these things all would help determining a more appropriate answer) solutions begin to emerge.
Since no image needs to be larger than 200 pixels in that case, and on a phone where your columns might be displaying only one image that you want full width, you can work with a maximum initial width of 480px wide images.
Higher Quality, Smaller Files
We'll assume you want them high quality. That's fine. You still need to reduce files size, and you do that with compression. Now, you may feel compressing a photo to 50% or even more makes it blurry, and it certainly will at low ppi (pixels per inch) settings.
The Secret To Better Compression
What you need to do is change default image editor settings from traditional defaults like 72 or 90 ppi, and crank them up to 300, 400, 500, or more--and THEN apply compression. If that image is 480px wide, and you've only got 72ppi, compression will quickly erode quality. However, having several hundred extra pixels per inch will allow more information to be stored. Then, you can apply much higher compression rates, and shrink file sizes down quite a bit more.
The Oversized Image Approach
Another trick is to do the same thing, and slightly oversize the image. If 480px is the max size for your thumbnails/small pics, make them actually 540-600 px wide, with 400-500ppi and compress them at really high settings. The browser will resize to the max width of 480 px...but then you have a performance hit there. Everything is a trade off. You can blur backgrounds in images as well, allowing the foreground/main focus of the photo to be of higher quality while the background requires less information, yielding smaller file sizes.
Not Suitable For Batch Processing
This should be done individually for each image, batch editing does not generally get the most out of this technique, because the color information is so different in each photo. One photo might be best quality and smallest size for your purposes at 300ppi and 50% quality, another at 500ppi and 35% quality. You'll want to do this not just for your gallery thumbnails, but multiple images. No point in serving up a 1400px wide full size desktop to someone who's browsing your site with 480px wide/resolution display after all. Use media queries to serve up the appropriate ballpark sized image, and have a small, medium and large variant. Done right, you don't even need to be serving larger images to those browsing with phones...the gallery images they are viewing are good enough.
The compression setting is not so heavily determinate of the final image quality as the number of pixels you have to compress goes up. More pixels to work with, the better quality at higher compression settings.
Design Considerations and Smart Image Loading
Break It Up Into Smaller Content Chunks
Also, consider the process/design of your gallery. Do you have 20 items? 100? 400? Are you trying to show them all on one page? Break it up into small numbers...12-20 per page. Smaller and fewer images will load faster, and can remain responsive, with links for those who want a larger or higher quality image. No need to show a huge, high quality image to someone browsing with their phone.
Pre-fetching and Loading
Server side scripting, and even some javascript solutions can help with this. You might do things like limiting each gallery page to four rows of four images, and then after page load, have a javascript that pre-fetches the first four images that will display on page 2. If your visitor goes to page two after scrolling through page one, the first four images are loaded in cache, and display quickly while the others load normally, giving the experience of a faster page load.
If the visitor goes to another page in the site, you didn't waste bandwidth on 12 images and only cost you the bandwidth of four. Smart design might be to use those first four images on page two of the gallery elsewhere on the site...so that first gallery page visit actually sped up page load elsewhere and does not in fact give up bandwidth for loading 4 unnecessary images. Think the process through, and solutions will suggest themselves.
Resources
Anyway, here are relevant articles/posts/links you may find helpful in understanding all of this:
Are Compressive Images A Good Solution For High Resolution Displays?
http://www.vanseodesign.com/web-design/compressive-image-tests/
Reducing image sizes (ResponsiveDesign.is)
https://responsivedesign.is/articles/reducing-image-sizes
Search benfrain dot com for this post:
How to serve high-resolution website images for retina displays
And a tool you might find useful...
adaptive-images dot com
With the new screen resolution and aspect ratio of iPhone 5 it seems there are two approaches to take advantage of the new screen real estate when handling images that are full screen (or mostly full screen) images:
Include two images bg#2x.png and bg-h568#2x.png and check the device's main screen bounds to figure out which to load.
Only include the one file but make it the full iPhone 5 resolution
(There are other methods also: stretching, tiling, and / or drawing with Quartz.)
Both of these options have drawbacks:
Including two files is going to increase your app's bundle size
Loading a iPhone 5 retina image on a 4 will clip the image data when displayed on screen, but will still consume the same amount of memory resources
I am particularly interested in understanding the details of the second drawback better.
I am thinking it would save the app bundle (and my designer) if I could use this consolidating images idea further...
Let's say I have an image that take up 320pt x 480pt in portrait and 480pt x 320pt in landscape (width x height).
(EDIT: Initially I hadn't considered a couple of other important tidbits. For sake of completeness, I'm including those thoughts here)
If we create a single image file at a 480pt x 480pt resolution instead of landscape and portrait files, it would prevent the app bundle from having two files that which each contain 320pt x 320pt area of duplicate image data. However, this large square image would include four 80pt x 80pt quadrants in the corners that won't be seen. All things equal, we should still have 20% less image data in the app bundle.
Having re-read Bill's "iBooks Author Experiment", the memory requirements are very easy to figure out:
Landscape or Portrait: 640 * 960 * 4 = 2,457,600
Large Square: 960 * 960 * 4 = 3,686,400
The memory required to load the large square is 50% more then loading the image sized correctly for the screen. Clearly the savings in the app bundle don't measure up to this hit on resources!
But, what about extending this to a giant square that would encompass iPhone 5 and iPhone 4 - 568pt x 568pt? Instead of four files (iP4 landscape, iP4 portrait, iP5 landscape, iP5 portrait), there would be only one file in the app bundle (giant square). The savings could be about 60%.
What about the resources hit?
iPhone 5 Landscape or Portrait: 1136 * 640 * 4 = 2,908,160
Giant Square: 1136 * 1136 * 4 = 5,161,984
That's 110% more memory for iPhone 4, and 77.5% more memory for iPhone 5.
My original question was roughly two parts: 1. Do I understand this correctly and 2. is this tradeoff a wise one to make.
I hope my edits (spurred by Bill's answer) show that I do understand this stuff now. And, if the new found knowledge is correct, then #2 pretty much answers itself. :-)
Generally you get it.
Assuming that you have the big square image and it's some kind of tiled pattern (linen etc) then you could use ImageIO to load the cropped image at the size you need which would use some extra memory decompressing the image but once done it would consume only what is needed to fill the background.
I would suggest against scaling the image.
On the flip side though, shipping an app with 3 images (foo, foo#2x and foo-h568#2x) instead of 1 makes way more sense. The only way I'd say you should go with one image is if you are close to the 3G/LTE download size of (I think) 50MB.
I am building a map system that requires a large image (native 13K pixels wide by 20K pixels tall) to be overlayed onto an area of the US covering about 20 kilometers or so. I have the file size of the image in jpg format down to 23 MB and it loads onto the map fairly quickly. I can zoom in and out and it looks great. It's even located exactly where I need it to be (geographically). However, that 25 MB file is causing Firefox to consume an additional 1GB of memory!!! I am using Memory Restart extension on Firefox and without the image overlay, the memory usage is about 360 MB to 400 MB, which seems to be about the norm for regular usage, browsing other websites etc. But when I add the image layer, the memory usage jumps to 1.4 GB. I'm at a complete loss to explain WHY that is and how to fix it. Any ideas would be greatly appreciated.
Andrew
The file only takes up 23 MB as a JPEG. However, the JPEG format is compressed, and any program (such as FireFox) that wants to actually render the image has to uncompress it and store every pixel in memory. You have 13k by 20k pixels, which makes 260M pixels. Figure at least 3 bytes of color info per pixel, that's 780 MB. It might be using 4 bytes, to have each pixel aligned at a word boundary, which would be 1040 MB.
As for how to fix it, well, I don't know if you can, except by reducing the image size. If the image contains only a small number of colors (for instance, a simple diagram drawn in a few primary colors), you might be able to save it in some format that uses indexed colors, and then FireFox might be able to render it using less memory per pixel. It all depends on the rendering code.
Depending on what you're doing, perhaps you could set things up so that the whole image is at lower resolution, then when the user zooms in they get a higher-resolution image that covers less area.
Edit: to clarify that last bit: right now you have the entire photograph at full resolution, which is simple but needs a lot of memory. An alternative would be to have the entire photograph at reduced resolution (maximum expected screen resolution), which would take less memory; then when the user zooms in, you have the image at full resolution, but not the entire image - just the part that's been zoomed in (which likewise needs less memory).
I can think of two approaches: break up the big image into "tiles" and load the ones you need (not sure how well that would work), or use something like ImageMagick to construct the smaller image on-the-fly. You'd probably want to use caching if you do it that way, and you might need to code up a little "please wait" message to show while it's being constructed, since it could take several seconds to process such a large image.
Is there a file format limit to the PNG pixel size?
I am trying to visualize a 30.000x30.000 pixels PNG image with Firefox, but I get an error. The image opens correcly in Preview.app, although very slowly. The file size is not big, just around 3 MiB (1 bit black/white image). I am wondering if there's a technical file-format reason for this.
A naive implementation of resizing would require the image to be blown up to 2.7GB in size before it is displayed. This would clearly be too large for a normal 32-bit program to handle.
The PNG specification doesn't appear to place any limits on the width and height of an image; these are 4 byte unsigned integers, which could be up to 4294967295. http://www.libpng.org/pub/png/spec/iso/index-object.html#11IHDR
That is an odd image, but I am sure there is a reason to have such a huge image.
I can't really address the size limit, but I can address a way to get around it. Create a set of tiles of some size, and then as the user scrolls, bring tiles into view using CSS to position them correctly. You might even be able to get away with bringing up all the tiles at once, with a slew of smaller images.
But I am curious, what is the application that needs such a huge image displayed without scaling out?
Erick
So here's the situation:
I have a CALayer that is the size of my screen, and I'm setting the contents property to a 2 Mb JPEG that's roughly 3500 x 2000 pixels in size with a resolution of 240ppi.
I'd expect there to be a slight overhead involved in using the CALayer, but my sample application (which only does exactly what's above) shows usage of about 33Mb RSIZE, 22Mb RPVT and 30Mb RSHRD. I've noticed that these numbers are much better when running the application as 64-bit than they are running as a 32-bit process.
I'm doing everything I can think of in the real application that this example comes from, including resampling my CGImageRefs to only be the size of the layer, but this seems extraneous to me - shouldn't it be simpler?
Has anyone come across good methods to reduce the amount of memory CALayers and CGImageRefs use?
First, you're going to run into problems with an image that size in a plain CALayer, because you may hit the texture size limit of 2048 x 2048 (depending on your graphics card). Applications like this are what CATiledLayer is designed for. Bill Dudney has some code examples on his blog (a large PDF), as well as with the code that accompanies his book.
It isn't surprising to me that such a large image would take so much memory, given that it will be stored as an uncompressed bitmap in your CGImage. Aside from scaling your image to the resolution you need, and tiling it with CATiledLayer, I can't think of much. Are you releasing the CGImageRef once you've assigned it to the contents of the CAlayer? You won't need to hang onto it at that point.