I'm looking to do some javascript powered animation via image clipping. Here's an example of what I'm talking about: http://www.def-logic.com/_dhtml/freejack/hero1.gif
I know png uses a kind of prediction in its compression, what would be the best way to lay out an image like the one above so that I get the most out of the compression? I'm especially interested when the images are very similar, more so than the one above, so there is a lot of potential for compression due to redundancy.
For example, is there specific size of tile that would work well?
For example, is there specific size of tile that would work well?
Not really. PNG prediction is strictly local (it uses the 3 neighbours pixels), and the prediction ("filter") strategy can be chosen on a line basis.
That kind of redundancy is not very detectable in PNG compression (not in JPG or practically any other, actually).
If you have the freedom to select the distribution of tiles (few or many per row), you can try vary that, it can have some small influence (to have an image with many short lines instead of few long lines can give the filter better opportunities to select different filters) but, again, I'd bet that the difference will be very small.
Related
I am working on a project I wanted to do for quite a while. I wanted to make an all-round huffman compressor, which will work, not just in theory, on various types of files, and I am writing it in python:
text - which is, for obvious reasons, the easiet one to implement, already done, works wonderfully.
images - this is where I am struggling. I don't know how to approach images and how to read them in a simple way that it'd actually help me compress them easily.
I've tried reading them pixel by pixel, but somehow, it actually enlarges the picture instead of compressing it.
What I've tried:
Reading the image pixel by pixel using Image(PIL), get all the pixels in a list, create a freq table (for each pixel) and then encrypt it. Problem is, imo, that I am reading each pixel and trying to make a freq table out of that. That way, I get way too many symbols, which leads to too many lengthy huffman codes (over 8 bits).
I think I may be able to solve this problem by reading a larger set of pixels or anything of that sort because then I'd have a smaller code table and therefore less lengthy huffman codes. If I leave it like that, I can, in theory, get 255^3 sized code table (since each pixel is (0-255, 0-255, 0-255)).
Is there any way to read larger amount of pixels at a time (>1 pixel) or is there a better way to approach images when all needed is to compress?
Thank you all for reading so far, and a special thank you for anyone who tries to lend a hand.
edited: If huffman is a real bad compression algorithm for images, are there any better ones you can think off? The project I'm working on can take different algorithms for different file types if it is neccessary.
Encoding whole pixels like this often results in far too many unique symbols, that each are used very few times. Especially if the image is a photograph or if it contains many coloured gradients. A simple way to fix this is splitting the image into its R, G and B colour planes and encoding those either separately or concatenated, either way the actual elements that are being encoded are in the range 0..255 and not multi-dimensional.
But as you suspect, exploiting just 0th order entropy is not so great for many images, especially photographs. As example of what some existing formats do, PNG uses filters to take some advantage of spatial correlation (great for smooth gradients), JPG uses quantized discrete cosine transforms and (usually) a colour space transformation to YCbCr (to decorrelate the channels, and to crush Chroma more mercilessly than Luma) and (usually) Chroma subsampling, JPEG2000 uses wavelets and colour space transformation both in its lossy and lossless forms (though different wavelets, and a different colour space transformation) and also supports subsampling though dropping a wavelet scale achieves a similar effect.
I'm working on a system that does fuzzy image deduplication.
Right now, I have a functional system that can do large-scale phash fuzzy image searching and deduplication via either DCT-based or gradient-based perceptual hashes.
However, while determining if an image has been reduced in size is programatically trivial, how can I determine which image is the parent of which?
Basically, if I have two images with the same resolution, where one is a resaved version of the other (either in a different format (jpg/png), or simply recompressed), how can I determine which one is the original in a reliable manner?
(Note: Assume all metadata has been stripped from the images, I wish it were that simple.)
Bonus points if any solution is fairly easy to implement in python.
This isn't a positive answer, but I spent a while of time evaluating the use of average entropy per-pixel to determine if it could be a useful metric for determining how compressed an image is.
I have a write up here.
Some excerpts:
Variance in entropy across compression levels on SIPI reference image database images.
In retrospect, the x-axis should be labeled "JPEG Quality level". Higher numbers mean better quality
While per-pixel entropy does decline sharply at extremely aggressive compression levels, it does not vary in a way that correlates well with compression level.
This means that any attempt to compare two images by inspecting the entropy will likely have issues unless one knows exactly what compression level the image had been resaved at.
I'm curious about whether there are approaches or algorithms one might use to downscale an image based on the amount of detail or entropy in the image such that the new size is determined to be a resolution at which most of the detail of the original image would be preserved.
For example, if one takes an out-of-focus or shaky image with a camera, there would be less detail or high frequency content than if the camera had taken the image in focus or from a fixed position relative to the scene being depicted. The size of the lower entropy image could be reduced significantly and still maintain most of the detail if one were to scale this image back up to the original size. However, in the case of the more detailed image, one wouldn't be able to reduce the image size as much without losing significant detail.
I certainly understand that many lossy image formats including JPEG do something similar in the sense that the amount of data needed to store an image of a given resolution is proportional to the entropy of the image data, but I'm curious, mostly for my own interest, if there might be a computationally efficient approach for scaling resolution to image content.
It's possible, and one could argue that most lossy image compression schemes from JPEG-style DCT stuff to fractal compression are essentially doing this in their own particular ways.
Note that such methods almost always operate on small image chunks rather than the big picture, so as to maximise compression in lower detail regions rather than being constrained to apply the same settings everywhere. The latter would likely make for poor compression and/or high loss on most "real" images, which commonly contain a mixture of detail levels, though there are exceptions like your out-of-focus example.
You would need to define what constitutes "most of the detail of the original image", since perfect recovery would only be possible for fairly contrived images. And you would also need to specify the exact form of rescaling to be used each way, since that would have a rather dramatic impact on the quality of the recovery. Eg, simple pixel repetition would better preserve hard edges but ruin smooth gradients, while linear interpolation should reproduce gradients better but could wreak havoc with edges.
A simplistic, off-the-cuff approach might be to calculate a 2D power spectrum and choose a scaling (perhaps different vertically and horizontally) that preserves the frequencies that contain "most" of the content. Basically this would be equivalent to choosing a low-pass filter that keeps "most" of the detail. Whether such an approach counts as "computationally efficient" may be a moot point...
For an image-upload tool I want to detect the (subjective) quality of an image automatically, resulting in a rating of the quality.
I have the following idea to realize this heuristically:
Obviously incorporate the resolution into the rating.
Compress it to JPG (75%), decompress it and compare jpg-size vs. decompressed size to gain a ratio. The blurrier the image is, the higher the ratio.
Obviously my approach would use up a lot of cycles and memory if large images are rated, although this would do in my scenario (fat server, not many uploads), and I could always build in a "short circuit" around the more expensive steps if the image exceeds a certain resolution.
Is there something else I can try, or is there a way to do this more efficiently?
Assesing the image (the same goes for sound or video) quality is not an easy task, and there are numerous publications tackling the problem.
Much depends on the nature of the image - different set of criteria is appropriate for artificially created images (i.e. diagrams) or natural images (i.e. photographs). There are subtle effects that have to be taken into consideration - like color masking, luminance masking, contrast perception. For some images a given compression ratio is perfectly adequate, while for other it will result in significant loss of quality.
Here is a free-access publication giving a brief introduction to the subject of image quality evaluation.
The method you mentioned - compressing the image and comparing the result with the original is far from perfect. What will be the metric that you plan to use? MSE? MSE per block? For sure it is not too difficult to implement, but the results will be difficult to interpret (consider images with high-frequency components and without them).
And if you want to delve more into the are of image quality assessment there is also a lot of research done by the machine learning community.
You could try looking in the EXIF tags of the image (using something like exiftool), what you get will vary a lot. On my SLR, for example, you even get which of the focus points were active when the image was taken. There may also be something about compression quality.
The other thing to check is the image histogram - watch out for images biased to the left, which suggests under-exposure or lots of saturated pixels.
For image blur you could look at the high frequency components of the Fourier transform, this is probably accessing parameters relating to the JPG compression anyway.
This is a bit of a tricky area because most "rules" you might be able to implement could arguably be broken for artistic effect.
I'd like to shoot down the "obviously incorporate resolution" idea. Resolution tells you nothing. I can scale an image by a factor of 2 , quadrupling the number of pixels. This adds no information whatsoever, nor does it improve quality.
I am not sure about the "compress to JPG" idea. JPG is a photo-oriented algorithm. Not all images are photos. Besides, a blue sky compresses quite well. Uniformly grey even better. Do you think exact cloud types determine the image quality?
Sharpness is a bad idea, for similar reasons. Depth of Field is not trivially related to image quality. Items photographed against a black background will have a lot of pixels with quite low intensity, intentionally. Again, this does not signal underexposure, so the histogram isn't a good quality indicator by itself either.
But what if the photos are "commercial?" Does the value of the existing technology work if the photos are of every-day objects and purposefully non-artistic?
If I hire hundreds of people to take pictures of park benches I want to quickly know which pictures are of better quality (in-focus, well-lit) and which aren't. I don't want pictures of kittens, people, sunsets, etc.
Or what if the pictures are supposed to be of items for a catalog? No models, just garments. Would image-quality processing help there?
I'm also really interested working out how blurry a photograph is.
What about this:
measure the byte size of the image when compressed as JPEG
downscale the image to 1/4th
upscale it 4x, using some kind of basic interpolation
compress that version using JPEG
compare the sizes of the two compressed images.
If the size did not go down a lot (past some percentage threshold), then downscaling and upscaling did not lose much information, therefore the original image is the same as something that has been zoomed.
I have a very large background image (about 940x940 pixels) and I'm wondering if anyone has tips for compressing a file this large further than Photoshop can handle? The best compression without serious loss of quality from Photoshop is PNG 8 (250 KB); does anyone know of a way to compress an image down further than this (maybe compress a PNG after it's been saved)?
I don't normally deal with optimizing images this large, so I was hoping someone would have some pointers.
It will first depend on what kind of image you are trying to compress. The two basic categories are:
Picture
Illustration
For pictures (such as photographs), a lossy compression format like JPEG will be best, as it will remove details that aren't easily noticed by human visual perception. This will allow very high compression rates for the quality. The downside is that excessive compression will result in very noticeable compression artifacts.
For illustrations that contain large areas of the same color, using a lossless compression format like PNG or GIF will be the best approach. Although not technically correct, you can think of PNG and GIF will compress repetitions the same color very well, similar to run-length encoding (RLE).
Now, as you've mentioned PNG specifically, I'll go into that discussion from my experience of using PNGs.
First, compressing a PNG further is not a viable option, as it's not possible to compress data that has already been compressed. This is true with any data compression; removing the entropy from the source data (basically, repeating patterns which can be represented in more compact ways) leads to the decrease in the amount of space needed to store the information. PNG already employs methods to efficiently compress images in a lossless fashion.
That said, there is at least one possible way to drop the size of a PNG further: by reducing the number of colors stored in the image. By using "indexed colors" (basically embedding a custom palette in the image itself), you may be able to reduce the size of the file. However, if the image has many colors to begin with (such as having color gradients or a photographic image) then you may not be able to reduce the number of colors used in a image without perceptible loss of quality.
Basically it will come down to some trial-and-error to see if the changes to the image will cause any change in image quailty and file size.
The comment by Paul Fisher reminded me that I also probably wouldn't recommend using GIF either. Paul points out that PNG compresses static line art better than GIF for nearly every situation.
I'd also point out that GIF only supports 8-bit images, so if an image has more than 256 colors, you'll have to reduce the colors used.
Also, Kent Fredric's comment about reducing the color depth has, in some situtations, caused a increase in file size. Although this is speculation, it may be possible that dithering is causing the image to become less compressible (as dithering introduces pixels with different color to simulate a certain other color, kind of like mixing pigment of different color paint to end up with another color) by introducing more entropy into the image.
Have a look at http://www.irfanview.com/, is an oldy but a goody.
Have found this is able to do multipass png compression pretty well, and does batch processing way faster than PS.
There is also PNGOUT available here http://advsys.net/ken/utils.htm, which is apparently very good.
Heres a point the other posters may not have noticed that I found out experimentally:
On some installations, the default behaviour is to save a full copy of the images colour profile along with the image.
That is, the device calibration map, usually SRGB or something similar, that tells using agents how to best map the colour to real world-colours instead of device independant ones.
This image profile is however quite large, and can make some of the files you would expect to be very small to be very large, for instance, a 1px by 1px image consuming a massive 25kb. Even a pure BMP format ( uncompressed ) can represent 1 pixel in less.
This profile is generally not needed for the web, so, when saving your photoshop images, make sure to export them without this profile, and you'll notice a marked size improvement.
You can strip this data using another tool such as gimp, but it can be a little time consuming if there are many files.
pngcrush can further compress PNG files without any data loss, it applies different combinations of the encoding and compression options to see which one works best.
If the image is photographic in nature, JPEG will compress it far better than PNG8 for the same loss in quality.
Smush.It claims to go "beyond the limitations of Photoshop". And it's free and web-based.
It depends a lot on the type of image. If it has a lot of solid colors and patterns, then PNG or GIF are probably your best bet. But if it's a photo-realistic image then JPG will be better - and you can crank down the quality of JPG to the point where you get the compression / quality tradeoff you're looking for (Photoshop is very good at showing you a preview of the final image as you adjust the quality).
The "compress a PNG after it's been saved" part looks like a deep misunderstanding to me. You cannot magically compress beyond a certain point without information loss.
First point to consider is whether the resolution has to be this big. Reducing the resolution by 10% in both directions reduces the file size by 19%.
Next, try several different compression algorithms with different grades of compression versus information/quality loss. If the image is sketchy, you might get away with quite rigorous JPEG compression.
I would tile it, Unless you are absolutely sure that you audience has bandwidth.
next is jpeg2k.
To get more out of a JPEG file you can use the 'Modified Quality Setting' of the "Save as Web" dialog.
Create a mask/selection that contains white where you want to keep the most detail, eq around Text. You can use Quick-Mask to draw the mask with a brush. It helps to Feather the selection, this results in a nice white to black transition in the next step.
save this mask/selection as a channel and give the channel a name
Use File->Save as Web
Select JPEG as file format
Next to the Quality box there is a small button with a circle on it. Click that. Select the saved channel in step 2 and play with the quality setting for the white and black part of the channel content.
http://www.jpegmini.com is a new service that creates standard jpgs with an impressively small filesize. I've had good success with it.
For best quality single images, I highly recommend RIOT. You can see the original image, aside from the changed one.
The tool is free and really worth trying out.
JPEG2000 gives compression ratios on photographic quality images that are significantly higher than JPEG (or PNG). Also, JPEG2000 has both "lossy" and "lossless" compression options that can be tuned quite nicely to your individual needs.
I've always had great luck with jpeg. Make sure to configure photoshop to not automatically save thumbnails in jpegs. In my experience I get the greatest bang/buck ratio by using 3 pass progressive compression, though baseline optimized works pretty well. Choose very low quality levels (e.g. 2 or 3) and experiment until you've found a good trade off.
PNG images are already compressed internally, in a manner that doesn't benefit from more compression much (and may actually expand if you try to compress it).
You can:
Reduce the resolution from 940x940 to something smaller like 470x470.
Reduce the color depth
Compress using a lossy compression tool like JPEG
edit: Of course 250KB is large for a web background. You might also want to rethink the graphic design that requires this.
Caesium is the best tool i have ever seen.