I'm working on a software project in which I have to compare a set of 'input' images against another 'source' set of images and find out if there is a match between any of them. The source images cannot be edited/modified in any way; the input images can be scaled/cropped in order to find a match. The images can be in BMP,JPEG,GIF,PNG,TIFF of any dimensions.
A constraint: I'm not allowed to use any external libraries. ImageMagick is an exception and can be used.
I intend to use Java/Python. The software is purely command-line based.
I was reading on SO and some common image comparing algorithms. I'm planning to take 2 approaches.
1. I could use Histograms/buckets to find out the RGB values of the 2 images being compared.
2. Use SIFT/SURF to fin keypoint descriptors and find the euclidean distance between them and output the result based on the resultant distance.
The 2 images in comparison can be in different formats. An intuitive thought is that before analysis/comparison, the 2 images must be converted to a common format.I reasoned that the image should be converted to the one with lesser quality e.g. if the 2 input images are BMP and JPEG, convert the BMP to JPEG. This can be thought of as a pre-processing step.
My question:
Is image conversion to a common format required? Can 2 images of different formats be compared? IF they have to be converted before comparison, is my assumption of comparing from higher quality(BMP) to lower(JPEG) correct? It'd also be helpful if someone can suggest some algorithms for image conversion.
EDIT
A match is said to be found if the pattern image is found in the source image.
Say for example the source image consists of a football field with one player. If the pattern image contains the player EXACTLY as he is in the source image, then its a match.
No, conversion to a common format on disk is not required, and likely not helpful. If you extract feature descriptors from an image (SIFT/SURF, for example), it matters much less how the original images were stored on disk. The feature descriptors should be invariant to small compression artifacts.
A bit more...
Suppose you have a BMP that is an image of object X in your source dataset.
Then, in your input/query dataset, you have another image of object X, but it has been saved as a JPEG.
You have no idea how what noise was introduced in the encoding process that produced either of these images. There is lighting differences, atmospheric effects, lens effects, sensor noise, tone-mapping, gammut-mapping. Some of these vary from image to image, others vary from camera to camera. All this is done before the image even gets saved to storage in the camera. Yes, there are also JPEG compression artifacts, but to assume the BMP is "higher" quality and then degrade it through JPEG compression will not help. Perhaps the BMP has even gone through JPEG compression before being saved as a BMP.
I am working on an iPad application which has hundreds of photo-quality images. I would have naturally assumed to store these images as JPEGs so as to optimize the app file size. However, Apple's guidelines state:
Use the PNG format for images. The PNG format provides lossless image content, meaning that saving image data to a PNG format and then reading it back results in the exact same pixel values. PNG also has an optimized storage format designed for faster reading of the image data. It is the preferred image format for iOS.
However, if I store the same images as JPEGs at 100% quality, the size of them drops to about half that of the PNG lossless versions.
Is there really that much of a performance hit to use JPEG instead of PNG? If I am viewing these images in a carousel or gallery style, do I really need to worry about the performance and use PNGs instead?
Thanks!
Regarding the quality PNG is good for application kind of images, but JPEG is preferred for photos. Choose the lowest JPEG quality that gives good enough quality for your images.
Regarding speed, size also matters. I have no IPad to test with, but the smaller file size to read from flash or network might very well out weight any additional decompression cost. The only way to find out is to measure on your actual device.
There is a performance consideration but while PNG is preferred for quality, given your application, I'd suggest JPEG would be preferable.
Pure performance isn't the only factor of interest or concern; an iPad has only a finite space available to it, and filling that up with image data that most users are not going to need or want seems preferable to using more computational power for most cases.
One other thing to consider - on a gallery, you are strongly recommended to generate thumbnails which give you the best of both worlds: the smaller, more accessible image for general use and the full original image for 'best'.
If in doubt, benchmark with both and see how big the difference is in your application - and if the difference is something you can live with versus the space saving, go with JPEG.
In medical imaging, there appears to be two ways of storing huge gigapixel images:
Use lots of JPEG images (either packed into files or individually) and cook up some bizarre index format to describe what goes where. Tack on some metadata in some other format.
Use TIFF's tile and multi-image support to cleanly store the images as a single file, and provide downsampled versions for zooming speed. Then abuse various TIFF tags to store metadata in non-standard ways. Also, store tiles with overlapping boundaries that must be individually translated later.
In both cases, the reader must understand the format well enough to understand how to draw things and read the metadata.
Is there a better way to store these images? Is TIFF (or BigTIFF) still the right format for this? Does XMP solve the problem of metadata?
The main issues are:
Storing images in a way that allows for rapid random access (tiling)
Storing downsampled images for rapid zooming (pyramid)
Handling cases where tiles are overlapping or sparse (scanners often work by moving a camera over a slide in 2D and capturing only where there is something to image)
Storing important metadata, including associated images like a slide's label and thumbnail
Support for lossy storage
What kind of (hopefully non-proprietary) formats do people use to store large aerial photographs or maps? These images have similar properties.
It seems like starting with TIFF or BigTIFF and defining a useful subset of tags + XMP metadata might be the way to go. FITS is no good since it is basically for lossless data and doesn't have a very appropriate metadata mechanism.
The problem with TIFF is that it just allows too much flexibility, but a subset of TIFF should be acceptable.
The solution may very well be http://ome-xml.org/ and http://ome-xml.org/wiki/OmeTiff.
It looks like DICOM now has support:
ftp://medical.nema.org/MEDICAL/Dicom/Final/sup145_ft.pdf
You probably want FITS.
Arbitrary size
1--3 dimensional data
Extensive header
Widely used in astronomy and endorsed by NASA and the IAU
I'm a pathologist (and hobbyist programmer) so virtual slides and digital pathology are a huge interest of mine. You may be interested in the OpenSlide project. They have characterized a number of the proprietary formats from the large vendors (Aperio, BioImagene, etc). Most seem to consist of a pyramidal zoomed (scanned at different microscopic objectives, of course), large tiff files containing multiple tiled tiffs or compressed (JPEG or JPEG2000) images.
The industry standard is DICOM Sup 145; getting vendors to adopt it though has been sluggish, but inventing yet another format would probably not be helpful.
PNG might work for you. It can handle large images, metadata, and the PNG format can have some interlacing, so you can get up to (down to?) an n/8 x n/8 downsampled image pretty easily.
I'm not sure if PNG can do rapid random access. It is chunked, but that might not be enough.
You could represent sparse data with the transparency channel.
JPEG2000 might be worth a look, some interesting efforts from National libraries in this space.
I have seen many types of image extensions but have never understood the real differences between them. Are there any links out there that clearly explain their differences?
Are there standards to consider when choosing a particular type of image to use in an application? What do we use for web applications?
Yes. They are different file formats (and their file extensions).
Wikipedia entries for each of the formats will give you quite a bit of information:
JPEG (or JPG, for the file extension; Joint Photographic Experts Group)
PNG (Portable Network Graphics)
BMP (Bitmap)
GIF (Graphics Interchange Format)
TIFF (or TIF, for the file extension; Tagged Image File Format)
Image formats can be separated into three broad categories:
lossy compression,
lossless compression,
uncompressed,
Uncompressed formats take up the most amount of data, but they are exact representations of the image. Bitmap formats such as BMP generally are uncompressed, although there also are compressed BMP files as well.
Lossy compression formats are generally suited for photographs. It is not suited for illustrations, drawings and text, as compression artifacts from compressing the image will standout. Lossy compression, as its name implies, does not encode all the information of the file, so when it is recovered into an image, it will not be an exact representation of the original. However, it is able to compress images very effectively compared to lossless formats, as it discards certain information. A prime example of a lossy compression format is JPEG.
Lossless compression formats are suited for illustrations, drawings, text and other material that would not look good when compressed with lossy compression. As the name implies, lossless compression will encode all the information from the original, so when the image is decompressed, it will be an exact representation of the original. As there is no loss of information in lossless compression, it is not able to achieve as high a compression as lossy compression, in most cases. Examples of lossless image compression is PNG and GIF. (GIF only allows 8-bit images.)
TIFF and BMP are both "wrapper" formats, as the data inside can depend upon the compression technique that is used. It can contain both compressed and uncompressed images.
When to use a certain image compression format really depends on what is being compressed.
Related question: Ruthlessly compressing large images for the web
You should be aware of a few key factors...
First, there are two types of compression: Lossless and Lossy.
Lossless means that the image is made smaller, but at no detriment to the quality. Lossy means the image is made (even) smaller, but at a detriment to the quality. If you saved an image in a Lossy format over and over, the image quality would get progressively worse and worse.
There are also different colour depths (palettes): Indexed color and Direct color.
With Indexed it means that the image can only store a limited number of colours (usually 256) that are chosen by the image author, with Direct it means that you can store many thousands of colours that have not been chosen by the author.
BMP - Lossless / Indexed and Direct
This is an old format. It is Lossless (no image data is lost on save) but there's also little to no compression at all, meaning saving as BMP results in VERY large file sizes. It can have palettes of both Indexed and Direct, but that's a small consolation. The file sizes are so unnecessarily large that nobody ever really uses this format.
Good for: Nothing really. There isn't anything BMP excels at, or isn't done better by other formats.
GIF - Lossless / Indexed only
GIF uses lossless compression, meaning that you can save the image over and over and never lose any data. The file sizes are much smaller than BMP, because good compression is actually used, but it can only store an Indexed palette. This means that there can only be a maximum of 256 different colours in the file. That sounds like quite a small amount, and it is.
GIF images can also be animated and have transparency.
Good for: Logos, line drawings, and other simple images that need to be small. Only really used for websites.
JPEG - Lossy / Direct
JPEGs images were designed to make detailed photographic images as small as possible by removing information that the human eye won't notice. As a result it's a Lossy format, and saving the same file over and over will result in more data being lost over time. It has a palette of thousands of colours and so is great for photographs, but the lossy compression means it's bad for logos and line drawings: Not only will they look fuzzy, but such images will also have a larger file-size compared to GIFs!
Good for: Photographs. Also, gradients.
PNG-8 - Lossless / Indexed
PNG is a newer format, and PNG-8 (the indexed version of PNG) is really a good replacement for GIFs. Sadly, however, it has a few drawbacks: Firstly it cannot support animation like GIF can (well it can, but only Firefox seems to support it, unlike GIF animation which is supported by every browser). Secondly it has some support issues with older browsers like IE6. Thirdly, important software like Photoshop have very poor implementation of the format. (Damn you, Adobe!) PNG-8 can only store 256 colours, like GIFs.
Good for: The main thing that PNG-8 does better than GIFs is having support for Alpha Transparency.
Important Note: Photoshop does not support Alpha Transparency for PNG-8 files. (Damn you, Photoshop!) There are ways to convert Photoshop PNG-24 to PNG-8 files while retaining their transparency, though. One method is PNGQuant, another is to save your files with Fireworks.
PNG-24 - Lossless / Direct
PNG-24 is a great format that combines Lossless encoding with Direct color (thousands of colours, just like JPEG). It's very much like BMP in that regard, except that PNG actually compresses images, so it results in much smaller files. Unfortunately PNG-24 files will still be much bigger than JPEGs, GIFs and PNG-8s, so you still need to consider if you really want to use one.
Even though PNG-24s allow thousands of colours while having compression, they are not intended to replace JPEG images. A photograph saved as a PNG-24 will likely be at least 5 times larger than a equivalent JPEG image, which very little improvement in visible quality. (Of course, this may be a desirable outcome if you're not concerned about filesize, and want to get the best quality image you can.)
Just like PNG-8, PNG-24 supports alpha-transparency, too.
Generally these are either:
Lossless compression
Lossless compression algorithms reduce file size without losing image quality, though they are not compressed into as small a file as a lossy compression file. When image quality is valued above file size, lossless algorithms are typically chosen.
Lossy compression
Lossy compression algorithms take advantage of the inherent limitations of the human eye and discard invisible information. Most lossy compression algorithms allow for variable quality levels (compression) and as these levels are increased, file size is reduced. At the highest compression levels, image deterioration becomes noticeable as "compression artifacting". The images below demonstrate the noticeable artifacting of lossy compression algorithms; select the thumbnail image to view the full size version.
Each format is different as described below:
JPEG
JPEG (Joint Photographic Experts Group) files are (in most cases) a lossy format; the DOS filename extension is JPG (other OS might use JPEG). Nearly every digital camera can save images in the JPEG format, which supports 8 bits per color (red, green, blue) for a 24-bit total, producing relatively small files. When not too great, the compression does not noticeably detract from the image's quality, but JPEG files suffer generational degradation when repeatedly edited and saved. Photographic images may be better stored in a lossless non-JPEG format if they will be re-edited, or if small "artifacts" (blemishes caused by the JPEG's compression algorithm) are unacceptable. The JPEG format also is used as the image compression algorithm in many Adobe PDF files.
TIFF
The TIFF (Tagged Image File Format) is a flexible format that normally saves 8 bits or 16 bits per color (red, green, blue) for 24-bit and 48-bit totals, respectively, using either the TIFF or the TIF filenames. The TIFF's flexibility is both blessing and curse, because no single reader reads every type of TIFF file. TIFFs are lossy and lossless; some offer relatively good lossless compression for bi-level (black&white) images. Some digital cameras can save in TIFF format, using the LZW compression algorithm for lossless storage. The TIFF image format is not widely supported by web browsers. TIFF remains widely accepted as a photograph file standard in the printing business. The TIFF can handle device-specific colour spaces, such as the CMYK defined by a particular set of printing press inks.
PNG
The PNG (Portable Network Graphics) file format was created as the free, open-source successor to the GIF. The PNG file format supports truecolor (16 million colours) while the GIF supports only 256 colours. The PNG file excels when the image has large, uniformly coloured areas. The lossless PNG format is best suited for editing pictures, and the lossy formats, like JPG, are best for the final distribution of photographic images, because JPG files are smaller than PNG files. Many older browsers currently do not support the PNG file format, however, with Internet Explorer 7, all contemporary web browsers fully support the PNG format. The Adam7-interlacing allows an early preview, even when only a small percentage of the image data has been transmitted.
GIF
GIF (Graphics Interchange Format) is limited to an 8-bit palette, or 256 colors. This makes the GIF format suitable for storing graphics with relatively few colors such as simple diagrams, shapes, logos and cartoon style images. The GIF format supports animation and is still widely used to provide image animation effects. It also uses a lossless compression that is more effective when large areas have a single color, and ineffective for detailed images or dithered images.
BMP
The BMP file format (Windows bitmap) handles graphics files within the Microsoft Windows OS. Typically, BMP files are uncompressed, hence they are large; the advantage is their simplicity, wide acceptance, and use in Windows programs.
Use for Web Pages / Web Applications
The following is a brief summary for these image formats when using them with a web page / application.
PNG is great for IE6 and up (will require a small CSS patch to get transparency working well). Great for illustrations and photos.
JPG is great for photos online
GIF is good for illustrations when you do not wish to move to PNG
BMP shouldn't be used online within web pages - wastes bandwidth
Source: Image File Formats
Since others have covered the differences, I'll hit the uses.
TIFF is usually used by scanners. It makes huge files and is not really used in applications.
BMP is uncompressed and also makes huge files. It is also not really used in applications.
GIF used to be all over the web but has fallen out of favor since it only supports a limited number of colors and is patented.
JPG/JPEG is mainly used for anything that is photo quality, though not for text. The lossy compression used tends to mar sharp lines.
PNG isn't as small as JPEG but is lossless so it's good for images with sharp lines. It's in common use on the web now.
Personally, I usually use PNG everywhere I can. It's a good compromise between JPG and GIF.
JPG > Joint Photographic Experts Group
1 JPG images support 16 million colors and are best suited for photographs and complex graphics
2 JPGs do not support transparency.
PNG > Portable Network Graphics
1 It's used as an alternative to the GIF file format when the GIF technology was copyrighted and required permission to use.
2 PNGs allow for 5 to 25 percent greater compression than GIFs, and with a wider range of colors. PNGs use two-dimensional interlacing, which makes them load twice as fast as GIF images.”
3 Image that has a lot of colors or requires advanced variable transparency, PNG is the preferred file type.
GIF > Graphics Interchange Format
1 Reduces the number of colors in an image to 256.
2 GIFs also support transparency.
3 GIFs have the unique ability to display a sequence of images, similar to videos, called an animated GIF.
4 If the image has few colors and does not require any advanced alpha transparency effect, GIF is the way to go.
SVG > Scalable Vector Graphics
1 SVGs are a web standard based on XML that describe both static images and animations in two dimensions.
2 SVG allows you to create very high-quality graphics and animations that do not lose detail as their size increases/decreases.
These names refers to different ways to encode pixel image data (JPG and JPEG are the same thing, and TIFF may just enclose a jpeg with some additional metadata).
These image formats may use different compression algorithms, different color representations, different capability in carrying additional data other than the image itself, and so on.
For web applications, I'd say jpeg or gif is good enough. Jpeg is used more often due to its higher compression ratio, and gif is typically used for light weight animation where a flash (or something similar) is an over kill, or places where transparent background is desired. PNG can be used too, but I don't have much experience with that. BMP and TIFF probably are not good candidates for web applications.
What coobird and Gerald said.
Additionally, JPEG is the file format name. JPG is commonly used abbreviated file extension for this format, as you needed to have a 3-letter file extension for earlier Windows systems. Likewise with TIFF and TIF.
Web browsers at the moment only display JPEG, PNG and GIF files - so those are the ones that can be shown on web pages.
PNG supports alphachannel transparency.
TIFF can have extended options I.e.
Geo referencing for GIS applications.
I recommend only ever using JPEG for photographs, never for images like clip art, logos, text, diagrams, line art.
Favor PNG.
The named ones are all raster graphics, but beside that don't forget the more and more important vectorgraphics.
There are compressed and uncompressed types (in a more or less way), but they're all lossless. Most important are:
SVG / SVGZ
EPS
EMF / (WMF)
The file extension tells you how the image is saved. Some of those formats just save the bits as they are, some compress the image in different ways, including lossless and lossy methods. The Web can tell you, although I know some of the patient responders will outline them here.
The web favors gif, jpg, and png, mostly. JPEG is the same (or very close) to jpg.
For the specified difference and usage between the varies of image formats have a good discussion above already.
However, I want to add something for the overall process of capturing a picture and storing them.
The capturing process
Or you can say the construct process(as we can draw or make pictures with computers now). If you take a photograph with a camera, you are already using lots of sensors(CCD or CMOS) and algorithms(Bayer Pattern Filter, Sub-sampling and quantization, etc.) Also there are stuff like Pixel Format and Color Space. After you got the basic pixel information, there must be a way for storing them.
The basic image file structure
For storing the pixels info a file, we need a convention and related algorithms. For saving space, there are compression, but basically problem is encoding the pixels to bytes and decoding the bytes to pixels for display.
A typical image file may be consisted by several parts, basically two:meta data or file header and pixel data section. The meta data tells about the image itself, maybe height and width, file format, etc. And the pixel data section is the real section who deals with the real picture.
Storing and Displaying
As we said earlier, files are stored in hard disk and are in bytes/bits. So image files have no priority but also bytes stream actually. For displaying, maybe we should get something to know how monitor works. Typical PC monitors use RGB model for displaying.
Hope this helps:-)