The following is from TIFF 6.0 Specification Section 9: PackBits Compression
That is the essence of the algorithm. Here are some additional rules:
Pack each row separately. Do not compress across row boundaries.
The number of uncompressed bytes per row is defined to be (ImageWidth + 7)
/ 8. If the uncompressed bitmap is required to have an even number of bytes per
row, decompress into word-aligned buffers.
If a run is larger than 128 bytes, encode the remainder of the run as one or more
additional replicate runs
The first and the third items are easy to understand but I am confused about the second one specifically this: The number of uncompressed bytes per row is defined to be (ImageWidth + 7) / 8. Isn't that only true for 1 bit bi-level image. But to my knowledge, packbits is a byte oriented compression algorithm, it could be used for any type of tiff.
Could someone who knows about tiff and packbits give me some hints?
The TIFF document from this site: http://www.fileformat.info/format/tiff/corion-packbits.htm
has the following at the top:
Abstract
This document describes a simple compression scheme for bilevel
scanned and paint type files.
Motivation
The TIFF specification defines a number of compression schemes.
Compression type 1 is really no compression, other than basic
pixel packing. Compression type 2, based on CCITT 1D
compression, is powerful, but not trivial to implement.
Compression type 5 is typically very effective for most bilevel
images, as well as many deeper images such as palette color and
grayscale images, but is also not trivial to implement. PackBits
is a simple but often effective alternative
So it is clear the additional rules are with respect to bilevel images. For some reason, the above abstract and description are missing from the pdf version of TIFF6.0.
Related
According to this site
BMP (Bitmap) is a uncompressed raster graphics image format
1) So does it mean that, BMP doesn't follow compression while image storing at all ?
2) If it does follow compression then it should be called lossy ? But it's lossless why so ?
Also when it is said,
Lossless means that the image is made smaller, but at no detriment to
the quality
3) If the image is made smaller then how can it remain the same.Making it smaller means that it has to follow some compression right?
Edit:
4) JPEG is also a bitmap format then why is it not lossless ?
First, BMP does not allow image compression at all, pixel values are written as is, no compression or size reduction transformation is used. It's uncompressed, so it's not lossy, it's lossless. It is actually possible to compress images (and also audio) in a lossless manner, that is, mathematical operations are performed on the data that remove redundant data, thus decreasing the overall size, since these operations are invertible they are also able to recover the original data (image, audio, etc...). Technically, a bitmap is a 2 dimensional array of pixel values, but a bitmap is widely known as the uncompressed .bmp image format. Compression has two variants, lossy compression, where you drop some portion of your data that can't be recovered, hence lossy; and lossless, where you drop portions of your data that can be recovered by the inverse process. A full treatment on the subject inevitably has to deal with information theory and Shannon's result on coding theory. A simple place to start is with run-length enconding and Lempel-Ziv compression algorithm for lossless compression, and JPEG compression using wavelets for lossy compression.
I'm looking for a way to compress dicom files and send them to a remote server (nodejs in my case)
I tried bz2 compression and it seems to work very well on large dicom files (tested it with 10 Mb file which gave me a 5Mb compressed file).
When it comes to small size (like a 250Kb file ), I get a size reduced by a very few kb (5 to 10 kb in most cases) which it won't be worth it
can someone please explain me why bz2 is working very well with large dicom files and is there a better way to compress dicom files that I can use in order to send them via internet.
thanks in advance.
If you want to compress a DICOM dataset with images, recommendation is to use one of the compression types supported by DICOM Standard. This includes lossy and lossless JPEG, JPEG 2000, JPEG-LS, and RLE to name a few. Standard also supports encoding extended grayscale (12-16 bit grayscale) using the standard based compression techniques.
The Transfer Syntax element (0002, 0010) will indicate whether image in the DICOM dataset is already compressed or not. As for example, recompressing an already compressed image is going to appear having less compression ratio compared to the original. So the best way to measure is to compare with the original uncompressed image. If your original image is already compressed, you can calculate the uncompressed image size using (Rows x Columns x Bits Allocated / 8 x Sample Per Pixel x Number of Frames). Also, the compression ratio will vary based on the image type (color vs grayscale) and compression technique used. Typically, you will get much better compression when dealing with true color image vs a grayscale image such as an X-RAY.
As for using HTTP for uploading the file, you can also use DICOM standard defined service such as DICOMWeb (STOW-RS) REST service.
I work for LEAD Technologies and if you want to test various compressions on your DICOM files, we have a demo exe (Transfer Syntax) that ships with our free 60 day evaluation SDK that you can use for testing. Also, there is a demo for testing the DICOMWeb REST services. You can download the evaluation copy from our web site.
There is not one solution that perfectly fits all...
BZ2 is based on the principle that "colours" (or gray values, but I will use "colours" in this explanation) which frequently occur in the image are encoded with less bits than colours which are rare. Thus, as a rule of thumb: the bigger the image the better the compression ratio.
JPEG is a different approach which decomposes the image into tiles and optimizes the encoding for each tile. Thus the compression ratio is less dependent on the image size than it is for BZ2. JPEG comes in different flavors (lossy, lossless, JPEG 2000 which can create different serializations of the compressed data for different purposes, e.g. progressive refinement).
Less popular compression algorithms which are valid in DICOM but not widely supported by DICOM products are:
RLE (Run Length Encoding) - the pixel data is described by pairs of colour and number of pixels, so it compresses very well when you have large homogenous areas in the image. In all other cases it is rather increasing the size of the "compressed" image
JPEG-LS - I do not know how it is working internally, but it provides a lossless algorithm and a lossy algorithm in which you can control the loss of information (maximum difference of a pixel value after compression to the original pixel value). It is said to achieve better ratios than traditional JPEG, but as it is not widely supported, I have not used it in practice yet.
If you do not want to select the compression algorithm depending on the image type, JPEG-Lossless is probably a good compromise for you. In typical medical images, it achieves an average compression ratio of roughly 1:, a bit more with JPEG-2000.
Ok, so I tried to use the imagemagick command:
"convert picA.png -interlace line picB.png"
to make an interlace version of my .png images. Most of the time, I got the resulting image is larger than the original one, which is kinda normal. However, on certain image, the resulting image size is smaller.
So I just wonder why does that happen? I really don't want my new image to lose any quality because of the command.
Also, is there any compatibility problem with interlaced .png image?
EDIT: I guess my problem is that the original image was not compressed as best as it could be.
The following only applies to the cases where the pixel size is >= 8 bits. I didn't investigate for other cases but I expect similar outcomes.
A content-identical interlaced PNG image file will almost always be greater because of the additional data for filter type descriptions required to handle the passes scanlines. This is what I explained in details in this web page based on the PNG RFC RFC2083.
In short, this is because the sum of the below number of bytes for interlaced filter types description per interlacing pass is almost always greater than the image height (which is the number of filter types for non-interlaced images):
nb_pass1_lines = CEIL(height/8)
nb_pass2_lines = (width>4?CEIL(height/8):0)
nb_pass3_lines = CEIL((height-4)/8)
nb_pass4_lines = (width>2?CEIL(height/4):0)
nb_pass5_lines = CEIL((height-2)/4)
nb_pass6_lines = (width>1?CEIL(height/2):0)
nb_pass7_lines = FLOOR(height/2)
Though, theoretically, it can be that the data entropy/complexity accidentally gets lowered enough by the Adam7 interlacing so that, with the help of filtering, the usually additional space needed for filter types with interlacing may be compensated through the deflate compression used for the PNG format. This would be a particular case to be proven as the entropy/complexity is more likely to increase with interlacing because the image data is made less consistent through the interlacing deconstruction.
I used the word "accidentally" because reducing the data entropy/complexity is not the purpose of the Adam7 interlacing. Its purpose is to allow the progressive loading and display of the image through a passes mechanism. While, reducing the entropy/complexity is the purpose of the filtering for PNG.
I used the word "usually" because, as shown in the explanation web page, for example, a 1 pixel image will be described through the same length of uncompressed data whether interlaced or not. So, in this case, no additional space should be needed.
When it comes to the PNG file size, a lower size for interlaced can be due to:
Different non-pixel encoding related content embedded in the file such as palette (in the case of color type =! 3) and non-critical chunks such as chromaticities, gamma, number of significant bits, default background color, histogram, transparency, physical pixel dimensions, time, text, compressed text. Note that some of those non-pixel encoding related content can lead to different display of the image depending on the software used and the situation.
Different pixel encoding related content (which can change the image quality) such as bit depth, color type (and thus the use of palette or not with color type = 3), image size,... .
Different compression related content such as better filtering choices, accidental lower data entropy/complexity due to interlacing as explained above (theoretical particular case), higher compression level (as you mentioned)
If I had to check whether 2 PNG image files are equivalent pixel wise, I would use the following command in a bash prompt:
diff <( convert non-interlaced.png rgba:- ) <( convert interlaced.png rgba:- )
It should return no difference.
For the compatibility question, if the PNG encoder and PNG decoder implement the mandatory aspects of the PNG RFC, I see no reason for the interlacing to lead to a compatibility issue.
Edit 2018 11 13:
Some experiments based on auto evolved distributed genetic algorithms with niche mechanism (hosted on https://en.oga.jod.li ) are explained here:
https://jod.li/2018/11/13/can-an-interlaced-png-image-be-smaller-than-the-equivalent-non-interlaced-image/
Those experiments show that it is possible for equivalent PNG images to have a smaller size interlaced than non-interlaced. The best images for this are tall, they have a one pixel width and have pixel content that appear random. Though, the shape is not the only important aspect for the interlaced image to be smaller than the non-interlaced image as random cases with the same shape lead to different size differences.
So, yes, some PNG images can be identical pixel wise and for non-pixel related content but have a smaller size interlaced than non-interlaced.
So I just wonder why does that happen?
From section Interlacing and pass extraction of the PNG spec.
Scanlines that do not completely fill an integral number of bytes are padded as defined in 7.2: Scanlines.
NOTE If the reference image contains fewer than five columns or fewer than five rows, some passes will be empty.
I would assume the behavior your experiencing is the result of the Adam7 method requiring additional padding.
I decided I'd attempt an image compression (From pixel RGBs) idea I had for a bit of extra credit in class. I've finished, but I find the levels of compression I get varies HUGELY from image to image. With this image, I'm getting a file size 1.25x the size of the coresponding PNG. With this image however, I'm getting a file size 22.5x the size of the PNG.
My compression works by first assigning each color in the image with an int(starting from 0), then using that int rather than the actual color in the file. The file is formatted as:
0*qj8c1*50i2p2*pg93*9zlds4*2rk5*4ok4r6*8mv1w7*2r25l8*3m89o9*9yp7c10*111*2clz112*g1j13*2w34z14*auq15*3zhg616*mmhc17*5lgsi18*25lw919*7ip84+0!0!0!0!0!0!0!0!0!0!0!0!0!1!1!1!#2!2!2!2!2!2!2!2!2!2!2!2!2!3!3!3!#4!4!4!4!4!4!4!4!4!4!4!4!4!5!5!5!#6!6!6!6!6!6!6!6!6!6!6!6!6!0!0!0!#3!3!3!3!3!3!3!3!3!3!3!3!3!2!2!2!#1!1!1!1!1!1!1!1!1!1!1!1!1!4!4!4!#7!7!7!7!7!7!7!7!7!7!7!7!7!6!6!6!#5!5!5!5!5!5!5!5!5!5!5!5!5!8!8!8!#9!9!9!9!9!9!9!9!9!9!9!9!9!10!10!10!#8!8!8!8!8!8!8!8!8!8!8!8!8!11!11!11!#12!12!12!12!12!12!12!12!12!12!12!12!12!7!7!7!#13!13!13!13!13!13!13!13!13!13!13!13!13!14!14!14!#15!15!15!15!15!15!15!15!15!15!15!15!15!16!16!16!#17!17!17!17!17!17!17!17!17!17!17!17!17!13!13!13!#18!18!18!18!18!18!18!18!18!18!18!18!18!15!15!15!#10!10!10!10!10!10!10!10!10!10!10!10!10!19!19!19!#
Where the first bit (With the *s and base36 numbers) is the dictionary defining the colors, and the second part seperated by !s is the actual image.
Why am I seeing the level of compression vary so hugely image from image? Is there a flaw in my compression algorithm?
Edit: The fact that the actual level of compression is poor compared to JPEG or PNG isn't an issue, I wasn't expecting to rival any major formats.
Thanks
I've done a bit of fiddling around with PNGs over the last couple days and I am upset with my findings. I'm concluding that the majority of my results deal with compression. So this weekend I'm going to dive into advanced compression articles. I wanted to share my findings so far. To see if anyone has any advice on achieving my goal and to maybe point me in the right direction.
I am currently working on a project where I need to obtain the smallest possible file size within a window of less than 15 seconds.
The majority of the images I am working with are PNG-8bpp with a full 256 color palette. Most of these images I could represent accurately with 5bpp (32 colors).
PNG indexed however only supports 1,2,4, and 8bpp. So my idea was to strip the PNG format to the minimal information I needed and write an encoder/decoder to support IDAT sections with 3,5,6, or 7bpp.
Test 1:
Original File: 61.5KB, 750 * 500, 8pp Palette, 256 colors, No tRNS
After Optimizations (Reductions to 4bpp, Strip Anx Chunks, & PNGOUT): 49.2KB 4bpp, 16 Colors
Human Interpretation: I can see 6 distinguishable colors.
Since I only need six colors to represent the image I decide to encode the IDAT using 3bpp to give me a max palette of 8 colors. First I uncompressed the IDAT which results in a new file size of 368KB. After applying a 3bpp to IDAT my new uncompressed file size is 274KB. I was off to what seemed to be a good start... Next I applied deflate to my new IDAT section. Result... 59KB.
10KB larger than using 4bpp.
Test 2:
Original File: 102KB, 1000 * 750, 8bpp, 256 Colors, tRNS 1 fully transparent color
After Optimization: 79KB, 8bpp, 193 colors, tRNS 1 full transparent color
Human Interpretation: I need about 24 colors to represent this picture.
24 colors can be represented in 5bpp at 32 colors. Using the same technique above I was able to achieve much better results over uncompressed but again I lost at compression. Final size compressed... 84KB. Then I tried at 6,7bpp... same result poorer compression that at 8bpp.
Just to be sure I saved all the uncompressed images and tried several other compression algorithms... LZMA, BZIP2, PAQ8... same result smaller compression size at 8bpp than at 5,6, or 7bpp AND smaller size at 4bpp than at 3bpp.
Why is this occuring? Can I tweak/modify a compression algorithm to target a PNG like format that uses a 5,6, or 7bpp format that beast 8bpp compression? Is it worth the time... and yes saving another 10KB would be worth it.
What you're seeing is that by using odd pixel sizes, your effective compression decreases because of the way PNG compression works. The advantage of PNG compression over just using straight FLATE/ZIP compression is the filtering. PNG compression tries to exploit horizontal and vertical symmetry with a small assortment of pre-processing filters. These filters work on byte boundaries and are effective with pixel sizes of 4/8/16/24/32/48/64 bits. When you move to an odd size pixel (3/5/6/7 bits) you are defeating the filtering because identically colored pixels won't "cancel each other out" horizontally when filtered on 8-bit boundaries.
Even if the filtering weren't an issue, the way that FLATE compression works, reducing the pixel size from 8 to 7 or 6 bits won't have much effect either because it also assumes a symbol size of 8-bits.
In conclusion...the only benefit you can achieve by using odd sizes of pixels is that the uncompressed data will be smaller. By breaking the pixels' byte boundary symmetry, you defeat much of the benefit of PNG compression.
GIF compression supports all pixel sizes from 1 to 8 bits. It defines the symbol size as the pixel size and doesn't use any pre-filtering. An 8-bit GIF image, if compressed as 7-bit pixels, wouldn't suffer less compression, but also wouldn't benefit because the compression depends more on the repetition of the pixels than the symbol size.
DEFLATE compression used by PNG has two main techniques:
finds repeating byte sequences and encodes them as backreferences
encodes bytes using Huffman coding
By changing pixel length from 8-bit you're out of sync with byte boundaries and DEFLATE won't be able to encode repeating pixel runs as repeated bytes.
And thanks to Huffman coding it doesn't matter that 8-bit pixels have unused bits, because the coding will encode bytes with variable-width codes assigning shortest ones to most frequently occurring values.