I have a sequence of large images I would like to load into matlab and then apply some processing too. Due to the images size, reading them in takes a long time, and fills the computer memory very fast.
However, I am only interested in the middle section of the images, a region of about 100 by 100 pixels or so.
Is there a way to only read in that section of the image, therefore saving time, and memory?
Currently I am using:
ROIx = 450:550;
ROIy = 650:750;
image = double( imread( filename ) );
image = image(ROIx, ROIy);
However, imread() loads the whole image, and this takes a long time. Is there a way only to read the part I am interested in?
(One procedure would be to go through and crop each image into a smaller one and resave it. But I would prefer not to crop the images).
Thanks,
labjunky
Matlab 2012a indicates that you can read parts of images from JPEG2000 and TIFF images. Look at the documentation for imread, inspect the option 'PixelRegion' for reading TIFFs.
You will need to use something like fopen to open the file and read the contents in parts manually. You will need to take care of lot of encoding/decoding of course. Or the other way round a little more would be to increase your system's swap space. If you are getting out of memory errors.
Related
When I take pictures with my camera the file sizes seem exceedingly large. In this example the original is 5186kb. I wrote a java program to just read the image and then write it again, removing any information except the pixel values.
The first time it is rewritten, the file size goes down to 1005kb, over a 500% reduction! To make sure I wasn't losing data to compression I iterated the program 100 times on the resulting images and the file size stayed at exactly 1005kb with no loss in image quality.
My question is, what is the camera storing in the other 4181kb? Some sort of metadata I know, but it seems like a lot. I would like to know what I am losing by resizing my images like this.
Assuming the file format you are using is .jpg, the original file was saved in a higher value of jpg compression say 95%, while when you resave the file, you probably were using say 85% jpg compression value.
The size doesn't change in consequent saves as the compression value stays the same
I am searching for a library (in any language) that is capable of reading a region of an image file (any format) without having to initially read that entire image file.
I have come across a few options such as vips, which does indeed not keep the entire image in memory, but still seems to need to read it entirely to begin with.
I realize this may not be available for compressed formats such as jpegs, but in theory it sounds like bmps or tiffs should allow for this type of reading.
libvips will read just the part you need, when it can. For example, if you crop 100x100 pixels from the top-left of a large PNG, it's fast:
$ time vips crop wtc.png x.jpg 0 0 100 100
real 0m0.063s
user 0m0.041s
sys 0m0.023s
(the four numbers are left, top, width, height of the area to be cropped from wtc.png and written to x.jpg)
But a 100x100 pixel region from near the bottom is rather slow, since it has to read and decompress the pixels before the pixels you want to get to the right point in the file:
$ time vips crop wtc.png x.jpg 0 9000 100 100
real 0m3.063s
user 0m2.884s
sys 0m0.181s
JPG and strip TIFF work in the same way, though it's less obvious since they are much faster formats.
Some formats support true random-access read. For example, tiled TIFF is fast everywhere, since libvips can use libtiff to read only the tiles it needs:
$ vips copy wtc.png wtc.tif[tile]
$ time vips crop wtc.tif x.jpg 0 0 100 100
real 0m0.033s
user 0m0.013s
sys 0m0.021s
$ time vips crop wtc.tif x.jpg 0 9000 100 100
real 0m0.037s
user 0m0.021s
sys 0m0.017s
OpenSlide, vips, tiled OpenEXR, FITS, binary PPM/PGM/PBM, HDR, RAW, Analyze, Matlab and probably some others all support true random access like this.
If you're interested in more detail, there's a chapter in the API docs describing how libvips opens a file:
http://libvips.github.io/libvips/API/current/How-it-opens-files.md.html
Here's crop plus save in Python using pyvips:
import pyvips
image = pyvips.Image.new_from_file(input_filename, access='sequential')
tile = image.crop(left, top, width, height)
tile.write_to_file(output_filename)
The access= is a flag that hints to libvips that it's OK to stream this image, in case the underlying file format does not support random access. You don't need this for formats that do support random access, like tiled TIFF.
You don't need to write to a file. For example, this will make a buffer object containing the file encoded as a JPG:
buffer = tile.write_to_buffer('.jpg', Q=85)
Or this will write directly to stdout:
target = pyvips.Target.new_from_descriptor(0)
tile.write_to_target('.jpg', Q=85)
The Q=85 is an optional argument to set the JPG Q factor. You can set any of the file save options.
ITK can do it with some formats. There is a method CanStreamRead which returns true for formats which support streaming, such as MetaImageIO. An example can be found here. You can ask more detailed questions on ITK's forum.
If have control over the file format, I would suggest you use tiled TIFF files. These are typically used in digital pathology whole slide images, with average sizes of 100kx30k pixels or so.
LibTiff makes it easy to read the tiles corresponding to a selected ROI. Tiles can be compressed without making it less efficient to read a small region (no need to decode whole scan lines).
The BMP format (uncompressed) is simple enough that you can write the function yourself.
TIFF is a little less easy, as there are so many subformats. But the TIFF library (TIFFlib) supports a "tile-oriented" I/O mode. http://www.libtiff.org/libtiff.html#Tiles
I don't know of such a library solution.
Low level, file-read access is format specific and in particular, file mapping is OS specific.
If you have access to the raw bytes then assuming you know the width, height, depth and number of channels etc. then calculating file offsets is trivial so just roll your own.
If you're transferring the extracted data over a network you might consider compressing the extracted ROI in-memory if it's relatively big before sending it over the network.
I know that this question has been asked before. But, I could not find a clear answer for it. I have data for a very high resolution colorful image with the size of 50,000 by 60,000 with the data type unit8. I cannot save the entire image by using imwrite. I have gotten the error that says:
"Images must contain fewer than 2^32 - 1 bytes of data"
Is there a way to save the entire image in MATLAB?
right now, I have to break the data into smaller pieces (sub-images) and then use imwrite to write each piece to a png file. The output format of the file is not important.
Your image occupies 8*50000*60000*3 = 7.2000e+10 bytes of data that is 16.7638 times more than MATLAB image size limit. Why no to split it in 20 pieces, save them and then merge them manually? If you split your image into 6 8x50000x3000x3 pieces, they would all fit into 2^32 limit.
I am sure OP has enough aptitude to do this, but I'll explain the procedure anyway. Convert your image into 50000x60000x3 array and do the following:
x = 0:3000:60000;
for i = 1:length(x)-1
imwrite(A(:,x(i)+1:x(i+1),:),strcat('image',num2str(i),'.png'),'png');
end
This would create 20 images for you with names 'image1.png', 'image2.png' and so on. Then, you can merge these images manually using this first google search result. Perhaps, there is some fancier way to do this, but I think this is the easiest one.
Another question has an answer which worked for me: if your image is stored as a double matrix, convert to uint8 with im2uint8(img), then save.
I'm porting some CF 2.0 VB.Net apps to a newer version of a handset that has twice the screen resolution. So I have to double the dimensions of everything otherwise it all gets squished up into the top LH corner of the screen.
One screen had a bitmap which was 250K in size, and after I doubled the dimensions naturally it blew out to one MB. This isn't real good on a handheld, so I fired up irfanview and converted it to a .GIF. The .GIF was only 60KB in size, with no discernible change in the quality of the image.
To me, it seems a no-brainer : Convert all Bitmaps to Gif (or JPG) and get the same results for a fraction of the disk space (and probably quicker form loading times).
But does anyone know of a situation where you would use a bitmap in preference to a GIF/JPEG? I cannot find any.
I really can't think of any realistic example where you would prefer an bitmap to a GIF. Since GIF is a lossless format you loose no information when storing images. So after reading the file in your app you will have the same image data as if you have read a bitmap. And like you said: The file will be smaller and thus will probably will be read faster from disk.
JPEG is different because it's a lossy format, meaning you will lose information when storing images in it. You will need to decide if the loss of information is meaningful in your app.
Bitmaps would be preferable if and only if reading files from disk where faster than decompressing the file in memory.
And to be precise you would prefer bitmaps when storing images in main memory, so you can work easily on the data in your code. Which is actually what you most likely already have when you have loaded a file using an image library.
To cut a long story shorts, a BMP is stored as a series of pixels along with their colour. This is useful if you want to do such things as pattern recognition, movement detection and such like.
Bitmaps are typically used for their convenience - you can knock one up in paint without having specialist graphics software.
One of the nice features of JPEG 2000 is that it's easy to downres an image -- just feed out fewer bytes.
Is there a similar way to scale the image, such that I could send a thumbnail using bytes in the full-size file, without resampling?
No, to resize you will still have to read the original size, resample to the new size, and then recompress.
You can write your file such that it has layers with powers-of-2 sizes, with a very small relative incremental increase in size, and extract those layers.