Matlab: crop image with a sliding window? - image

does anybody know how to crop an image with a sliding window in Matlab?
e.g. I have an image of 1000x500 pixels, I would like to crop from this image blocks of 50x50 pixels... Of course I have to handle uneven divisions, but it is not necessary to have block of the same size.

Some details that have helped me in the past related to (i) ways to divide an image while block processing and (ii) "uneven division", as mentioned by OP.
(i) Ways to divide/process an image:
1. Process non-overlapping blocks:
Using default parameter {'BorderSize',[0 0]}, this can be handled with blockproc as below.
Example for (i)-1: Note the blocked nature of the output. Here each non-overlapping block of size 32 x 32 is used to calculate the std2() and the output std2 value is used to fill that particular block. The input and output are of size 32 x 32.
fun = #(block_struct) std2(block_struct.data) * ones(size(block_struct.data));
I2 = blockproc('moon.tif',[32 32],fun);
figure; subplot(1, 2, 1);
imshow('moon.tif'); title('input');
subplot(1,2, 2)
imshow(I2,[]); title('output');
Input and Output Image:
(i)-2: Process overlapping blocks:
Using parameter {'BorderSize',[V H]}: V rows are added above and below the block and H columns are added to the left and right of the block. The block that is processed has (N + 2*V) rows and (M + 2*H) columns. Using default parameter {'TrimBorder',true}, the border of the output is trimmed to the original input block size of N rows and M columns.
Example for (i)-2: Below code using blockproc uses {'BorderSize',[15 15]} with [N M] = [1 1]. This is similar to filtering each pixel of the image with a custom kernel. So the input to the processing unit is a block of size (1 + 2*15) rows and (1 + 2*15) columns. And since {'TrimBorder',true} by default, the std2 of the 31 rows by 31 columns block is provided as output for each pixel. The output is of size 1 by 1 after trimming border. Consequently, note that this example output is 'non-blocked' in contrast to the previous example. This code takes much longer time to process all the pixels.
fun = #(block_struct) std2(block_struct.data) * ones(size(block_struct.data));
I2 = blockproc('moon.tif',[1 1],fun,'BorderSize',[15 15]);
figure; subplot(1, 2, 1);
imshow('moon.tif'); title('input');
subplot(1,2, 2)
imshow(I2,[]); title('output');
Input and Output Image:
(ii) "Uneven division":
1. Zero/replicate/symmetric padding:
Zero padding so that an integer multiple of the blocks (N rows by M cols sized) can cover the [image + bounding zeros] in the uneven dimension. This can be achieved by using the default parameter {'PadMethod', 0} along with {'PadPartialBlocks' , true} ( which is false by default ). If a bounding region of zeros causes a high discontinuity in values computed from the bounding blocks, {'PadMethod', 'replicate'} or {'PadMethod', 'symmetric'} can be used.
2. Assume an "Active Region" within the image for block processing
For the case of processing each pixel, as in case (i)-2, we could assuming a bounding region of floor(block_size/2) pixels on all sides along the periphery of the image that is used as "Dummy" region. The Active region for block processing is contained within the Dummy region.
Something similar is used in imaging sensors where Dummy Pixels located at the periphery of an imaging array of Active Pixels allow for an operation like the color interpolation of all active area pixels. As color interpolation usually needs a 5x5 pixel mask to interpolate the color values of a pixel a bounding Dummy periphery of 2 pixels can be used.
Assuming MATLAB indexing, the region ( floor(block_size/2) + 1 ) to ( Input_Image_Rows - floor(block_size)/2) ) Rows by ( floor(block_size/2) + 1 ) to ( Input_ImageCols - floor(block_size)/2) ) Columns is considered as Active region (assuming a square block of side, block_size) which undergoes block processing for each pixel as in (i)-2.
Assuming a square block size of 5 by 5, this is shown by the following:
block_size = 5;
buffer_size = floor(block_size/2);
for i = (buffer_size+1):(image_rows-buffer_size)
for j = (buffer_size+1):(image_cols-buffer_size)
... % block processing for each pixel Image(i,j)
end
end
Matlab ver: R2013a

I would first look into the function blockproc to see if it can do what you want.
If you're sure you want to manually crop the image into blocks, you can use this script. It both writes the cropped images to .png files and saves the cropped images in the pages of a 3D array. You can modify it as you need.
This script assumes the image in evenly divisible by the block size. If it isn't, you'll need to pad it with zeros.
[rowstmp,colstmp]= size(myImage);
block_height = 50;
block_width = 50;
blocks_per_row = rows/block_height;
blocks_per_col = cols/block_width;
number_of_blocks = blocks_per_row*blocks_per_col;
%// pad image with zeros if needed
if ~(mod(rowstmp-1,block_height)==0)
rows = ceil(rowstmp/block_height)*block_height;
end
if ~(mod(colstmp-1,block_width)==0)
cols = ceil(colstmp/block_width)*block_width;
end
Im = uint8(zeros(rows,cols));
Im(1:rowstmp,1:colstmp) = myImage;
%// make sure these image have type uint8 so they save properly
cropped_image = uint8(zeros(rows,cols));
img_stack = uint8(zeros(rows,cols,number_of_blocks));
%// loop over the image blocks
for i = 1:blocks_per_row
for j = 1:blocks_per_col
%// get the cropped image from the original image
idxI = 1+(i-1)*block_height:i*block_height;
idxJ = 1+(j-1)*block_width :j*block_width;
cropped_image(idxI,idxJ) = Im(idxI,idxJ);
%//imshow(cropped_image)
%// write the cropped image to the current folder
filename = sprintf('block_row%d_col%d.png',i,j);
imwrite(cropped_image,filename);
cropped_image(idxI,idxJ) = 0;
%// keep all the blocks in a 3D array if we want to use them later
img_stack(:,:,(i-1)*blocks_per_col+j);
end
end

Related

Inpaint an image matlab [duplicate]

This question already has an answer here:
Efficient inpaint with neighbouring pixels
(1 answer)
Closed 6 years ago.
I am trying to replace all pixels with certain value in an image with the average values of the neighbors. Can interp2 be useful here? I tried this -
I = imread('test_image.JPG');
[r c] = size(I);
class_of_image = class(I);
[xi,yi] = meshgrid(1:0.5:c,1:0.5:r);
I1 = cast(interp2(double(image),xi,yi,'linear'),class_of_image);
[x_z,y_z] = find(I1==0);
I1(x_z,y_z) = I1(x_z-1,y_z)+I1(x_z+1,y_z)+I1(x_z,y_z-1)+I1(x_z,y_z+1);
This fails spectacularly with an error message - Index exceeds matrix dimensions.
I realize that the error is in trying to access I1 indices beyond r and c. Is there a generic way to incorporate this in the code?
Please help!
If you are trying to replace pixels in an image that are at a certain value to be the average of its 4 neighbours, then you don't have to use interp2. It looks like you are doubling the size of the image and then sampling from that image when you're done.
If you want to do what you're asking, you need to use column-major indices to facilitate the vectorized access of pixels. Specifically, you need to use sub2ind to help determine the locations you need to access in your matrix.
However, you will need to account for pixels that go out of bounds. There are many ways to accommodate this, but what I will implement is known as zero-padding where the border pixels are simply set to 0. I would create a zero-padded image where the top and bottom rows as well as the left and right values are all some sentinel value (like -1), use find on this image to find the coordinates then do the inpainting. Make sure you set the border pixels back to 0 before doing this so that you don't use -1 as part of the inpainting. You would then crop the border pixels of this new image when you're done to obtain the final output image.
Therefore, if you want to perform your "inpainting" try this instead:
% Read in image
I = imread('test_image.JPG');
% Create padded image with border pixels set to -1
Ipad = -ones(size(I) + 2);
% Place image in the middle
Ipad(2:end-1,2:end-1) = I;
% Find zero pixels
[r,c] = find(I == 0);
% Now set border pixels to 0
Ipad(Ipad == -1) = 0;
% Find column major indices for those elements that are 0
% as well as their 4 neighbours
ind = sub2ind(size(I), r, c);
ind_up = sub2ind(size(I), r-1, c);
ind_down = sub2ind(size(I), r+1, c);
ind_left = sub2ind(size(I), r, c-1);
ind_right = sub2ind(size(I), r, c+1);
% Perform the inpainting by averaging
Ipad(ind) = (Ipad(ind_up) + Ipad(ind_down) + Ipad(ind_left) + Ipad(ind_right))/4;
% Store the output in I1 after removing border pixels
I1 = Ipad(2:end-1,2:end-1);
However, a possibly shorter way to do this even though you would operate on the entire image would be to perform 2D convolution using a 3 x 3 kernel whose elements are 1 in the cardinal directions and ensuring that you divide by 4 to find the average value per location. After, you would simply copy over those values in the output that are 0 in the original image. You can use conv2 to do that and make sure you specify the 'same' flag to ensure that the output size is the same as the input size. The behaviour of conv2 when you approach the border elements is to zero-pad, which is what I did already in the first implementation:
% Read in image
I = imread('test_image.JPG');
% Specify kernel
kernel = [0 1 0; 1 0 1; 0 1 0] / 4;
% Perform convolution - make sure you cast image to double
% as convolution in 2D only works for floating-point types
out = conv2(double(I), kernel, 'same');
% Copy over those values from the output that match the value
% to be inpainted for the input. Also cast back to original
% data type.
I1 = I;
I1(I == 0) = cast(out(I == 0), class(I));

How can I crop several and random batches from an image?

I have an image of size (224 x 224) and I want to extract a number of random patches from the original image using Matlab (let say 5 patches). One of these patch should be at the centre of the original image. The patch size is (128 x 128).
I have tried this to crop just the centre patch:
II = imread('img.png')
[p3, p4] = size(II);
q1 = 50; // size of the crop box
i3_start = floor((p3-q1)/2); % or round instead of floor; using neither gives warning
i3_stop = i3_start + q1;
i4_start = floor((p4-q1)/2);
i4_stop = i4_start + q1;
II = II(i3_start:i3_stop, i4_start:i4_stop, :);
figure ,imshow(II);
I've tried to accomplish this in the following way:
A=imread('Lena.bmp');%sample image
rnd_x = randperm(size(A,1)-128,5);%choose 5 tandom unique points on x-axis
rnd_y = randperm(size(A,2)-128,5);%choose 5 tandom unique points on y-axis
for ii = 1:5
piece{ii} = A((rnd_x(ii):(rnd_x(ii)+127)),(rnd_y(ii):(rnd_y(ii)+127)),1:3);%Convert chosen numbers to image pieces
figure(ii)
imshow(piece{ii});
end
This takes image like this:
This gives 5 pics like this:
Here our image size is 512x512. So, if we want to cut the 128x128 piece from it, we need to seek from 385x385 grid (512-127). We find 5 random points on the grid expressed in rnd_x and rnd_y. Finally, we take the found points as the upper-left corners of the pieces and construct 128x128 images from them. The 5 pieces are recorded in piece cell array.
EDIT: forgot to add how to extract the center patch. The following code performs the task:
A=imread('Lena.bmp');%sample image
if mod(size(A,1),2)
A = A(1:(end-1),:,:);
end
if mod(size(A,2),2)
A = A(:,1:(end-1),:);
end
while size(A,1) > 128
A = A(2:(end-1),:,:);
end
while size(A,2) > 128
A = A(:,2:(end-1),:);
end
imshow(A)
The code removes one pixel from each side until we get the 128-pixel image.
Careful! In your code, if you load a color image (3 channels) and call size with only two outputs, you will have an incorrect value for p4.
Use three outputs when loading images to avoid this problem:
[nrows ncols nchannels] = size(II);
Your code correctly extracts a (q1 x q1) from the center of the image.
If you want a random patch just generate a random integer for the top-left column of the patch with the correct range to ensure that it doesn't fall outside the image. You can generate random integers using the function randi.
i3_start = randi(floor((p3-q1));
i4_start = randi(floor((p4-q1));
The rest of the code is the same. If you want several patches you can generate several values when calling the randi function with a second and third parameter for the desired number of rows and columns. And then process each patch inside a for loop.
BTW: In the third line you have an invalid Matlab comment (use % for comments). Also you should name your variables with more intuitive names.
Eg: [nrows ncols nchannels] = size(II);

Resize an image with bilinear interpolation without imresize

I've found some methods to enlarge an image but there is no solution to shrink an image. I'm currently using the nearest neighbor method. How could I do this with bilinear interpolation without using the imresize function in MATLAB?
In your comments, you mentioned you wanted to resize an image using bilinear interpolation. Bear in mind that the bilinear interpolation algorithm is size independent. You can very well use the same algorithm for enlarging an image as well as shrinking an image. The right scale factors to sample the pixel locations are dependent on the output dimensions you specify. This doesn't change the core algorithm by the way.
Before I start with any code, I'm going to refer you to Richard Alan Peters' II digital image processing slides on interpolation, specifically slide #59. It has a great illustration as well as pseudocode on how to do bilinear interpolation that is MATLAB friendly. To be self-contained, I'm going to include his slide here so we can follow along and code it:
Please be advised that this only resamples the image. If you actually want to match MATLAB's output, you need to disable anti-aliasing.
MATLAB by default will perform anti-aliasing on the images to ensure the output looks visually pleasing. If you'd like to compare apples with apples, make sure you disable anti-aliasing when comparing between this implementation and MATLAB's imresize function.
Let's write a function that will do this for us. This function will take in an image (that is read in through imread) which can be either colour or grayscale, as well as an array of two elements - The image you want to resize and the output dimensions in a two-element array of the final resized image you want. The first element of this array will be the rows and the second element of this array will be the columns. We will simply go through this algorithm and calculate the output pixel colours / grayscale values using this pseudocode:
function [out] = bilinearInterpolation(im, out_dims)
%// Get some necessary variables first
in_rows = size(im,1);
in_cols = size(im,2);
out_rows = out_dims(1);
out_cols = out_dims(2);
%// Let S_R = R / R'
S_R = in_rows / out_rows;
%// Let S_C = C / C'
S_C = in_cols / out_cols;
%// Define grid of co-ordinates in our image
%// Generate (x,y) pairs for each point in our image
[cf, rf] = meshgrid(1 : out_cols, 1 : out_rows);
%// Let r_f = r'*S_R for r = 1,...,R'
%// Let c_f = c'*S_C for c = 1,...,C'
rf = rf * S_R;
cf = cf * S_C;
%// Let r = floor(rf) and c = floor(cf)
r = floor(rf);
c = floor(cf);
%// Any values out of range, cap
r(r < 1) = 1;
c(c < 1) = 1;
r(r > in_rows - 1) = in_rows - 1;
c(c > in_cols - 1) = in_cols - 1;
%// Let delta_R = rf - r and delta_C = cf - c
delta_R = rf - r;
delta_C = cf - c;
%// Final line of algorithm
%// Get column major indices for each point we wish
%// to access
in1_ind = sub2ind([in_rows, in_cols], r, c);
in2_ind = sub2ind([in_rows, in_cols], r+1,c);
in3_ind = sub2ind([in_rows, in_cols], r, c+1);
in4_ind = sub2ind([in_rows, in_cols], r+1, c+1);
%// Now interpolate
%// Go through each channel for the case of colour
%// Create output image that is the same class as input
out = zeros(out_rows, out_cols, size(im, 3));
out = cast(out, class(im));
for idx = 1 : size(im, 3)
chan = double(im(:,:,idx)); %// Get i'th channel
%// Interpolate the channel
tmp = chan(in1_ind).*(1 - delta_R).*(1 - delta_C) + ...
chan(in2_ind).*(delta_R).*(1 - delta_C) + ...
chan(in3_ind).*(1 - delta_R).*(delta_C) + ...
chan(in4_ind).*(delta_R).*(delta_C);
out(:,:,idx) = cast(tmp, class(im));
end
Take the above code, copy and paste it into a file called bilinearInterpolation.m and save it. Make sure you change your working directory where you've saved this file.
Except for sub2ind and perhaps meshgrid, everything seems to be in accordance with the algorithm. meshgrid is very easy to explain. All you're doing is specifying a 2D grid of (x,y) co-ordinates, where each location in your image has a pair of (x,y) or column and row co-ordinates. Creating a grid through meshgrid avoids any for loops as we will have generated all of the right pixel locations from the algorithm that we need before we continue.
How sub2ind works is that it takes in a row and column location in a 2D matrix (well... it can really be any amount of dimensions you want), and it outputs a single linear index. If you're not aware of how MATLAB indexes into matrices, there are two ways you can access an element in a matrix. You can use the row and column to get what you want, or you can use a column-major index. Take a look at this matrix example I have below:
A =
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
If we want to access the number 9, we can do A(2,4) which is what most people tend to default to. There is another way to access the number 9 using a single number, which is A(11)... now how is that the case? MATLAB lays out the memory of its matrices in column-major format. This means that if you were to take this matrix and stack all of its columns together in a single array, it would look like this:
A =
1
6
11
2
7
12
3
8
13
4
9
14
5
10
15
Now, if you want to access element number 9, you would need to access the 11th element of this array. Going back to the interpolation bit, sub2ind is crucial if you want to vectorize accessing the elements in your image to do the interpolation without doing any for loops. As such, if you look at the last line of the pseudocode, we want to access elements at r, c, r+1 and c+1. Note that all of these are 2D arrays, where each element in each of the matching locations in all of these arrays tell us the four pixels we need to sample from in order to produce the final output pixel. The output of sub2ind will also be 2D arrays of the same size as the output image. The key here is that each element of the 2D arrays of r, c, r+1, and c+1 will give us the column-major indices into the image that we want to access, and by throwing this as input into the image for indexing, we will exactly get the pixel locations that we want.
There are some important subtleties I'd like to add when implementing the algorithm:
You need to make sure that any indices to access the image when interpolating outside of the image are either set to 1 or the number of rows or columns to ensure you don't go out of bounds. Actually, if you extend to the right or below the image, you need to set this to one below the maximum as the interpolation requires that you are accessing pixels to one over to the right or below. This will make sure that you're still within bounds.
You also need to make sure that the output image is cast to the same class as the input image.
I ran through a for loop to interpolate each channel on its own. You could do this intelligently using bsxfun, but I decided to use a for loop for simplicity, and so that you are able to follow along with the algorithm.
As an example to show this works, let's use the onion.png image that is part of MATLAB's system path. The original dimensions of this image are 135 x 198. Let's interpolate this image by making it larger, going to 270 x 396 which is twice the size of the original image:
im = imread('onion.png');
out = bilinearInterpolation(im, [270 396]);
figure;
imshow(im);
figure;
imshow(out);
The above code will interpolate the image by increasing each dimension by twice as much, then show a figure with the original image and another figure with the scaled up image. This is what I get for both:
Similarly, let's shrink the image down by half as much:
im = imread('onion.png');
out = bilinearInterpolation(im, [68 99]);
figure;
imshow(im);
figure;
imshow(out);
Note that half of 135 is 67.5 for the rows, but I rounded up to 68. This is what I get:
One thing I've noticed in practice is that upsampling with bilinear has decent performance in comparison to other schemes like bicubic... or even Lanczos. However, when you're shrinking an image, because you're removing detail, nearest neighbour is very much sufficient. I find bilinear or bicubic to be overkill. I'm not sure about what your application is, but play around with the different interpolation algorithms and see what you like out of the results. Bicubic is another story, and I'll leave that to you as an exercise. Those slides I referred you to does have material on bicubic interpolation if you're interested.
Good luck!

Increase image contrast using look up table in MATLAB

I am trying to do some image processing for which I am given an 8-bit grayscale image. I am supposed to change the contrast of the image by generating a lookup table that increases the contrast for pixel values between 50 and 205. I have generated a look up table using the following MATLAB code.
a = 2;
x = 0:255;
lut = 255 ./ (1+exp(-a*(x-127)/32));
When I plot lut, I get a graph shown below:
So far so good, but how do I go about increasing the contrast for pixel values between 50 and 205? Final plot of the transform mapping should be something like:
Judging from your comments, you simply want a linear map where intensities that are < 50 get mapped to 0, intensities that are > 205 get mapped to 255, and everything else is a linear mapping in between. You can simply do this by:
slope = 255 / (205 - 50); % // Generate equation of the line -
% // y = mx + b - Solve for m
intercept = -50*slope; %// Solve for b --> b = y - m*x, y = 0, x = 50
LUT = uint8(slope*(0:255) + intercept); %// Generate points
LUT(1:51) = 0; %// Anything < intensity 50 set to 0
LUT(206:end) = 255; %// Anything > intensity 205 set to 255
The LUT now looks like:
plot(0:255, LUT);
axis tight;
grid;
Take note at how I truncated the intensities when they're < 50 and > 205. MATLAB starts indexing at index 1, and so we need to offset the intensities by 1 so that they correctly map to pixel intensities which start at 0.
To finally apply this to your image, all you have to do is:
out = LUT(img + 1);
This is assuming that img is your input image. Again, take note that we had to offset the input by +1 as MATLAB starts indexing at location 1, while intensities start at 0.
Minor Note
You can easily do this by using imadjust, which basically does this for you under the hood. You call it like so:
outAdjust = imadjust(in, [low_in; high_in], [low_out; high_out]);
low_in and high_in represent the minimum and maximum input intensities that exist in your image. Note that these are normalized between [0,1]. low_out and high_out adjust the intensities of your image so that low_in maps to low_out, high_in maps to high_out, and everything else is contrast stretched in between. For your case, you would do:
outAdjust = imadjust(img, [0; 1], [50/255; 205/255]);
This should stretch the contrast such that the input intensity 50 maps to the output intensity 0 and the input intensity 205 maps to the output intensity 255. Any intensities < 50 and > 205 get automatically saturated to 0 and 255 respectively.
You need to take each pixel in your image and replace it with the corresponding value in the lookup table. This can be done with some nested for loops, but it is not the most idiomatic way to do it. I would recommend using arrayfun with a function that replaces a pixel.
new_image = arrayfun(#(pixel) lut(pixel), image);
It might be more efficient to use the code that generates lut directly on the image. If performance is a concern and you don't need to use a lookup table, try comparing both methods.
new_image = 255 ./ (1 + exp(-image * (x-127) / 32));
Note that the new_image variable will no longer be of type uint8. If you need to display it again (say, with imshow) you will need to convert it back by writing uint8(new_image).

How to process a "Very Large image file" in MATLAB and find out the pixel with largest value

I have to process a very large image ( say 10 MB image file or even more).I have to remove artifacts and dead pixels in MATLAB
I have read about Block Processing of Large Images, but have no idea how to apply it to a 16 bit image.
I am referring to removal of pixels which have highest value into the average value of surrounding pixel .my code is not working on my image which is 80 MB of size
numIterations = 30;
avgPrecisionSize = 16; % smaller is better, but takes longer
% Read in the image grayscale:
originalImage = double(rgb2gray(imread('C:\Documents and Settings\admin\Desktop\TM\image5.tif')));
% get the bad pixels where = 0 and dilate to make sure they get everything:
badPixels = (originalImage == 0);
badPixels = imdilate(badPixels, ones(12));
%# Create a big gaussian and an averaging kernel to use:
G = fspecial('gaussian',[1 1]*100,50);
H = fspecial('average', [1,1]*avgPrecisionSize);
%# User a big filter to get started:
newImage = imfilter(originalImage,G,'same');
newImage(~badPixels) = originalImage(~badPixels);
% Now average to
for count = 1:numIterations
newImage = imfilter(newImage, H, 'same');
newImage(~badPixels) = originalImage(~badPixels);
end
%% Plot the results
figure(123);
clf;
% Display the mask:
subplot(1,2,1);
imagesc(badPixels);
axis image
title('Region Of the Bad Pixels');
% Display the result:
subplot(1,2,2);
imagesc(newImage);
axis image
set(gca,'clim', [0 255])
title('Infilled Image');
colormap gray
newImage2 = roifill(originalImage, badPixels);
figure(44);
clf;
imagesc(newImage2);
colormap gray
You are doing a few things which are obvious problems (but it might depend specifically on how far you can get into the code before you run out of memory)
1) You are immediately converting the whole image to double
2) You are identifying certain pixels which you want to replace, but passing the whole image to imfilter and then throwing away (presumably) most of the output:
newImage = imfilter(originalImage,G,'same'); % filter across the entire image
newImage(~badPixels) = originalImage(~badPixels); % replace all the good pixels!
Without converting to double, why not first check where the bad pixels are, do your processing on subregions of the appropriate size around those pixels (the subregions can be converted to double and back), and then reassemble the image at the end?
blockproc may work if you can write your filtering option as a function which takes in an image area and returns the correct area - you'll have to use the border_size option appropriately and make sure your function just returns the original image without bothering to do any filtering if it hits a block with no bad pixels in. You can even have it write out to file as well.

Resources