I have to use region-based to classify foreground (FG) and background (BG). I read many papers about that problem. However, almost papers that I read, they often using mean feature to compare the mean square error such as
(I(x)-mean(FG)).^2>(I(x)-mean(BG)).^2=>x belong to BG
Some authors add some condition that used statistical reigon (add sigma term). How about the other feature to discribe image region. Could you suggest to me some feature? Thank you so much
Try this function I created in Matlab called 'Quantisation', note this is for grey scale images. it automatically finds thresholds in the image and will classify all pixels under 1 of the categories, FG or BG:
function [quant2_A_min,quant2_A_max] = Quantization(fname)
% If there is less that one input argument 'B20.BMP' will be read in.
if nargin <1
fname='B20.BMP'
end
% define fname as the variable 'X'.
X=fname;
%splits the image into 2 levels by obtaining 2 threshold values.
thresh = multithresh(X,1);
%Contructs a vector of max values so that the max value in each
%quantization interval is assigned to the 2 levels of o/p image
valuesMax = [thresh max(X(:))];
[quant2_A_max, index] = imquantize(X,thresh,valuesMax);
%Contructs a vector of min values so that the min value in each
%quantization interval is assigned to the 2 levels of the o/p image
valuesMin = [min(X(:)) thresh];
%use the output argument index to assign the MIN values to the output image
%instead if called imquantize again.
quant2_A_min = valuesMin(index);
%Display both 2 level images side by side
imshowpair(quant2_A_min,quant2_A_max,'montage');...
title('Quantised Images (Min & Max)', 'FontSize',14,...
'fontweight','bold');
end
Related
I would like to calculate the median of each pixel in a set of images or "video". However, when MATLAB starts calculating this, it takes a very long time and finishes randomly with an index error. Why?
This is the code:
V = VideoReader('hall_monitor.avi');
info = get(V);
M = info.Width;
N = info.Height;
nb_frames_bk = 5;
v_pixel = zeros([nb_frames_bk 3]);
IB=zeros([M N 3],'double');
for i=1:M
for j=1:N
for k=1:nb_frames_bk
frm=read(V,k);
v_pixel(k,:)=frm(i,j,:);
end
IB(i,j,:)=median(v_pixel(:,:));
end
end
IB=uint8(IB);
imshow(IB);
This code can benefit from a lot of refactoring. For one thing, you are re-reading frames when you can just read them once, store them and use them after you're done.
Secondly, iterating over all pixels to compute your median is going to be very slow. From what it looks like in your code, for each spatial position over the first nb_frames_bk frames, you collect all of the RGB values within these frames and calculate the median RGB value.
Also as a minor note, you are getting a dimension exceeds error because you defined the output matrix wrong. You defined it as M x N with M being the width and N being the height. This needs to be swapped. Remember that matrices are defined as height first, width second. However, this is unnecessary with what I'm going to suggest for implementing this properly.
Instead of reading the frames one at a time, specify a range of frames. This way, you will get a 4D matrix where the first three dimensions references an image, with the fourth dimension representing the frame number. You can then take the median in the fourth dimension to find the median RGB value over all frames.
In other words, simply do this:
V = VideoReader('hall_monitor.avi');
nb_frames_bk = 5;
frms = read(V, [1 nb_frames_bk]);
IB = median(frms, 4);
imshow(IB);
This is much better, to the point and guaranteed to be faster. You also don't need to obtain the width and height of each frame as it is no longer needed as we are no longer looping over each pixel.
I am learning image analysis and trying to average set of color images and get standard deviation at each pixel
I have done this, but it is not by averaging RGB channels. (for ex rchannel = I(:,:,1))
filelist = dir('dir1/*.jpg');
ims = zeros(215, 300, 3);
for i=1:length(filelist)
imname = ['dir1/' filelist(i).name];
rgbim = im2double(imread(imname));
ims = ims + rgbim;
end
avgset1 = ims/length(filelist);
figure;
imshow(avgset1);
I am not sure if this is correct. I am confused as to how averaging images is useful.
Also, I couldn't get the matrix holding standard deviation.
Any help is appreciated.
If you are concerned about finding the mean RGB image, then your code is correct. What I like is that you converted the images using im2double before accumulating the mean and so you are making everything double precision. As what Parag said, finding the mean image is very useful especially in machine learning. It is common to find the mean image of a set of images before doing image classification as it allows the dynamic range of each pixel to be within a normalized range. This allows the training of the learning algorithm to converge quickly to the optimum solution and provide the best set of parameters to facilitate the best accuracy in classification.
If you want to find the mean RGB colour which is the average colour over all images, then no your code is not correct.
You have summed over all channels individually which is stored in sumrgbims, so the last step you need to do now take this image and sum over each channel individually. Two calls to sum in the first and second dimensions chained together will help. This will produce a 1 x 1 x 3 vector, so using squeeze after this to remove the singleton dimensions and get a 3 x 1 vector representing the mean RGB colour over all images is what you get.
Therefore:
mean_colour = squeeze(sum(sum(sumrgbims, 1), 2));
To address your second question, I'm assuming you want to find the standard deviation of each pixel value over all images. What you will have to do is accumulate the square of each image in addition to accumulating each image inside the loop. After that, you know that the standard deviation is the square root of the variance, and the variance is equal to the average sum of squares subtracted by the mean squared. We have the mean image, now you just have to square the mean image and subtract this with the average sum of squares. Just to be sure our math is right, supposing we have a signal X with a mean mu. Given that we have N values in our signal, the variance is thus equal to:
Source: Science Buddies
The standard deviation would simply be the square root of the above result. We would thus calculate this for each pixel independently. Therefore you can modify your loop to do that for you:
filelist = dir('set1/*.jpg');
sumrgbims = zeros(215, 300, 3);
sum2rgbims = sumrgbims; % New - for standard deviation
for i=1:length(filelist)
imname = ['set1/' filelist(i).name];
rgbim = im2double(imread(imname));
sumrgbims = sumrgbims + rgbim;
sum2rgbims = sum2rgbims + rgbim.^2; % New
end
rgbavgset1 = sumrgbims/length(filelist);
% New - find standard deviation
rgbstdset1 = ((sum2rgbims / length(filelist)) - rgbavgset.^2).^(0.5);
figure;
imshow(rgbavgset1, []);
% New - display standard deviation image
figure;
imshow(rgbstdset1, []);
Also to make sure, I've scaled the display of each imshow call so the smallest value gets mapped to 0 and the largest value gets mapped to 1. This does not change the actual contents of the images. This is just for display purposes.
I've found some methods to enlarge an image but there is no solution to shrink an image. I'm currently using the nearest neighbor method. How could I do this with bilinear interpolation without using the imresize function in MATLAB?
In your comments, you mentioned you wanted to resize an image using bilinear interpolation. Bear in mind that the bilinear interpolation algorithm is size independent. You can very well use the same algorithm for enlarging an image as well as shrinking an image. The right scale factors to sample the pixel locations are dependent on the output dimensions you specify. This doesn't change the core algorithm by the way.
Before I start with any code, I'm going to refer you to Richard Alan Peters' II digital image processing slides on interpolation, specifically slide #59. It has a great illustration as well as pseudocode on how to do bilinear interpolation that is MATLAB friendly. To be self-contained, I'm going to include his slide here so we can follow along and code it:
Please be advised that this only resamples the image. If you actually want to match MATLAB's output, you need to disable anti-aliasing.
MATLAB by default will perform anti-aliasing on the images to ensure the output looks visually pleasing. If you'd like to compare apples with apples, make sure you disable anti-aliasing when comparing between this implementation and MATLAB's imresize function.
Let's write a function that will do this for us. This function will take in an image (that is read in through imread) which can be either colour or grayscale, as well as an array of two elements - The image you want to resize and the output dimensions in a two-element array of the final resized image you want. The first element of this array will be the rows and the second element of this array will be the columns. We will simply go through this algorithm and calculate the output pixel colours / grayscale values using this pseudocode:
function [out] = bilinearInterpolation(im, out_dims)
%// Get some necessary variables first
in_rows = size(im,1);
in_cols = size(im,2);
out_rows = out_dims(1);
out_cols = out_dims(2);
%// Let S_R = R / R'
S_R = in_rows / out_rows;
%// Let S_C = C / C'
S_C = in_cols / out_cols;
%// Define grid of co-ordinates in our image
%// Generate (x,y) pairs for each point in our image
[cf, rf] = meshgrid(1 : out_cols, 1 : out_rows);
%// Let r_f = r'*S_R for r = 1,...,R'
%// Let c_f = c'*S_C for c = 1,...,C'
rf = rf * S_R;
cf = cf * S_C;
%// Let r = floor(rf) and c = floor(cf)
r = floor(rf);
c = floor(cf);
%// Any values out of range, cap
r(r < 1) = 1;
c(c < 1) = 1;
r(r > in_rows - 1) = in_rows - 1;
c(c > in_cols - 1) = in_cols - 1;
%// Let delta_R = rf - r and delta_C = cf - c
delta_R = rf - r;
delta_C = cf - c;
%// Final line of algorithm
%// Get column major indices for each point we wish
%// to access
in1_ind = sub2ind([in_rows, in_cols], r, c);
in2_ind = sub2ind([in_rows, in_cols], r+1,c);
in3_ind = sub2ind([in_rows, in_cols], r, c+1);
in4_ind = sub2ind([in_rows, in_cols], r+1, c+1);
%// Now interpolate
%// Go through each channel for the case of colour
%// Create output image that is the same class as input
out = zeros(out_rows, out_cols, size(im, 3));
out = cast(out, class(im));
for idx = 1 : size(im, 3)
chan = double(im(:,:,idx)); %// Get i'th channel
%// Interpolate the channel
tmp = chan(in1_ind).*(1 - delta_R).*(1 - delta_C) + ...
chan(in2_ind).*(delta_R).*(1 - delta_C) + ...
chan(in3_ind).*(1 - delta_R).*(delta_C) + ...
chan(in4_ind).*(delta_R).*(delta_C);
out(:,:,idx) = cast(tmp, class(im));
end
Take the above code, copy and paste it into a file called bilinearInterpolation.m and save it. Make sure you change your working directory where you've saved this file.
Except for sub2ind and perhaps meshgrid, everything seems to be in accordance with the algorithm. meshgrid is very easy to explain. All you're doing is specifying a 2D grid of (x,y) co-ordinates, where each location in your image has a pair of (x,y) or column and row co-ordinates. Creating a grid through meshgrid avoids any for loops as we will have generated all of the right pixel locations from the algorithm that we need before we continue.
How sub2ind works is that it takes in a row and column location in a 2D matrix (well... it can really be any amount of dimensions you want), and it outputs a single linear index. If you're not aware of how MATLAB indexes into matrices, there are two ways you can access an element in a matrix. You can use the row and column to get what you want, or you can use a column-major index. Take a look at this matrix example I have below:
A =
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
If we want to access the number 9, we can do A(2,4) which is what most people tend to default to. There is another way to access the number 9 using a single number, which is A(11)... now how is that the case? MATLAB lays out the memory of its matrices in column-major format. This means that if you were to take this matrix and stack all of its columns together in a single array, it would look like this:
A =
1
6
11
2
7
12
3
8
13
4
9
14
5
10
15
Now, if you want to access element number 9, you would need to access the 11th element of this array. Going back to the interpolation bit, sub2ind is crucial if you want to vectorize accessing the elements in your image to do the interpolation without doing any for loops. As such, if you look at the last line of the pseudocode, we want to access elements at r, c, r+1 and c+1. Note that all of these are 2D arrays, where each element in each of the matching locations in all of these arrays tell us the four pixels we need to sample from in order to produce the final output pixel. The output of sub2ind will also be 2D arrays of the same size as the output image. The key here is that each element of the 2D arrays of r, c, r+1, and c+1 will give us the column-major indices into the image that we want to access, and by throwing this as input into the image for indexing, we will exactly get the pixel locations that we want.
There are some important subtleties I'd like to add when implementing the algorithm:
You need to make sure that any indices to access the image when interpolating outside of the image are either set to 1 or the number of rows or columns to ensure you don't go out of bounds. Actually, if you extend to the right or below the image, you need to set this to one below the maximum as the interpolation requires that you are accessing pixels to one over to the right or below. This will make sure that you're still within bounds.
You also need to make sure that the output image is cast to the same class as the input image.
I ran through a for loop to interpolate each channel on its own. You could do this intelligently using bsxfun, but I decided to use a for loop for simplicity, and so that you are able to follow along with the algorithm.
As an example to show this works, let's use the onion.png image that is part of MATLAB's system path. The original dimensions of this image are 135 x 198. Let's interpolate this image by making it larger, going to 270 x 396 which is twice the size of the original image:
im = imread('onion.png');
out = bilinearInterpolation(im, [270 396]);
figure;
imshow(im);
figure;
imshow(out);
The above code will interpolate the image by increasing each dimension by twice as much, then show a figure with the original image and another figure with the scaled up image. This is what I get for both:
Similarly, let's shrink the image down by half as much:
im = imread('onion.png');
out = bilinearInterpolation(im, [68 99]);
figure;
imshow(im);
figure;
imshow(out);
Note that half of 135 is 67.5 for the rows, but I rounded up to 68. This is what I get:
One thing I've noticed in practice is that upsampling with bilinear has decent performance in comparison to other schemes like bicubic... or even Lanczos. However, when you're shrinking an image, because you're removing detail, nearest neighbour is very much sufficient. I find bilinear or bicubic to be overkill. I'm not sure about what your application is, but play around with the different interpolation algorithms and see what you like out of the results. Bicubic is another story, and I'll leave that to you as an exercise. Those slides I referred you to does have material on bicubic interpolation if you're interested.
Good luck!
I have two black and white images and I need to calculate the mutual information.
Image 1 = X
Image 2 = Y
I know that the mutual information can be defined as:
MI = entropy(X) + entropy(Y) - JointEntropy(X,Y)
MATLAB already has built-in functions to calculate the entropy but not to calculate the joint entropy. I guess the true question is: How do I calculate the joint entropy of two images?
Here is an example of the images I'd like to find the joint entropy of:
X =
0 0 0 0 0 0
0 0 1 1 0 0
0 0 1 1 0 0
0 0 0 0 0 0
0 0 0 0 0 0
Y =
0 0 0 0 0 0
0 0 0.38 0.82 0.38 0.04
0 0 0.32 0.82 0.68 0.17
0 0 0.04 0.14 0.11 0
0 0 0 0 0 0
To calculate the joint entropy, you need to calculate the joint histogram between two images. The joint histogram is essentially the same as a normal 1D histogram but the first dimension logs intensities for the first image and the second dimension logs intensities for the second image. This is very similar to what is commonly referred to as a co-occurrence matrix. At location (i,j) in the joint histogram, it tells you how many intensity values we have encountered that have intensity i in the first image and intensity j in the second image.
What is important is that this logs how many times we have seen this pair of intensities at the same corresponding locations. For example, if we have a joint histogram count of (7,3) = 2, this means that when we were scanning both images, when we encountered the intensity of 7, at the same corresponding location in the second image, we encountered the intensity of 3 for a total of 2 times.
Constructing a joint histogram is very simple to do.
First, create a 256 x 256 matrix (assuming your image is unsigned 8-bit integer) and initialize them to all zeroes. Also, you need to make sure that both of your images are the same size (width and height).
Once you do that, take a look at the first pixel of each image, which we will denote as the top left corner. Specifically, take a look at the intensities for the first and second image at this location. The intensity of the first image will serve as the row while the intensity of the second image will serve as the column.
Find this location in the matrix and increment this spot in the matrix by 1.
Repeat this for the rest of the locations in your image.
After you're done, divide all entries by the total number of elements in either image (remember they should be the same size). This will give us the joint probability distribution between both images.
One would be inclined to do this with for loops, but as it is commonly known, for loops are notoriously slow and should be avoided if at all possible. However, you can easily do this in MATLAB in the following way without loops. Let's assume that im1 and im2 are the first and second images you want to compare to. What we can do is convert im1 and im2 into vectors. We can then use accumarray to help us compute the joint histogram. accumarray is one of the most powerful functions in MATLAB. You can think of it as a miniature MapReduce paradigm. Simply put, each data input has a key and an associated value. The goal of accumarray is to bin all of the values that belong to the same key and do some operation on all of these values. In our case, the "key" would be the intensity values, and the values themselves are the value of 1 for every intensity value. We would then want to add up all of the values of 1 that map to the same bin, which is exactly how we'd compute a histogram. The default behaviour for accumarray is to add all of these values. Specifically, the output of accumarray would be an array where each position computes the sum of all values that mapped to that key. For example, the first position would be the summation of all values that mapped to the key of 1, the second position would be the summation of all values that mapped to the key of 2 and so on.
However, for the joint histogram, you want to figure out which values map to the same intensity pair of (i,j), and so the keys here would be a pair of 2D coordinates. As such, any intensities that have an intensity of i in the first image and j in the second image in the same spatial location shared between the two images go to the same key. Therefore in the 2D case, the output of accumarray would be a 2D matrix where each element (i,j) contains the summation of all values that mapped to key (i,j), similar to the 1D case that was mentioned previously which is exactly what we are after.
In other words:
indrow = double(im1(:)) + 1;
indcol = double(im2(:)) + 1; %// Should be the same size as indrow
jointHistogram = accumarray([indrow indcol], 1);
jointProb = jointHistogram / numel(indrow);
With accumarray, the first input are the keys and the second input are the values. A note with accumarray is that if each key has the same value, you can simply assign a constant to the second input, which is what I've done and it's 1. In general, this is an array with the same number of rows as the first input. Also, take special note of the first two lines. There will inevitably be an intensity of 0 in your image, but because MATLAB starts indexing at 1, we need to offset both arrays by 1.
Now that we have the joint histogram, it's really simple to calculate the joint entropy. It is similar to the entropy in 1D, except now we are just summing over the entire joint probability matrix. Bear in mind that it will be very likely that your joint histogram will have many 0 entries. We need to make sure that we skip those or the log2 operation will be undefined. Let's get rid of any zero entries now:
indNoZero = jointHistogram ~= 0;
jointProb1DNoZero = jointProb(indNoZero);
Take notice that I searched the joint histogram instead of the joint probability matrix. This is because the joint histogram consists of whole numbers while the joint probability matrix will lie between 0 and 1. Because of the division, I want to avoid comparing any entries in this matrix with 0 due to numerical roundoff and instability. The above will also convert our joint probability matrix into a stacked 1D vector, which is fine.
As such, the joint entropy can be calculated as:
jointEntropy = -sum(jointProb1DNoZero.*log2(jointProb1DNoZero));
If my understanding of calculating entropy for an image in MATLAB is correct, it should calculate the histogram / probability distribution over 256 bins, so you can certainly use that function here with the joint entropy that was just calculated.
What if we have floating-point data instead?
So far, we have assumed that the images that you have dealt with have intensities that are integer-valued. What if we have floating point data? accumarray assumes that you are trying to index into the output array using integers, but we can still certainly accomplish what we want with this small bump in the road. What you would do is simply assign each floating point value in both images to have a unique ID. You would thus use accumarray with these IDs instead. To facilitate this ID assigning, use unique - specifically the third output from the function. You would take each of the images, put them into unique and make these the indices to be input into accumarray. In other words, do this instead:
[~,~,indrow] = unique(im1(:)); %// Change here
[~,~,indcol] = unique(im2(:)); %// Change here
%// Same code
jointHistogram = accumarray([indrow indcol], 1);
jointProb = jointHistogram / numel(indrow);
indNoZero = jointHistogram ~= 0;
jointProb1DNoZero = jointProb(indNoZero);
jointEntropy = -sum(jointProb1DNoZero.*log2(jointProb1DNoZero));
Note that with indrow and indcol, we are directly assigning the third output of unique to these variables and then using the same joint entropy code that we computed earlier. We also don't have to offset the variables by 1 as we did previously because unique will assign IDs starting at 1.
Aside
You can actually calculate the histograms or probability distributions for each image individually using the joint probability matrix. If you wanted to calculate the histograms / probability distributions for the first image, you would simply accumulate all of the columns for each row. To do it for the second image, you would simply accumulate all of the rows for each column. As such, you can do:
histogramImage1 = sum(jointHistogram, 1);
histogramImage2 = sum(jointHistogram, 2);
After, you can calculate the entropy of both of these by yourself. To double check, make sure you turn both of these into PDFs, then compute the entropy using the standard equation (like above).
How do I finally compute Mutual Information?
To finally compute Mutual Information, you're going to need the entropy of the two images. You can use MATLAB's built-in entropy function, but this assumes that there are 256 unique levels. You probably want to apply this for the case of there being N distinct levels instead of 256, and so you can use what we did above with the joint histogram, then computing the histograms for each image in the aside code above, and then computing the entropy for each image. You would simply repeat the entropy calculation that was used jointly, but apply it to each image individually:
%// Find non-zero elements for first image's histogram
indNoZero = histogramImage1 ~= 0;
%// Extract them out and get the probabilities
prob1NoZero = histogramImage1(indNoZero);
prob1NoZero = prob1NoZero / sum(prob1NoZero);
%// Compute the entropy
entropy1 = -sum(prob1NoZero.*log2(prob1NoZero));
%// Repeat for the second image
indNoZero = histogramImage2 ~= 0;
prob2NoZero = histogramImage2(indNoZero);
prob2NoZero = prob2NoZero / sum(prob2NoZero);
entropy2 = -sum(prob2NoZero.*log2(prob2NoZero));
%// Now compute mutual information
mutualInformation = entropy1 + entropy2 - jointEntropy;
Hope this helps!
I want to create a program that can count the number of objects in an image. All went smoothly except for the images that have:
objects that have more than 1 color.
overlapping objects.
Following is my program that can only count the number of objects in an image where the object has only 1 color and not overlapped. I use the function bwlabel.
a=imread('Tumpukan Buku2.jpg');
a_citra_keabuan = rgb2gray(a);
threshold = graythresh(a_citra_keabuan);
a_bww = im2bw(a_citra_keabuan,threshold);
a_bw=~a_bww;
[labeled,numObjects]=bwlabel(a_bw);
[m,n]=size(a_bw);
s = regionprops(labeled, 'Centroid');
B = bwboundaries(a_bw);
imshow(a_bw)
hold on
for k = 1:numel(s)
c = s(k).Centroid;
text(c(1), c(2), sprintf('%d', k), ...
'HorizontalAlignment', 'center', ...
'VerticalAlignment', 'middle');
end
for k = 1:length(B)
boundary = B{k};
plot(boundary(:,2), boundary(:,1), 'g', 'LineWidth', 0.2)
end
hold off
Here is the result for an image that has objects in 1 color:
and here is the wrong result for the image with object that has > 1 color and overlapped:
How to solve this problem?
First, you need to clearly define your input data- what types of objects do you want to detect (books, people, any types of objects?), what is the range of environmental conditions (smooth background vs. textured, lighting, perspective).
Then try out various image segmentation techniques and seeing what works for your range of input data. There is no "right" answer - it all depends on your data.
You might also try to incorporate prior information- things that you know when evaluating a scene, that a computer will not know by just evaluating pixels.
For example, maybe all objects are of some minimum size. So your algorithm can filter to only return objects having pixelArea > minArea. Maybe you only expect one objet of each color. So if the color histogram of two detected objects matches to within a given tolerance, consider them to be the same object.