I have 1024 images, each of which has the size 150(y) x 270(x) which is stored as 3D array with the size of 150 x 270 x 1024
First 160 images are very similar to each other but not entirely identical;
however, after 160th image, pictures start to change drastically.
So, here is what I want to achieve:
I want to find the index of the image from which the images start to drastically change
I have tried to compare correlation between image #1 and other 1023 images by:
for ii = 1:1023
R(ii) = corr2(input(:,:,1),input(:,:,ii+1)); % input = 3D array (150 x 270 x 1024)
end
and see if there is any change in correlation coefficient at around 160th image, but wasn't successful.
What method can I use to detect changes in these images and to find the index at which my images start to change dramatically?
EDIT
following are some of the images I have (index in the title)
I guess the change is not as dramatic as I first described and when you look at image 160 and 161, the change is subtle but as it goes on, you can clearly see that the image definitely changes at the bottom part
These images are results of ultrasonic testing and wave propagation from PZT sensor starts at the bottom part of the image
Probably, this is not a full answer to your problem, but I hope I can give you some ideas to further work on. But before, I have to mention this: Image processing is two-dimensional signal processing! ;-)
After reading your question the 34th time (or so), I finally saw, that you're comparing "image" #1 with all others. Instead, you should compare neighbouring "images". I have put together the following script incorporating your approach of using corr2:
load('fullSet.mat');
%% Normalize values to [0, 1] with respect to whole volume.
%% Just for nicer image showing.
%minV = min(fullSet(:));
%fullSet = fullSet - minV;
%maxV = max(fullSet(:));
%fullSet = fullSet / maxV;
% Determine number of "images".
n = size(fullSet, 3);
% "Step size".
s = 1;
% Initialize R.
R = zeros(n-s, 1);
% Compute correlation coefficients.
for ii = 1:n-s
R(ii) = corr2(fullSet(:, :, ii), fullSet(:, :, ii + s));
end
% Show output.
figure(1);
subplot(3, 1, 1);
plot(1:n-s, R);
title('R');
xlim([0 n-s]);
subplot(3, 1, 2);
plot(1:n-s-1, diff(R, 1));
title('1st derivative of R');
xlim([0 n-s-1]);
subplot(3, 1, 3);
plot(1:n-s-2, diff(R, 2));
title('2nd derivative of R');
xlim([0 n-s-2]);
The "step size" defines which "image" should be compared to the current one, i.e. s = 1 is used to compare the current "image" with the next "image", s = 2 is used to compare the current "image" with the second-next "image", and so on.
The results for s = 1:
The results for s = 5:
The results for s = 10:
As you can see, there is a distinct change around [160, 200]. I also computed the 1st and 2nd derivatives, because here you can also see changes later in your "volume" - if this is of interest for you.
Please let me know, if you need further explanations on my script or if you need further help in general.
Related
I am learning image analysis and trying to average set of color images and get standard deviation at each pixel
I have done this, but it is not by averaging RGB channels. (for ex rchannel = I(:,:,1))
filelist = dir('dir1/*.jpg');
ims = zeros(215, 300, 3);
for i=1:length(filelist)
imname = ['dir1/' filelist(i).name];
rgbim = im2double(imread(imname));
ims = ims + rgbim;
end
avgset1 = ims/length(filelist);
figure;
imshow(avgset1);
I am not sure if this is correct. I am confused as to how averaging images is useful.
Also, I couldn't get the matrix holding standard deviation.
Any help is appreciated.
If you are concerned about finding the mean RGB image, then your code is correct. What I like is that you converted the images using im2double before accumulating the mean and so you are making everything double precision. As what Parag said, finding the mean image is very useful especially in machine learning. It is common to find the mean image of a set of images before doing image classification as it allows the dynamic range of each pixel to be within a normalized range. This allows the training of the learning algorithm to converge quickly to the optimum solution and provide the best set of parameters to facilitate the best accuracy in classification.
If you want to find the mean RGB colour which is the average colour over all images, then no your code is not correct.
You have summed over all channels individually which is stored in sumrgbims, so the last step you need to do now take this image and sum over each channel individually. Two calls to sum in the first and second dimensions chained together will help. This will produce a 1 x 1 x 3 vector, so using squeeze after this to remove the singleton dimensions and get a 3 x 1 vector representing the mean RGB colour over all images is what you get.
Therefore:
mean_colour = squeeze(sum(sum(sumrgbims, 1), 2));
To address your second question, I'm assuming you want to find the standard deviation of each pixel value over all images. What you will have to do is accumulate the square of each image in addition to accumulating each image inside the loop. After that, you know that the standard deviation is the square root of the variance, and the variance is equal to the average sum of squares subtracted by the mean squared. We have the mean image, now you just have to square the mean image and subtract this with the average sum of squares. Just to be sure our math is right, supposing we have a signal X with a mean mu. Given that we have N values in our signal, the variance is thus equal to:
Source: Science Buddies
The standard deviation would simply be the square root of the above result. We would thus calculate this for each pixel independently. Therefore you can modify your loop to do that for you:
filelist = dir('set1/*.jpg');
sumrgbims = zeros(215, 300, 3);
sum2rgbims = sumrgbims; % New - for standard deviation
for i=1:length(filelist)
imname = ['set1/' filelist(i).name];
rgbim = im2double(imread(imname));
sumrgbims = sumrgbims + rgbim;
sum2rgbims = sum2rgbims + rgbim.^2; % New
end
rgbavgset1 = sumrgbims/length(filelist);
% New - find standard deviation
rgbstdset1 = ((sum2rgbims / length(filelist)) - rgbavgset.^2).^(0.5);
figure;
imshow(rgbavgset1, []);
% New - display standard deviation image
figure;
imshow(rgbstdset1, []);
Also to make sure, I've scaled the display of each imshow call so the smallest value gets mapped to 0 and the largest value gets mapped to 1. This does not change the actual contents of the images. This is just for display purposes.
I would like to resize a 512X512 image into 363X762 image which will be larger than the original image(of size 512X512). Those extra pixel values must be different values in the range of 0-255.
I tried the following code:
I=imread('photo.jpg'); %photo.jpg is a 512X512 image
B=zeros(363,726);
sizeOfMatrixB=size(B);
display(sizeOfMatrixB);
B(1:262144)=I(1:262144);
imshow(B);
B(262155:263538)=0;
But I think this is a lengthy one and the output is also not as desired. Could anyone suggest me with a better piece of code to perform this. Thank you in advance.
I think that the code you have is actually pretty close to ideal except that you have a lot of hard-coded values in there. Those should really be computed on the fly. We can do that using numel to count the number of elements in B.
B = zeros(363, 726);
%// Assign the first 262144 elements of B to the values in I
%// all of the rest will remain as 0
B(1:numel(I)) = I;
This flexibility is important and the importance is actually demonstrated via the typo in your last line:
B(262155:263538)=0;
%// Should be
B(262144:263538)=0;
Also, you don't need these extra assignments to zero at the end because you initialize the matrix to be all zeros in the first place.
A Side Note
It looks like you are spreading the original image data for each column across multiple columns. I'm guessing this isn't what you want. You probably only want to grab the first 363 rows of I to be placed into B. You can do that this way:
B = zeros(363, 726);
B(1:size(B, 1), 1:size(I, 2)) = I(1:size(B, 1), :);
Update
If you want the other values to be something other than zero, you can initialize your matrix to be that value instead.
value = 2;
B = zeros(363, 726) + value;
B(1:numel(I)) = I;
If you want them to be random integers between 0 and 255, use randi to initialize the matrix.
B = randi([0 255], 363, 726);
B(1:numel(I)) = I;
I have an image of size (224 x 224) and I want to extract a number of random patches from the original image using Matlab (let say 5 patches). One of these patch should be at the centre of the original image. The patch size is (128 x 128).
I have tried this to crop just the centre patch:
II = imread('img.png')
[p3, p4] = size(II);
q1 = 50; // size of the crop box
i3_start = floor((p3-q1)/2); % or round instead of floor; using neither gives warning
i3_stop = i3_start + q1;
i4_start = floor((p4-q1)/2);
i4_stop = i4_start + q1;
II = II(i3_start:i3_stop, i4_start:i4_stop, :);
figure ,imshow(II);
I've tried to accomplish this in the following way:
A=imread('Lena.bmp');%sample image
rnd_x = randperm(size(A,1)-128,5);%choose 5 tandom unique points on x-axis
rnd_y = randperm(size(A,2)-128,5);%choose 5 tandom unique points on y-axis
for ii = 1:5
piece{ii} = A((rnd_x(ii):(rnd_x(ii)+127)),(rnd_y(ii):(rnd_y(ii)+127)),1:3);%Convert chosen numbers to image pieces
figure(ii)
imshow(piece{ii});
end
This takes image like this:
This gives 5 pics like this:
Here our image size is 512x512. So, if we want to cut the 128x128 piece from it, we need to seek from 385x385 grid (512-127). We find 5 random points on the grid expressed in rnd_x and rnd_y. Finally, we take the found points as the upper-left corners of the pieces and construct 128x128 images from them. The 5 pieces are recorded in piece cell array.
EDIT: forgot to add how to extract the center patch. The following code performs the task:
A=imread('Lena.bmp');%sample image
if mod(size(A,1),2)
A = A(1:(end-1),:,:);
end
if mod(size(A,2),2)
A = A(:,1:(end-1),:);
end
while size(A,1) > 128
A = A(2:(end-1),:,:);
end
while size(A,2) > 128
A = A(:,2:(end-1),:);
end
imshow(A)
The code removes one pixel from each side until we get the 128-pixel image.
Careful! In your code, if you load a color image (3 channels) and call size with only two outputs, you will have an incorrect value for p4.
Use three outputs when loading images to avoid this problem:
[nrows ncols nchannels] = size(II);
Your code correctly extracts a (q1 x q1) from the center of the image.
If you want a random patch just generate a random integer for the top-left column of the patch with the correct range to ensure that it doesn't fall outside the image. You can generate random integers using the function randi.
i3_start = randi(floor((p3-q1));
i4_start = randi(floor((p4-q1));
The rest of the code is the same. If you want several patches you can generate several values when calling the randi function with a second and third parameter for the desired number of rows and columns. And then process each patch inside a for loop.
BTW: In the third line you have an invalid Matlab comment (use % for comments). Also you should name your variables with more intuitive names.
Eg: [nrows ncols nchannels] = size(II);
I am performing a whisker-tracking experiments. I have high-speed videos (500fps) of rats whisking against objects. In each such video I tracked the shape of the rat's snout and whiskers. Since tracking is noisy, the number of whiskers in each frame may be different (see 2 consecutive frames in attached image, notice the yellow false-positive whisker appearing in the left frame but not the right one).
See example 1:
As an end result of tracking, I get, for each frame, a varying number of variable-length vectors; each vector corresponding to a single whisker. At this point I would like to match the whiskers between frames. I have tried using Matlab's sample align to do this, but it works only somewhat properly. Its results are attached below (in attached image showing basepoint of all whiskers over 227 frames).
See example 2:
I would like to run some algorithm to cluster the whiskers correctly, such that each whisker is recognized as itself and separated from other over the course of many frames. In other words, I would like each slightly sinusoidal trajectory in the second image to be recognized as one trajectory. Whatever sorting algorithm I use should take into account that whiskers may disappear and reappear between consecutive frames. Unfortunately, I'm all out of ideas...
Any help?
Once again, keep in mind that for each point in attached image 2, I have many data points, since this is only a plot of whisker basepoint, while in actuality I have data for the entire whisker length.
This is how I would deal with the problem. Assuming that data vectors of different size are in a cell type called dataVectors, and knowing the number of whiskers (nSignals), I would try to extend the data to a second dimension derived from the original data and then perform k-means on two dimensions.
So, first I would get the maximum size of the vectors in order to convert the data to a matrix and do NaN-padding.
maxSize = -Inf;
for k = 1:nSignals
if length(dataVectors{k}.data) > maxSize
maxSize = length(dataVectors{k}.data);
end
end
Now, I would make the data 2D by elevating it to power of two (or three, your choice). This is just a very simple transformation. But you could alternatively use kernel methods here and project each vector against the rest; however, I don't think this is necessary, and if your data is really big, it could be inefficient. For now, raising the data to the power of two should do the trick. The result is stored in a second dimension.
projDegree = 2;
projData = zeros(nSignals, maxSize, 2).*NaN;
for k = 1:nSignals
vecSize = length(dataVectors{k}.data);
projData(k, 1:vecSize, 1) = dataVectors{k}.data;
projData(k, 1:vecSize, 2) = dataVectors{k}.data.*projDegree;
end
projData = reshape(projData, [], 2);
Here, projData will have in row 1 and column 1, the original data of the first whisker (or signal as I call it here), and column 2 will have the new dimension. Let's suppose that you have 8 whiskers in total, then, projData will have the data of the first whisker in row 1, 9, 17, and so on. The data of the second whisker in row 2, 10, 18, and so forth. That is important if you want to work your way back to the original data. Also, you can try with different projDegrees but I doubt it will make a lot of difference.
Now we perform k-means on the 2D data; however, we provide the initial points instead of letting it determine them with k-means++. The initial points, as I propose here, are the first data point of each vector for each whisker. In this manner, k-means will depart from there and will move to clusters means accordingly. We save the results in idxK.
idxK = kmeans(projData,nSignals, 'Start', projData(1:nSignals, :));
And there you have it. The variable idxK will tell you which data point belongs to what cluster.
Below is a working example of my proposed solution. The first part is simply trying to produce data that looks like your data, you can skip it.
rng(9, 'twister')
nSignals = 8; % number of whiskers
n = 1000; % number of data points
allData = zeros(nSignals, n); % all the data will be stored here
% this loop will just generate some data that looks like yours
for k = 1:nSignals
x = sort(rand(1,n));
nPeriods = round(rand*9)+1; % the sin can have between 1-10 periods
nShiftAmount = round(randn*30); % shift between ~ -100 to +100
y = sin(x*2*pi*nPeriods) + (randn(1,n).*0.5);
y = y + nShiftAmount;
allData(k, :) = y;
end
nanIdx = round(rand(1, round(n*0.05)*nSignals).*((n*nSignals)-1))+1;
allData(nanIdx) = NaN; % about 5% of the data is now missing
figure(1);
for k = 1:nSignals
nanIdx = ~isnan(allData(k, :));
dataVectors{k}.data = allData(k, nanIdx);
plot(dataVectors{k}.data, 'kx'), hold on;
end
% determine the max size
maxSize = -Inf;
for k = 1:nSignals
if length(dataVectors{k}.data) > maxSize
maxSize = length(dataVectors{k}.data);
end
end
% making the data now into two dimensions and NaN pad
projDegree = 2;
projData = zeros(nSignals, maxSize, 2).*NaN;
for k = 1:nSignals
vecSize = length(dataVectors{k}.data);
projData(k, 1:vecSize, 1) = dataVectors{k}.data;
projData(k, 1:vecSize, 2) = dataVectors{k}.data.*projDegree;
end
projData = reshape(projData, [], 2);
figure(2); plot(projData(:,1), projData(:,2), 'kx');
% run k-means using the first points of all measure as the initial points
idxK = kmeans(projData,nSignals, 'Start', projData(1:nSignals, :));
figure(3);
liColors = [{'yx'},{'mx'},{'cx'},{'bx'},{'kx'},{'gx'},{'rx'},{'gd'}];
for k = 1:nSignals
plot(projData(idxK==k,1), projData(idxK==k,2), liColors{k}), hold on;
end
% plot results on original data
figure(4);
for k = 1:nSignals
plot(projData(idxK==k,1), liColors{k}), hold on;
end
Let me know if this helps.
I've found some methods to enlarge an image but there is no solution to shrink an image. I'm currently using the nearest neighbor method. How could I do this with bilinear interpolation without using the imresize function in MATLAB?
In your comments, you mentioned you wanted to resize an image using bilinear interpolation. Bear in mind that the bilinear interpolation algorithm is size independent. You can very well use the same algorithm for enlarging an image as well as shrinking an image. The right scale factors to sample the pixel locations are dependent on the output dimensions you specify. This doesn't change the core algorithm by the way.
Before I start with any code, I'm going to refer you to Richard Alan Peters' II digital image processing slides on interpolation, specifically slide #59. It has a great illustration as well as pseudocode on how to do bilinear interpolation that is MATLAB friendly. To be self-contained, I'm going to include his slide here so we can follow along and code it:
Please be advised that this only resamples the image. If you actually want to match MATLAB's output, you need to disable anti-aliasing.
MATLAB by default will perform anti-aliasing on the images to ensure the output looks visually pleasing. If you'd like to compare apples with apples, make sure you disable anti-aliasing when comparing between this implementation and MATLAB's imresize function.
Let's write a function that will do this for us. This function will take in an image (that is read in through imread) which can be either colour or grayscale, as well as an array of two elements - The image you want to resize and the output dimensions in a two-element array of the final resized image you want. The first element of this array will be the rows and the second element of this array will be the columns. We will simply go through this algorithm and calculate the output pixel colours / grayscale values using this pseudocode:
function [out] = bilinearInterpolation(im, out_dims)
%// Get some necessary variables first
in_rows = size(im,1);
in_cols = size(im,2);
out_rows = out_dims(1);
out_cols = out_dims(2);
%// Let S_R = R / R'
S_R = in_rows / out_rows;
%// Let S_C = C / C'
S_C = in_cols / out_cols;
%// Define grid of co-ordinates in our image
%// Generate (x,y) pairs for each point in our image
[cf, rf] = meshgrid(1 : out_cols, 1 : out_rows);
%// Let r_f = r'*S_R for r = 1,...,R'
%// Let c_f = c'*S_C for c = 1,...,C'
rf = rf * S_R;
cf = cf * S_C;
%// Let r = floor(rf) and c = floor(cf)
r = floor(rf);
c = floor(cf);
%// Any values out of range, cap
r(r < 1) = 1;
c(c < 1) = 1;
r(r > in_rows - 1) = in_rows - 1;
c(c > in_cols - 1) = in_cols - 1;
%// Let delta_R = rf - r and delta_C = cf - c
delta_R = rf - r;
delta_C = cf - c;
%// Final line of algorithm
%// Get column major indices for each point we wish
%// to access
in1_ind = sub2ind([in_rows, in_cols], r, c);
in2_ind = sub2ind([in_rows, in_cols], r+1,c);
in3_ind = sub2ind([in_rows, in_cols], r, c+1);
in4_ind = sub2ind([in_rows, in_cols], r+1, c+1);
%// Now interpolate
%// Go through each channel for the case of colour
%// Create output image that is the same class as input
out = zeros(out_rows, out_cols, size(im, 3));
out = cast(out, class(im));
for idx = 1 : size(im, 3)
chan = double(im(:,:,idx)); %// Get i'th channel
%// Interpolate the channel
tmp = chan(in1_ind).*(1 - delta_R).*(1 - delta_C) + ...
chan(in2_ind).*(delta_R).*(1 - delta_C) + ...
chan(in3_ind).*(1 - delta_R).*(delta_C) + ...
chan(in4_ind).*(delta_R).*(delta_C);
out(:,:,idx) = cast(tmp, class(im));
end
Take the above code, copy and paste it into a file called bilinearInterpolation.m and save it. Make sure you change your working directory where you've saved this file.
Except for sub2ind and perhaps meshgrid, everything seems to be in accordance with the algorithm. meshgrid is very easy to explain. All you're doing is specifying a 2D grid of (x,y) co-ordinates, where each location in your image has a pair of (x,y) or column and row co-ordinates. Creating a grid through meshgrid avoids any for loops as we will have generated all of the right pixel locations from the algorithm that we need before we continue.
How sub2ind works is that it takes in a row and column location in a 2D matrix (well... it can really be any amount of dimensions you want), and it outputs a single linear index. If you're not aware of how MATLAB indexes into matrices, there are two ways you can access an element in a matrix. You can use the row and column to get what you want, or you can use a column-major index. Take a look at this matrix example I have below:
A =
1 2 3 4 5
6 7 8 9 10
11 12 13 14 15
If we want to access the number 9, we can do A(2,4) which is what most people tend to default to. There is another way to access the number 9 using a single number, which is A(11)... now how is that the case? MATLAB lays out the memory of its matrices in column-major format. This means that if you were to take this matrix and stack all of its columns together in a single array, it would look like this:
A =
1
6
11
2
7
12
3
8
13
4
9
14
5
10
15
Now, if you want to access element number 9, you would need to access the 11th element of this array. Going back to the interpolation bit, sub2ind is crucial if you want to vectorize accessing the elements in your image to do the interpolation without doing any for loops. As such, if you look at the last line of the pseudocode, we want to access elements at r, c, r+1 and c+1. Note that all of these are 2D arrays, where each element in each of the matching locations in all of these arrays tell us the four pixels we need to sample from in order to produce the final output pixel. The output of sub2ind will also be 2D arrays of the same size as the output image. The key here is that each element of the 2D arrays of r, c, r+1, and c+1 will give us the column-major indices into the image that we want to access, and by throwing this as input into the image for indexing, we will exactly get the pixel locations that we want.
There are some important subtleties I'd like to add when implementing the algorithm:
You need to make sure that any indices to access the image when interpolating outside of the image are either set to 1 or the number of rows or columns to ensure you don't go out of bounds. Actually, if you extend to the right or below the image, you need to set this to one below the maximum as the interpolation requires that you are accessing pixels to one over to the right or below. This will make sure that you're still within bounds.
You also need to make sure that the output image is cast to the same class as the input image.
I ran through a for loop to interpolate each channel on its own. You could do this intelligently using bsxfun, but I decided to use a for loop for simplicity, and so that you are able to follow along with the algorithm.
As an example to show this works, let's use the onion.png image that is part of MATLAB's system path. The original dimensions of this image are 135 x 198. Let's interpolate this image by making it larger, going to 270 x 396 which is twice the size of the original image:
im = imread('onion.png');
out = bilinearInterpolation(im, [270 396]);
figure;
imshow(im);
figure;
imshow(out);
The above code will interpolate the image by increasing each dimension by twice as much, then show a figure with the original image and another figure with the scaled up image. This is what I get for both:
Similarly, let's shrink the image down by half as much:
im = imread('onion.png');
out = bilinearInterpolation(im, [68 99]);
figure;
imshow(im);
figure;
imshow(out);
Note that half of 135 is 67.5 for the rows, but I rounded up to 68. This is what I get:
One thing I've noticed in practice is that upsampling with bilinear has decent performance in comparison to other schemes like bicubic... or even Lanczos. However, when you're shrinking an image, because you're removing detail, nearest neighbour is very much sufficient. I find bilinear or bicubic to be overkill. I'm not sure about what your application is, but play around with the different interpolation algorithms and see what you like out of the results. Bicubic is another story, and I'll leave that to you as an exercise. Those slides I referred you to does have material on bicubic interpolation if you're interested.
Good luck!