this is my first time asking question in stackoverflow. I tried hard not to make any guideline violations, but let me know if there is any—please understand since it is my first time.
I have a code in Mathematica, and I think it is REALLY slow.
I have a stack of images—couple hundreds, and I was creating a simple algorithm in Mathematica to combine images into a single image by averaging pixel data across the entire stack.
For example, assume that I have a pixel at 520 x 23, and I want to get all the number(data) from every single images in the stack and average that table. I want to repeat this process for every single pixel in my image. Then, I will be able to reconstruct the data into an image using Image[]. For the testing, I used 1024 by 1024 image, but I want to use 8000 by 8000 eventually.
Here is my code:
idlist = Map[ImageData, ilist];
Table[Table[Map[Mean,Transpose[Table[idlist[[a,b,c]],{a,1,Length[idlist]}]]],{b,1,1024}], {c, 1, 1024}]
I found out that my nested Table was faster than using a single Table for some weird reasons.
1.How should I optimize this?
2.Is there a benefit using any parallel computing for this specific implementation?
I'm not sure if I'm following your coding, but to get the mean pixel value for a bunch of images, you could simply do the following (it's fast):
imDim = 100;
n = 80;(*number of images*)
(*generate random image list*)
ilist = Array[RandomImage[1, {imDim, imDim}, ColorSpace -> "RGB"] &, n];
(*take mean*)
Image#Mean[ImageData /# ilist]
For a large collection of random images it should converge to homogeneous gray (RGB[.5,.5,.5]). Let's test it:
imDim = 3;
n = 2000;(*number of images*)
(*generate random image list*)
ilist = Array[RandomImage[1, {imDim, imDim}, ColorSpace -> "RGB"] &, n];
(*take mean*)
Image#Mean[ImageData /# ilist]
Related
This is the dilenma I'm having. For my application, I need to match an image L1 with the matching one L2 in a set of images. L1 and L2 are the exact same image, except L1 is much smaller (It will need to be upscale?), and could be artifacted a little on the edges, but nevertheless, they are from the exact same source image. Color DOES matter, in that using color information will remove possible ambiguities between the current image and the one it is to be matched with. Using OpenCV (or perhaps there may be a better alternative?), what is the best way to find the matching image (L2).
To reiterate, the image to be matched with is not rotated or distorted in anyway, only resized.
I guess there would be a function which rates how close the image to be matched is to all of the images in the set provided. Then we choose the one with the highest rating as the match. I'm not sure as to how to compare the images though. Any help would be great. Thanks.
Go to github and check out opencv-master\samples\cpp\matcher_simple.cpp(or matching_to_many_images.cpp)
not only can it satisfy your need but it also works for images with perspective distortion (eg. rotation, affine transformation and illumination variation). simply put, it's very robust.
but SIFT and SURF are patented, you might not be able to use it for commercial applications, which sucks. but there are many alternatives, just google around!
OpenCV has a tutorial on similarity measurement for images.
You will need to upscale L1 before doing the comparison, or downscale L2. If you are comparing L2 against lots of images it probably makes more sense to scale L2 down (because you then don't have to call resize for every image being compared against, and there are fewer pixels to be compared).
e.g.
cv::Mat L1 = ...;
cv::Mat L2 = ...;
cv::Mat L2small;
cv::resize(L2, L2small, L1.size());
double pnsr = getPSNR(L1, L2small);
// where code for getPSNR() is in the tutorial
I think you may be able to use something similar to Bag-of-words model used to measure document similarity. Take a look at this: link
I'm reproducing the equation below:
G = X'*X
where X = [x1 x2 ... xn]
In your case, use the normalized histogram of the image as vector xi.
I think you wouldn't have to resize the images in this approach and it would be faster.
EDIT
I tried this in Matlab using some sample images provided in opencv samples:
im1 = imread('baboon.jpg');
im2 = imread('board.jpg');
im3 = imread('fruits.jpg');
im4 = imread('fruits - small.jpg'); % fruits.jpg scaled down 25% using mspaint
% using grayscale for simplicity
gr1 = rgb2gray(im1);
gr2 = rgb2gray(im2);
gr3 = rgb2gray(im3);
gr4 = rgb2gray(im4);
[cnt_baboon, x] = imhist(gr1);
[cnt_board, x] = imhist(gr2);
[cnt_fruits, x] = imhist(gr3);
[cnt_fruits_small, x] = imhist(gr4);
% X: not normalized
X = [cnt_baboon cnt_board cnt_fruits cnt_fruits_small];
H = X'*X;
N = sqrt(diag(H)*diag(H)');
% normalize. this would be faster
G = H./N
The resulting G matrix:
G =
1.0000 0.8460 0.7748 0.7729
0.8460 1.0000 0.8741 0.8686
0.7748 0.8741 1.0000 0.9947
0.7729 0.8686 0.9947 1.0000
You can see that G(3,4) (and G(4,3)) are very close to 1.
i think you are looking for histogram matching. There are inbuilt functions for histogram matching, for example bhattacharya distance etc, and they don't require both of your images to be of same size also.
Just check this link on opencv site,
link
UPDATE
Here is my code that is meant to add up the two matrices and using element by element addition and then divide by two.
function [ finish ] = stackAndMeanImage (initFrame, finalFrame)
cd 'C:\Users\Disc-1119\Desktop\Internships\Tracking\Octave\highway\highway (6-13-2014 11-13-41 AM)';
pkg load image;
i = initFrame;
f = finalFrame;
astr = num2str(i);
tmp = imread(astr, 'jpg');
d = f - i
for a = 1:d
a
astr = num2str(i + 1);
read_tmp = imread(astr, 'jpg');
read_tmp = rgb2gray(read_tmp);
tmp = tmp :+ read_tmp;
tmp = tmp / 2;
end
imwrite(tmp, 'meanimage.JPG');
finish = 'done';
end
Here are two example input images
http://imgur.com/5DR1ccS,AWBEI0d#1
And here is one output image
http://imgur.com/aX6b0kj
I am really confused as to what is happening. I have not implemented what the other answers have said yet though.
OLD
I am working on an image processing project where I am now manually choosing images that are 'empty' or only have the background, so that my algorithm can compute the differences and then do some more analysis, I have a simple piece of code that computes the mean of the two images, which I have converted to grayscale matrices, but this only works for two images, because when I find the mean of two, then take this mean and find the mean of this versus the next image, and do this repeatedly, I end up with a washed out white image that is absolutely useless. You can't even see anything.
I found that there is a function in Matlab called imFuse that is able to average images. I was wondering if anyone knew the process that imFuse uses to combine images, I am happy to implement this into Octave, or if anyone knew of or has already written a piece of code that achieves something similiar to this. Again, I am not asking for anyone to write code for me, just wondering what the process for this is and if there are already pre-existing functions out there, which I have not found after my research.
Thanks,
AeroVTP
You should not end up with a washed-out image. Instead, you should end up with an image, which is technically speaking temporally low-pass filtered. What this means is that half of the information content is form the last image, one quarter from the second last image, one eight from the third last image, etc.
Actually, the effect in a moving image is similar to a display with slow response time.
If you are ending up with a white image, you are doing something wrong. nkjt's guess of type challenges is a good one. Another possibility is that you have forgotten to divide by two after summing the two images.
One more thing... If you are doing linear operations (such as averaging) on images, your image intensity scale should be linear. If you just use the RGB values or some grayscale values simply calculated from them, you may get bitten by the nonlinearity of the image. This property is called the gamma correction. (Admittedly, most image processing programs just ignore the problem, as it is not always a big challenge.)
As your project calculates differences of images, you should take this into account. I suggest using linearised floating point values. Unfortunately, the linearisation depends on the source of your image data.
On the other hand, averaging often the most efficient way of reducing noise. So, there you are in the right track assuming the images are similar enough.
However, after having a look at your images, it seems that you may actually want to do something else than to average the image. If I understand your intention correctly, you would like to get rid of the cars in your road cam to give you just the carless background which you could then subtract from the image to get the cars.
If that is what you want to do, you should consider using a median filter instead of averaging. What this means is that you take for example 11 consecutive frames. Then for each pixel you have 11 different values. Now you order (sort) these values and take the middle (6th) one as the background pixel value.
If your road is empty most of the time (at least 6 frames of 11), then the 6th sample will represent the road regardless of the colour of the cars passing your camera.
If you have an empty road, the result from the median filtering is close to averaging. (Averaging is better with Gaussian white noise, but the difference is not very big.) But your averaging will be affected by white or black cars, whereas median filtering is not.
The problem with median filtering is that it is computationally intensive. I am very sorry I speak very broken and ancient Octave, so I cannot give you any useful code. In MatLab or PyLab you would stack, say, 11 images to a M x N x 11 array, and then use a single median command along the depth axis. (When I say intensive, I do not mean it couldn't be done in real time with your data. It can, but it is much more complicated than averaging.)
If you have really a lot of traffic, the road is visible behind the cars less than half of the time. Then the median trick will fail. You will need to take more samples and then find the most typical value, because it is likely to be the road (unless all cars have similar colours). There it will help a lot to use the colour image, as cars look more different from each other in RGB or HSV than in grayscale.
Unfortunately, if you need to resort to this type of processing, the path is slightly slippery and rocky. Average is very easy and fast, median is easy (but not that fast), but then things tend to get rather complicated.
Another BTW came into my mind. If you want to have a rolling average, there is a very simple and effective way to calculate it with an arbitrary length (arbitrary number of frames to average):
# N is the number of images to average
# P[i] are the input frames
# S is a sum accumulator (sum of N frames)
# calculate the sum of the first N frames
S <- 0
I <- 0
while I < N
S <- S + P[I]
I <- I + 1
# save_img() saves an averaged image
while there are images to process
save_img(S / N)
S <- -P[I-N] + S + P[I]
I <- I + 1
Of course, you'll probably want to use for-loops, and += and -= operators, but still the idea is there. For each frame you only need one subtraction, one addition, and one division by a constant (which can be modified into a multiplication or even a bitwise shift in some cases if you are in a hurry).
I may have misunderstood your problem but I think what you're trying to do is the following. Basically, read all images into a matrix and then use mean(). This is providing that you are able to put them all in memory.
function [finish] = stackAndMeanImage (ini_frame, final_frame)
pkg load image;
dir_path = 'C:\Users\Disc-1119\Desktop\Internships\Tracking\Octave\highway\highway (6-13-2014 11-13-41 AM)';
imgs = cell (1, 1, d);
## read all images into a cell array
current_frame = ini_frame;
for n = 1:(final_frame - ini_frame)
fname = fullfile (dir_path, sprintf ("%i", current_frame++));
imgs{n} = rgb2gray (imread (fname, "jpg"));
endfor
## create 3D matrix out of all frames and calculate mean across 3rd dimension
imgs = cell2mat (imgs);
avg = mean (imgs, 3);
## mean returns double precision so we cast it back to uint8 after
## rescaling it to range [0 1]. This assumes that images were all
## originally uint8, but since they are jpgs, that's a safe assumption
avg = im2uint8 (avg ./255);
imwrite (avg, fullfile (dir_path, "meanimage.jpg"));
finish = "done";
endfunction
My short question
How to detect the black dots in the following images? (I paste only one test image to make the question look compact. More images can be found →here←).
My long question
As shown above, the background color is roughly blue, and the dots color is "black". If pick one black pixel and measure its color in RGB, the value can be (0, 44, 65) or (14, 69, 89).... Therefore, we cannot set a range to tell the pixel is part of the black dot or the background.
I test 10 images of different colors, but I hope I can find a method to detect the black dots from more complicated background which may be made up of three or more colors, as long as human eyes can identify the black dots easily. Some extremely small or blur dots can be omitted.
Previous work
Last month, I have asked a similar question at stackoverflow, but have not got a perfect solution, some excellent answers though. Find more details about my work if you are interested.
Here are the methods I have tried:
Converting to grayscale or the brightness of image. The difficulty is that I can not find an adaptive threshold to do binarization. Obviously, turning a color image to grayscale or using the brightness (HSV) will lose much useful information. Otsu algorithm which calculates adaptive threshold can not work either.
Calculating RGB histogram. In my last question, natan's method is to estimate the black color by histogram. It is time-saving, but the adaptive threshold is also a problem.
Clustering. I have tried k-means clustering and found it quite effective for the background that only has one color. The shortage (see my own answer) is I need to set the number of clustering center in advance but I don't know how the background will be. What's more, it is too slow! My application is for real time capturing on iPhone and now it can process 7~8 frames per second using k-means (20 FPS is good I think).
Summary
I think not only similar colors but also adjacent pixels should be "clustered" or "merged" in order to extract the black dots. Please guide me a proper way to solve my problem. Any advice or algorithm will be appreciated. There is no free lunch but I hope a better trade-off between cost and accuracy.
I was able to get some pretty nice first pass results by converting to HSV color space with rgb2hsv, then using the Image Processing Toolbox functions imopen and imregionalmin on the value channel:
rgb = imread('6abIc.jpg');
hsv = rgb2hsv(rgb);
openimg = imopen(hsv(:, :, 3), strel('disk', 11));
mask = imregionalmin(openimg);
imshow(rgb);
hold on;
[r, c] = find(mask);
plot(c, r, 'r.');
And the resulting images (for the image in the question and one chosen from your link):
You can see a few false positives and missed dots, as well as some dots that are labeled with multiple points, but a few refinements (such as modifying the structure element used in the opening step) could clean these up some.
I was curios to test with my old 2d peak finder code on the images without any threshold or any color considerations, really crude don't you think?
im0=imread('Snap10.jpg');
im=(abs(255-im0));
d=rgb2gray(im);
filter=fspecial('gaussian',16,3.5);
p=FastPeakFind(d,0,filter);
imagesc(im0); hold on
plot(p(1:2:end),p(2:2:end),'r.')
The code I'm using is a simple 2D local maxima finder, there are some false positives, but all in all this captures most of the points with no duplication. The filter I was using was a 2d gaussian of width and std similar to a typical blob (the best would have been to get a matched filter for your problem).
A more sophisticated version that does treat the colors (rgb2hsv?) could improve this further...
Here is an extraodinarily simplified version, that can be extended to be full RGB, and it also does not use the image procesing library. Basically you can do 2-D convolution with a filter image (which is an example of the dot you are looking for), and from the points where the convolution returns the highest values, are the best matches for the dots. You can then of course threshold that. Here is a simple binary image example of just that.
%creating a dummy image with a bunch of small white crosses
im = zeros(100,100);
numPoints = 10;
% randomly chose the location to put those crosses
points = randperm(numel(im));
% keep only certain number of points
points = points(1:numPoints);
% get the row and columns (x,y)
[xVals,yVals] = ind2sub(size(im),points);
for ii = 1:numel(points)
x = xVals(ii);
y = yVals(ii);
try
% create the crosses, try statement is here to prevent index out of bounds
% not necessarily the best practice but whatever, it is only for demonstration
im(x,y) = 1;
im(x+1,y) = 1;
im(x-1,y) = 1;
im(x,y+1) = 1;
im(x,y-1) = 1;
catch err
end
end
% display the randomly generated image
imshow(im)
% create a simple cross filter
filter = [0,1,0;1,1,1;0,1,0];
figure; imshow(filter)
% perform convolution of the random image with the cross template
result = conv2(im,filter,'same');
% get the number of white pixels in filter
filSum = sum(filter(:));
% look for all points in the convolution results that matched identically to the filter
matches = find(result == filSum);
%validate all points found
sort(matches(:)) == sort(points(:))
% get x and y coordinate matches
[xMatch,yMatch] = ind2sub(size(im),matches);
I would highly suggest looking at the conv2 documentation on MATLAB's website.
I am searching for a way to find the location of a button on my screen using matlab. Now i have created the following code for this:
With this code i make a screenshot of my computer, load it in matlab and make it double.
Then do the same with a target image, the image that needs to be found on my screen, and then try to match this to find matrix to the total matrix and display the coordinates of the top left corner of this image.
The problem is that with this method and a screen resolution of 1920*1080 it takes about 15 min to go trough this full proces.
So my question is, is there an easier/faster way to find the coördinates(or the center of) an image on my screen, or within another image? So far i have found none, even when using the image processing toolbox.
clear all
close all
robo = java.awt.Robot;
t = java.awt.Toolkit.getDefaultToolkit();
rectangle = java.awt.Rectangle(t.getScreenSize());
image = robo.createScreenCapture(rectangle);
filehandle = java.io.File('screencapture.png');
javax.imageio.ImageIO.write(image,'png',filehandle);
scrimg=im2double(imread('screencapture.png')); % screenshot
sfimg=im2double(imread('searchfor.png')); % image to look for
[mA,nA,zA] = size(sfimg);
[mB,nB,zB] = size(scrimg);
F = zeros((mB-mA+1)*(nB-nA+1),3);
k = 0;
for p = 1:mB-mA+1
for q = 1:nB-nA+1
for r = 1:zB-zA+1
iets=[p q r];
disp(iets)
if all(all(sfimg==scrimg(p:p+mA-1,q:q+nA-1,r:r+zA-1)))
k = k + 1;
F(k,:) = [p,q,r];
end
end
end
end
F(k+1:end,:) = [];
Check out the following links:
http://repository.asu.edu/items/15110
http://repository.asu.edu/attachments/94025/content//tmp/package-fN0ldX/Velocimeter.pdf
http://repository.asu.edu/attachments/94026/content//tmp/package-fN0ldX/ThesisPresentationr2.pdf
I did 2d tracking from videos in matlab against some very particular geometries. It was radically faster than 15 minutes. The part you are interested in is "measure on image".
The basic idea is that you can use fourier-domain 2d operations, a "max" and some local polynomial fitting to get reliable sub-pixel alignment.
Best of luck
A quicker way is to instead of searching for the entire image is to take a smaller subset of pixels say:
n=4;
SegmentOfImage=sfimg(1:(mA/n),1:(nA/n));
and search for that segment. Obviously as n increases the algorithm would be quicker.
If you want to then make sure that it was actually that segment you can make another comparison on the whole image but now you know the index to start with.
Note:
1) This algorithm can actually take longer if there are a lot of objects with identical pixels to
SegmentOfImage.
2) Also, I believe that this code in general would work only if the matrices are exactly the same. For example if the size of sfimg is not the same as scrimg it would fail.
Hope this helps
I have an image that I want to divide in three parts and find the centroid of the parts separately and display them on original image, I used blkproc for dividing the image in [1 3] grids, but can't display the centroids. Here is the code I wrote,
i=imread('F:\line3.jpg');
i2=rgb2gray(i);
bw=im2bw(i2);
imshow(bw)
fun=#(x) regionprops(x,'centroid');
b=blkproc(bw,[1 3],fun);
But I can't get to display the centroids, as well as get their values. Any help will be much appreciated.
You can just use the plot command to plot over the top of the image.
Whatever you [X,Y] centroid coordinates are, say cx(1:3) and cy(1:3)
numCentroids is the number of centroids you are plotting.
hold on;
for ii = 1:length(numCentroids)
plot(cx(ii),cy(ii),'Marker','s','MarkerSize',10,'MarkerFaceColor','r','MarkerEdgeColor','k')
end
If you wanted to write more elegant code, you could run the plot command once across all your centroids and then make the line style type invisible. The answer I supplied should work though.
Here's an example image with made up centroids.
Strong recommendation - use blockproc instead of blkproc. It is better designed and easier to use.
Now, first of all, the second input to blockproc is the blocksize and not the grid size. So if you want to divide your image into [1 3] grid, which I understand as a single row of three blocks, then you should set your blocksize as:
blocksize = [size(i,1) ceil(size(i,2)/3)];
The second thing is to turn off the 'TrimBorder' parameter in blockproc. The code would look something like:
fun=#(x) regionprops(x,'centroid');
blocksize = [size(i,1) ceil(size(i,2)/3)];
b=blockproc(bw,blocksize,fun,'TrimBorder',false);
One minor thing - I would recommend not using the variable name 'i'. By default it represents the imaginary number i = sqrt(-1); in Matlab.