I am searching for a way to find the location of a button on my screen using matlab. Now i have created the following code for this:
With this code i make a screenshot of my computer, load it in matlab and make it double.
Then do the same with a target image, the image that needs to be found on my screen, and then try to match this to find matrix to the total matrix and display the coordinates of the top left corner of this image.
The problem is that with this method and a screen resolution of 1920*1080 it takes about 15 min to go trough this full proces.
So my question is, is there an easier/faster way to find the coördinates(or the center of) an image on my screen, or within another image? So far i have found none, even when using the image processing toolbox.
clear all
close all
robo = java.awt.Robot;
t = java.awt.Toolkit.getDefaultToolkit();
rectangle = java.awt.Rectangle(t.getScreenSize());
image = robo.createScreenCapture(rectangle);
filehandle = java.io.File('screencapture.png');
javax.imageio.ImageIO.write(image,'png',filehandle);
scrimg=im2double(imread('screencapture.png')); % screenshot
sfimg=im2double(imread('searchfor.png')); % image to look for
[mA,nA,zA] = size(sfimg);
[mB,nB,zB] = size(scrimg);
F = zeros((mB-mA+1)*(nB-nA+1),3);
k = 0;
for p = 1:mB-mA+1
for q = 1:nB-nA+1
for r = 1:zB-zA+1
iets=[p q r];
disp(iets)
if all(all(sfimg==scrimg(p:p+mA-1,q:q+nA-1,r:r+zA-1)))
k = k + 1;
F(k,:) = [p,q,r];
end
end
end
end
F(k+1:end,:) = [];
Check out the following links:
http://repository.asu.edu/items/15110
http://repository.asu.edu/attachments/94025/content//tmp/package-fN0ldX/Velocimeter.pdf
http://repository.asu.edu/attachments/94026/content//tmp/package-fN0ldX/ThesisPresentationr2.pdf
I did 2d tracking from videos in matlab against some very particular geometries. It was radically faster than 15 minutes. The part you are interested in is "measure on image".
The basic idea is that you can use fourier-domain 2d operations, a "max" and some local polynomial fitting to get reliable sub-pixel alignment.
Best of luck
A quicker way is to instead of searching for the entire image is to take a smaller subset of pixels say:
n=4;
SegmentOfImage=sfimg(1:(mA/n),1:(nA/n));
and search for that segment. Obviously as n increases the algorithm would be quicker.
If you want to then make sure that it was actually that segment you can make another comparison on the whole image but now you know the index to start with.
Note:
1) This algorithm can actually take longer if there are a lot of objects with identical pixels to
SegmentOfImage.
2) Also, I believe that this code in general would work only if the matrices are exactly the same. For example if the size of sfimg is not the same as scrimg it would fail.
Hope this helps
Related
For my master thesis I have to examine several similarly structured images with Matlab. I have already coded the actual evaluation. My problem is that the images have a black frame of different thickness. This will cause the evaluation to be falsified. So far I cut the border by hand. So I wanted to ask if someone could help me cut this frame using a Matlab code.
Below is an example picture,
I tried to solve this problem with the edge-plug in (https://de.mathworks.com/help/images/ref/edge.html) but it didn´t worked.
Maybe you can give me some advice
I wouldn't be using the edge detection as you have linked to in your question.
Rather, I would focus on reducing the dimensions of the image and analyzing the "reduced image". In the following code you can see an example of reducing the image into two representative lines, and analyzing the line to detect the specimen.
% Load the image.
Img = imread('pfi36.png');
% Level to pass for the "edge" detection.
lvl = 50;
% Add a buffer around the edges.
buf = 10;
% Reduce the Image to 1D along the x and y driections respectively.
lrData = mean(Img,1);
tdData = mean(Img,2);
% Find the indices of the edges.
indL = find(lrData>lvl,1,'first');
indR = find(lrData>lvl,1,'last');
indB = find(tdData>lvl,1,'last');
% Show the Cropped image.
newImg = Img(1:(indB+buf),(indL-buf):(indR+buf));
imshow(newImg);
I have a segmented image as shown here
i want to fit a curve along the top pixels of the segmented image(show as red curve) and i want to find the top point along the curve show in blue. I have already worked on basic idea like traversing through the top to bottom and collecting the top point along each column. i want to know is there any easy solution for this problem like directly taking out the boundary pixels and find the top point.I am using MATLAB for this problem
%download the image
img = logical(imread('http://i.stack.imgur.com/or2iX.png'));
%for some reason it appeared RGB with big solid borders.
%to monochrome
img = img(:,:,1);
%remove borders
img = img(~all(img,2), ~all(img,1));
%split into columns
cimg = num2cell(img,1);
%find first nonzero element per column
ridx = cellfun(#(x) find(x,1,'first'), cimg);
figure, imshow(img)
hold on
%image dim1 is Y, dim2 is X
plot(1:size(img,2),ridx-1,'r','linewidth',2)
%find top point
[yval, xval] = min(ridx);
If you want a smoother curve, try polyfit/polyval
#EDIT
If we want the line to have break at gaps between connected components, we should change the code to something like
bord_idx = sub2ind(size(img), ridx, 1:size(img,2));
regs=regionprops(bwlabel(img),'pixelidxlist');
regs_idx = struct2cell(regs);
split_step = cellfun(#(x) sum(ismember(bord_idx,x)), regs_idx);
split_step = split_step(split_step>0);
split_yvals = mat2cell(ridx',split_val);
split_xvals = mat2cell([1:size(img,2)]',split_val);
figure, imshow(img)
hold on
for k = 1:length(split_step),
plot(split_xvals{k}, split_yvals{k}, 'r', 'linewidth', 2),
end
However, the result is not ideal if one region is positioned over the other. If the "shadowed" points are needed, you should try bwtraceboundary or convexhull and find where the border turns down
As far as "simplest matlab solution" by which I think you mean built in matlab functions: imclose()->edge()->bwboundaries()->findpeaks()'on each boundary'->'filter results based on width and magnitude of peaks'. *you will need to tune all the parameters in these functions, I am just listing what would get you there if appropriately applied.
As far as processing speed is concerned, I think I would have done exactly what you did, basically collecting the top edge from a top down column search and then looking for the point of highest inflection. As soon as you start doing processing of any type, you start doing several operations per pixel which will quickly become more expensive than your initial search (just requires that your image and target are simple enough)
That being said, here are some ideas that may help:
1:If you run a sufficiently heavy closing (dilate->erode), that should fill in all that garbage at the bottom.
2: If you know that your point of interest is not at left or right of picture (boundaries), you could take the right and left edge points and calculate a slope to be applied as an offset to flatten the whole image.
3: If your image always has the large dark linear region below the peak as seen here, you could locate those edges with houghlines looking for verticals and then search only the columns between them.
4: If speed is a concern, you could do a more sophisticated search pattern than left to right, as your peak has a pretty good distribution around it which could help with faster localization of maxima.
UPDATE
Here is my code that is meant to add up the two matrices and using element by element addition and then divide by two.
function [ finish ] = stackAndMeanImage (initFrame, finalFrame)
cd 'C:\Users\Disc-1119\Desktop\Internships\Tracking\Octave\highway\highway (6-13-2014 11-13-41 AM)';
pkg load image;
i = initFrame;
f = finalFrame;
astr = num2str(i);
tmp = imread(astr, 'jpg');
d = f - i
for a = 1:d
a
astr = num2str(i + 1);
read_tmp = imread(astr, 'jpg');
read_tmp = rgb2gray(read_tmp);
tmp = tmp :+ read_tmp;
tmp = tmp / 2;
end
imwrite(tmp, 'meanimage.JPG');
finish = 'done';
end
Here are two example input images
http://imgur.com/5DR1ccS,AWBEI0d#1
And here is one output image
http://imgur.com/aX6b0kj
I am really confused as to what is happening. I have not implemented what the other answers have said yet though.
OLD
I am working on an image processing project where I am now manually choosing images that are 'empty' or only have the background, so that my algorithm can compute the differences and then do some more analysis, I have a simple piece of code that computes the mean of the two images, which I have converted to grayscale matrices, but this only works for two images, because when I find the mean of two, then take this mean and find the mean of this versus the next image, and do this repeatedly, I end up with a washed out white image that is absolutely useless. You can't even see anything.
I found that there is a function in Matlab called imFuse that is able to average images. I was wondering if anyone knew the process that imFuse uses to combine images, I am happy to implement this into Octave, or if anyone knew of or has already written a piece of code that achieves something similiar to this. Again, I am not asking for anyone to write code for me, just wondering what the process for this is and if there are already pre-existing functions out there, which I have not found after my research.
Thanks,
AeroVTP
You should not end up with a washed-out image. Instead, you should end up with an image, which is technically speaking temporally low-pass filtered. What this means is that half of the information content is form the last image, one quarter from the second last image, one eight from the third last image, etc.
Actually, the effect in a moving image is similar to a display with slow response time.
If you are ending up with a white image, you are doing something wrong. nkjt's guess of type challenges is a good one. Another possibility is that you have forgotten to divide by two after summing the two images.
One more thing... If you are doing linear operations (such as averaging) on images, your image intensity scale should be linear. If you just use the RGB values or some grayscale values simply calculated from them, you may get bitten by the nonlinearity of the image. This property is called the gamma correction. (Admittedly, most image processing programs just ignore the problem, as it is not always a big challenge.)
As your project calculates differences of images, you should take this into account. I suggest using linearised floating point values. Unfortunately, the linearisation depends on the source of your image data.
On the other hand, averaging often the most efficient way of reducing noise. So, there you are in the right track assuming the images are similar enough.
However, after having a look at your images, it seems that you may actually want to do something else than to average the image. If I understand your intention correctly, you would like to get rid of the cars in your road cam to give you just the carless background which you could then subtract from the image to get the cars.
If that is what you want to do, you should consider using a median filter instead of averaging. What this means is that you take for example 11 consecutive frames. Then for each pixel you have 11 different values. Now you order (sort) these values and take the middle (6th) one as the background pixel value.
If your road is empty most of the time (at least 6 frames of 11), then the 6th sample will represent the road regardless of the colour of the cars passing your camera.
If you have an empty road, the result from the median filtering is close to averaging. (Averaging is better with Gaussian white noise, but the difference is not very big.) But your averaging will be affected by white or black cars, whereas median filtering is not.
The problem with median filtering is that it is computationally intensive. I am very sorry I speak very broken and ancient Octave, so I cannot give you any useful code. In MatLab or PyLab you would stack, say, 11 images to a M x N x 11 array, and then use a single median command along the depth axis. (When I say intensive, I do not mean it couldn't be done in real time with your data. It can, but it is much more complicated than averaging.)
If you have really a lot of traffic, the road is visible behind the cars less than half of the time. Then the median trick will fail. You will need to take more samples and then find the most typical value, because it is likely to be the road (unless all cars have similar colours). There it will help a lot to use the colour image, as cars look more different from each other in RGB or HSV than in grayscale.
Unfortunately, if you need to resort to this type of processing, the path is slightly slippery and rocky. Average is very easy and fast, median is easy (but not that fast), but then things tend to get rather complicated.
Another BTW came into my mind. If you want to have a rolling average, there is a very simple and effective way to calculate it with an arbitrary length (arbitrary number of frames to average):
# N is the number of images to average
# P[i] are the input frames
# S is a sum accumulator (sum of N frames)
# calculate the sum of the first N frames
S <- 0
I <- 0
while I < N
S <- S + P[I]
I <- I + 1
# save_img() saves an averaged image
while there are images to process
save_img(S / N)
S <- -P[I-N] + S + P[I]
I <- I + 1
Of course, you'll probably want to use for-loops, and += and -= operators, but still the idea is there. For each frame you only need one subtraction, one addition, and one division by a constant (which can be modified into a multiplication or even a bitwise shift in some cases if you are in a hurry).
I may have misunderstood your problem but I think what you're trying to do is the following. Basically, read all images into a matrix and then use mean(). This is providing that you are able to put them all in memory.
function [finish] = stackAndMeanImage (ini_frame, final_frame)
pkg load image;
dir_path = 'C:\Users\Disc-1119\Desktop\Internships\Tracking\Octave\highway\highway (6-13-2014 11-13-41 AM)';
imgs = cell (1, 1, d);
## read all images into a cell array
current_frame = ini_frame;
for n = 1:(final_frame - ini_frame)
fname = fullfile (dir_path, sprintf ("%i", current_frame++));
imgs{n} = rgb2gray (imread (fname, "jpg"));
endfor
## create 3D matrix out of all frames and calculate mean across 3rd dimension
imgs = cell2mat (imgs);
avg = mean (imgs, 3);
## mean returns double precision so we cast it back to uint8 after
## rescaling it to range [0 1]. This assumes that images were all
## originally uint8, but since they are jpgs, that's a safe assumption
avg = im2uint8 (avg ./255);
imwrite (avg, fullfile (dir_path, "meanimage.jpg"));
finish = "done";
endfunction
I finally solved my problem here with lennon310.
I have a picture of thousands of thin peaks in Time-Frequency picture.
I cannot see all the same time in one picture.
Depending on the physical width of my time window, some windows appear and some come visible.
Pictures of my data which I plot by imagesc
All pictures are from the same data points T, F, B.
How can you plot all peaks at once in a picture in Matlab?
You need to resize the image using resampling to prevent the aliasing effect (that craigim described as unavoidable).
For example, the MATLAB imresize function can perform anti-aliasing. Don't use the "nearest" resize method, that's what you have now.
Extension to #BenVoigt's answer
My try
B = abs(B);
F1 = filter2(B,T); % you need a different filter kernel because resolution is lower
T = filter2(B,F);
F = F1;
image1 = imagesc(B);
display1 = imresize(image1, [600 600], 'bilinear');
imshow(T*t, F*fs, display1);
where are some problems.
I get again picture where the resolution is not enough
2nd Extension to BenVoigt's answer
My suggestion for one kernel filter is with convolution of relative random error
data(find(data ~= 0)) = sin(pi .* data(find(data ~= 0))) ./ (pi*data(find(data ~= 0)));
data(find(data == 0)) = 1; % removing lastly the discontinuity
data1 = data + 0.0000001 * mean(abs(data(:))) * randn(size(data));
data = conv(data, data1);
Is this what BenVoigt means by the kernel filter for distribution?
This gives results like
where the resolution is still a problem.
The central peaks tend to multiply easily if I resize the window.
I had old code active in the above picture but it did not change the result.
The above code is still not enough for the kernel filter of the display.
Probably, some filter functions has to be applied to the time and frequency axis separately still, something like:
F1 = filter2(B,T); % you need a different kernel filter because resolution is lower
T = filter2(B,F);
F = F1;
These filters mess up the values on the both axis.
I need to understand them better to fix this.
But first to understand if they are the right way to go.
The figure has be resized still.
The size of the data was 5001x1 double and those of F and T 13635x1 double.
So I think I should resize lastly after setting axis, labels and title by
imresize(image, [13635 13635], 'bilinear');
since the distirbution is bilinear.
3rd Extension to BenVoigt's answer
I plot the picture now by
imagesc([0 numel(T)], [0 numel(F)], B);
I have a big aliasing problem in my pictures.
Probably, something like this should be to manipulate the Time-Frequency Representation
T = filter2(B,t); % you need a different filter kernel because resolution is lower
F = filter2(B,fs);
4th extension to BenVoigt's answer and comment
I remove the filters and addition of random relative errors.
I set the size of T,F,B, and run
imagesc([0 numel(T)], [0 numel(F)], B, [0 numel(B)])
I get still with significant aliasing but different picture
This isn't being flippant, but I think the only way is to get a wider monitor with a higher pixel density. MATLAB and your video card can only show so many pixels on the screen, and must decide which to show and which to leave out. The data is still there, it just isn't getting displayed. Since you have a lot of very narrow lines, some of them are going to get skipped when decisions are made as to which pixels to light up. Changing the time window size changes which lines get aliased away and which ones get lucky enough to show up on the screen.
My suggestions are, in no particular order: convolute your lines with a Gaussian along the time axis to broaden them, thus increasing the likelihood that part of the peak will appear on the screen. Print them out on a 600 dpi printer and see if they appear. Make several plots, each zooming in on a separate time window.
3rd Extension to BenVoigt's answer
My attempt to tell imagesc how big to make the image, instead of letting to use the monitor size, so it won't away data.
imagesc(T*t, F*fs, B, [50 50]);
% imagesc(T*t, F*fs, B');
% This must be last
imresize(image, 'bilinear', [64 30],);
I am not sure where I should specify the amount of pixels in x-axis and y-axis, either in the command imagesc or imresize.
What is the correct way of resizing the image here?
I am trying to generate the following "effect" from a basic shape in MATLAB:
But I don't even know how this process is called. Let's say I have an image containing the brown shape, what I want is generate the contours outside of it, that get smoother as they get bigger.
Is there either a name for this effect, a function to do this in MATLAB or an algorithm that does it from scratch?
thanks
I think you are looking for bwdist.
The image you are displaying looks like the positive part of a distance function from the boundary of your shape. You can perform this easily in Matlab using the examples on the aforementioned manual page.
Try this:
I = imread('brown_image.png');
I_bw = (rgb2gray(I) > 0); % or whatever, just so I_bw is 1 in the 'brown' region
r = 10;
se1 = strel('disk', r);
se2 = strel('disk', r-1);
imshow(imdilate(I_bw, se1) - imdilate(I_bw, se2))
Requires image processing toolbox, but the basic idea is to dilate the image twice with dilation elements that differ by 1 (or however thick you want the contours to be) and subtract the result of the smaller one from the bigger one. You could then color them however you want.