I have an image which has a rectangle drawn in it. The rectangle can be of any design but the background isn't a single color. It's a photo taken from a phone camera like this one.
I want to crop the to inner picture (scenary) in the image.
How can I do this in MATLAB?
I tried this code
img = im2double(imread('https://i.stack.imgur.com/iS2Ht.jpg'));
BW = im2bw(img);
dim = size(BW)
col = round(dim(2)/2)-90;
row = min(find(BW(:,col)))
boundary = bwtraceboundary(BW,[row, col],'N');
r = [min(boundary) , max(boundary)];
img_cropped = img(r(1) : r(3) , r(2) : r(4) , :);
imshow(img_cropped);
but it works only for one image, this one
and not the one above or this one
I need to find a code which works for any image with a specific rectangle design.Any help will be aprreciated.Thank you
The processing below considers the following:
backgroung is monochrome, possible with gradient
a border is monochrome(gradient), is easily distinguishable from background, not barocco/rococco style
a picture is kind of real world picture with lots of details, not Malevich's Black Square
So we first search the picture and flatten background by entropy filtering
img=imread('http://i.stack.imgur.com/KMBRg.jpg');
%dimensions for neighbourhood are just a guess
cross_nhood = false(11,11); cross_nhood(:,6)=1;cross_nhood(6,:)=1;
img_ent = entropyfilt(img./255,repmat(cross_nhood,[1 1 3]));
img_ent_gray = rgb2gray(img_ent);
Then we find corners using Harris detector and choose 4 point: two leftmost and two rightmost, and crop the image thus removing background (with precision up to inclination). I use r2011a, you may have a bit different functions, refer to MATLAB help
harris_pts = corner(img_ent_gray);
corn_pts = sortrows(harris_pts,1);
corn_pts = [corn_pts(1:2,:);...
corn_pts(size(corn_pts,1)-1:size(corn_pts,1),:)];
crop_img=img(min(corn_pts(:,2)):max(corn_pts(:,2)),...
min(corn_pts(:,1)):max(corn_pts(:,1)),:);
corn_pts(:,1)=corn_pts(:,1) - min(corn_pts(:,1));
corn_pts(:,2)=corn_pts(:,2) - min(corn_pts(:,2));
corn_pts = corn_pts + 1;
An here is a problem: the lines between the corner points are inclined at a bit different angle. It can be both problem of corner detection and image capture (objective distortion and/or a bit wrong acquisition angle). There is no straightforward, always right solution. I'd better choose the biggest inclination (it will crop the picture a little) and start processing the image line by line (to split image use Bresenham algorithm) till any or most, you to choose, pixels belongs to the picture, not the inner border. The distinguishable feature can be local entropy, std of colour values, specific colour threshold, different methods of statistics, whatever.
Another approach is to do colour segmentation, I like most Gram-Shmidt orthogonalization or a*-b* color segmentation. However, you'll get all the same problems if image is skewed and part of a picture matches the colour of a border (see last picture, bottom left corner).
Related
I have a segmented image as shown here
i want to fit a curve along the top pixels of the segmented image(show as red curve) and i want to find the top point along the curve show in blue. I have already worked on basic idea like traversing through the top to bottom and collecting the top point along each column. i want to know is there any easy solution for this problem like directly taking out the boundary pixels and find the top point.I am using MATLAB for this problem
%download the image
img = logical(imread('http://i.stack.imgur.com/or2iX.png'));
%for some reason it appeared RGB with big solid borders.
%to monochrome
img = img(:,:,1);
%remove borders
img = img(~all(img,2), ~all(img,1));
%split into columns
cimg = num2cell(img,1);
%find first nonzero element per column
ridx = cellfun(#(x) find(x,1,'first'), cimg);
figure, imshow(img)
hold on
%image dim1 is Y, dim2 is X
plot(1:size(img,2),ridx-1,'r','linewidth',2)
%find top point
[yval, xval] = min(ridx);
If you want a smoother curve, try polyfit/polyval
#EDIT
If we want the line to have break at gaps between connected components, we should change the code to something like
bord_idx = sub2ind(size(img), ridx, 1:size(img,2));
regs=regionprops(bwlabel(img),'pixelidxlist');
regs_idx = struct2cell(regs);
split_step = cellfun(#(x) sum(ismember(bord_idx,x)), regs_idx);
split_step = split_step(split_step>0);
split_yvals = mat2cell(ridx',split_val);
split_xvals = mat2cell([1:size(img,2)]',split_val);
figure, imshow(img)
hold on
for k = 1:length(split_step),
plot(split_xvals{k}, split_yvals{k}, 'r', 'linewidth', 2),
end
However, the result is not ideal if one region is positioned over the other. If the "shadowed" points are needed, you should try bwtraceboundary or convexhull and find where the border turns down
As far as "simplest matlab solution" by which I think you mean built in matlab functions: imclose()->edge()->bwboundaries()->findpeaks()'on each boundary'->'filter results based on width and magnitude of peaks'. *you will need to tune all the parameters in these functions, I am just listing what would get you there if appropriately applied.
As far as processing speed is concerned, I think I would have done exactly what you did, basically collecting the top edge from a top down column search and then looking for the point of highest inflection. As soon as you start doing processing of any type, you start doing several operations per pixel which will quickly become more expensive than your initial search (just requires that your image and target are simple enough)
That being said, here are some ideas that may help:
1:If you run a sufficiently heavy closing (dilate->erode), that should fill in all that garbage at the bottom.
2: If you know that your point of interest is not at left or right of picture (boundaries), you could take the right and left edge points and calculate a slope to be applied as an offset to flatten the whole image.
3: If your image always has the large dark linear region below the peak as seen here, you could locate those edges with houghlines looking for verticals and then search only the columns between them.
4: If speed is a concern, you could do a more sophisticated search pattern than left to right, as your peak has a pretty good distribution around it which could help with faster localization of maxima.
In the grayscale image shown below, how can I accurately detect the white region having sharp boundary (marked with red color)?
In this particular image, a simple thresholding might work, however, I have several images in which there are similar areas around corner of images which I want to ignore.
Also, there might be more than one regions of interest, both having different intensities. One can be as bright as it is in the example image, other can be of medium intensity.
However, the only difference between the interested and non-interested areas is as follows:
The interest areas have sharp well defined boundaries.
Non-interested areas don't have sharp boundaries. They tend to gradually merge with neighbourhood areas.
Image without mark for testing:
When you say sharp boundaries, you have to think gradient. The sharper the boundaries, the bigger the gradient. Therefore apply a gradient and you will see that it will be stronger around the shapes you want to segment.
But in your case, you can also observe that the area you want to segment is also the brightest. So I would also try a noise reduction (median filter) plus a convolution filter (simple average) in order to homogenize the different zones, then thresholding by keeping only the brightest/right peak.
im = imread('o2XfN.jpg');
figure
imshow(im)
smooth = imgaussfilt(im,.8); %"blur" the image to take out noisey pixels
big = double(smooth); % some functions don't work with UINT8, I didn't check for these
maxiRow = quantile(big,.99); % .99 qualtile... think quartile from stats
maxiCol = quantile(maxiRow,.98); % again for the column
pixels = find(big>=maxiCol); % which pixels have the highest values
logicMat = false(size(big)); %initalize a logic matrix of zeros
logicMat(pixels) = 1; %set the pixels that passed to logic pass
figure
imshow(logicMat)
It is not extremely clear what you want to do with the regions that you are finding. Also, a few more sample images would be helpful to debug a code. What I posted above may work for that one image, but it is unlikely that it will work for every image that you are processing.
My short question
How to detect the black dots in the following images? (I paste only one test image to make the question look compact. More images can be found →here←).
My long question
As shown above, the background color is roughly blue, and the dots color is "black". If pick one black pixel and measure its color in RGB, the value can be (0, 44, 65) or (14, 69, 89).... Therefore, we cannot set a range to tell the pixel is part of the black dot or the background.
I test 10 images of different colors, but I hope I can find a method to detect the black dots from more complicated background which may be made up of three or more colors, as long as human eyes can identify the black dots easily. Some extremely small or blur dots can be omitted.
Previous work
Last month, I have asked a similar question at stackoverflow, but have not got a perfect solution, some excellent answers though. Find more details about my work if you are interested.
Here are the methods I have tried:
Converting to grayscale or the brightness of image. The difficulty is that I can not find an adaptive threshold to do binarization. Obviously, turning a color image to grayscale or using the brightness (HSV) will lose much useful information. Otsu algorithm which calculates adaptive threshold can not work either.
Calculating RGB histogram. In my last question, natan's method is to estimate the black color by histogram. It is time-saving, but the adaptive threshold is also a problem.
Clustering. I have tried k-means clustering and found it quite effective for the background that only has one color. The shortage (see my own answer) is I need to set the number of clustering center in advance but I don't know how the background will be. What's more, it is too slow! My application is for real time capturing on iPhone and now it can process 7~8 frames per second using k-means (20 FPS is good I think).
Summary
I think not only similar colors but also adjacent pixels should be "clustered" or "merged" in order to extract the black dots. Please guide me a proper way to solve my problem. Any advice or algorithm will be appreciated. There is no free lunch but I hope a better trade-off between cost and accuracy.
I was able to get some pretty nice first pass results by converting to HSV color space with rgb2hsv, then using the Image Processing Toolbox functions imopen and imregionalmin on the value channel:
rgb = imread('6abIc.jpg');
hsv = rgb2hsv(rgb);
openimg = imopen(hsv(:, :, 3), strel('disk', 11));
mask = imregionalmin(openimg);
imshow(rgb);
hold on;
[r, c] = find(mask);
plot(c, r, 'r.');
And the resulting images (for the image in the question and one chosen from your link):
You can see a few false positives and missed dots, as well as some dots that are labeled with multiple points, but a few refinements (such as modifying the structure element used in the opening step) could clean these up some.
I was curios to test with my old 2d peak finder code on the images without any threshold or any color considerations, really crude don't you think?
im0=imread('Snap10.jpg');
im=(abs(255-im0));
d=rgb2gray(im);
filter=fspecial('gaussian',16,3.5);
p=FastPeakFind(d,0,filter);
imagesc(im0); hold on
plot(p(1:2:end),p(2:2:end),'r.')
The code I'm using is a simple 2D local maxima finder, there are some false positives, but all in all this captures most of the points with no duplication. The filter I was using was a 2d gaussian of width and std similar to a typical blob (the best would have been to get a matched filter for your problem).
A more sophisticated version that does treat the colors (rgb2hsv?) could improve this further...
Here is an extraodinarily simplified version, that can be extended to be full RGB, and it also does not use the image procesing library. Basically you can do 2-D convolution with a filter image (which is an example of the dot you are looking for), and from the points where the convolution returns the highest values, are the best matches for the dots. You can then of course threshold that. Here is a simple binary image example of just that.
%creating a dummy image with a bunch of small white crosses
im = zeros(100,100);
numPoints = 10;
% randomly chose the location to put those crosses
points = randperm(numel(im));
% keep only certain number of points
points = points(1:numPoints);
% get the row and columns (x,y)
[xVals,yVals] = ind2sub(size(im),points);
for ii = 1:numel(points)
x = xVals(ii);
y = yVals(ii);
try
% create the crosses, try statement is here to prevent index out of bounds
% not necessarily the best practice but whatever, it is only for demonstration
im(x,y) = 1;
im(x+1,y) = 1;
im(x-1,y) = 1;
im(x,y+1) = 1;
im(x,y-1) = 1;
catch err
end
end
% display the randomly generated image
imshow(im)
% create a simple cross filter
filter = [0,1,0;1,1,1;0,1,0];
figure; imshow(filter)
% perform convolution of the random image with the cross template
result = conv2(im,filter,'same');
% get the number of white pixels in filter
filSum = sum(filter(:));
% look for all points in the convolution results that matched identically to the filter
matches = find(result == filSum);
%validate all points found
sort(matches(:)) == sort(points(:))
% get x and y coordinate matches
[xMatch,yMatch] = ind2sub(size(im),matches);
I would highly suggest looking at the conv2 documentation on MATLAB's website.
I have tried image subtraction in MatLab, but realised that there is a big blue patch on the image. Please see image for more details.
Another images showing where the blue patch approximately cover till.
The picture on the left in the top 2 images shows the picture after subtraction.You can ignore the picture on the right of the top 2 images. This is one of the original image:
and this is the background I am subtracting.
The purpose is to get the foreground image and blob it, followed by counting the number of blobs to see how many books are stacked vertically from their sides. I am experimenting how blobs method works on matlab.
Do anybody have any idea? Below is the code on how I carry out my background subtraction as well as display it. Thanks.
[filename, user_canceled] = imgetfile;
fullFileName=filename;
rgbImage = imread(fullFileName);
folder = fullfile('C:\Users\Aaron\Desktop\OPENCV\Book Detection\Sample books');
baseFileName = 'background.jpg';
fullFileName = fullfile(folder, baseFileName);
backgroundImage =imread(fullFileName);
rgbImage= rgbImage - backgroundImage;
%display foreground image after background substraction%%%%%%%%%%%%%%
subplot( 1,2,1);
imshow(rgbImage, []);
Because the foreground objects (i.e. the books) are opaque, the background does not affect those pixels at all. In other words, you are subtracting out something that is not there. What you need is a method of detecting which pixels in your image correspond to foreground, and which correspond to background. Unfortunately, solving this problem might be at least as difficult as the problem you set out to solve in the first place.
If you just want a pixel-by-pixel comparison with the background you could try something like this:
thresh = 250;
imdiff = sum(((rgbImage-backgroundImage).^2),3);
mask = uint8(imdiff > thresh);
maskedImage = rgbImage.*cat(3,mask,mask,mask);
imshow(maskedImage, []);
You will have to play around with the threshold value until you get the desired masking. The problem you are going to have is that the background is poorly suited for the task. If you had the books in front of a green screen for example, you could probably do a much better job.
You are getting blue patches because you are subtracting two color RGB images. Ideally, in the difference image you expect to get zeros for the background pixels, and non-zeros for the foreground pixels. Since you are in RGB, the foreground pixels may end up having some weird color, which does not really matter. All you care about is that the absolute value of the difference is greater than 0.
By the way, your images are probably uint8, which is unsigned. You may want to convert them to double using im2double before you do the subtraction.
I want to process an image in matlab
The image consists out of a solid back ground and two specimens (top and bottom side). I already have a code that separate the top and bottom and make it two images. But the part what I don't get working is to crop the image to the glued area only (red box in the image, I've only marked the top one). However, the cropped image should be a rectangle just like the red box (the yellow background, can be discarded afterwards).
I know this can be done with imcrop, but this requires manual input from the user. The code needs to be automated such that it is possible to process more images without user input. All image will have the same colors (red for glue, black for material).
Can someone help me with this?
edit: Thanks for the help. I used the following code to solve the problem. However, I couldn't get rid of the black part right of the red box. This can be fix by taping that part off before making pictures. The code which I used looks a bit weird, but it succeeds in counting the black region in the picture and getting a percentage.
a=imread('testim0.png');
level = graythresh(a);
bw2=im2bw(a, level);
rgb2=bw2rgb(bw2);
IM2 = imclearborder(rgb2,4);
pic_negative = ait_imgneg(IM2);
%% figures
% figure()
% image(rgb2)
%
% figure()
% imshow(pic_negative)
%% Counting percentage
g=0;
for j=1:size(rgb2,2)
for i=1:size(rgb2,1)
if rgb2(i,j,1) <= 0 ...
& rgb2(i,j,2) <= 0 ...
& rgb2(i,j,3) <= 0
g=g+1;
end
end
end
h=0;
for j=1:size(pic_negative,2)
for i=1:size(pic_negative,1)
if pic_negative(i,j)== 0
h=h+1;
end
end
end
per=g/(g+h)
If anyone has some suggestions to improve the code, I'm happy to hear it.
For a straight-forward image segmentation into 2 regions (background, foreground) based on color (yellow, black are prominent in your case), an option can be clustering image color values using kmeans algorithm. For additional robustness you can transform the image from RGB to Lab* colorspace.
The example for your case follows the MATLAB Imape Processing example here.
% read and transform to L*a*b space
im_rgb = double(imread('testim0.png'))./256;
im_lab = applycform(im_rgb, makecform('srgb2lab'));
% keep only a,b-channels and form feature vector
ab = double(lab_I(:,:,2:3));
[nRows, nCols, ~] = size(ab);
ab = reshape(ab,nRows * nCols,2);
% apply k-means for 2 regions, repeat c times, e.g. c = 5
nRegions = 2;
[cluster_idx cluster_center] = kmeans(ab,nRegions, 'Replicates', 5);
% get foreground-background mask
im_regions = reshape(cluster_idx, nRows, nCols);
You can use the resulting binary image to index the regions of interest (or find the boundary) in the original reference image.
Images are saved as matrices. If you know the bounds in pixels of the crop box want to crop you can execute the crop using indexing.
M = rand(100); % create a 100x100 matrix or load it from an image
left = 50;
right = 75;
top = 80;
bottom = 10;
croppedM = M(bottom:top, left:right);
%save croppedm
You can easily get the unknown bounded crop by
1) Contour plotting the image,
2) find() on the result for the max/min X/ys,
3) use #slayton's method to perform the actual crop.
EDIT: Just looked at your actual image - it won't be so easy. But color enhance/threshold your image first, and the contours should work with reasonable accuracy. Needless to say, this requires tweaking to your specific situation.
since you've already been able to seperate the top and bottom, and also able to segment the region you want (including the small part of the right side that you don't want), I propose you just add a fix at the end of the code via the following.
after segmentation, sum each column Blue intensity value, so that you are compressing the image from 2d to 1d. so if original region is width=683 height=59, the new matrix/image will be simply width=683 height=1.
now, you can apply a small threshold to determine where the edge should lie, and apply a crop to the image at that location. Now you get your stats.