Let's say I have an image of a ball like this one:
I want to separate the colors of the ball to the color groups. In this case I should have 2 main color groups - "brown" and "white". The "brown" group will have all the brown pixels and the "white" group will have all the white pixels.
I'm using matlab for this task. The way that I thought to do is:
to look at the RGB channels. I used scatter to look if I could clearly see some groups, but I didn't.
to look at the bayer vales. But couldn't see any groups either.
to run an edge detector. Then, in each enclosed area I'll find the mean of the pixels. The areas that will have similar mean values (within a certain threshold) will belong to the same group. It seemed to sort of to work but in many case it didn't
Any other ideas?
This task is called segmentation, in your case each color is a segment and segments are not always continuous.
Searching segmentation examples for Matlab should yield a lot of code examples and theorems.
Note one thing, there is no ground truth solution, you can't say how many segments there are for each image since it is subjective question. In a general case you can run clustering algorithm on the color values which will break the image to color segments, there are algorithms which will find the number of groups automatically - this can be a good start for the number of color groups in your image.
quick search yielded these works, they can get you started with ideas:
Image segmentation with matlab
Using EM for image segmentation
While image segmentation would be the correct way to treat color separation, if your image is simple, you can try to do it brute-force.
Here, converting to HSV would be easier to handle with the image.
For the white parts of the image:
I=imread('ball.jpg');
H=rgb2hsv(I);
% separate dimensions
h=H(:,:,1);
s=H(:,:,2);
v=H(:,:,3);
% color conditions
v(v<0.8 | s>0.7 | h>0.7 )=NaN;
h(isnan(v))=NaN;
s(isnan(v))=NaN;
% convert image back
W=cat(3,h,s,v);
White_image=hsv2rgb(W);
figure; imagesc(White_image);
And for the brown parts:
% separate dimensions
h=H(:,:,1);
s=H(:,:,2);
v=H(:,:,3);
% color conditions
v(s<0.6 | v>0.8 )=NaN;
h(isnan(v))=NaN;
s(isnan(v))=NaN;
% convert image back
B=cat(3,h,s,v);
Brown_image=hsv2rgb(B);
figure; imagesc(Brown_image); axis off
Related
I have a figure looks like this
I want to find the coordinates of all intersections of three hexagons.
How can I do this? Should I use OpenCV?
I am still trying to think of a faster/better method, but I think the following should work:
threshold your image to pure blacks and whites
generate and save a list of all black pixels for later
label your image so that each white hexagon is effectively flood-filled with a unique color (or shade of grey) - some folks call this "labelling", some call it "Blob Analysis", some call it "Connected Component Analysis". Whatever it is called, you will get something like this:
Now look at each black pixel from the list you saved in the second step and count how many different colours other than black are in the surrounding 9x9, or 15x15 area. If it's three it is probably an intersection like you are looking for.
Of course there are variations on this - you could implement a "minimum distance from other intersection" on top, for example. Or a "black line thinning first". Or a dilation of each blob to erode the black lines and make the three colours closer together. You could scale your image down (being careful to use NEAREST_NEIGHBOUR rather than interpolation) after labelling to reduce processing time - if important.
You can try to find these features using Harris corner detector.
Also check if findContours with analysis of result intersections could give you useful information.
In the grayscale image shown below, how can I accurately detect the white region having sharp boundary (marked with red color)?
In this particular image, a simple thresholding might work, however, I have several images in which there are similar areas around corner of images which I want to ignore.
Also, there might be more than one regions of interest, both having different intensities. One can be as bright as it is in the example image, other can be of medium intensity.
However, the only difference between the interested and non-interested areas is as follows:
The interest areas have sharp well defined boundaries.
Non-interested areas don't have sharp boundaries. They tend to gradually merge with neighbourhood areas.
Image without mark for testing:
When you say sharp boundaries, you have to think gradient. The sharper the boundaries, the bigger the gradient. Therefore apply a gradient and you will see that it will be stronger around the shapes you want to segment.
But in your case, you can also observe that the area you want to segment is also the brightest. So I would also try a noise reduction (median filter) plus a convolution filter (simple average) in order to homogenize the different zones, then thresholding by keeping only the brightest/right peak.
im = imread('o2XfN.jpg');
figure
imshow(im)
smooth = imgaussfilt(im,.8); %"blur" the image to take out noisey pixels
big = double(smooth); % some functions don't work with UINT8, I didn't check for these
maxiRow = quantile(big,.99); % .99 qualtile... think quartile from stats
maxiCol = quantile(maxiRow,.98); % again for the column
pixels = find(big>=maxiCol); % which pixels have the highest values
logicMat = false(size(big)); %initalize a logic matrix of zeros
logicMat(pixels) = 1; %set the pixels that passed to logic pass
figure
imshow(logicMat)
It is not extremely clear what you want to do with the regions that you are finding. Also, a few more sample images would be helpful to debug a code. What I posted above may work for that one image, but it is unlikely that it will work for every image that you are processing.
My short question
How to detect the black dots in the following images? (I paste only one test image to make the question look compact. More images can be found →here←).
My long question
As shown above, the background color is roughly blue, and the dots color is "black". If pick one black pixel and measure its color in RGB, the value can be (0, 44, 65) or (14, 69, 89).... Therefore, we cannot set a range to tell the pixel is part of the black dot or the background.
I test 10 images of different colors, but I hope I can find a method to detect the black dots from more complicated background which may be made up of three or more colors, as long as human eyes can identify the black dots easily. Some extremely small or blur dots can be omitted.
Previous work
Last month, I have asked a similar question at stackoverflow, but have not got a perfect solution, some excellent answers though. Find more details about my work if you are interested.
Here are the methods I have tried:
Converting to grayscale or the brightness of image. The difficulty is that I can not find an adaptive threshold to do binarization. Obviously, turning a color image to grayscale or using the brightness (HSV) will lose much useful information. Otsu algorithm which calculates adaptive threshold can not work either.
Calculating RGB histogram. In my last question, natan's method is to estimate the black color by histogram. It is time-saving, but the adaptive threshold is also a problem.
Clustering. I have tried k-means clustering and found it quite effective for the background that only has one color. The shortage (see my own answer) is I need to set the number of clustering center in advance but I don't know how the background will be. What's more, it is too slow! My application is for real time capturing on iPhone and now it can process 7~8 frames per second using k-means (20 FPS is good I think).
Summary
I think not only similar colors but also adjacent pixels should be "clustered" or "merged" in order to extract the black dots. Please guide me a proper way to solve my problem. Any advice or algorithm will be appreciated. There is no free lunch but I hope a better trade-off between cost and accuracy.
I was able to get some pretty nice first pass results by converting to HSV color space with rgb2hsv, then using the Image Processing Toolbox functions imopen and imregionalmin on the value channel:
rgb = imread('6abIc.jpg');
hsv = rgb2hsv(rgb);
openimg = imopen(hsv(:, :, 3), strel('disk', 11));
mask = imregionalmin(openimg);
imshow(rgb);
hold on;
[r, c] = find(mask);
plot(c, r, 'r.');
And the resulting images (for the image in the question and one chosen from your link):
You can see a few false positives and missed dots, as well as some dots that are labeled with multiple points, but a few refinements (such as modifying the structure element used in the opening step) could clean these up some.
I was curios to test with my old 2d peak finder code on the images without any threshold or any color considerations, really crude don't you think?
im0=imread('Snap10.jpg');
im=(abs(255-im0));
d=rgb2gray(im);
filter=fspecial('gaussian',16,3.5);
p=FastPeakFind(d,0,filter);
imagesc(im0); hold on
plot(p(1:2:end),p(2:2:end),'r.')
The code I'm using is a simple 2D local maxima finder, there are some false positives, but all in all this captures most of the points with no duplication. The filter I was using was a 2d gaussian of width and std similar to a typical blob (the best would have been to get a matched filter for your problem).
A more sophisticated version that does treat the colors (rgb2hsv?) could improve this further...
Here is an extraodinarily simplified version, that can be extended to be full RGB, and it also does not use the image procesing library. Basically you can do 2-D convolution with a filter image (which is an example of the dot you are looking for), and from the points where the convolution returns the highest values, are the best matches for the dots. You can then of course threshold that. Here is a simple binary image example of just that.
%creating a dummy image with a bunch of small white crosses
im = zeros(100,100);
numPoints = 10;
% randomly chose the location to put those crosses
points = randperm(numel(im));
% keep only certain number of points
points = points(1:numPoints);
% get the row and columns (x,y)
[xVals,yVals] = ind2sub(size(im),points);
for ii = 1:numel(points)
x = xVals(ii);
y = yVals(ii);
try
% create the crosses, try statement is here to prevent index out of bounds
% not necessarily the best practice but whatever, it is only for demonstration
im(x,y) = 1;
im(x+1,y) = 1;
im(x-1,y) = 1;
im(x,y+1) = 1;
im(x,y-1) = 1;
catch err
end
end
% display the randomly generated image
imshow(im)
% create a simple cross filter
filter = [0,1,0;1,1,1;0,1,0];
figure; imshow(filter)
% perform convolution of the random image with the cross template
result = conv2(im,filter,'same');
% get the number of white pixels in filter
filSum = sum(filter(:));
% look for all points in the convolution results that matched identically to the filter
matches = find(result == filSum);
%validate all points found
sort(matches(:)) == sort(points(:))
% get x and y coordinate matches
[xMatch,yMatch] = ind2sub(size(im),matches);
I would highly suggest looking at the conv2 documentation on MATLAB's website.
I want to make filters like shown here
these are my target filters but can you please guide me how to go for them
how i can make filters like these?
which algorithms i need to follow? and which step i need to take as beginner?
Which is the better and easiest way to get the values of RGB and shades of filters .
copy of image from link above by spektre:
the source image is the first after camera in the first line.
very hard to say from single non test-screen image.
the black and white filter
is easy just convert RGB to intensity i and then instead RGB write iii color. The simplest not precise conversion is
i=(R+G+B)/3
but better way is use of weights
i=w0*R+w1*G+w2*B
where w0+w1+w2=1 the values can be found by a little google search effort
the rest
some filters seem like over exponated colors or weighted colors like this:
r=w0*r; if (r>255) r=255;
g=w1*g; if (g>255) g=255;
b=w2*b; if (b>255) b=255;
write an app with 3 scrollbars for w0,w1,w2 in range <0-10> and redraw image with above formula. After little experimenting you should find w0,w1,w2 for most of the filters ... The rest can be mix of colors like this:
r=w00*r+w01*g+w02*b; if (r>255) r=255;
g=w10*r+w11*g+w12*b; if (g>255) g=255;
b=w20*r+w21*g+w22*b; if (b>255) b=255;
or:
i=(r+g+b)/3
r=w0*r+w3*i; if (r>255) r=255;
g=w1*g+w3*i; if (g>255) g=255;
b=w2*b+w3*i; if (b>255) b=255;
btw if you want the closest similarity you can:
find test colors in input image
like R shades, G shades , B shades , RG,RB,BG,RGB shades from 0-255. Then get colors from filtered image at the same position and draw depedency graphs for each shade draw R,G,B intensities.
One axis is input image color intensity and the other one is R,G,B intensity of filtered color. Then you should see which formula is used directly and can also compute the weights from it. This is how over-exponation works for Red color
if the lines are not lines but curves
then some kind of gamma correction is used so formulas use polynomial of higher order (power of 2,3,4...) mostly power of 2 suffice. In that case the weights can be also negative !!!
some filters could use different color spaces
for example transform RGB to HSV shift hue and convert back to RGB. That will shift colors a little.
I am looking for algorithms to, given an image containing a crossword
crop the image to just the crossword
distinguish between regular and barred crosswords
extract the grid size and the positions of the black squares/bars
The crossword itself can be assumed to be regular (i.e. I am interested in crosswords that have been generated by some program and published as an image, rather than scanned paper-based crosswords), and I would like the program to run without needing any inputs other than the image bitmap.
I can think of some brute-force multi-pass ways to do this (essentially using variants of imagemagick's hit-and-miss filter and then looping over the image looking for leftover dots) but I'm hoping for better ideas from people who actually know about image processing.
This is a really broad question, but I wil try to give you some pointers.
These are the steps you need to take:
Detect the position of the crossword.
Detect the grid of the crossword. For this, you will need some Computer Vision algorithm (for example the Hough lines detector).
For each cell, you need to find if it have a character or not. To do so, you just simply analize the "amount" of white color does the cell have
For the cells containing a character you need to recognize it. To do so, you need an OCR, and I recommend you Tesseract.
Create your own algorithm for solving the crosswords. You can use this.
And here (1,2,3) you have an example of a Sudoku Solver in Python. The first steps are common to your problem so you can use OpenCV to solve it like this:
import cv2
import numpy as np
#Load the Black and White image
img = cv2.imread('sudoku.jpg')
gray = cv2.cvtColor(img,cv2.COLOR_BGR2GRAY)
gray = cv2.GaussianBlur(gray,(5,5),0)
thresh = cv2.adaptiveThreshold(gray,255,1,1,11,2)
#Detect the lines of the sudoku
contours, hierarchy = cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)
#Detect the square of the Sudoku
biggest = None
max_area = 0
for i in contours:
area = cv2.contourArea(i)
if area > 100:
peri = cv2.arcLength(i,True)
approx = cv2.approxPolyDP(i,0.02*peri,True)
if area > max_area and len(approx)==4:
biggest = approx
max_area = area
Using a screenshot of the linked crossword as example, I assume that:
the crossword grid is crisp, i.e. the horizontal and vertical grid lines are drawn at exact pixels with a constant dark colour and that there is no noise inside the grid cells,
the crossword is black or another relatively dark colour ("black") on white or light grey ("white"),
the clue numbers are written in the top left corner,
the crossword is rectangular and regular.
You can then scan the image from top to bottom to find horizontal black lines of sufficient length. A line starts with a black pixel and ends with a white pixel. Other pixels are indicators that it is not a line. (This is to weed out text and buttons.) Do the same for vertical lines.
Ideally, you now have the crossword lines. If your image is not cropped to the crossword, you might have false positives, such as the button borders. To find the crossword lines, sort them by length and look for the largest contiguous block of the same length. These should be your crossword lines unless you hae some degenerate cases
Now do a nested loop of horizontal and vertical lines, but skip the first line. Look two or three pixels to the northwest of the intersection of the lines. If the pixel is dark, that's a blank. If it is light, it's a cell. This heuristic seems to work well. I say dark and light here, bacause some crosswords use grey cells to save on ink when printing and some cell are highlighted in the screenshot.
If you end up with no blanks, you have a barred crossword. You can find the bars by checking whether one of the pixels to the left and right of a cell border is black.
Lastly, a tip: If you want to use your algorithm to find the cells in a crossword generated with the Crossword Compiler, look at the source. You will find a link to a Javascript file /puzzles/sample/cryptic_demo/cryptic_demo_xml.js, which contans the crossword as XML string, which also gives you the clues as a bonus.
Older versions of the Crossword Compiler, such as the one used for the Independent Cryptic hide their data in a file loaded from an applet. The format of that file is binary, but not too hard to read if you know the original data.
Try hough transform to find squares and when you get the squares check using histogram whether it is a dark or white square using threshold on its gray scale values
Thinking of an alternative way to do this.
This is similar in many respects to object recongition, computer vision
One way would be to use a framework like openCV which, trained with some samples of what you want to detect, can detect any similar results
(a javascript library for object detection based on Viola-Jones algorithm, used also by openCV and of which am the author is HAAR.js)
Apart from this (or a similar alternative to this) there is a possibility of constructing a "visual" template of a crossword you want to detect (in a scale-invariant way)
and scan the images looking for correlations of parts of the image with the template (complexity O(N*M), N size of image, M size of template)
Since crossword grids have relatively constant shapes (especially fixed outputs of crossword compilers) it should be relative easy to create a prototype template and have success in matching (and aligning) the detected regions to extract the shape information