This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Detecting thin lines in blurry image
So as the title says, I am trying to detect boundaries of patterns. In the images attached, you can basically see three different patterns.
Close stripe lines
One thick L shaped line
The area between 1 & 2
I am trying to separate these three, in say 3 separate images. Depend on where the answers go, I will upload more images if needed. Both idea or code will be helpful.
You can solve (for some values of "solve") this problem using morphology. First, to make the image more uniform, remove irrelevant minima. One way to do this is using the h-dome transform for regional minima, which suppresses minima of height < h. Now, we want to join the thin lines. That is accomplished by a morphological opening with a horizontal line of length l. If the lines were merged, then the regional minima of the current image is the background. So we can fill holes to obtain the relevant components. The following code summarizes these tasks:
f = rgb2gray(imread('http://i.stack.imgur.com/02X9Z.jpg'));
hm = imhmin(f, h);
o = imopen(hm, strel('line', l, 0));
result = imfill(~imregionalmin(o), 'holes');
Now, you need to determine h and l. The parameter h is expected to be easier since it is not related to the scale of the input, and in your example, values in the range [10, 30] work fine. To determine l maybe a granulometry analysis could help. Another way is to check if the result contains two significant connected components, corresponding to the bigger L shape and the region of the thin lines. There is no need to increase l one by one, you could perform something that resembles a binary search.
Here are the hm, o and result images with h = 30 and l = 15 (l in [13, 19] works equally good here). This approach gives flexibility on parameter choosing, making it easier to pick/find good values.
To calculate the area in the space between the two largest components, we could merge them and simply count the black pixels inside the new connected component.
You can pass a window (10x10 pixels?) and collect features for that window. The features could be something as simple as the cumulative gradients (edges) within that window. This would distinguish the various areas as long as the window is big enough.
Then using each window as a data point, you can do some clustering, or if the patterns don't vary that much you can do some simple thresholds to determine which data points belong to which patterns (the larger gradient sums belong to the small lines: more edges, while the smallest gradient sums belong to the thickest lines: only one edge, and those in between belong to the other "in-between" pattern .
Once you have this classification, you can create separate images if need be.
Just throwing out ideas. You can binarize the image and do connected component labelling. Then perform some analysis on the connected components such as width to discriminate between the regions.
Related
I want to identify lego bricks for building a lego sorting machine (I use c++ with opencv).
That means I have to distinguish between objects which look very similar.
The bricks are coming to my camera individually on a flat conveyer. But they might lay in any possible way: upside down, on the side or "normal".
My approach is to teach the sorting machine the bricks by taping them with the camera in lots of different positions and rotations. Features of each and every view are calculated by surf-algorythm.
void calculateFeatures(const cv::Mat& image,
std::vector<cv::KeyPoint>& keypoints,
cv::Mat& descriptors)
{
// detector == cv::SurfFeatureDetector(10)
detector->detect(image,keypoints);
// extractor == cv::SurfDescriptorExtractor()
extractor->compute(image,keypoints,descriptors);
}
If there is an unknown brick (the brick that i want to sort) its features also get calculated and matched with known ones.
To find wrongly matched features I proceed as described in the book OpenCV 2 Cookbook:
with the matcher (=cv::BFMatcher(cv::NORM_L2)) the two nearest neighbours in both directions are searched
matcher.knnMatch(descriptorsImage1, descriptorsImage2,
matches1,
2);
matcher.knnMatch(descriptorsImage2, descriptorsImage1,
matches2,
2);
I check the ratio between the distances of the found nearest neighbours. If the two distances are very similar it's likely that a false value is used.
// loop for matches1 and matches2
for(iterator matchIterator over all matches)
if( ((*matchIterator)[0].distance / (*matchIterator)[1].distance) > 0.65 )
throw away
Finally only symmatrical match-pairs are accepted. These are matches in which not only n1 is the nearest neighbour to feature f1, but also f1 is the nearest neighbour to n1.
for(iterator matchIterator1 over all matches)
for(iterator matchIterator2 over all matches)
if ((*matchIterator1)[0].queryIdx == (*matchIterator2)[0].trainIdx &&
(*matchIterator2)[0].queryIdx == (*matchIterator1)[0].trainIdx)
// good Match
Now only pretty good matches remain. To filter out some more bad matches I check which matches fit the projection of img1 on img2 using the fundamental matrix.
std::vector<uchar> inliers(points1.size(),0);
cv::findFundamentalMat(
cv::Mat(points1),cv::Mat(points2), // matching points
inliers,
CV_FM_RANSAC,
3,
0.99);
std::vector<cv::DMatch> goodMatches
// extract the surviving (inliers) matches
std::vector<uchar>::const_iterator itIn= inliers.begin();
std::vector<cv::DMatch>::const_iterator itM= allMatches.begin();
// for all matches
for ( ;itIn!= inliers.end(); ++itIn, ++itM)
if (*itIn)
// it is a valid match
The result is pretty good. But in cases of extreme alikeness faults still occur.
In the picture above you can see that a similar brick is recognized well.
However in the second picture a wrong brick is recognized just as well.
Now the question is how I could improve the matching.
I had two different ideas:
The matches in the second picture trace back to the features really fitting, but only if the visual field is intensely changed. To recognize a brick I have to compare it in many different positions anyway (at least as shown in figure three). This means I know that I am only allowed to minimally change the visual field. The information how intensely the visual field is changed should be hidden in the fundamental matrix. How can I read out of this matrix how far the position in the room has changed? Especially the rotation and strong scaling should be of interest; if the brick once is taped farer on the left side this shouldn't matter.
Second idea:
I calculated the fundamental matrix out of 2 pictures and filtered out features that don't fit the projections - shouldn't there be a way to do the same using three or more pictures? (keyword Trifocal tensor). This way the matching should become more stable. But I neither know how to do this using OpenCV nor could I find any information on this on google.
I don't have a complete answer, but I have a few suggestions.
On the image analysis side:
It looks like your camera setup is pretty constant. Easy to just separate the brick from the background. I also see your system finding features in the background. This is unnecessary. Set all non-brick pixels to black to remove them from the analysis.
When you have located just the brick, your first step should be to just filter likely candidates based on the size (i.e. number of pixels) in the brick. That way the example faulty match you show is already less likely.
You can take other features into account such as the aspect ratio of the bounding box of the brick, the major and minor axes (eigevectors of the covariance matrix of the central moments) of the brick etc.
These simpler features will give you a reasonable first filter to limit your search space.
On the mechanical side:
If bricks are actually coming down a conveyor you should be able to "straighten" the bricks along a straight edge using something like a rod that lies at an angle to the direction of the conveyor across the belt so that the bricks arrive more uniformly at your camera like so.
Similar to the previous point, you could use something like a very loose brush suspended across the belt to topple bricks standing up as they pass.
Again both these points will limit your search space.
So I’m trying to find the rotational angle for stripe lines in images like the attached photo.
The only assumption is that the lines are parallel, and their orientation is about 90 degrees approximately more or less [say 5 degrees tolerance].
I have to make sure the stripe lines in the result image will be %100 vertical. The quality of the images varies as well as their histogram/greyscale values. So methods based on non-adaptive thresholding already failed for my cases [I’m not interested in thresholding based methods if I cannot make it adaptive]. Also, there are some random black clusters on top of the stripe lines sometimes.
What I did so far:
1) Of course HoughLines is the first option, but I couldn’t make it work for all my images, I had some partial success though following this great article:
http://felix.abecassis.me/2011/09/opencv-detect-skew-angle/.
The main reason of failure to my understanding was that, I needed to fine tune the parameters for different images. Parameters such as Canny/BW/Morphological edge detection (If needed) | parameters for minLinelength/maxLineGap/etc. For sure there’s a way to hack into this and make it work, but, to me this is a fragile solution!
2) What I’m working on right now, is to divide the image to a top slice and a bottom slice, then find the peaks and valleys of each slice. Then basically find the angle using the width of the image and translation of peaks. I’m currently working on finding which peak of the top slice belongs to which of the bottom slice, since there will be some false positive peaks in my computation due to existence of black/white clusters on top of the strip lines.
Example: Location of peaks for slices:
Top Slice = { 1, 33,67,90,110}
BottomSlice = { 3, 14, 35,63,90,104}
I am actually getting similar vectors when extracting peaks. So as can be seen, the length of vector might vary, any idea how can I get a group like:
{{1,3},{33,35},{67,63},{90,90},{110,104}}
I’m open to any idea about improving any of these algorithms or a completely new approach. If needed, I can upload more images.
If you can get a list of points for a single line, a linear regression will give you a formula for the straight line that best fits the points. A simple trig operation will convert the line formula to an angle.
You can probably use some line thinning operation to turn the stripes into a list of points.
You can run an accumulator of spatial derivatives along different angles. If you want a half-degree precision and a sample of 5 lines, you have a maximum 10*5*1500 = 7.5m iterations. You can safely reduce the sampling rate along the line tenfold, which will give you a sample size of 150 points per sample, reducing the number of iterations to less than a million. Somewhere around that point the operation of straightening the image ought to become the bottleneck.
I'm making a program to view 3D CAD models and would like to build in automated exploded views. All the assemblies that will be viewed are axi-symmetric. Some may not be, but the majority are. I'd like to figure out an algorithm for automatically moving parts in an assembly into an exploded view position. Here is an example of what I want to achieve through an algorithm (minus the labels of course):
The only value I have to work with is the center of the bounding box of each part. If more information than that is needed, I can calculate more information, but it seems like it should be sufficient. The rough approach I have in mind is to calculate a vector from the origin of the assembly to the center of each part along the axi-symmetric axis, then calculate a radial vector to the center of the part with respect to the center axis. From there, I'd need to figure out some calculation that would be able to scale the position of each part along some combination of those two vectors. That's the part where I'm not quite sure what direction to go with this. The image I've included shows the exact functionality I'd like, but I want to be able to scale the position by any float value to expand or contract the exploded view, with 1.0 being the original assembled model. Any ideas?
Your question is quite broad and thus my explanation became somehow lengthy. I'll propose two variants of an explosion algorithm for both axial and radial treatment.
To illustrate them with an example I'll use the following numbers (bounding boxes along the axis only, only five parts):
P1: [ 0,10] (battery)
P2: [10,14] (motor)
P3: [14,16] (cog)
P4: [16,24] (bit holder)
P5: [18,26] (gear casing)
While parts P1 to P4 exactly touch each other, P4 and P5 actually overlap.
The first one is an algorithm which basically scales the distances by a factor, such as you proposed. It will suffer if size of pieces is much different in an assembly but also for overlapping parts (e.g. in your example along the axis the extension of circle cog is much smaller than bit holder).
Let the scaling factor be f, then the center of each bounding box is scaled by f, but extension is not. Parts then would be
P1: 5 + [-5,5] => P1': 5*f + [-5,5]
P2: 12 + [-2,2] => P2': 12*f + [-2,2]
P3: 15 + [-1,1] => P3': 15*f + [-1,1]
P4: 20 + [-4,4] => P4': 20*f + [-4,4]
P5: 22 + [-4,4] => P5': 22*f + [-4,4]
The distance between the parts P1' to P4 is then given by
P2' - P1' : (12*f-2) - (5*f+5) = 7*(f-1)
P3' - P2' : (15*f-1) - (12*f+2) = 3*(f-1)
P4' - P3' : (20*f-4) - (15*f+1) = 5*(f-5)
As expected the difference is zero for f=0 but for any exploded view the distance strongly depends on the sizes of the separate parts. I don't think that this will look too good if variation of sizes is bigger.
Additionally for overlapping parts
P5' - P4' : (22*f-4) - (20*f+4) = 2*f-8
they still overlap for reasonable f.
Another possibility would be to define not a scaling factor for the axis but a constant part-distance d. Then bounding boxes would be aligned like the following:
P1': [ 0,10]
P2': [10,14]+d
P3': [14,16]+2*d
P4': [16,24]+3*d
P5': [18,26]+4*d+6
Note that in the last line we added 24-8=6, i.e. the overlap in order to differentiate the two parts.
While this algorithm handles the above mentioned cases in a (in my opinion) better way we have to add special care to parts which cover multiple other parts and should not be included in the grouping (e.g. handle top in your case).
One possibility would be to group the parts into groups in a first step and then apply the algorithm to the bounding box of these groups. Afterwards it can be applied to parts in each group again, omitting the parts which cover more than one subgroup. In your case it would be (note nested grouping is possible):
[
([battery,(switch,circuit switch),motor],handle top),
motor cog,
tri-cog,
red-cog,
circle-cog,
bit-holder,
(gear casing,spring,lock knob)
]
You might see that I have introduced two different kind of groups: parts/groups in square braces are handled by the algorithm, i.e. a spacing is added between each part/subgroup inside such a group, while the groups inside round braces are not exploded.
Up to now we did not handled the radial explosion because it nicely decouples from the axis treatment. But again the same both approaches can be used for radial explosion also. But again in my opinion the second algorithm yields more pleasant results. E.g. the groups can be done as follows for radial treatment:
[
(battery,switch,<many parts>,gear casing),
(switch,spring),
(handle top, lock knob)
]
In this case we would add an additional component r to all radial centers in the second group and 2*r to all in the third group.
Note that the simple scaling algorithm runs without special user guidance (once the scaling factor is given) while the second one uses additional information (the grouping).
I hope this rather long explanation gives you some ideas how to proceed further. If my explanations are unclear at some point or if you have further questions please feel free to comment.
I have some map files consisting of 'polylines' (each line is just a list of vertices) representing tunnels, and I want to try and find the tunnel 'center line' (shown, roughly, in red below).
I've had some success in the past using Delaunay triangulation but I'd like to avoid that method as it does not (in general) allow for easy/frequent modification of my map data.
Any ideas on how I might be able to do this?
An "algorithm" that works well with localized data changes.
The critic's view
The Good
The nice part is that it uses a mixture of image processing and graph operations available in most libraries, may be parallelized easily, is reasonable fast, may be tuned to use a relatively small memory footprint and doesn't have to be recalculated outside the modified area if you store the intermediate results.
The Bad
I wrote "algorithm", in quotes, just because I developed it and surely is not robust enough to cope with pathological cases. If your graph has a lot of cycles you may end up with some phantom lines. More on this and examples later.
And The Ugly
The ugly part is that you need to be able to flood fill the map, which is not always possible. I posted a comment a few days ago asking if your graphs can be flood filled, but didn't receive an answer. So I decided to post it anyway.
The Sketch
The idea is:
Use image processing to get a fine line of pixels representing the center path
Partition the image in chunks commensurated to the tunnel thinnest passages
At each partition, represent a point at the "center of mass" of the contained pixels
Use those pixels to represent the Vertices of a Graph
Add Edges to the Graph based on a "near neighbour" policy
Remove spurious small cycles in the induced Graph
End- The remaining Edges represent your desired path
The parallelization opportunity arises from the fact that the partitions may be computed in standalone processes, and the resulting graph may be partitioned to find the small cycles that need to be removed. These factors also allow to reduce the memory needed by serializing instead of doing calcs in parallel, but I didn't go trough this.
The Plot
I'll no provide pseudocode, as the difficult part is just that not covered by your libraries. Instead of pseudocode I'll post the images resulting from the successive steps.
I wrote the program in Mathematica, and I can post it if is of some service to you.
A- Start with a nice flood filled tunnel image
B- Apply a Distance Transformation
The Distance Transformation gives the distance transform of image, where the value of each pixel is replaced by its distance to the nearest background pixel.
You can see that our desired path is the Local Maxima within the tunnel
C- Convolve the image with an appropriate kernel
The selected kernel is a Laplacian-of-Gaussian kernel of pixel radius 2. It has the magic property of enhancing the gray level edges, as you can see below.
D- Cutoff gray levels and Binarize the image
To get a nice view of the center line!
Comment
Perhaps that is enough for you, as you ay know how to transform a thin line to an approximate piecewise segments sequence. As that is not the case for me, I continued this path to get the desired segments.
E- Image Partition
Here is when some advantages of the algorithm show up: you may start using parallel processing or decide to process each segment at a time. You may also compare the resulting segments with the previous run and re-use the previous results
F- Center of Mass detection
All the white points in each sub-image are replaced by only one point at the center of mass
XCM = (Σ i∈Points Xi)/NumPoints
YCM = (Σ i∈Points Yi)/NumPoints
The white pixels are difficult to see (asymptotically difficult with param "a" age), but there they are.
G- Graph setup from Vertices
Form a Graph using the selected points as Vertex. Still no Edges.
H- select Candidate Edges
Using the Euclidean Distance between points, select candidate edges. A cutoff is used to select an appropriate set of Edges. Here we are using 1.5 the subimagesize.
As you can see the resulting Graph have a few small cycles that we are going to remove in the next step.
H- Remove Small Cycles
Using a Cycle detection routine we remove the small cycles up to a certain length. The cutoff length depends on a few parms and you should figure it empirically for your graphs family
I- That's it!
You can see that the resulting center line is shifted a little bit upwards. The reason is that I'm superimposing images of different type in Mathematica ... and I gave up trying to convince the program to do what I want :)
A Few Shots
As I did the testing, I collected a few images. They are probably the most un-tunnelish things in the world, but my Tunnels-101 went astray.
Anyway, here they are. Remember that I have a displacement of a few pixels upwards ...
HTH !
.
Update
Just in case you have access to Mathematica 8 (I got it today) there is a new function Thinning. Just look:
This is a pretty classic skeletonization problem; there are lots of algorithms available. Some algorithms work in principle on outline contours, but since almost everyone uses them on images, I'm not sure how available such things will be. Anyway, if you can just plot and fill the sewer outlines and then use a skeletonization algorithm, you could get something close to the midline (within pixel resolution).
Then you could walk along those lines and do a binary search with circles until you hit at least two separate line segments (three if you're at a branch point). The midpoint of the two spots you first hit, or the center of a circle touching the three points you first hit, is a good estimate of the center.
Well in Python using package skimage it is an easy task as follows.
import pylab as pl
from skimage import morphology as mp
tun = 1-pl.imread('tunnel.png')[...,0] #your tunnel image
skl = mp.medial_axis(tun) #skeleton
pl.subplot(121)
pl.imshow(tun,cmap=pl.cm.gray)
pl.subplot(122)
pl.imshow(skl,cmap=pl.cm.gray)
pl.show()
I want to draw some data into a texture: many items in a row. They aren't created in order, and they may all be different sizes (think of a memory heap). Each data item is a small rectangle and I want to be able to distinguish them apart, so I'd like each of them to have a unique colour.
Now I could just use rand() to generate RGB values and hope they are all different, but I suspect I won't get good distribution in RGB space. Is there a better way than this? E.g. what is a good way of cycling through different colours before they (nearly) repeat?
The colours don't have to match with any data in the items. I just want to be able to look at many values and see that they are different, as they are adjacent.
I could figure something out but I think this is an interesting question. :)
Using the RGB color model is not a good way to get a good color mix. It's better to use another color model to generate your color, and then convert from that color model to RGB.
I suggest you the HSV or HSL color model instead, in particular you want to vary the Hue.
If you want X different color values, vary them from 0 to 360 with a step size of 360 divided by X.
Whats your sample space... how many items are we talking.
You could build up an array of RGB Triples from
for(int r = 0; r < 255; r = r+16)
for(int g = 0; g < 255; g = g+16)
for(int b = 0; b < 255; b = b+16)
// take r, g, b and add it to a list
Then randomise your list and iterate through it.
that'd give you 16^3 (4096) different colors before a repeated color.
In general RGB isn't a great color space for doing these sorts of things because it's perceptually nonlinear, for starters. This means that equal distances moved between RGB triplets do not look equally different to our eyes.
I'd probably work in the L*c*h* space (see also) space, or HSL space, and just generate a uniform spacing in hue. These spaces have been designed to be approximately perceptually linear.
Google "delta e cie 2000"; the colour-difference formula is useful for determining apparent (visual) distance between 2 colours. (On a monitor; there's a different formula for pigments.) It operates on colours in Lab space (props to simon), but applies a perceptual calculation of difference.
We found that a number around 1.5 was sufficient to ensure visually distinct colours (i.e. you can tell the difference if they are near one another), but if you want identifiable colours (you can find any colour in a legend) you'll need to bump that up.
As to creating a set of colours... I'd probably start at some corner of Lab space, and walk around it using a step size that gives large enough visual differences (note: it's not linear, so step size will probably have to be adaptive) and then randomize the list.
This is very similar to the four-colour problem relating to colouring maps, this might yield some interesting solutions for you:
Four colour theorem
If you just need a set of perceptually-distinct colors (and not an algorithm to generate them) I have created a free tool on my website that does just that:
http://phrogz.net/css/distinct-colors.html
Instead of just using even spacing in RGB or HSV space (which are not uniformly distributed with respect to human perception) the tool allows you to generate a grid of values in HSV space and it then uses the CMC(I:c) standard for color distance to throw out colors that are perceptually too close to each other. (The 'threshold' slider on the second tab allows you to control how visually distinct the colors must be, showing you the results in realtime.)
In the end, you can sort your list of generated colors by various criteria, and then evenly 'shuffle' that list so that you are guaranteed to have visually-distinct values adjacent to each other in the list. (I recommend an 'Interleave' value of about 5.)
As of this writing the tool works well with Chrome, Safari, and (via a shim) Firefox; IE9 does not support HTML5 range input sliders, which the UI uses extensively for interactive exploration.