I need to extract the ALL Wall Edges (including floor,wall intersections and wall,door intersections) from the following image.If I use the canny detection and hough transform (probabilistic). It gives me to many redundant and unnecessary lines. I was looking if I could refine the canny image before hough transform is run on it.
Input Image
This following is the canny image given by the canny detection algorithm
I am using canny parameters as 0,20 for min and max threshold. I can't use a very high value for max threshold otherwise I will lose wall edges but gradient will be low there compared to rest of the image.
I thought of identifying a high density cluster of points in a window and set them to zero if it is above some threshold.
The following is the canny image obtained after that. You can see the wall edges are preserved.
Can anyone suggest me a better way of handling this problem? I mean refining the canny image so that I can identify cluster of random points and getting away with those but setting them to zero . I was thinking of checking for colinear points in a window but don't know how effective that would be?
Any Comments would be welcome
I think you can filter out longest and nearly vertical lines, after using hough transform. Check out this link.
SimpleCV is just a shortcut library including OpenCV functions, you dont need to use it. I dont think you will encounter problems implementing the algorithm after getting the idea.
Edit: Ok, I thought more about your problem. Setting clusters to zero as a preprocessing step is not bad actually. What about increasing the window size step by step? I mean after obtaining second image, apply another cluster filter with 2*window size, same threshold. I think you can go on like this, as the wall edges are hard to be cancelled out.
Another way, use a rectangular window (width >= 5*height) for cluster filtering as you need vertical edges.
Another way, play with erosion and dilation and filter out blobs having large area.
Another way, check out the top part of the image, there is only the wall edges and the chandelier. You can search horizontally for a white pattern, then follow its neighbours to specify the length of connected points. Then filter out longer ones.
Related
I have a challenging problem to solve. The Figure shows green lines, that are derived from an image and the red lines are the edges derived from another image. Both the images are taken from the same camera, so the intrinsic parameters are same. Only, the exterior parameters are different, i.e. there is a slight rotation and translation while taking the 2nd image. As it can be seen in the figure, the two sets of lines are pretty close. My task is to find correspondence between the edges derived from the 1st image and the edges derived from the second image.
I have gone through a few sources, that mention taking corresponding the nearest line segment, by calculating Euclidean distances between the endpoints of an edge of image 1 to the edges of image 2. However, this method is not acceptable for my case, as there are edges in image 1, near to other edges in image 2 that are not corresponding, and this will lead to a huge number of mismatches.
After a bit of more research, few more sources referred to Hausdorff distance. I believe that this could really be a solution to my problem and the paper
"Rucklidge, William J. "Efficiently locating objects using the
Hausdorff distance." International Journal of Computer Vision 24.3
(1997): 251-270."
seemed to be really interesting.
If, I got it correct the paper formulated a function for calculating translation of model edges to image edges. However, while implementation in MATLAB, I'm completely lost, where to begin. I will be much obliged if I can be directed to a pseudocode of the same algorithm or MATLAB implementation of the same.
Additionally, I am aware of
"Apply Hausdorff distance to tile image classification" link
and
"Hausdorff regression"
However, still, I'm unsure how to minimise Hausdorff distance.
Note1: Computational cost is not of concern now, but faster algorithm is preferred
Note2: I am open to other algorithms and methods to solve this as long as there is a pseudocode available or an open implementation.
Have you considered MATLAB's image registration tools?
With imregister(https://www.mathworks.com/help/images/ref/imregister.html), you can just insert both images, 1 as reference, one as "moving" and it will register them together using an affine transform. The function call is just
[optimizer, metric] = imregconfig('monomodal');
output_registered = imregister(moving,fixed,'affine',optimizer,metric);
For better visualization, use the RegistrationEstimator command to open up a gui in which you can import the 2 images and play around with it to register your images. From there you can export code for future images.
Furthermore if you wish to account for non-rigid transforms there is imregdemons(https://www.mathworks.com/help/images/ref/imregdemons.html) which works much the same way.
You can compute the Hausdorff distance using Matlab's bwdist function. You would compute the distance transform of one image, evaluate it at the edge points of the other, and take the maximum value. (You can also take the sum instead, in which case it is called the chamfer distance.) For this problem you'll probably want the symmetric Hausdorff distance, so you would do the computation in both directions.
Both Hausdorff and chamfer distance measure the match quality of a particular alignment. To find the best registration you'll need to try multiple alignment transformations and evaluate them all looking for the best one. As suggested in another answer, you may find it easier to use registration existing tools than to write your own.
Suppose I have an image of a scene as depicted above. A sort of a pole with a blob on it next to possibly similar objects with no blobs.
How can I find the blob marked by the red circle (a binary image indicating which pixels belong to the blob).
Note that the pole together with the blob may be rotated arbitrarily and also size may vary.
Can you try to do it in below 4 steps?
Circle detection like: writing robust (color and size invariant) circle detection with opencv (based on Hough transform or other features)
Line detection, like: Finding location of rectangles in an image with OpenCV
Identify rectangle position by combining neighboring lines (For each line segment you have the start and end point position, you also know the direction of each line segment. So that you can figure out if two connecting line segments (whose endpoints are close) are orthogonal. Your goal is to find 3 such segments for each rectangle.)
Check the relative position of each circle and rectangle to see if any pair can form the knob shape.
One approach could be using Viola-Jones object detection framework.
Though the framework is mostly used for face detection - it is actually designed for generic objects you feed to the algorithm.
The algorithm basic idea is to feed samples of "good object" (what you are looking for) and "bad objects" to a machine learning algorithm - which generates patterns from the images as its features.
During Classification - using a sliding window the algorithm will search for a "match" to the object (the classifier returned a positive answer).
The algorithm uses supervised learning and thus requires a labeled set of examples (both positive and negative ones)
I'm sure there is some boundary-map algorithm in image processing to do this.
Otherwise, here is a quick fix: pick a pixel at the center of the
"undiscovered zone", which initially is the whole image.
trace the horizantal and vertical lines at 4 directions each ending at the
borders of the zone and find the value changes from 0 to 1 or the vice verse.
Trace each such value switch and complete the boundary of each figure (Step-A).
Do the same for the zones
that still are undiscovered: start at some center
point and skim thru the lines connecting the center to the image border or to a
pixel at the boundary of a known zone.
In Step-A, you can also check to see whether the boundary you traced is
a line or a curve. Whenever it is a curve, you need only two points on it--
points at some distance from one another for the accuracy of the calculation.
The lines perpendicular each to these two points of tangency
intersect at the center of the circle red in your figure.
You can segment the image. Then use only the pixels in the segments to contribute to a Hough-transform to find the circles.
Then you will only have segments with circle in them. You can use a modified hough transform to find rectangles. The 'best' rectangle and square combination will then be your match. This is very computationally intentsive.
Another approach, if you already have these binary pictures, is to transform to a (for example 256 bin) sample by taking the distance to the centroid compared to the distance travelled along the edge. If you start at the point furthest away from the centroid you have a fairly rotational robust featurevector.
I've got a raster grid of values that looks something like the image below (white is high values, the black background value is zero).
I'm trying to write some kind of path-following code to start at the end of one of the lines and trace to the other end, going via the highest possible values (that is, the whiter the pixels chosen to be in the line the better) but still getting to the other end.
I've been struggling with this for a while, and can't seem to get anything I try to work. So I wondered, has a generic algorithm already been developed for this sort of problem? I've done a lot of searching, but most path algorithms seem to be designed to work on vectors/networks, not raster grids like this.
Any ideas?
The simplest idea probably is to use the A* algorithm, where each pixel is a node, and the cost of the node is the pixel darkness.
Update: Found a nice tutorial.
One way to do this:
Filter the image to get it closer to black and white only pixels.
Draw a line through the white pixels. To do this, start at a white pixel. Draw a line from that pixel to each other white pixel a distance of 2 (or 3 or so) away, but ignore pixels near a previous line. Keep going until you've covered every pixel not close (2 or 3 pixels) from a line. You'll have to do some minor adjustments here to get it to work well.
Connect the endpoints of the lines you've drawn. If there are two endpoints near (1 or 2 pixels?) one another, connect them. You should end up with a few lines made up of a lot of short segments, possibly with some loops and forks.
Get rid of any small loops in the lines, and seperate the lines at forks, so you have a few lines made of a lot of short segments.
Reduce points. For each line, check to see if it is nearly straight. If so, remove all the interior points. If not, check the two halves of the line recursively until you get down to the minimum segment lengths.
You can optionally fit a spline curve through the lines at this point.
Profit.
It will take some tweaking to get it to work well, but it is possible to do it this way. One other variant is to outline the white sections, if they are wider than 1 or 2 or 3 pixels, and combine the double lines afterward.
I don't think you'll need a genetic algorithm or anything ridiculous; good old fashion recursion and dynamic programming should suffice. I am initially thinking, that you should be able to accomplish your goal by doing a breadth first search. From your starting point, you visit all the neighbors with scores greater then that paths value --all cells start out at infinity, and costs to black cells would be infinity, and these are the paths you can prune off). Once at your destination, if reachable, you should be able to backtrack to find the path. It's greedy, but if your paths are well behaved like these are, it should be fine.
For paths with more gray and twists and turns, it might be a good idea to convert the raster image to a graph, with the edge weight being the the gray scale values of the neighbors (or difference in gray scale values, depending on what this data actually means). So, you should be able to use any algorithm for shortest paths based on that interpretation.
If you are doing this on big scale or for research you might try whit http://en.wikipedia.org/wiki/Ant_colony_optimization, but if you are doing this for money just pick up something like flood fill http://en.wikipedia.org/wiki/Flood_fill
Based on this original idea, that many of you have probably seen before:
http://rogeralsing.com/2008/12/07/genetic-programming-evolution-of-mona-lisa/
I wanted to try taking a different approach:
You have a target image. Let's say you can add one triangle at a time. There exists some triangle (or triangles in case of a tie) that maximizes the image similarity (fitness function). If you could brute force through all possible shapes and colors, you would find it. But that is prohibitively expensive. Searching all triangles is a 10-dimensional space: x1, y1, x2, y2, x3, y3, r, g, b, a.
I used simulated annealing with pretty good results. But I'm wondering if I can further improve on this. One thought was to actually analyze the image difference between the target image and current image and look for "hot spots" that might be good places to put a new triangle.
What algorithm would you use to find the optimal triangle (or other shape) that maximizes image similarity?
Should the algorithm vary to handle coarse details and fine details differently? I haven't let it run long enough to start refining the finer image details. It seems to get "shy" about adding new shapes the longer it runs... it uses low alpha values (very transparent shapes).
Target Image and Reproduced Image (28 Triangles):
Edit! I had a new idea. If shape coordinates and alpha value are given, the optimal RGB color for the shape can be computed by analyzing the pixels in the current image and the target image. So that eliminates 3 dimensions from the search space, and you know the color you're using is always optimal! I've implemented this, and tried another run using circles instead of triangles.
300 Circles and 300 Triangles:
I would start experimenting with vertex-colours (have a different RGBA value for each vertex), this will slightly increase the complexity but massively increase the ability to quickly match the target image (assuming photographic images which tend to have natural gradients in them).
Your question seems to suggest moving away from a genetic approach (i.e. trying to find a good triangle to fit rather than evolving it). However, it could be interpreted both ways, so I'll answer from a genetic approach.
A way to focus your mutations would be to apply a grid over the image, calculate which grid-square is the least-best match of the corresponding grid-square in the target image and determine which triangles intersect with that grid square, then flag them for a greater chance of mutation.
You could also (at the same time) improve fine-detail by doing a smaller grid-based check on the best matching grid-square.
For example if you're using an 8x8 grid over the image:
Determine which of the 64 grid squares is the worst match and flag intersecting (or nearby/surrounding) triangles for higher chance of mutation.
Determine which of the 64 grid-squares is the best match and repeat with another smaller 8x8 grid within that square only (i.e. 8x8 grid within that best grid-square). These can be flagged for likely spots for adding new triangles, or just to fine-tune the detail.
An idea using multiple runs:
Use your original algorithm as the first run, and stop it after a predetermined number of steps.
Analyze the first run's result. If the result is pretty good on most part of the image but was doing badly in a small part of the image, increase the emphasis of this part.
When running the second run, double the error contribution from the emphasized part (see note). This will cause the second run to do a better match in that area. On the other hand, it will do worse in the rest of the image, relative to the first run.
Repeatedly perform many runs.
Finally, use a genetic algorithm to merge the results - it is allowed to choose from triangles generated from all of the previous runs, but is not allowed to generate any new triangles.
Note: There was in fact some algorithms for calculating how much the error contribution should be increased. It's called http://en.wikipedia.org/wiki/Boosting. However, I think the idea will still work without using a mathematically precise method.
Very interesting problem indeed ! My way of analyzing such problem was usage of evolutionary strategy optimization algorithm. It's not fast and is suitable if number of triangles is small. I've not achieved good approximations of original image - but that is partly because my original image was too complex - so I didn't tried a lot of algorithm restarts to see what other sub-optimal results EVO could produce... In any case - this is not bad as abstract art generation method :-)
i think that algorithm is at real very simple.
P = 200 # size of population
max_steps = 100
def iteration
create P totally random triangles (random points and colors)
select one triangle that has best fittness
#fitness computing is described here: http://rogeralsing.com/2008/12/09/genetic-programming-mona-lisa-faq/
put selected triangle on the picture (or add it to array of triangles to manipulate them in future)
end
for i in 1..max_steps {iteration}
Photoshop has a lot of cool artistic filters, and I'd love to understand the underlying algorithms.
One algorithm that's particularly interesting is the Cutout filter (number 2 at the link above).
It has three tunable parameters, Number of Levels, Edge Simplicity, and Edge Fidelity. Number of levels appears to drive a straightforward posterization algorithm, but what the other sliders do technically eludes me.
I would think that they're doing something related to Vornoi diagrams or k-means partitionion, but poking around on wikipedia hasn't resulted in anything that maps obviously to what Photoshop is doing, especially considering how fast the filter renders itself.
Is there any source for technical descriptions of the Photoshop filters? Alternatively, do you have any thoughts about how this particular filter might be implemented?
Edge detection is usually a Sobel or Canny filter then the edges are joined together with a chain code.
Look at something like the OpenCV library for details
Did you see this post. It explains how to get the same result using ImageMagic, and IM is opensource.
Very old question but maybe someone searching for an answer and maybe this helps.
Opencv's findcontours and approxPolyDP functions can do this. But we need to prepare the image before main process.
First; find most used N colors with k-means. For example find 8 colors.Find contours for each color and then calculate contourArea for all colors one by one (We will have N=8 layers). After that draw filled contours after approxPolyDP for each color from biggest ContourArea to smaller with its pre-calculated color.
My another suggestion is eliminate very small contours while calculating contourArea.
Photoshop cutout effects parameters;
Number Of Levels=K-Means-find most used N colors.
Edge Simplicity=I guess gaussian blur or other removing noise filters like bilateral filter or meanshift filter with edge preserving will be useful for this step.This step can be executed after K-Means and before finding contours.
Edge fidelity=openCV's approxPolyDP epsilon parameter.
I'm not sure it could be some kind of cell shading, but it also looks like a median filter with a very big kernel size or which was applied several times.
The edge simplicity/fidelity might be options which help decide whether or not to take in account an adjacent pixel (or one which falls inside the kernel) based on difference of color with the current pixel.
Maybe not exactly what you are looking for, but if you like knowing how filters work, you could check out the source code of GIMP. I can't say if GIMP has an equivalent of cutout filter you mentioned, but it's worth taking a look if you are truly interested in this field.
The number of levels seems to resemble how cell-shading is done and this is how I'd implement that part in this case: you simply take this histogram of the image and divide it into the "No. of levels" amount of sections then calculate an average for each section. Each color in the histogram will then use that average in stead of their original color.
The other two parameters require some more thinking but 'Edge simplicity' seems to detonate the number of segments the shapes are build up off. Or rather: the number of refinements applied to some crude Image Segmentation Algorithms. The fidelity slider seems to do something similar; it probably controls some kind of threshold for when the refinements should take place.
This might help
Got a simple solution, which would theoretically produce something similar to that filter.
Somehow similar to what Ismael C suggested.
Edge Simplicity controls window size. Maybe window should be weighted.
But unlike it happens for regular windowed filters this one would take only a fixed size portion of random pixels from this window. The size of the portion is controlled with Fidelity parameter.
Set the pixel color to the median of the sample.
Given we have some posterization algorithm, it is applied afterwards.
Here we go!
Please report results if you implement it.
PS. I really doubt that segmentation is used at all.
I imagine it's probably some thresholding, edge-detection (Sobel/Canny/Roberts/whatever) and posterisation.
From tinkering with it I've found out that:
it's deterministic
it doesn't do any kind of pixel based posterization to achieve final effect
it probably doesn't use any kind of pixel based edge detection, it seems to work rather with areas then edges.
it calculates the shapes closed polygons to draw (some of the polygon edges might overlap with image edges).
when the edges of polygons are known then color of each area enclosed in edges (not necessarily belonging to one polygon) is colored with average color of pixels of original image that area covers.
edge of polygon can intersect with itself.
Especially visible for high edge simplicity.
as 'line simplicity' drops, the number of polygon edges increases, but also number of polygons increases.
edge fidelity influences line polygon edge count but does not influence polygon count
high edge fidelity (=3) causes single polygon to have very long and very short edges at the same time, low fidelity (=1) causes single polygon to have all edges roughly the similar length
high edge simplicity and low edge fidelity seem to prefer polygons anchored at edges of image, even at cost of sanity.
Altogether it looks like simplified version of Live Trace algorithm from Adobe Illustrator that uses polygons instead of curves.
... or maybe not.