I have an edge map of a scene and would like to extract the edge which best separates the sky and terrain. This seems to be well framed as a graph traversal problem. However, popular search algorithms such as A* are reliant upon the use of a starting and ending point (other than the first and last column respectively). Are there any algorithms for graph search which do not require these parameters? I would also like to maximize some global features of the extracted edge such as smoothness.
Note: speed is a significant issue, this needs to be done in real time.
Computer vision researchers have attacked this type of problem with minimum cuts. Wikipedia has a whole article about graph cuts in computer vision. I'll sketch here the algorithm proposed by Greig, Porteous, and Seheult, who were the first to make this connection.
Suppose that we have a function from pixel colors to log likelihoods of how likely that pixel is to be sky versus terrain. Prepare a graph with a source vertex, a sink vertex, and a vertex for each pixel. Connect the source to each pixel with capacity equal to the log likelihood of that pixel being sky. Connect each pixel to the sink with capacity equal to the log likelihood of that pixel being terrain. For each pair of adjacent pixels, connect them with capacity equal to the log likelihood of them having different classifications. Compute a minimum cut. All of the vertices on the source side of the cut are classified as sky, and all of the vertices on the sink side of the cut are classified as terrain.
Alternatively, if the terrain is known to be at the bottom of the image and the sky is known to be at the top, connect the source instead to each of the top pixels and connect the bottom pixels to the sink, with infinite capacity. Then we can dispense with the log likelihoods for classifying pixels based on color, leaving the edge capacities to vary with the similarity of adjacent pixel colors.
Related
Is there any algorithm that would allow to approximate a path on the x-y plane (i.e. an ordered suite of points defined by x and y) with a limited number of line segments and arcs of circles (constant curvature)? The resulting curve needs to be C1 (continuity of slope).
The maximum number or segments and arcs could be a parameter. An additional interesting constraint would be to prevent two consecutive circles of arcs without an intermediate line segment joining them.
I do not see any way to do this, and I do not think that there exists a method for it, but any hint towards this objective is welcome.
Example:
Sample file available here
Consider this path. It looks like a line, but is actually an ordered suite of very close points. There is no noise and the order of the sequence of points is well known.
I would like to approximate this curve with a minimum number of succession of line segments and circular arcs (let's say 10 line segments and 10 circular arcs) and a C1 continuity. The number of segments/arcs is not an objective itself but I need any parameter which would allow to reduce/increase this number to attain a certain simplicity of the parametrization, at the cost of accuracy loss.
Solution:
Here is my solution, based on Spektre's answer. Red curve is original data. Black lines are segments and blue curves are circle arcs. Green crosses are arc centers with radii shown and blue ones are points where segments potentially join.
Detect line segments, based on slope max deviation and segment minimal length as parameters. The slope of the new segment step is compared with the average step of the existing segment. I would prefer an optimization-based method, but I do not think that it exists for disjoint segments with unknown number, position and length.
Join segments with tangent arcs. To close the system, the radius is chosen such that the segments extremities are the least possible moved. A minimum radius constraint has been added for my purposes. I believe that there will be some special cases to treat in the inflexion points are far away when (e.g. lines are nearly parallel) and interact with neigboring segments.
so you got a point cloud ... for such Usually points close together are considered connected so:
you need to add info about what points are close to which ones
points close only to 2 neighbors signaling interior of curve/line. Only one neighbor means endpoint of curve/lines and more then 2 means intersection or too close almost or parallel lines/curves. No neighbors means either noise or just a dot.
group path segments together
This is called connected component analysis. So you need to form polylines from your neighbor info table.
detect linear path chunks
these have the same slope among neighboring segments so you can join them to single line.
fit the rest with curves
Here related QAs:
Finding holes in 2d point sets?
Algorithms: Ellipse matching
How approximation search works see the sublinks there are quite a bit of examples of fitting
Trace a shape into a polygon of max n sides
[Edit1] simple line detection from #3 on your data
I used 5.0 deg angle change as threshold for lines and also minimal size fo detected line as 50 samples (too lazy to compute length assuming constant point density). The result looks like this:
dots are detected line endpoints, green lines are the detected lines and white "lines" are the curves so I do not see any problem with this approach for now.
Now the problem is with the points left (curves) I think there should be also geometric approach for this as it is just circular arcs so something like this
Formula to draw arcs ending in straight lines, Y as a function of X, starting slope, ending slope, starting point and arc radius?
And this might help too:
Circular approximation of polygon (or its part)
the C1 requirement demands the you must have alternating straights and arcs. Also realize if you permit a sufficient number of segments you can trivially fit every pair of points with a straight and use a tiny arc to satisfy slope continuity.
I'd suggest this algorithm,
1 best fit with a set of (specified N) straight segments. (surely there are well developed algorithms for that.)
2 consider the straight segments fixed and at each joint place an arc. Treating each joint individually i think you have a tractable problem to find the optimum arc center/radius to satisfy continuity and improve the fit.
3 now that you are pretty close attempt to consider all arc centers and radii (segments being defined by tangency) as a global optimization problem. This of course blows up if N is large.
A typical constraint when approximating a given curve by some other curve is to bound the approximate curve to an epsilon-hose within the original curve (in terms if Minkowski sum with a disk of fixed radius epsilon).
For G1- or C2-continuous approximation (which people from CNC/CAD like) with biarcs (and a straight-line segment could be seen as an arc with infinite radius) former colleagues of mine developed an algorithm that gives solutions like this [click to enlarge]:
The above picture is taken from the project website: https://www.cosy.sbg.ac.at/~held/projects/apx/apx.html
The algorithm is fast, that is, it runs in O(n log n) time and is based on the generalized Voronoi diagram. However, it does not give an approximation with the exact minimum number of elements. If you look for the theoretical optimum I would refer to a paper by Drysdale et al., Approximation of an Open Polygonal Curve with
a Minimum Number of Circular Arcs and Biarcs, CGTA, 2008.
I'm trying to code the livewire algorithm but I'm a little stuck because the algorithm explained in the article "Intelligent Scissors for Image Composition" is a little messy and I don't understand complety how to apply certain things for example: How to calculate de local cost map and other stuff.
So please can anyone give a hand and explain it step by step in just simple words?
I would apreciate any help
Thanks.
You should read Mortensen, Eric N., and William A. Barrett. "Interactive segmentation with intelligent scissors." Graphical models and image processing 60.5 (1998): 349-384. which contains more details about the algorithm than the shorter paper "Intelligent Scissors for Image Composition."
Here is a high-level overview:
The Intelligent Scissors algorithm uses a variant of Dijkstra's graph search algorithm to find a minimum cost path from a seed pixel to a destination pixel (the position of the mouse cursor during interactive segmentation).
1) Local costs
Each edge from a pixel p to a pixel q has a local cost, which is a linear combination of the local costs (adjusted by the distance between p and q to account for diagonal pixels):
Laplacian zero-crossing f_Z(q)
Gradient magnitude f_G(q)
Gradient direction f_D(p,q)
Edge pixel value f_P(q)
Inside pixel value f_I(q)
Outside pixel value f_O(q)
Some of these local costs are static and can be computed offline. f_Z and f_G are computed at different scales (meaning with different size kernels) to better represent the edge a pixel q. f_G, f_P, f_I, f_O are dynamically (or have a dynamic component as is the case for f_G) computed for on-the-fly training.
2) On-the-fly training
To prevent snapping to a different edge with a lower cost than the current one being followed, the algorithm uses on-the-fly training to assign a lower cost to neighboring pixels that "look like" past pixels along the current edge.
This is done by building a histogram of image value features along the last 64 or 128 edge pixels. The image value features are computed by scaling and rounding f'_G (where f_G = 1 - f'_G), f_P, f_I, and f_O as to have integer values in [0 255] or [0 1023] which can be used to index the histograms.
The histograms are inverted and scaled to compute dynamic cost maps m_G, m_P, m_I, and m_O. The idea is that a low cost neighbor q should fit in the histogram of the 64 or 128 pixels previously seen.
The paper gives pseudo code showing how to compute these dynamic costs given a list of previously chosen pixels on the path.
3) Graph search
The static and dynamic costs are combined together into a single cost to move from pixel p to one of its 8 neighbors q. Finding the lowest cost path from a seed pixel to a destination pixel is done by essentially using Dijkstra's algorithm with a min-priority queue. The paper gives pseudo code.
How would you go about simplifying a maze that is represented in terms of pixels into nodes based on the same color? I'm wring a program that solves a maze (represented as an image) using A* Algorithm. The walls are represented as black pixels and the rest as white pixels. However, I'm worried that the space complexity will be very high for large mazes. Therefore I'm trying to come up with a way to maybe group pixels of the same color together in order to create a graph of nodes which represents the entire matrix. That way when I run A* I can go node by node instead of pixel by pixel which is much more simple.
I'm looking for an algorithm that converts a regular grid of heights (e.g. 1024x1024) to a triangular irregular network. Here is an image showing an example of a triangular irregular network:
I've looked in the internet for an algorithms to convert it, but I just couldn't find one. Basically the triangle density depends on the roughness and/or pixel error (when rasterized), or something like that.
Here's an idea for a two-step algorithm: Do a Delaunay triangulation based on a rough mesh first, then smoothe out the triangles recursively until a certain error criterion is met.
For the first step, identify a set of vertices for the Delaunay triangulation. These vertices coincide with pixel coordinates. Extreme points that are either higher or lower than all four neighbouring pixels should be in the set as should ridge points on the borders where the adjacent pixels along the border are either lower or higher. This should give a coarse triangular mesh. You can get a finer mesh by also including pixels that have a high curvature.
In the second step, iterate through all triangles. Scan through the triangle along the pixel grid and accumulate an error square for each pixel inside the triangle and also identify the points of maximum and minimum signed error. If the average error per pixel does not meet your criterion, add the points of lowest and highest error to your triangulation. Verify the new triangles and re-triangulate as necessary.
Notes:
The coarse triangulation in step one should be reasonably fast. If the height map is ragged, you might end up with too many vertices in the ragged area. In that case, the hight map might be smoothed with a Gaussian filter before applying the algorithm.
The recursive re-triangulation is probably not so fast, because determining the error requires scanning the triangles over and over. (The process should get faster as the triangle size decreases, but still.) A good criterion for finding vertices in step 1 might speed up step 2.
You can scan a triangle by finding the bounding box of pixels. Find the barycentric coordinates s, t of the lower left point of the bounding box and also the barycentric increments (dsx, dtx) and (dsy, dty) that correspond to a pixel move in the x and y directions. You can then scan the bounding box in two loops over the included pixels (x, y), calculate the barycentric coordinates (s, t) from your delta vectors and accumulate the error if you are inside the triangle, i.e. when s > 0, t > 0 and s + t < 1.
I haven't implemented this algorithm (yet - it is an interesting task), but I imagine that finding a good balance between speed and mesh quality is a matter of tailoring error criteria and vertex selection to the current height map.
what is the efficient way to understand that a curve is closed or not?
maybe one way is flood fill algorithm and use it to check it ;If your flood fill leaves a pre-determined bounding box, you are outside the shape. Otherwise if your flood fill terminates, then you're within the shape.
but is it an efficient way?
tnx.
Look at a curve as a graph, vertices are pixels and edges are between neighbouring pixels. Testing is done:
Simple curve is if all vertices have two neighbours and graph is connected.
More not intersected simple curves is if all vertices have two neighbours and graph isn't connected. Number of part is find with graph partitioning.
8 curve is if all vertices except one have 2 neighbours, that one have 4 neighbours, and graph is connected.
...
Testing graph/subgraph connectivity and partitioning is done by graph traversal.
Could you walk along the curve with two separate pointers?
If so, do that and set one pointer to traverse twice as fast.
If the loop is closed, the pointers will overlap at one point.
This should be O(n)..
Lets say the degree of each pixel is the number of pixels in its neighborhood.
Walk through your pixel array,
if any pixel has odd degree, then the curve is not closed.
Explanation:
For an even degree pixel, for each path that enters, there is a path that leaves it.
This is not true for odd degrees.