Divide an image by grouping similar pixels into rectangles - algorithm

consider an image like this:
by grouping pixels by color into distinct rectangles, different configurations might be achieved, for example:
the goal is to find one of the best configurations, i.e. a configuration which has the least possible number of rectangles (rectangles sizes are not important).
any idea on how to design an efficient algorithm which is able to solve this problem?
EDIT:
i think the best answer is the one by #dshin, as they proved that this problem is a NP-HARD one so there probably isn't any efficient solution that is able to guarantee an optimal result.
other answers provide reasonable compromises to get an acceptable solution, but that won't always be the optimal one.

Each connected colored region is a rectilinear polygon that can be considered independently, and so your problem amounts to solving the minimum rectangle covering for rectilinear polygons. This is a well-studied problem that finds applications in some fields, like VLSI.
For convex rectilinear polygons, there is an algorithm that finds the optimal solution in polynomial time, described in this 1984 thesis.
The non-convex case is NP-hard (reference), so an efficient optimal solution likely does not exist. But there are several algorithms which produce good empirical results. This 1990 publication describes three separate algorithms, each of which are guaranteed to use at most twice as many rectangles as the optimal solution. This 2016 publication describes an algorithm that uses the common IP + LP relaxation technique, which apparently produces better results in real-life problem instances, although lacking in theoretical guarantees. Unfortunately, both publications are behind paywalls, and I haven't been able to find free resources that describe the algorithms.
If you are just looking for something reasonable, and your problem instances are not pathological in nature, then the algorithms described in other answers are probably good enough.

I don't have a proof but my feeling is a greedy approach should solve this problem:
Start on the upper left (or in whichever corner)
Expand rectangle 1px to the right as long as colors match
Expand rectangle 1px to the bottom as long as all colors in that row match
Line by line and column by column, find the next pixel that is not already part of a square (maybe keep track of visited pixels in a second array) and repeat 2 and 3.
You can switch lines and columns and go up and left or whatever instead and end up with different configurations, but from playing this through in my mind I think the number of rectangles should always be the same.

The idea here is based on the following links: Link 1 and Link 2.
In both the cases, the largest possible rectangle is computed within a given polygon/shape. Check both the above links for details.
We can extend the idea above to the problem at hand.
Steps:
Filter the image by color (say red)
Find the largest possible rectangle in the red region. After doing so mask it.
Repeat to find the next biggest rectangle until all the portions in red have been covered.
Repeat the above for every unique color.
Overview:

Related

Reverse Rectangle Packing

I have a connected shape that consists of squares put together, e.g. take a squared paper and draw a line along the existing lines that ends at its beginning and does not cross itself.
The goal is now to find an algorithm (not brute-force) that fills this shape with as few, non-overlapping rectangles as possible.
I'm looking for the optimal solution. As can be seen in the images, the naive greedy approach (take the largest rectangle) does not work.
(Optimal)
(Greedy)
My scenario is vertices reduction, but I'm sure there are other use-cases as well.
Note: This problem seems basic, but I was not able to find a solution elsewhere. Also, is this problem NP-hard?
Edit: I just realized that, in my scenario, filling the shape with as few non-overlapping triangles as possible, would give an even better result.
I've spend a lot of time researching this, since I asked the initial question. For the first problem (optimally filling the shape with rectangles), I've written the solution here under the header "Optimal Greedy Meshing":
http://blackflux.wordpress.com/2014/03/01/meshing-in-voxel-engines-part-2/
The complexity is actually better (faster) than for optimally triangulating a polygon without holes. The slowest part is the Hopcroft-Karp algorithm.
Treating the shape as a polygon is also discussed in the linked blog post. Note that I'm also considering holes.
The first problem is harder than the one with triangles; for triangles, see the algorithms in
http://en.wikipedia.org/wiki/Polygon_triangulation
which can do it without any extra vertices.

Cutting Heuristic Solving Algorithm for n-edged polygone

I want to programm a tool that can place objects on a rectangle with the minumum of waste, this problem is also known as the cutting problem.
So i looked around to find some algorithms and i found out there are a few for rectangles but not that much for n-edged polygones.
my first approach was to get a bounding box for the polygone, then run the normal rectangle algorithm. After that you cound slowly try to increase the number of edges but still have only isometric lines (only vertical and horizontal), to approximate the polygone.
I wonder if there is any good algorithm that implement such thing, but is more common than create my own stuff.
the other way ive come up with could be something with two dimensional knapsack and some sorting heuristics that sort the best fitting polygones and try to put them on the rectangle.
But all i come up with has some good detection of special polygones (such as a square or normal rectangle) but does not work on common polygones.

Algorithm to align and compare two sets of vectors which may be incomplete and ignoring scaling?

Here is the problem:
I have many sets of points, and want to come up with a function that can take one set and rank matches based on their similarity to the first. Scaling, translation, and rotation do not matter, and some points may be missing from any of the sets of points. The best match is the one that if scaled and translated in the ideal way has the least mean square error between points (maybe with a cap on penalty, or considering only the best fraction of points to handle missing points).
I am trying to come up with a good way to do this, and am wondering if there are any well known algorithms that can handle this type of problem? Just the name of something would be awesome! I lack a formal CSCI or math education, and am doing the best to teach myself.
A few things I have tried
The first thing that comes to mind is to normalize the points somehow, but I dont think that this is helpful because the missing points may throw things off.
The best way I can think of is to estimate a starting point by translating to match their centroids, scaling so that the largest distances from the centroid of the sets match. From there, do an A* search, scaling, rotating, and translating until I reach a maximum, and then compare the two sets. (I hope I am using the term A* correctly, I mean trying small translations and scalings and selecting the move giving the best match) I think this will find the global maximum most of the time, but is not guaranteed to. I am looking for a better way that will always be correct.
Thanks a ton for the help! It has been fun and interesting trying to figure this out so far, so I hope it is for you as well.
There's a very clever algorithm for identifying starfields. You find 4 points in a diamond shape and then using the two stars farthest apart you define a coordinate system locating the other two stars. This is scale and rotation invariant because the locations are relative to the first two stars. This forms a hash. You generate several of these hashes and use those to generate candidates. Once you have the candidates you look for ones where multiple hashes have the correct relationships.
This is described in a paper and a presentation on http://astrometry.net/ .
This paper may be useful: Shape Matching and Object Recognition Using Shape Contexts
Edit:
There is a couple of relatively simple methods to solve the problem:
To combine all possible pairs of points (one for each set) to nodes, connect these nodes where distances in both sets match, then solve the maximal clique problem for this graph. Since the maximal clique problem is NP-complete, the complexity is probably O(exp(n^2)), so if you have too many points, don't use this algorithm directly, use some approximation.
Use Generalised Hough transform to match two sets of points. This approach has less complexity (O(n^4)). But it is more complicated, so I cannot explain it here.
You can find the details in computer vision books, for example "Machine vision: theory, algorithms, practicalities" by E. R. Davies (2005).

Finding a maximal square from a finite set of tiles (approximation)

I have a final set of tiles in which every edge can have on of four colors.
The task is to find a maximal possible square build from a given set (finite) of this tiles. Tiles can be rotated.
I need to design 3 algorithms for finding a solution for this task. One complete and two aproximations.
Obviously it is my task for Algorithms class so Im not asking about complete solutions (as this would be unfair) but for some directions.
Im already designed a kind of complete algorithm (using backtracking - search for a square of size sqrt(n) - if it could not be found try finding smaller and so on) but I have no idea how to create aproximation algorithms. I think one will be kind of stupid which will find a good answer only in specific cases just to document that it is not a good aproach but still I need one much faster then backtracking and quite good one.
Also is this problem NP-hard one? My backtracking algorithm is exponential one but it doesnt mean that there cannot be a better one...
EDIT: I have complete algorithm with exponential time, could some one give me some hints how to build some kind of aproximation for this problem with polynomial time or something better then exponential?
EDIT2: I have the idea that this problem can be changed to a problem of reducting a graph to square grid graph ( http://mathworld.wolfram.com/GridGraph.html ). Still there is a problem if the tiles can be arranged in such a way to build a grid, but this could be a good point to start. Are there any, for example, greedy or any other aproximation algorithms for reducting graph to square-grid graph?
Suppose your backtracking algorithm constructs k-by-k squares for increasing values of k.
You can extend the backtracking algorithm with heuristics. So instead of choosing the next tile randomly, choose and attach a tile such that the colors of the free tiles "agree with" those on the square. The big problem is to find the "agreement" heuristics. One possible heuristics is to find the least common color on the free tiles and use it.

Fill arbitrary 2D shape with given set of rectangles

I have a set of rectangles and arbitrary shape in 2D space. The shape is not necessary a polygon (it may be a circle), and rectangles have different widths and heights. The task is to approximate the shape with rectangles as close as possible. I can't change rectangles dimensions, but rotation is permitted.
It sounds very similar to packing problem and covering problem but covering area is not rectangular...
I guess it's NP problem, and I'm pretty sure there should be some papers that show good heuristics to solve it, but I don't know what to google? Where should I start?
Update: One idea just came into my mind but I'm not sure if it's worth investigating. What if we consider bounding shape as a physical mold filled with water. Each rectangle is considered as a positively charged particle with size. Now drop the smallest rectangle to it. Then drop the next by size at random point. If rectangles too close they repel each other. Keep adding rectangles until all are used. Could this method work?
I think you could look for packing and automatic layout generation algorithms. Automatic VLSI layout generation algorithms might need similar things, just like textile layout questions...
This paper Hegedüs: Algorithms for covering polygons by rectangles seems to address a similar problem. And since this paper is from 1982, it might be interesting to look at the papers which cite this one. Additionally, this meeting seems to be discussing research problems related to this, so might be a starting point for keywords or names who do research in this idea.
I don't know if the computational geometry research has algorithms for your specific problem, or if these algorithms are easy/practical enough to implement. Here is how I would approach it if I had to do it without being able to look up previous work. This is just a direction, by far not a solution...
Formulate it as an optimization problem. You have discrete variables of which rectangles you choose (yes or no) and continuous variables (location and orientation of the triangles). Now you can set up two independent optimizations: a discrete optimization which picks the rectangles; and a continuous that optimizes for the location and orientation once rectangles are given. Interleave these two optimizations. Of course the difficulty lies in the formulation of optimizations, and designing your error energy such that it does not get stuck in some strange configurations (local minima). I'd try to get the continuous as a least squares problem such that I can use standard optimizations libraries.
I think this problem is suitable for solving with genetic algorithm and/or evolutionary strategy algorithm. I've done similar box packing problem with the help of evolutionary strategy algorithm of some kind. Check this out in my blog.
So if you will use such approach - encode into chromosomes box:
x coordinate
y coordinate
angle
Then try to minimize such fitness function-
y = w1 * box_intersection_area +
w2 * box_area_out_of_shape +
w3 * average_circle_radius_in_free_space
Choose weights w1,w2,w3 such as to affect importance of factors. When genetic algorithm will find partial solution - remove boxes which still overlaps together or are out of shape - and you will have at least legal (but not necessary optimal) solution.
good luck in this interesting problem !
It is NP hard indeed and since it has hi-tech application, reasonably efficients approximate strategies are not even in patents, let alone published papers.
The best you can do with a limited budget is to start by limiting the problem. Assume that all rectangles are exactly the same, Assume that all rectangles which are binary sub-divisions of your standard rectangle are also allowed since you can efficiently pre-pack them to fit your core division. For extra points you can also form several fixed schemas for gluing core rectangles to cover a few larger shapes with substantially different proportions. Assume that you can change dimensions of your standard rectangle/cell as long as the rest (pre-packing and gluing schema) remains the same - this gives you parameters to decide approximate size of the core rectangle based on rectangles you are given.
Now you can play with aspect ratios to approximate the error such limited system could guarantee. For the first iterations assume that it can have 50% error with a simple sub-division schema and then change schema to reduce the error but without increasing asymptotic complexity of pre-packing. At the end of the day you are always just assigning given rectangles to your pre-calculated and now fixed grid and binary sub-divisions - meaning you are not trying to do a layout or backtrack at all - you are always happy with the first approximate fit into the grid.
Work on defining classes of rectangles that pack well with your schema - that's again to keep the whole process inverted - you are never trying to actually fit what you are given - you are defining what you have to be given in order to fir it well - then you punt the rest as error since it is approximation.
Then you can try to do a bit more, but not much more - any slip into backtracking or nailing arbitrary small error and it's exponential.
If you are at a research facility and can get some supercomputer time - run a set of exhaustive searches with pathological mixes there just to see how optimal packing may look like and to see if you can derive a few more sub-division schemas and/or classes of rectangle sets.
That should be enough for the first 2 yrs or research :-)

Resources