I'm looking for a data structure that provides indexing for Rectangles. I need the insert algorithm to be as fast as possible since the rectangles will be moving around the screen (think of dragging a rectangle with your mouse to a new position).
I've looked into R-Trees, R+Trees, kD-Trees, Quad-Trees and B-Trees but from my understanding insert's are usually slow. I'd prefer to have inserts at sub-linear time complexity so maybe someone can prove me wrong about either of the listed data structures.
I should be able to query the data structure for what rectangles are at point(x, y) or what rectangles intersect rectangle(x, y, width, height).
EDIT: The reason I want insert so fast is because if you think of a rectangle being moved around the screen, they're going to have to be removed and then re-inserted.
Thanks!
I'd use a multiscale grid approach (equivalent to quad-trees in some form).
I'm assuming you're using integer coordinates (i.e. pixels) and have plenty of space to hold all the pixels.
Have an array of lists of rectangles, one for each pixel. Then, bin two-by-two and do it again. And again, and again, and again, until you have one pixel that covers everything.
Now, the key is that you insert your rectangles at the level that is a good match for the size of the rectangle. This will be something like (pixel size) ~= min(height,width)/2. Now for each rectangle you have only a handful of inserts to do into the lists (you could bound it above by a constant, e.g. pick something that has between 4 and 16 pixels).
If you want to seek for all rectangles at x,y you look in the list of the smallest pixel, and then in the list of the 2x2 binned pixel that contains it, and then in the 4x4 etc.; you should have log2(# of pixels) steps to look through. (For larger pixels, you then have to check whether (x,y) was really in the rectangle; you expect about half of them to be successful on borders, and all of them to be successful inside the rectangle, so you'd expect no worse than 2x more work than if you looked up the pixel directly.)
Now, what about insert? That's very inexpensive--O(1) to stick yourself on the front of a list.
What about delete? That's more expensive; you have to look through and heal each list for each pixel you're entered in. That's approximately O(n) in the number of rectangles overlapping at that position in space and of approximately the same size. If you have really large numbers of rectangles, then you should use some other data structure to hold them (hash set, RB tree, etc.).
(Note that if your smallest rectangle must be larger than a pixel, you don't need to actually form the multiscale structure all the way to the pixel level; just go down until the smallest rectangle won't get hopelessly lost inside your binned pixel.)
The data structures you mention are quite a mixed bag: in particular B-Trees should be fast (cost to insert grows with the logarithm of the number of items present) but won't speed up your intersection queries.
Ignoring that - and hoping for the best - the spatial data structures come in two parts. The first part tells you how to build a tree structure from the data. The second part tells you how to keep track of information at each node that describes the items stored below that node, and how to use it to speed up queries.
You can usually pinch the ideas about keeping track of information at each node without using the (expensive) ideas about exactly how the tree should be built. For instance, you could create a key for each rectangle by bit-interleaving the co-ordinates of its points and then use a perfectly ordinary tree structure (such as a B-tree or an AVL tree or a Red-Black tree) to store it, while still keeping information at each node. This might, in practice, speed up your queries enough - although you wouldn't be able to tell that until you implemented and tested it on real data. The purpose of the tree-building instructions in most schemes is to provide performance guarantees.
Two postscripts:
1) I like Patricia trees for this - they are reasonably easy to implement, and adding or deleting entries does not disturb the tree structure much, so you won't have too much work to do updating information stored at nodes.
2) Last time I looked at a window system, it didn't bother about any of this clever stuff at all - it just kept a linear list of items and searched all the way through it when it needed to: that was fast enough.
This is perhaps an extended comment rather than an answer.
I'm a bit puzzled about what you really want. I could guess that you want a data structure to support quick answers to questions such as 'Given the ID of a rectangle, return its current coordinates'. Is that right ?
Or do you want to answer 'what rectangle is at position (x,y)' ? In that case an array with dimensions matching the height and width of your display might suffice, with each element in the array being a (presumably short) list of the rectangles on that pixel.
But then you state that you need an insert algorithm to be as fast as possible to cope with rectangles moving constantly. If you had only, say, 10 rectangles on screen, you could simply have a 10-element array containing the coordinates of each of the rectangles. Updating their positions would not then require any inserts into the data structure.
How many rectangles ? How quickly are they created ? and destroyed ? How do you want to cope with overlaps ? Is a rectangle just a boundary, or does it include the interior ?
Related
I'm trying to find a spatial index structure suitable for a particular problem : using a union-find data structure, I want to connect\associate points that are within a certain range of each other.
I have a lot of points and I'm trying to optimize an existing solution by using a better spatial index.
Right now, I'm using a simple 2D grid indexing each square of width [threshold distance] of my point map, and I look for potential unions by searching for points in adjacent squares in the grid.
Then I compute the squared Euclidean distance to the adjacent cells combinations, which I compare to my squared threshold, and I use the union-find structure (optimized using path compression and etc.) to build groups of points.
Here is some illustration of the method. The single black points actually represent the set of points that belong to a cell of the grid, and the outgoing colored arrows represent the actual distance comparisons with the outside points.
(I'm also checking for potential connected points that belong to the same cells).
By using this pattern I make sure I'm not doing any distance comparison twice by using a proper "neighbor cell" pattern that doesn't overlap with already tested stuff when I iterate over the grid cells.
Issue is : this approach is not even close to being fast enough, and I'm trying to replace the "spatial grid index" method with something that could maybe be faster.
I've looked into quadtrees as a suitable spatial index for this problem, but I don't think it is suitable to solve it (I don't see any way of performing repeated "neighbours" checks for a particular cell more effectively using a quadtree), but maybe I'm wrong on that.
Therefore, I'm looking for a better algorithm\data structure to effectively index my points and query them for proximity.
Thanks in advance.
I have some comments:
1) I think your problem is equivalent to a "spatial join". A spatial join takes two sets of geometries, for example a set R of rectangles and a set P of points and finds for every rectangle all points in that rectangle. In Your case, R would be the rectangles (edge length = 2 * max distance) around each point and P the set of your points. Searching for spatial join may give you some useful references.
2) You may want to have a look at space filling curves. Space filling curves create a linear order for a set of spatial entities (points) with the property that points that a close in the linear ordering are usually also close in space (and vice versa). This may be useful when developing an algorithm.
3) Have look at OpenVDB. OpenVDB has a spatial index structure that is highly optimized to traverse 'voxel'-cells and their neighbors.
4) Have a look at the PH-Tree (disclaimer: this is my own project). The PH-Tree is a somewhat like a quadtree but uses low level bit operations to optimize navigation. It is also Z-ordered/Morten-ordered (see space filling curves above). You can create a window-query for each point which returns all points within that rectangle. To my knowledge, the PH-Tree is the fastest index structure for this kind of operation, especially if you typically have only 9 points in a rectangle. If you are interested in the code, the V13 implementation is probably the fastest, however the V16 should be much easier to understand and modify.
I tried on my rather old desktop machine, using about 1,000,000 points I can do about 200,000 window queries per second, so it should take about 5 second to find all neighbors for every point.
If you are using Java, my spatial index collection may also be useful.
A standard approach to this is the "sweep and prune" algorithm. Sort all the points by X coordinate, then iterate through them. As you do, maintain the lowest index of the point which is within the threshold distance (in X) of the current point. The points within that range are candidates for merging. You then do the same thing sorting by Y. Then you only need to check the Euclidean distance for those pairs which showed up in both the X and Y scans.
Note that with your current union-find approach, you can end up unioning points which are quite far from each other, if there are a bunch of nearby points "bridging" them. So your basic approach -- of unioning groups of points based on proximity -- can induce an arbitrary amount of distance error, not just the threshold distance.
I have a set of very many axis-aligned rectangles which maybe nested and intersecting. I want to be able to find all the rectangles that enclose/bound a query point. What would be a good approach for this?
EDIT : Additional information-
1. By very many I meant ~100 million or more.
2. The rectangles are distributed across a huge span (span of a country). There is no restriction on the sizes.
3. Yes the rectangles can be pre-processed and stored in a tree structure.
4. No real-time insertions and deletions are required.
5. I only need to find all the rectangles enclosing/bounding a given query point. I do not need the Nearest Neighbors.
As you might have guessed, this is for a real-time geo-fencing application on a mobile unit and hence -
6. The search need not be repeated for rectangles sufficiently far from the point.
I've tried KD trees and Quad-Trees by approximating each Rectangle to a point. They've given me variable performances depending on the size of the rectangles.
Is there a more direct way of doing it ? How about r trees?
I would consider using a quadtree. (Posting from mobile so it's too much effort to link, but Wikipedia has a decent explanation.) You can split at the left, right, top, and bottom bound of any rectangle, and store each rectangle in the node representing the smallest region that contains the rectangle. To search for a point, you go down the quadtree towards the point and check every rectangle that you encounter along that path.
This will work well for small rectangles, but if many rectangles cover almost the entire region you'll still have to check all of those.
You need to look at the R*-tree data structure.
In contrast to many other structures, the R*-tree is well capable of storing (overlapping) rectangles. It is not limited to point data. You will be able to find many publications on how to best approximate polygons before putting them into the index, too. Also, it scales up to pretty large data, as it can operate on disk, too.
R*-trees are faster when bulk loaded; as this can be used to reduce overlap of index pages and ensure a near-perfectly balanced tree, whereas dynamic insertions only guarantee each page to be at least half full or so. I.e. a bulk loaded tree will often use only half as much memory / storage.
For 2d data, and your type of queries, a quadtree or grid may however work just well enough. It depends on how much local data density varies.
I'll do my best to make my case scenario simple.
1-) Let's suppose we need to stores a lot of rectangles in some kind of array/data structure. Each rectangle are of different sizes and are positioned at various parts of the screen. Rectangles can't overlap together.
2-) Then we click on the screen with the mouse at point [x,y].
Now, we need to determine if we clicked on a part of one of the rectangles. Well, that would be insane to iterate through all the rectangles to make some kind of comparisons, especially if there is a huge number of them.
What would be the fastest technique/algorithm to do it with as little steps as possible? What would be the best data structure to use in such case?
One way would be to use a quadtree to store the rectangles: The root represents the whole area that contains all rectangles, and this area is then recursively subdivided as required.
If you want to test if a certain coordinate is within one of the rectangles, you start at the root, and walk down the tree until either you find a rectangle or not.
This can be done in O(log n) time.
So working on an little project but thinking about making maps efficient. I have a grid of numbers say
100110
011011
010110
If you've ever played dungeon keeper, the idea is a 0 is a flat dug out square, and 1 is a still standing square.
I want to take advantage of the grid layout and be able to minimise the number of vertexes used. So instead of using individuals cubes for an area like:
1111
1111
1111
I want to just use 8.
Any idea on the best approach to this? or even just knows the name of the type of algorithm i should use. Something that can do it quickly on the fly would be preferable so not to bottle neck rendering.
I agree that this is probably not gonna be a performance issue, but you could represent your map in a compressed map by using a (slightly modified) unbalanced Quad-tree.
Start by your map consisting only of 1's. You can store this as a box of size n*n in the root node of your tree.
IF you want to dig out one of the boxes you recursively walk down the tree, splitting the n*n box (or whatever you find there) using the default quad tree rules (= split an n*n box into four n/2*n/2 boxes, etc.). At some point you'll arrive in a leaf of the tree that only contains the single box (the one you want to dig out) and you may change it from 1 to 0.
Additionally, after the leaf has changed and your recursive calls return (= you walk back up the tree towards the root node), you can check neighboring boxes for whether they may be merged. (If you have two neighboring boxes that are both dug out, you can merge them).
Another technique that is sometimes used when indexing low-dimensional data like this is a space filling curve. One that has good average locality and is reversible is the Hilbert curve. Basically, you may enumerate your boxes (dug out ones and filled ones) along the space filling curve and then use simple run-length compression.
The tree-idea allows you to reduce the number of rendered geometry (you can rescale texture, etc. to emulate n*n boxes by a single larger box). The space filling curve probably will only save you some memory.
I've read so many articles, but none seem to answer this question. Or maybe I'm just not understanding. I'm attempting to build a quadtree so that it can represent an image. The leaf nodes are to hold pixels, and non-leaf nodes will hold the average value pixel of its children.
My question is:
How does it work that the leaf nodes only hold pixels? Why don't the other nodes hold pixels? And how do we know how many times to subdivide our original root node to represent that given image? Do we just subdivide it n times, where n is the height and width (for a square)?
Edit: So how do I keep track of leaf nodes, so I know when to add pixels at that location? Right now I have a helper function that divides the regions for me, keeping track of width and height.
Quadtrees work best for square images whose size is a power of 2 (for example, most textures). You shouldn't think of each node as representing a "pixel". Instead, think of it as representing a "square block of pixels of size 2^k". In the case of final leaves, k is 0, so each leaf node represents a square block of pixels of size 1, that is, a single pixel. Internal nodes in the tree represent increasingly large sections of image.
Why do only leaf nodes hold pixels? Ask yourself if a non-leaf node held a pixel, then what would its children hold? Since you can't subdivide a pixel, the answer is obviously nothing -- there can be no such nodes.
How do we know how many times to subdivide? Well, there are multiple ways to do it, of course, depending on why you're building the quadtree. In general, the areas of the image with more entropy -- more "detail" -- should be subdivided more, while the lower-entropy, "flatter" areas can be divided less. There are a number of different algorithms for choosing when and where to subdivide. Generally, they compare pixel values within a region, and split when the differences are above some threshold.
how does it work that the leaf nodes
only hold pixels? Why dont the other
nodes hold pixels?
This depends on what you're using the Quadtree for. You can link any kind of information to the other nodes, f.e. a pointer to the upper-left corner and the width/height of the rectangle this node describes, but you won't need it in most cases (or need things like the average values you can precompute to speed things up).
And how do we know how many times to
subdivide our original root node to
represent that given image?
With every subdivision, you half the width and height of a region, so for a square image of size n you'll need to subdivide log2(n) times, for a non-square image of size n*m you'll need at most max(log2(n), log2(m)) steps.
I think the best way to answer your question is to answer two questions that you didn't ask.
What is a quadtree?
How can this be
applied to modelling systems of
erratic density?
A quadtree is a binary tree in two dimensions. That's why there are (up to) four children for every non-leaf node. This allows you to apply an index to a plane just as a database uses a binary-tree or some variation thereof to index a single dimension, with the same highly advantageous sparse phase space representation properties.
The application of this to image compression and progressive display is pretty obvious: if you do a tree-walk limited to a depth of n then you get 4^n items of picture info spanning the entire image space. If you one level deeper, each pixel splits into four. JPEG2000 works like this, if I recall correctly. I said "items of picture info" because they need not be single bit; items could be 32bit ARGB or any other property (or properties) describing the space at that point.