I have a polygon soup of triangles that I would like to construct a BSP tree for. My current program simply constructs a BSP tree by inserting a random triangle from the model one at a time until all the triangles are consumed, then it checks the depth and breadth of the tree and remembers the best score it achieved (lowest depth, lowest breadth).
By definition, the best depth would be log2(n) (or less if co-planar triangles are grouped?) where n is the number of triangles in my model, and the best breadth would be n (meaning no splitting has occurred). But, there are certain configurations of triangles for which this pinnacle would never be reached.
Is there an efficient test for checking the quality of my BSP tree? Specifically, I'm trying to find a way for my program to know it should stop looking for a more optimal construction.
Construction of an optimal tree is an NP-complete problem. Determining if a given tree is optimal is essentially the same problem.
From this BSP faq:
The problem is one of splitting versus
tree balancing. These are mutually
exclusive requirements. You should
choose your strategy for building a
good tree based on how you intend to
use the tree.
Randomly building BSP trees until you chance upon a good one will be really, really inefficient.
Instead of choosing a tri at random to use as a split-plane, you want to try out several (maybe all of them, or maybe a random sampling) and pick one according to some heuristic. The heuristic is typically based on (a) how balanced the resulting child nodes would be, and (b) how many tris it would split.
You can trade off performance and quality by considering a smaller or larger sampling of tris as candidate split-planes.
But in the end, you can't hope to get a totally optimal tree for any real-world data so you might have to settle for 'good enough'.
Try to pick planes that (could potentially) get split by the most planes as splitting planes. Splitting planes can't be split.
Try to pick a plane that has close to the same number of planes in front as in back.
Try to pick a plane that doesn't cause too many splits.
Try to pick a plane that is coplanar with a lot of other surfaces
You'll have to sample this criteria and come up with a scoring system to decide which one is most likely to be a good choice for a splitting plane. For example, the further off balance, the more score it loses. If it causes 20 splits, then penalty is -5 * 20 (for example). Choose the one that scores best. You don't have to sample every polygon, just search for a pretty good one.
Related
I have a problem understanding this algorithm, labeled "Algorithm 1" from this paper.
It says:
If at leaf, check for intersection with contained triangles.
What am I missing? As far as I know, the kd-tree nodes only hold one value and pointers to left and right children. Do you know any reference kd-tree structures for me to investigate? At insert, I calculate middlepoints for each axis and based on that, I place one triangle for each node.
In case of GPU Ray Tracing, KD-Trees are used as acceleration structures. We group geometry into larger chunks to cull ray misses early. However, it might happen, that the tree gets too deep at certain nodes. To avoid this, we can limit the height of the tree. This is why we might end up with more geometry in a leaf.
Note, that Algorithm 3 also mentions this case.
The exit step
24: Intersect ray with contained geometry
Clarifying "too deep":
When a branch goes much deeper than it's neighbours, the threads diverge, causing performance degradation .
More on the topic here: Nvidia RTX best practices
Storing only one triangle per node is probably sub-optimal. Generally, it is faster to iterate through a moderately sized bucket of objects than to navigate the same sized binary tree.
Also, there is no guarantee that you will be able to sort out all your triangles such that your k-D tree has one per node. Hierarchical bounding volumes (where, unlike k-D trees, sibling volumes may overlap) should be more flexible in this respect, but they are also conventionally used with buckets of target objects, rather than one object per node.
Problem
I'm working with openstreetmap-data and want to test for point-features in which polygon they lie. In total there are 10.000s of polygons and 100.000.000 of points. I can hold all this data in memory. The polygons usually have 1000s of points, hence making point-in-polygon-tests is very expensive.
Idea
I could index all polygons with an R-Tree, allowing me to only check the polygons whose bounding-box is hit.
Probable new problem
As the polygons are touching each other (think of administrative boundaries) there are many points in the bounding-box of more than one polygon, hence forcing many point-in-polygon-tests.
Question
Do you have any better suggestion than using an R-Tree?
Quad-Trees will likely work worse than rasterization - they are essentially a repeated rasterization to 2x2 images... But definitely exploit rasterization for all the easy cases, because testing the raster should be as fast as it gets. And if you can solve 90% of your points easily, you have more time for the remaining.
Also make sure to first remove duplicates. The indexes often suffer from duplicates, and they are obviously redundant to test.
R*-trees are probably a good thing to try, but you need to really carefully implement them.
The operation you are looking for is a containment spatial join. I don't think there is any implementation around that you could use - but for your performance issues, I would carefully implement it myself anyway. Also make sure to tune parameters and profile your code!
The basic idea of the join is to build two trees - one for the points, one for the polygons.
You then start with the root nodes of each tree, and repeat the following recursively until the leaf level:
If one is a non-directory node:
If the two nodes do not overlap: return
Decide by an heuristic (you'll need to figure this part out, "larger extend" may do for a start) which directory node to expand.
Recurse into each new node, plus the other non-opened node as new pair.
Leaf nodes:
fast test point vs. bounding box of polygon
slow test point in polygon
You can further accelerate this if you have a fast interior-test for the polygon, in particular for rectangle-in-polygon. It may be good enough if it is approximative, as long as it is fast.
For more detailed information, search for r-tree spatial join.
Try using quad trees.
Basically you can recursivelly partion space into 4 parts and then for each part you should know:
a) polygons which are superset of given part
b) polygons which intersect given part
This gives some O(log n) overhead factor which you might not be happy with.
The other option is to just partion space using grid. You should keep same information or each part of the grid as in the case above. This does only have some constant overhead.
Both this options assume, that the distribution of polygons is somehow uniform.
There is an other option, if you can process points offline (in other words you can pick the processing order of points). Then you can use some sweeping line techniques, where you sort points by one coordinate, you iterate over points in this sorted order and maintain only interesting set of polygons during iteration.
I am curious to know why the Watts-Strogatz random graph generation model uses a ring
lattice in its algorithm.
I am creating a spatially embedded network, where nodes are randomly placed on a grid. Each
node will connect to its k-nearest neighbors. Then, at random with probability p, connections
are rewired.
In principle, this sounds exactly the same as the Watts-Strogatz algorithm, but nodes are
not neatly organised in a lattice. In terms of the logical topology, are there any significant
differences?
To answer your first question (why using a ring): to my opinion, they used a ring lattice because it's the simplest form of lattice, and they didn't need to use a more complex form to illustrate their point. By using the ring as a starting point and by applying their rewiring process, they showed they could obtain the desired topological properties.
For your second question (regarding your own method), I think the effect depends on the spatial distribution of the nodes. Also, what is the exact rule you use to create a link between two nodes? Do both nodes need to be among the k-nearest neighbors of one another? (in which case the maximal degree is k), or do you apply only a unilateral condition? (and then, the degree can be much larger than k depending on the spatial distribution).
I am doing some research into how feasible it is to use voxels to represent largish (256x256x256 voxels) battlegrounds with destructible terrain for server-hosted multiplayer games. Only one battleground will exist for any game at a time. However, to be able to broadcast rooms and changes to their terrain, I am trying to find an algorithm that can group the voxels into the fewest rectangular blocks as possible.
As a simplistic example, if the bottom half of the level was completely filled with voxels of one type and the top half with voxels of another type, the level should be divided into two blocks, one representing the bottom half of the level, and the other representing the top. Ideally, this algorithm should be able to run in real time so any deformation in the terrain could be accounted for on a per-frame basis and broadcast to the clients. This should enable the clients to render the terrain efficiently without worrying about duplicating terrain destruction logic in the clients.
Here are the approaches I have tried and the problems I find with them. The block counts reported are for filling a 256^3 space by randomly dropping more than 4,194,304 "EARTH" voxels into as far close to the bottom as they can go for the randomly selected (x,z) coordinate. Only "EARTH" blocks are counted.
Octrees: very fast, but if I split at the middle of a space, it produces a ridiculously large number of blocks (800,000+).
k-D trees splitting to minimize weighted entropy after a split: slightly slower than octrees, but much fewer blocks (~350,000).
k-D trees splitting to maximize information gain ratio: twice as fast as than the previous k-D trees method, generating far fewer blocks (~167,000), but still to slow.
k-D trees splitting to minimize Gower similarity: very slow, but generates fewer blocks than any other k-D tree method (~155,000).
Greedily grabbing the largest subset of non-intersecting, largest volume blocks available when considering each unclaimed "EARTH" block as the block's defining point: even threaded, this algorithm is obscenely slow (~16 minutes with 8 threads on an 8-core system), but it generates the fewest blocks of all (<65,536).
Greedily grabbing the largest subset of non-intersecting blocks while treating a block's dimensions as objective scores to maximize: much slower than the other greedy approach and generates a few thousand more blocks, too.
Considering that a maximum acceptable number of blocks is one block per (x,y) coordinate to represent a single column, only the greedy volume is having results that could be considered even close to optimal.
I do not know how to compute the minimum block count without using a brute force approach that never ends, so I do not know whether it is possible to do better than the greedy volume approach. Furthermore, I do not know how I can make it any faster. Can anyone give me an algorithm to try or at least point me in the right direction? I'd like to another way to approach my problem if this cannot be done any better.
Problem Statement:
I have the following problem:
There are more than a billion points in 3D space. The goal is to find the top N points which has largest number of neighbors within given distance R. Another condition is that the distance between any two points of those top N points must be greater than R. The distribution of those points are not uniform. It is very common that certain regions of the space contain a lot of points.
Goal:
To find an algorithm that can scale well to many processors and has a small memory requirement.
Thoughts:
Normal spatial decomposition is not sufficient for this kind of problem due to the non-uniform distribution. irregular spatial decomposition that evenly divide the number of points may help us the problem. I will really appreciate that if someone can shed some lights on how to solve this problem.
Use an Octree. For 3D data with a limited value domain that scales very well to huge data sets.
Many of the aforementioned methods such as locality sensitive hashing are approximate versions designed for much higher dimensionality where you can't split sensibly anymore.
Splitting at each level into 8 bins (2^d for d=3) works very well. And since you can stop when there are too few points in a cell, and build a deeper tree where there are a lot of points that should fit your requirements quite well.
For more details, see Wikipedia:
https://en.wikipedia.org/wiki/Octree
Alternatively, you could try to build an R-tree. But the R-tree tries to balance, making it harder to find the most dense areas. For your particular task, this drawback of the Octree is actually helpful! The R-tree puts a lot of effort into keeping the tree depth equal everywhere, so that each point can be found at approximately the same time. However, you are only interested in the dense areas, which will be found on the longest paths in the Octree without even having to look at the actual points yet!
I don't have a definite answer for you, but I have a suggestion for an approach that might yield a solution.
I think it's worth investigating locality-sensitive hashing. I think dividing the points evenly and then applying this kind of LSH to each set should be readily parallelisable. If you design your hashing algorithm such that the bucket size is defined in terms of R, it seems likely that for a given set of points divided into buckets, the points satisfying your criteria are likely to exist in the fullest buckets.
Having performed this locally, perhaps you can apply some kind of map-reduce-style strategy to combine spatial buckets from different parallel runs of the LSH algorithm in a step-wise manner, making use of the fact that you can begin to exclude parts of your problem space by discounting entire buckets. Obviously you'll have to be careful about edge cases that span different buckets, but I suspect that at each stage of merging, you could apply different bucket sizes/offsets such that you remove this effect (e.g. perform merging spatially equivalent buckets, as well as adjacent buckets). I believe this method could be used to keep memory requirements small (i.e. you shouldn't need to store much more than the points themselves at any given moment, and you are always operating on small(ish) subsets).
If you're looking for some kind of heuristic then I think this result will immediately yield something resembling a "good" solution - i.e. it will give you a small number of probable points which you can check satisfy your criteria. If you are looking for an exact answer, then you are going to have to apply some other methods to trim the search space as you begin to merge parallel buckets.
Another thought I had was that this could relate to finding the metric k-center. It's definitely not the exact same problem, but perhaps some of the methods used in solving that are applicable in this case. The problem is that this assumes you have a metric space in which computing the distance metric is possible - in your case, however, the presence of a billion points makes it undesirable and difficult to perform any kind of global traversal (e.g. sorting of the distances between points). As I said, just a thought, and perhaps a source of further inspiration.
Here are some possible parts of a solution.
There are various choices at each stage,
which will depend on Ncluster, on how fast the data changes,
and on what you want to do with the means.
3 steps: quantize, box, K-means.
1) quantize: reduce the input XYZ coordinates to say 8 bits each,
by taking 2^8 percentiles of X,Y,Z separately.
This will speed up the whole flow without much loss of detail.
You could sort all 1G points, or just a random 1M,
to get 8-bit x0 < x1 < ... x256, y0 < y1 < ... y256, z0 < z1 < ... z256
with 2^(30-8) points in each range.
To map float X -> 8 bit x, unrolled binary search is fast —
see Bentley, Pearls p. 95.
Added: Kd trees
split any point cloud into different-sized boxes, each with ~ Leafsize points —
much better than splitting X Y Z as above.
But afaik you'd have to roll your own Kd tree code
to split only the first say 16M boxes, and keep counts only, not the points.
2) box: count the number of points in each 3d box,
[xj .. xj+1, yj .. yj+1, zj .. zj+1].
The average box will have 2^(30-3*8) points;
the distribution will depend on how clumpy the data is.
If some boxes are too big or get too many points, you could
a) split them into 8,
b) track the centre of the points in each box,
otherwide just take box midpoints.
3)
K-means clustering
on the 2^(3*8) box centres.
(Google parallel "k means" -> 121k hits.)
This depends strongly on K aka Ncluster, also on your radius R.
A rough approach would be to grow a
heap
of the say 27*Ncluster boxes with the most points,
then take the biggest ones subject to your Radius constraint.
(I like to start with a
Minimum spanning tree,
then remove the K-1 longest links to get K clusters.)
See also
Color quantization .
I'd make Nbit, here 8, a parameter from the beginning.
What is your Ncluster ?
Added: if your points are moving in time, see
collision-detection-of-huge-number-of-circles on SO.
I would also suggest to use an octree. The OctoMap framework is very good at dealing with huge 3D point clouds. It does not store all the points directly, but updates the occupancy density of every node (aka 3D box).
After the tree is built, you can use a simple iterator to find the node with the highest density. If you would like to model the point density or distribution inside the nodes, the OctoMap is very easy to adopt.
Here you can see how it was extended to model the point distribution using a planar model.
Just an idea. Create a graph with given points and edges between points when distance < R.
Creation of this kind of graph is similar to spatial decomposition. Your questions can be answered with local search in graph. First are vertices with max degree, second is finding of maximal unconnected set of max degree vertices.
I think creation of graph and search can be made parallel. This approach can have large memory requirement. Splitting domain and working with graphs for smaller volumes can reduce memory need.