I am trying to figure out what algorithms there are to do surface reconstruction from 3D range data. At a first glance, it seems that the Ball pivoting algorithm (BPA) and Poisson surface reconstruction are the more established methods?
What are the established, more robust algorithm in the field other than BPA and Poisson surface reconstruction algorithm?
Recommended research publications?
Is there available source code?
I have been facing this dilemma for some months now, and made exhaustive research.
Algorithms
Mainly there are 2 categories of algorithms: computation geometry, and implicit surfaces.
Computation Geometry
They fit the mesh on the existing points.
Probably the most famous algorithm of this group is powercrust, because it is theoretically well-established - it guarantees watertight mesh.
Ball Pivoting is patented by IBM. Also, it is not suitable for pointclouds with varying point density.
Implicit functions
One fits implicit functions on the pointcloud, then uses a marching-cube-like algorithm to extract the zero-set of the function into a mesh.
Methods in this category differ mainly by the different implicit functions used.
Poisson, Hoppe's, and MPU are the most famous algorithms in this category. If you are new to the topic, i recommend to read Hoppe's thesis, it is very explanatory.
The algorithms of this category usually can be implemented so that they are able to process huge inputs very efficiently, and one can scale their quality<->speed trade-off. They are not disturbed by noise, varying point-density, holes. A disadvantage of them is that they require consistently oriented surface normals at the input points.
Implementations
You will find small number of free implementations. However it depends on whether You are going to integrate it into free software (in this case GPL license is acceptable for You) or into a commercial software (in this case You need a more liberal license). The latter is very rare.
One is in VTK. I suspect it to be difficult to integrate (no documentation is available for free), it has a strange, over-complicated architecture, and is not designed for high-performance applications. Also has some limitations for the allowed input pointclouds.
Take a look at this Poisson implementation, and after that share your experience about it with me please.
Also:
here are a few high-performance algorithms, with surface reconstruction among them.
CGAL is a famous 3d library, but it is free only for free projects.
Meshlab is a famous application with GPL.
Also (Added August 2013):
The library PCL has a module dedicated to surface reconstruction and is in active development (and is part of Google's Summer of Code). The surface module contains a number of different algorithms for reconstruction. PCL also has the ability to estimate surface normals, incase you do not have them provided with your point data, this functionality can be found in the features module. PCL is released under the terms of the BSD license and is open source software, it is free for commercial and research use.
If you want make some direct experiments with various surface reconstruction algorithms you should try MeshLab, the mesh-processing system, it is open source and it contains implementations of many of the previously cited surface reconstruction algorithms, like:
Poisson Surface Recon
a couple of MLS based approach,
a ball pivoting implementation
a variant of the Curless volume based approach
Delaunay based techniques (Alpha shapes and Voronoi filtering)
tools for computing normals from scattered point sets
and many other tools for comparing/measuring/cleaning/simplifying the resulting meshes.
Sources are protected by GPL, so you could not use them in a commercial closed source project, but it is very important to get the right feeling about the properties of the various surface reconstruction algorithms (how sensitive to noise they are, the speed, the robustness to outliers, how they preserve fine details etc etc) before starting to implement one of them.
You might start looking at some recent work in the field - currently something like Fast low-memory streaming MLS reconstruction of point-sampled surfaces by Gianmauro Cuccuru, Enrico Gobbetti, Fabio Marton, Renato Pajarola, and Ruggero Pintus. Its citations can get you going through the literature pretty quickly.
While not a mesh representation, an ex-colleague recommended me this link
to source code for a Thin Plate Spline method:
Link
Anyone tried it?
Not sure if it's exactly right for your case, since it seems weird that you omitted it, but marching cubes is commonly mentioned in cases like these.
As I had this problem too, I did develop and implement my own point cloud crust algorithm. The sources, as well as the documentation, can be found on github.com: https://github.com/meixxi/PointCloudCrust. The algorithm is implemented in Java.
Maybe, this can help you. You can find also a short python script on the page which illustrates how to use the library. Have fun!
Here on GitHub, is a open source Mesh Processing Library in C++ by Dr. Hugues Hoppe, in which the surface reconstruction program Recon is a good option for your problem...
There is 3D Delaunay tool by Geometric Tools. This tool is used DirecX and OpenGL. Unfortunately, you may need buy a book to see actual example code of the library. You still read the code and figure out.
Matlab also introduced a surface reconstruction tool using Delaunay, delaunayTriangulation class.
You might be interested in Alpha Shapes.
Related
I do not have a background in image recognition/feature extraction, but I am in need of a way to extract trees from an image without the background vegetation.
Seen above is a small example of the kind of imagery I'm working with. I have access to multi-spectral imagery as well (though I haven't seen it yet) including NDVI, NIR, Red-edge.
From researching the problem at hand, I am aware that feature extraction is an active area of research and it seems that often supervised and unsupervised machine learning is employed in combination with statistical voodoo such as "PCA". Being able to differentiate between trees and background vegetation has been noted as an area of difficulty in some papers I skimmed over in my research.
There are notable features about the imagery I am working with. First of all, the palm trees have a very distinctive shape. Not only this, but there are obvious differences in the texture of the trees vs the texture of the background vegetation.
I am not an academic, and as such I only have access to publicly available papers for my research. I am looking for relevant algorithms that could help me extract the features of interest to me (trees) that either have an implementation (ideally in C or bindings to C, though I'm aware that it is not a commonly used language in this field) or with publicly available papers/tutorials/sites/etc. detailing the algorithm so that I could implement it myself.
Thanks in advance for any help!
Look into OpenCV, It has a lot of options for supervised/semi supervised Learning methods. As you have mentioned there is a visible texture difference between the tress and background vegetation, a good place for you would be to start would be color based segmentation and evolving it to use textures as well. OpenCV ML tutorial is a good starting point. Moreover you can also combine the NDVI data to create a stronger feature set.
I'm going to match the sketch face (drawing photo) in to the color photo. so for the research i want to find out what are the challenges that matching sketch drawing in to color faces. for now i have find out that
resolution pixel difference
texture difference
distance difference
and color (not much effect)
I want to know (in technical terms) what are other challenges and what are available OPEN CV and JAVA CV method and algorithms to overcome that challenges?
Here is some example of the sketches and the photos that are known to match them:
This problem is called multi-modal face recognition. There has been a lot of interest in comparing a high quality mugshot (modality 1) to low quality surveillance images (modality 2), another is frontal images to profiles, or pictures to sketches like the OP is interested in. Partial Least Squares (PLS) and Tied Factor Analysis (TFA) have been used for this purpose.
A key difficulty is computing two linear projections from the image in modality 1 (and modality 2) to a space where two points being close means that the individual is the same. This is the key technical step. Here are some papers on this approach:
Abhishek Sharma, David W Jacobs : Bypassing Synthesis: PLS for
Face Recognition with Pose, Low-Resolution and Sketch. CVPR
2011.
S.J.D. Prince, J.H. Elder, J. Warrell, F.M. Felisberti, Tied Factor
Analysis for Face Recognition across Large Pose Differences, IEEE
Patt. Anal. Mach. Intell, 30(6), 970-984, 2008. Elder is a specialist in this area and has a variety of papers on the topic.
B. Klare, Z. Li and A. K. Jain, Matching forensic sketches to
mugshot photos, IEEE Pattern Analysis and Machine Intelligence, 29
Sept. 2010.
As you can understand this is an active research area/problem. In terms using OpenCV to overcome the difficulties, let me give you an analogy: you need to build build a house (match sketches to photos) and you're asking how will having a Stanley hammer (OpenCV) will help. Sure, it will probably help. But you'll also need a lot of other resources: wood, time/money, pipes, cable, etc.
I think that James Elder's old work on the completeness of the edge map (using reconstruction by solving the Laplace equation) is quite relevant here. See the results at the end of this paper: http://elderlab.yorku.ca/~elder/publications/journals/ElderIJCV99.pdf
You could give Eigenfaces a try, though i never tested them with sketches i think they could a least be a good starting point for your research.
See Wiki: http://en.wikipedia.org/wiki/Eigenface and the Tutorial for OpenCV: http://docs.opencv.org/modules/contrib/doc/facerec/facerec_tutorial.html (including not only Eigenfaces!)
OpenCV can be used for feature extraction and machine learning required for this task. I guess you can start with the papers in the answers above, start with some basic features and prototype a classifier with OpenCV.
I guess you might also want to detect and match feature points on the faces. If you use this approach, you will have to do the feature point detectors on your own (training the Viola-Jones detector in OpenCV with your own data is an option).
I'm looking for some algorithm (preferably if source code available)
for image registration.
Image deformation can't been described by homography matrix(because I think that distortion not symmetrical and not
homogeneous),more specifically deformations are like barrel/distortion and trapezoid distortion, maybe some rotation of image.
I want to obtain pairs of pixel of two images and so i can obtain representation of "deformation field".
I google a lot and find out that there are some algorithm base on some phisics ideas, but it seems that they can converge
to local maxima, but not global.
I can affort program to be semi-automatic, it means some simple user interation.
maybe some algorithms like SIFT will be appropriate?
but I think it can't provide "deformation field" with regular sufficient density.
if it important there is no scale changes.
example of complicated field
http://www.math.ucla.edu/~yanovsky/Research/ImageRegistration/2DMRI/2DMRI_lambda400_grid_only1.png
What you are looking for is "optical flow". Searching for these terms will yield you numerous results.
In OpenCV, there is a function called calcOpticalFlowFarneback() (in the video module) that does what you want.
The C API does still have an implementation of the classic paper by Horn & Schunck (1981) called "Determining optical flow".
You can also have a look at this work I've done, along with some code (but be careful, there are still some mysterious bugs in the opencl memory code. I will release a corrected version later this year.): http://lts2www.epfl.ch/people/dangelo/opticalflow
Besides OpenCV's optical flow (and mine ;-), you can have a look at ITK on itk.org for complete image registration chains (mostly aimed at medical imaging).
There's also a lot of optical flow code (matlab, C/C++...) that can be found thanks to google, for example cs.brown.edu/~dqsun/research/software.html, gpu4vision, etc
-- EDIT : about optical flow --
Optical flow is divided in two families of algorithms : the dense ones, and the others.
Dense algorithms give one motion vector per pixel, non-dense ones one vector per tracked feature.
Examples of the dense family include Horn-Schunck and Farneback (to stay with opencv), and more generally any algorithm that will minimize some cost function over the whole images (the various TV-L1 flows, etc).
An example for the non-dense family is the KLT, which is called Lucas-Kanade in opencv.
In the dense family, since the motion for each pixel is almost free, it can deal with scale changes. Keep in mind however that these algorithms can fail in the case of large motions / scales changes because they usually rely on linearizations (Taylor expansions of the motion and image changes). Furthermore, in the variational approach, each pixel contributes to the overall result. Hence, parts that are invisible in one image are likely to deviate the algorithm from the actual solution.
Anyway, techniques such as coarse-to-fine implementations are employed to bypass these limits, and these problems have usually only a small impact. Brutal illumination changes, or large occluded / unoccluded areas can also be explicitly dealt with by some algorithms, see for example this paper that computes a sparse image of "innovation" alongside the optical flow field.
i found some software medical specific, but it's complicate and it's not work with simple image formats, but seems that it do that I need.
http://www.csd.uoc.gr/~komod/FastPD/index.html
Drop - Deformable Registration using Discrete Optimization
I have a lot of polygons. Ideally, all the polygons must not overlap one other, but they can be located adjacent to one another.
But practically, I would have to allow for slight polygon overlap ( defined by a certain tolerance) because all these polygons are obtained from user hand drawing input, which is not as machine-precised as I want them to be.
My question is, is there any software library components that:
Allows one to input a range of polygons
Check if the polygons are overlapped more than a prespecified tolerance
If yes, then stop, or else, continue
Create mesh in terms of coordinates and elements for the polygons by grouping common vertex and edges together?
More importantly, link back the mesh edges to the original polygon(s)'s edge?
Or is there anyone tackle this issue before?
This issue is a daily "bread" of GIS applications - this is what is exactly done there. We also learned that at a GIS course. Look into GIS systems how they address this issue. E.g. ArcGIS define so called topology rules and has some functions to check if the edited features are topologically correct. See http://webhelp.esri.com/arcgisdesktop/9.2/index.cfm?TopicName=Topology_rules
This is pretty long, only because the question is so big. I've tried to group my comments based on your bullet points.
Components to draw polygons
My guess is that you'll have limited success without providing more information - a component to draw polygons will be very much coupled to the language and UI paradigm you are using for the rest of your project, ie. code for a web component will look very different to a native component.
Perhaps an alternative is to separate this element of the process out from the rest of what you're trying to do. There are some absolutely fantastic pre-existing editors that you can use to create 2d and 3d polygons.
Inkscape is an example of a vector graphics editor that makes it easy to enter 2d polygons, and has the advantage of producing output SVG, which is reasonably easy to parse.
In three dimensions Blender is an open source editor that can be used to produce arbitrary geometries that can be exported to a number of formats.
If you can use a google-maps API (possibly in an native HTML rendering control), and you are interested in adding spatial points on a map overlay, you may be interested in related click-to-draw polygon question on stackoverflow. From past experience, other map APIs like OpenLayers support similar approaches.
Check whether polygons are overlapped
Thomas T made the point in his answer, that there are families of related predicates that can be used to address this and related queries. If you are literally just looking for overlaps and other set theoretic operations (union, intersection, set difference) in two dimensions you can use the General Polygon Clipper
You may also need to consider the slightly more generic problem when two polygons that don't overlap or share a vertex when they should. You can use a Minkowski sum to dilate (enlarge) two and three dimensional polygons to avoid such problems. The Computational Geometry Algorithms Library has robust implementations of these algorithms.
I think that it's more likely that you are really looking for a piece of software that can perform vertex welding, Christer Ericson's book Real-time Collision Detection includes extensive and very readable description of the basics in this field, and also on related issues of edge snapping, crack detection, T-junctions and more. However, even though code snippets are included for that book, I know of no ready made library that addresses these problems, in particular, no complete implementation is given for anything beyond basic vertex welding.
Obviously all 3D packages (blender, maya, max, rhino) all include built in software and tools to solve this problem.
Group polygons based on vertices
From past experience, this turned out to be one of the most time consuming parts of developing software to solve problems in this area. It requires reasonable understanding of graph theory and algorithms to traverse boundaries. It is worth relying upon a solid geometry or graph library to do the heavy lifting for you. In the past I've had success with igraph.
Link the updated polygons back to the originals.
Again, from past experience, this is just a case of careful bookkeeping, and some very careful design of your mesh classes up-front. I'd like to give more advice, but even after spending a big chunk of the last six months on this, I'm still struggling to find a "nice" way to do this.
Other Comments
If you're interacting with users, I would strongly recommend avoiding this issue where possible by using an editor that "snaps", rounding all user entered points onto a grid. This will hopefully significantly reduce the amount of work that you have to do.
Yes, you can use OGR. It has python bindings. Specifically, the Geometry class has an Intersects method. I don't fully understand what you want in points 4 and 5.
I am interested to read and understand the 2D mesh algorithms. A search on Google reveals a lot of papers and sources, however most are too academic and not much on beginner's side.
So, would anyone here recommend any reading sources ( suitable for the beginners), or open source implementation that I can learn from the start? Thanks.
Also, compared to triangular mesh generation, I have more interest in quadrilateral mesh and mix mesh( quad and tri combined).
I second David's answer regarding Jonathan Shewchuk's site as a good starting point.
In terms of open source software, it depends on what you are looking for exactly.
If you are interested in mesh generation, you can have a look at CGAL's code. Understanding the low level parts of CGAL's code is too much for a beginner. However, having a look at the higher level algorithms can be quite interesting even for a beginner. Also note that the documentation of CGAL is very detailed.
You can also have a look at TetGen, but its source code is monolithic and is not documented (it is more of an end user software rather than a library, even if it can also be called simply from other programs). Still, it is fairly readable, and the user manual contains a short presentation of mesh generation, with some references.
If you are also interested in mesh processing, you can have a look at OpenMesh.
More informations about your goals would definitely help providing more relevant pointers.
The first link on your Google search takes you to Jonathan Shewchuk's site. This is not actually a bad place to start. He has a program called triangle which you can download for 2D triangulation. On that page there is a link to references used in creating triangle, including a link to a description of the triangluation algorithm.
There are several approaches to mesh generation. One of the most common is to create a Delaunay triangulation. Triangulating a set of points is fairly simple and there are several algorithms which do that, including Watson's and Rupert's as used in triangle
When you want to create a constrained triangulation, where the edges of the triangulation match the edges of your input shape it is a bit harder, because you need to recover certain edges.
I would start by understanding Delaunay triangulation. Then maybe look at some of the other meshing algorithms.
Some of the common topics that you will find in mesh generation papers are
Robustness - that is how to deal with floating point round off errors.
Mesh quality - ensuring the shapes of the triangles/tetrahedrons are close to equilateral. Whether this is important depends on why you are creating the mesh. For analysis work it is very important,
How to choose where to insert the nodes in the mesh to give a good mesh distribution.
Meshing speed
Quadrilateral/Hexahedral mesh generation. This is harder than using triangles/tetrahedra.
3D mesh generation is much harder than 2D so a lot of the papers are on 3D generation
Mesh generation is a large topic. It would be helpful if you could give some more information on what aspects (eg 2D or 3D) that you are interested in. If you can give some idea of what you ant to do then maybe I can find some better sources of information.