I have point clouds (each point has a colour) of objects and images that show these objects. I want to find interest points in 2D/3D and match those so I know which parts of my image (at least those that had interest points) are found in the point cloud.
So I would need to find interest points first, get their descriptors and match them. If possible, this should work with current fast and memory conserving algorithms like BRISK or ORB (no patented algorithms!) from OpenCV. But I don't know how to implement them for 3D. Is this even possible? I found a paper (Hough Transform and 3D SURF for robust three dimensional classification) that talked about a 3D extension to SURF which would be a start but I can't find any info about that 3D extension. Even then, the question would be how feasible such an extension would be for BRISK or others current algorithms.
So please, give me advice on how to proceed.
It's called epipolar geometry and stereo matching.
1) You would need two images(2D) that you generated 3D point cloud from.
2) From those two images, you can create fundamental matrix and then generate epipolar points. Quite easy to do if you do it in MATLAB, not sure about OpenCV.
3) Those epipolar points from two separate images will draw lines to 3D world.
I suggest you to read about epipolar geometry and stereo matching for 2D -> 3D
Related
Trying to find the contour boundary points of a set of images as a list of x,y coordinates. Here is a set of sample images and I'm looking for the boundaries of the white, gray and the inner blue regions of the donut(cardiac segments). I'm able to get the coordinate of of each color based on levels however, finding the boundary coordinates efficiently is a challenge. Tried convex hull but with limited success. Any advice would be appreciated. Ideally a C++ library that may have a routine to efficiently compute the list. Since,there are lots of such images, efficiency is a key factor.
A list of images containing contours of interest.
OpenCV is a good image processing library with lots of features including findContours. There is a also a GPU support but i dont think for the findContours algorithm. But its free and easy to implement. Maybe the performance is still good enough.
I have a .xyz file that has irregularly spaced points and gives the position and surface normal (ie XYZIJK). Are there algorithms out there that can reconstruct the surface that factor in the IJK vectors? Most algorithms I have found assume that surface normals aren't known.
This would ultimately be used to plot surface error data (from the nominal surface) using python 3.x, and I'm sure I will have many more follow on questions once I find a good reconstruction algorithm.
The state of the art right now is Poisson Surface Reconstruction and its screened variant. Code for both is available, e.g. under http://www.cs.jhu.edu/~misha/Code/PoissonRecon/Version8.0/. It is also implemented in MeshLab if you want to take a quick look.
If you want to take a look at other methods, check out this STAR. Page three has a table of a couple of approaches and their inputs.
i am going to develope a system which will take a 2D still image as a input & 3D image as a output.
So the steps are:
1. creating a depth map from 2D image
2. creating 3D image from depth map and original image.
Can anybody suggest me the algorithms to generate the depth map of 2D image?
As far as I know, there's no 100% bullet proof algorithm that can convert a 2D image to a 3D model. Simply said, there's not enough information inside a 2D image to fully construct something 3D. Some 3D TV sets manage to do some fake 3D from the 2D input but nothing really convincing (and sometimes wrong.)
What famous softwares do (like the one in the Kinect), is use several sources instead of one single 2D image. With pictures from different angles, you can track some particular features in the images and with geometric computations output something 3D. See http://en.wikipedia.org/wiki/3D_reconstruction_from_multiple_images for full explanation.
If you're stuck with a single image, the best known tool is the human eye... Humans can easily reconstruct 3D from a picture, by unconsciously merging several factors, such as their experience of the scene, the focus blur, "far-away fog effect", etc... So the best way for you to have a result, is to do the depth map yourself in any image editing software...
Julien
is it possible to construct a 3d model of a still object if various images along with depth data was gathered from various angles, what I was thinking was have a sort of a circular conveyor belt where a kinect would be placed and the conveyor belt while the real object that is to be reconstructed in 3d space sits in the middle. The conveyor belt thereafter rotates around the image in a circle and lots of images are captured (perhaps 10 image per second) which would allow the kinect to catch an image from every angle including the depth data, theoretically this is possible. The model would also have to be recreated with the textures.
What I would like to know is whether there are any similar projects/software already available and any links would be appreciated
Whether this is possible within perhaps 6 months
How would I proceed to do this? Such as any similar algorithm you could point me to and such
Thanks,
MilindaD
It is definitely possible and there are a lot of 3D scanners which work out there, with more or less the same principle of stereoscopy.
You probably know this, but just to contextualize: The idea is to get two images from the same point and to use triangulation to compute the 3d coordinates of the point in your scene. Although this is quite easy, the big issue is to find the correspondence between the points in your 2 images, and this is where you need a good software to extract and recognize similar points.
There is an open-source project called Meshlab for 3d vision, which includes 3d reconstruction* algorithms. I don't know the details of the algorithms, but the software is definitely a good entrance point if you want to play with 3d.
I used to know some other ones, I will try to find them and add them here:
Insight3d
(*Wiki page has no content, redirects to login for editing)
Check out https://bitbucket.org/tobin/kinect-point-cloud-demo/overview which is a code sample for the Kinect for Windows SDK that does specifically this. Currently it uses the bitmaps captured by the depth sensor, and iterates through the byte array to create a point cloud in a PLY format that can read by MeshLab. The next stage of us is to apply/refine a delanunay triangle algoirthim to form a mesh instead of points, which a texture can be applied. A third stage would then me a mesh merging formula to combine multiple caputres from the Kinect to form a full 3D object mesh.
This is based on some work I done in June using Kinect for the purposes of 3D printing capture.
The .NET code in this source code repository will however get you started with what you want to achieve.
Autodesk has a piece of software that will do what you are asking for it is called "Photofly". It is currently in the labs section. Using a series of images taken from multiple angles the 3d geometry is created and then photo mapped with your images to create the scene.
If you interested more in theoretical (i mean if you want to know how) part of this problem,
here is some document from Microsoft Research about moving depth camera and 3D reconstruction.
Try out VisualSfM (http://ccwu.me/vsfm/) by Changchang Wu (http://ccwu.me/)
It takes multiple images from different angles of the scene and outputs a 3D point cloud.
The algorithm is called "Structure from Motion".
Brief idea of the algorithm : It involves extracting feature points in each image; finding correspondences between them across images; building feature tracks, estimating camera matrices and thereby the 3D coordinates of the feature points.
I saw a question on reverse projecting 4 2D points to derive the corners of a rectangle in 3D space. I have a kind of more general version of the same problem:
Given either a focal length (which can be solved to produce arcseconds / pixel) or the intrinsic camera matrix (a 3x2 matrix that defines the properties of the pinhole camera model being used - it's directly related to focal length), compute the camera ray that goes through each pixel.
I'd like to take a series of frames, derive the candidate light rays from each frame, and use some sort of iterative solving approach to derive the camera pose from each frame (given a sufficiently large sample, of course)... All of that is really just massively-parallel implementations of a generalized Hough algorithm... it's getting the candidate rays in the first place that I'm having the problem with...
A friend of mine found the source code from a university for the camera matching in PhotoSynth. I'd Google around for it, if I were you.
That's a good suggestion... and I will definitely look into it (photosynth kind of resparked my interest in this subject - but I've been working on it for months for robochamps) - but it's a sparse implementation - it looks for "good" features (points in the image that should be easily identifiable in other views of the same image), and while I certainly plan to score each match based on how good the feature it's matching is, I want the full dense algorithm to derive every pixel... or should I say voxel lol?
After a little poking around, isn't it the extrinsic matrix that tells you where the camera actually is in 3-space?
I worked at a company that did a lot of this, but I always used the tools that the algorithm guys wrote. :)