I am using Poisson surface reconstruction algorithm to reconstruct triangulated mesh surface from points. However, Poisson will always generate a watertight surface, which fills all holes with interpolation.
For some small holes that is the result of data missing, this hole filling is desirable. But for some big holes, I do not want hole filling and just want the surface to remain open.
The figure above shows my idea, the left one is a pointset with normal, the right one is reconstructed surface. I want the top of this surface remains open rather than current watertight result.
Can anyone provide me with some advice, how may I keep these big holes in Poisson surface reconstruction? Or is there any other algorithms that could solve this?
P.S.
Based on the accepted answer to this question, I understand surface reconstruction algorithms could be categorized as explicit ones and implicit ones. Poisson is implicit ones, and explicit ones could naturally handle big hole problem. But since the points data I have are mostly sparse and noisy, I would prefer an implicit one like Poisson.
Your screenshots look like you might be using MeshLab's implementation which is based on an old implementation. This implementation is not capable of trimming the surface.
The latest implementation, however, contains the SurfaceTrimmer that does exactly what you want. Take a look at the examples at the bottom of the page to see how to use it.
To use SurfaceTrimmer program, you have to first use SSDRecon program to reconstruct a mesh surface with --density, then setting a trim value would exactly remove faces under a specific threshold.
Below is a sample usage of that program on the demo eagle data
./SSDRecon --in eagle.points.ply --out eagle.screened.color.ply --depth 10 --density
./SurfaceTrimmer --in eagle.screened.color.ply --out eagle.screened.color.trimmed.ply --trim 7
Related
I have a .xyz file that has irregularly spaced points and gives the position and surface normal (ie XYZIJK). Are there algorithms out there that can reconstruct the surface that factor in the IJK vectors? Most algorithms I have found assume that surface normals aren't known.
This would ultimately be used to plot surface error data (from the nominal surface) using python 3.x, and I'm sure I will have many more follow on questions once I find a good reconstruction algorithm.
The state of the art right now is Poisson Surface Reconstruction and its screened variant. Code for both is available, e.g. under http://www.cs.jhu.edu/~misha/Code/PoissonRecon/Version8.0/. It is also implemented in MeshLab if you want to take a quick look.
If you want to take a look at other methods, check out this STAR. Page three has a table of a couple of approaches and their inputs.
Many certain resources about raytracing tells about:
"shoot rays, find the first obstacle to cut it"
"shoot secondary rays..."
"or, do it reverse and approximate/interpolate"
I didnt see any algortihm that uses a diffusion algorithm. Lets assume a point-light is a point that has more density than other cells(all space is divided into cells), every step/iteration of lighting/tracing makes that source point to diffuse into neighbours using a velocity field and than their neighbours and continues like that. After some satisfactory iterations(such as 30-40 iterations), the density info of each cell is used for enlightment of objects in that cell.
Point light and velocity field:
But it has to be a like 1000x1000x1000 size and this would take too much time and memory to compute. Maybe just computing 10x10x10 and when finding an obstacle, partitioning that area to 100x100x100(in a dynamic kd-tree fashion) can help generating lighting/shadows for acceptable resolution? Especially for vertex-based illumination rather than triangle.
Has anyone tried this approach?
Note: Velocity field is here to make light diffuse to outwards mostly(not %100 but %99 to have some global illumination). Finite-element-method can make this embarassingly-parallel.
Edit: any object that is hit by a positive-density will be an obstacle to generate a new velocity field around the surface of it. So light cannot go through that object but can be mirrored to another direction.(if it is a lens object than light diffuse harder through it) So the reflection of light can affect other objects with a higher iteration limit
Same kd-tree can be used in object-collision algorithms :)
Just to take as a grain of salt: a neural-network can be trained for advection&diffusion in a 30x30x30 grid and that can be used in a "gpu(opencl/cuda)-->neural-network ---> finite element method --->shadows" way.
There's a couple problems with this as it stands.
The first problem is that, fundamentally, a photon in the Newtonian sense doesn't react or change based on the density of other photons around. So using a density field and trying to light to follow the classic Navier-Stokes style solutions (which is what you're trying to do, based on the density field explanation you gave) would result in incorrect results. It would also, given enough iterations, result in complete entropy over the scene, which is also not what happens to light.
Even if you were to get rid of the density problem, you're still left with the the problem of multiple photons going different directions in the same cell, which is required for global illumination and diffuse lighting.
So, stripping away the problem portions of your idea, what you're left with is a particle system for photons :P
Now, to be fair, sudo-particle systems are currently used for global illumination solutions. This type of thing is called Photon Mapping, but it's only simple to implement a direct lighting solution using it :P
General: I'm hoping that the use-case I'm about to describe is a simple case of an optical flow problem and since I don't have much knowledge on the subject, I was wondering if anyone has any suggestions on how I can approach solving my problem.
Research I've already done: I have began reading the High Accuracy Optical Flow Estimation Based on a Theory for Warping paper and am planning on looking over the Particle Video paper. I have found a MATLAB High Accuracy Optical Flow implementation of optical flow. However, the papers (and the code) seem to describe concepts that are very involved and may require a lot of time for me to dig in and understand. I am hoping that the solution to my problem may be more simple.
Problem: I have a sequence of images. The images depict a material breakage process, where the material and background are black and the cracks are white. I am interested in traversing the sequence of images in reverse in an attempt to map all of the cracks that have formed in the breakage process to the first black image. You can think of the material as a large puzzle and I am trying to put the pieces back together in the reverse order that they broke.
In each image, there can be some cracks that are just emerging and/or some cracks that have been fully formed (and thus created a fragment). Throughout the breakage process, some fragments may separate and break further. The fragments can also move farther away from one another (the change is slight between subsequent frames).
Desired Output: All of the cracks/lines in the sequence mapped to the first image in the sequence.
Additional Notes: Images are available in grayscale format (i.e. original) as well as in binary format, where the cracks have been outlined in white and the background is completely black. See below for some image examples.
The top row shows the original images and the bottom row shows the binary images. As you can see, the crack that goes down the middle grows wider and wider as the image sequence progresses. Thus, the bottom crack moves together with the lower fragment. When traversing the sequence in reverse, I hope to algorithmically realize that the middle crack comes together as one (and map it correctly to the first image), and also map the bottom crack correctly, keeping its correct correspondence (size and position) with the bottom fragment.
A sequence typically contains about 30~40 images, so I've just shown the beginning subset. Also, although these images don't show it, it is possible that a particular image only contains the beginning of the crack (i.e. its initial appearance) and in subsequent images it gets longer and longer and may join with other cracks.
Language: Although not necessary, I would like to implement the solution using MATLAB (just because most of the other code that relates to the project has been done in MATLAB). However, if OpenCV may be easier, I am flexible in my language/library usage.
Any ideas are greatly appreciated.
Traverse forward rather than reverse, and don't use optical flow. Use the fracture lines to segment the black parts, track the centroid of each black segment over time. Whenever a new fracture line appears that cuts across a black segment, split the segment into two and continue tracking each segment separately.
From this you should be able to construct a tree structure representing the segmentation of the black parts over time. The fracture lines can be added as metadata to this tree, perhaps assigning fracture lines to the segment node in which they first appeared.
I would advise you to follow your initial idea of backtracking the cracks. Yo kind of know how the cracks look like so you can track all the points that belong to the crack. You just track all the white points with an optical flow tracker, start with Lukas-Kanade tracker and see where you get. The high-accuracy optical flow method is a global one and more general, I'll track all the pixels in the image trying to keep some smoothness everywhere. The LK is a local method that will just use a small window around each point to do the tracking. The problem is that appart from the cracks all the pixels are plain black so nothing to track there, you'll just waist time trying to track something that you can't track and you don't need to track.
If lines are very straight you might end up with what's called the aperture problem and you'll get inaccurate results. You can also try some shape fitting/deformation based on snakes.
I agree to damian. Most optical flow methods like the HAOF rely on the first-order taylor approximation of the intensity constancy constrian equation I(x,t)=I(x+v,t+dt). That mean the solution depends on image derivatives where the gradient determine the motion vector magnitude and angle i.e. you need a certain amount of texture. However the very low texture of your non-binarised images could be enough. You could try histogram equalization to increase the contrast of your input data but it is important to apply the same transformation for both input images. e.g. as follows:
cv::Mat equalizeMat(grayInp1.rows, grayInp1.cols * 2 , CV_8UC1);
grayInp1.copyTo(equalizeMat(cv::Rect(0,0,grayInp1.cols,grayInp1.rows)));
grayInp2.copyTo(equalizeMat(cv::Rect(grayInp1.cols,0,grayInp2.cols,grayInp2.rows)));
cv::equalizeHist(equalizeMat,equalizeMat);
equalizeMat(cv::Rect(0,0,grayInp1.cols,grayInp1.rows)).copyTo(grayInp1);
equalizeMat(cv::Rect(grayInp1.cols,0,grayInp2.cols,grayInp2.rows)).copyTo(grayInp2);
// estimate optical flow
To give you some background as to what I'm doing: I'm trying to quantitatively record variations in flow of a compressible fluid via image analysis. One way to do this is to exploit the fact that the index of refraction of the fluid is directly related to its density. If you set up some kind of image behind the flow, the distortion in the image due to refractive index changes throughout the fluid field leads you to a density gradient, which helps to characterize the flow pattern.
I have a set of routines that do this successfully with a regular 2D pattern of dots. The dot pattern is slightly distorted, and by comparing the position of the dots in the distorted image with that in the non-distorted image, I get a displacement field, which is exactly what I need. The problem with this method is resolution. The resolution is limited to the number of dots in the field, and I'm exploring methods that give me more data.
One idea I've had is to use a regular grid of horizontal and vertical lines. This image will distort the same way, but instead of getting only the displacement of a dot, I'll have the continuous distortion of a grid. It seems like there must be some standard algorithm or procedure to compare one geometric grid to another and infer some kind of displacement field. Nonetheless, I haven't found anything like this in my research.
Does anyone have some ideas that might point me in the right direction? FYI, I am not a computer scientist -- I'm an engineer. I say that only because there may be some obvious approach I'm neglecting due to coming from a different field. But I can program. I'm using MATLAB, but I can read Python, C/C++, etc.
Here are examples of the type of images I'm working with:
Regular: Distorted:
--------
I think you are looking for the Digital Image Correlation algorithm.
Here you can see a demo.
Here is a Matlab Implementation.
From Wikipedia:
Digital Image Correlation and Tracking (DIC/DDIT) is an optical method that employs tracking & image registration techniques for accurate 2D and 3D measurements of changes in images. This is often used to measure deformation (engineering), displacement, and strain, but it is widely applied in many areas of science and engineering.
Edit
Here I applied the DIC algorithm to your distorted image using Mathematica, showing the relative displacements.
Edit
You may also easily identify the maximum displacement zone:
Edit
After some work (quite a bit, frankly) you can come up to something like this, representing the "displacement field", showing clearly that you are dealing with a vortex:
(Darker and bigger arrows means more displacement (velocity))
Post me a comment if you are interested in the Mathematica code for this one. I think my code is not going to help anybody else, so I omit posting it.
I would also suggest a line tracking algorithm would work well.
Simply start at the first pixel line of the image and start following each of the vertical lines downwards (You just need to start this at the first line to get the starting points. This can be done by a simple pattern that moves orthogonally to the gradient of that line, ergo follows a line. When you reach a crossing of a horizontal line you can measure that point (in x,y coordinates) and compare it to the corresponding crossing point in your distorted image.
Since your grid is regular you know that the n'th measured crossing point on the m'th vertical black line are corresponding in both images. Then you simply compare both points by computing their distance. Do this for each line on your grid and you will get, by how far each crossing point of the grid is distorted.
This following a line algorithm is also used in basic Edge linking algorithms or the Canny Edge detector.
(All this are just theoretic ideas and I cannot provide you with an algorithm to it. But I guess it should work easily on distorted images like you have there... but maybe it is helpful for you)
I am thinking of implement a image processing based solution for industrial problem.
The image is consists of a Red rectangle. Inside that I will see a matrix of circles. The requirement is to count the number of circles under following constraints. (Real application : Count the number of bottles in a bottle casing. Any missing bottles???)
The time taken for the operation should be very low.
I need to detect the red rectangle as well. My objective is to count the
items in package and there are no
mechanism (sensors) to trigger the
camera. So camera will need to capture
the photos continuously but the
program should have a way to discard
the unnecessary images.
Processing should be realtime.
There may be a "noise" in image capturing. You may see ovals instead of circles.
My questions are as follows,
What is the best edge detection algorithm that matches with the given
scenario?
Are there any other mechanisms that I can use other than the edge
detection?
Is there a big impact between the language I use and the performance of
the system?
AHH - YOU HAVE NOW TOLD US THE BOTTLES ARE IN FIXED LOCATIONS!
IT IS AN INCREDIBLY EASIER PROBLEM.
All you have to do is look at each of the 12 spots and see if there is a black area there or not. Nothing could be easier.
You do not have to do any edge or shape detection AT ALL.
It's that easy.
You then pointed out that the box might be rotatated, things could be jiggled. That the box might be rotated a little (or even a lot, 0 to 360 each time) is very easily dealt with. The fact that the bottles are in "slots" (even if jiggled) massively changes the nature of the problem. You're main problem (which is easy) is waiting until each new red square (crate) is centered under the camera. I just realised you meant "matrix" literally and specifically in the sentence in your original questions. That changes everything totally, compared to finding a disordered jumble of circles. Finding whether or not a blob is "on" at one of 12 points, is a wildly different problem to "identifying circles in an image". Perhaps you could post an image to wrap up the question.
Finally I believe Kenny below has identified the best solution: blob analysis.
"Count the number of bottles in a bottle casing"...
Do the individual bottles sit in "slots"? ie, there are 4x3 = 12 holes, one for each bottle.
In other words, you "only" have to determine if there is, or is not, a bottle in each of the 12 holes.
Is that correct?
If so, your problem is incredibly easier than the more general problem of a pile of bottles "anywhere".
Quite simply, where do we see the bottles from? The top, sides, bottom, or? Do we always see the tops/bottoms, or are they mixed (ie, packed top-to-tail). These issues make huge, huge differences.
Surf/Sift = overkill in this case you certainly don't need it.
If you want real time speed (about 20fps+ on a 800x600 image) I recommend using Cuda to implement edge detection using a standard filter scheme like sobel, then implement binarization + image closure to make sure the edges of circles are not segmented apart.
The hardest part will be fitting circles. This is assuming you already got to the step where you have taken edges and made sure they are connected using image closure (morphology.) At this point I would proceed as follows:
run blob analysis/connected components to segment out circles that do not touch. If circles can touch the next step will be trickier
for each connected componet/blob fit a circle or rectangle using RANSAC which can run in realtime (as opposed to Hough Transform which I believe is very hard to run in real time.)
Step 2 will be much harder if you can not segment the connected components that form circles seperately, so some additional thought should be invested on how to guarantee that condition.
Good luck.
Edit
Having thought about it some more, I feel like RANSAC is ideal for the case where the circle connected components do touch. RANSAC should hypothetically fit the circle to only a part of the connected component (due to its ability to perform well in the case of mostly outlier points.) This means that you could add an extra check to see if the fitted circle encompasses the entire connected component and if it does not then rerun RANSAC on the portion of the connected component that was left out. Rinse and repeat as many times as necessary.
Also I realize that I say circle but you could just as easily fit an ellipse instead of circles using RANSAC.
Also, I'd like to comment that when I say CUDA is a good choice I mean CUDA is a good choice to implement the sobel filter + binirization + image closing on. Connected components and RANSAC are probably best left to the CPU, but you can try pushing them onto CUDA though I don't know how much of an advantage a GPU will give you for those 2 over a CPU.
For the circles, try the Hough transform.
other mechanisms: dunno
Compiled languages will possibly be faster.
SIFT should have a very good response to circular objects - it is patented, though. GLOHis a similar algorithm, but I do not know if there are any implementations readily available.
Actually, doing some more research, SURF is an improved version of SIFT with quite a few implementations available, check out the links on the wikipedia page.
Sum of colors + convex hull to detect boundary. You need, mostly, 4 corners of a rectangle, and not it's sides?
No motion, no second camera, a little choice - lot of math methods against a little input (color histograms, color distribution matrix). Dunno.
Java == high memory consumption, Lisp == high brain consumption, C++ == memory/cpu/speed/brain use optimum.
If the contrast is good, blob analysis is the algorithm for the job.