Trying to find the contour boundary points of a set of images as a list of x,y coordinates. Here is a set of sample images and I'm looking for the boundaries of the white, gray and the inner blue regions of the donut(cardiac segments). I'm able to get the coordinate of of each color based on levels however, finding the boundary coordinates efficiently is a challenge. Tried convex hull but with limited success. Any advice would be appreciated. Ideally a C++ library that may have a routine to efficiently compute the list. Since,there are lots of such images, efficiency is a key factor.
A list of images containing contours of interest.
OpenCV is a good image processing library with lots of features including findContours. There is a also a GPU support but i dont think for the findContours algorithm. But its free and easy to implement. Maybe the performance is still good enough.
Related
The shape and positions of all the polygons are known beforehand. The polygons are not overlapping and will be of different colors and shapes, and there could be quite many of them. The polygons are defined in floating point based coordinates and will be painted on top of a JPEG photo as annotation.
How could I create the resulting image file as fast as possible after I get to know which color I should give each polygon?
If it would save time I would like to perform as much as possible of the computations beforehand. All information regarding geometry and positions of the polygons are known in advance. The JPEG photo is also known in advance. The only information not known beforehand is the color of each polygon.
The JPEG photo has a size of 250x250 pixels, so that would also be the image size of the resulting rasterised image.
The computations will be done on a Linux computer with a standard graphics card, so OpenGL might be a viable option. I know there are also rasterisation libraries like Cairo that could be used to paint polygons. What I wonder is if I could take advantage of the fact that I know so much of the input in advance and use that to speed up the computation. The only thing missing is the color of each polygon.
Preferably I would like to find a solution that would only precompute things in the form of data files. In other words as soon as the polygon colors are known, the algorithm would load the other information from datafiles (JPEG file, polygon geometry file and/or possibly precomputed datafiles). Of course it would be faster to start the computation out with a "warm" state ready in the GPU/CPU/RAM but I'd like to avoid that. The choice of programming language is not so import, but could for instance be C++.
To give some more background information: The JavaScript library OpenSeadragon that is running in a web browser requests image tiles from a web server. The idea is that measurement points (i.e. the polygons) could be plotted on-the-fly on to pregenerated Zooming Images (DZI format) by the web server. So for one image tile the algorithm would only need to be run one time. The aim is low latency.
I have been working a self project in image processing and robotics where instead robot as usual detecting colors and picking out the object, it tries to detect the holes(resembling different polygons) on the board. For a better understanding of the setup here is an image:
As you can see I have to detect these holes, find out their shapes and then use the robot to fit the object into the holes. I am using a kinect depth camera to get the depth image. The pic is shown below:
I was lost in thought of how to detect the holes with the camera, initially using masking to remove the background portion and some of the foreground portion based on the depth measurement,but this did not work out as, at different orientations of the camera the holes would merge with the board... something like inranging (it fully becomes white). Then I came across adaptiveThreshold function
adaptiveThreshold(depth1,depth3,255,ADAPTIVE_THRESH_GAUSSIAN_C,THRESH_BINARY,7,-1.0);
With noise removal using erode, dilate, and gaussian blur; which detected the holes in a better manner as shown in the picture below. Then I used the cvCanny edge detector to get the edges but so far it has not been good as shown in the picture below.After this I tried out various feature detectors from SIFT, SURF, ORB, GoodFeaturesToTrack and found out that ORB gave the best times and the features detected. After this I tried to get the relative camera pose of a query image by finding its keypoints and matching those keypoints for good matches to be given to the findHomography function. The results are as shown below as in the diagram:
In the end i want to get the relative camera pose between the two images and move the robot to that position using the rotational and translational vectors got from the solvePnP function.
So is there any other method by which I could improve the quality of the
holes detected for the keypoints detection and matching?
I had also tried contour detection and approxPolyDP but the approximated shapes are not really good:
I have tried tweaking the input parameters for the threshold and canny functions but
this is the best I can get
Also ,is my approach to get the camera pose correct?
UPDATE : No matter what I tried I could not get good repeatable features to map. Then I read online that a depth image is cheap in resolution and its only used for stuff like masking and getting the distances. So , it hit me that the features are not proper because of the low resolution image with its messy edges. So I thought of detecting features on a RGB image and using the depth image to get only the distances of those features. The quality of features I got were literally off the charts.It even detected the screws on the board!! Here are the keypoints detected using GoodFeaturesToTrack keypoint detection..
I met an another hurdle while getting the distancewith the distances of the points not coming out properly. I searched for possible causes and it occured to me after quite a while that there was a offset in the RGB and depth images because of the offset between the cameras.You can see this from the first two images. I then searched the net on how to compensate this offset but could not find a working solution.
If anyone one of you could help me in compensate the offset,it would be great!
UPDATE: I could not make good use of the goodFeaturesToTrack function. The function gives the corners in Point2f type .If you want to compute the descriptors we need the keypoints and converting Point2f to Keypoint with the code snippet below leads to the loss of scale and rotational invariance.
for( size_t i = 0; i < corners1.size(); i++ )
{
keypoints_1.push_back(KeyPoint(corners1[i], 1.f));
}
The hideous result from the feature matching is shown below .
I have to start on different feature matchings now.I'll post further updates. It would be really helpful if anyone could help in removing the offset problem.
Compensating the difference between image output and the world coordinates:
You should use good old camera calibration approach for calibrating the camera response and possibly generating a correction matrix for the camera output (in order to convert them into real scales).
It's not that complicated once you have printed out a checkerboard template and capture various shots. (For this application you don't need to worry about rotation invariance. Just calibrate the world view with the image array.)
You can find more information here: http://www.vision.caltech.edu/bouguetj/calib_doc/htmls/own_calib.html
--
Now since I can't seem to comment on the question, I'd like to ask if your specific application requires the machine to "find out" the shape of the hole on the fly. If there are finite amount of hole shapes, you may then model them mathematically and look for the pixels that support the predefined models on the B/W edge image.
Such as (x)^2+(y)^2-r^2=0 for a circle with radius r, whereas x and y are the pixel coordinates.
That being said, I believe more clarification is needed regarding the requirements of the application (shape detection).
If you're going to detect specific shapes such as the ones in your provided image, then you're better off using a classifer. Delve into Haar classifiers, or better still, look into Bag of Words.
Using BoW, you'll need to train a bunch of datasets, consisting of positive and negative samples. Positive samples will contain N unique samples of each shape you want to detect. It's better if N would be > 10, best if >100 and highly variant and unique, for good robust classifier training.
Negative samples would (obviously), contain stuff that do not represent your shapes in any way. It's just for checking the accuracy of the classifier.
Also, once you have your classifier trained, you could distribute your classifier data (say, suppose you use SVM).
Here are some links to get you started with Bag of Words:
https://gilscvblog.wordpress.com/2013/08/23/bag-of-words-models-for-visual-categorization/
Sample code:
http://answers.opencv.org/question/43237/pyopencv_from-and-pyopencv_to-for-keypoint-class/
I have point clouds (each point has a colour) of objects and images that show these objects. I want to find interest points in 2D/3D and match those so I know which parts of my image (at least those that had interest points) are found in the point cloud.
So I would need to find interest points first, get their descriptors and match them. If possible, this should work with current fast and memory conserving algorithms like BRISK or ORB (no patented algorithms!) from OpenCV. But I don't know how to implement them for 3D. Is this even possible? I found a paper (Hough Transform and 3D SURF for robust three dimensional classification) that talked about a 3D extension to SURF which would be a start but I can't find any info about that 3D extension. Even then, the question would be how feasible such an extension would be for BRISK or others current algorithms.
So please, give me advice on how to proceed.
It's called epipolar geometry and stereo matching.
1) You would need two images(2D) that you generated 3D point cloud from.
2) From those two images, you can create fundamental matrix and then generate epipolar points. Quite easy to do if you do it in MATLAB, not sure about OpenCV.
3) Those epipolar points from two separate images will draw lines to 3D world.
I suggest you to read about epipolar geometry and stereo matching for 2D -> 3D
I'd like to implement a Filter that allows resampling of an image by moving a number of control points that mark edges and tangent directions. The goal is to be able to freely transform an image as seen in Photoshop when you use "Free Transform" and chose Warpmode "Custom". The image is fitted into a some kind of Spline-Patch (if that is a valid name) that can be manipulated.
I understand how simple splines (paths) work but how do you connect them to form a patch?
And how can you sample such a patch to render the morphed image? For each pixel in the target I'd need to know what pixel in the source image corresponds. I don't even know where to start searching...
Any helpful info (keywords, links, papers, reference implementations) are greatly appreciated!
This document will get you a good insight into warping: http://www.gson.org/thesis/warping-thesis.pdf
However, this will include filtering out high frequencies, which will make the implementation a lot more complicated but will give a better result.
An easy way to accomplish what you want to do would be to loop through every pixel in your final image, plug the coordinates into your splines and retrieve the pixel in your original image. This pixel might have coordinates 0.4/1.2 so you could bilinearly interpolate between 0/1, 1/1, 0/2 and 1/2.
As for splines: there are many resources and solutions online for the 1D case. As for 2D it gets a bit trickier to find helpful resources.
A simple example for the 1D case: http://www-users.cselabs.umn.edu/classes/Spring-2009/csci2031/quad_spline.pdf
Here's a great guide for the 2D case: http://en.wikipedia.org/wiki/Bicubic_interpolation
Based upon this you could derive an own scheme for splines for the 2D case. Define a bivariate (with x and y) polynomial and set your constraints to solve for the coefficients of the polynomial.
Just keep in mind that the borders of the spline patches have to be consistent (both in value and derivative) to avoid ugly jumps.
Good luck!
I am working on an automatic image stitching algorithm using MATLAB. So far, I have downloaded a source code much like the one that I had in mind and so, I'm currently studying how the code work.
The problem is, when stitching two or more images together, their color intensity will most probably be different from each other so the stitched seams will be visible to the eye... So, right now, I'm trying to find out how to redistribute their color intensity using the images gradients so that the whole stitched image will have the same color intensity.
I hope someone can help me out there and if so, thank you very much...
If the images overlap by a significant amount, and the stitching algorithm does a very good job of registering the overlap region, a very simple solution would be to blend the pixel values from the two images together in the overlap region, using a weighted average with weights going from 0-1 depending on the distance from the edge of the overlap region.
blendedPixel = (imageApixel * weightA) + (imageBpixel * weightB)
where weightA is approaches 1 as we get closer to the imageA side of the overlap region, weightB approaches 1 as we get closer to the imageB side of the overlap region, and the sum of weightA and weightB is always 1.
The above solution is not particularly principled, and does depend on the stitching algorithm doing a very good job of image registration in the overlap region.
Another, more principled solution to the problem would be to remove the source of the intensity difference, attempting to homogenize the response of the pixels across the image plane.
The form of this solution will depend on the source of the intensity difference, which will depend on the optics and the scene lighting conditions.
For example when dealing with photographs of outdoor scenes, taken at the same time from the same location, then the dominant effect will likely be "vignetting" effects, which can be due to a variety of different causes, including differences between the various paths taken by the light through camera optics.
As another example, when dealing with photographs taken through a microscope of a sample illuminated at an oblique angle, the dominant effect will likely be due to the difference in illumination between those parts of the image closest to the light and those far away.
Vignetting generally manifests itself as a radially symmetric function centred around the projection of the optical axis of the lens onto the image plane. To correct for vignetting, you should try to fit a suitable radially symmetric function.
Lighting changes can take different functional forms, but fitting a straightforward linear approximation is sufficient in many cases.
Depending upon the scene, and the number and variability of the images that have available, you may need to take calibration images to fit these functions properly.
The above approaches make assumptions about the functional forms of the sources of the intensity differences, but not about the scene or it's statistics.
Yet another approach might be to make some assumptions about the scene, for example, that all significant information is represented by spatial frequencies above some threshold. You can then remove all low image intensity spatial frequency components. This will "flatten" the image, removing much of the low-frequency vignetting and lighting issues.
This approach might be applicable to microscopy images, sattelite images, or images of other scenes within which most of the interest lies in the detail, rather than in the drama of the composition.
There are a number of papers that tackle this problem, many at a level of technical sophistication rather beyond the above discussion. For example, see D Goldman, "Vignette and Exposure Calibration and Compensation", IEEE Trans Pattern Analysis and Machine Intelligence, vol 32, no 12, pp2276-2288