I am looking for a recommendation for an introduction to image processing algorithms (face and shape recognition, etc.) and wondered if anyone had an good recommendations, either for books, whitepapers or websites.
I am starting from knowing very little about image recognition and did some maths at University (a long time ago).
Any help or pointers would be greatly appreciated.
I found this blog very helpful.
There are quite a lot of topics related to CV that you might want to read up on.
Some of the topics:
Kernels
Blurring
Eigenfaces (face detection)
Connected components
Canny edge detection
Gradients
haar-like features
Histogram Of Gradients
Image segmentation
The two books that are pretty good on this subject are:
Computer Vision: Shapiro
Computer Vision A Modern Approach: Forsyth et al
I used the CV: A modern approach for a CV class I took a semester or two ago. It is fairly concise and includes explanations of how the techniques work. Its not for the faint of heart. Also: Forsyth is a well known author of many CV Papers.
Be aware that Computer Vision is in general very math heavy, so if you feel that your linear algebra skills are not up to date, then update them before attempting to read anything. Knowing your way around some basic signal processing will also be of great help.
For basic shape recognition like lines and circles an edge detector coupled with a simple(?) Hough transform could be enough to do the trick.
If you want to find other stuff that is not faces or basic shapes (cars, people, ...) then you are in for some really heavy reading as this is a quite large area of research with lots of different methods for feature extraction and classification.
If you want to look at faces only, then I suggest finding literature that deals with this specifically to not drown in a sea of math heavy information.
MIT OpenCourseWare has an image recognition course. Unfortunately, there are no video lectures for this course yet, but you'll find lecture notes and other materials.
http://ocw.mit.edu/OcwWeb/Electrical-Engineering-and-Computer-Science/6-801Fall-2004/CourseHome/index.htm
Here is a great book that you can download for free. http://szeliski.org/Book/
Related
I do not have a background in image recognition/feature extraction, but I am in need of a way to extract trees from an image without the background vegetation.
Seen above is a small example of the kind of imagery I'm working with. I have access to multi-spectral imagery as well (though I haven't seen it yet) including NDVI, NIR, Red-edge.
From researching the problem at hand, I am aware that feature extraction is an active area of research and it seems that often supervised and unsupervised machine learning is employed in combination with statistical voodoo such as "PCA". Being able to differentiate between trees and background vegetation has been noted as an area of difficulty in some papers I skimmed over in my research.
There are notable features about the imagery I am working with. First of all, the palm trees have a very distinctive shape. Not only this, but there are obvious differences in the texture of the trees vs the texture of the background vegetation.
I am not an academic, and as such I only have access to publicly available papers for my research. I am looking for relevant algorithms that could help me extract the features of interest to me (trees) that either have an implementation (ideally in C or bindings to C, though I'm aware that it is not a commonly used language in this field) or with publicly available papers/tutorials/sites/etc. detailing the algorithm so that I could implement it myself.
Thanks in advance for any help!
Look into OpenCV, It has a lot of options for supervised/semi supervised Learning methods. As you have mentioned there is a visible texture difference between the tress and background vegetation, a good place for you would be to start would be color based segmentation and evolving it to use textures as well. OpenCV ML tutorial is a good starting point. Moreover you can also combine the NDVI data to create a stronger feature set.
I'm going to match the sketch face (drawing photo) in to the color photo. so for the research i want to find out what are the challenges that matching sketch drawing in to color faces. for now i have find out that
resolution pixel difference
texture difference
distance difference
and color (not much effect)
I want to know (in technical terms) what are other challenges and what are available OPEN CV and JAVA CV method and algorithms to overcome that challenges?
Here is some example of the sketches and the photos that are known to match them:
This problem is called multi-modal face recognition. There has been a lot of interest in comparing a high quality mugshot (modality 1) to low quality surveillance images (modality 2), another is frontal images to profiles, or pictures to sketches like the OP is interested in. Partial Least Squares (PLS) and Tied Factor Analysis (TFA) have been used for this purpose.
A key difficulty is computing two linear projections from the image in modality 1 (and modality 2) to a space where two points being close means that the individual is the same. This is the key technical step. Here are some papers on this approach:
Abhishek Sharma, David W Jacobs : Bypassing Synthesis: PLS for
Face Recognition with Pose, Low-Resolution and Sketch. CVPR
2011.
S.J.D. Prince, J.H. Elder, J. Warrell, F.M. Felisberti, Tied Factor
Analysis for Face Recognition across Large Pose Differences, IEEE
Patt. Anal. Mach. Intell, 30(6), 970-984, 2008. Elder is a specialist in this area and has a variety of papers on the topic.
B. Klare, Z. Li and A. K. Jain, Matching forensic sketches to
mugshot photos, IEEE Pattern Analysis and Machine Intelligence, 29
Sept. 2010.
As you can understand this is an active research area/problem. In terms using OpenCV to overcome the difficulties, let me give you an analogy: you need to build build a house (match sketches to photos) and you're asking how will having a Stanley hammer (OpenCV) will help. Sure, it will probably help. But you'll also need a lot of other resources: wood, time/money, pipes, cable, etc.
I think that James Elder's old work on the completeness of the edge map (using reconstruction by solving the Laplace equation) is quite relevant here. See the results at the end of this paper: http://elderlab.yorku.ca/~elder/publications/journals/ElderIJCV99.pdf
You could give Eigenfaces a try, though i never tested them with sketches i think they could a least be a good starting point for your research.
See Wiki: http://en.wikipedia.org/wiki/Eigenface and the Tutorial for OpenCV: http://docs.opencv.org/modules/contrib/doc/facerec/facerec_tutorial.html (including not only Eigenfaces!)
OpenCV can be used for feature extraction and machine learning required for this task. I guess you can start with the papers in the answers above, start with some basic features and prototype a classifier with OpenCV.
I guess you might also want to detect and match feature points on the faces. If you use this approach, you will have to do the feature point detectors on your own (training the Viola-Jones detector in OpenCV with your own data is an option).
I was wondering if there is Aforge.NET algorithm that is intended for human activity recognition?
For example, I would like to recognizing drowning while capturing frames from surveillance camera on the beach.
I saw there are algorithms for motion detection, but what I need is motion detection plus logic to process that motion so that computer can conclude does that motion fit into drowning category or any other category I tell him.
Comments would be appreciated.
you might need to develop your own algorythms, i do that too with Aforge
Aforge basicly allows me for simple video aquisition, while my math does the interesting stuff.
In your case..
Detect spots with people
Zoom in to them ??
then it becommes tricky how to distinguish someone who dives from someone who sinks ?..
Also there are waves who can get in front of the person your trying to follow..
usually this recognition comes down to simple observations, like someone pulling his hands up is not like a circle of a swimming head..
you got to think how a beach guard can see the difference, what are the main visual clues and how can you convert them to bitmap math.
Consider using Accord.NET - it's a library based on AForge.NET that contains many machine learning algorithms. However you must write all the logic, as you call it, by yourself.
Another possibility is to use Emgu CV which has some motion detection algorithms.
Recently I've been messing about with algorithms on images, partly for fun and partly to keep my programming skills sharp.
I've just implemented a 'nearest-neighbour' algorithm that picks n random pixels in an image, and then converts the colour of each other pixel in the image to the colour of its nearest neighbour in the set of n chosen pixels. The result is a kind of "frosted glass" effect on the image, for a reasonably large value of n (if n is too small then the image gets blocky).
I'm just wondering if anyone has any other good/fun algorithms on images that might be interesting to implement?
Tom
This book, Digital Image Processing, is one of the most commonly used books in image processing classes, and it will teach you a lot of basic techniques that will help you understand other algorithms better, like the ones Ants Aasma suggested.
Try making an Andy Warhol print. It's pretty easy in Java. For more ideas, just look at the filters available in GIMP or a similar program.
Marching Squares is a computer vision algorithm. Try using that to convert black and white raster images to object based scenes.
Turns the image into a pizza
Take N images, relate them via an MC-Escher-style painting
"Explode" an image from the inside out
Convert the image into a single-color blocks (piet-style) based on all the colours within.
How about tie-dye algorithm?
Fun to toy with and easy to code filters are:
kaleidoscope
lens
twirl
There are a lot of other filters, but especially the kaleidoscope gives much bang for the bucks. I have made my own graphics editor with lots of filters and is also looking for inspiration.
Instead of coding image filters, I personally would love to code Diffusion Curves, but unfortunately have little time for fun.
If you want to try something more challenging look for SIGGRAPH papers on the web. There are some really nifty image algorithms presented at that conference. Seam carving is one cool example that is reasonably straightforward to implement.
If you want something more challenging try to complete the symmetry of broken objects
I am trying to figure out what algorithms there are to do surface reconstruction from 3D range data. At a first glance, it seems that the Ball pivoting algorithm (BPA) and Poisson surface reconstruction are the more established methods?
What are the established, more robust algorithm in the field other than BPA and Poisson surface reconstruction algorithm?
Recommended research publications?
Is there available source code?
I have been facing this dilemma for some months now, and made exhaustive research.
Algorithms
Mainly there are 2 categories of algorithms: computation geometry, and implicit surfaces.
Computation Geometry
They fit the mesh on the existing points.
Probably the most famous algorithm of this group is powercrust, because it is theoretically well-established - it guarantees watertight mesh.
Ball Pivoting is patented by IBM. Also, it is not suitable for pointclouds with varying point density.
Implicit functions
One fits implicit functions on the pointcloud, then uses a marching-cube-like algorithm to extract the zero-set of the function into a mesh.
Methods in this category differ mainly by the different implicit functions used.
Poisson, Hoppe's, and MPU are the most famous algorithms in this category. If you are new to the topic, i recommend to read Hoppe's thesis, it is very explanatory.
The algorithms of this category usually can be implemented so that they are able to process huge inputs very efficiently, and one can scale their quality<->speed trade-off. They are not disturbed by noise, varying point-density, holes. A disadvantage of them is that they require consistently oriented surface normals at the input points.
Implementations
You will find small number of free implementations. However it depends on whether You are going to integrate it into free software (in this case GPL license is acceptable for You) or into a commercial software (in this case You need a more liberal license). The latter is very rare.
One is in VTK. I suspect it to be difficult to integrate (no documentation is available for free), it has a strange, over-complicated architecture, and is not designed for high-performance applications. Also has some limitations for the allowed input pointclouds.
Take a look at this Poisson implementation, and after that share your experience about it with me please.
Also:
here are a few high-performance algorithms, with surface reconstruction among them.
CGAL is a famous 3d library, but it is free only for free projects.
Meshlab is a famous application with GPL.
Also (Added August 2013):
The library PCL has a module dedicated to surface reconstruction and is in active development (and is part of Google's Summer of Code). The surface module contains a number of different algorithms for reconstruction. PCL also has the ability to estimate surface normals, incase you do not have them provided with your point data, this functionality can be found in the features module. PCL is released under the terms of the BSD license and is open source software, it is free for commercial and research use.
If you want make some direct experiments with various surface reconstruction algorithms you should try MeshLab, the mesh-processing system, it is open source and it contains implementations of many of the previously cited surface reconstruction algorithms, like:
Poisson Surface Recon
a couple of MLS based approach,
a ball pivoting implementation
a variant of the Curless volume based approach
Delaunay based techniques (Alpha shapes and Voronoi filtering)
tools for computing normals from scattered point sets
and many other tools for comparing/measuring/cleaning/simplifying the resulting meshes.
Sources are protected by GPL, so you could not use them in a commercial closed source project, but it is very important to get the right feeling about the properties of the various surface reconstruction algorithms (how sensitive to noise they are, the speed, the robustness to outliers, how they preserve fine details etc etc) before starting to implement one of them.
You might start looking at some recent work in the field - currently something like Fast low-memory streaming MLS reconstruction of point-sampled surfaces by Gianmauro Cuccuru, Enrico Gobbetti, Fabio Marton, Renato Pajarola, and Ruggero Pintus. Its citations can get you going through the literature pretty quickly.
While not a mesh representation, an ex-colleague recommended me this link
to source code for a Thin Plate Spline method:
Link
Anyone tried it?
Not sure if it's exactly right for your case, since it seems weird that you omitted it, but marching cubes is commonly mentioned in cases like these.
As I had this problem too, I did develop and implement my own point cloud crust algorithm. The sources, as well as the documentation, can be found on github.com: https://github.com/meixxi/PointCloudCrust. The algorithm is implemented in Java.
Maybe, this can help you. You can find also a short python script on the page which illustrates how to use the library. Have fun!
Here on GitHub, is a open source Mesh Processing Library in C++ by Dr. Hugues Hoppe, in which the surface reconstruction program Recon is a good option for your problem...
There is 3D Delaunay tool by Geometric Tools. This tool is used DirecX and OpenGL. Unfortunately, you may need buy a book to see actual example code of the library. You still read the code and figure out.
Matlab also introduced a surface reconstruction tool using Delaunay, delaunayTriangulation class.
You might be interested in Alpha Shapes.