Related
I have the following image:
What I want to do is "id" the individual strips based on their dominant color. What is the best approach to do this?
What I've done is used the image's value (HSV) and make a distribution on that value's occurrence. The problem is, for strip0 values [27=32191, 28=5433, others=8] strip1 values [26=7107, 27=23111, others=22]. I can't get a definitive distinction.
The project's main goal is to compare an actual yellow-colored paper to the strips and determine which strip is the most similar.
First, since you know the boundaries of each strip in the reference image, the only problem possible here is that your reference image is noisy. A relatively overkill way to handle that is clustering the colors in each strip and taking the cluster's centroid as the representative color of the strip. In order to get a more meaningful response here, consider the CIELAB colorspace for this step. Doing this, and converting the results back to RGB, for the first strip I get the rgb triplet (0.949375, 0.879872, 0.147898), and for the second strip (0.945324, 0.857322, 0.129756) (each channel in range [0, 1]).
When you get a new image, you perform the same operation. But there are a lot of problems here. For instance, how are you handling the white balance in this input image ? Supposing you have no such problem, then now it is only a matter of finding the nearest color to the one you just found by the same process. To find the nearest color you have to use a meaningful colorspace for such thing too, and CIELAB is recommended again since the well established Delta-E functions are defined on it. See http://en.wikipedia.org/wiki/Color_difference for some such metrics, the simplest being the euclidean distance in CIELAB.
Calibrate your equipment. If you do not calibrate your equipment, you will have arbitrary errors between the test sample and the reference. Lighting is part of your equipment.
Use edge detection and your knowledge of the reference strip's geometry (strips are equal width) to determine sampling regions. For each sampling region, extract an internal patch.
For the test strip, compute an image where each pixel is the max difference within a sampling window (e.g. 5x5). This will let you identify a relatively homogeneous region which is dissimilar to the outside region (i.e. the paper). Extract a patch.
Use downsampling to find an integrated color for each patch per svnpenn's advice. You can look at other computation methods later, but this should work quite well.
For weights wh, ws, wv, compute similarity = whabs(h0-h1) + wsabs(s0-s1) + wv*abs(v0-v1) between the test color and each reference color. You can look at other distance measures later, but this should work quite well. Start with equal weights. One perk to this method is that it behaves well regardless of the dimension or combination of dimensions under which the reference strip varies.
Sort the results to find the most similar and second most similar matches. Note that similarity is set up so zero is an exact match, and a big number is a poor match. Use the ratio of these two results to estimate the quality of the most similar match - if the first two matches are very close, it's probably not a great match to either.
You can scan through all the colors and use a hashtable to keep track of how many pixels of each color there are.
Take those numbers and, remembering which colors they correspond to, sort them in decreasing order.
Look at the sorted list of numbers and find the difference between each consecutive pair of numbers. Keep track the indices in the list of the two numbers that resulted in each difference. Sort this difference list.
Look at the maximum number in the difference list. You now have the biggest drop-off between two sets of pixels. Go find which was the bigger one. Everything with this number of pixels and above is a dominant color. Everything below is a sub-dominant color. Now you know how many dominant colors you have, and what they are.
Should be pretty easy from there to do whatever it is you want to do.
The only time this wouldn't work is if some of the noise was of the same color as a strip, so much so that it corrupted your data.
In this case, you would use a different approach, which you can also use in the first case - looking at runs. Go through the pixels, and each time you find a new color, look at how many of the following pixels are of the same color.
Use the method described earlier to cluster the colors into dominant and non-dominant, for the same result.
In both cases, if you know that the picture is of vertical strips, you could limit the number of horizontal lines of colors you look at to make things go faster.
You could split the image into sections, then resize each section to one pixel. This is an example using the whole image
$ convert Y82IirS.jpg -resize 1x1 txt:
# ImageMagick pixel enumeration: 1,1,255,srgb
0,0: (220,176, 44) #DCB02C srgb(220,176,44)
Average colour of an image
Which approach would you suggest for automatically classifying type found in images? The samples are likely large, with black text on a white background.
The categories are defined here, with some examples on each (Google Books link): http://bit.ly/9Mnu7P This is an extended version of the VOX-ATypI classification system.
My initial thoughts on this were to train the system with lots of single character samples from each category, but I'm wondering if there's a better way that would eliminate the need to do the comparison one letter at a time.
First, you need to extract features for classification. Typefaces are generally distinguished by the thickness of lines, the presence of serifs, "circularity" of character parts. Thus, the possible features are:
The fraction of the number of black pixels on the fixed area.
Try to apply math morphology erosion few times (and/or use different masks) and compute this fraction
Compute the mean compactness of a character: perimeter^2 / area
After applying erosion, count the number of connected components for a character
Compute the elongation and other image moments, also the direction
etc
I see two options here: either compute mean features for all characters, or try to classify letters first, and than classify the font based on some specific letters (so, you train the different classifier for a different letter). It's hard to say which one is better in your case.
As for specific learning algorithm, Random Forest seems to be a good place to start. There's an implementation in the OpenCV library.
I want to implement the two above mentioned image resampling algorithms (bicubic and Lanczos) in C++. I know that there are dozens of existing implementations out there, but I still want to make my own. I want to make it partly because I want to understand how they work, and partly because I want to give them some capabilities not found in mainstream implementations (like configurable multi-CPU support and progress reporting).
I tried reading Wikipedia, but the stuff is a bit too dry for me. Perhaps there are some nicer explanations of these algorithms? I couldn't find anything either on SO or Google.
Added: Seems like nobody can give me a good link about these topics. Can anyone at least try to explain them here?
The basic operation principle of both algorithms is pretty simple. They're both convolution filters. A convolution filter that for each output value moves the convolution functions point of origin to be centered on the output and then multiplies all the values in the input with the value of the convolution function at that location and adds them together.
One property of convolution is that the integral of the output is the product of the integrals of the two input functions. If you consider the input and output images, then the integral means average brightness and if you want the brightness to remain the same the integral of the convolution function needs to add up to one.
One way how to understand them is to think of the convolution function as something that shows how much input pixels influence the output pixel depending on their distance.
Convolution functions are usually defined so that they are zero when the distance is larger than some value so that you don't have to consider every input value for every output value.
For lanczos interpolation the convolution function is based on the sinc(x) = sin(x*pi)/x function, but only the first few lobes are taken. Usually 3:
lanczos(x) = {
0 if abs(x) > 3,
1 if x == 0,
else sin(x*pi)/x
}
This function is called the filter kernel.
To resample with lanczos imagine you overlay the output and input over eachother, with points signifying where the pixel locations are. For each output pixel location you take a box +- 3 output pixels from that point. For every input pixel that lies in that box, calculate the value of the lanczos function at that location with the distance from the output location in output pixel coordinates as the parameter. You then need to normalize the calculated values by scaling them so that they add up to 1. After that multiply each input pixel value with the corresponding scaling value and add the results together to get the value of the output pixel.
Because lanzos function has the separability property and, if you are resizing, the grid is regular, you can optimize this by doing the convolution horizontally and vertically separately and precalculate the vertical filters for each row and horizontal filters for each column.
Bicubic convolution is basically the same, with a different filter kernel function.
To get more detail, there's a pretty good and thorough explanation in the book Digital Image Processing, section 16.3.
Also, image_operations.cc and convolver.cc in skia have a pretty well commented implementation of lanczos interpolation.
While what Ants Aasma says roughly describes the difference, I don't think it is particularly informative as to why you might do such a thing.
As far as links go, you are asking a very basic question in image processing, and any decent introductory textbook on the subject will describe this. If I remember correctly, Gonzales and Woods is decent on it, but I'm away from my books and can't check.
Now on to the particulars, it should help to think about what you are doing fundamentally. You have a square lattice of measurements that you want to interpolate new values for. In the simple case of upsampling, lets imagine you want a new measurement in between every one that you already have (e.g. double the resolution).
Now you won't get the "correct" value, because in general you don't have that information. So you have to estimate it. How to do this? A very simple way would be to linearly interpolate. Everyone knows how to do this with two points, you just draw a line between them, and read the new value off the line (in this case, at the half way point).
Now an image is two dimensional, so you really want to do this in both the left-right and up-down directions. Use the result for your estimate and voila you have "bilinear" interpolation.
The main problem with this is that it isn't very accurate, although it's better (and slower) than the "nearest neighbor" approach which is also very local and fast.
To address the first problem, you want something better than a linear fit of two points, you want to fit something to more data points (pixels), and something that can be nonlinear. A good trade off on accuracy and computational cost is something called a cubic spline. So this will give you a smooth fit line, and again you approximate your new "measurement" by the value it takes in the middle. Do this in both directions and you've got "bicubic" interpolation.
So that's more accurate, but still heavy. One way to address the speed issue is to use a convolution, which has the nice property that in the Fourier domain, it's just a multiplication, so we can implement it quite quickly. But you don't need to worry about the implementation to understand that the convolution result at any point is one function (your image) being integrated in product another, typically much smaller support (the part that is non-zero) function called the kernel), after that kernel has been centered over that particular point. In the discrete world, these are just sums of the products.
It turns out that you can design a convolution kernel that has properties quite like the cubic spline, and use that to get a fast "bicubic"
Lancsoz resampling is a similar thing, with slightly different properties in the kernel, which primarily means they will have different characteristic artifacts. You can look up the details of these kernel functions easily enough (I'm sure wikipedia has them, or any intro text). The implementations used in graphics programs tend to be highly optimized and sometimes have specialized assumptions which make them more efficient but less general.
I would like suggest the following article for a basic understanding of different image interpolation methods image interpolation via convolution. If you want to try more interpolation methods, the imageresampler is a nice open source project to begin with.
In my opinion image interpolation can be understood from two aspects, one is from function fitting perspective, and one is from convolution perspective. For example, the spline interpolation explained in image interpolation via convolution is well explained from function fitting perspective in Cubic interpolation.
Additionally, image interpolation is always related to a specific application, for example image zooming, image rotation and so on. In fact for a specific application, image interpolation can be implemented i.n a smart way. For example, image rotation can be implemented via a three-shearing method, and during each shearing operation different one-dimension interpolation algorithms can be implemented.
I've been reading up a bit on anti-aliasing and it seems to make sense, but there is one thing I'm not too sure of. How exactly do you find the maximum frequency of a signal (in the context of graphics).
I realize there's more than one case so I assume there is more than one answer. But first let me state a simple algorithm that I think would represent maximum frequency so someone can tell me if I'm conceptualizing it the wrong way.
Let's say it's for a 1 dimensional,finite, and greyscale image (in pixels). Am I correct in assuming you could simply scan the entire pixel line (in the spatial domain) looking for a for the minimum oscillation and the inverse of that smallest oscillation would be the maximum frequency?
Ex values {23,26,28,22,48,49,51,49}
Frequency:Pertaining to Set {}
(1/2) = .5 : {28,22}
(1/4) = .25 : {22,48,49,51}
So would .5 be the maximum frequency?
And what would be the ideal way to calculate this for a similar pixel line as the one above?
And on a more theoretical note, what if your sampling input was infinite (more like the real world)? Would a valid process be sort of like:
Predetermine a decent interval for point sampling
Determine max frequency from point sampling
while(2*maxFrequency > pointSamplingInterval)
{
pointSamplingInterval*=2
Redetermine maxFrequency from point sampling (with new interval)
}
I know these algorithms are fraught with inefficiencies, so what are some of the preferred ways? (Not looking for something crazy-optimized, just fundamentally better concepts)
The proper way to approach this is using a Fourier Transform (in practice, an FFT,or fast fourier transform)
The theory works as follows: if you have an set of pixels with color/grayscale, then we can say that the image is represented by pixels in the "spatial domain"; that is, each individual number specifies the image at a particular spatial location.
However, what we really want is a representation of the image in the "frequency domain". Instead of each individual number specifying each pixel, each number represents the amplitude of a particular frequency in the image as a whole.
The tool which converts from the "spatial domain" to the "frequency domain" is the Fourier Transform. The output of the FT will be a sequence of numbers specifying the relative contribution of different frequencies.
In order to find the maximum frequency, you perform the FT, and look at the amplitudes that you get for the high frequencies - then it is just a matter of searching from the highest frequency down until you hit your "minimum significant amplitude" threshold.
You can code your own FFT, but it is much easier in practice to use a pre-packaged library such as FFTW
You don't scan a signal for the highest frequency and then choose your sampling frequency: You choose a sampling frequency that's high enough to capture the things you want to capture, and then you filter the signal to remove high frequencies. You throw away everything higher than half the sampling rate before you sample it.
Am I correct in assuming you could
simply scan the entire pixel line (in
the spatial domain) looking for a for
the minimum oscillation and the
inverse of that smallest oscillation
would be the maximum frequency?
If you have a line of pixels, then the sampling is already done. It's too late to apply an antialiasing filter. The highest frequency that could be present is half the sampling frequency ("1/2px", I guess).
And on a more theoretical note, what
if your sampling input was infinite
(more like the real world)?
Yes, that's when you use the filter. First, you have a continuous function, like a real-life image (infinite sampling rate). Then you filter it to remove everything above fs/2, then you sample it at fs (digitize the image into pixels). Cameras don't actually do any filtering, which is why you get Moire patterns when you photograph bricks, etc.
If you're anti-aliasing computer graphics, you have to think of the ideal continuous mathematical function first, and think through how you would filter it and digitize it to produce the output on the screen.
For instance, if you want to generate a square wave with a computer, you can't just naively alternate between maximum and minimum values. That would be just like sampling a real life signal without filtering first. The higher harmonics wrap back into the baseband and cause lots of spurious spikes in the spectrum. You need to generate points as if they were sampled from a filtered continuous mathematical function:
I think this article from the O'Reilly site might also be useful to you ... http://www.onlamp.com/pub/a/python/2001/01/31/numerically.html ... in there they're referring to frequency analysis of sound files but you it gives you the idea.
I think what you need is an application of Fourier Analysis (http://en.wikipedia.org/wiki/Fourier_analysis). I've studied this but never used it so take it with a pinch of salt but I believe if you apply it correctly to your set of numbers you will get a set of frequencies which are components of the series and then you can pick off the highest one.
I can't point you at a piece of code that does this but I'm sure it would be out there somewhere .
I have a list of numeric values. I may normalize the values if needed.
I need to transform this list to a list of colors (in HSL, RGB or any other color model — I can always do conversion later).
For any given value the color must be the same every time.
The more different two given numeric values are, the more contrast corresponding values should be.
All used colors must be as contrast to each other as possible (this is a soft limitation, rough solution would do).
Note that list is rather large (thousands of numbers), so simply squeezing all numbers into a single color channel would produce too dense results.
You could consider using a 3D space-filling curve through your chosen colour space. I'll second Mark's CIELAB suggestion, wish I'd known about that last time I had to solve a similar problem.
Whatever algorithm you finally settle on, you might try the CIELAB color space. It normalizes the differences in human color perception, so that equal numeric spacing gives equal perceptual differences.
See: How to automatically generate N "distinct" colors?
It would be best to normalize your values, and run them through the code I suggested (where hue == your value), building a map/hash. (You can use a hash-style function instead, which is probably more efficient.)
You can "randomize" lightness (or brightness, depending on your model) and saturation using some predetermined bits of your number, for example.
Why not use shades of gray? Just calculate the min/max values and use that to translate each number into a different shade from white to black.
I know it's not colors, but in my opinion it'll be easier to interpret the results. I can tell what it means when something is darker vs. lighter, but who is to say that, for example, green is a higher value than orange?