What's up with CITemperatureAndTint having vector inputs? - opengl-es

OK, so the Core Image filter Temperature and Tint has two inputs, neutral and targetNeutral. However, my biggest issue is the fact that they're both two-component vectors, meaning each has two numeric inputs. I would expect the first to be from say 2500 to 10000. What would the vector be for?

The essential purpose of performing temperature and tint adjustment is to correct the white balance of a captured image: to account for the ambient illumination of the scene and adjust colors so that the image appears more like it was shot in "white" light (roughly 6500K).
Temperature relates to the warmth or coolness of an image, and is normally characterized qualitatively as orange-ish or bluish.
Tint refers to the deviation toward green or magenta of colors at the same temperature. Note that tint (defined as such) is mostly independent of color temperature. (Take a look at a CIE diagram with the Planck locus and isotherms drawn on it to develop your intuition about this. Here's one: http://en.wikipedia.org/wiki/File:Planckian-locus.png).
So, when you are interested in performing white balance adjustment (whether to make an image appear more realistic, or for artistic purposes), there are four different parameters you must supply: the temperature of the initial image, the tint of the white point as it appears in the initial image, the desired color temperature of the output image, and how tint-shifted the "neutral" tones should appear in the output image. The temperature/tint combination of an image is a function of the ambient light in the scene and the response of the material being imaged, and both temperature and tint are necessary to meaningfully characterize the white balance of a captured image.
This is why CITemperatureAndTint takes two vectors: it wants the two pairs of (temperature, tint) just described.
Now, if you want to create a UI for controlling white balance, you don't actually have to give the user control of all four of these values. Instead, hold the second vector (TargetNeutral) at a constant (6500, 0) and allow the user to adjust the other vector (Neutral). With this arrangement, the user will be selecting the perceived color temperature and tint shift of the original image. (You might choose instead to hold the Neutral vector constant and allow the user to tweak the Target Neutral vector; this may be more appropriate in contexts where the user wants to adjust the white balance artistically, but the correlation between the selected values and the resulting image is not as obvious).

#warrenm said in great detail, I will add the code to those who need to use it directly!
Temperature:
var value: CGFloat = 0 // Min: -3000, Max: 3000
return image.applyingFilter("CITemperatureAndTint", parameters: [
"inputNeutral": CIVector.init(x: value + 6500, y: 0),
"inputTargetNeutral": CIVector.init(x: 6500, y: 0)
])
Tint:
var value: CGFloat = 0 // Min: -100, Max: 100
return image.applyingFilter("CITemperatureAndTint", parameters: [
"inputNeutral": CIVector.init(x: 6500, y: value),
"inputTargetNeutral": CIVector.init(x: 6500, y: 0)
])

Related

Calculate 3D distance based on change in intensity

I have three sections (top, mid, bot) of grayscale images (3D). In each section, I have a point with coordinates (x,y) and intensity values [0-255]. The distance between each section is 20 pixels.
I created an illustration to show how those images were generated using a microscope:
Illustration
Illustration (side view): red line is the object of interest. Blue stars represents the dots which are visible in top, mid, bot section. The (x,y) coordinates of these dots are known. The length of the object remains the same but it can rotate in space - 'out of focus' (illustration shows a rotating line at time point 5). At time point 1, the red line is resting (in 2D image: 2 dots with a distance equal to the length of the object).
I want to estimate the x,y,z-coordinate of the end points (represents as stars) by using the changes in intensity, the knowledge about the length of the object and the information in the sections I have. Any help would be appreciated.
Here is an example of images:
Bot section
Mid section
Top section
My 3D PSF data:
https://drive.google.com/file/d/1qoyhWtLDD2fUy2zThYUgkYM3vMXxNh64/view?usp=sharing
Attempt so far:
enter image description here
I guess the correct approach would be to record three images with slightly different z-coordinates for your bot and your top frame, then do a 3D-deconvolution (using Richardson-Lucy or whatever algorithm).
However, a more simple approach would be as I have outlined in my comment. If you use the data for a publication, I strongly recommend to emphasize that this is just an estimation and to include the steps how you have done it.
I'd suggest the following procedure:
Since I do not have your PSF-data, I fake some by estimating the PSF as a 3D-Gaussiamn. Of course, this is a strong simplification, but you should be able to get the idea behind it.
First, fit a Gaussian to the PSF along z:
[xg, yg, zg] = meshgrid(-32:32, -32:32, -32:32);
rg = sqrt(xg.^2+yg.^2);
psf = exp(-(rg/8).^2) .* exp(-(zg/16).^2);
% add some noise to make it a bit more realistic
psf = psf + randn(size(psf)) * 0.05;
% view psf:
%
subplot(1,3,1);
s = slice(xg,yg,zg, psf, 0,0,[]);
title('faked PSF');
for i=1:2
s(i).EdgeColor = 'none';
end
% data along z through PSF's center
z = reshape(psf(33,33,:),[65,1]);
subplot(1,3,2);
plot(-32:32, z);
title('PSF along z');
% Fit the data
% Generate a function for a gaussian distibution plus some background
gauss_d = #(x0, sigma, bg, x)exp(-1*((x-x0)/(sigma)).^2)+bg;
ft = fit ((-32:32)', z, gauss_d, ...
'Start', [0 16 0] ... % You may find proper start points by looking at your data
);
subplot(1,3,3);
plot(-32:32, z, '.');
hold on;
plot(-32:.1:32, feval(ft, -32:.1:32), 'r-');
title('fit to z-profile');
The function that relates the intensity I to the z-coordinate is
gauss_d = #(x0, sigma, bg, x)exp(-1*((x-x0)/(sigma)).^2)+bg;
You can re-arrange this formula for x. Due to the square root, there are two possibilities:
% now make a function that returns the z-coordinate from the intensity
% value:
zfromI = #(I)ft.sigma * sqrt(-1*log(I-ft.bg))+ft.x0;
zfromI2= #(I)ft.sigma * -sqrt(-1*log(I-ft.bg))+ft.x0;
Note that the PSF I have faked is normalized to have one as its maximum value. If your PSF data is not normalized, you can divide the data by its maximum.
Now, you can use zfromI or zfromI2 to get the z-coordinate for your intensity. Again, I should be normalized, that is the fraction of the intensity to the intensity of your reference spot:
zfromI(.7)
ans =
9.5469
>> zfromI2(.7)
ans =
-9.4644
Note that due to the random noise I have added, your results might look slightly different.

HTML canvas fillRect with low opacity doesn't affect dark pixels

Repeatedly drawing a semi-opaque black rectangle over the entire canvas before each animation frame is an easy way to get an afterimage effect for moving shapes and it gives me exactly what I need - up to a point. With too slow a fade it doesn't fade all the way to black. Here's an example:
var canv = document.createElement('canvas');
document.body.appendChild(canv);
var ctx = canv.getContext('2d');
ctx.fillStyle = 'rgba(0, 0, 0, 1)';
ctx.fillRect(0, 0, 100, 100);
ctx.fillStyle = 'rgba(255, 255, 255, 1)';
ctx.fillRect(20, 20, 60, 60);
window.requestAnimationFrame(doFade);
function doFade() {
// Never fades away completely
ctx.fillStyle = 'rgba(0, 0, 0, 0.02)';
ctx.fillRect(20, 20, 60, 60);
window.requestAnimationFrame(doFade);
}
jsfiddle
This looks to me like a numeric precision problem - you can't expect the canvas to keep floating point pixel values around - but I'm not sure how to get around this.
I tried reading the image into a pattern, blanking the canvas, and then filling with the pattern at lower opacity in the hope that I could make rounding error work in my favor, but it seems to have the same result.
Short of reading out the image data and setting to black any pixels below a certain threshold, which would be prohibitively slow, I'm running out of ideas and could use some suggestions.
Thanks!
I thought I'd share my solution for the benefit of anyone else who might run into this problem. I was hoping to avoid doing any pixel-level manipulation, but beyond a certain threshold it's just not possible with the built-in canvas operations because the underlying bitmap is only 8 bits per channel and small fades will work out to less than one least significant bit and won't have any effect on the image data.
My solution was to create an array representing the age of each pixel. After each frame is drawn, I scan the imageData array, looking only at the alpha channel. If the alpha is 255 I know the pixel has just been written, so I set the age to 0 and set the alpha to 254. For any other non-zero alpha values, I increment the pixel age and then set the new alpha based on the pixel age.
The mapping of pixel age to alpha value is done with a lookup table that's populated when the fade rate is set. This lets me use whatever decay curve I want without extra math during the rendering loop.
The CPU utilization is a bit higher, but it's not too much of a performance hit and it can do smooth fades over several seconds and always fades entirely to black eventually.

How does the algorithm to color the song list in iTunes 11 work? [closed]

Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
The new iTunes 11 has a very nice view for the song list of an album, picking the colors for the fonts and background in function of album cover. Anyone figured out how the algorithm works?
I approximated the iTunes 11 color algorithm in Mathematica given the album cover as input:
How I did it
Through trial and error, I came up with an algorithm that works on ~80% of the albums with which I've tested it.
Color Differences
The bulk of the algorithm deals with finding the dominant color of an image. A prerequisite to finding dominant colors, however, is calculating a quantifiable difference between two colors. One way to calculate the difference between two colors is to calculate their Euclidean distance in the RGB color space. However, human color perception doesn't match up very well with distance in the RGB color space.
Therefore, I wrote a function to convert RGB colors (in the form {1,1,1}) to YUV, a color space which is much better at approximating color perception:
(EDIT: #cormullion and #Drake pointed out that Mathematica's built-in CIELAB and CIELUV color spaces would be just as suitable... looks like I reinvented the wheel a bit here)
convertToYUV[rawRGB_] :=
Module[{yuv},
yuv = {{0.299, 0.587, 0.114}, {-0.14713, -0.28886, 0.436},
{0.615, -0.51499, -0.10001}};
yuv . rawRGB
]
Next, I wrote a function to calculate color distance with the above conversion:
ColorDistance[rawRGB1_, rawRGB2_] :=
EuclideanDistance[convertToYUV # rawRGB1, convertToYUV # rawRGB2]
Dominant Colors
I quickly discovered that the built-in Mathematica function DominantColors doesn't allow enough fine-grained control to approximate the algorithm that iTunes uses. I wrote my own function instead...
A simple method to calculate the dominant color in a group of pixels is to collect all pixels into buckets of similar colors and then find the largest bucket.
DominantColorSimple[pixelArray_] :=
Module[{buckets},
buckets = Gather[pixelArray, ColorDistance[#1,#2] < .1 &];
buckets = Sort[buckets, Length[#1] > Length[#2] &];
RGBColor ## Mean # First # buckets
]
Note that .1 is the tolerance for how different colors must be to be considered separate. Also note that although the input is an array of pixels in raw triplet form ({{1,1,1},{0,0,0}}), I return a Mathematica RGBColor element to better approximate the built-in DominantColors function.
My actual function DominantColorsNew adds the option of returning up to n dominant colors after filtering out a given other color. It also exposes tolerances for each color comparison:
DominantColorsNew[pixelArray_, threshold_: .1, n_: 1,
numThreshold_: .2, filterColor_: 0, filterThreshold_: .5] :=
Module[
{buckets, color, previous, output},
buckets = Gather[pixelArray, ColorDistance[#1, #2] < threshold &];
If[filterColor =!= 0,
buckets =
Select[buckets,
ColorDistance[ Mean[#1], filterColor] > filterThreshold &]];
buckets = Sort[buckets, Length[#1] > Length[#2] &];
If[Length # buckets == 0, Return[{}]];
color = Mean # First # buckets;
buckets = Drop[buckets, 1];
output = List[RGBColor ## color];
previous = color;
Do[
If[Length # buckets == 0, Return[output]];
While[
ColorDistance[(color = Mean # First # buckets), previous] <
numThreshold,
If[Length # buckets != 0, buckets = Drop[buckets, 1],
Return[output]]
];
output = Append[output, RGBColor ## color];
previous = color,
{i, n - 1}
];
output
]
The Rest of the Algorithm
First I resized the album cover (36px, 36px) & reduced detail with a bilateral filter
image = Import["http://i.imgur.com/z2t8y.jpg"]
thumb = ImageResize[ image, 36, Resampling -> "Nearest"];
thumb = BilateralFilter[thumb, 1, .2, MaxIterations -> 2];
iTunes picks the background color by finding the dominant color along the edges of the album. However, it ignores narrow album cover borders by cropping the image.
thumb = ImageCrop[thumb, 34];
Next, I found the dominant color (with the new function above) along the outermost edge of the image with a default tolerance of .1.
border = Flatten[
Join[ImageData[thumb][[1 ;; 34 ;; 33]] ,
Transpose # ImageData[thumb][[All, 1 ;; 34 ;; 33]]], 1];
background = DominantColorsNew[border][[1]];
Lastly, I returned 2 dominant colors in the image as a whole, telling the function to filter out the background color as well.
highlights = DominantColorsNew[Flatten[ImageData[thumb], 1], .1, 2, .2,
List ## background, .5];
title = highlights[[1]];
songs = highlights[[2]];
The tolerance values above are as follows: .1 is the minimum difference between "separate" colors; .2 is the minimum difference between numerous dominant colors (A lower value might return black and dark gray, while a higher value ensures more diversity in the dominant colors); .5 is the minimum difference between dominant colors and the background (A higher value will yield higher-contrast color combinations)
Voila!
Graphics[{background, Disk[]}]
Graphics[{title, Disk[]}]
Graphics[{songs, Disk[]}]
Notes
The algorithm can be applied very generally. I tweaked the above settings and tolerance values to the point where they work to produce generally correct colors for ~80% of the album covers I tested. A few edge cases occur when DominantColorsNew doesn't find two colors to return for the highlights (i.e. when the album cover is monochrome). My algorithm doesn't address these cases, but it would be trivial to duplicate iTunes' functionality: when the album yields less than two highlights, the title becomes white or black depending on the best contrast with the background. Then the songs become the one highlight color if there is one, or the title color faded into the background a bit.
More Examples
With the answer of #Seth-thompson and the comment of #bluedog, I build a little Objective-C (Cocoa-Touch) project to generate color schemes in function of an image.
You can check the project at :
https://github.com/luisespinoza/LEColorPicker
For now, LEColorPicker is doing:
Image is scaled to 36x36 px (this reduce the compute time).
It generates a pixel array from the image.
Converts the pixel array to YUV space.
Gather colors as Seth Thompson's code does it.
The color's sets are sorted by count.
The algorithm select the three most dominant colors.
The most dominant is asigned as Background.
The second and third most dominants are tested using the w3c color contrast formula, to check if the colors has enought contrast with the background.
If one of the text colors don't pass the test, then is asigned to white or black, depending of the Y component.
That is for now, I will be checking the ColorTunes project (https://github.com/Dannvix/ColorTunes) and the Wade Cosgrove project for new features. Also I have some new ideas for improve the color scheme result.
Wade Cosgrove of Panic wrote a nice blog post describing his implementation of an algorithm that approximates the one in iTunes. It includes a sample implementation in Objective-C.
You might also checkout ColorTunes which is a HTML implementation of the Itunes album view which is using the MMCQ (median cut color quantization) algorithm.
I just wrote a JS library implementing roughly the same algorithm that the one described by #Seth. It is freely available on github.com/arcanis/colibrijs, and on NPM as colibrijs.
With #Seth's answer I implemented the algorithm to get the dominant color in the two lateral borders of a picture using PHP and Imagick.
https://gist.github.com/philix/5688064#file-simpleimage-php-L81
It's being used to fill the background of cover photos in http://festea.com.br
I asked the same question in a different context and was pointed over to http://charlesleifer.com/blog/using-python-and-k-means-to-find-the-dominant-colors-in-images/ for a learning algorithm (k Means) that rougly does the same thing using random starting points in the image. That way, the algorithm finds dominant colors by itself.

Color tint and temperature

Though I have found a lot of topics on color tint and temperature, but till now I have not seen any definite solution, which is the reason I am creating this post..My apologies for that.
I am interested in adjusting color temp and tint in images from RGB values, somewhat similar to the iPhoto application found in iOS where it can be adjusted with a slider bar from left to right.
Whatever I have found, temp and tint are orthogonal properties, where temp adjustment is along the blue (left; cool colors)--yellow(right; warm colors) and tint along the green (left) -- magenta (right) axis.
How do I adjust them using formulas from RGB values i.e., uderlying implementation of the color temp and tint slider bars.
I can convert them to HSV space and then I can rotate the hue wheel channel towards those (blue, yello, green, magenta) angles, but how to do them in a systematic fashion similar to the slider bar implementation by changing gradually from low level (middle of the slider bar) to high level (right/left ends of the slider bar).
Thanks!
You should try using HSL instead of HSV. HSL saturation separates itself from the hue and luminosity has very definitive range when it comes to mathematical calculation.
In HSL, to add tint you move the L factor between 50-100 and to add shade the L factor varies between 0-50. Also saturation for HSL controls the tone directly unlike HSV.
For temperature, you have to devise your own stratagy changing the color between red and blue but one golden hint that I can give you is "every pure RGB color has one of 3 color values as zero, second fixed to 255 and 3rd varies with the factor of 255/60.
Hope this helps-
Whereas color temparature is a physical value, its expression
in terms of RGB values
not
trivial. If all you need is a pair of orthogonal axes in the RGB colorspace for the visual adjustment of white balance, they can be defined with relative ease in such a way as to resemble the true color temperature and its derivative the tint.
Let us name our RGB temperature BY—for the balance between blue and yellow, and our RGB tint GR—for the balance balance between green and red. Now, these functions must satisfy the following obvious requirements:
They shall not depend on brightness, or be invariant to multiplication of all the RGB components by the same factor:
BY(r,g,b) = BY(kr, kg, kb),
GR(r,g,b) = GR(kr, kg, kb).
They shall be zero for neutral gray:
BY(0,0,0) = 0,
GR(0,0,0) = 0.
They shall belong the to same range, symmetrical around zero point. I will use [-1..+1]
Any combination of BY and GR shall define a valid color.
Now, one of the ways to define them could be:
BY = (r + g - 2b)/(r + g + 2b),
GR = (r - g )/(r + g) .
so that each pair of BY and GR determines a specific proportion
r:g:b = (1 + BY)(1 + GR)
(1 + BY)(1 - GR)
1 - BY
The following image shows the colors of maximum brightness on our BY-GR plane. BY is directed right, GR down, and the neutral point (0,0) is at the center:
Proper
adjustment of white balance consists of multiplication of the linear RGB values by individual factors:
r_new = wb_r * r_old
g_new = wb_g * g_old
b_new = wb_b * b_old
It happens to work on gamma-compressed RGB too, but not so well on sRGB, because of a
piece-wise
definition of its transfer function, but the distortion will be small and often unnoticeable. If you want a perfect adjustment, however, make sure to work in linear RGB.
Once a BY-GR pair is chosen and the corresponding RGB proportion calculated, only one degree of freedom remains—the overall multiplier (see req. 1). Choose it so that no pixels become clipped.

Algorithm challenge: Generate color scheme from an image

Background
So, I'm working on a fresh iteration of a web app. And, we've found that our users are obsessed with being lazy. Really lazy. In fact, the more work we do for them, the more they love the service. A portion of the existing app requires the user to select a color scheme to use. However, we have an image (a screenshot of the user's website), so why can't we just satiate their laziness and do it for them? Answer: We can, and it will be a fun programming exercise! :)
The Challenge
Given an image, how do you create a corresponding color scheme? In other words, how do you select the primary X colors in an image (where X is defined by the web app). The image used in our particular situation is a screenshot of the user's website, taken at full resolution (e.g. 1280x1024). (Note: Please simply describe your algorithm - there's no need to post actual pseudocode.)
Bonus points (street cred points, not actual SO points) for:
Describing an algorithm that is simple yet effective. Code is how we create - keep it simple and beautiful.
Allowing the user to tweak the color scheme according to various 'moods' such as 'Colorful', 'Bright', 'Muted', 'Deep', etc. (a la Kuler)
Describing a method for reliably determining the main text color used in the website screenshot (will likely require its own, separate, algo).
Inspiration
There are several existing sites that perform a similar function. Feel free to check them out and ask yourself, "How would I duplicate this? How could I improve it?"
http://www.pictaculous.com/
http://www.cssdrive.com/imagepalette/index.php
http://kuler.adobe.com/#create/fromanimage
To find the primary X colors, screencap the app. Run a color histogram on the image. The top X colors in the histogram are the theme. Edit: if gradients are used, you'll want to pick distinct "peaks" of colors; that is, you may have a whole bunch of colors right around "orange" if orange is one of the main colors used in the gradients. Effectively, just enforce a certain amount of distance between your colors chosen from the histogram.
Tweaking the color scheme can best be done in HSV space; convert your colors to HSV space, and if the users want it to be "Brighter", increase the Value, if they want it to be more "Colorful", increase the Saturation, etc.
Determining the text color can best be done by characterizing areas of high variability (high frequency in Fourier space). Within those areas, you should have either: two colors, text and background, in which case your text is the lesser-used color; or you'll have several colors, text and background image colors, in which case the text color is the most common color.
You can take a look at:
https://github.com/dcollien/Dreamcoat
which does this in CoffeeScript (literate coffee, so it's well documented)
Demo here: http://dcollien.github.io/Dreamcoat/test.html
It has both a colour quantization approach, and a KMeans approach which are combined.
I do this to find the palette used for images (of artwork).
I start with imagemagick and resize a large image down to a workable size (i.e. 400 px on largest dimension.) This actually helps convert subtle local color differences into fewer pixels with an average of those colors.
Loop through each pixel present in the resized image, reading the RGB values for each pixel, convert the RGB to HSB and store the HSB values to an array.
For each pixel color found, I then divide the Hue range (0,255) by 16, the Saturation range (0,100) by 10, and Brightness range (0,100) by 10. Round the result up or down to an integer. This helps group pixels into categories of similar colors.
So a pixel with HSB of 223,64,76 would be in the category 14,6,8
Within each category, you can still find the exact color of each pixel, but for the most part the categories themselves are a close color match to the source image.
Choose to divide the HSB into finer divisions if you want better color replication from the categories. ie. divide each H,S,B by 8,5,5 instead of 16,10,10.
Count up the most prevalent color categories, sort and display. I discard color categories with very few pixel counts.
Note: This is really designed for artwork that has very few pixels with identical color values (i.e. paintings with shadows and gradients.)
For the most part, an HTML page probably has more pixels that exactly match a specific color value (i.e. background color, text color, etc. are all going to be the same color wherever they appear.)
Color quantization is the same process used to choose the palette for low-color GIFs. To get a color palette from a photographic image, I used Nick Rabinowitz’ quantize.js which is based on Leptonica’s MMCQ (modified median cut quantization).
Live web app, about.
Divide the screen image into a grid of r-many rectangles, in an n by m "grid", each with width (total width / n) and height (total height / m).
1a. Assign a weight to high-profile areas of the screen, such as the left-off-center area.
1b. For each rectangle, assign the pixels into a space of (color,frequency)
For each rectangle R, frequency distribution f_R, and weight W_R:
2a. Determine the i-th scheme color (e.g. i = 1 <--> background color) by scanning the "top frequency", "second frequency" (i.e. f_R[i,:]) for each block.
2b. For each i, put it in a score table, (color_i,score) where score = f_R[i,"frequency"] * W_R
2c. The top scorer for each i will be the i-th scheme color.
Theoretically, if you have a lot of "blue on white" or "red on black", you should get white primary, blue secondary, or black primary, red secondary, for example.
For your text color, either base this directly on a calculation off of background color, or choose secondary color, and if the V difference of HSV is too low, base the color off of the computed scheme color, but augment the V value.
PseudoCode:
float[][] weights =
{ { 1.0, 3.0, 5.0, 5.0, 3.0, 1.0, 1.0, 1.0, 1.0 },
{ 2.0, 6.0, 7.0, 7.0, 6.0, 2.0, 3.0, 3.0, 2.0 },
{ 2.0, 8.0, 9.0, 9.0, 7.0, 3.0, 6.0, 6.0, 3.0 },
{ 2.0, 8.0, 9.0, 9.0, 7.0, 2.0, 3.0, 3.0, 2.0 },
{ 2.0, 7.0, 9.0, 9.0, 7.0, 2.0, 1.0, 1.0, 1.0 },
{ 2.0, 6.0, 7.0, 7.0, 6.0, 2.0, 3.0, 3.0, 1.0 },
{ 1.0, 3.0, 5.0, 5.0, 3.0, 2.0, 6.0, 6.0, 2.0 },
{ 1.0, 1.0, 2.0, 2.0, 1.0, 2.0, 6.0, 6.0, 2.0 },
{ 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 3.0, 3.0, 1.0 } };
// Leave the following implementations to the imagination:
void DivideImageIntoRegions( Image originalImage, out Image[][] regions );
void GetNthMostCommonColorInRegion( Image region, int n, out Color color );
TKey FindMaximum<TKey, TValue>( Map<TKey, TValue> map );
// The method:
Color[] GetPrimaryScheme( Image image, int ncolors, int M = 9, int N = 9 )
{
Color[] scheme = new Color[ncolors];
Image[][] regions = new Image[M][N];
DivideImageIntoRegions( image, regions );
for( int i = 0; i < ncolors; i++ )
{
Map<Color, float> colorScores = new Map<Color, float>();
for( int m = 0; m < M; m++ )
for( int n = 0; n < N; n++ )
{
Color theColor;
GetNthMostCommonColorInRegion( region, i, theColor );
if( colorScores[theColor] == null )
{ colorScores[theColor] = 0; }
colorScores[theColor] += weights[m][n];
}
scheme[i] = FindMaximum( colorScores );
}
return scheme;
}
Looking at the above, it's clear that if there is a region with little variability, it will have the same second-most-common color as most-common color. To adjust, the second-most-common color in such a case might be null, which one could guard against:
if( theColor != null )
continue;
if( colorScores[theColor] == null )
{ colorScores[theColor] = 0; }
colorScores[theColor] += weights[m][n];
}
The name of the type of algorithm you want is Color Quantization.
Unfortunately I don't have any source code available for you, but I'm sure a google search could turn something up.
In particular, the Dr. Dobb's Journal article on the subject seems promising.
Similar to McWafflestix's solution, the specifics will need to be tweaked, but my general approach would be...
(I agree that HSV is the right space)
Grab a histogram of the image, filter it to smooth the noise, and find the highest score where V and S are in a (possibly dynamic) gamut of likely "subject" colors. A red bird on a blue sky will require that we be smart enough not to base our scheme on the blue, but on the red. This may require some guesses about photo composition, like "centered in the frame" and "rule of thirds" analysis could give you a probability of a color being relevant.
Regardless, this is our the base color.
Along the lines of Kuler, calculate colors that compliment the base by moving around the color wheel. Extra points for a calculated compliment if it also appeared prominently in the histogram from step 1.
Use the base color and calculated compliments to derive pleasing adjunct colors, such as lighter and darker versions of each, more or less saturated, etc.
There are already a lot of good suggestion how to find the primary colors and I would try similar approaches. For finding the text color, I have another suggestion.
Calculate the histogram for each line in the image from top to bottom. Every time you reach the base line of a line there should be a strong drop in the frequency of the text color. The frequency will remain low until you reach the upper case letters of the next line followd by a second step when you reach the lower case letters.
If there is another strong peak that becomes even larger when you hit the base line, you have found the background color. A gradient background will smooth this peak and the changes of the peaks - when you enter or leave a new line - will be smoothed by antialiasing.
Below are some suggestions and discussion around different approaches for generating a color scheme from an image:
First, embed/plot your pixels in some color space. This can be RGB, HSL, or some other color space. Then you can use one of the following to generate a color scheme:
Creating a histogram of the color space - This involves splitting the space into a grid, and counting the pixels in each grid cell. Select the top N cells (histogram buckets) with the most pixels, and average the pixels in each to produce a color per cell. This can be your color scheme.
Median Cut or some other space partitioning technique - This is a nice improvement over #1, as it will split the space by looking at the data.
Clustering Pixels - Cluster the pixels into groups using one of many clustering techniques (k-means, mean-shift, etc.). Then average the pixels in each group to generate a color scheme.
I wrote a more detailed post on the three above approaches here
I also wrote an interactive web app that allows you to load an image an create generate a color palette using one of the above three approaches. You can find the code for it on github
I'm a bit late to this, but I would implement a Kohonen Map (http://en.wikipedia.org/wiki/Self-organizing_map) in a 3d colour space. The number of points on the map would be the number of distinct colours you wanted for your palette, then train your map using all of the pixels from the image. I've not tried this myself but I'm sure someone else has thought of it already.
Average the hue, saturation and brightness separately while keeping the min/max values.
Lock the target hue of all the colours to the average and interpolate the saturation and brightness for the x points between the boundaries. This should return a scheme with a colour cast the same as the foto but with a simple variation. Maybe you'll even get the Apple look.
Just hope you don't get 3 shades of dog puke.

Resources