I want to write an image enhancement algorithm which is similar to photoshop's highlight and shadows alteration feature. Can you help me regarding what does this feature of photoshop do internally to an image?
Simple approach
To begin with, you can already find already some clue in their documentation: https://helpx.adobe.com/photoshop/using/adjust-shadow-highlight-detail.html
It's quite hard to guess from those documents which algorithm they use exactly. Below I will only try to explain some approaches I would use if I was facing this problem. Don't expect there a clear algorithm, but use my answer as pointers to drive you at least to a path.
As I understood, this algorithm improve the contrast in a local scale, meaning for each pixel it will adjust the value based on the neighborhood.
To do so you have several input parameters:
Neighborhood size (or Kernel)
Highlight Threshold: Everything above is considered as belonging to highlight
Shadow Threshold: Everything below is considered as belonging to shadow
Other ones are mentioned in the documentation, but they are not useful to understand the algorithmic concept.
1. Determine to which category the pixel belong: Highlight / Shadow / none.
For this part you might consider using either the grayscale image or the Value channel from HSV transformation.
I would take a look to the pixel and its neighborhood.
Compute statistics of the local distribution (mean and variance).
I will compare the mean to the thresholds value define previously, then use the variance to distinguish if the pixel is noisy or belonging to a contour, which on those case I'll expect a huge variance.
2. Apply the processing
In case the pixel is belonging to the shadow or highlight class you want to improve its contrast, not the "gray" but the "color" contrast.
Dumb approach:
Will be to weight your color channel according to their intra-variances.
Here is an example: Consider your pixel being: (32, 35, 50)(R,G,B) and belonging to shadow class. I will determine 3 coefficients Rc, Gc, Bc which are defined between 0.5 - 1.5 (arbitrary) which apply to the respective channel.
Since the Blue is dominant I would have a high coefficient for the blue like 1.3 and lower the importance of R and G channel with a coefficient about 0.8.
To compute these coefficients you can think to look at color variance, meaning differences between the color channels themselves and differences between each channels and the pixel mean.
Other (high-level) approaches
Laplacian Pyramids
Using the pyramids to distinguish the details in different scales and the laplacian to improve the contrast.
http://mcclanahoochie.com/blog/portfolio/opencl-image-pyramid-detail-enhancement/
https://www.darktable.org/2017/11/local-laplacian-pyramids/
Those links could be really helpful for you, especially because the sources are available and the concept are well explained.
I would advise you to continue your quest to look deeper in darktable. It's a powerful free/open-source alternative to Lightroom.
I already find some interesting stuff just by looking at their blog.
Sorry for this incomplete answer, I'll probably come back there to improve it.
All comments and suggestions are more than welcome
You can follow the following technique. It is not accurate but imitates well.
lumR = 0.299;
lumG = 0.587;
lumB = 0.114;
// we have to find luminance of the pixel
// here 0.0 <= source.r/source.g/source.b <= 1.0
// and 0.0 <= luminance <= 1.0
luminance = sqrt( lumR*pow(source.r,2.0) + lumG*pow(source.g,2.0) + lumB*pow(source.b,2.0));
// here highlights and and shadows are our desired filter amounts
// highlights/shadows should be >= -1.0 and <= +1.0
// highlights = shadows = 0.0 by default
// you can change 0.05 and 8.0 according to your needs but okay for me
h = highlights * 0.05 * ( pow(8.0, luminance) - 1.0 );
s = shadows * 0.05 * ( pow(8.0, 1.0 - luminance) - 1.0 );
output.r = source.r + h + s;
output.g = source.g + h + s;
output.b = source.b + h + s;
Related
I'm having a little bit of difficulty wrapping my head around the correct terminology to use in phrasing my question, so I'll just take a stab at it and perhaps I can get some help in clarifying it along the way toward a solution.
I want to detect some coloured lights in an image, so I need a way to:
a) determine the colour of pixels
b) determine how "intense" or "bright" they are
c) use the two values above as a threshold or criteria for whether or not to discard a given pixel
I figured brightness alone will probably not be a good way to do this, since there will be non-zero ambient light.
Thanks!
EDIT: So using MATLAB's colour thresholder, I was able to isolate the coloured lights by restricting the hue range in HSV space. Just trying to figure out a way to do this via the command line.
Well there are two separate steps. 1 is finding out what you want to isolate, and 2 is isolation
1)Seems like you got this figured out. But for the future you can use the "imtool" command. It is nearly the same as imshow, but it allows you to inspect pixel values(RGB, you would convert these to HSV using rgb2hsv), crop images, zoom, measure distances, etc. It can be really helpful.
imtool(my_im)
will open up the window, pretty simple.
2)Now that you have your values you want to isolate them. The term you are looking for is MASKING A misk is typically a binary matrix/vector with 1's (true) corresponding to areas of interest and 0's (false) elsewhere. Matlab calls these "logical" arrays. So lets just say you found your areas of interest were as follows
hue=0.2 to 0.3, saturation=don't care, brightness= greater than .5
you would create your mask by doing binary comparisons on the pixels. I will split this into three steps just so you can make sense of everything.
%% MASKING STEPS
hue_idx = 1; sat_idx =2 ; bright_idx = 3;
hue_mask = ((my_hsv_im(:,:,hue_idx ) > 0.2) & (my_hsv_im(:,:,hue_idx ) < 0.3));
%note we have no saturation mask, because it would be filled with ones
%since we dont care about the saturation values
brightness_mask = (my_hsv_im(:,:,bright_idx ) > 0.5);
total_mask = hue_mask & brightness_mask;
%% ALL THE REST
%now we mask your image, recall that 1's are ares of interest and 0's are
%nothing so just multiply your image by your mask
% the mask is a logical array size MxNx1, we need to convert it to the same
%type as our image in order to multiply them
mask_3d(:,:,hue_idx) = total_mask;
mask_3d(:,:,sat_idx) = total_mask;
mask_3d(:,:,bright_idx) = total_mask;
mask_3d = uint8(mask_3d); %this step is pretty important, if your image
%is a double use double(mask_3d) instead
masked_rgb_im = my_im .* mask_3d;
%does some plotting just for fun
figure(10);
subplot(2,3,1);imshow(my_im);title('original image');
subplot(2,3,2);imshow(hue_mask);title('hue mask');
subplot(2,3,3);imshow(brightness_mask);title('bright mask');
subplot(2,3,4);imshow(total_mask);title('total mask');
subplot(2,3,5:6);imshow(masked_rgb_im );title('masked image');
I have to remove gaussian noise from this image (before, I had to filter it and add the noise). Then, I have to use function "o" and my grade is based on how low result of this function will be. I am trying and trying different things, but I can't remove this noise so I can get a good grade :/ any help please?
img=imread('liftingbody.png');
img=double(img)/255;
maska1=[1 1 1; 1 5 1; 1 1 1]/13;
odfiltrowany=imfilter(img,maska1);
zaszumiony=imnoise(odfiltrowany,'gaussian');
nowy=wiener2(zaszumiony);
nowy4=medfilt2(nowy);
o=1/512.*sqrt(sum(sum(img-nowy4).^2));
subplot(311); imshow(img);
subplot(312); imshow(zaszumiony);
subplot(313); imshow(nowy);
Try convoluting a Gaussian filter with your noisy image to remove Gaussian noise like below:
nowx=conv2(zaszumiony,fspecial('gaussian',[3 3],1.5),'same')/(sum(sum(fspecial('gaussian',[3 3],1.5))));
It should reduce your o function somewhat.
Try playing around with the strength of the filter (i.e. the 1.5 value) and the size of the kernel (i.e. [3 3] value) to reduce the noise to a minimum.
Adding to #ALM865's answer, you can also use imfilter. In fact, this is the recommended function that you use for images as imfilter has optimizations in place specifically for images. conv2 is the more general function for any 2D signal.
I have also answered how to choose the standard deviation and ultimately the size of your a Gaussian filter / kernel here: By which measures should I set the size of my Gaussian filter in MATLAB?
In essence, once you choose which standard deviation you want, you find a floor(6*sigma) + 1 x floor(6*sigma) + 1 Gaussian kernel to use in your filtering operation. Assuming that sigma = 2, you would get a 13 x 13 kernel. As ALM865 has said, you can create a Gaussian kernel using fspecial. You specify the 'gaussian' flag, followed by the size of the kernel and the standard deviation after. As such:
sigma = 2;
width = 6*sigma + 1;
kernel = fspecial('gaussian', [width width], sigma);
out = imfilter(zaszumiony, kernel, 'replicate');
imfilter takes in the image you want to filter, the convolution kernel you want to use to filter the image, and an optional flag that specifies what happens along the image pixel borders when the kernel doesn't fit completely inside the image. 'replicate' means that it simply copies the pixels along the borders, thus replicating them. There are other options, such as padding with a value (usually zero), circular padding and symmetric padding.
Play around with the standard deviation until you get what you believe is a good result.
So the Wikipedia page for path tracing (http://en.wikipedia.org/wiki/Path_tracing) contains a naive implementation of the algorithm with the following explanation underneath:
"All these samples must then be averaged to obtain the output color. Note this method of always sampling a random ray in the normal's hemisphere only works well for perfectly diffuse surfaces. For other materials, one generally has to use importance-sampling, i.e. probabilistically select a new ray according to the BRDF's distribution. For instance, a perfectly specular (mirror) material would not work with the method above, as the probability of the new ray being the correct reflected ray - which is the only ray through which any radiance will be reflected - is zero. In these situations, one must divide the reflectance by the probability density function of the sampling scheme, as per Monte-Carlo integration (in the naive case above, there is no particular sampling scheme, so the PDF turns out to be 1)."
The part I'm having trouble understanding is the part in bold. I am familiar with PDFs but I am not quite sure how they fit into here. If we stick to the mirror example, what would be the PDF value we would divide by? Why? How would I go about finding the PDF value to divide by if I was using an arbitrary BRDF value such as a Phong reflection model or Cook-Torrance reflection model, etc? Lastly, why do we divide by the PDF instead of multiply? If we divide, don't we give more weight to a direction with a lower probability?
Let's assume that we have only materials without color (greyscale). Then, their BDRF at each point can be expressed as a single valued function
float BDRF(phi_in, theta_in, phi_out, theta_out, pointWhereObjWasHit);
Here, phi and theta are the azimuth and zenith angles of the two rays under consideration. For pure Lambertian reflection, this function would look like this:
float lambertBRDF(phi_in, theta_in, phi_out, theta_out, pointWhereObjWasHit)
{
return albedo*1/pi*cos(theta_out);
}
albedo ranges from 0 to 1 - this measures how much of the incoming light is reemitted. The factor 1/pi ensures that the integral of BRDF over all outgoing vectors does not exceed 1. With the naive approach of the Wikipedia article (http://en.wikipedia.org/wiki/Path_tracing), one can use this BRDF as follows:
Color TracePath(Ray r, depth) {
/* .... */
Ray newRay;
newRay.origin = r.pointWhereObjWasHit;
newRay.direction = RandomUnitVectorInHemisphereOf(normal(r.pointWhereObjWasHit));
Color reflected = TracePath(newRay, depth + 1);
return emittance + reflected*lambertBDRF(r.phi,r.theta,newRay.phi,newRay.theta,r.pointWhereObjWasHit);
}
As mentioned in the article and by Ross, this random sampling is unfortunate because it traces incoming directions (newRay's) from which little light is reflected with the same probability as directions from which there is lots of light. Instead, directions whence much light is reflected to the observer should be selected preferentially, to have an equal sample rate per contribution to the final color over all directions. For that, one needs a way to generate random rays from a probability distribution. Let's say there exists a function that can do that; this function takes as input the desired PDF (which, ideally should be be equal to the BDRF) and the incoming ray:
vector RandomVectorWithPDF(function PDF(p_i,t_i,p_o,t_o,point x), Ray incoming)
{
// this function is responsible to create random Rays emanating from x
// with the probability distribution PDF. Depending on the complexity of PDF,
// this might somewhat involved. It is possible, however, to do it for Lambertian
// reflection (how exactly is math, not programming):
vector randomVector;
if(PDF==lambertBDRF)
{
float phi = uniformRandomNumber(0,2*pi);
float rho = acos(sqrt(uniformRandomNumber(0,1)));
float theta = pi/2-rho;
randomVector = getVectorFromAzimuthZenithAndNormal(phi,zenith,normal(incoming.whereObjectWasHit));
}
else // deal with other PDFs
return randomVector;
}
The code in the TracePath routine would then simply look like this:
newRay.direction = RandomVectorWithPDF(lambertBDRF,r);
Color reflected = TracePath(newRay, depth + 1);
return emittance + reflected;
Because the bright directions are preferred in the choice of samples, you do not have to weight them again by applying the BDRF as a scaling factor to reflected. However, if PDF and BDRF are different for some reason, you would have to scale down the output whenever PDF>BDRF (if you picked to many from the respective direction) and enhance it when you picked to little .
In code:
newRay.direction = RandomVectorWithPDF(PDF,r);
Color reflected = TracePath(newRay, depth + 1);
return emittance + reflected*BDRF(...)/PDF(...);
The output is best, however, if BDRF/PDF is equal to 1.
The question remains why can't one always choose the perfect PDF which is exactly equal to the BDRF? First, some random distributions are harder to compute than others. For example, if there was a slight variation in the albedo parameter, the algorithm would still do much better for the non-naive sampling than for uniform sampling, but the correction term BDRF/PDF would be needed for the slight variations. Sometimes, it might even be impossible to do it at all. Imagine a colored object with different reflective behavior of red green and blue - you could either render in three passes, one for each color, or use an average PDF, which fits all color components approximately, but none perfectly.
How would one go about implementing something like Phong shading? For simplicity, I still assume that there is only one color component, and that the ratio of diffuse to specular reflection is 60% / 40% (the notion of ambient light makes no sense in path tracing). Then my code would look like this:
if(uniformRandomNumber(0,1)<0.6) //diffuse reflection
{
newRay.direction=RandomVectorWithPDF(lambertBDRF,r);
reflected = TracePath(newRay,depth+1)/0.6;
}
else //specular reflection
{
newRay.direction=RandomVectorWithPDF(specularPDF,r);
reflected = TracePath(newRay,depth+1)*specularBDRF/specularPDF/0.4;
}
return emittance + reflected;
Here specularPDF is a distribution with a narrow peak around the reflected ray (theta_in=theta_out,phi_in=phi_out+pi) for which a way to create random vectors is available, and specularBDRF returns the specular intensity from Phong's model (http://en.wikipedia.org/wiki/Phong_reflection_model).
Note how the PDFs are modified by 0.6 and 0.4 respectively.
I'm by no means an expert in ray tracing, but this seems to be classic Monte Carlo:
You have lots of possible rays, and you choose one uniformly at random and then average over lots of trials.
The distribution you used to choose one of the rays was uniform (they were all equally as likely)
so you don't have to do any clever re-normalising.
However, Perhaps there are lots of possible rays to choose, but only a few would possibly lead to useful results.We therefore bias towards picking those 'useful' possibilities with higher probability, and then re-normalise (we are not choosing the rays uniformly any more, so we can't just take the average). This is
importance sampling.
The mirror example seems to be the following: only one possible ray will give a useful result.
If we choose a ray at random then the probability we hit that useful ray is zero: this is a property
of conditional probability on continuous spaces (it's not actually continuous, it's implicitly discretised
by your computer, so it's not quite true...): the probability of hitting something specific when there are infinitely many things must be zero.
Thus we are re-normalising by something with probability zero - standard conditional probability definitions
break when we consider events with probability zero, and that is where the problem would come from.
I am using Laplacian of Gaussian for edge detection using a combination of what is described in http://homepages.inf.ed.ac.uk/rbf/HIPR2/log.htm and http://wwwmath.tau.ac.il/~turkel/notes/Maini.pdf
Simply put, I'm using this equation :
for(int i = -(kernelSize/2); i<=(kernelSize/2); i++)
{
for(int j = -(kernelSize/2); j<=(kernelSize/2); j++)
{
double L_xy = -1/(Math.PI * Math.pow(sigma,4))*(1 - ((Math.pow(i,2) + Math.pow(j,2))/(2*Math.pow(sigma,2))))*Math.exp(-((Math.pow(i,2) + Math.pow(j,2))/(2*Math.pow(sigma,2))));
L_xy*=426.3;
}
}
and using up the L_xy variable to build the LoG kernel.
The problem is, when the image size is larger, application of the same kernel is making the filter more sensitive to noise. The edge sharpness is also not the same.
Let me put an example here...
Suppose we've got this image:
Using a value of sigma = 0.9 and a kernel size of 5 x 5 matrix on a 480 × 264 pixel version of this image, we get the following output:
However, if we use the same values on a 1920 × 1080 pixels version of this image (same sigma value and kernel size), we get something like this:
[Both the images are scaled down version of an even larger image. The scaling down was done using a photo editor, which means the data contained in the images are not exactly similar. But, at least, they should be very near.]
Given that the larger image is roughly 4 times the smaller one... I also tried scaling the sigma by factor of 4 (sigma*=4) and the output was... you guessed it right, a black canvas.
Could you please help me realize how to implement a LoG edge detector that finds the same features from an input signal, even if the incoming signal is scaled up or down (scaling factor will be given).
Looking at your images, I suppose you are working in 24-bit RGB. When you increase your sigma, the response of your filter weakens accordingly, thus what you get in the larger image with a larger kernel are values close to zero, which are either truncated or so close to zero that your display cannot distinguish.
To make differentials across different scales comparable, you should use the scale-space differential operator (Lindeberg et al.):
Essentially, differential operators are applied to the Gaussian kernel function (G_{\sigma}) and the result (or alternatively the convolution kernel; it is just a scalar multiplier anyways) is scaled by \sigma^{\gamma}. Here L is the input image and LoG is Laplacian of Gaussian -image.
When the order of differential is 2, \gammais typically set to 2.
Then you should get quite similar magnitude in both images.
Sources:
[1] Lindeberg: "Scale-space theory in computer vision" 1993
[2] Frangi et al. "Multiscale vessel enhancement filtering" 1998
I'm trying to implement blend modes from the PDF specification, for my own pleasure, in SASS.
PDF Specification:
http://www.adobe.com/content/dam/Adobe/en/devnet/acrobat/pdfs/PDF32000_2008.pdf
Page 322 is the alpha compositing section.
The input values are in RGBA format, but now I'm trying to implement blending of the alpha component. How do I excatly go about doing it? From what I gather my values should be between 0.0 and 1.0, that's done. However from the specs it seems that you should blend for each color channel? Do I just average it out to get back to RGBA form for my alpha component?
Any help is appriciated, I don't mind reading blog, books etc. to get my answer, as this is purely an intellectual exercise.
Thanks in advance,
Emil
The SVG spec has a lot of good equations for various blending modes. And yes, you do have to calculate both the new alpha and the new colour -- for each channel. For standard blending modes, the alpha is calculated this way:
alpha_final = alpha_bg + alpha_fg - alpha_bg * alpha_fg
Note: I see you're considering alpha to be between 0 and 1, which is good. Alpha values in CSS are always defined as float values from 0 to 1; it's good to stick with this convention, because it makes the calculations immensely easier.
It helps to 'premultiply' each colour channel by its alpha; these are more helpful for interpreting and using the usual formulae:
colour_bg_a = colour_bg * alpha_bg
In other words:
red_bg_a = red_bg * alpha_bg
green_bg_a = green_bg * alpha_bg
blue_bg_a = blue_bg * alpha_bg
Then, for plain-jane alpha compositing (like overlaying sheets of tracing paper, also known as src-over in Porter and Duff's original paper and the SVG alpha compositing spec), you take each channel and calculate it thus:
colour_final_a = colour_fg_a + colour_bg_a * (1 - alpha_fg)
The last step is to 'un-multiply' each final colour channel value by the final alpha:
colour_final = colour_final_a / alpha_final
and put it into your mixin somehow:
rgba(red_final, green_final, blue_final, alpha_final)
The other blending modes (multiply, difference, screen, etc) are slightly more complicated formulas, but the concept for every single mode is the same:
Separate the R, G, B, and A values of both the foreground and background colours
Calculate the alpha of the new, final colour with the above formula
Pre-multiply all the R, G, and B values by their alpha value
Calculate the new, final R, G, and B values (insert blending mode formula here)
Un-multiply the final R, G, and B values by the final alpha
Clip the final R, G, and B values so that they are between 0 and 255 (necessary for some modes, but not all)
Put the colour back together again!
If you're still interested in this, I've been doing the very thing in Stylus. You can see my progress here: https://github.com/pdaoust/stylus-helpers/blob/master/blend.styl You might be able to use it as a starting point for your own Sass mixin.
The first thing I do is convert all the R, G, and B values from 0 - 255 values to 0 - 1 float values for the purposes of the calculations. I don't know if that's necessary, and it does require converting them back to 0 - 255 values. It felt right to me, and Porter and Duff worked in 0 - 1 float values in their original paper.
(I'm encountering trouble with some of the compositing modes, which produce wildly different results from the expected results that the SVG spec pictures. I suspect that the spec gives the wrong equations. If anyone knows about Porter/Duff blending modes, I'd be very grateful for their help!)