I am using Laplacian of Gaussian for edge detection using a combination of what is described in http://homepages.inf.ed.ac.uk/rbf/HIPR2/log.htm and http://wwwmath.tau.ac.il/~turkel/notes/Maini.pdf
Simply put, I'm using this equation :
for(int i = -(kernelSize/2); i<=(kernelSize/2); i++)
{
for(int j = -(kernelSize/2); j<=(kernelSize/2); j++)
{
double L_xy = -1/(Math.PI * Math.pow(sigma,4))*(1 - ((Math.pow(i,2) + Math.pow(j,2))/(2*Math.pow(sigma,2))))*Math.exp(-((Math.pow(i,2) + Math.pow(j,2))/(2*Math.pow(sigma,2))));
L_xy*=426.3;
}
}
and using up the L_xy variable to build the LoG kernel.
The problem is, when the image size is larger, application of the same kernel is making the filter more sensitive to noise. The edge sharpness is also not the same.
Let me put an example here...
Suppose we've got this image:
Using a value of sigma = 0.9 and a kernel size of 5 x 5 matrix on a 480 × 264 pixel version of this image, we get the following output:
However, if we use the same values on a 1920 × 1080 pixels version of this image (same sigma value and kernel size), we get something like this:
[Both the images are scaled down version of an even larger image. The scaling down was done using a photo editor, which means the data contained in the images are not exactly similar. But, at least, they should be very near.]
Given that the larger image is roughly 4 times the smaller one... I also tried scaling the sigma by factor of 4 (sigma*=4) and the output was... you guessed it right, a black canvas.
Could you please help me realize how to implement a LoG edge detector that finds the same features from an input signal, even if the incoming signal is scaled up or down (scaling factor will be given).
Looking at your images, I suppose you are working in 24-bit RGB. When you increase your sigma, the response of your filter weakens accordingly, thus what you get in the larger image with a larger kernel are values close to zero, which are either truncated or so close to zero that your display cannot distinguish.
To make differentials across different scales comparable, you should use the scale-space differential operator (Lindeberg et al.):
Essentially, differential operators are applied to the Gaussian kernel function (G_{\sigma}) and the result (or alternatively the convolution kernel; it is just a scalar multiplier anyways) is scaled by \sigma^{\gamma}. Here L is the input image and LoG is Laplacian of Gaussian -image.
When the order of differential is 2, \gammais typically set to 2.
Then you should get quite similar magnitude in both images.
Sources:
[1] Lindeberg: "Scale-space theory in computer vision" 1993
[2] Frangi et al. "Multiscale vessel enhancement filtering" 1998
Related
I have this 3D array in MATLAB (V: vertical, H: horizontal, t: time frame)
Figures below represent images obtained using imagesc function after slicing the array in terms of t axis
area in black represents damage area and other area is intact
each frame looks similar but has different amplitude
I am trying to visualize only defect area and get rid of intact area
I tried to use 'threshold' method to get rid of intact area as below
NewSet = zeros(450,450,200);
for kk = 1:200
frame = uwpi(:,:,kk);
STD = std(frame(:));
Mean = mean(frame(:));
for ii = 1:450
for jj =1:450
if frame(ii, jj) > 2*STD+Mean
NewSet(ii, jj, kk) = frame(ii, jj);
else
NewSet(ii, jj, kk) = NaN;
end
end
end
end
However, since each frame has different amplitude, result becomes
Is there any image processing method to get rid of intact area in this case?
Thanks in advance
You're thresholding based on mean and standard deviation, basically assuming your data is normally distributed and looking for outliers. But your model should try to distinguish values around zero (noise) vs higher values. Your data is not normally distributed, mean and standard deviation are not meaningful.
Look up Otsu thresholding (MATLAB IP toolbox has it). It's model does not perfectly match your data, but it might give reasonable results. Like most threshold estimation algorithms, it uses the image's histogram to determine the optimal threshold given some model.
Ideally you'd model the background peak in the histogram. You can find the mode, fit a Gaussian around it, then cut off at 2 sigma. Or you can use the "triangle method", which finds the point along the histogram that is furthest from the line between the upper end of the histogram and the top of the background peak. A little more complex to explain, but trivial to implement. We have this implemented in DIPimage (http://www.diplib.org), M-file code is visible so you can see how it works (look for the function threshold)
Additionally, I'd suggest to get rid of the loops over x and y. You can type frame(frame<threshold) = nan, and then copy the whole frame back into NewSet in one operation.
Do I clearly understand the question, ROI is the dark border and all it surrounds? If so I'd recommend process in 3D using some kind of region-growing technique like watershed or active snakes with markers by imregionalmin. The methods should provide segmentation result even if the border has small holes. Than just copy segmented object to a new 3D array via logic indexing.
I am trying to compute the integral image (aka summed area table) of a texture I have in the GPU memory (a camera capture), the goal being to compute the adaptive threshold of said image. I'm using OpenGL ES 2.0, and still learning :).
I did a test with a simple gaussian blur shader (vertical/horizontal pass), which is working fine, but I need a way bigger variable average area for it to give satisfactory results.
I did implement a version of that algorithm on CPU before, but I'm a bit confused on how to implement that on a GPU.
I tried to do a (completely incorrect) test with just something like this for every fragment :
#version 100
#extension GL_OES_EGL_image_external : require
precision highp float;
uniform sampler2D u_Texture; // The input texture.
varying lowp vec2 v_TexCoordinate; // Interpolated texture coordinate per fragment.
uniform vec2 u_PixelDelta; // Pixel delta
void main()
{
// get neighboring pixels values
float center = texture2D(u_Texture, v_TexCoordinate).r;
float a = texture2D(u_Texture, v_TexCoordinate + vec2(u_PixelDelta.x * -1.0, 0.0)).r;
float b = texture2D(u_Texture, v_TexCoordinate + vec2(0.0, u_PixelDelta.y * 1.0)).r;
float c = texture2D(u_Texture, v_TexCoordinate + vec2(u_PixelDelta.x * -1.0, u_PixelDelta.y * 1.0)).r;
// compute value
float pixValue = center + a + b - c;
// Result stores value (R) and original gray value (G)
gl_FragColor = vec4(pixValue, center, center, 1.0);
}
And then another shader to get the area that I want and then get the average. This is obviously wrong as there's multiple execution units operating at the same time.
I know that the common way of computing a prefix sum on a GPU is to do it in two pass (vertical/horizontal, as discussed here on this thread or or here), but isn't there a problem here as there is a data dependency on each cell from the previous (top or left) one ?
I can't seem to understand the order in which the multiple execution units on a GPU will process the different fragments, and how a two-pass filter can solve that issue. As an example, if I have some values like this :
2 1 5
0 3 2
4 4 7
The two pass should give (first columns then rows):
2 1 5 2 3 8
2 4 7 -> 2 6 13
6 8 14 6 14 28
How can I be sure that, as an example, the value [0;2] will be computed as 6 (2 + 4) and not 4 (0 + 4, if the 0 hasn't been computed yet) ?
Also, as I understand that fragments are not pixels (If I'm not mistaken), would the values I store back in one of my texture in the first pass be the same in another pass if I use the exact same coordinates passed from the vertex shader, or will they be interpolated in some way ?
Tommy and Bartvbl address your questions about a summed-area table, but your core problem of an adaptive threshold may not need that.
As part of my open source GPUImage framework, I've done some experimentation with optimizing blurs over large radii using OpenGL ES. Generally, increasing blur radii leads to a significant increase in texture sampling and calculations per pixel, with an accompanying slowdown.
However, I found that for most blur operations you can apply a surprisingly effective optimization to cap the number of blur samples. If you downsample the image before blurring, blur at a smaller pixel radius (radius / downsampling factor), and then linearly upsample, you can arrive at a blurred image that is the equivalent of one blurred at a much larger pixel radius. In my tests, these downsampled, blurred, and then upsampled images look almost identical to the ones blurred based on the original image resolution. In fact, precision limits can lead to larger-radii blurs done at a native resolution breaking down in image quality past a certain size, where the downsampled ones maintain the proper image quality.
By adjusting the downsampling factor to keep the downsampled blur radius constant, you can achieve near constant-time blurring speeds in the face of increasing blur radii. For a adaptive threshold, the image quality should be good enough to use for your comparisons.
I use this approach in the Gaussian and box blurs within the latest version of the above-linked framework, so if you're running on Mac, iOS, or Linux, you can evaluate the results by trying out one of the sample applications. I have an adaptive threshold operation based on a box blur that uses this optimization, so you can see if the results there are what you want.
AS per the above, it's not going to be fantastic on a GPU. But assuming the cost of shunting data between the GPU and CPU is more troubling it may still be worth persevering.
The most obvious prima facie solution is to split horizontal/vertical as discussed. Use an additive blending mode, create a quad that draws the whole source image then e.g. for the horizontal step on a bitmap of width n issue a call that requests the quad be drawn n times, the 0th time at x = 0, the mth time at x = m. Then ping pong via an FBO, switching the target of buffer of the horizontal draw into the source texture for the vertical.
Memory accesses are probably O(n^2) (i.e. you'll probably cache quite well, but that's hardly a complete relief) so it's a fairly poor solution. You could improve it by divide and conquer by doing the same thing in bands — e.g. for the vertical step, independently sum individual rows of 8, after which the error in every row below the final is the failure to include whatever the sums are on that row. So perform a second pass to propagate those.
However an issue with accumulating in the frame buffer is clamping to avoid overflow — if you're expecting a value greater than 255 anywhere in the integral image then you're out of luck because the additive blending will clamp and GL_RG32I et al don't reach ES prior to 3.0.
The best solution I can think of to that, without using any vendor-specific extensions, is to split up the bits of your source image and combine channels after the fact. Supposing your source image were 4 bit and your image less than 256 pixels in both directions, you'd put one bit each in the R, G, B and A channels, perform the normal additive step, then run a quick recombine shader as value = A + (B*2) + (G*4) + (R*8). If your texture is larger or smaller in size or bit depth then scale up or down accordingly.
(platform specific observation: if you're on iOS then you've hopefully already got a CVOpenGLESTextureCache in the loop, which means you have CPU and GPU access to the same texture store, so you might well prefer to kick this step off to GCD. iOS is amongst the platforms supporting EXT_shader_framebuffer_fetch; if you have access to that then you can write any old blend function you like and at least ditch the combination step. Also you're guaranteed that preceding geometry has completed before you draw so if each strip writes its totals where it should and also to the line below then you can perform the ideal two-pixel-strips solution with no intermediate buffers or state changes)
What you attempt to do cannot be done in a fragment shader. GPU's are by nature very different to CPU's by executing their instructions in parallel, in massive numbers at the same time. Because of this, OpenGL does not make any guarantees about execution order, because the hardware physically doesn't allow it to.
So there is not really any defined order other than "whatever the GPU thread block scheduler decides".
Fragments are pixels, sorta-kinda. They are pixels that potentially end up on screen. If another triangle ends up in front of another, the previous calculated colour value is discarded. This happens regardless of whatever colour was stored at that pixel in the colour buffer previously.
As for creating the summed area table on the GPU, I think you may first want to look at GLSL "Compute Shaders", which are specifically made for this sort of thing.
I think you may be able to get this to work by creating a single thread for each row of pixels in the table, then have every thread "lag behind" by 1 pixel compared to the previous row.
In pseudocode:
int row_id = thread_id()
for column_index in (image.cols + image.rows):
int my_current_column_id = column_index - row_id
if my_current_column_id >= 0 and my_current_column_id < image.width:
// calculate sums
The catch of this method is that all threads should be guaranteed to execute their instructions simultaneously without getting ahead of one another. This is guaranteed in CUDA, but I'm not sure whether it is in OpenGL compute shaders. It may be a starting point for you, though.
It may look surprising for the beginner but the prefix sum or SAT calculation is suitable for parallelization. As the Hensley algorithm is the most intuitive to understand (also implemented in OpenGL), more work-efficient parallel methods are available, see CUDA scan. The paper from Sengupta discuss parallel method which seems state-of-the-art efficient method with reduce and down swap phases. These are valuable materials but they do not enter OpenGL shader implementations in detail. The closest document is the presentation you have found (it refers to Hensley publication), since it has some shader snippets. This is the job which is doable entirely in fragment shader with FBO Ping-Pong. Note that the FBO and its texture need to have internal format set to high precision - GL_RGB32F would be best but I am not sure if it is supported in OpenGL ES 2.0.
I have a greyscale image similar to the one below that I have achieved after some post-processing steps (image 0001). I would like a vector corresponding to the bottom of the lower bright strip (as depicted in image 0001b). I can use im2bw with various thresholds to achieve the vectors in image 0002 (the higher the threshold value the higher the tendency for the vector line to blip upwards, the lower the threshold the higher the tendency for the line to blip downwards)..and then I was thinking of going through each vector and measuring arclength over some increment (maybe 100 pixels or so) and choosing that vector with the lowest arclength...and adding that 100 pixel stretch to the final vector, creating a frankenstein-like vector using the straightest segments from each of the thresholded vectors.. I should also mention that when there are multiple straightish/parallel vectors, the top one is the best fit.
First off, is there some better strategy I should be employing here to find that line on image 0001? (this needs to be fast so some long fitting code wouldn't work). If my current Frankenstein's monster solution works, any suggestions as to how to best go about this?
Thanks in advance
image=im2bw(image,0.95); %or 0.85, 0.75, 0.65, 0.55
vec=[];
for v=1:x
for x=1:z
if image(c,v)==1
vec(v)=c;
end
end
end
vec=fastsmooth(vec,60,20,1);
Here is the modified version of what I originally did. It works well on on your images. If you want subpixel resolution, you can implement an active contour model with some fitting function.
files = dir('*.png');
filenames = {files.name};
for ifile=1:length(filenames)
%%
% read image
im0 = double(imread(filenames{ifile}));
%%
% remove background by substracting a convolution with a mask
lobj=100;
convmask = ones(lobj,1)/lobj;
im=im0-conv2(im0,convmask,'same');
im(im<0)=0;
imagesc(im);colormap gray;axis image;
%%
% use canny edge filter, alowing extremely weak edge to exist
bw=edge(im,'canny',[0.01,0.3]);
% use close operation on image to close gaps between lines
% the kernel is a flat rectangular so that it helps to connect horizontal
% gaps
se=strel('rectangle',[10,30]);
bw=imdilate(bw,se);
% thin the lines to be single pixel line
bw=bwmorph(bw,'thin',inf);
% connect H bridge
bw=bwmorph(bw,'bridge');
imagesc(bw);colormap gray;axis image;
%% smooth the image, find the decreasing region, and apply the mask
imtmp = imgaussfilt(im0,3);
imtmp = diff(imtmp);
imtmp = [imtmp(1,:);imtmp];
intensity_decrease_mask = imtmp < 0;
bw = bw & intensity_decrease_mask;
imagesc(bw);colormap gray;axis image;
%%
% find properties of the lines, and find the longest lines
cc=regionprops(bw,'Area','PixelList','Centroid','MajorAxisLength','PixelIdxList');
% now select any lines that is larger than eighth of the image width
cc=cc([cc.MajorAxisLength]>size(bw,2)/8);
%%
% select lines that has average intensity larger than gray level
for i=1:length(cc)
cc(i).meanIntensity = mean(im0(sub2ind(size(im0),cc(i).PixelList(:,2), ...
cc(i).PixelList(:,1) )));
end
cc=cc([cc.meanIntensity]>150);
cnts=reshape([cc.Centroid],2,length(cc))';
%%
% calculate the minimum distance to the bottom right of each edge
for i=1:length(cc)
cc(i).distance2bottomright = sqrt(min((cc(i).PixelList(:,2)-size(im,1)).^2 ...
+ (cc(i).PixelList(:,1)-size(im,2)).^2));
end
% select the bottom edge
[~,minindex]=min([cc.distance2bottomright]);
bottomedge = cc(minindex);
%% clean up the lines a little bit
bwtmp = false(size(bw));
bwtmp(bottomedge.PixelIdxList)=1;
% find the end points to the most left and right
endpoints = bwmorph(bwtmp, 'endpoints');
[endy,endx] = find(endpoints);
[~,minind]=min(endx);
[~,maxind]=max(endx);
pos_most_left = [endx(minind),endy(minind)];
pos_most_right = [endx(maxind),endy(maxind)];
% select the shortest path between left and right
dists = bwdistgeodesic(bwtmp,pos_most_left(1),pos_most_left(2)) + ...
bwdistgeodesic(bwtmp,pos_most_right(1),pos_most_right(2));
dists(isnan(dists))=inf;
bwtmp = imregionalmin(dists);
bottomedge=regionprops(bwtmp,'PixelList');
%% plot the lines
imagesc(im0);colormap gray;axis image;hold on;axis off;
for i=1:length(cc)
plot(cc(i).PixelList(:,1),cc(i).PixelList(:,2),'b','linewidth',2);hold on;
end
plot(bottomedge.PixelList(:,1),bottomedge.PixelList(:,2),'r','linewidth',2);hold on;
print(gcf,num2str(ifile),'-djpeg');
% pause
end
I am not sure this answers your question directly, but I have a lot of experiencing fitting arrays (or matrices in my case) to 3D raster images. We were using relatively low power machines (standard i7 processors 32 gb ram), and had to perform the fitting very quickly (<30 seconds). We also had to validate the fit with a variety of parameters (and again these were 3D rasters fit to a point cloud matrix).
Anyways, the process we used was the fminsearch function internal to Matlab. Documentation can be found here: http://www.mathworks.com/help/optim/functionlist.html
We would start with a plain point-cloud and perform successive manipulations on a per pixel basis to adjust the point-cloud to the raster. Essentially walking through each pixel in the raster to produce the lowest offset between the point cloud and the raster.
I will try to search for some code this afternoon and update my answer, but I might explore this option for your case. I would imagine you could fit a curve to certain pixels (e.g. white pixels) both rapidly and accurately by setting up an optimization function.
I also could help more if I understood your objective better. Are you just trying to fit a line to the high-albedo/white areas?
In the way of example: I can fit a 3D point cloud to the following image by starting with a standard point cloud, the 3D raster, and a minimization function (in this case just RMS error of each individual point in the z axis). Throw an fmin function on there and in a few seconds you get a modified point cloud that fits much better than the standard.
I have to remove gaussian noise from this image (before, I had to filter it and add the noise). Then, I have to use function "o" and my grade is based on how low result of this function will be. I am trying and trying different things, but I can't remove this noise so I can get a good grade :/ any help please?
img=imread('liftingbody.png');
img=double(img)/255;
maska1=[1 1 1; 1 5 1; 1 1 1]/13;
odfiltrowany=imfilter(img,maska1);
zaszumiony=imnoise(odfiltrowany,'gaussian');
nowy=wiener2(zaszumiony);
nowy4=medfilt2(nowy);
o=1/512.*sqrt(sum(sum(img-nowy4).^2));
subplot(311); imshow(img);
subplot(312); imshow(zaszumiony);
subplot(313); imshow(nowy);
Try convoluting a Gaussian filter with your noisy image to remove Gaussian noise like below:
nowx=conv2(zaszumiony,fspecial('gaussian',[3 3],1.5),'same')/(sum(sum(fspecial('gaussian',[3 3],1.5))));
It should reduce your o function somewhat.
Try playing around with the strength of the filter (i.e. the 1.5 value) and the size of the kernel (i.e. [3 3] value) to reduce the noise to a minimum.
Adding to #ALM865's answer, you can also use imfilter. In fact, this is the recommended function that you use for images as imfilter has optimizations in place specifically for images. conv2 is the more general function for any 2D signal.
I have also answered how to choose the standard deviation and ultimately the size of your a Gaussian filter / kernel here: By which measures should I set the size of my Gaussian filter in MATLAB?
In essence, once you choose which standard deviation you want, you find a floor(6*sigma) + 1 x floor(6*sigma) + 1 Gaussian kernel to use in your filtering operation. Assuming that sigma = 2, you would get a 13 x 13 kernel. As ALM865 has said, you can create a Gaussian kernel using fspecial. You specify the 'gaussian' flag, followed by the size of the kernel and the standard deviation after. As such:
sigma = 2;
width = 6*sigma + 1;
kernel = fspecial('gaussian', [width width], sigma);
out = imfilter(zaszumiony, kernel, 'replicate');
imfilter takes in the image you want to filter, the convolution kernel you want to use to filter the image, and an optional flag that specifies what happens along the image pixel borders when the kernel doesn't fit completely inside the image. 'replicate' means that it simply copies the pixels along the borders, thus replicating them. There are other options, such as padding with a value (usually zero), circular padding and symmetric padding.
Play around with the standard deviation until you get what you believe is a good result.
I have some data set where each object has a Value and Price. I want to apply Gaussian Blur to their Price using their Value. Since my data has only 1 component to use in blurring, I am trying to apply 1D Gaussian blur.
My code does this:
totalPrice = 0;
totalValue = 0;
for each object.OtherObjectsWithinPriceRange()
totalPrice += price;
totalValue += Math.Exp(-value*value);
price = totalPrice/totalValue;
I see good results, but the 1D Gaussian blur algorithms I see online seems to use deviations, sigma, PI, etc. Do I need them, or are they strictly for 2D Gaussian blurs? They combine these 1D blur passes as vertical and horizontal so they are still accounting for 2D.
Also I display the results as colors but the white areas are a little over 1 (white). How can I normalize this? Should I just clamp the values to 1? That's why I am wondering if I am using the correct formula.
Your code applies some sort of a blur, though definitely not Gaussian. The Gaussian blur would look something like
kindaSigma = 1;
priceBlurred = object.price;
for each object.OtherObjectsWithinPriceRange()
priceBlurred += price*Math.Exp(-value*value/kindaSigma/kindaSigma);
and that only assuming that value is proportional to a "distance" between the object and other objects within price range, whatever this "distance" in your application means.
To your questions.
2D Gaussian blur is completely equivalent to a combination of vertical and horizontal 1D Gaussian blurs done one ofter another. That's how thee 2D Gaussian blur is usually implemented in practice.
You don't need any PI or sigmas as a multiplicative factor for the Gaussian - those have an effect of merely scaling an image and can be safely ignored.
The sigma (standard deviation) under the exponent has a major impact on the result, but it is not possible for me to tell you if you need it or not. It depends on your application.
Want more blur: use larger kindaSigma in the snippet above.
Want less blur: use smaller kindaSigma.
When kindaSigma is too small, you won't notice any blur at all. When kindaSigma is too large, the Gaussian blur effectively transforms itself into a moving average filter.
Play with it and choose what you need.
I am not sure I understand your normalization question. In image processing it is common to store each color component (R,G,B) as unsigned char. So black color is represented by (0,0,0) and white color by (255,255,255). Of course, you are free to decided to choose a different presentation form and take white color as 1. But keep in mind that for the visualization packages that are using standard 8-bit presentation, the value of 1 means almost black color. So you will likely need to manipulate and renormalize your image before display.