MATLAB: layer detection, vector combination and selection by tortuosity/arclength - image

I have a greyscale image similar to the one below that I have achieved after some post-processing steps (image 0001). I would like a vector corresponding to the bottom of the lower bright strip (as depicted in image 0001b). I can use im2bw with various thresholds to achieve the vectors in image 0002 (the higher the threshold value the higher the tendency for the vector line to blip upwards, the lower the threshold the higher the tendency for the line to blip downwards)..and then I was thinking of going through each vector and measuring arclength over some increment (maybe 100 pixels or so) and choosing that vector with the lowest arclength...and adding that 100 pixel stretch to the final vector, creating a frankenstein-like vector using the straightest segments from each of the thresholded vectors.. I should also mention that when there are multiple straightish/parallel vectors, the top one is the best fit.
First off, is there some better strategy I should be employing here to find that line on image 0001? (this needs to be fast so some long fitting code wouldn't work). If my current Frankenstein's monster solution works, any suggestions as to how to best go about this?
Thanks in advance
image=im2bw(image,0.95); %or 0.85, 0.75, 0.65, 0.55
vec=[];
for v=1:x
for x=1:z
if image(c,v)==1
vec(v)=c;
end
end
end
vec=fastsmooth(vec,60,20,1);

Here is the modified version of what I originally did. It works well on on your images. If you want subpixel resolution, you can implement an active contour model with some fitting function.
files = dir('*.png');
filenames = {files.name};
for ifile=1:length(filenames)
%%
% read image
im0 = double(imread(filenames{ifile}));
%%
% remove background by substracting a convolution with a mask
lobj=100;
convmask = ones(lobj,1)/lobj;
im=im0-conv2(im0,convmask,'same');
im(im<0)=0;
imagesc(im);colormap gray;axis image;
%%
% use canny edge filter, alowing extremely weak edge to exist
bw=edge(im,'canny',[0.01,0.3]);
% use close operation on image to close gaps between lines
% the kernel is a flat rectangular so that it helps to connect horizontal
% gaps
se=strel('rectangle',[10,30]);
bw=imdilate(bw,se);
% thin the lines to be single pixel line
bw=bwmorph(bw,'thin',inf);
% connect H bridge
bw=bwmorph(bw,'bridge');
imagesc(bw);colormap gray;axis image;
%% smooth the image, find the decreasing region, and apply the mask
imtmp = imgaussfilt(im0,3);
imtmp = diff(imtmp);
imtmp = [imtmp(1,:);imtmp];
intensity_decrease_mask = imtmp < 0;
bw = bw & intensity_decrease_mask;
imagesc(bw);colormap gray;axis image;
%%
% find properties of the lines, and find the longest lines
cc=regionprops(bw,'Area','PixelList','Centroid','MajorAxisLength','PixelIdxList');
% now select any lines that is larger than eighth of the image width
cc=cc([cc.MajorAxisLength]>size(bw,2)/8);
%%
% select lines that has average intensity larger than gray level
for i=1:length(cc)
cc(i).meanIntensity = mean(im0(sub2ind(size(im0),cc(i).PixelList(:,2), ...
cc(i).PixelList(:,1) )));
end
cc=cc([cc.meanIntensity]>150);
cnts=reshape([cc.Centroid],2,length(cc))';
%%
% calculate the minimum distance to the bottom right of each edge
for i=1:length(cc)
cc(i).distance2bottomright = sqrt(min((cc(i).PixelList(:,2)-size(im,1)).^2 ...
+ (cc(i).PixelList(:,1)-size(im,2)).^2));
end
% select the bottom edge
[~,minindex]=min([cc.distance2bottomright]);
bottomedge = cc(minindex);
%% clean up the lines a little bit
bwtmp = false(size(bw));
bwtmp(bottomedge.PixelIdxList)=1;
% find the end points to the most left and right
endpoints = bwmorph(bwtmp, 'endpoints');
[endy,endx] = find(endpoints);
[~,minind]=min(endx);
[~,maxind]=max(endx);
pos_most_left = [endx(minind),endy(minind)];
pos_most_right = [endx(maxind),endy(maxind)];
% select the shortest path between left and right
dists = bwdistgeodesic(bwtmp,pos_most_left(1),pos_most_left(2)) + ...
bwdistgeodesic(bwtmp,pos_most_right(1),pos_most_right(2));
dists(isnan(dists))=inf;
bwtmp = imregionalmin(dists);
bottomedge=regionprops(bwtmp,'PixelList');
%% plot the lines
imagesc(im0);colormap gray;axis image;hold on;axis off;
for i=1:length(cc)
plot(cc(i).PixelList(:,1),cc(i).PixelList(:,2),'b','linewidth',2);hold on;
end
plot(bottomedge.PixelList(:,1),bottomedge.PixelList(:,2),'r','linewidth',2);hold on;
print(gcf,num2str(ifile),'-djpeg');
% pause
end

I am not sure this answers your question directly, but I have a lot of experiencing fitting arrays (or matrices in my case) to 3D raster images. We were using relatively low power machines (standard i7 processors 32 gb ram), and had to perform the fitting very quickly (<30 seconds). We also had to validate the fit with a variety of parameters (and again these were 3D rasters fit to a point cloud matrix).
Anyways, the process we used was the fminsearch function internal to Matlab. Documentation can be found here: http://www.mathworks.com/help/optim/functionlist.html
We would start with a plain point-cloud and perform successive manipulations on a per pixel basis to adjust the point-cloud to the raster. Essentially walking through each pixel in the raster to produce the lowest offset between the point cloud and the raster.
I will try to search for some code this afternoon and update my answer, but I might explore this option for your case. I would imagine you could fit a curve to certain pixels (e.g. white pixels) both rapidly and accurately by setting up an optimization function.
I also could help more if I understood your objective better. Are you just trying to fit a line to the high-albedo/white areas?
In the way of example: I can fit a 3D point cloud to the following image by starting with a standard point cloud, the 3D raster, and a minimization function (in this case just RMS error of each individual point in the z axis). Throw an fmin function on there and in a few seconds you get a modified point cloud that fits much better than the standard.

Related

Draw a curve in the segmented image using matlab

I have a segmented image as shown here
i want to fit a curve along the top pixels of the segmented image(show as red curve) and i want to find the top point along the curve show in blue. I have already worked on basic idea like traversing through the top to bottom and collecting the top point along each column. i want to know is there any easy solution for this problem like directly taking out the boundary pixels and find the top point.I am using MATLAB for this problem
%download the image
img = logical(imread('http://i.stack.imgur.com/or2iX.png'));
%for some reason it appeared RGB with big solid borders.
%to monochrome
img = img(:,:,1);
%remove borders
img = img(~all(img,2), ~all(img,1));
%split into columns
cimg = num2cell(img,1);
%find first nonzero element per column
ridx = cellfun(#(x) find(x,1,'first'), cimg);
figure, imshow(img)
hold on
%image dim1 is Y, dim2 is X
plot(1:size(img,2),ridx-1,'r','linewidth',2)
%find top point
[yval, xval] = min(ridx);
If you want a smoother curve, try polyfit/polyval
#EDIT
If we want the line to have break at gaps between connected components, we should change the code to something like
bord_idx = sub2ind(size(img), ridx, 1:size(img,2));
regs=regionprops(bwlabel(img),'pixelidxlist');
regs_idx = struct2cell(regs);
split_step = cellfun(#(x) sum(ismember(bord_idx,x)), regs_idx);
split_step = split_step(split_step>0);
split_yvals = mat2cell(ridx',split_val);
split_xvals = mat2cell([1:size(img,2)]',split_val);
figure, imshow(img)
hold on
for k = 1:length(split_step),
plot(split_xvals{k}, split_yvals{k}, 'r', 'linewidth', 2),
end
However, the result is not ideal if one region is positioned over the other. If the "shadowed" points are needed, you should try bwtraceboundary or convexhull and find where the border turns down
As far as "simplest matlab solution" by which I think you mean built in matlab functions: imclose()->edge()->bwboundaries()->findpeaks()'on each boundary'->'filter results based on width and magnitude of peaks'. *you will need to tune all the parameters in these functions, I am just listing what would get you there if appropriately applied.
As far as processing speed is concerned, I think I would have done exactly what you did, basically collecting the top edge from a top down column search and then looking for the point of highest inflection. As soon as you start doing processing of any type, you start doing several operations per pixel which will quickly become more expensive than your initial search (just requires that your image and target are simple enough)
That being said, here are some ideas that may help:
1:If you run a sufficiently heavy closing (dilate->erode), that should fill in all that garbage at the bottom.
2: If you know that your point of interest is not at left or right of picture (boundaries), you could take the right and left edge points and calculate a slope to be applied as an offset to flatten the whole image.
3: If your image always has the large dark linear region below the peak as seen here, you could locate those edges with houghlines looking for verticals and then search only the columns between them.
4: If speed is a concern, you could do a more sophisticated search pattern than left to right, as your peak has a pretty good distribution around it which could help with faster localization of maxima.

MATLAB image processing technique

I have this 3D array in MATLAB (V: vertical, H: horizontal, t: time frame)
Figures below represent images obtained using imagesc function after slicing the array in terms of t axis
area in black represents damage area and other area is intact
each frame looks similar but has different amplitude
I am trying to visualize only defect area and get rid of intact area
I tried to use 'threshold' method to get rid of intact area as below
NewSet = zeros(450,450,200);
for kk = 1:200
frame = uwpi(:,:,kk);
STD = std(frame(:));
Mean = mean(frame(:));
for ii = 1:450
for jj =1:450
if frame(ii, jj) > 2*STD+Mean
NewSet(ii, jj, kk) = frame(ii, jj);
else
NewSet(ii, jj, kk) = NaN;
end
end
end
end
However, since each frame has different amplitude, result becomes
Is there any image processing method to get rid of intact area in this case?
Thanks in advance
You're thresholding based on mean and standard deviation, basically assuming your data is normally distributed and looking for outliers. But your model should try to distinguish values around zero (noise) vs higher values. Your data is not normally distributed, mean and standard deviation are not meaningful.
Look up Otsu thresholding (MATLAB IP toolbox has it). It's model does not perfectly match your data, but it might give reasonable results. Like most threshold estimation algorithms, it uses the image's histogram to determine the optimal threshold given some model.
Ideally you'd model the background peak in the histogram. You can find the mode, fit a Gaussian around it, then cut off at 2 sigma. Or you can use the "triangle method", which finds the point along the histogram that is furthest from the line between the upper end of the histogram and the top of the background peak. A little more complex to explain, but trivial to implement. We have this implemented in DIPimage (http://www.diplib.org), M-file code is visible so you can see how it works (look for the function threshold)
Additionally, I'd suggest to get rid of the loops over x and y. You can type frame(frame<threshold) = nan, and then copy the whole frame back into NewSet in one operation.
Do I clearly understand the question, ROI is the dark border and all it surrounds? If so I'd recommend process in 3D using some kind of region-growing technique like watershed or active snakes with markers by imregionalmin. The methods should provide segmentation result even if the border has small holes. Than just copy segmented object to a new 3D array via logic indexing.

Find peak (regions) in 2D data

I am looking to find peak regions in 2D data (if you will, grayscale images or 2D landscapes, created through a Hough transform). By peak region I mean a locally maximal peak, yet NOT a single point but a part of the surrounding contributing region that goes with it. I know, this is a vague definition, but maybe the word mountain or the images below will give you an intuition of what I mean.
The peaks marked in red (1-4) are what I want, the ones in pink (5-6) examples for the "grey zone", where it would be okay if those smaller peaks are not found but also okay if they are.
Images contain between 1-20 peaked regions, different in height. The 2D data for above surf plot is shown below with a possible result (orange corresponds to Peak 1, green corresponds to Peak 2 a/b, ...). Single images for tests can be found in the description links:
Image left: input image - - - - middle: (okaish) result - - - - right: result overlayed over image.
The result above was produced using simple thresholding (MATLAB code):
% thresh_scale = 15; % parameter: how many thresholding steps
% thresh_perc = 6; % parameter: threshold at which we clip
thresh = multithresh(H,thresh_scale);
q_image = imquantize(H, thresh);
q_image(q_image <= thresh_perc) = 0; % regions under threshold are thrown away
q_image(q_image > thresh_perc) = 1; % ... while all others are preserved
q_image = imbinarize(q_image); % binarize for further processing
B = bwareaopen(q_image, nhood_minsize); % Filter really small regions
[L, L_num] = bwlabel(B); % <- result % Label connected components
Some values like these (15 and 6) often work fine if there are few similar peaks, but this isn't consistent if more peaks are present or they vary a lot. I mainly have two problems, that also can't be fixed by simply adjusting the parameters:
higher peaks can mask lower (but clearly distinguishable) peaks. Since the threshold is relative to the highest peak, other peaks may fall below.
in some cases the valley between two peaks is above the threshold, merging several peaks into one (as can be observed with Peak 2 a/b).
I also don't want a huge region for a high peak, so the peak region should probably be defined as some percentage of the mountain. I figured instead of a global thresholding, I'd rather have a method that finds peak regions relative to their immediate environment. I've looked into mean-shift and MSER segmentation, but those seem to be suited for segmenting real images, not kind of synthetic data.
Somehow I imagined filling a negative of the landscape with a certain amount of water would give me the regions I'm looking for: basins that fill and spread with how the surrounding regions are shaped. Like pouring water over below image and the resulting waterpools are the regions I'm looking for.
I thought that is what the floodfill or watershed algorithm do, but floodfill seems like something completely else and the watershed results are not at all what I had in mind, also when applying some preprocessing that I thought could help (clipped at 1/10):
Or when using the same clipping threshold as with above example (clipped at 6/15):
Produced with this code (MATLAB):
thresh = multithresh(H, 10); % set to either 10 || 15 for the examples
q_image = imquantize(H, thresh);
mask = false(size(q_image)); % create clipping mask...
mask(q_image > 1) = true; % ... to remove lowest 10% || lowest 6/15
% show with: figure, imshow(mask);
% OPTIONAL: Gaussian smoothing
H = imgaussfilt(H, 2); % apply before adding Inf values
% OPTIONAL: H-minima transform
H = imhmin(H, 10); % parameter is threshold for suppressing shallow minima
H = -H; % Complement the image
H(~mask) = Inf; % force "ground" pixels to Inf
L = watershed(D);
L(~mask) = 0; % clip "ground" from result
imshow(label2rgb(L,'lines',[.5 .5 .5])); % show result
My question now: Is there an algorithm that fills a landscape and gives me the resulting waterpools (for various amounts of water poured) to do what I've tried to achieve with above methods? Or any other suggestion is welcome. I'm implementing MATLAB (or if need be Python), but I can work with any code or pseude-code.
To distinguish this from this question, my maxima are not separated by zero-values. What I want is similar, yet none of the suggestions there are helpful (hill-climbing/simulated annealing will give you only one point...).
This question is also interesting, but it solves the problem with constraints (assume exactly 5 peaks of a certain size) that make the suggested approaches not useful for my case.
In such peak finding problems, I mostly use morphological operations. Since the Hough transform results are mostly noisy, I prefer blurring it first, then apply tophat and extended maxima transform. Then for each local maximum, find the region around it with adaptive thresholding. Here is a sample code:
im=imread('udIuy.png');
% blur
im=imgaussfilt(im,1);
% tophat transform
im2=imtophat(im,strel('disk',5));
% extended maximums
im3=imextendedmax(im2,10);
% Extract each blob
s=regionprops(im3,'Centroid','PixelIdxList');
figure,imagesc(im),axis image
for i=1:numel(s)
x=ceil(s(i).Centroid);
tmp=im*0;
tmp(s(i).PixelIdxList)=1;
tmp2=tmp.*im2;
% The maximum amplitude and location
[refV,b]=max(tmp2(:));
[x2,y2]=ind2sub(size(im),b);
% select the region around local max amplitude
tmp=bwselect(im2>refV*0.6,y2,x2,4);
[xi,yi]=find(tmp);
hold on, plot(yi,xi,'r.')
hold on, text(y2+10,x2,num2str(i),'Color','white','FontSize',16)
end

Zero out pixels that aren't at a particular intensity of a single colour

I'm having a little bit of difficulty wrapping my head around the correct terminology to use in phrasing my question, so I'll just take a stab at it and perhaps I can get some help in clarifying it along the way toward a solution.
I want to detect some coloured lights in an image, so I need a way to:
a) determine the colour of pixels
b) determine how "intense" or "bright" they are
c) use the two values above as a threshold or criteria for whether or not to discard a given pixel
I figured brightness alone will probably not be a good way to do this, since there will be non-zero ambient light.
Thanks!
EDIT: So using MATLAB's colour thresholder, I was able to isolate the coloured lights by restricting the hue range in HSV space. Just trying to figure out a way to do this via the command line.
Well there are two separate steps. 1 is finding out what you want to isolate, and 2 is isolation
1)Seems like you got this figured out. But for the future you can use the "imtool" command. It is nearly the same as imshow, but it allows you to inspect pixel values(RGB, you would convert these to HSV using rgb2hsv), crop images, zoom, measure distances, etc. It can be really helpful.
imtool(my_im)
will open up the window, pretty simple.
2)Now that you have your values you want to isolate them. The term you are looking for is MASKING A misk is typically a binary matrix/vector with 1's (true) corresponding to areas of interest and 0's (false) elsewhere. Matlab calls these "logical" arrays. So lets just say you found your areas of interest were as follows
hue=0.2 to 0.3, saturation=don't care, brightness= greater than .5
you would create your mask by doing binary comparisons on the pixels. I will split this into three steps just so you can make sense of everything.
%% MASKING STEPS
hue_idx = 1; sat_idx =2 ; bright_idx = 3;
hue_mask = ((my_hsv_im(:,:,hue_idx ) > 0.2) & (my_hsv_im(:,:,hue_idx ) < 0.3));
%note we have no saturation mask, because it would be filled with ones
%since we dont care about the saturation values
brightness_mask = (my_hsv_im(:,:,bright_idx ) > 0.5);
total_mask = hue_mask & brightness_mask;
%% ALL THE REST
%now we mask your image, recall that 1's are ares of interest and 0's are
%nothing so just multiply your image by your mask
% the mask is a logical array size MxNx1, we need to convert it to the same
%type as our image in order to multiply them
mask_3d(:,:,hue_idx) = total_mask;
mask_3d(:,:,sat_idx) = total_mask;
mask_3d(:,:,bright_idx) = total_mask;
mask_3d = uint8(mask_3d); %this step is pretty important, if your image
%is a double use double(mask_3d) instead
masked_rgb_im = my_im .* mask_3d;
%does some plotting just for fun
figure(10);
subplot(2,3,1);imshow(my_im);title('original image');
subplot(2,3,2);imshow(hue_mask);title('hue mask');
subplot(2,3,3);imshow(brightness_mask);title('bright mask');
subplot(2,3,4);imshow(total_mask);title('total mask');
subplot(2,3,5:6);imshow(masked_rgb_im );title('masked image');

Detect black dots from color background

My short question
How to detect the black dots in the following images? (I paste only one test image to make the question look compact. More images can be found →here←).
My long question
As shown above, the background color is roughly blue, and the dots color is "black". If pick one black pixel and measure its color in RGB, the value can be (0, 44, 65) or (14, 69, 89).... Therefore, we cannot set a range to tell the pixel is part of the black dot or the background.
I test 10 images of different colors, but I hope I can find a method to detect the black dots from more complicated background which may be made up of three or more colors, as long as human eyes can identify the black dots easily. Some extremely small or blur dots can be omitted.
Previous work
Last month, I have asked a similar question at stackoverflow, but have not got a perfect solution, some excellent answers though. Find more details about my work if you are interested.
Here are the methods I have tried:
Converting to grayscale or the brightness of image. The difficulty is that I can not find an adaptive threshold to do binarization. Obviously, turning a color image to grayscale or using the brightness (HSV) will lose much useful information. Otsu algorithm which calculates adaptive threshold can not work either.
Calculating RGB histogram. In my last question, natan's method is to estimate the black color by histogram. It is time-saving, but the adaptive threshold is also a problem.
Clustering. I have tried k-means clustering and found it quite effective for the background that only has one color. The shortage (see my own answer) is I need to set the number of clustering center in advance but I don't know how the background will be. What's more, it is too slow! My application is for real time capturing on iPhone and now it can process 7~8 frames per second using k-means (20 FPS is good I think).
Summary
I think not only similar colors but also adjacent pixels should be "clustered" or "merged" in order to extract the black dots. Please guide me a proper way to solve my problem. Any advice or algorithm will be appreciated. There is no free lunch but I hope a better trade-off between cost and accuracy.
I was able to get some pretty nice first pass results by converting to HSV color space with rgb2hsv, then using the Image Processing Toolbox functions imopen and imregionalmin on the value channel:
rgb = imread('6abIc.jpg');
hsv = rgb2hsv(rgb);
openimg = imopen(hsv(:, :, 3), strel('disk', 11));
mask = imregionalmin(openimg);
imshow(rgb);
hold on;
[r, c] = find(mask);
plot(c, r, 'r.');
And the resulting images (for the image in the question and one chosen from your link):
You can see a few false positives and missed dots, as well as some dots that are labeled with multiple points, but a few refinements (such as modifying the structure element used in the opening step) could clean these up some.
I was curios to test with my old 2d peak finder code on the images without any threshold or any color considerations, really crude don't you think?
im0=imread('Snap10.jpg');
im=(abs(255-im0));
d=rgb2gray(im);
filter=fspecial('gaussian',16,3.5);
p=FastPeakFind(d,0,filter);
imagesc(im0); hold on
plot(p(1:2:end),p(2:2:end),'r.')
The code I'm using is a simple 2D local maxima finder, there are some false positives, but all in all this captures most of the points with no duplication. The filter I was using was a 2d gaussian of width and std similar to a typical blob (the best would have been to get a matched filter for your problem).
A more sophisticated version that does treat the colors (rgb2hsv?) could improve this further...
Here is an extraodinarily simplified version, that can be extended to be full RGB, and it also does not use the image procesing library. Basically you can do 2-D convolution with a filter image (which is an example of the dot you are looking for), and from the points where the convolution returns the highest values, are the best matches for the dots. You can then of course threshold that. Here is a simple binary image example of just that.
%creating a dummy image with a bunch of small white crosses
im = zeros(100,100);
numPoints = 10;
% randomly chose the location to put those crosses
points = randperm(numel(im));
% keep only certain number of points
points = points(1:numPoints);
% get the row and columns (x,y)
[xVals,yVals] = ind2sub(size(im),points);
for ii = 1:numel(points)
x = xVals(ii);
y = yVals(ii);
try
% create the crosses, try statement is here to prevent index out of bounds
% not necessarily the best practice but whatever, it is only for demonstration
im(x,y) = 1;
im(x+1,y) = 1;
im(x-1,y) = 1;
im(x,y+1) = 1;
im(x,y-1) = 1;
catch err
end
end
% display the randomly generated image
imshow(im)
% create a simple cross filter
filter = [0,1,0;1,1,1;0,1,0];
figure; imshow(filter)
% perform convolution of the random image with the cross template
result = conv2(im,filter,'same');
% get the number of white pixels in filter
filSum = sum(filter(:));
% look for all points in the convolution results that matched identically to the filter
matches = find(result == filSum);
%validate all points found
sort(matches(:)) == sort(points(:))
% get x and y coordinate matches
[xMatch,yMatch] = ind2sub(size(im),matches);
I would highly suggest looking at the conv2 documentation on MATLAB's website.

Resources