Unknown issue with image refinement process - image

As introduced in this work (code), the following code sould refine an image (i.e. a gray scale saliency map).
Refinement Function:
function sal = Refinement(y, dim)
th_2 = graythresh(y);
if dim == 1
sal = y;
sal(y<th_2) = 10*(y(y < th_2))/th_2 - 10;
sal(y>=th_2) = 10*(y(y >= th_2) - th_2)/(1-th_2);
sal = 1 ./ (1 + exp(-sal)) + y;
sal = normalization(sal, 0);
elseif dim == 2
[r, c] = size(y);
y_col = reshape(y,[1 r*c]);
sal_col = y_col;
sal_col(y_col<th_2) = 10*(y_col(y_col < th_2))/th_2 - 10;
sal_col(y_col>=th_2) = 10*(y_col(y_col >= th_2) - th_2)/(1-th_2);
sal_col = 1 ./ (1 + exp(-sal_col)) + y_col;
sal = reshape(sal_col, [r c]);
end
end
normalization function:
function matrix = normalization(mat, flag)
% INPUT :
% flag: 1 denotes that the mat is a 3-d matrix;
% 0 denotes that the mat is a matrix;
%
if flag ~= 0
dim = size(mat,3);
matrix = mat;
for i = 1:dim
matrix(:,:,i) = ( mat(:,:,i) - min(min(mat(:,:,i)))) / ( max(max(mat(:,:,i))) - min(min( mat(:,:,i))) + eps);
end
else
matrix = ( mat - min(min(mat)))/( max(max(mat)) - min(min(mat)) + eps);
end
However, after applying the function values of the image matrix will be changed, the result remains the same as the image before the refinement.
Is there a conceptual error with that or the implementation failed?
P.S. The input image (saliency map) for refinery is something like below. in the refined saliency map, the foreground (satrfish in this image) should stand out(becomes homogeneously white as possible) and the background noise should be removed (becomes homogenously black as possible):

Related

How to properly process images with mixed noise types

The picture with noise is like this.
Noised picture: Image3.bmp
I was doing image processing in MatLab with some built-in and self-implemented filters.
I have already tried a combination of bilateral, median and gaussian. bilateral and gaussian code are at the end of this post.
img3 = double(imread('Image3.bmp')); % this is the noised image
lena = double(imread('lena_gray.jpg')); % this is the original one
img3_com = bilateral(img3, 3, 2, 80);
img3_com = medfilt2(img3_com, [3 3], 'symmetric');
img3_com = gaussian(img3_com, 3, 0.5);
img3_com = bilateral(double(img3_com), 6, 100, 13);
SNR3_com = snr(img3_com,img3_com - lena); % 17.1107
However, the result is not promising with SNR of only 17.11.
Filtered image: img3_com
The original picture is like this.
Clean original image: lena_gray.jpg
Could you please give me any possible ideas about how to process it? Like what noise generators generated the noised image and what filtering methods or image processing method I can use to deal with it. Appreciate!!!
My bilateral function bilateral.m
function img_new = bilateral(img_gray, window, sigmaS, sigmaI)
imgSize = size(img_gray);
img_new = zeros(imgSize);
for i = 1:imgSize(1)
for j = 1:imgSize(2)
sum = 0;
simiSum = 0;
for a = -window:window
for b = -window:window
x = i + a;
y = j + b;
p = img_gray(i,j);
q = 0;
if x < 1 || y < 1 || x > imgSize(1) || y > imgSize(2)
% q=0;
continue;
else
q = img_gray(x,y);
end
gaussianFilter = exp( - double((a)^2 + (b)^2)/ (2 * sigmaS^2 ) - (double(p-q)^2)/ (2 * sigmaI^2 ));
% gaussianFilter = gaussian((a^2 + b^2)^(1/2), sigma) * gaussian(abs(p-q), sigma);
sum = sum + gaussianFilter * q;
simiSum = simiSum + gaussianFilter;
end
end
img_new(i,j) = sum/simiSum;
end
end
% disp SNR
lena = double(imread('lena_gray.jpg'));
SNR1_4_ = snr(img_new,img_new - lena);
disp(SNR1_4_);
My gaussian implementation gaussian.m
function img_gau = gaussian(img, hsize, sigma)
h = fspecial('gaussian', hsize, sigma);
img_gau = conv2(img,h,'same');
% disp SNR
lena = double(imread('lena_gray.jpg'));
SNR1_4_ = snr(img_gau,img_gau - lena);
disp(SNR1_4_);

How to calculate the mean of 3D matrices in an image without NaN?

I need to calculate the mean of a 3D matrices (last step in the code). However, there are many NaNs in the (diff_dataframe./dataframe_vor) calculation. So when I use this code, some results will be NaN. How could I calculate the mean of this matrix by ignoring the NaNs? I attached the code as below.
S.amplitude = 1:20;%:20;
S.blocksize = [1 2 3 4 5 6 8 10 12 15 20];
S.frameWidth = 1920;
S.frameHeight = 1080;
S.quality=0:10:100;
image = 127*ones(S.frameHeight,S.frameWidth,3);
S.yuv2rgb = [1 0 1.28033; 1 -0.21482 -0.38059; 1 2.12798 0];
i_bs = 0;
for BS = S.blocksize
i_bs = i_bs + 1;
hblocks = S.frameWidth / BS;
vblocks = S.frameHeight / BS;
i_a = 0;
dataU = randi([0 1],vblocks,hblocks);
dataV = randi([0 1],vblocks,hblocks);
dataframe_yuv = zeros(S.frameHeight, S.frameWidth, 3);
for x = 1 : hblocks
for y = 1 : vblocks
dataframe_yuv((y-1)*BS+1:y*BS, ...
(x-1)*BS+1:x*BS, 2) = dataU(y,x) * 2 - 1;
dataframe_yuv((y-1)*BS+1:y*BS, ...
(x-1)*BS+1:x*BS, 3) = dataV(y,x) * 2 - 1;
end
end
dataframe_rgb(:,:,1) = S.yuv2rgb(1,1) * dataframe_yuv(:,:,1) + ...
S.yuv2rgb(1,2) * dataframe_yuv(:,:,2) + ...
S.yuv2rgb(1,3) * dataframe_yuv(:,:,3);
dataframe_rgb(:,:,2) = S.yuv2rgb(2,1) * dataframe_yuv(:,:,1) + ...
S.yuv2rgb(2,2) * dataframe_yuv(:,:,2) + ...
S.yuv2rgb(2,3) * dataframe_yuv(:,:,3);
dataframe_rgb(:,:,3) = S.yuv2rgb(3,1) * dataframe_yuv(:,:,1) + ...
S.yuv2rgb(3,2) * dataframe_yuv(:,:,2) + ...
S.yuv2rgb(3,3) * dataframe_yuv(:,:,3);
for A = S.amplitude
i_a = i_a + 1;
i_q = 0;
image1p = round(image + dataframe_rgb * A);
image1n = round(image - dataframe_rgb * A);
dataframe_vor = ((image1p-image1n)/2)/255;
for Q = S.quality
i_q = i_q + 1;
namestrp = ['greyjpegs/Img_BS' num2str(BS) '_A' num2str(A) '_Q' num2str(Q) '_1p.jpg'];
namestrn = ['greyjpegs/Img_BS' num2str(BS) '_A' num2str(A) '_Q' num2str(Q) '_1n.jpg'];
imwrite(image1p/255,namestrp,'jpg', 'Quality', Q);
imwrite(image1n/255,namestrn,'jpg', 'Quality', Q);
error_mean(i_bs, i_a, i_q) = mean2((abs(diff_dataframe./dataframe_vor)));
end
end
end
mean2 is a shortcut function that's part of the image processing toolbox that finds the entire average of a 2D region which doesn't include handling NaN. In that case, simply remove all values that are NaN and find the resulting average. Note that the removal of NaN unrolls the 2D region into a 1D vector, so we can simply use mean in this case. As an additional check, let's make sure there are no divide by 0 errors, so also check for Inf as well.
Therefore, replace this line:
error_mean(i_bs, i_a, i_q) = mean2((abs(diff_dataframe./dataframe_vor)));
... with:
tmp = abs(diff_dataframe ./ dataframe_vor);
mask = ~isnan(tmp) | ~isinf(tmp);
tmp = tmp(mask);
if isempty(tmp)
error_mean(i_bs, i_a, i_q) = 0;
else
error_mean(i_bs, i_a, i_q) = mean(tmp);
We first assign the desired operation to a temporary variable, use isnan and isinf to remove out the offending values, then find the average of the rest. One intricacy is that if your entire region is NaN or Inf, then the removal of all these entries in the region results in the empty vector, and finding the mean of this undefined. A separate check is there to be sure that if it's empty, simply assign the value of 0 instead.

How can i convert image velocity to object velocity in matlab?

I am using for my project the "LucasKanade" code in matlab. It gives me as output 2 matrices (u and v). These are(i believe so) the velocities of the image in the x and y axes respectively. Now how can i convert these velocities to object velocities(eg in meters/second)?
Thanks in advance
"LucasKanade" code:
function [u, v] = LucasKanade(im1, im2, windowSize);
%LucasKanade lucas kanade algorithm, without pyramids (only 1 level);
%REVISION: NaN vals are replaced by zeros
[fx, fy, ft] = ComputeDerivatives(im1, im2);
u = zeros(size(im1));
v = zeros(size(im2));
halfWindow = floor(windowSize/2);
for i = halfWindow+1:size(fx,1)-halfWindow
for j = halfWindow+1:size(fx,2)-halfWindow
curFx = fx(i-halfWindow:i+halfWindow, j-halfWindow:j+halfWindow);
curFy = fy(i-halfWindow:i+halfWindow, j-halfWindow:j+halfWindow);
curFt = ft(i-halfWindow:i+halfWindow, j-halfWindow:j+halfWindow);
curFx = curFx';
curFy = curFy';
curFt = curFt';
curFx = curFx(:);
curFy = curFy(:);
curFt = -curFt(:);
A = [curFx curFy];
U = pinv(A'*A)*A'*curFt;
u(i,j)=U(1);
v(i,j)=U(2);
end;
end;
u(isnan(u))=0;
v(isnan(v))=0;
%u=u(2:size(u,1), 2:size(u,2));
%v=v(2:size(v,1), 2:size(v,2));
%%
function [fx, fy, ft] = ComputeDerivatives(im1, im2);
%ComputeDerivatives Compute horizontal, vertical and time derivative
% between two gray-level images.
if (size(im1,1) ~= size(im2,1)) | (size(im1,2) ~= size(im2,2))
error('input images are not the same size');
end;
if (size(im1,3)~=1) | (size(im2,3)~=1)
error('method only works for gray-level images');
end;
fx = conv2(im1,0.25* [-1 1; -1 1]) + conv2(im2, 0.25*[-1 1; -1 1]);
fy = conv2(im1, 0.25*[-1 -1; 1 1]) + conv2(im2, 0.25*[-1 -1; 1 1]);
ft = conv2(im1, 0.25*ones(2)) + conv2(im2, -0.25*ones(2));
% make same size as input
fx=fx(1:size(fx,1)-1, 1:size(fx,2)-1);
fy=fy(1:size(fy,1)-1, 1:size(fy,2)-1);
ft=ft(1:size(ft,1)-1, 1:size(ft,2)-1);

Convert RGB to HSV

i want to convert RGB values to HSV values . But if I devide 9 by 28, octave calculate 0. Can anyone explain me the reason??
function [hsv] = RGBtoHSV()
im = imread('picture.png');
R = im(:,:,1);
G = im(:,:,2);
B = im(:,:,3);
len = length(R); % R, G, B should have the same length
for i = 1:len
MAX = max([R(i),G(i),B(i)]);
MIN = min([R(i),G(i),B(i)]);
S = 0;
if MAX == MIN
H = 0;
elseif MAX == R(i)
disp(G(i) - B(i)); % 9
disp(MAX - MIN); % 28
H = 0.6 * ( 0 + ( (G(i) - B(i)) / MAX - MIN) ); % 0
disp(H) % why i get 0 if try to calculate ( 0 + ( (G(i) - B(i)) / MAX - MIN)?
....
end
return;
end
endfunction
RGBtoHSV()
Chris :D
You must cast the image into Double by doing:
im = double(imread('picture.png'));
This will solve your issues which happens since the image is type UINT8.
You can also use Octave's builtin rgb2hsv function instead of writing your own.
im_rgb = imread ("picture.png");
im_hsv = rgb2hsv (im_rgb);
If this is an exercise, then I'd suggest you look at its source, enter type rgb2hsv at the Octave prompt, and see how its implemented.

How to accelerate matlab code?

I'm using matlab to implement a multilayer neural network. In the code I represent
the value of each node AS netValue{k}
the weight between layer k and k + 1 AS weight{k}
etc.
Since these data is three-dimensional, I have to use cell to hold a 2-D matrix to enable matrix multiply.
So it becomes really really slow to train the model, which I expect to have resulted from the usage of cell.
Can anyone tell me how to accelerate this code? Thanks
clc;
close all;
clear all;
input = [-2 : 0.4 : 2;-2:0.4:2];
ican = 4;
depth = 4; % total layer - 1, by convension
[featureNum , sampleNum] = size(input);
levelNum(1) = featureNum;
levelNum(2) = 5;
levelNum(3) = 5;
levelNum(4) = 5;
levelNum(5) = 2;
weight = cell(0);
for k = 1 : depth
weight{k} = rand(levelNum(k+1), levelNum(k)) - 2 * rand(levelNum(k+1) , levelNum(k));
threshold{k} = rand(levelNum(k+1) , 1) - 2 * rand(levelNum(k+1) , 1);
end
runCount = 0;
sumMSE = 1; % init MSE
minError = 1e-5;
afa = 0.1; % step of "gradient ascendence"
% training loop
while(runCount < 100000 & sumMSE > minError)
sumMSE = 0; % sum of MSE
for i = 1 : sampleNum % sample loop
netValue{1} = input(:,i);
for k = 2 : depth
netValue{k} = weight{k-1} * netValue{k-1} + threshold{k-1}; %calculate each layer
netValue{k} = 1 ./ (1 + exp(-netValue{k})); %apply logistic function
end
netValue{depth+1} = weight{depth} * netValue{depth} + threshold{depth}; %output layer
e = 1 + sin((pi / 4) * ican * netValue{1}) - netValue{depth + 1}; %calc error
assistS{depth} = diag(ones(size(netValue{depth+1})));
s{depth} = -2 * assistS{depth} * e;
for k = depth - 1 : -1 : 1
assistS{k} = diag((1-netValue{k+1}).*netValue{k+1});
s{k} = assistS{k} * weight{k+1}' * s{k+1};
end
for k = 1 : depth
weight{k} = weight{k} - afa * s{k} * netValue{k}';
threshold{k} = threshold{k} - afa * s{k};
end
sumMSE = sumMSE + e' * e;
end
sumMSE = sqrt(sumMSE) / sampleNum;
runCount = runCount + 1;
end
x = [-2 : 0.1 : 2;-2:0.1:2];
y = zeros(size(x));
z = 1 + sin((pi / 4) * ican .* x);
% test
for i = 1 : length(x)
netValue{1} = x(:,i);
for k = 2 : depth
netValue{k} = weight{k-1} * netValue{k-1} + threshold{k-1};
netValue{k} = 1 ./ ( 1 + exp(-netValue{k}));
end
y(:, i) = weight{depth} * netValue{depth} + threshold{depth};
end
plot(x(1,:) , y(1,:) , 'r');
hold on;
plot(x(1,:) , z(1,:) , 'g');
hold off;
Have you used the profiler to find out what functions are actually slowing down your code? It shows what lines take the most time to execute.

Resources