Related
I have some difficulties to reconstruct a 3D scene from a 2D image under Matlab.
I would like to create a top view of scene removing the perspective, in other words, realize an inverse perspective mapping.
Let's assume we know the camera position, orientation and its parameters. Moreover we consider all the captured points lie on the same plane XY.
Then, it is easy to prove that a pixel at a (u,v) location in the image will move to the coordinate (X,Y,0) in the 3D space with:
X=-((u*P(3,4)-P(1,4))*(v*P(3,1)-P(2,1)) + (v*P(3,4)-P(2,4))*(P(1,1)-u*P(3,1)))/((u*P(3,2)-P(1,2))*(v*P(3,1)-P(2,1)) + (v*P(3,2)-P(2,2))*(P(1,1)-u*P(3,1)));
Y=(X*(u*P(3,2)-P(1,2)) + (u*P(3,4)-P(1,4)))/(P(1,1)-u*P(3,1));
P is the projection matrix such that: P=[KR KT] with K,R and T respectively the intrinsic, rotation and translation matrices.
Once all the 3D locations of each pixel are computed, I would like to display the XY plane with the color information of the original pixel as if it was a 2D image.
However, a pixel (u,v) can mapped in 3D space to a non integer location meaning that I get a non-regular scatter plot were each (X,Y) point contain a color information.
I tried to divide the XY plane into small windows and then compute the average color of all points into each squares but it is very slow.
Please find my code below.
Some help would be appreciated.
Thank you in advance,
Pm
% This program aims to convert a 2D image into a 3D scenario. Let's assume that
% all the points in the image lie on the same plan in the 3D space.
% Program:
% 1-Generate the 3D scenario with 4 squares from diferrent colors
% 2-Take a picture of these squares by projecting the scene into an image 2D space
% 3-Reconstruct the 3D scene from the 2D picture
% Author: Pierre-Marie Damon
clear all; close all; clc;
%% 4 SQUARES DEFINITION
c=10;
sq1_3D = [ 0 0 0;
0 c 0;
c c 0;
c 0 0];
sq2_3D = [ 0 0 0;
c 0 0;
c -c 0;
0 -c 0];
sq3_3D = [ 0 0 0;
0 -c 0;
-c -c 0;
-c 0 0];
sq4_3D = [ 0 0 0;
-c 0 0;
-c c 0;
0 c 0];
SQ_3D = [sq1_3D;
sq2_3D;
sq3_3D;
sq4_3D];
%% CAMERA DEFINITION
% Image resolution:
image_size = [640,480];
% Intrinsic matrix:
fx=150; fy=150; % fx, fy: focal lengths in pixel
x0=image_size(1)/2; y0=image_size(2)/2; % x0, y0: optical center projection coordinates in the 2D cxamera space
K = [fx 0 x0 ;
0 fy y0 ;
0 0 1 ];
% 3D camera orientation:
Rot_cam = [ 0 0 1;
-1 0 0;
0 -1 0]';
% 3D camera rotations:
yaw = -20*pi/180;
roll = -0*pi/180;
pitch = -20*pi/180;
% 3D camera position:
O_Rcam = [-20;0;10];
h_cam = O_Rcam(3); % camera height
% Projection & transformation matrices
R = rotationVectorToMatrix([pitch,yaw,roll])*Rot_cam; % Rotation matrix
T = -R*O_Rcam; % Translation matrix
P = [K*R K*T; zeros(1,3) 1]; %Projection Matrix
%% PROJECTION FROM 3D TO THE 2D IMAGE
SQ_2D = (P*[SQ_3D ones(size(SQ_3D,1),1)]')'; % homogeneous coordinates
SQ_2D = SQ_2D(:,1:2)./repmat(SQ_2D(:,3),1,2); % Normalization
% Square splits:
sq1_2D = SQ_2D(1:4,:);
sq2_2D = SQ_2D(5:8,:);
sq3_2D = SQ_2D(9:12,:);
sq4_2D = SQ_2D(13:16,:);
%% PLOT THE 3D SCENARIO
figure('units','normalized','outerposition',[0 0 1 1]);
f1=subplot(1,2,1);hold on; grid on; view(50, 30); axis equal;
xlabel('X');ylabel('Y');zlabel('Z'); title('3D Scene');
% Plot the camera
cam = plotCamera('Location',O_Rcam,'Orientation',R,'Size',1,'AxesVisible',0);
% Plot the squares
patch([sq1_3D(:,1)],[sq1_3D(:,2)],'r','EdgeColor','k');
patch([sq2_3D(:,1)],[sq2_3D(:,2)],'g','EdgeColor','k');
patch([sq3_3D(:,1)],[sq3_3D(:,2)],'b','EdgeColor','k');
patch([sq4_3D(:,1)],[sq4_3D(:,2)],'m','EdgeColor','k');
%% PLOT THE 2D IMAGE
f2=subplot(1,2,2); hold on; grid on; axis equal; set(gca,'YDir','Reverse'); title('2D Image'); axis off;
xlim([0 image_size(1)]); ylim([0 image_size(2)]);
% plot the projected squares
patch ([sq1_2D(:,1)],[sq1_2D(:,2)],'r','EdgeColor','none');
patch ([sq2_2D(:,1)],[sq2_2D(:,2)],'g','EdgeColor','none');
patch ([sq3_2D(:,1)],[sq3_2D(:,2)],'b','EdgeColor','none');
patch ([sq4_2D(:,1)],[sq4_2D(:,2)],'m','EdgeColor','none');
% Plot the image borders
plot([0 image_size(1)],[0 0],'k','linewidth',3);
plot([image_size(1) image_size(1)],[0 image_size(2)],'k','linewidth',3);
plot([0 image_size(1)],[image_size(2) image_size(2)],'k','linewidth',3);
plot([0 0],[0 image_size(2)],'k','linewidth',3);
%% GENERATE A JPG IMAGE
figure; hold on; grid on; set(gca,'YDir','Reverse'); axis off;
hFig = gcf; hAx = gca; % get the figure and axes handles
set(hFig,'units','normalized','outerposition',[0 0 1 1]); % set the figure to full screen
set(hAx,'Unit','normalized','Position',[0 0 1 1]); % set the axes to full screen
set(hFig,'menubar','none'); % hide the toolbar
set(hFig,'NumberTitle','off'); % to hide the title
xlim([0 image_size(1)]); ylim([0 image_size(2)]);
% Plot the squares
patch ([sq1_2D(:,1)],[sq1_2D(:,2)],'r','EdgeColor','none');
patch ([sq2_2D(:,1)],[sq2_2D(:,2)],'g','EdgeColor','none');
patch ([sq3_2D(:,1)],[sq3_2D(:,2)],'b','EdgeColor','none');
patch ([sq4_2D(:,1)],[sq4_2D(:,2)],'m','EdgeColor','none');
% Create the image
set(gcf,'PaperUnits','inches','PaperPosition',[0 0 image_size(1)/100 image_size(2)/100])
print -djpeg Image.jpg -r100
save('ImageParam.mat', 'K','R','T','P','O_Rcam' )
%% 3D RECONSTRUCTION FROM 2D IMAGE
clear all;
close all;
clc;
load('ImageParam');
I=imread('Image.jpg');
I1 = rgb2gray(I);
figure;imshow(I1);impixelinfo;
I2 = zeros(size(I1));
k=1; i=1; j=1;
tic
for y=size(I2,1):-1:1
for x=1:size(I2,2)
% The formula below comes from the projection equations of
% the camera and the additional constraint that all the points lie on the XY
% plane
X(k)=-((x*P(3,4)-P(1,4))*(y*P(3,1)-P(2,1)) + (y*P(3,4)-P(2,4))*(P(1,1)-x*P(3,1)))/((x*P(3,2)-P(1,2))*(y*P(3,1)-P(2,1)) + (y*P(3,2)-P(2,2))*(P(1,1)-x*P(3,1)));
Y(k)=(X(k)*(x*P(3,2)-P(1,2)) + (x*P(3,4)-P(1,4)))/(P(1,1)-x*P(3,1));
Z(k)=0;
C(k)=I1(y,x); % Color (gray intensity) information
k=k+1;
end
end
toc
figure;hold on;axis equal;
plot(X,Y,'.');
grid on;
I'm trying to convert an image with many circles with the same center, from Cartesian to Polar (so that the new image will be the circles but lines instead of the circles, see the image below), and that's working out just fine using the following code:
[r, c] = size(img);
r=floor(r/2);
c=floor(c/2);
[X, Y] = meshgrid(-c:c-1,-r:r-1);
[theta, rho] = cart2pol(X, Y);
subplot(221), imshow(img), axis on;
hold on;
subplot(221), plot(xCenter,yCenter, 'r+');
subplot(222), warp(theta, rho, zeros(size(theta)), img);
view(2), axis square;
The problem is, I don't understand why does it even work? (obviously it's not my code), I mean, when I use the function cart2pol I don't even use the image, it's just some vectors x and y generated from the meshgrid function..
and another problem is, I want somehow to have a new image (not just to be able to draw it with the wrap function) which is the original image but by the theta and rho coordinates (meaning the same pixels but rearranged)... I'm not even sure how to ask this, in the end I want to have an image which is a matrix so that I can sum each row and turn the matrix into a column vector...
You can think of your image as being a 2D matrix, where each pixel has an X and Y coordinate
[(1,1) (1,2) (1,3) .... (1,c)]
[(2,1) (2,2) (2,3) .... (2,c)]
[(3,1) (3,2) (3,3) .... (3,c)]
[.... .... .... .... .... ]
[(r,1) (r,2) (r,3) .... (r,c)]
In the code that you posted, it maps each of these (X,Y) coordinates to it's equivalent polar coordinate (R, theta) using the center of the image floor(c/2) and floor(r/2) as the reference point.
% Map pixel value at (1,1) to it's polar equivalent
[r,theta] = cart2pol(1 - floor(r/2),1 - floor(c/2));
So whatever pixel value was used for (1,1) should now appear in your new polar coordinate space at (r,theta). It is important to note that to do this conversion, no information about the actual pixel values in the image matters, rather we just want to perform this transformation for each pixel within the image.
So first we figure out where the center of the image is:
[r, c] = size(img);
r = floor(r / 2);
c = floor(c / 2);
Then we figure out the (X,Y) coordinates for every point in the image (after the center has already been subtracted out
[X, Y] = meshgrid(-c:c-1,-r:r-1);
Now convert all of these cartesian points to polar coordinates
[theta, rho] = cart2pol(X, Y);
All that warp now does, is say "display the value of img at (X,Y) at it's corresponding location in (theta, rho)"
warp(theta, rho, zeros(size(theta)), img);
Now it seems that you want a new 2D image where the dimensions are [nTheta, nRho]. To do this, you could use griddata to interpolate your scattered (theta, rho) image (which is displayed by warp above) to a regular grid.
% These is the spacing of your radius axis (columns)
rhoRange = linspace(0, max(rho(:)), 100);
% This is the spacing of your theta axis (rows)
thetaRange = linspace(-pi, pi, 100);
% Generate a grid of all (theta, rho) coordinates in your destination image
[T,R] = meshgrid(thetaRange, rhoRange);
% Now map the values in img to your new image domain
theta_rho_image = griddata(theta, rho, double(img), T, R);
Take a look at all the interpolation methods for griddata to figure out which is most appropriate for your scenario.
There were a couple other issues (like the rounding of the center) which caused the result to be slightly incorrect. A fully working example is provided below
% Create an image of circles
radii = linspace(0, 40, 10);
rows = 100;
cols = 100;
img = zeros(rows, cols);
for k = 1:numel(radii)
t = linspace(0, 2*pi, 1000);
xx = round((cos(t) * radii(k)) + (cols / 2));
yy = round((sin(t) * radii(k)) + (rows / 2));
toremove = xx > cols | xx < 1 | yy > rows | yy < 1;
inds = sub2ind(size(img), xx(~toremove), yy(~toremove));
img(inds) = 1;
end
[r,c] = size(img);
center_row = r / 2;
center_col = c / 2;
[X,Y] = meshgrid((1:c) - center_col, (1:r) - center_row);
[theta, rho] = cart2pol(X, Y);
rhoRange = linspace(0, max(rho(:)), 1000);
thetaRange = linspace(-pi, pi, 1000);
[T, R] = meshgrid(thetaRange, rhoRange);
theta_rho_image = griddata(theta, rho, double(img), T, R);
figure
subplot(1,2,1);
imshow(img);
title('Original Image')
subplot(1,2,2);
imshow(theta_rho_image);
title('Polar Image')
And the result
I'm trying to plot small images on a larger plot... Actually its isomap algorithm, I got many points, now each point correspond to some image, I know which image is it... The porblem is how to load that image and plot on the graph?
One more thing I have to plot both image and the points, so, basically the images will overlap the points.
Certainly, the type of image given here
Something like this should get you started. You can use the low-level version of the image function to draw onto a set of axes.
% Define some random data
N = 5;
x = rand(N, 1);
y = rand(N, 1);
% Load an image
rgb = imread('ngc6543a.jpg');
% Draw a scatter plot
scatter(x, y);
axis([0 1 0 1]);
% Offsets of image from associated point
dx = 0.02;
dy = 0.02;
width = 0.1;
height = size(rgb, 1) / size(rgb, 2) * width;
for i = 1:N
image('CData', rgb,...
'XData', [x(i)-dx x(i)-(dx+width)],...
'YData', [y(i)-dy y(i)-(dy+height)]);
end
BOUNTY STATUS UPDATE:
I discovered how to map a linear lens, from destination coordinates to source coordinates.
How do you calculate the radial distance from the centre to go from fisheye to rectilinear?
1). I actually struggle to reverse it, and to map source coordinates to destination coordinates. What is the inverse, in code in the style of the converting functions I posted?
2). I also see that my undistortion is imperfect on some lenses - presumably those that are not strictly linear. What is the equivalent to-and-from source-and-destination coordinates for those lenses? Again, more code than just mathematical formulae please...
Question as originally stated:
I have some points that describe positions in a picture taken with a fisheye lens.
I want to convert these points to rectilinear coordinates. I want to undistort the image.
I've found this description of how to generate a fisheye effect, but not how to reverse it.
There's also a blog post that describes how to use tools to do it; these pictures are from that:
(1) : SOURCE Original photo link
Input : Original image with fish-eye distortion to fix.
(2) : DESTINATION Original photo link
Output : Corrected image (technically also with perspective correction, but that's a separate step).
How do you calculate the radial distance from the centre to go from fisheye to rectilinear?
My function stub looks like this:
Point correct_fisheye(const Point& p,const Size& img) {
// to polar
const Point centre = {img.width/2,img.height/2};
const Point rel = {p.x-centre.x,p.y-centre.y};
const double theta = atan2(rel.y,rel.x);
double R = sqrt((rel.x*rel.x)+(rel.y*rel.y));
// fisheye undistortion in here please
//... change R ...
// back to rectangular
const Point ret = Point(centre.x+R*cos(theta),centre.y+R*sin(theta));
fprintf(stderr,"(%d,%d) in (%d,%d) = %f,%f = (%d,%d)\n",p.x,p.y,img.width,img.height,theta,R,ret.x,ret.y);
return ret;
}
Alternatively, I could somehow convert the image from fisheye to rectilinear before finding the points, but I'm completely befuddled by the OpenCV documentation. Is there a straightforward way to do it in OpenCV, and does it perform well enough to do it to a live video feed?
The description you mention states that the projection by a pin-hole camera (one that does not introduce lens distortion) is modeled by
R_u = f*tan(theta)
and the projection by common fisheye lens cameras (that is, distorted) is modeled by
R_d = 2*f*sin(theta/2)
You already know R_d and theta and if you knew the camera's focal length (represented by f) then correcting the image would amount to computing R_u in terms of R_d and theta. In other words,
R_u = f*tan(2*asin(R_d/(2*f)))
is the formula you're looking for. Estimating the focal length f can be solved by calibrating the camera or other means such as letting the user provide feedback on how well the image is corrected or using knowledge from the original scene.
In order to solve the same problem using OpenCV, you would have to obtain the camera's intrinsic parameters and lens distortion coefficients. See, for example, Chapter 11 of Learning OpenCV (don't forget to check the correction). Then you can use a program such as this one (written with the Python bindings for OpenCV) in order to reverse lens distortion:
#!/usr/bin/python
# ./undistort 0_0000.jpg 1367.451167 1367.451167 0 0 -0.246065 0.193617 -0.002004 -0.002056
import sys
import cv
def main(argv):
if len(argv) < 10:
print 'Usage: %s input-file fx fy cx cy k1 k2 p1 p2 output-file' % argv[0]
sys.exit(-1)
src = argv[1]
fx, fy, cx, cy, k1, k2, p1, p2, output = argv[2:]
intrinsics = cv.CreateMat(3, 3, cv.CV_64FC1)
cv.Zero(intrinsics)
intrinsics[0, 0] = float(fx)
intrinsics[1, 1] = float(fy)
intrinsics[2, 2] = 1.0
intrinsics[0, 2] = float(cx)
intrinsics[1, 2] = float(cy)
dist_coeffs = cv.CreateMat(1, 4, cv.CV_64FC1)
cv.Zero(dist_coeffs)
dist_coeffs[0, 0] = float(k1)
dist_coeffs[0, 1] = float(k2)
dist_coeffs[0, 2] = float(p1)
dist_coeffs[0, 3] = float(p2)
src = cv.LoadImage(src)
dst = cv.CreateImage(cv.GetSize(src), src.depth, src.nChannels)
mapx = cv.CreateImage(cv.GetSize(src), cv.IPL_DEPTH_32F, 1)
mapy = cv.CreateImage(cv.GetSize(src), cv.IPL_DEPTH_32F, 1)
cv.InitUndistortMap(intrinsics, dist_coeffs, mapx, mapy)
cv.Remap(src, dst, mapx, mapy, cv.CV_INTER_LINEAR + cv.CV_WARP_FILL_OUTLIERS, cv.ScalarAll(0))
# cv.Undistort2(src, dst, intrinsics, dist_coeffs)
cv.SaveImage(output, dst)
if __name__ == '__main__':
main(sys.argv)
Also note that OpenCV uses a very different lens distortion model to the one in the web page you linked to.
(Original poster, providing an alternative)
The following function maps destination (rectilinear) coordinates to source (fisheye-distorted) coordinates. (I'd appreciate help in reversing it)
I got to this point through trial-and-error: I don't fundamentally grasp why this code is working, explanations and improved accuracy appreciated!
def dist(x,y):
return sqrt(x*x+y*y)
def correct_fisheye(src_size,dest_size,dx,dy,factor):
""" returns a tuple of source coordinates (sx,sy)
(note: values can be out of range)"""
# convert dx,dy to relative coordinates
rx, ry = dx-(dest_size[0]/2), dy-(dest_size[1]/2)
# calc theta
r = dist(rx,ry)/(dist(src_size[0],src_size[1])/factor)
if 0==r:
theta = 1.0
else:
theta = atan(r)/r
# back to absolute coordinates
sx, sy = (src_size[0]/2)+theta*rx, (src_size[1]/2)+theta*ry
# done
return (int(round(sx)),int(round(sy)))
When used with a factor of 3.0, it successfully undistorts the images used as examples (I made no attempt at quality interpolation):
Dead link
(And this is from the blog post, for comparison:)
If you think your formulas are exact, you can comput an exact formula with trig, like so:
Rin = 2 f sin(w/2) -> sin(w/2)= Rin/2f
Rout= f tan(w) -> tan(w)= Rout/f
(Rin/2f)^2 = [sin(w/2)]^2 = (1 - cos(w))/2 -> cos(w) = 1 - 2(Rin/2f)^2
(Rout/f)^2 = [tan(w)]^2 = 1/[cos(w)]^2 - 1
-> (Rout/f)^2 = 1/(1-2[Rin/2f]^2)^2 - 1
However, as #jmbr says, the actual camera distortion will depend on the lens and the zoom. Rather than rely on a fixed formula, you might want to try a polynomial expansion:
Rout = Rin*(1 + A*Rin^2 + B*Rin^4 + ...)
By tweaking first A, then higher-order coefficients, you can compute any reasonable local function (the form of the expansion takes advantage of the symmetry of the problem). In particular, it should be possible to compute initial coefficients to approximate the theoretical function above.
Also, for good results, you will need to use an interpolation filter to generate your corrected image. As long as the distortion is not too great, you can use the kind of filter you would use to rescale the image linearly without much problem.
Edit: as per your request, the equivalent scaling factor for the above formula:
(Rout/f)^2 = 1/(1-2[Rin/2f]^2)^2 - 1
-> Rout/f = [Rin/f] * sqrt(1-[Rin/f]^2/4)/(1-[Rin/f]^2/2)
If you plot the above formula alongside tan(Rin/f), you can see that they are very similar in shape. Basically, distortion from the tangent becomes severe before sin(w) becomes much different from w.
The inverse formula should be something like:
Rin/f = [Rout/f] / sqrt( sqrt(([Rout/f]^2+1) * (sqrt([Rout/f]^2+1) + 1) / 2 )
I blindly implemented the formulas from here, so I cannot guarantee it would do what you need.
Use auto_zoom to get the value for the zoom parameter.
def dist(x,y):
return sqrt(x*x+y*y)
def fisheye_to_rectilinear(src_size,dest_size,sx,sy,crop_factor,zoom):
""" returns a tuple of dest coordinates (dx,dy)
(note: values can be out of range)
crop_factor is ratio of sphere diameter to diagonal of the source image"""
# convert sx,sy to relative coordinates
rx, ry = sx-(src_size[0]/2), sy-(src_size[1]/2)
r = dist(rx,ry)
# focal distance = radius of the sphere
pi = 3.1415926535
f = dist(src_size[0],src_size[1])*factor/pi
# calc theta 1) linear mapping (older Nikon)
theta = r / f
# calc theta 2) nonlinear mapping
# theta = asin ( r / ( 2 * f ) ) * 2
# calc new radius
nr = tan(theta) * zoom
# back to absolute coordinates
dx, dy = (dest_size[0]/2)+rx/r*nr, (dest_size[1]/2)+ry/r*nr
# done
return (int(round(dx)),int(round(dy)))
def fisheye_auto_zoom(src_size,dest_size,crop_factor):
""" calculate zoom such that left edge of source image matches left edge of dest image """
# Try to see what happens with zoom=1
dx, dy = fisheye_to_rectilinear(src_size, dest_size, 0, src_size[1]/2, crop_factor, 1)
# Calculate zoom so the result is what we wanted
obtained_r = dest_size[0]/2 - dx
required_r = dest_size[0]/2
zoom = required_r / obtained_r
return zoom
I took what JMBR did and basically reversed it. He took the radius of the distorted image (Rd, that is, the distance in pixels from the center of the image) and found a formula for Ru, the radius of the undistorted image.
You want to go the other way. For each pixel in the undistorted (processed image), you want to know what the corresponding pixel is in the distorted image.
In other words, given (xu, yu) --> (xd, yd). You then replace each pixel in the undistorted image with its corresponding pixel from the distorted image.
Starting where JMBR did, I do the reverse, finding Rd as a function of Ru. I get:
Rd = f * sqrt(2) * sqrt( 1 - 1/sqrt(r^2 +1))
where f is the focal length in pixels (I'll explain later), and r = Ru/f.
The focal length for my camera was 2.5 mm. The size of each pixel on my CCD was 6 um square. f was therefore 2500/6 = 417 pixels. This can be found by trial and error.
Finding Rd allows you to find the corresponding pixel in the distorted image using polar coordinates.
The angle of each pixel from the center point is the same:
theta = arctan( (yu-yc)/(xu-xc) ) where xc, yc are the center points.
Then,
xd = Rd * cos(theta) + xc
yd = Rd * sin(theta) + yc
Make sure you know which quadrant you are in.
Here is the C# code I used
public class Analyzer
{
private ArrayList mFisheyeCorrect;
private int mFELimit = 1500;
private double mScaleFESize = 0.9;
public Analyzer()
{
//A lookup table so we don't have to calculate Rdistorted over and over
//The values will be multiplied by focal length in pixels to
//get the Rdistorted
mFisheyeCorrect = new ArrayList(mFELimit);
//i corresponds to Rundist/focalLengthInPixels * 1000 (to get integers)
for (int i = 0; i < mFELimit; i++)
{
double result = Math.Sqrt(1 - 1 / Math.Sqrt(1.0 + (double)i * i / 1000000.0)) * 1.4142136;
mFisheyeCorrect.Add(result);
}
}
public Bitmap RemoveFisheye(ref Bitmap aImage, double aFocalLinPixels)
{
Bitmap correctedImage = new Bitmap(aImage.Width, aImage.Height);
//The center points of the image
double xc = aImage.Width / 2.0;
double yc = aImage.Height / 2.0;
Boolean xpos, ypos;
//Move through the pixels in the corrected image;
//set to corresponding pixels in distorted image
for (int i = 0; i < correctedImage.Width; i++)
{
for (int j = 0; j < correctedImage.Height; j++)
{
//which quadrant are we in?
xpos = i > xc;
ypos = j > yc;
//Find the distance from the center
double xdif = i-xc;
double ydif = j-yc;
//The distance squared
double Rusquare = xdif * xdif + ydif * ydif;
//the angle from the center
double theta = Math.Atan2(ydif, xdif);
//find index for lookup table
int index = (int)(Math.Sqrt(Rusquare) / aFocalLinPixels * 1000);
if (index >= mFELimit) index = mFELimit - 1;
//calculated Rdistorted
double Rd = aFocalLinPixels * (double)mFisheyeCorrect[index]
/mScaleFESize;
//calculate x and y distances
double xdelta = Math.Abs(Rd*Math.Cos(theta));
double ydelta = Math.Abs(Rd * Math.Sin(theta));
//convert to pixel coordinates
int xd = (int)(xc + (xpos ? xdelta : -xdelta));
int yd = (int)(yc + (ypos ? ydelta : -ydelta));
xd = Math.Max(0, Math.Min(xd, aImage.Width-1));
yd = Math.Max(0, Math.Min(yd, aImage.Height-1));
//set the corrected pixel value from the distorted image
correctedImage.SetPixel(i, j, aImage.GetPixel(xd, yd));
}
}
return correctedImage;
}
}
I found this pdf file and I have proved that the maths are correct (except for the line vd = *xd**fv+v0 which should say vd = **yd**+fv+v0).
http://perception.inrialpes.fr/CAVA_Dataset/Site/files/Calibration_OpenCV.pdf
It does not use all of the latest co-efficients that OpenCV has available but I am sure that it could be adapted fairly easily.
double k1 = cameraIntrinsic.distortion[0];
double k2 = cameraIntrinsic.distortion[1];
double p1 = cameraIntrinsic.distortion[2];
double p2 = cameraIntrinsic.distortion[3];
double k3 = cameraIntrinsic.distortion[4];
double fu = cameraIntrinsic.focalLength[0];
double fv = cameraIntrinsic.focalLength[1];
double u0 = cameraIntrinsic.principalPoint[0];
double v0 = cameraIntrinsic.principalPoint[1];
double u, v;
u = thisPoint->x; // the undistorted point
v = thisPoint->y;
double x = ( u - u0 )/fu;
double y = ( v - v0 )/fv;
double r2 = (x*x) + (y*y);
double r4 = r2*r2;
double cDist = 1 + (k1*r2) + (k2*r4);
double xr = x*cDist;
double yr = y*cDist;
double a1 = 2*x*y;
double a2 = r2 + (2*(x*x));
double a3 = r2 + (2*(y*y));
double dx = (a1*p1) + (a2*p2);
double dy = (a3*p1) + (a1*p2);
double xd = xr + dx;
double yd = yr + dy;
double ud = (xd*fu) + u0;
double vd = (yd*fv) + v0;
thisPoint->x = ud; // the distorted point
thisPoint->y = vd;
This can be solved as an optimization problem. Simply draw on curves in images that are supposed to be straight lines. Store the contour points for each of those curves. Now we can solve the fish eye matrix as a minimization problem. Minimize the curve in points and that will give us a fisheye matrix. It works.
It can be done manually by adjusting the fish eye matrix using trackbars! Here is a fish eye GUI code using OpenCV for manual calibration.
I was trying to figure out how to make a swirl in the photo, tried looking everywhere for what exactly you do to the pixels. I was talking with a friend and we kinda talked about using sine functions for the redirection of pixels?
Let's say you define your swirl using 4 parameters:
X and Y co-ordinates of the center of the swirl
Swirl radius in pixels
Number of twists
Start with a source image and create a destination image with the swirl applied. For each pixel (in the destination image), you need to adjust the pixel co-ordinates based on the swirl and then read a pixel from the source image. To apply the swirl, figure out the distance of the pixel from the center of the swirl and it's angle. Then adjust the angle by an amount based on the number of twists that fades out the further you get from the center until it gets to zero when you get to the swirl radius. Use the new angle to compute the adjusted pixel co-ordinates to read from. In pseudo code it's something like this:
Image src, dest
float swirlX, swirlY, swirlRadius, swirlTwists
for(int y = 0; y < dest.height; y++)
{
for(int x = 0; x < dest.width; x++)
{
// compute the distance and angle from the swirl center:
float pixelX = (float)x - swirlX;
float pixelY = (float)y - swirlY;
float pixelDistance = sqrt((pixelX * pixelX) + (pixelY * pixelY));
float pixelAngle = arc2(pixelY, pixelX);
// work out how much of a swirl to apply (1.0 in the center fading out to 0.0 at the radius):
float swirlAmount = 1.0f - (pixelDistance / swirlRadius);
if(swirlAmount > 0.0f)
{
float twistAngle = swirlTwists * swirlAmount * PI * 2.0;
// adjust the pixel angle and compute the adjusted pixel co-ordinates:
pixelAngle += twistAngle;
pixelX = cos(pixelAngle) * pixelDistance;
pixelY = sin(pixelAngle) * pixelDistance;
}
// read and write the pixel
dest.setPixel(x, y, src.getPixel(swirlX + pixelX, swirlY + pixelY));
}
}