Check for pixel values in a neighborhood - image

I'm trying to write a MATLAB script that does the following:
Given: pixel coordinates(x,y) for a .jpg image
Goal: Check, within a 5 pixel radius of given coordinates, if there is a pixel of a certain value.
For example, let's say I'm given the coordinates (100,100), then I want to check the neighborhood of (100,100) within my image for any pixels that are black (0,0,0). So perhaps, pixel (103, 100) and (104,100) might have the value (0,0,0).
Current code:
x_coord = uint32(coord(:,1));
y_coord = uint32(coord(:,2));
count = 0;
for i = 1:length(x_coord)
%(img(x,y) returns pixel value at that (x,y)
%Note 0 = black. Indicating that, at that position, the image is just
% black
if img(x_coord(i),y_coord(i)) == 0
count = count + 1;
end
end
It currently only checks at an exact location. Not in a local neighborhood. How to could I extend this?
EDIT: Also note, as long as there as at least one pixel in the neighborhood with the value, I increment count. I'm not trying to enumerate how many pixels in the neighborhood have that value, just trying to find evidence of at least one pixel that has that value.
EDIT:
Even though I am unable to identify an error with the code, I am not able to get the exact results I want. Here is the code I am using.
val = 0; %pixel value to check
N = 50; % neighbourhood radius
%2D grid of coordinates surrounding center coordinate
[R, C] = ndgrid(1 : size(img, 1), 1 : size(img, 2));
for kk = 1 : size(coord, 1)
r = coord(kk, 1); c = coord(kk, 2); % Get pixel locations
% mask of valid locations within the neighbourhood (avoid boundary problems)
mask = (R - r).^2 + (C - c).^2 <= N*N;
pix = img(mask); % Get the valid pixels
valid = any(pix(:) ~= val);
% Add either 0 or 1 depending if we have found any matching pixels
if(valid == 1)
img = insertMarker(img, [r c], 'x', 'color', 'red', 'size', 10);
imwrite(img, images(i).name,'tiff');
end
count = count + valid;
end

An easier way to do this would be to use indexing to grab a neighbourhood, then to check to see if any of the pixels in the neighbourhood have the value that you're looking for, use any on a flattened version of this neighbourhood. The trick with grabbing the right neighbourhood is to first generate a 2D grid of coordinates that span the entire dimensions of your image, then simply use the equation of a circle with the centre of it being each coordinate you are looking at and determine those locations that satisfy the following equation:
(x - a)^2 + (y - b)^2 <= N^2
N is the radius of the observation window, (a, b) is a coordinate of interest while (x, y) is a coordinate in the image. Use meshgrid to generate the coordinates.
You would use the above equation to create a logical mask, index into your image to pull the locations that are valid within the mask and check how many pixels match the one you want. Another added benefit with the above approach is that you are not subject to any out of bounds errors. Because you are pre-generating the list of all valid coordinates in your image, generating the mask will confine you within the boundaries of the image so you never have to check for out of boundaries conditions.... even when you specify coordinates to search that are out of bounds.
Specifically, assuming your image is stored in img, you would do:
count = 0; % Remembers total count of pixels matching a value
val = 0; % Value to match
N = 50; % Radius of neighbourhood
% Generate 2D grid of coordinates
[x, y] = meshgrid(1 : size(img, 2), 1 : size(img, 1));
% For each coordinate to check...
for kk = 1 : size(coord, 1)
a = coord(kk, 1); b = coord(kk, 2); % Get the pixel locations
mask = (x - a).^2 + (y - b).^2 <= N*N; % Get a mask of valid locations
% within the neighbourhood
pix = img(mask); % Get the valid pixels
count = count + any(pix(:) == val); % Add either 0 or 1 depending if
% we have found any matching pixels
end

The proposed solution:
fc = repmat(-5:5,11,1);
I = (fc.^2+fc'.^2)<=25;
fc_x = fc(I);
fc_y = fc'; fc_y = fc_y(I);
for i = 1:length(x_coord)
x_toCheck = fc_x + x_coord(i);
y_toCheck = fc_y + y_coord(i);
I = x_toCheck>0 & x_toCheck<=yourImageWidth;
I = I.*(y_toCheck>0 & y_toCheck<=yourImageHeight);
x_toCheck = x_toCheck(logical(I));
y_toCheck = y_toCheck(logical(I));
count = sum(img(x_toCheck(:),y_toCheck(:)) == 0);
end
If your img function can only check one pixel at a time, just add a for loop:
for i = 1:length(x_coord)
x_toCheck = fc_x + x_coord(i);
y_toCheck = fc_y + y_coord(i);
I = x_toCheck>0 & x_toCheck<=yourImageWidth;
I = I.*(y_toCheck>0 & y_toCheck<=yourImageHeight);
x_toCheck = x_toCheck(logical(I));
y_toCheck = y_toCheck(logical(I));
for j = 1:length(x_toCheck)
count = count + (img(x_toCheck(j),y_toCheck(j)) == 0);
end
end
Step-by-step:
You first need to get all the coordinates within 5 pixels range of the given coordinate.
We start by building a square of 11 pixels in length/width.
fc = repmat(-5:5,11,1);
fc_x = fc;
fc_y = fc';
plot(fc_x,fc_y,'.');
We now need to build a filter to get rid of those points outside the 5-pixel radius.
I = (fc.^2+fc'.^2)<=25;
Apply the filter, so we can get a circle of 5-pixel radius.
fc_x = fc_x(I);
fc_y = fc_y(I);
Next translate the centre of the circle to the given coordinate:
x_toCheck = fc_x + x_coord(i);
y_toCheck = fc_y + y_coord(i);
You need to check whether part of the circle is outside the range of your image:
I = x_toCheck>0 & x_toCheck<=yourImageWidth;
I = I.*(y_toCheck>0 & y_toCheck<=yourImageHeight);
x_toCheck = x_toCheck(logical(I));
y_toCheck = y_toCheck(logical(I));
Finally count the pixels:
count = sum(img(x_toCheck,y_toCheck) == 0);

Related

Interp2 of image with transformed coordinates

I have 2 greyscale images that i am trying to align using scalar scaling 1 , rotation matrix [2,2] and translation vector [2,1]. I can calculate image1's transformed coordinates as
y = s*R*x + t;
Below the resulting images are shown.
The first image is image1 before transformation,
the second image is image1 (red) with attempted interpolation using interp2 shown on top of image2 (green)
The third image is when i manually insert the pixel values from image1 into an empty array (that has the same size as image2) using the transformed coordinates.
From this we can see that the coordinate transformation must have been successful, as the images are aligned although not perfectly (which is to be expected since only 2 coordinates were used in calculating s, R and t) .
How come interp2 is not producing a result more similar to when i manually insert pixel values?
Below the code for doing this is included:
Interpolation code
function [transformed_image] = interpolate_image(im_r,im_t,s,R,t)
[m,n] = size(im_t);
% doesn't help if i use get_grid that the other function is using here
[~, grid_xr, grid_yr] = get_ipgrid(im_r);
[x_t, grid_xt, grid_yt] = get_ipgrid(im_t);
y = s*R*x_t + t;
yx = reshape(y(1,:), m,n);
yy = reshape(y(2,:), m,n);
transformed_image = interp2(grid_xr, grid_yr, im_r, yx, yy, 'nearest');
end
function [x, grid_x, grid_y] = get_ipgrid(image)
[m,n] = size(image);
[grid_x,grid_y] = meshgrid(1:n,1:m);
x = [reshape(grid_x, 1, []); reshape(grid_y, 1, [])]; % X is [2xM*N] coordinate pairs
end
The manual code
function [transformed_image] = transform_image(im_r,im_t,s,R,t)
[m,n] = size(im_t);
[x_t, grid_xt, grid_yt] = get_grid(im_t);
y = s*R*x_t + t;
ymat = reshape(y',m,n,2);
yx = ymat(:,:,1);
yy = ymat(:,:,2);
transformed_image = zeros(m,n);
for i = 1:m
for j = 1:n
% make sure coordinates are inside
if (yx(i,j) < m & yy(i,j) < n & yx(i,j) > 0.5 & yy(i,j) > 0.5)
transformed_image(round(yx(i,j)),round(yy(i,j))) = im_r(i,j);
end
end
end
end
function [x, grid_x, grid_y] = get_grid(image)
[m,n] = size(image);
[grid_y,grid_x] = meshgrid(1:n,1:m);
x = [grid_x(:) grid_y(:)]'; % X is [2xM*N] coordinate pairs
end
Can anyone see what i'm doing wrong with interp2? I feel like i have tried everything
Turns out i got interpolation all wrong.
In my question i calculate the coordinates of im1 in im2.
However the way interpolation works is that i need to calculate the coordinates of im2 in im1 such that i can map the image as shown below.
This means that i also calculated the wrong s,R and t since they were used to transform im1 -> im2, where as i needed im2 -> im1. (this is also called the inverse transform). Below is the manual code, that is basically the same as interp2 with nearest neighbour interpolation
function [transformed_image] = transform_image(im_r,im_t,s,R,t)
[m,n] = size(im_t);
[x_t, grid_xt, grid_yt] = get_grid(im_t);
y = s*R*x_t + t;
ymat = reshape(y',m,n,2);
yx = ymat(:,:,1);
yy = ymat(:,:,2);
transformed_image = zeros(m,n);
for i = 1:m
for j = 1:n
% make sure coordinates are inside
if (yx(i,j) < m & yy(i,j) < n & yx(i,j) > 0.5 & yy(i,j) > 0.5)
transformed_image(i,j) = im_r(round(yx(i,j)),round(yy(i,j)));
end
end
end
end

How to rotate a 2D theoretical path to fit or be overlaid on a path from videoprocessing

I track the motion of an object from a video file in MATLAB and save the locations from each frame in a numberOfFrames x 2 array.
I know the theoretical path or intended path. When recording the movie the camera is at some unknown angle in space. Therefore, the path is skewed. The only information I have is the scaling between the pixels and millimeters by using the object diameter.
Now I would like to rotate the intended path, and move it around until it is overlaid on the tracked motion path.
I start with my theoretical path (Pth) then rotate it in 3-dimensions to "Pthr". After that I loop over each point in "M". And for each point in "M", I look for the closest point from "Pthr". Then, I repeat for the next point in "M". This probably has a problem of choosing the same point in "Pthr" for multiple points in "M".
I noticed this is sensitive to my initial guess and it gives terrible results.
Also, M is not a perfect path, since it is experimental measurements it is no where near perfect. measured vs. theoretical unrotated path
% M = [Mx,My], is location in x and y, from motion tracking.
% scale = 20; % pixels/mm, using the size of object
% I build the theoretical path (Pth) goes from to (0,0,0) to (0,3,0) to
% (3,3,0) to (3,0,0) to be approximately the same length as M
Pthup = linspace(0,3,num)';
Pthdwn = linspace(3,0,num)';
Pth0 = zeros(size(Pthup));
Pth3 = 3*ones(size(Pthup));
% Pth is approximately same length as M
Pth = scale*[Pth0 Pthup Pth0;Pthup Pth3 Pth0;Pth3 Pthdwn Pth0];
% using fmincon in matlab to minimize the sum of the square
lb = [145 0 -45 min(min(M)) min(min(M))]; %upper bound
ub = [180 90 45 max(max(M)) max(max(M))]; %lower bound
coro = [180 0 0 mean(Mx) mean(My)]; %initial guess
% initial guess (theta(x),theta(y),theta(z), shift in x, shift in y)
cnt = 0; er = 1;
while (abs(er)>0.1)
[const,fval] = fmincon(#(cor)findOrientation(cor,Pth,M),coro,[],[],[],[],lb,ub);
er = sum(const-coro);
coro = const;
cnt = 1+cnt;
if (cnt>50)
cnt = cnt;
break
end
end
%% function findOrientation keeps rotating Pth until it is closest to M
function [Eo] = findOrientation(cor,Pth,M)
% cor = [angle of rotations, center coordinate];(degrees, non-dimensiolaized in pixels)
% M = measured coordinates from movie in pixel
% coor: is output of the form [x-coordiante,y-coordinate, absolute distance from Center(i,:)]
% F = sum of least square, sum(coor(:,3))
%% Rotation of theoretical path about z,y,x and shifting in it in xy
thx = cor(1);
thy = cor(2);
thz = cor(3);
xy = cor([4:5]);
% T = [cosd(thn) -sind(thn);
% sind(thn) cosd(thn)]; %rotation matrix in 3D
Tz = [cosd(thz) -sind(thz) 0;
sind(thz) cosd(thz) 0;0 0 1]; %rotation matrix
Ty = [cosd(thy) 0 -sind(thy);0 1 0;
sind(thy) 0 cosd(thy)]; %rotation matrix
Tx = [1 0 0;0 cosd(thx) -sind(thx);
0 sind(thx) cosd(thx)]; %rotation matrix
Pthr = zeros(size(Pth));
for i = 1:size(Pth,1)
xp = Tz*Pth(i,:)';
xp = Ty*xp;
xp = Tx*xp;
Pthr(i,:) = xp.';
end
Pthr = Pthr(:,[1,2]); % omit third value because it is 2D
Pthr = Pthr + [cor([4:5])];
rin = sqrt(Pthr(:,1).^2+Pthr(:,2).^2); %theoretical radius
Centern = sqrt(M(:,1).^2 + M(:,2).^2);%measured radius
for i = 1:size(M,1) %loop over each point in tracked motion
sub = Pthr-M(i,:); %subtracting M(i,:) from all Pthr
for j = 1:length(sub)
dist(j,1) = norm(sub(j,:));% distance from M(i,:) to all ri
end
%index is based on the min absolute distance between Pthr and M(i,:). It chooses the closest Pthr to a specific M(i,:)
[mn, index] = min(dist);
erri = abs(rin(index)-Centern(i))./rin(index);
coor(i,:) = erri;
end
Eo = sum(coor);

Shortest distance between a specific object to other objects

Following these two posts that deals with finding the distance between objects in binary image, how can I only output/calculate only the shortest distance between a specific object to the rest (for examples, {1->3}, {2->5}, {3->1}, {4->7)?
https://www.mathworks.com/matlabcentral/answers/164955-distance-between-several-objects-in-binary-image
Pairwise distance between all centroid coordinate combinations - Matlab
Script:
clc;
clear all;
I = rgb2gray(imread('E:/NCircles.png'));
imshow(I);
BW = imbinarize(I,'adaptive');
BW = imfill(BW, 'holes');
BW = bwlabel(BW);
s = regionprops(BW,'Area', 'BoundingBox', 'Eccentricity', 'MajorAxisLength', 'MinorAxisLength', 'Orientation', 'Perimeter','Centroid');
imshow(BW)
hold on
for k = 1:numel(s)
c = s(k).Centroid;
text(c(1), c(2), sprintf('%d', k), 'HorizontalAlignment', 'center', 'VerticalAlignment', 'middle');
end
boundaries = bwboundaries(BW);
numberOfBoundaries = size(boundaries, 1);
for k = 1 : numberOfBoundaries
thisBoundary = boundaries{k};
plot(thisBoundary(:,2), thisBoundary(:,1), 'r', 'LineWidth', 3);
end
hold off;
% Define object boundaries
numberOfBoundaries = size(boundaries, 1)
for b1 = 1 : numberOfBoundaries
for b2 = 1 : numberOfBoundaries
if b1 == b2
% Can't find distance between the region and itself
continue;
end
boundary1 = boundaries{b1};
boundary2 = boundaries{b2};
boundary1x = boundary1(:, 2);
boundary1y = boundary1(:, 1);
x1=1;
y1=1;
x2=1;
y2=1;
overallMinDistance = inf; % Initialize.
% For every point in boundary 2, find the distance to every point in boundary 1.
for k = 1 : size(boundary2, 1)
% Pick the next point on boundary 2.
boundary2x = boundary2(k, 2);
boundary2y = boundary2(k, 1);
% For this point, compute distances from it to all points in boundary 1.
allDistances = sqrt((boundary1x - boundary2x).^2 + (boundary1y - boundary2y).^2);
% Find closest point, min distance.
[minDistance(k), indexOfMin] = min(allDistances);
if minDistance(k) < overallMinDistance
x1 = boundary1x(indexOfMin);
y1 = boundary1y(indexOfMin);
x2 = boundary2x;
y2 = boundary2y;
overallMinDistance = minDistance(k);
end
end
% Find the overall min distance
minDistance = min(minDistance);
% Report to command window.
fprintf('The minimum distance from region %d to region %d is %.3f pixels\n', b1, b2, minDistance);
% Draw a line between point 1 and 2
line([x1, x2], [y1, y2], 'Color', 'y', 'LineWidth', 3);
end
end
Given BW and boundaries as defined above, and a source object from which to calculate distances to all other objects:
source_object = 1; % label of source object in BW
Construct a distance image such that the value of each pixel is its distance from the source object using bwdist:
% anonymous function to convert cell array of subsripts
% into cell array of indices
indsfun = #(a) sub2ind(size(BW), a(:,1), a(:,2));
% use function on all of the cell's boundary objects
object_inds = cellfun(indsfun, boundaries, 'UniformOutput', false);
source_image = zeros(size(BW)); % create image containing only source object
source_image(object_inds{source_object}) = 1;
% compute distance from source to all other pixels in image
dist_image = bwdist(source_image, 'euclidean'); % replace with desired metric
imagesc(dist_image); % not necessary, but gives a cool image
Now, for each object in the original image, find the minimum distance of its boundary to the source object boundary:
min_dist = zeros(1,numel(boundaries)); % hold minimum distances
for target_object = 1:numel(boundaries)
% get the distance values at the indices of the target object
% and store the minimum.
min_dist(target_object) = min(dist_image(object_inds{target_object}));
end
In the end, min_dist will contain the minimum (boundary) distance from the source object to all other objects. A sample run on your image gives the following Euclidean distances:
min_dist =
Columns 1 through 7:
0.00000 67.54258 60.00000 207.23416 154.48625 168.79869 319.01410
Columns 8 through 13:
236.05296 324.71063 344.05814 367.00000 469.07996 509.00000

matlab: efficient computation of local histograms within circular neighboorhoods

I've an image over which I would like to compute a local histogram within a circular neighborhood. The size of the neighborhood is given by a radius. Although the code below does the job, it's computationally expensive. I run the profiler and the way I'm accessing to the pixels within the circular neighborhoods is already expensive.
Is there any sort of improvement/optimization based maybe on vectorization? Or for instance, storing the neighborhoods as columns?
I found a similar question in this post and the proposed solution is quite in the spirit of the code below, however the solution is still not appropriate to my case. Any ideas are really welcomed :-) Imagine for the moment, the image is binary, but the method should also ideally work with gray-level images :-)
[rows,cols] = size(img);
hist_img = zeros(rows, cols, 2);
[XX, YY] = meshgrid(1:cols, 1:rows);
for rr=1:rows
for cc=1:cols
distance = sqrt( (YY-rr).^2 + (XX-cc).^2 );
mask_radii = (distance <= radius);
bwresponses = img(mask_radii);
[nelems, ~] = histc(double(bwresponses),0:255);
% do some processing over the histogram
...
end
end
EDIT 1 Given the received feedback, I tried to update the solution. However, it's not yet correct
radius = sqrt(2.0);
disk = diskfilter(radius);
fun = #(x) histc( x(disk>0), min(x(:)):max(x(:)) );
output = im2col(im, size(disk), fun);
function disk = diskfilter(radius)
height = 2*ceil(radius)+1;
width = 2*ceil(radius)+1;
[XX,YY] = meshgrid(1:width,1:height);
dist = sqrt((XX-ceil(width/2)).^2+(YY-ceil(height/2)).^2);
circfilter = (dist <= radius);
end
Following on the technique I described in my answer to a similar question you could try to do the following:
compute the index offsets from a particular voxel that get you to all the neighbors within a radius
Determine which voxels have all neighbors at least radius away from the edge
Compute the neighbors for all these voxels
Generate your histograms for each neighborhood
It is not hard to vectorize this, but note that
It will be slow when the neighborhood is large
It involves generating an intermediate matrix that is NxM (N = voxels in image, M = voxels in neighborhood) which could get very large
Here is the code:
% generate histograms for neighborhood within radius r
A = rand(200,200,200);
radius = 2.5;
tic
sz=size(A);
[xx yy zz] = meshgrid(1:sz(2), 1:sz(1), 1:sz(3));
center = round(sz/2);
centerPoints = find((xx - center(1)).^2 + (yy - center(2)).^2 + (zz - center(3)).^2 < radius.^2);
centerIndex = sub2ind(sz, center(1), center(2), center(3));
% limit to just the points that are "far enough on the inside":
inside = find(xx > radius+1 & xx < sz(2) - radius & ...
yy > radius + 1 & yy < sz(1) - radius & ...
zz > radius + 1 & zz < sz(3) - radius);
offsets = centerPoints - centerIndex;
allPoints = 1:prod(sz);
insidePoints = allPoints(inside);
indices = bsxfun(#plus, offsets, insidePoints);
hh = histc(A(indices), 0:0.1:1); % <<<< modify to give you the histogram you want
toc
A 2D version of the same code (which might be all you need, and is considerably faster):
% generate histograms for neighborhood within radius r
A = rand(200,200);
radius = 2.5;
tic
sz=size(A);
[xx yy] = meshgrid(1:sz(2), 1:sz(1));
center = round(sz/2);
centerPoints = find((xx - center(1)).^2 + (yy - center(2)).^2 < radius.^2);
centerIndex = sub2ind(sz, center(1), center(2));
% limit to just the points that are "far enough on the inside":
inside = find(xx > radius+1 & xx < sz(2) - radius & ...
yy > radius + 1 & yy < sz(1) - radius);
offsets = centerPoints - centerIndex;
allPoints = 1:prod(sz);
insidePoints = allPoints(inside);
indices = bsxfun(#plus, offsets, insidePoints);
hh = histc(A(indices), 0:0.1:1); % <<<< modify to give you the histogram you want
toc
You're right, I don't think that colfilt can be used as you're not applying a filter. You'll have to check the correctness, but here's my attempt using im2col and your diskfilter function (I did remove the conversion to double so it now output logicals):
function circhist
% Example data
im = randi(256,20)-1;
% Ranges - I do this globally for the whole image rather than for each neighborhood
mini = min(im(:));
maxi = max(im(:));
edges = linspace(mini,maxi,20);
% Disk filter
radius = sqrt(2.0);
disk = diskfilter(radius); % Returns logical matrix
% Pad array with -1
im_pad = padarray(im, (size(disk)-1)/2, -1);
% Convert sliding neighborhoods to columns
B = im2col(im_pad, size(disk), 'sliding');
% Get elements from each column that correspond to disk (logical indexing)
C = B(disk(:), :);
% Apply histogram across columns to count number of elements
out = histc(C, edges)
% Display output
figure
imagesc(out)
h = colorbar;
ylabel(h,'Counts');
xlabel('Neighborhood #')
ylabel('Bins')
axis xy
function disk = diskfilter(radius)
height = 2*ceil(radius)+1;
width = 2*ceil(radius)+1;
[XX,YY] = meshgrid(1:width,1:height);
dist = sqrt((XX-ceil(width/2)).^2+(YY-ceil(height/2)).^2);
disk = (dist <= radius);
If you want to set your ranges (edges) based on each neighborhood then you'll need to make sure that the vector is always the same length if you want to build a big matrix (and then the rows of that matrix won't correspond to each other).
You should note that the shape of the disk returned by fspecial is not as circular as what you were using. It's meant to be used a smoothing/averaging filter so the edges are fuzzy (anti-aliased). Thus when you use ~=0 it will grab more pixels. It'd stick with your own function, which is faster anyways.
You could try processing with an opposite logic (as briefly explained in the comment)
hist = zeros(W+2*R, H+2*R, Q);
for i = 1:R+1;
for j = 1:R+1;
if ((i-R-1)^2+(j-R-1)^2 < R*R)
for q = 0:1:Q-1;
hist(i:i+W-1,j:j+H-1,q+1) += (image == q);
end
end
end
end

Cubic to equirectangular projection algorithm

I have a cube map texture which defines a surrounding, however I need to pass it to a program which only works with latitude/longitude maps. I am really at lost here on how to do the translation. Any help here?
In other words, I need to come from here:
To this (I think that image has an aditional -90° rotation over the x axis):
update: I got the official names of the projections. By the way, I found the opposite projection here
A general procedure for projecting raster images like this is:
for each pixel of the destination image:
calculate the corresponding unit vector in 3-dimensional space
calculate the x,y coordinate for that vector in the source image
sample the source image at that coordinate and assign the value to the destination pixel
The last step is simply interpolation. We will focus on the other two steps.
The unit vector for a given latitude and longitude is (+z towards the north pole, +x towards the prime meridian):
x = cos(lat)*cos(lon)
y = cos(lat)*sin(lon)
z = sin(lat)
Assume the cube is +/- 1 unit around the origin (i.e. 2x2x2 overall size).
Once we have the unit vector, we can find the face of the cube it's on by looking at the element with the largest absolute value. For example, if our unit vector was <0.2099, -0.7289, 0.6516>, then the y element has the largest absolute value. It's negative, so the point will be found on the -y face of the cube. Normalize the other two coordinates by dividing by the y magnitude to get the location within that face. So, the point will be at x=0.2879, z=0.8939 on the -y face.
I'd like to share my MATLAB implementation of this conversion. I also borrowed from the OpenGL 4.1 specification, Chapter 3.8.10 (found here), as well as Paul Bourke's website (found here). Make sure you look under the subheading: Converting to and from 6 cubic environment maps and a spherical map.
I also used Sambatyon's post above as inspiration. It started off as a port from Python over to MATLAB, but I made the code so that it is completely vectorized (i.e. no for loops). I also take the cubic image and split it up into 6 separate images, as the application I'm building has the cubic image in this format. Also there is no error checking with the code, and that this assumes that all of the cubic images are of the same size (n x n). This also assumes that the images are in RGB format. If you'd like to do this for a monochromatic image, simply comment out those lines of code that require access to more than one channel. Here we go!
function [out] = cubic2equi(top, bottom, left, right, front, back)
% Height and width of equirectangular image
height = size(top, 1);
width = 2*height;
% Flags to denote what side of the cube we are facing
% Z-axis is coming out towards you
% X-axis is going out to the right
% Y-axis is going upwards
% Assuming that the front of the cube is towards the
% negative X-axis
FACE_Z_POS = 1; % Left
FACE_Z_NEG = 2; % Right
FACE_Y_POS = 3; % Top
FACE_Y_NEG = 4; % Bottom
FACE_X_NEG = 5; % Front
FACE_X_POS = 6; % Back
% Place in a cell array
stackedImages{FACE_Z_POS} = left;
stackedImages{FACE_Z_NEG} = right;
stackedImages{FACE_Y_POS} = top;
stackedImages{FACE_Y_NEG} = bottom;
stackedImages{FACE_X_NEG} = front;
stackedImages{FACE_X_POS} = back;
% Place in 3 3D matrices - Each matrix corresponds to a colour channel
imagesRed = uint8(zeros(height, height, 6));
imagesGreen = uint8(zeros(height, height, 6));
imagesBlue = uint8(zeros(height, height, 6));
% Place each channel into their corresponding matrices
for i = 1 : 6
im = stackedImages{i};
imagesRed(:,:,i) = im(:,:,1);
imagesGreen(:,:,i) = im(:,:,2);
imagesBlue(:,:,i) = im(:,:,3);
end
% For each co-ordinate in the normalized image...
[X, Y] = meshgrid(1:width, 1:height);
% Obtain the spherical co-ordinates
Y = 2*Y/height - 1;
X = 2*X/width - 1;
sphereTheta = X*pi;
spherePhi = (pi/2)*Y;
texX = cos(spherePhi).*cos(sphereTheta);
texY = sin(spherePhi);
texZ = cos(spherePhi).*sin(sphereTheta);
% Figure out which face we are facing for each co-ordinate
% First figure out the greatest absolute magnitude for each point
comp = cat(3, texX, texY, texZ);
[~,ind] = max(abs(comp), [], 3);
maxVal = zeros(size(ind));
% Copy those values - signs and all
maxVal(ind == 1) = texX(ind == 1);
maxVal(ind == 2) = texY(ind == 2);
maxVal(ind == 3) = texZ(ind == 3);
% Set each location in our equirectangular image, figure out which
% side we are facing
getFace = -1*ones(size(maxVal));
% Back
ind = abs(maxVal - texX) < 0.00001 & texX < 0;
getFace(ind) = FACE_X_POS;
% Front
ind = abs(maxVal - texX) < 0.00001 & texX >= 0;
getFace(ind) = FACE_X_NEG;
% Top
ind = abs(maxVal - texY) < 0.00001 & texY < 0;
getFace(ind) = FACE_Y_POS;
% Bottom
ind = abs(maxVal - texY) < 0.00001 & texY >= 0;
getFace(ind) = FACE_Y_NEG;
% Left
ind = abs(maxVal - texZ) < 0.00001 & texZ < 0;
getFace(ind) = FACE_Z_POS;
% Right
ind = abs(maxVal - texZ) < 0.00001 & texZ >= 0;
getFace(ind) = FACE_Z_NEG;
% Determine the co-ordinates along which image to sample
% based on which side we are facing
rawX = -1*ones(size(maxVal));
rawY = rawX;
rawZ = rawX;
% Back
ind = getFace == FACE_X_POS;
rawX(ind) = -texZ(ind);
rawY(ind) = texY(ind);
rawZ(ind) = texX(ind);
% Front
ind = getFace == FACE_X_NEG;
rawX(ind) = texZ(ind);
rawY(ind) = texY(ind);
rawZ(ind) = texX(ind);
% Top
ind = getFace == FACE_Y_POS;
rawX(ind) = texZ(ind);
rawY(ind) = texX(ind);
rawZ(ind) = texY(ind);
% Bottom
ind = getFace == FACE_Y_NEG;
rawX(ind) = texZ(ind);
rawY(ind) = -texX(ind);
rawZ(ind) = texY(ind);
% Left
ind = getFace == FACE_Z_POS;
rawX(ind) = texX(ind);
rawY(ind) = texY(ind);
rawZ(ind) = texZ(ind);
% Right
ind = getFace == FACE_Z_NEG;
rawX(ind) = -texX(ind);
rawY(ind) = texY(ind);
rawZ(ind) = texZ(ind);
% Concatenate all for later
rawCoords = cat(3, rawX, rawY, rawZ);
% Finally determine co-ordinates (normalized)
cubeCoordsX = ((rawCoords(:,:,1) ./ abs(rawCoords(:,:,3))) + 1) / 2;
cubeCoordsY = ((rawCoords(:,:,2) ./ abs(rawCoords(:,:,3))) + 1) / 2;
cubeCoords = cat(3, cubeCoordsX, cubeCoordsY);
% Now obtain where we need to sample the image
normalizedX = round(cubeCoords(:,:,1) * height);
normalizedY = round(cubeCoords(:,:,2) * height);
% Just in case.... cap between [1, height] to ensure
% no out of bounds behaviour
normalizedX(normalizedX < 1) = 1;
normalizedX(normalizedX > height) = height;
normalizedY(normalizedY < 1) = 1;
normalizedY(normalizedY > height) = height;
% Place into a stacked matrix
normalizedCoords = cat(3, normalizedX, normalizedY);
% Output image allocation
out = uint8(zeros([size(maxVal) 3]));
% Obtain column-major indices on where to sample from the
% input images
% getFace will contain which image we need to sample from
% based on the co-ordinates within the equirectangular image
ind = sub2ind([height height 6], normalizedCoords(:,:,2), ...
normalizedCoords(:,:,1), getFace);
% Do this for each channel
out(:,:,1) = imagesRed(ind);
out(:,:,2) = imagesGreen(ind);
out(:,:,3) = imagesBlue(ind);
I've also made the code publicly available through github and you can go here for it. Included is the main conversion script, a test script to show its use and a sample set of 6 cubic images pulled from Paul Bourke's website. I hope this is useful!
Project changed name to libcube2cyl. Same goodness, better working examples both in C and C++.
Now also available in C.
I happened to solve the exact same problem as you described.
I wrote this tiny C++ lib called "Cube2Cyl", you can find the detailed explanation of algorithm here: Cube2Cyl
Please find the source code from github: Cube2Cyl
It is released under MIT licence, use it for free!
So, I found a solution mixing this article on spherical coordinates from wikipedia and the Section 3.8.10 from the OpenGL 4.1 specification (plus some hacks to make it work). So, assuming that the cubic image has a height h_o and width w_o, the equirectangular will have a height h = w_o / 3 and a width w = 2 * h. Now for each pixel (x, y) 0 <= x <= w, 0 <= y <= h in the equirectangular projection, we want to find the corresponding pixel in the cubic projection, I solve it using the following code in python (I hope I didn't make mistakes while translating it from C)
import math
# from wikipedia
def spherical_coordinates(x, y):
return (math.pi*((y/h) - 0.5), 2*math.pi*x/(2*h), 1.0)
# from wikipedia
def texture_coordinates(theta, phi, rho):
return (rho * math.sin(theta) * math.cos(phi),
rho * math.sin(theta) * math.sin(phi),
rho * math.cos(theta))
FACE_X_POS = 0
FACE_X_NEG = 1
FACE_Y_POS = 2
FACE_Y_NEG = 3
FACE_Z_POS = 4
FACE_Z_NEG = 5
# from opengl specification
def get_face(x, y, z):
largest_magnitude = max(x, y, z)
if largest_magnitude - abs(x) < 0.00001:
return FACE_X_POS if x < 0 else FACE_X_NEG
elif largest_magnitude - abs(y) < 0.00001:
return FACE_Y_POS if y < 0 else FACE_Y_NEG
elif largest_magnitude - abs(z) < 0.00001:
return FACE_Z_POS if z < 0 else FACE_Z_NEG
# from opengl specification
def raw_face_coordinates(face, x, y, z):
if face == FACE_X_POS:
return (-z, -y, x)
elif face == FACE_X_NEG:
return (-z, y, -x)
elif face == FACE_Y_POS:
return (-x, -z, -y)
elif face == FACE_Y_NEG:
return (-x, z, -y)
elif face == FACE_Z_POS:
return (-x, y, -z)
elif face == FACE_Z_NEG:
return (-x, -y, z)
# computes the topmost leftmost coordinate of the face in the cube map
def face_origin_coordinates(face):
if face == FACE_X_POS:
return (2*h, h)
elif face == FACE_X_NEG:
return (0, 2*h)
elif face == FACE_Y_POS:
return (h, h)
elif face == FACE_Y_NEG:
return (h, 3*h)
elif face == FACE_Z_POS:
return (h, 0)
elif face == FACE_Z_NEG:
return (h, 2*h)
# from opengl specification
def raw_coordinates(xc, yc, ma):
return ((xc/abs(ma) + 1) / 2, (yc/abs(ma) + 1) / 2)
def normalized_coordinates(face, x, y):
face_coords = face_origin_coordinates(face)
normalized_x = int(math.floor(x * h + 0.5))
normalized_y = int(math.floor(y * h + 0.5))
# eliminates black pixels
if normalized_x == h:
--normalized_x
if normalized_y == h:
--normalized_y
return (face_coords[0] + normalized_x, face_coords[1] + normalized_y)
def find_corresponding_pixel(x, y):
spherical = spherical_coordinates(x, y)
texture_coords = texture_coordinates(spherical[0], spherical[1], spherical[2])
face = get_face(texture_coords[0], texture_coords[1], texture_coords[2])
raw_face_coords = raw_face_coordinates(face, texture_coords[0], texture_coords[1], texture_coords[2])
cube_coords = raw_coordinates(raw_face_coords[0], raw_face_coords[1], raw_face_coords[2])
# this fixes some faces being rotated 90°
if face in [FACE_X_NEG, FACE_X_POS]:
cube_coords = (cube_coords[1], cube_coords[0])
return normalized_coordinates(face, cube_coords[0], cube_coords[1])
at the end we just call find_corresponding_pixel for each pixel in the equirectangular projection
I think from your algorithm in Python you might have inverted x and y in the calculation of theta and phi.
def spherical_coordinates(x, y):
return (math.pi*((y/h) - 0.5), 2*math.pi*x/(2*h), 1.0)
from Paul Bourke's website here
theta = x pi
phi = y pi / 2
and in your code you are using y in the theta calculation and x in the phi calculation.
Correct me if I am wrong.

Resources