I am encountering unexplained behaviour when using the cv2.remap function in Opencv. I have made a function that translates the y coordinate of each pixel upwards by a number that differs on each x-coordinate. The number is calculated according to an underlying curve that starts from x=0 and ends at x=640 (for a 360 x 640 picture)
About a week ago I posted a question on the same subject but with a different problem - the top part of the image was cropped. This was answered and I understood that the problem was that several pixels from the original image were mapped to the same positions in the new image and therefore the pixels that were first mapped to the top parts were overwritten by others from the lower parts.
However the issue I am facing now is a bit different which is why I posted a new question. In this instance the problem is that since I am lifting all y coordinates upwards I would expect to have the bottom section of the image blank. This is achieved with the below piece of code
original = cv2.imread('C:\\Users\\User\\Desktop\\alaskan-landscaps3.jpg')
y_size,x_size=original.shape[:2]
map_x = np.zeros(original.shape[:2], np.float32)
map_y = np.zeros(original.shape[:2], np.float32)
for i in range(0, y_size):
for j in range(0,x_size):
map_y[i][j]=i+yvalues[j]
map_x[i][j] = j
where yvalues[j] is the y-coordinate of the curve at x-coordinate j. Sure enough, when printing the y-mappings for the first 10 rows of the map_y array at x=383 the lowest number is 199.102
for j in range(0, 360):
print('At x=383,y=' + str(j) + ' mapping of y-coordinate is:' + str(map_y[j][383]))
At x=383,y=0 mapping of y-coordinate is:199.102
At x=383,y=1 mapping of y-coordinate is:200.102
At x=383,y=2 mapping of y-coordinate is:201.102
At x=383,y=3 mapping of y-coordinate is:202.102
At x=383,y=4 mapping of y-coordinate is:203.102
At x=383,y=5 mapping of y-coordinate is:204.102
At x=383,y=6 mapping of y-coordinate is:205.102
When I use a more complicated warping function I get the below results for the same first 10 lines - the idea being that a number of consecutive pixels in the vertical direction should be mapped to the same pixel in the new image
At x=383,y=0mapping is:199.102
At x=383,y=1mapping is:199.102
At x=383,y=2mapping is:199.102
At x=383,y=3mapping is:199.102
At x=383,y=4mapping is:199.102
At x=383,y=5mapping is:200.102
At x=383,y=6mapping is:200.102
At x=383,y=7mapping is:200.102
At x=383,y=8mapping is:200.102
At x=383,y=9mapping is:201.102
At x=383,y=10mapping is:201.102
However, this time, the image is completely filled in and there is no blank space at the bottom part. You can see both images below:
(the pixelated effect is not what I intended but that's for me to fix :) ). I have checked all the entries of the specific column in the map_y array for x=383 and all values are > 199.102 so it appears that is not the same problem as in the previous case i.e there are no pixels higher up in the column that get mapped to the lower part.
So my question in, what is different between the two mappings to have caused this drastic change? I expected that since the mappings of the second image also start from a high value there would have been a blank space at the bottom as well
Apologies for the long post but I have been trying to work out the reason for a few days and came out none the wiser. Any help will be appreciated.
I think you misunderstand how remap() works.
Let me quote from the OpenCV documentation for the function initUndistortRectifyMap():
"The function actually builds the maps for the inverse mapping algorithm that is used by remap(). That is, for each pixel (u, v) in the destination image, the function computes the corresponding coordinates in the source image."
So your line
At x=383,y=0mapping is:199.102
actually means that the value of pixel (383, 0) in the mapped image is taken from pixel (383, 199.102) in the original image.
A pixel is not mapped, if:
a) the value of map_x[i][j] or map_y[i][j] is -1 (-16 in case of short map type)
or
b) the value is outside the boundaries of the source image.
Related
I have a segmented image as shown here
i want to fit a curve along the top pixels of the segmented image(show as red curve) and i want to find the top point along the curve show in blue. I have already worked on basic idea like traversing through the top to bottom and collecting the top point along each column. i want to know is there any easy solution for this problem like directly taking out the boundary pixels and find the top point.I am using MATLAB for this problem
%download the image
img = logical(imread('http://i.stack.imgur.com/or2iX.png'));
%for some reason it appeared RGB with big solid borders.
%to monochrome
img = img(:,:,1);
%remove borders
img = img(~all(img,2), ~all(img,1));
%split into columns
cimg = num2cell(img,1);
%find first nonzero element per column
ridx = cellfun(#(x) find(x,1,'first'), cimg);
figure, imshow(img)
hold on
%image dim1 is Y, dim2 is X
plot(1:size(img,2),ridx-1,'r','linewidth',2)
%find top point
[yval, xval] = min(ridx);
If you want a smoother curve, try polyfit/polyval
#EDIT
If we want the line to have break at gaps between connected components, we should change the code to something like
bord_idx = sub2ind(size(img), ridx, 1:size(img,2));
regs=regionprops(bwlabel(img),'pixelidxlist');
regs_idx = struct2cell(regs);
split_step = cellfun(#(x) sum(ismember(bord_idx,x)), regs_idx);
split_step = split_step(split_step>0);
split_yvals = mat2cell(ridx',split_val);
split_xvals = mat2cell([1:size(img,2)]',split_val);
figure, imshow(img)
hold on
for k = 1:length(split_step),
plot(split_xvals{k}, split_yvals{k}, 'r', 'linewidth', 2),
end
However, the result is not ideal if one region is positioned over the other. If the "shadowed" points are needed, you should try bwtraceboundary or convexhull and find where the border turns down
As far as "simplest matlab solution" by which I think you mean built in matlab functions: imclose()->edge()->bwboundaries()->findpeaks()'on each boundary'->'filter results based on width and magnitude of peaks'. *you will need to tune all the parameters in these functions, I am just listing what would get you there if appropriately applied.
As far as processing speed is concerned, I think I would have done exactly what you did, basically collecting the top edge from a top down column search and then looking for the point of highest inflection. As soon as you start doing processing of any type, you start doing several operations per pixel which will quickly become more expensive than your initial search (just requires that your image and target are simple enough)
That being said, here are some ideas that may help:
1:If you run a sufficiently heavy closing (dilate->erode), that should fill in all that garbage at the bottom.
2: If you know that your point of interest is not at left or right of picture (boundaries), you could take the right and left edge points and calculate a slope to be applied as an offset to flatten the whole image.
3: If your image always has the large dark linear region below the peak as seen here, you could locate those edges with houghlines looking for verticals and then search only the columns between them.
4: If speed is a concern, you could do a more sophisticated search pattern than left to right, as your peak has a pretty good distribution around it which could help with faster localization of maxima.
I need to match a path recorded by lidar (x/y/height) onto a map tile (pixel height field)
I can assume the problem is 2.5D (ie a unique height for each point, no caverns) and the region is small enough that the grid is uniform (don't need to consider curvature). Naturally the track data is noisy and I don't have any known locations in advance.
Rather than do a full 3D point based Iterative Closest Point are there any simple algorithms for purely surface path matching I should take a look at ?
Specifically it seems to be an image processing problem(x,y,height=intensity) so some sort of snake matching algorithm?
Instead of brute force (calculate error for full path from each starting point) you can only expand the best points and potentially save a lot of work.
Normalize:
Subtract (pathMinX, pathMinY) from all points in the path.
Subtract (gridMinX, gridMinY) from all points in the grid.
Find pathMaxX, pathMaxY, gridMaxX, gridMaxY.
deltaMaxX = gridMaxX - pathMaxX, deltaMaxY = gridMaxY - pathMaxY.
Create an array or list with (deltaMaxX + 1) * (deltaMaxY + 1) nodes for all the combinations of deltaX and deltaY. Each node has to hold the following information:
index, initialize with 0
deltaX and deltaY, initialize with the loop counters
error, initialize with (path[0].height - grid[path[0].x + deltaX][path[0].y + deltaY].height)^2
Sort the array by error.
While arr[0].error <= arr[1].error:
arr[0].index++
if arr[0].index == n: return (arr[0].deltaX, arr[0].deltaY)
arr[0].error += (path[arr[0].index].height - grid[path[arr[0].index].x + arr[0].deltaX][path[arr[0].index].y + arr[0].deltaY].height)^2
Remove node 0 and insert it at the correct position, so that the array becomes sorted again.
Repeat step 6. and 7. until a solution was returned.
You can (should, I think it's worth it) further improve the algorithm if you first sort the points in the path by extreme heights (lowest and highest points first and then moving towards the average height in regard to the grid, e.g. sort by abs(height - averageGridHeight) descending). Like this large errors are produced early and therefore branches can be cut a lot earlier.
Purely, theoretical, but what if you encode the 3D data as grayscale images (3D path as grayscale image, height map is already something like that probably, just need to ensure scales make sense).
If you have the 3D path as a grayscale image and the height map as a grayscale image perhaps you could do a needle in haystack search using computer vision techniques. For example in, OpenCV there are a couple of techniques for finding a subimage in a larger image:
Template Matching - (overly simplified) uses a sliding window doing pixel comparison
Chamfer Matching - making more use of edges - probably more suitable for your goal. bare in mind I ran into an allocation bug last time using this: it does work in the end, but needs a bit of love (doing malloc fixes and keeping track of the cost to get read of false positives) but there are options to handle the scale difference
Template matching examples:
Chamfer matching example:
I have a greyscale image similar to the one below that I have achieved after some post-processing steps (image 0001). I would like a vector corresponding to the bottom of the lower bright strip (as depicted in image 0001b). I can use im2bw with various thresholds to achieve the vectors in image 0002 (the higher the threshold value the higher the tendency for the vector line to blip upwards, the lower the threshold the higher the tendency for the line to blip downwards)..and then I was thinking of going through each vector and measuring arclength over some increment (maybe 100 pixels or so) and choosing that vector with the lowest arclength...and adding that 100 pixel stretch to the final vector, creating a frankenstein-like vector using the straightest segments from each of the thresholded vectors.. I should also mention that when there are multiple straightish/parallel vectors, the top one is the best fit.
First off, is there some better strategy I should be employing here to find that line on image 0001? (this needs to be fast so some long fitting code wouldn't work). If my current Frankenstein's monster solution works, any suggestions as to how to best go about this?
Thanks in advance
image=im2bw(image,0.95); %or 0.85, 0.75, 0.65, 0.55
vec=[];
for v=1:x
for x=1:z
if image(c,v)==1
vec(v)=c;
end
end
end
vec=fastsmooth(vec,60,20,1);
Here is the modified version of what I originally did. It works well on on your images. If you want subpixel resolution, you can implement an active contour model with some fitting function.
files = dir('*.png');
filenames = {files.name};
for ifile=1:length(filenames)
%%
% read image
im0 = double(imread(filenames{ifile}));
%%
% remove background by substracting a convolution with a mask
lobj=100;
convmask = ones(lobj,1)/lobj;
im=im0-conv2(im0,convmask,'same');
im(im<0)=0;
imagesc(im);colormap gray;axis image;
%%
% use canny edge filter, alowing extremely weak edge to exist
bw=edge(im,'canny',[0.01,0.3]);
% use close operation on image to close gaps between lines
% the kernel is a flat rectangular so that it helps to connect horizontal
% gaps
se=strel('rectangle',[10,30]);
bw=imdilate(bw,se);
% thin the lines to be single pixel line
bw=bwmorph(bw,'thin',inf);
% connect H bridge
bw=bwmorph(bw,'bridge');
imagesc(bw);colormap gray;axis image;
%% smooth the image, find the decreasing region, and apply the mask
imtmp = imgaussfilt(im0,3);
imtmp = diff(imtmp);
imtmp = [imtmp(1,:);imtmp];
intensity_decrease_mask = imtmp < 0;
bw = bw & intensity_decrease_mask;
imagesc(bw);colormap gray;axis image;
%%
% find properties of the lines, and find the longest lines
cc=regionprops(bw,'Area','PixelList','Centroid','MajorAxisLength','PixelIdxList');
% now select any lines that is larger than eighth of the image width
cc=cc([cc.MajorAxisLength]>size(bw,2)/8);
%%
% select lines that has average intensity larger than gray level
for i=1:length(cc)
cc(i).meanIntensity = mean(im0(sub2ind(size(im0),cc(i).PixelList(:,2), ...
cc(i).PixelList(:,1) )));
end
cc=cc([cc.meanIntensity]>150);
cnts=reshape([cc.Centroid],2,length(cc))';
%%
% calculate the minimum distance to the bottom right of each edge
for i=1:length(cc)
cc(i).distance2bottomright = sqrt(min((cc(i).PixelList(:,2)-size(im,1)).^2 ...
+ (cc(i).PixelList(:,1)-size(im,2)).^2));
end
% select the bottom edge
[~,minindex]=min([cc.distance2bottomright]);
bottomedge = cc(minindex);
%% clean up the lines a little bit
bwtmp = false(size(bw));
bwtmp(bottomedge.PixelIdxList)=1;
% find the end points to the most left and right
endpoints = bwmorph(bwtmp, 'endpoints');
[endy,endx] = find(endpoints);
[~,minind]=min(endx);
[~,maxind]=max(endx);
pos_most_left = [endx(minind),endy(minind)];
pos_most_right = [endx(maxind),endy(maxind)];
% select the shortest path between left and right
dists = bwdistgeodesic(bwtmp,pos_most_left(1),pos_most_left(2)) + ...
bwdistgeodesic(bwtmp,pos_most_right(1),pos_most_right(2));
dists(isnan(dists))=inf;
bwtmp = imregionalmin(dists);
bottomedge=regionprops(bwtmp,'PixelList');
%% plot the lines
imagesc(im0);colormap gray;axis image;hold on;axis off;
for i=1:length(cc)
plot(cc(i).PixelList(:,1),cc(i).PixelList(:,2),'b','linewidth',2);hold on;
end
plot(bottomedge.PixelList(:,1),bottomedge.PixelList(:,2),'r','linewidth',2);hold on;
print(gcf,num2str(ifile),'-djpeg');
% pause
end
I am not sure this answers your question directly, but I have a lot of experiencing fitting arrays (or matrices in my case) to 3D raster images. We were using relatively low power machines (standard i7 processors 32 gb ram), and had to perform the fitting very quickly (<30 seconds). We also had to validate the fit with a variety of parameters (and again these were 3D rasters fit to a point cloud matrix).
Anyways, the process we used was the fminsearch function internal to Matlab. Documentation can be found here: http://www.mathworks.com/help/optim/functionlist.html
We would start with a plain point-cloud and perform successive manipulations on a per pixel basis to adjust the point-cloud to the raster. Essentially walking through each pixel in the raster to produce the lowest offset between the point cloud and the raster.
I will try to search for some code this afternoon and update my answer, but I might explore this option for your case. I would imagine you could fit a curve to certain pixels (e.g. white pixels) both rapidly and accurately by setting up an optimization function.
I also could help more if I understood your objective better. Are you just trying to fit a line to the high-albedo/white areas?
In the way of example: I can fit a 3D point cloud to the following image by starting with a standard point cloud, the 3D raster, and a minimization function (in this case just RMS error of each individual point in the z axis). Throw an fmin function on there and in a few seconds you get a modified point cloud that fits much better than the standard.
On a shape from a logical image, I am trying to extract the field of view from any point inside the shape on matlab :
I tried something involving to test each line going through the point but it is really really long.(I hope to do it for each points of the shape or at least each point of it's contour wich is quite a few times)
I think a faster method would be working iteratively by the expansion of a disk from the considered point but I am not sure how to do it.
How can I find this field of view in an efficient way?
Any ideas or solution would be appreciated, thanks.
Here is a possible approach (the principle behind the function I wrote, available on Matlab Central):
I created this test image and an arbitrary point of view:
testscene=zeros(500);
testscene(80:120,80:120)=1;
testscene(200:250,400:450)=1;
testscene(380:450,200:270)=1;
viewpoint=[250, 300];
imsize=size(testscene); % checks the size of the image
It looks like this (the circle marks the view point I chose):
The next line computes the longest distance to the edge of the image from the viewpoint:
maxdist=max([norm(viewpoint), norm(viewpoint-[1 imsize(2)]), norm(viewpoint-[imsize(1) 1]), norm(viewpoint-imsize)]);
angles=1:360; % use smaller increment to increase resolution
Then generate a set of points uniformly distributed around the viewpoint.:
endpoints=bsxfun(#plus, maxdist*[cosd(angles)' sind(angles)'], viewpoint);
for k=1:numel(angles)
[CX,CY,C] = improfile(testscene,[viewpoint(1), endpoints(k,1)],[viewpoint(2), endpoints(k,2)]);
idx=find(C);
intersec(k,:)=[CX(idx(1)), CY(idx(1))];
end
What this does is drawing lines from the view point to each directions specified in the array angles and look for the position of the intersection with an obstacle or the edge of the image.
This should help visualizing the process:
Finally, let's use the built-in roipoly function to create a binary mask from a set of coordinates:
FieldofView = roipoly(testscene,intersec(:,1),intersec(:,2));
Here is how it looks like (obstacles in white, visible field in gray, viewpoint in red):
I have an image that I used to analyze in LabView using a method called Rake. Basically, what that method does is it finds all the significant edges along parallel lines on an image.
http://zone.ni.com/reference/en-XX/help/370281P-01/imaqvision/imaq_rake_3/ (as seen on the last image at the bottom of the link).
The beauty of this function is that it will give you all edge points that are larger than a certain edge strength, and each edge will only generate one edge point (thickness of the edge line is 1 pixel)
I want to use OpenCV to do something similar. The way I could imagine for doing this is
- deconstructing the Canny operator with a filter of my choice,
- hysterisis thresholding of the edge values with two thresholds
- followed by nonmaxima suppression
- read the pixels along that line and mark all pixels that are larger than my threshold
the problem is that the canny comes as a bundle and I cant find the nonmaxima suppression function by itself.
Does anybody know of a way to do something similar to the operation I've described?
Thanks
Not sure if I understand this question fully, but about the unbundled non-maximum suppression part:
One simple way for 2d non-maximum suppression is this:
dilate the image. Dilation in OpenCV sets the value of each pixel to the max() of the local neighborhood. Repeat a few times or use a larger kernel to get the desired radius.
Then compare the dilated image with the original and set all pixels with differing values to zero.
The remaining pixels are local maxima.
# some code I once used in OpenCV/Python
# given an image, sets all pixels to zero, unless they are local maxima
def supressNonMaxima(img):
localMax = cvCreateImage (cvGetSize(img), IPL_DEPTH_16U, 1)
cvDilate(img, localMax, null, 3) # max() with radius of 3
mask = cvCreateImage( cvGetSize(img), 8, 1)
cvCmp(img, localMax, mask, CV_CMP_LT)
cvSet(img,0,mask)