Ho can I imply in python a transformation with a centralization like imtransform in matlab (see it's exact semantics, it is acutely relevant).
For example in matlab:
for this tform:
tform = maketform('affine',[1 0 0; -1 1 0; 0 0 1]);
I get:
and in python in a big variety of affine transformation methods (piilow, opencv, skimage and e.t.c) I get it non-centralized and cut:
How can I choose my 3*3 matrix of the tform for python libraries, such that it will centralize the image after such skewing ?
MATLAB default behavior is expanding and centralizing the output image, but this behavior is unique to MATLAB.
There might be some Python equivalent that I am not aware of, but I would like to focus on OpenCV solution.
In OpenCV, you need to compute the coefficients of the transformation matrix, and compute the size of the output image in order to get the same result as in MATLAB.
Consider the following implementation details:
In OpenCV, the transformation matrix is transposed relative to MATLAB (different convention).
In Python the first index is [0, 0] and in MATLAB [1, 1].
You need to compute the dimensions (width and height) of the output image from advance.
You need the output dimensions to include the entire transformed image (all the corners of the transformed image should enter the output image).
My suggestion is transforming the four corners, and compute max_x - min_x and max_y - min_y of the transformed corners.
For centralizing the output, you need to compute the translation coefficients (last column in OpenCV matrix).
Assume: Source center is transformed (shifted) to destination center.
For computing the translation, you may use inverse transformation, and compute the translation (shift in pixels) from the source center to destination center.
Here is a Python code sample (using OpenCV):
import numpy as np
import cv2
# Read input image
src_im = cv2.imread('peppers.png')
# Build a transformation matrix (the transformation matrix is transposed relative to MATLAB)
t = np.float32([[1, -1, 0],
[0, 1, 0],
[0, 0, 1]])
# Use only first two rows (affine transformation assumes last row is [0, 0, 1])
#trans = np.float32([[1, -1, 0],
# [0, 1, 0]])
trans = t[0:2, :]
inv_t = np.linalg.inv(t)
inv_trans = inv_t[0:2, :]
# get the sizes
h, w = src_im.shape[:2]
# Transfrom the 4 corners of the input image
src_pts = np.float32([[0, 0], [w-1, 0], [0, h-1], [w-1, h-1]]) # https://stackoverflow.com/questions/44378098/trouble-getting-cv-transform-to-work (see comment).
dst_pts = cv2.transform(np.array([src_pts]), trans)[0]
min_x, max_x = np.min(dst_pts[:, 0]), np.max(dst_pts[:, 0])
min_y, max_y = np.min(dst_pts[:, 1]), np.max(dst_pts[:, 1])
# Destination matrix width and height
dst_w = int(max_x - min_x + 1) # 895
dst_h = int(max_y - min_y + 1) # 384
# Inverse transform the center of destination image, for getting the coordinate on the source image.
dst_center = np.float32([[(dst_w-1.0)/2, (dst_h-1.0)/2]])
src_projected_center = cv2.transform(np.array([dst_center]), inv_trans)[0]
# Compute the translation of the center - assume source center goes to destination center
translation = src_projected_center - np.float32([[(w-1.0)/2, (h-1.0)/2]])
# Place the translation in the third column of trans
trans[:, 2] = translation
# Transform
dst_im = cv2.warpAffine(src_im, trans, (dst_w, dst_h))
# Show dst_im as output
cv2.imshow('dst_im', dst_im)
cv2.waitKey()
cv2.destroyAllWindows()
# Store output for testing
cv2.imwrite('dst_im.png', dst_im)
MATLAB code for comparing results:
I = imread('peppers.png');
tform = maketform('affine',[1 0 0; -1 1 0; 0 0 1]);
J = imtransform(I, tform);
figure;imshow(J)
% MATLAB recommends using affine2d and imwarp instead of maketform and imtransform.
% tform = affine2d([1 0 0; -1 1 0; 0 0 1]);
% J = imwarp(I, tform);
% figure;imshow(J)
pyJ = imread('dst_im.png');
figure;imagesc(double(rgb2gray(J)) - double(rgb2gray(pyJ)));
title('MATLAB - Python Diff');impixelinfo
max_abs_diff = max(imabsdiff(J(:), pyJ(:)));
disp(['max_abs_diff = ', num2str(max_abs_diff)])
We are lucky to get zero difference - result of imwarp in MATLAB gives minor differences (but imtransform result is same as OpenCV).
Python output image (same as MATLAB output image):
Related
I created a numpy array by calculating the density of dwellings within an area through the following code:
def myplot(x, y, z, s, bins=10000):
heatmap, xedges, yedges = np.histogram2d(x, y, bins=bins, weights=z)
heatmap = gaussian_filter(heatmap, sigma=s)
extent = [xedges[0], xedges[-1], yedges[0], yedges[-1]]
return heatmap.T, extent
fig, axs = plt.subplots(2, 2)
# Generate some test data
x = buildings["x"]
y = buildings["y"]
weights = buildings["Area"]
sigmas = [0, 16, 32, 64]
for ax, s in zip(axs.flatten(), sigmas):
if s == 0:
ax.plot(x, y, weights, 'k.', markersize=5)
ax.set_title("Scatter plot")
else:
img, extent = myplot(x, y, weights, s)
ax.imshow(img, extent=extent, origin='lower', cmap=cm.jet)
ax.set_title("Smoothing with $\sigma$ = %d" % s)
plt.savefig('export_'+str(s)+'.png', dpi=150, bbox_inches='tight')
plt.show()
This is the result and works fine:
enter image description here
Now I need to save it as a geotif and I know the extreme coordinates of the box angles. I tried to do that using the following code:
# create a georeferenced box
transform = from_bounds(extent[0], extent[1],extent[2], extent[3], 10000, 10000)
# save the georeferenced tif
with rio.open('data.tif', 'w', driver='GTiff', height=10000, width=10000, count=1, dtype='float64', nodata=0, crs=32632, transform=transform) as dst:
dst.write(img, 1)
The problem is that the result is transpose and not on the right position. Could you help me to find the solution?
I tried to develop the code but did not work
You should simply use numpy.transpose on your array - it is a very fast operation that does not copy the array.
GDAL uses traditional C style raster coordinates. In numpy an array with shape (x, y) is x lines of y pixels, while in GDAL it is the other way around.
# save the georeferenced tif
with rio.open('data.tif', 'w', driver='GTiff', height=10000, width=10000, count=1, dtype='float64', nodata=0, crs=32632, transform=transform) as dst:
dst.write(img.tranpose(), 1)
I am using python open cv for image restoration and in the first step, I have to add gaussian noise to my "binary" image. My image pixel values are integers that can take values 0 or 1. How can I add gaussian noise to my image knowing SNR or noise variance?
I came up with this piece of code which adds 15 percent noise to my image but I don't know if this noise is normal Gaussian and how I can find its variance and SNR.
def add_noise(im):
im_noisy = im.copy()
for i in range(len(im_noisy)):
for j in range(len(im_noisy[0])):
r = np.random.rand()
if r < 0.15:
im_noisy[i][j] = -im_noisy[i][j]+1 #(converts 0 to 1 and vice versa)
return im_noisy
Since the image is binary, the solution is not well defined - we have to select a threshold that above the threshold the noise is 1.
The Gaussian noise supposes to be symmetric, so pixels below the minus threshold are going to be -1.
We may define the threshold by number of sigma's (for example: above 2 sigma, the noise is 1, and below -2 sigma the noise is -1).
When selecting threshold of 2 sigma, statistically we expect that about 2.5% of total pixels are going to be 1 and 2.5% are going to be -1.
Select lower threshold for higher percentage.
Create random normal (Gaussian) distribution image with mean=0 and sigma=1:
sigma = 1
gauss = np.random.normal(0, sigma, im.shape) # Create random normal (Gaussian) distribution image with mean=0 and sigma=1.
Convert to values to -1, 0, 1 - assume pixels value above 2 sigma are "1", below -2 sigma are -1 and other are "0" (2 sigma is an example, we may select other value):
binary_gauss = (gauss > 2*sigma).astype(np.int8)
binary_gauss[gauss < -2*sigma] = -1
After adding binary_gauss to im, clip the result to range [0, 1]:
noisey_im = (im + binary_gauss).clip(0, 1)
Code sample (first part reads a sample image, and convert to binary):
import numpy as np
import cv2 # Used only for testing
im = cv2.imread('chelsea.png', cv2.IMREAD_GRAYSCALE) # Read input image (for testing).
im = cv2.threshold(im, 0, 1, cv2.THRESH_OTSU)[1] # Convert image to binary (for testing).
im = im.astype(np.int8) # Convert to type int8 (8 bits singed)
sigma = 1
gauss = np.random.normal(0, sigma, im.shape) # Create random normal (Gaussian) distribution image with mean=0 and sigma=1.
binary_gauss = (gauss > 2*sigma).astype(np.int8) # Convert to binary - assume pixels with value above 2 sigmas are "1".
binary_gauss[gauss < -2*sigma] = -1 # Set all pixels below 2 sigma to "-1".
noisey_im = (im + binary_gauss).clip(0, 1) # Add noise image, and clip the result ot [0, 1].
noisey_im = noisey_im.astype(np.uint8) # Convert to type uint8
# Show images (for testing).
cv2.imshow('im', im*255)
cv2.imshow('binary_gauss', (binary_gauss+1).astype(np.uint8)*127)
cv2.imshow('noisey_im', noisey_im*255)
cv2.waitKey()
cv2.destroyAllWindows()
im (input image after converting to binary):
binary_gauss (noise image after threshold - values are -1, 0, 1):
noisey_im (im + binary_gauss after threshold):
I made an algorithm to measure an object using a reference, like this:
The reference is the frame and the other (AOL) is the desired object. My code obtained this result:
But the real AOL is 78.6. This is because of the perspective/angle of photograph. So I used in my code Deep Learning and I obtained the the reference and AOL mask, and I made a simple calculation based on the number of pixels for each mask to obtain AOL area (cm²), once I know the actual size of the reference. I tried to correct the angle/perpective based on the reference and I used the reference mask:
I tried to calculate quadrangle vertices based on the reference mask to correct the perspective. I created this code based on this reference Perspective correction in OpenCV using python:
# import the necessary packages
from scipy.spatial import distance as dist
from imutils import perspective
from imutils import contours
import numpy as np
import imutils
import cv2
import math
import matplotlib.pyplot as plt
# get the single external contours
# load the image, convert it to grayscale, and blur it slightly
image = cv2.imread("./ref/20210702_114527.png") ## Mask Image
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
gray = cv2.GaussianBlur(gray, (7, 7), 0)
# perform edge detection, then perform a dilation + erosion to
# close gaps in between object edges
edged = cv2.Canny(gray, 50, 100)
edged = cv2.dilate(edged, None, iterations=1)
edged = cv2.erode(edged, None, iterations=1)
# find contours in the edge map
cnts = cv2.findContours(edged.copy(), cv2.RETR_EXTERNAL,
cv2.CHAIN_APPROX_SIMPLE)
cnts = imutils.grab_contours(cnts)
# sort the contours from left-to-right and initialize the
# 'pixels per metric' calibration variable
(cnts, _) = contours.sort_contours(cnts)
pixelsPerMetric = None
orig = image.copy()
box = cv2.minAreaRect(min(cnts, key=cv2.contourArea))
box = cv2.cv.BoxPoints(box) if imutils.is_cv2() else cv2.boxPoints(box)
box = np.array(box, dtype="int")
# order the points in the contour such that they appear
# in top-left, top-right, bottom-right, and bottom-left
# order, then draw the outline of the rotated bounding
# box
box = perspective.order_points(box)
cv2.drawContours(orig, [box.astype("int")], -1, (0, 255, 0), 2)
# loop over the original points and draw them
for (x, y) in box:
cv2.circle(orig, (int(x), int(y)), 5, (0, 0, 255), -1)
print('Box: ',box)
cv2.imshow('Orig', orig)
img = cv2.imread("./meat/sair/20210702_114527.jpg") #original image
rows,cols,ch = img.shape
#pts1 = np.float32([[185,9],[304,80],[290, 134],[163,64]]) #ficou legal 6e.jpg
### Coletando os pontos
pts1 = np.float32(box)
### Draw the vertices on the original image
for (x, y) in pts1:
cv2.circle(img, (int(x), int(y)), 5, (0, 0, 255), -1)
ratio= 1.6
moldH=math.sqrt((pts1[2][0]-pts1[1][0])*(pts1[2][0]-pts1[1][0])+(pts1[2][1]-pts1[1][1])*(pts1[2][1]-pts1[1][1]))
moldW=ratio*moldH
pts2 = np.float32([[pts1[0][0],pts1[0][1]], [pts1[0][0]+moldW, pts1[0][1]], [pts1[0][0]+moldW, pts1[0][1]+moldH], [pts1[0][0], pts1[0][1]+moldH]])
#print('cardH: ',cardH,cardW)
M = cv2.getPerspectiveTransform(pts1,pts2)
print('M:', M)
print('pts1:', pts1)
print('pts2:', pts2)
offsetSize= 320
transformed = np.zeros((int(moldW+offsetSize), int(moldH+offsetSize)), dtype=np.uint8)
dst = cv2.warpPerspective(img, M, transformed.shape)
plt.subplot(121),plt.imshow(img),plt.title('Input')
plt.subplot(122),plt.imshow(dst),plt.title('Output')
plt.show()
And I got this:
No perspective correction. I have a lot of information like vertices, the correct size of the reference. Is it possible to do a mathematical correction based on quadrangle vertices, like a regression? Not necessarily correcting the image directly, unless there is a good method to correct the perspective image. Or maybe a different approach based on math? Thanks for your patience.
For Christoph:
This is the result position too:
pts1: [[ 9. 51.]
[392. 56.]
[388. 336.]
[ 5. 331.]]
I am trying to calculate the pose of image Y, given image X. Image Y is the same as image X rotated 90º degrees.
1 -So, for starters i find the matches between both images.
2 -Then i store all the good matches.
3 -The homography between the the matches from both images is calculated using cv2.RANSAC.
4 -Then for the X image, i transform the 2d matching points into 3d, adding 0 as the Z axis.
5 -Object points contain all points from matches of original image, while image points contain matches from the training image. Both array of points are filtered using the mask returned by homography.
6 -After that, i use cv2.calibrateCamera with these object points and image points.
7 -Finnaly i use cv2.projectPoints to get the projections of the axis
I know with that until step 5, the results are correct because i use cv2.drawMatches to see the matches. However this may not be the way to get what i want to achieve.
matches = flann.knnMatch(query_image.descriptors, descriptors, k=2)
good = []
for m, n in matches:
if m.distance < 0.70 * n.distance:
good.append(m)
current_good = good
src_pts = np.float32([selected_image.keypoints[m.queryIdx].pt for m in current_good]).reshape(-1, 1, 2)
dst_pts = np.float32([keypoints[m.trainIdx].pt for m in current_good]).reshape(-1, 1, 2)
homography, mask = cv2.findHomography(src_pts, dst_pts, cv2.RANSAC, 5.0)
test = np.zeros(((mask.ravel() > 0).sum(), 3),np.float32) #obj points
test1 = np.zeros(((mask.ravel() > 0).sum(), 2), np.float32) #img points
i=0
counter=0
for m in current_good:
if mask.ravel()[i] == 1:
test[counter][0] = selected_image.keypoints[m.queryIdx].pt[0]
test[counter][1] = selected_image.keypoints[m.queryIdx].pt[1]
test1[counter][0] = selected_image.keypoints[m.trainIdx].pt[0]
test1[counter][1] = selected_image.keypoints[m.trainIdx].pt[1]
counter+=1
i+=1
gray = cv2.cvtColor(self.train_image, cv2.COLOR_BGR2GRAY)
gray = np.float32(gray)
#here start my doubts about what i want to do and if it is possible to do it this way
ret, mtx, dist, rvecs, tvecs = cv2.calibrateCamera([test], [test1], gray.shape[::-1], None, None)
axis = np.float32([[3, 0, 0], [0, 3, 0], [0, 0, -3]]).reshape(-1, 3)
rvecs = np.array(rvecs, np.float32)
tvecs = np.array(tvecs, np.float32)
imgpts, jac = cv2.projectPoints(axis, rvecs, tvecs, mtx, dist)
However after all this, imgpts returned by cv2.projectPoints give results that don't make much sense to me, like :
[[[857.3185 109.317406]]
[[857.2196 108.360954]]
[[857.2846 107.579605]]]
I would like to have a normal to my image like it is shown here https://docs.opencv.org/trunk/d7/d53/tutorial_py_pose.html and i successfully got it to work using the chessboard image. But trying to adapt to a general image is giving me strange results.
Suppose i would like to draw an image like the following:
Where the pixel values are refined to 0 for black and white for 1.
These line are drawn with specific radius and angles
Now I create a 80 x 160 matrix
texturematrix = zeros(80,160);
then i want to change particular elements to be 1 according to the lines conditions
but how do i make them repeatedly with specific distance apart from each others effectively?
Thanks a lot everyone!
This might not be what you are looking for, but generating such an image could be done by plotting a set of lines, as follows:
% grid sizes
m = 6;
n = 5;
% line length and angle
len = 1;
theta = .1*pi;
[a,b] = meshgrid(1:m,1:n);
x = reshape([a(:),a(:)+len*cos(theta),nan(numel(a),1)]',[],1);
y = reshape([b(:),b(:)+len*sin(theta),nan(numel(b),1)]',[],1);
h = figure();
plot(x,y,'k', 'LineWidth', 2);
But this has nothing to do with a texture matrix. So, we construct a matrix of desired size:
set(gca, 'position',[0 0 1 1], 'units','normalized', 'YTick',[], 'XTick',[]);
frame = frame2im(getframe(h),[0 0 1 1]);
im = imresize(frame,[80 160]);
M = ~(im(2:end,2:end,1)==255);