I'm trying to blend two images with OpenCV in the most efficient way. Currently, I have this:
// Input matrices to mix
cv::Mat A(w, h, CV_8UC3);
cv::Mat B(w, h, CV_8UC3);
// Mix factor
cv::Mat alpha(w, h, CV_8UC1);
// Have to multiply alpha channel for mul() function
std::vector<cv::Mat> array{alpha, alpha, alpha};
cv::Mat alpha_multichannel;
cv::merge(array, alpha_multichannel);
cv::Mat result = A.mul(alpha_multichannel, 1./255) + B.mul(cv::Scalar(255, 255, 255) - alpha_multichannel, 1./255);
Currently this loops at least four times over the image (for the alpha_multichannel image, for A.mul, for B.mul and for the sum), although with a custom loop it could be done in one loop.
Is there a better way to do this?
Related
I am currently applying the Contrast Limited Adaptive Histogram Equalization algorithm together with an algorithm to perform the photo denoise.
My problem is that I am working with 360 photos. As the contrast generates different values at the edges when I join the photo, the edge line is highly noticeable. How can I mitigate that line? What changes should I make so that it is not noticeable and the algorithm is applied consistently?
Original Photo:
Code to Contrast Limited Adaptive Histogram Equalization
# CLAHE (Contrast Limited Adaptive Histogram Equalization)
clahe = cv2.createCLAHE(clipLimit=1., tileGridSize=(6, 6))
lab = cv2.cvtColor(image, cv2.COLOR_BGR2LAB) # convert from BGR to LAB color space
l, a, b = cv2.split(lab) # split on 3 different channels
l2 = clahe.apply(l) # apply CLAHE to the L-channel
lab = cv2.merge((l2, a, b)) # merge channels
img2 = cv2.cvtColor(lab, cv2.COLOR_LAB2BGR) # convert from LAB to BGR
Result:
360 performed:
It is highly notorious line of separation because it is not taken into account that the photo is joined later. What can I do?
Here's an answer for C++, you can probably convert it easily to python/numpy.
The idea is to use a border region before performing CLAHE and crop the image afterwards.
These are the subimage regions in the original image:
and they will be copied the the left/right of the image like this:
Maybe you can reduce the size of the border strongly:
int main()
{
cv::Mat img = cv::imread("C:/data/SO_360.jpg");
int borderSize = img.cols / 4;
// make image that can have some border region
cv::Mat borderImage = cv::Mat(cv::Size(img.cols + 2 * borderSize, img.rows), img.type());
// posX, posY, width, height of the subimages
cv::Rect leftBorderRegion = cv::Rect(0, 0, borderSize, borderImage.rows);
cv::Rect rightBorderRegion = cv::Rect(borderImage.cols - borderSize, 0, borderSize, borderImage.rows);
cv::Rect imgRegion = cv::Rect(borderSize, 0, img.cols, borderImage.rows);
// original image regions to copy:
cv::Rect left = cv::Rect(0, 0, borderSize, borderImage.rows);
cv::Rect right = cv::Rect(img.cols - borderSize, 0, borderSize, img.rows);
cv::Rect full = cv::Rect(0, 0, img.cols, img.rows);
// perform copying to subimage (left part of the img goes to right part of the border image):
img(left).copyTo(borderImage(rightBorderRegion));
img(right).copyTo(borderImage(leftBorderRegion));
img.copyTo(borderImage(imgRegion));
cv::imwrite("SO_360_border.jpg", borderImage);
//# CLAHE(Contrast Limited Adaptive Histogram Equalization)
//clahe = cv2.createCLAHE(clipLimit = 1., tileGridSize = (6, 6))
// apply the CLAHE algorithm to the L channel
cv::Ptr<cv::CLAHE> clahe = cv::createCLAHE();
clahe->setClipLimit(1);
clahe->setTilesGridSize(cv::Size(6, 6));
cv::Mat lab;
cv::cvtColor(borderImage, lab, cv::COLOR_BGR2Lab); // # convert from BGR to LAB color space
std::vector<cv::Mat> labChannels; //l, a, b = cv2.split(lab) # split on 3 different channels
cv::split(lab, labChannels);
//l2 = clahe.apply(l) # apply CLAHE to the L - channel
cv::Mat dst;
clahe->apply(labChannels[0], dst);
labChannels[0] = dst;
//lab = cv2.merge((l2, a, b)) # merge channels
cv::merge(labChannels, lab);
//img2 = cv2.cvtColor(lab, cv2.COLOR_LAB2BGR) # convert from LAB to BGR
cv::cvtColor(lab, dst, cv::COLOR_Lab2BGR);
cv::imwrite("SO_360_border_clahe.jpg", dst);
// to crop the image after performing clahe:
cv::Mat cropped = dst(imgRegion).clone();
cv::imwrite("SO_360_clahe.jpg", cropped);
}
Images:
input as in your original post.
After creating the border:
After performing CLAHE (with border):
After cropping the CLAHE-border-image:
I have a total of two textures, the first is used as a framebuffer to work with inside a computeshader, which is later blitted using BlitFramebuffer(...). The second is supposed to be an OpenGL array texture, which is used to look up textures and copy them onto the framebuffer. It's created in the following way:
var texarray uint32
gl.GenTextures(1, &texarray)
gl.ActiveTexture(gl.TEXTURE0 + 1)
gl.BindTexture(gl.TEXTURE_2D_ARRAY, texarray)
gl.TexParameteri(gl.TEXTURE_2D_ARRAY, gl.TEXTURE_MIN_FILTER, gl.LINEAR)
gl.TexImage3D(
gl.TEXTURE_2D_ARRAY,
0,
gl.RGBA8,
16,
16,
22*48,
0,
gl.RGBA, gl.UNSIGNED_BYTE,
gl.Ptr(sheet.Pix))
gl.BindImageTexture(1, texarray, 0, false, 0, gl.READ_ONLY, gl.RGBA8)
sheet.Pix is just the pixel array of an image loaded as a *image.NRGBA
The compute-shader looks like this:
#version 430
layout(local_size_x = 1, local_size_y = 1) in;
layout(rgba32f, binding = 0) uniform image2D img;
layout(binding = 1) uniform sampler2DArray texAtlas;
void main() {
ivec2 iCoords = ivec2(gl_GlobalInvocationID.xy);
vec4 c = texture(texAtlas, vec3(iCoords.x%16, iCoords.y%16, 7));
imageStore(img, iCoords, c);
}
When i run the program however, the result is just a window filled with the same color:
So my question is: What did I do wrong during the shader creation and what needs to be corrected?
For any open code questions, here's the corresponding repo
vec4 c = texture(texAtlas, vec3(iCoords.x%16, iCoords.y%16, 7))
That can't work. texture samples the texture at normalized coordinates, so the texture is in [0,1] (in the st domain, the third dimension is the layer and is correct here), coordinates outside of that ar handled via the GL_WRAP_... modes you specified (repeat, clamp to edge, clamp to border). Since int % 16 is always an integer, and even with repetition only the fractional part of the coordinate will matter, you are basically sampling the same texel over and over again.
If you need the full texture sampling (texture filtering, sRGB conversions etc.), you have to use the normalized coordinates instead. But if you only want to access individual texel data, you can use texelFetch and keep the integer data instead.
Note, since you set the texture filter to GL_LINEAR, you seem to want filtering, however, your coordinates appear as if you would want at to access the texel centers, so if you're going the texture route , thenvec3(vec2(iCoords.xy)/vec2(16) + vec2(1.0/32.0) , layer) would be the proper normalization to reach the texel centers (together with GL_REPEAT), but then, the GL_LINEAR filtering would yield identical results to GL_NEAREST.
I have a src pic like that (the source pic is too large to upload):
and I have a white background logo like that :
I tried to use the OpenCV code:
std::string file_name = "E:\\xxx\\IMG_0001.JPG";
cv::Mat image = cv::imread(file_name);
cv::Mat mask_not;
cv::Mat mask = cv::imread("E:\\xxx\\white_eva.jpg",0);
cv::Mat logo = cv::imread("E:\\xxx\\white_eva.jpg");
cv::bitwise_not(mask,mask_not);
cv::cvtColor(mask_not,mask_not,cv::COLOR_GRAY2BGR);
std::cout<<mask_not.type()<<std::endl;
cv::Mat imageROI;
imageROI = image(cv::Rect(image.cols-logo.cols-10,image.rows-logo.rows-10,logo.cols,logo.rows));
cv::imwrite("E:\\xxx\\imageROI.jpg",imageROI);
logo.copyTo(imageROI,mask_not);
cv::imwrite("E:\\xxx\\test.JPG",image);
The result like that:
As you can see from the result pic, there is a white edge around the logo. At first, I think the reason is that the mask doesn't large enough to mask the logo all. But as you can see the edge of the logo seems to show entirely. So, it confused me. The first question is that how can erase the white edge of the waterprint?
I tried adding morphology operations on your mask image. So this is what I could reach.
std::string file_name = "./image1.jpg";
cv::Mat image = cv::imread(file_name);
cv::Mat mask_not;
cv::Mat mask = cv::imread("./eva.jpg",0);
cv::Mat logo = cv::imread("./eva.jpg");
// MORPHOLOGY OPS HERE
cv::Mat element = cv::getStructuringElement(cv::MORPH_RECT,
Size(5, 5),
Point(-1, -1));
for (int i = 0; i < 20; ++i) {
cv::morphologyEx( mask, mask, cv::MORPH_CLOSE, element );
cv::morphologyEx( mask, mask, cv::MORPH_CLOSE, element );
cv::morphologyEx( mask, mask, cv::MORPH_CLOSE, element );
cv::medianBlur(mask, mask, 5);
}
cv::Mat element_dilate = cv::getStructuringElement(cv::MORPH_RECT,
Size(5, 5),
Point(-1, -1));
cv::dilate(mask, mask, element_dilate);
cv::bitwise_not(mask,mask_not);
cv::imshow("win", mask_not);
cv::waitKey(0);
cv::cvtColor(mask_not,mask_not,cv::COLOR_GRAY2BGR);
std::cout<<mask_not.type()<<std::endl;
cv::Mat imageROI;
imageROI = image(cv::Rect(image.cols-logo.cols-10,image.rows-logo.rows-10,logo.cols,logo.rows));
cv::imwrite("./imageROI.jpg",imageROI);
logo.copyTo(imageROI,mask_not);
cv::imwrite("./test.JPG",image);
Result image (I resized images a bit, so you may be need to change kernel sizes in morphology operations):
I know OpenCV only supports binary masks.
But I need to do an overlay where I have a grayscale mask that specifies transparency of the overlay.
Eg. if a pixel in the mask is 50% white it should mean a cv::addWeighted operation for that pixel with alpha=beta=0.5, gamma = 0.0.
Now, if there is no opencv library function, what algorithm would you suggest as the most efficient?
I did something like this for a fix.
typedef double Mask_value_t;
typedef Mat_<Mask_value_t> Mask;
void cv::addMasked(const Mat& src1, const Mat& src2, const Mask& mask, Mat& dst)
{
MatConstIterator_<Vec3b> it1 = src1.begin<Vec3b>(), it1_end = src1.end<Vec3b>();
MatConstIterator_<Vec3b> it2 = src2.begin<Vec3b>();
MatConstIterator_<Mask_value_t> mask_it = mask.begin();
MatIterator_<Vec3b> dst_it = dst.begin<Vec3b>();
for(; it1 != it1_end; ++it1, ++it2, ++mask_it, ++dst_it)
*dst_it = (*it1) * (1.0-*mask_it) + (*it2) * (*mask_it);
}
I have not optimized nor made safe this code yet with assertions.
Working assumptions: all Mat's and the Mask are the same size and Mat's are normal three channel color images.
I have a similar problem, where I wanted to apply a png with transparency.
My solution was using Mat expressions:
void AlphaBlend(const Mat& imgFore, Mat& imgDst, const Mat& alpha)
{
vector<Mat> vAlpha;
Mat imgAlpha3;
for(int i = 0; i < 3; i++) vAlpha.push_back(alpha);
merge(vAlpha,imgAlpha3)
Mat blend = imgFore.mul(imgAlpha3,1.0/255) +
imgDst.mul(Scalar::all(255)-imgAlpha3,1.0/255);
blend.copyTo(imgDst);
}
OpenCV supports RGBA images which you can create by using mixchannels or the split and merge functions to combine your images with your greyscale mask. I hope this is what you are looking for!
Using this method you can combine your grayscale mask with your image like so:
cv::Mat gray_image, mask, rgba_image;
std::vector<cv::Mat> result;
cv::Mat image = cv::imread(image_path);
cv::split(image, result);
cv::cvtColor(image, gray_image, CV_BGR2GRAY);
cv::threshold(gray_image, mask, 128, 255, CV_THRESH_BINARY);
result.push_back(mask);
cv::merge(result, rgba_image);
imwrite("rgba.png", rgba_image);
Keep in mind that you cannot view RGBA images using cv::imshow as described in read-rgba-image-opencv and you cannot save your image as jpeg since that format does not support transparency. It seems that you can combine channels using cv::cvtcolor as shown in opencv-2-3-convert-mat-to-rgba-pixel-array
I am trying to use OpenCV 2.3.1 to convert a 12-bit Bayer image to an 8-bit RGB image. This seems like it should be fairly straightforward using the cvCvtColor function, but the function throws an exception when I call it with this code:
int cvType = CV_MAKETYPE(CV_16U, 1);
cv::Mat bayerSource(height, width, cvType, sourceBuffer);
cv::Mat rgbDest(height, width, CV_8UC3);
cvCvtColor(&bayerSource, &rgbDest, CV_BayerBG2RGB);
I thought that I was running past the end of sourceBuffer, since the input data is 12-bit, and I had to pass in a 16-bit type because OpenCV doesn't have a 12-bit type. So I divided the width and height by 2, but cvCvtColor still threw an exception that didn't have any helpful information in it (the error message was "Unknown exception").
There was a similar question posted a few months ago that was never answered, but since my question deals more specifically with 12-bit Bayer data, I thought it was sufficiently distinct to merit a new question.
Thanks in advance.
Edit: I must be missing something, because I can't even get the cvCvtColor function to work on 8-bit data:
cv::Mat srcMat(100, 100, CV_8UC3);
const cv::Scalar val(255,0,0);
srcMat.setTo(val);
cv::Mat destMat(100, 100, CV_8UC3);
cvCvtColor(&srcMat, &destMat, CV_RGB2BGR);
I was able to convert my data to 8-bit RGB using the following code:
// Copy the data into an OpenCV Mat structure
cv::Mat bayer16BitMat(height, width, CV_16UC1, inputBuffer);
// Convert the Bayer data from 16-bit to to 8-bit
cv::Mat bayer8BitMat = bayer16BitMat.clone();
// The 3rd parameter here scales the data by 1/16 so that it fits in 8 bits.
// Without it, convertTo() just seems to chop off the high order bits.
bayer8BitMat.convertTo(bayer8BitMat, CV_8UC1, 0.0625);
// Convert the Bayer data to 8-bit RGB
cv::Mat rgb8BitMat(height, width, CV_8UC3);
cv::cvtColor(bayer8Bit, rgb8BitMat, CV_BayerGR2RGB);
I had mistakenly assumed that the 12-bit data I was getting from the camera was tightly packed, so that two 12-bit values were contained in 3 bytes. It turns out that each value was contained in 2 bytes, so I didn't have to do any unpacking to get my data into a 16-bit array that is supported by OpenCV.
Edit: See #petr's improved answer that converts to RGB before converting to 8-bits to avoid losing any color information during the conversion.
The Gillfish's answer technically works but during the conversion it uses smaller data structure (CV_8UC1) than the input (which is CV_16UC1) and loses some color information.
I would suggest first to decode the Bayer encoding but stay in 16-bits per channel (from CV_16UC1 to CV_16UC3) and later convert to CV_8UC3.
The modified Gillfish's code (assuming the camera gives image in 16bit Bayer encoding):
// Copy the data into an OpenCV Mat structure
cv::Mat mat16uc1_bayer(height, width, CV_16UC1, inputBuffer);
// Decode the Bayer data to RGB but keep using 16 bits per channel
cv::Mat mat16uc3_rgb(width, height, CV_16UC3);
cv::cvtColor(mat16uc1_bayer, mat16uc3_rgb, cv::COLOR_BayerGR2RGB);
// Convert the 16-bit per channel RGB image to 8-bit per channel
cv::Mat mat8uc3_rgb(width, height, CV_8UC3);
mat16uc3_rgb.convertTo(mat8uc3_rgb, CV_8UC3, 1.0/256); //this could be perhaps done more effectively by cropping bits
For anyone struggling with this, the above solution only works if your image actually comes in 16bit otherwise, as already suggested by the comments you should chop-off the 4 least significant bits. I achieved that with this. It's not very clean but it works.
unsigned short * image_12bit = (unsigned short*)data;
char out[rows * cols];
for(int i = 0; i < rows * cols; i++) {
out[i] = (char)((double)(255 * image_12bit[i]) / (double)(1 << 12));
}
cv::Mat bayer_image(rows, cols, CV_8UC1, (void*)out);
cv::cvtColor(bayer_image, *res, cv::COLOR_BayerGR2BGR);