I would like to convert a JPEG image into Y'UV420p using turbo jpeg. I think that this uses some 2x2 blocks with there being twice the information in the Y' that there is in each of the U and V components but I haven't been able to find an example that does this.
How do I extract these individual coponents from tjDecompressToYUV2 and what is the format of the buffer that is allocated for say an image whose dimensions are a multiple of a power of 2?
Say I have an image that is 1024 wide by 512 pixels high. How would I extract each Y' U and V value from the following code:
constexpr auto height = 512u;
constexpr auto width = 1024u;
unsigned char buffer[height * width * 3 / 2];
...
tjDecompressToYUV2(jpegDecompressor, jpegImage, jpegSize, buffer, width, 2, height, TJFLAG_FASTDCT);
...
ie. How are the components extracted from buffer?
Try using the tjDecompressToYUV function which gives you the 3 colorspaces on 3 different output buffers. The Y buffer should contain one byte (eight bit value) for the Y component. I have never used the chroma components so I cannot tell you how they are stored.
What is it that you are trying to do, as there might be another way to solve this problem?
Related
I'm using OpenGL for implementing some screen space filters. For debugging purposes, I would like to save a bunch of textures so a can compare individual pixel values. The problem is that these 16-bit float textures have negative values. Do you know of any image file formats that support negative values? How could I export them?
Yes there are some such formats ... What you need is to use non clamped floating formats. This is what I am using:
GL_LUMINANCE32F_ARB
glTexImage2D(GL_TEXTURE_2D, 0, GL_LUMINANCE32F_ARB, size, size, 0, GL_LUMINANCE, GL_FLOAT, data);
Here an example:
raytrace through 3D mesh
In there I am passing geometry in a float texture that is not clamped so it has full range not just <0,1> As you can see negative range is included too ...
There are few others but once I found the one was no point to look for more of them...
You can verify clamping using the GLSL debug prints
Also IIRC there was some function to set the clamping behavior of OpenGL but never used it (if my memory serves well).
[Edit1] exporting to file
this is a problem and unless yo are exporting to ASCII or math matrix files I do not know of any format supporting negative pixel values...
So you need to workaround... so simply convert range to non negative ones like this:
float max = ???; //some value that is safely bigger than biggest abs value in your images
for (each pixel x,y)
{
pixel(x,y) = 0.5*(max + pixel(x,y))
}
and then converting back if needed
for (each pixel x,y)
{
pixel(x,y) = (2.0*pixel(x,y)-max);
}
another common way is to normalize to range <0,1> which has the advantage that from RGB colors you can infere 3D direction... (like on normal/bump maps)
pixel(x,y) = 0.5 + 0.5*pixel(x,y)/max;
and back
pixel(x,y) = 2.0*max*pixel(x,y)-max;
I have this Python code:
cv2.addWeighted(src1, 4, cv2.GaussianBlur(src1, (0, 0), 10), src2, -4, 128)
How can I convert it to Matlab? So far I got this:
f = imread0('X.jpg');
g = imfilter(f, fspecial('gaussian',[size(f,1),size(f,2)],10));
alpha = 4;
beta = -4;
f1 = f*alpha+g*beta+128;
I want to subtract local mean color image.
Input image:
Blending output from OpenCV:
The documentation for cv2.addWeighted has the definition such that:
cv2.addWeighted(src1, alpha, src2, beta, gamma[, dst[, dtype]]) → dst
Also, the operations performed on the output image is such that:
(source: opencv.org)
Therefore, what your code is doing is exactly correct... at least for cv2.addWeighted. You take alpha, multiply this by the first image, then beta, multiply this by the second image, then add gamma on top of this. The only intricacy left to deal with is saturate, which means that any values that are beyond the dynamic range of the data type you are dealing with, you cap it at that much. Because there is a potential for negatives to occur in the result, the saturate option simply means to make any values that are negative 0 and any values that are greater than the maximum expected to that max. In this case, you'll want to make any values larger than 1 equal to 1. As such, it'll be a good idea to convert your image to double through im2double because you want to allow the addition and subtraction of values beyond the dynamic range to happen first, then you saturate after. By using the default image precision of the image (which is uint8), the saturation will happen even before the saturate operation occurs, and that'll give you the wrong results. Because you're doing this double conversion, you'll want to convert the addition of 128 for your gamma to 0.5 to compensate.
Now, the only slight problem is your Gaussian Blur. Looking at the documentation, by doing cv2.GaussianBlur(src1, (0, 0), 10), you are telling OpenCV to infer on the mask size while the standard deviation is 10. MATLAB does not infer the size of the mask for you, so you need to do this yourself. A common practice is to simply find six-times the standard deviation, take the floor and add 1. This is for both the horizontal and vertical dimensions of the mask. You can see my post here on the justification as to why this is common practice: By which measures should I set the size of my Gaussian filter in MATLAB?
Therefore, in MATLAB, you would do this with your Gaussian blur instead. BTW, it's simply imread, not imread0:
f = im2double(imread('http://i.stack.imgur.com/kl3Md.jpg')); %// Change - Reading image directly from StackOverflow
sigma = 10; %// Change
sz = 1 + floor(6*sigma); %// Change
g = imfilter(f, fspecial('gaussian', sz, sigma)); %// Change
%// Rest of the code is the same
alpha = 4;
beta = -4;
f1 = f*alpha+g*beta+0.5; %// Change
%// Saturate
f1(f1 > 1) = 1;
f1(f1 < 0) = 0;
I get this image:
Take a note that there is a slight difference in the way this appears between OpenCV in MATLAB... especially the hallowing around the eye. This is because OpenCV does something different when inferring the mask size for the Gaussian blur. This I'm not sure what is going on, but how I specified the mask size by looking at the standard deviation is one of the most common heuristics for it. Play around with the standard deviation until you get something you like.
I am trying to do a memcpy of the TangoImageBuffer data field, which, if the image is YUV, according to me should be (buffer->width * buffer->height * 3 * sizeof(uint8_t)) (sizeof just for kicks, I know it is 1), but this makes it segfault. if I just copy height*width bytes it works, and also with height*width*2, I do seem to be getting valid data, I just don't know which size should this field be.
My (relevant) code:
void onImageCallback(void *context, TangoCameraId id, const TangoImageBuffer *buffer)
{
memcpy(img_struct->image_buffer->getBuffer(), buffer->data, buffer->width * buffer->height * 3 * sizeof(uint8_t)));
}
Where image_buffer is a java ByteBuffer wrapping class which I am using in C++, inside it is allocating memory by calling new with the specified size (which in this case is the same I am trying to memcpy), and a jobject reference by doing a env->NewGlobalRef(env->NewDirectByteBuffer(buffer, this->bufferSize));, where this->bufferSize equals (buffer->width * buffer->height * 3 * sizeof(uint8_t))
I am pretty sure it is allocating the right amount of memory, as I have used it also for memcopying the xyzij buffer in other function (with the correspondent size difference, as those are floats) and it works just fine (yet I have also tried overallocating), so I know the problem is not the destination being too small.
In case this information might add to the question, I am using the regular color camera, so height should be 1280 and width 720 if I recall correctly.
Edit: Upon manually looking for the maximum amount of data I could copy without getting the segfault, it seems it tops on 1384448 (i.e. that works, 1384449 segfaults), which is roughly 1.5 times the size in pixels of the image, adding to my confusion.
The pixel format is documented as YV12 aka YUV420SP. It has full resolution for Y with 2x2 subsampling for U and V, thus U and V both have 1/4 as many samples as Y. The total number of samples is then width*height*(1 + 1/4 + 1/4) = 1.5*width*height.
does anybody know how to crop an image with a sliding window in Matlab?
e.g. I have an image of 1000x500 pixels, I would like to crop from this image blocks of 50x50 pixels... Of course I have to handle uneven divisions, but it is not necessary to have block of the same size.
Some details that have helped me in the past related to (i) ways to divide an image while block processing and (ii) "uneven division", as mentioned by OP.
(i) Ways to divide/process an image:
1. Process non-overlapping blocks:
Using default parameter {'BorderSize',[0 0]}, this can be handled with blockproc as below.
Example for (i)-1: Note the blocked nature of the output. Here each non-overlapping block of size 32 x 32 is used to calculate the std2() and the output std2 value is used to fill that particular block. The input and output are of size 32 x 32.
fun = #(block_struct) std2(block_struct.data) * ones(size(block_struct.data));
I2 = blockproc('moon.tif',[32 32],fun);
figure; subplot(1, 2, 1);
imshow('moon.tif'); title('input');
subplot(1,2, 2)
imshow(I2,[]); title('output');
Input and Output Image:
(i)-2: Process overlapping blocks:
Using parameter {'BorderSize',[V H]}: V rows are added above and below the block and H columns are added to the left and right of the block. The block that is processed has (N + 2*V) rows and (M + 2*H) columns. Using default parameter {'TrimBorder',true}, the border of the output is trimmed to the original input block size of N rows and M columns.
Example for (i)-2: Below code using blockproc uses {'BorderSize',[15 15]} with [N M] = [1 1]. This is similar to filtering each pixel of the image with a custom kernel. So the input to the processing unit is a block of size (1 + 2*15) rows and (1 + 2*15) columns. And since {'TrimBorder',true} by default, the std2 of the 31 rows by 31 columns block is provided as output for each pixel. The output is of size 1 by 1 after trimming border. Consequently, note that this example output is 'non-blocked' in contrast to the previous example. This code takes much longer time to process all the pixels.
fun = #(block_struct) std2(block_struct.data) * ones(size(block_struct.data));
I2 = blockproc('moon.tif',[1 1],fun,'BorderSize',[15 15]);
figure; subplot(1, 2, 1);
imshow('moon.tif'); title('input');
subplot(1,2, 2)
imshow(I2,[]); title('output');
Input and Output Image:
(ii) "Uneven division":
1. Zero/replicate/symmetric padding:
Zero padding so that an integer multiple of the blocks (N rows by M cols sized) can cover the [image + bounding zeros] in the uneven dimension. This can be achieved by using the default parameter {'PadMethod', 0} along with {'PadPartialBlocks' , true} ( which is false by default ). If a bounding region of zeros causes a high discontinuity in values computed from the bounding blocks, {'PadMethod', 'replicate'} or {'PadMethod', 'symmetric'} can be used.
2. Assume an "Active Region" within the image for block processing
For the case of processing each pixel, as in case (i)-2, we could assuming a bounding region of floor(block_size/2) pixels on all sides along the periphery of the image that is used as "Dummy" region. The Active region for block processing is contained within the Dummy region.
Something similar is used in imaging sensors where Dummy Pixels located at the periphery of an imaging array of Active Pixels allow for an operation like the color interpolation of all active area pixels. As color interpolation usually needs a 5x5 pixel mask to interpolate the color values of a pixel a bounding Dummy periphery of 2 pixels can be used.
Assuming MATLAB indexing, the region ( floor(block_size/2) + 1 ) to ( Input_Image_Rows - floor(block_size)/2) ) Rows by ( floor(block_size/2) + 1 ) to ( Input_ImageCols - floor(block_size)/2) ) Columns is considered as Active region (assuming a square block of side, block_size) which undergoes block processing for each pixel as in (i)-2.
Assuming a square block size of 5 by 5, this is shown by the following:
block_size = 5;
buffer_size = floor(block_size/2);
for i = (buffer_size+1):(image_rows-buffer_size)
for j = (buffer_size+1):(image_cols-buffer_size)
... % block processing for each pixel Image(i,j)
end
end
Matlab ver: R2013a
I would first look into the function blockproc to see if it can do what you want.
If you're sure you want to manually crop the image into blocks, you can use this script. It both writes the cropped images to .png files and saves the cropped images in the pages of a 3D array. You can modify it as you need.
This script assumes the image in evenly divisible by the block size. If it isn't, you'll need to pad it with zeros.
[rowstmp,colstmp]= size(myImage);
block_height = 50;
block_width = 50;
blocks_per_row = rows/block_height;
blocks_per_col = cols/block_width;
number_of_blocks = blocks_per_row*blocks_per_col;
%// pad image with zeros if needed
if ~(mod(rowstmp-1,block_height)==0)
rows = ceil(rowstmp/block_height)*block_height;
end
if ~(mod(colstmp-1,block_width)==0)
cols = ceil(colstmp/block_width)*block_width;
end
Im = uint8(zeros(rows,cols));
Im(1:rowstmp,1:colstmp) = myImage;
%// make sure these image have type uint8 so they save properly
cropped_image = uint8(zeros(rows,cols));
img_stack = uint8(zeros(rows,cols,number_of_blocks));
%// loop over the image blocks
for i = 1:blocks_per_row
for j = 1:blocks_per_col
%// get the cropped image from the original image
idxI = 1+(i-1)*block_height:i*block_height;
idxJ = 1+(j-1)*block_width :j*block_width;
cropped_image(idxI,idxJ) = Im(idxI,idxJ);
%//imshow(cropped_image)
%// write the cropped image to the current folder
filename = sprintf('block_row%d_col%d.png',i,j);
imwrite(cropped_image,filename);
cropped_image(idxI,idxJ) = 0;
%// keep all the blocks in a 3D array if we want to use them later
img_stack(:,:,(i-1)*blocks_per_col+j);
end
end
I am having 60 images of size 250x250x3 . Now I want to add it as a stack of images.
ie, I want to create a 4-D array, which holds all the images in the form of a mat file. So, it should be of size 250 x 250 x 3 x 60 .
I have tried the following. But when I displays the image it is full of white with some small marks only. This was the code .
X=zeros(250,250,3,60)
for i=1:60
X(:,:,:,i)=image1 and so on on every every loop.
Any way to create this mat.
The problem:
It seems like yout images are stored as uint8 type. When you pre-allocated your X you defined it as double (by default).
When Matlab displays an image there is a difference between a uint8 type image and double type image: for uint8 Matlab expects the intensities to range between [0..255]. However, when it comes to double type images Matlab expects the values to range between [0..1]. Thus, assigning uint8 image values to double type you have a double type image with values in the range [0..255] - which Matlab displays as white.
Solutions:
There are several ways you can solve your problem:
Define X as a uint8 type
X = zeros( [255, 255, 3, 60], 'uint8' )
This will save memory as uint8 takes single byte whereas double takes 8.
Alternatively, you can cast your images to double using im2double function that changes the data type and the range of intensities to [0..1].