View .bin file (YCbCr 4:2:2 format) - image

I am given a .bin file. I know that the elements in this file correspond to Y Cb Cr values (4:2:2). Also, the data type is 8 bits. How can I view this?
I found a pretty good site: http://rawpixels.net/ which does what is expected but for YUV format. I want for YCbCr format.
Priliminary google search gives conversion to RGB, which is not desired.
I have attached an example .bin file on dropbox. The size of image is 720 X 576.

From Wikipedia
Y′CbCr is often confused with the YUV color space, and typically the
terms YCbCr and YUV are used interchangeably, leading to some
confusion; when referring to signals in video or digital form, the
term "YUV" mostly means "Y′CbCr".
If you are on a linux-based system and have access to ffmpeg, the following command correctly displays the data
ffplay -f rawvideo -video_size 720x576 -pix_fmt yuyv422 38.bin
Another good tool for displaying of RGB/YCbCr images is vooya which is free for linux but not for windows.
My own tool, yuv-viewer works as well.
Hope this helps.

You can up-sample the 4:2:2 down-sampled chroma like this:
////////////////////////////////////////////////////////////////////////////////
// unpack.c
// Mark Setchell
//
// Convert YCbCr 4:2:2 format file to full YCbCr without chroma subsampling
//
// Compile with:
// gcc -o unpack unpack.c
// Run with:
// ./unpack < input.bin > output.bin
////////////////////////////////////////////////////////////////////////////////
#include <stdio.h>
#include <sys/uio.h>
#include <unistd.h>
#include <sys/types.h>
int main(){
unsigned char ibuf[4]; // Input data buffer format: Y Cb Y Cr
unsigned char obuf[6]; // Output data buffer format: Y Cb Cr Y Cb Cr
// Read 4 bytes at a time, and upsample chroma
while(fread(ibuf,4,1,stdin)==1){
obuf[0]=ibuf[0];
obuf[1]=ibuf[1];
obuf[2]=ibuf[3];
obuf[3]=ibuf[2];
obuf[4]=ibuf[1];
obuf[5]=ibuf[3];
fwrite(obuf,6,1,stdout);
}
return 0;
}
Then you would run this to up-sample:
./unpack < input.bin > output.bin
and then use ImageMagick convert to get a PNG (or JPEG, or TIF) like this:
convert -size 720x576 -depth 8 yuv:result.bin image.png
In theory, ImageMagick should be able to do the up sampling itself (and not need a C program) with a command line like this, but I can't seem to make it work:
convert -interlace none -sampling-factor 4:2:2 -size 720x576 -depth 8 yuv:input.bin image.jpg
If anyone knows why - please comment!

This is a slightly different version of my other answer, insofar as it up-samples the chroma, and also converts the YUV to RGB and then creates a NetPBM PNM format file. That means that you only need to install the pnmtopng utility from NetPBM to get to a PNM image - and NetPBM is much lighter weight and simpler to install than ImageMagick.
////////////////////////////////////////////////////////////////////////////////
// yuv2pnm.c
// Mark Setchell
//
// Convert YUV 4:2:2 format file to RGB PNM format without chroma subsampling
//
// Compile with:
// gcc -o yuv2pnm yuv2pnm.c
//
// Run with:
// ./yuv2pnm < input.bin > output.pnm
//
// and then use ImageMagick to go to PNG format, or JPEG or TIF, with
//
// convert output.pnm image.png
//
// or, all in one line (still with ImageMagick) to JPEG:
//
// ./yuv2pnm < input.bin | convert pnm:- image.jpg
//
// or, use the (simpler-to-install) NetPBM's "pnmtopng" to convert to a PNG file
//
// ./yuv2pnm < input.bin | pnmtopng - > image.png
////////////////////////////////////////////////////////////////////////////////
#include <stdio.h>
#define MIN(a,b) (a<b) ? a : b
void YUV2RGB(unsigned char Y,unsigned char U, unsigned char V,unsigned char *RGB)
{
int R,G,B;
R = Y + (1.370705 * (V-128));
G = Y - (0.698001 * (V-128)) - (0.337633 * (U-128));
B = Y + (1.732446 * (U-128));
RGB[0] = MIN(255,R);
RGB[1] = MIN(255,G);
RGB[2] = MIN(255,B);
}
int main(int argc,char* argv[]){
unsigned char buf[4]; // Input data buffer format: Y Cb Y Cr
unsigned char RGB[6]; // Output data buffer format: R G B R G B
int width=720;
int height=576;
// Write PNM header
fprintf(stdout,"P6\n");
fprintf(stdout,"%d %d\n",width,height);
fprintf(stdout,"255\n");
// Read 4 bytes at a time, upsample chroma and convert to 2 RGB pixels
while(fread(buf,4,1,stdin)==1){
YUV2RGB(buf[0],buf[1],buf[3],&RGB[0]);
YUV2RGB(buf[2],buf[1],buf[3],&RGB[3]);
fwrite(RGB,6,1,stdout);
}
return 0;
}
NetPBM format is described here. Note PNM is an abbreviation that includes PPM.

Find below formula to convert YUV data into RGB.
R = Y + 1.4075 * (V - 128)
G = Y - 0.3455 * (U - 128) - (0.7169 * (V - 128))
B = Y + 1.7790 * (U - 128)

Related

Turn off sw_scale conversion to planar YUV 32 byte alignment requirements

I am experiencing artifacts on the right edge of scaled and converted images when converting into planar YUV pixel formats with sw_scale. I am reasonably sure (although I can not find it anywhere in the documentation) that this is because sw_scale is using an optimization for 32 byte aligned lines, in the destination. However I would like to turn this off because I am using sw_scale for image composition, so even though the destination lines may be 32 byte aligned, the output image may not be.
Example.
Full output frame is 1280x720 yuv422p10le. (this is 32 byte aligned)
However into the top left corner I am scaling an image with an outwidth of 1280 / 3 = 426.
426 in this format is not 32 byte aligned, but I believe sw_scale sees that the output linesize is 32 byte aligned and overwrites the width of 426 putting garbage in the next 22 bytes of data thinking this is simply padding when in my case this is displayable area.
This is why I need to actually disable this optimization or somehow trick sw_scale into believing it does not apply while keeping intact the way the program works, which is otherwise fine.
I have tried adding extra padding to the destination lines so they are no longer 32 byte aligned,
this did not help as far as I can tell.
Edit with code Example. Rendering omitted for ease of use.
Also here is a similar issue, unfortunately as I stated there fix will not work for my use case. https://github.com/obsproject/obs-studio/pull/2836
Use the commented line of code to swap between a output width which is and isnt 32 byte aligned.
#include "libswscale/swscale.h"
#include "libavutil/imgutils.h"
#include "libavutil/pixelutils.h"
#include "libavutil/pixfmt.h"
#include "libavutil/pixdesc.h"
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
int main(int argc, char **argv) {
/// Set up a 1280x720 window, and an item with 1/3 width and height of the window.
int window_width, window_height, item_width, item_height;
window_width = 1280;
window_height = 720;
item_width = (window_width / 3);
item_height = (window_height / 3);
int item_out_width = item_width;
/// This line sets the item width to be 32 byte aligned uncomment to see uncorrupted results
/// Note %16 because outformat is 2 bytes per component
//item_out_width -= (item_width % 16);
enum AVPixelFormat outformat = AV_PIX_FMT_YUV422P10LE;
enum AVPixelFormat informat = AV_PIX_FMT_UYVY422;
int window_lines[4] = {0};
av_image_fill_linesizes(window_lines, outformat, window_width);
uint8_t *window_planes[4] = {0};
window_planes[0] = calloc(1, window_lines[0] * window_height);
window_planes[1] = calloc(1, window_lines[1] * window_height);
window_planes[2] = calloc(1, window_lines[2] * window_height); /// Fill the window with all 0s, this is green in yuv.
int item_lines[4] = {0};
av_image_fill_linesizes(item_lines, informat, item_width);
uint8_t *item_planes[4] = {0};
item_planes[0] = malloc(item_lines[0] * item_height);
memset(item_planes[0], 100, item_lines[0] * item_height);
struct SwsContext *ctx;
ctx = sws_getContext(item_width, item_height, informat,
item_out_width, item_height, outformat, SWS_FAST_BILINEAR, NULL, NULL, NULL);
/// Check a block in the normal region
printf("Pre scale normal region %d %d %d\n", (int)((uint16_t*)window_planes[0])[0], (int)((uint16_t*)window_planes[1])[0],
(int)((uint16_t*)window_planes[2])[0]);
/// Check a block in the corrupted region (should be all zeros) These values should be out of the converted region
int corrupt_offset_y = (item_out_width + 3) * 2; ///(item_width + 3) * 2 bytes per component Y PLANE
int corrupt_offset_uv = (item_out_width + 3); ///(item_width + 3) * (2 bytes per component rshift 1 for horiz scaling) U and V PLANES
printf("Pre scale corrupted region %d %d %d\n", (int)(*((uint16_t*)(window_planes[0] + corrupt_offset_y))),
(int)(*((uint16_t*)(window_planes[1] + corrupt_offset_uv))), (int)(*((uint16_t*)(window_planes[2] + corrupt_offset_uv))));
sws_scale(ctx, (const uint8_t**)item_planes, item_lines, 0, item_height,window_planes, window_lines);
/// Preform same tests after scaling
printf("Post scale normal region %d %d %d\n", (int)((uint16_t*)window_planes[0])[0], (int)((uint16_t*)window_planes[1])[0],
(int)((uint16_t*)window_planes[2])[0]);
printf("Post scale corrupted region %d %d %d\n", (int)(*((uint16_t*)(window_planes[0] + corrupt_offset_y))),
(int)(*((uint16_t*)(window_planes[1] + corrupt_offset_uv))), (int)(*((uint16_t*)(window_planes[2] + corrupt_offset_uv))));
return 0;
}
Example Output:
//No alignment
Pre scale normal region 0 0 0
Pre scale corrupted region 0 0 0
Post scale normal region 400 400 400
Post scale corrupted region 512 36865 36865
//With alignment
Pre scale normal region 0 0 0
Pre scale corrupted region 0 0 0
Post scale normal region 400 400 400
Post scale corrupted region 0 0 0
I believe sw_scale sees that the output linesize is 32 byte aligned and overwrites the width of 426 putting garbage in the next 22 bytes of data thinking this is simply padding when in my case this is displayable area.
That's actually correct, swscale indeed does that, good analysis. There's two ways to get rid of this:
disable all SIMD code using av_set_cpu_flags_mask(0).
write the re-scaled 426xN image in a temporary buffer and then manually copy the pixels into the unpadded destination plane.
The reason ffmpeg/swscale overwrite the destination is for performance. If you don't care about runtime and want the simplest code, use the first solution. If you do want performance and don't mind slightly more complicated code, use the second solution.

Get RGBA values from decoded PNG image

The task at hand is to read a PNG image as bytes, get the RGBA values of each pixel, apply modifications and set new pixels in the image. So far this is the code I have:
import 'package:image/image.dart' as img;
final byteData = await rootBundle.load('assets/images/image.png');
img.Image image = img.Image.fromBytes(1536, 2048, byteData.buffer.asInt8List(), format: img.Format.rgba);
print(image.getPixelSafe(0, 0));
Applying it to this image, I get the integer value 1196314761. How do I convert this integer into RGBA or hex format?
If this is the wrong way to get a specific pixel, how can I do so?
First, you'll need to properly decode the image, which you are not currently doing. Use the Decoder class instead of trying to directly pass your encoded data to the Image constructor.
img.Image image = PngDecoder().decodeImage(byteData.buffer.asInt8List());
The package you're using returns the color as
a Uint32 as #AABBGGRR
To get the colors you want individually, just extract each of those bytes from the Uint32. The following code does a bitwise & to obtain only the least significant byte. Bit shifts are done to get higher bytes.
int input = image.getPixelSafe(0, 0);
int red = input & 0xff;
int green = input >> 8 & 0xff;
int blue = input >> 8 * 2 & 0xff;
int alpha = input >> 8 * 3 & 0xff;
To get this as a hex string you can do red.toRadixString(16).

Lossless RGB24 to YUV444 transformation

I am currently attempting to undergo lossless compression of RGB24 files using H264 on FFMPEG. However, the color space transformation used in the H264 compression (RGB24 -> YUV444) has proven to be lossy (I'm guessing due to quantisation error).
Is there anything else I can use (eg a program) to transform my RGB24 files to YUV losslessly, before compressing them with lossless H264?
The ultimate goal is to compress an RGB24 file then decompress it, with the decompressed file exactly matching the original file. eg RGB24 -> YUV444 -> compressed YUV44 -> decompressed YUV444 -> RGB24.
Is this at all possible?
This is a copy/paste from my answer here:
RGB-frame encoding - FFmpeg/libav
lets look at the colorspace conversion.
void YUVfromRGB(double& Y, double& U, double& V, const double R, const double G, const double B)
{
Y = 0.257 * R + 0.504 * G + 0.098 * B + 16;
U = -0.148 * R - 0.291 * G + 0.439 * B + 128;
V = 0.439 * R - 0.368 * G - 0.071 * B + 128;
}
And plug in some dummy values:
R = 255, G = 255, B = 255
Y = 235
R = 0, G = 0, B = 0
Y = 16
As you can see, the range 0 -> 255 is squished to 16 -> 235. Thus we have shown that there are some colors in the RGB colorspace that do not exist in the (digital) YUV color space. Hence the conversion is lossy by definition.

Xcode convert rawdata from 8 bit/channel to 16 bit/channel using apple native framework

I have rgb raw data, which is 8 bit/channel, i need to convert it to 16 bit /channel, any native framework can do so, thanks.
so input is image width, height, rawdata, any suggestion?
If you just want to convert 8bpc raw image into 16bpc raw image (without any special color processing and filtering) - you don't need any special framework. You can do it by youself by conbverting pixel by pixel... In case of unsigned short 16 bit raw conversion will looks like:
size_t ent_cnt = width*height*channel_count;
unsigned short *dst = new unsigned short[ent_cnt];
unsigned short *dst_ptr = dst;
unsigned char *src_ptr = src;
while (ent_cnt --)
*dst_ptr++ = (unsigned short)(*src_ptr++) << 8;
return dst;

Convert 12-bit Bayer image to 8-bit RGB using OpenCV

I am trying to use OpenCV 2.3.1 to convert a 12-bit Bayer image to an 8-bit RGB image. This seems like it should be fairly straightforward using the cvCvtColor function, but the function throws an exception when I call it with this code:
int cvType = CV_MAKETYPE(CV_16U, 1);
cv::Mat bayerSource(height, width, cvType, sourceBuffer);
cv::Mat rgbDest(height, width, CV_8UC3);
cvCvtColor(&bayerSource, &rgbDest, CV_BayerBG2RGB);
I thought that I was running past the end of sourceBuffer, since the input data is 12-bit, and I had to pass in a 16-bit type because OpenCV doesn't have a 12-bit type. So I divided the width and height by 2, but cvCvtColor still threw an exception that didn't have any helpful information in it (the error message was "Unknown exception").
There was a similar question posted a few months ago that was never answered, but since my question deals more specifically with 12-bit Bayer data, I thought it was sufficiently distinct to merit a new question.
Thanks in advance.
Edit: I must be missing something, because I can't even get the cvCvtColor function to work on 8-bit data:
cv::Mat srcMat(100, 100, CV_8UC3);
const cv::Scalar val(255,0,0);
srcMat.setTo(val);
cv::Mat destMat(100, 100, CV_8UC3);
cvCvtColor(&srcMat, &destMat, CV_RGB2BGR);
I was able to convert my data to 8-bit RGB using the following code:
// Copy the data into an OpenCV Mat structure
cv::Mat bayer16BitMat(height, width, CV_16UC1, inputBuffer);
// Convert the Bayer data from 16-bit to to 8-bit
cv::Mat bayer8BitMat = bayer16BitMat.clone();
// The 3rd parameter here scales the data by 1/16 so that it fits in 8 bits.
// Without it, convertTo() just seems to chop off the high order bits.
bayer8BitMat.convertTo(bayer8BitMat, CV_8UC1, 0.0625);
// Convert the Bayer data to 8-bit RGB
cv::Mat rgb8BitMat(height, width, CV_8UC3);
cv::cvtColor(bayer8Bit, rgb8BitMat, CV_BayerGR2RGB);
I had mistakenly assumed that the 12-bit data I was getting from the camera was tightly packed, so that two 12-bit values were contained in 3 bytes. It turns out that each value was contained in 2 bytes, so I didn't have to do any unpacking to get my data into a 16-bit array that is supported by OpenCV.
Edit: See #petr's improved answer that converts to RGB before converting to 8-bits to avoid losing any color information during the conversion.
The Gillfish's answer technically works but during the conversion it uses smaller data structure (CV_8UC1) than the input (which is CV_16UC1) and loses some color information.
I would suggest first to decode the Bayer encoding but stay in 16-bits per channel (from CV_16UC1 to CV_16UC3) and later convert to CV_8UC3.
The modified Gillfish's code (assuming the camera gives image in 16bit Bayer encoding):
// Copy the data into an OpenCV Mat structure
cv::Mat mat16uc1_bayer(height, width, CV_16UC1, inputBuffer);
// Decode the Bayer data to RGB but keep using 16 bits per channel
cv::Mat mat16uc3_rgb(width, height, CV_16UC3);
cv::cvtColor(mat16uc1_bayer, mat16uc3_rgb, cv::COLOR_BayerGR2RGB);
// Convert the 16-bit per channel RGB image to 8-bit per channel
cv::Mat mat8uc3_rgb(width, height, CV_8UC3);
mat16uc3_rgb.convertTo(mat8uc3_rgb, CV_8UC3, 1.0/256); //this could be perhaps done more effectively by cropping bits
For anyone struggling with this, the above solution only works if your image actually comes in 16bit otherwise, as already suggested by the comments you should chop-off the 4 least significant bits. I achieved that with this. It's not very clean but it works.
unsigned short * image_12bit = (unsigned short*)data;
char out[rows * cols];
for(int i = 0; i < rows * cols; i++) {
out[i] = (char)((double)(255 * image_12bit[i]) / (double)(1 << 12));
}
cv::Mat bayer_image(rows, cols, CV_8UC1, (void*)out);
cv::cvtColor(bayer_image, *res, cv::COLOR_BayerGR2BGR);

Resources