Consider the following Python invocations, that exercise ffmpeg's ability to convert a 8-bit input into 16-bit integer values or 32-bit float values:
import cv2
import subprocess
import numpy as np
ffmpeg = "ffmpeg -hide_banner -loglevel error -y"
flags = "-sws_flags accurate_rnd+bitexact+full_chroma_int+neighbor -sws_dither none"
for input_color_range in ("tv", "pc"):
# Generate YUV444 video and encode it losslessly
subprocess.check_call(rf"{ffmpeg} -y {flags} -color_range {input_color_range} -f lavfi -i yuvtestsrc {flags} -pix_fmt yuv444p -color_range {input_color_range} -x264-params qp=0 -frames 1 -c:v libx264 video_temp.mp4")
# Extract YUV444 frame, as well as 8-bit int, 16-bit int and 32-bit float frame
subprocess.check_call(rf"{ffmpeg} -y {flags} -i video_temp.mp4 {flags} -f rawvideo video_temp_444_frame1.yuv")
subprocess.check_call(rf"{ffmpeg} -y {flags} -i video_temp.mp4 {flags} -color_range pc -pix_fmt rgb24 video_temp_444_frame1_8u.png")
subprocess.check_call(rf"{ffmpeg} -y {flags} -i video_temp.mp4 {flags} -color_range pc -pix_fmt rgb48be video_temp_444_frame1_16u.png")
subprocess.check_call(rf"{ffmpeg} -y {flags} -i video_temp.mp4 {flags} -color_range pc -pix_fmt gbrpf32be -f rawvideo video_temp_444_frame1_32f.gbrpf32be")
# Reead these frames into (height, width, 3) Numpy arrays containing YUV or RGB data
data_8u = cv2.imread("video_temp_444_frame1_8u.png", cv2.IMREAD_UNCHANGED)[..., ::-1]
data_16u = cv2.imread("video_temp_444_frame1_16u.png", cv2.IMREAD_UNCHANGED)[..., ::-1]
data_yuv = np.rollaxis(np.frombuffer(open("video_temp_444_frame1.yuv", "rb").read()).view(np.uint8).reshape((3, 240, 320)), 0, 3).copy()
data_32f = np.rollaxis(np.frombuffer(open("video_temp_444_frame1_32f.gbrpf32be", "rb").read()).view(np.dtype(">f4")).reshape((3, 240, 320)), 0, 3).copy()
data_32f[..., (0, 1, 2)] = data_32f[..., (2, 0, 1)]
# This pixel in yuvtestsrc corresponds to limited-range YUV=(235, 128,
# 128)=(100%, 0.0, 0.0), which should correspond to 100% white, i.e.
# RGB=(100%, 100%, 100%).
if input_color_range == "tv":
i, j = 10, 294
# This pixel in yuvtestsrc corresponds to full-range YUV=(255, 128,
# 128)=(100%, 0.0, 0.0), which should correspond to 100% white, i.e.
# RGB=(100%, 100%, 100%).
elif input_color_range == "pc":
i, j = 10, 319
else:
raise Exception(input_color_range)
# Print pixel values
print("")
print(f"Values for {input_color_range}-range input video at ({i}, {j}):")
print("- 8-bit YUV input = %s" % data_yuv[i, j, :])
print("- 8-bit RGB output = %s (= %s)" % (data_8u[i, j, :], data_8u[i, j, :] / 255))
print("- 16-bit RGB output = %s (= %s)" % (data_16u[i, j, :], data_16u[i, j, :] / 65535))
print("- Float RGB output = %s" % data_32f[i, j, :])
The script first generates a video frame from ffmpeg's YUV test source, encoded with no chroma subsampling (4:4:4) as losslessly as possible.
That video is then used as a reference source to extract the following frames:
The input reference YUV data
A 8-bit RGB conversion
A 16-bit RGB conversion
A 32-bit floating-point RGB conversion
One pixel is extracted from each frame, which should contain a 100% white value. The output of the last series of print statements is the following:
Values for tv-range input video at (10, 294):
- 8-bit YUV input = [235 128 128]
- 8-bit RGB output = [255 255 255] (= [1. 1. 1.])
- 16-bit RGB output = [65283 65283 65283] (= [0.99615473 0.99615473 0.99615473])
- Float RGB output = [0.9961547 0.9961547 0.9961547]
Values for pc-range input video at (10, 319):
- 8-bit YUV input = [255 128 128]
- 8-bit RGB output = [255 255 255] (= [1. 1. 1.])
- 16-bit RGB output = [65280 65280 65280] (= [0.99610895 0.99610895 0.99610895])
- Float RGB output = [0.99610895 0.99610895 0.99610895]
While the 8-bit RGB output values are correct, none of the others correctly output a 100% white signal. Why is that?
Related
I see this code and need add a image of background
like this
enter image description here
how I could do this add a background instead black background
please help me
I get this code from another question but I dont know how change the background
Create video from array of pixel values in C++
#include <iostream>
#include "CImg.h"
using namespace std;
using namespace cimg_library;
int main()
{
const unsigned int width=1024;
const unsigned int height=768;
// Basic frame we will draw in
CImg<unsigned char> image(width,height,1,3);
unsigned char magenta[] = {255,0,255};
// We are going to output 300 frames of 1024x768 RGB raw video
// ... making a 10s long video at 30fps
int radius=100;
int cx=100;
int cy=100;
for(int frame=0;frame<300;frame++){
// Start with black - it shows fewer stains ;-)
image.fill(0);
image.draw_circle(cx,cy,radius,magenta);
// Move and re-colour circle
cx+=2; cy++; if(magenta[1]!=255){magenta[1]++;}
// Output to ffmpeg to make video, in planar GBR format
// i.e. run program like this
// ./main | ffmpeg -y -f rawvideo -pixel_format gbrp -video_size 1024x768 -i - -c:v h264 -pix_fmt yuv420p video.mov
char* s=reinterpret_cast<char*>(image.data()+(width*height)); // Get start of G plane
std::cout.write(s,width*height); // Output it
s=reinterpret_cast<char*>(image.data()+2*(width*height)); // Get start of B plane
std::cout.write(s,width*height); // Output it
s=reinterpret_cast<char*>(image.data()); // Get start of R plane
std::cout.write(s,width*height); // Output it
}
}
I compile using this
./main | ffmpeg -y -f rawvideo -pixel_format gbrp -video_size 1024x768 -i - -c:v h264 -pix_fmt yuv420p video.mov
All you need to do is load the background image at the start and then make a copy of it to draw on at each iterations:
#include <iostream>
#include "CImg.h"
using namespace std;
using namespace cimg_library;
int main()
{
// Background frame we will draw
CImg<unsigned char> bg("background.jpg");
bg.resize(1024,768);
int width=bg.width();
int height=bg.height();
unsigned char magenta[] = {255,0,255};
// We are going to output 300 frames of RGB raw video
// ... making a 10s long video at 30fps
int radius=100;
int cx=100;
int cy=100;
for(int frame=0;frame<300;frame++){
// Take copy of background image
CImg<unsigned char> image(bg,false);
image.draw_circle(cx,cy,radius,magenta);
// Move and re-colour circle
cx+=2; cy++; if(magenta[1]!=255){magenta[1]++;}
// Output to ffmpeg to make video, in planar GBR format
// i.e. run program like this
// ./main | ffmpeg -y -f rawvideo -pixel_format gbrp -video_size 1024x768 -i - -c:v h264 -pix_fmt yuv420p video.mov
char* s=reinterpret_cast<char*>(image.data()+(width*height)); // Get start of G plane
std::cout.write(s,width*height); // Output it
s=reinterpret_cast<char*>(image.data()+2*(width*height)); // Get start of B plane
std::cout.write(s,width*height); // Output it
s=reinterpret_cast<char*>(image.data()); // Get start of R plane
std::cout.write(s,width*height); // Output it
}
}
ffmpeg -y -i input.mp4 -i "logo2.png" -filter_complex "[0:v]setpts=PTS/1.15,boxblur=2:1,scale=iw/1.75:-1,pad=iw+26:ih+26:13:13:color=blue [v1]; movie=bgmu.mp4:loop=999,setpts=N/(FRAME_RATE*TB) [v2]; [v2][v1]overlay=shortest=1:x=W-w-30:y=H-h-22 [v3]; [v3][1:v]overlay=0:0,setdar=16/9; [0:a]atempo=1.15, aecho=0.4:0.66:2:0.2, chorus=0.5:0.9:50|80:0.4|0.42:0.25|0.4:2|1.4, firequalizer=gain_entry='entry(100,0); entry(400, -4); entry(1000, -6); entry(2000, 0)',equalizer = f = 1000: width_type = q: width = 1: g = 2, equalizer = f = 100: width_type = q: width = 2: g = 5,pan=stereo|c0
In [v3][1:v]overlay=0:0, the first 0 is x-coordinate measured from left edge of [v3] and 2nd 0 is y-coordinate measured from top of [v3]. So change those values. You can use W, H, w and h in your expressions, which are the width and height of [v3] and [1:v] respectively.
[v3][1:v]overlay=(W-w)/2:H-h-10 means to overlay it horizontally center and vertically at the bottom leaving a margin of 10 pixels.
I am given a .bin file. I know that the elements in this file correspond to Y Cb Cr values (4:2:2). Also, the data type is 8 bits. How can I view this?
I found a pretty good site: http://rawpixels.net/ which does what is expected but for YUV format. I want for YCbCr format.
Priliminary google search gives conversion to RGB, which is not desired.
I have attached an example .bin file on dropbox. The size of image is 720 X 576.
From Wikipedia
Y′CbCr is often confused with the YUV color space, and typically the
terms YCbCr and YUV are used interchangeably, leading to some
confusion; when referring to signals in video or digital form, the
term "YUV" mostly means "Y′CbCr".
If you are on a linux-based system and have access to ffmpeg, the following command correctly displays the data
ffplay -f rawvideo -video_size 720x576 -pix_fmt yuyv422 38.bin
Another good tool for displaying of RGB/YCbCr images is vooya which is free for linux but not for windows.
My own tool, yuv-viewer works as well.
Hope this helps.
You can up-sample the 4:2:2 down-sampled chroma like this:
////////////////////////////////////////////////////////////////////////////////
// unpack.c
// Mark Setchell
//
// Convert YCbCr 4:2:2 format file to full YCbCr without chroma subsampling
//
// Compile with:
// gcc -o unpack unpack.c
// Run with:
// ./unpack < input.bin > output.bin
////////////////////////////////////////////////////////////////////////////////
#include <stdio.h>
#include <sys/uio.h>
#include <unistd.h>
#include <sys/types.h>
int main(){
unsigned char ibuf[4]; // Input data buffer format: Y Cb Y Cr
unsigned char obuf[6]; // Output data buffer format: Y Cb Cr Y Cb Cr
// Read 4 bytes at a time, and upsample chroma
while(fread(ibuf,4,1,stdin)==1){
obuf[0]=ibuf[0];
obuf[1]=ibuf[1];
obuf[2]=ibuf[3];
obuf[3]=ibuf[2];
obuf[4]=ibuf[1];
obuf[5]=ibuf[3];
fwrite(obuf,6,1,stdout);
}
return 0;
}
Then you would run this to up-sample:
./unpack < input.bin > output.bin
and then use ImageMagick convert to get a PNG (or JPEG, or TIF) like this:
convert -size 720x576 -depth 8 yuv:result.bin image.png
In theory, ImageMagick should be able to do the up sampling itself (and not need a C program) with a command line like this, but I can't seem to make it work:
convert -interlace none -sampling-factor 4:2:2 -size 720x576 -depth 8 yuv:input.bin image.jpg
If anyone knows why - please comment!
This is a slightly different version of my other answer, insofar as it up-samples the chroma, and also converts the YUV to RGB and then creates a NetPBM PNM format file. That means that you only need to install the pnmtopng utility from NetPBM to get to a PNM image - and NetPBM is much lighter weight and simpler to install than ImageMagick.
////////////////////////////////////////////////////////////////////////////////
// yuv2pnm.c
// Mark Setchell
//
// Convert YUV 4:2:2 format file to RGB PNM format without chroma subsampling
//
// Compile with:
// gcc -o yuv2pnm yuv2pnm.c
//
// Run with:
// ./yuv2pnm < input.bin > output.pnm
//
// and then use ImageMagick to go to PNG format, or JPEG or TIF, with
//
// convert output.pnm image.png
//
// or, all in one line (still with ImageMagick) to JPEG:
//
// ./yuv2pnm < input.bin | convert pnm:- image.jpg
//
// or, use the (simpler-to-install) NetPBM's "pnmtopng" to convert to a PNG file
//
// ./yuv2pnm < input.bin | pnmtopng - > image.png
////////////////////////////////////////////////////////////////////////////////
#include <stdio.h>
#define MIN(a,b) (a<b) ? a : b
void YUV2RGB(unsigned char Y,unsigned char U, unsigned char V,unsigned char *RGB)
{
int R,G,B;
R = Y + (1.370705 * (V-128));
G = Y - (0.698001 * (V-128)) - (0.337633 * (U-128));
B = Y + (1.732446 * (U-128));
RGB[0] = MIN(255,R);
RGB[1] = MIN(255,G);
RGB[2] = MIN(255,B);
}
int main(int argc,char* argv[]){
unsigned char buf[4]; // Input data buffer format: Y Cb Y Cr
unsigned char RGB[6]; // Output data buffer format: R G B R G B
int width=720;
int height=576;
// Write PNM header
fprintf(stdout,"P6\n");
fprintf(stdout,"%d %d\n",width,height);
fprintf(stdout,"255\n");
// Read 4 bytes at a time, upsample chroma and convert to 2 RGB pixels
while(fread(buf,4,1,stdin)==1){
YUV2RGB(buf[0],buf[1],buf[3],&RGB[0]);
YUV2RGB(buf[2],buf[1],buf[3],&RGB[3]);
fwrite(RGB,6,1,stdout);
}
return 0;
}
NetPBM format is described here. Note PNM is an abbreviation that includes PPM.
Find below formula to convert YUV data into RGB.
R = Y + 1.4075 * (V - 128)
G = Y - 0.3455 * (U - 128) - (0.7169 * (V - 128))
B = Y + 1.7790 * (U - 128)
I used libx264 in ffmpeg to encode video, I used the configuration below.
enCodecContext->bit_rate = 300000;
enCodecContext->width = 80;
enCodecContext->height = 60;
enCodecContext->time_base = (AVRational) {1, 25};
enCodecContext->gop_size = 10;
enCodecContext->max_b_frames = 1;
enCodecContext->pix_fmt = PIX_FMT_YUV420P;
enCodecContext->qcompress = 0.6;
av_opt_set(enCodecContext->priv_data, "preset", "slow", 0);
But when I called avcodec_encode_video2 with enCodecContext, I got the error Input picture width (40) is greater than stride (0).
avcodec_encode_video2(enCodecContext, &filteredAVPacket, pFilteredAVFrame, &got_packet_ptr);
The pFilteredAVFrame->width and pFilteredAVFrame->height is 80 and 60 respectively.
Did I missed something when configured libx264, How can I get a workable configuration for libx264 to encode my video?
x264 is fine. You must fill in the AVPicture.linestride variable for your image planes. The stride describes how the image is laid out in memory. The stride must be at least as big as the image width. In the case of YUV 4:2:0, the stride must be at least half the width on the second and third plane.
https://msdn.microsoft.com/en-us/library/windows/desktop/aa473780(v=vs.85).aspx
I am currently attempting to undergo lossless compression of RGB24 files using H264 on FFMPEG. However, the color space transformation used in the H264 compression (RGB24 -> YUV444) has proven to be lossy (I'm guessing due to quantisation error).
Is there anything else I can use (eg a program) to transform my RGB24 files to YUV losslessly, before compressing them with lossless H264?
The ultimate goal is to compress an RGB24 file then decompress it, with the decompressed file exactly matching the original file. eg RGB24 -> YUV444 -> compressed YUV44 -> decompressed YUV444 -> RGB24.
Is this at all possible?
This is a copy/paste from my answer here:
RGB-frame encoding - FFmpeg/libav
lets look at the colorspace conversion.
void YUVfromRGB(double& Y, double& U, double& V, const double R, const double G, const double B)
{
Y = 0.257 * R + 0.504 * G + 0.098 * B + 16;
U = -0.148 * R - 0.291 * G + 0.439 * B + 128;
V = 0.439 * R - 0.368 * G - 0.071 * B + 128;
}
And plug in some dummy values:
R = 255, G = 255, B = 255
Y = 235
R = 0, G = 0, B = 0
Y = 16
As you can see, the range 0 -> 255 is squished to 16 -> 235. Thus we have shown that there are some colors in the RGB colorspace that do not exist in the (digital) YUV color space. Hence the conversion is lossy by definition.