I'm trying to use Ffmpeg for creating a hevc realtime stream from a Decklink input. The goal is high quality HDR stream usage with 10 bits.
The Decklink SDI input is fed RGB 10 bits, which is well handled by ffmpeg with the decklink option -raw_format rgb10, which gets recognized by ffmpeg as 'gbrp10le'.
I have a Nvidia pascal-based card, which supports yuv444 10 bit (as 'yuv444p16le') and when when using '-c:v hevc_nvenc' the auto_scaler kicks in and converts to 'yuv444p16le', which I guess is the same conversion as giving '-pix_fmt yuv444p16le'.
This is working very well in 1920x1080 resolution, but in 4096x2160 resolution ffmpeg can't keep up realtime 24 or 25 fps, and I get input buffer overruns.
The culprit seems to be the RGB->YUV conversion in ffmpeg swscale because;
When piping the Decklink 4K RGB input with '-c:v copy' straight to /dev/null, there's is no problems with buffer underruns,
And when feeding the Decklink YUV and giving '-raw_format yuv422p10’ (no YUV444 input for decklink seems available for decklink in ffmpeg) I get no underrun and everything works well in 4K. Even if I set '-pix_fmt yuv444p16le'.
Any ideas how I could accomplish a 4K hevc in NVENC with the 10-bit RGB signal from the Decklink? Is there a way to make NVENC accept and use the RGB data without first converting to YUV? Or is there maybe a way to convert gbrp10le->yuv444p16le with cuda or scale_npp filter? I have compiled ffmpeg with npp and cuda, but I cannot figure out if I can get it to work with RGB? Whenever I try to do '-vf "hwupload_cuda"', auto_scaler kicks in and tries to convert to yuv on the cpu, which again creates underruns.
Another thing I guess could help is if there was a way to make the swscale cpu filter(or if there is another suitable filter?) use multiple threads? Right now it seems to only use one thread at a time, maxing out at 99% on my Ryzen 3950x (3,5GHz, 32 threads).
Example ffmpeg output:
$ ffmpeg -loglevel verbose -f decklink -raw_format rgb10 -i "Blackmagic Card 1" -c:v hevc_nvenc -preset medium -profile:v main10 -cbr 1 -b:v 20M -f nut - > /dev/null
--
Stream #0:1: Video: r210, 1 reference frame, gbrp10le(progressive), 4096x2160, 6635520 kb/s, 25 tbr, 1000k tbn, 1000k tbc
--
[graph 0 input from stream 0:1 # 0x4166180] w:4096 h:2160 pixfmt:gbrp10le tb:1/1000000 fr:25000/1000 sar:0/1
[auto_scaler_0 # 0x4168480] w:iw h:ih flags:'bicubic' interl:0
[format # 0x4166080] auto-inserting filter 'auto_scaler_0' between the filter 'Parsed_null_0' and the filter 'format'
[auto_scaler_0 # 0x4168480] w:4096 h:2160 fmt:gbrp10le sar:0/1 -> w:4096 h:2160 fmt:yuv444p16le sar:0/1 flags:0x4
[hevc_nvenc # 0x4139640] Loaded Nvenc version 11.0
--
Stream #0:0: Video: hevc (Rext), 1 reference frame (HEVC / 0x43564548), yuv444p16le(tv, progressive), 4096x2160 (0x0), q=2-31, 2000 kb/s, 25 fps, 51200 tbn
--
[decklink # 0x40f0900] Decklink input buffer overrun!:02.52 bitrate= 30471.3kbits/s speed=0.627x
Related
We have a .ts input file that contains (among other streams) a video stream and MISB 0604-compliant KLV data stream. The output of ffprobe for these stream are:
Stream #0:0[0x111]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], 29.97 fps, 29.97 tbr, 90k tbn, 59.94 tbc
...
Stream #0:2[0x1001]: Data: klv (KLVA / 0x41564C4B)
We are hoping to extract every Nth frame of the video as a .tiff. We also need to associate each of these frames with a corresponding KLV packet from the MISB 0604-compliant data stream.
The following command that select filters and adjusts the original FPS by the corresponding ratio does result in the expected frames being saved out as TIFF (in this case the original video has 1187 frames, and I expect to get 12 frames from the select filter).
ffmpeg -y -i 2205Z.ts -map 0:0 -vf "select='not(mod(n,100))'",fps=30000/1001/100 -compression_algo raw -pix_fmt rgb24 %05d.tif
However I can't seem to get any filters working on the data stream. For example using filter:d does not throw an error, but also doesn't seem to actually filter. My question is whether ffmpeg can be used to save out a "downsampled" data stream corresponding to the downsampling operations performed on the video stream above?
Using a recent git master build, run
ffmpeg -i 2205Z.ts -map 0:2 -bsf "noise=drop=mod(n\,100)" -f segment -segment_format data -segment_time 0.01 %d.bin
The noise bsf uses an expression to drop, not select i.e. select=EXPR === drop=not(EXPR).
Add -loglevel verbose to see details about which packets are kept, in a format like this:
[noise # 000001cd418a68c0] Stream #2 packet 1099 pts 3420417 - amount 0 drop 1
[noise # 000001cd418a68c0] Stream #2 packet 1100 pts 3423419 - amount 0 drop 0
[noise # 000001cd418a68c0] Stream #2 packet 1101 pts 3426423 - amount 0 drop 1
I record a stream similar to this (but I pipe the image to stdout and process the frames in real time):
ffmpeg -debug_ts -y -i http://my_stream.com -strict -2 out.mp4
Now I need to map my frames to the pkt_pts. This pkt_pts I can get by running:
ffprobe -i http://my_stream.com -show_frames -select_streams v:0
Although starting ffprobe is not an option since I need to excactly know which pkt_pts belongs to what frame.
Therefore I'm using the -debug_ts in my ffmpeg call, which gives me a lot of output to stderr, which I then parse out. The only output line which matches the number of frames is the one which starts with: muxer <- type:video This line does contain a pkt_pts, but it always starts with 0 (not the global pkt_pts).
Is there a way to map a frame, to the global pkt_pts in real time with ffmpeg?
You can run your original command, and extract lines starting with
demuxer -> ist_index:X
where X is the index number of the stream you want.
In each line, e.g.
demuxer -> ist_index:12 type:video next_dts:10851838034 next_dts_time:10851.8 next_pts:10851705034 next_pts_time:10851.7 pkt_pts:976665480 pkt_pts_time:10851.8 pkt_dts:976665480 pkt_dts_time:10851.8 off:0 off_time:0
-pkt_pts divided by stream timebase, 90000 for TS input, will give you full precision pkt_pts_time. If you don't know the timebase, see the input dump for your stream
Stream #0:0: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p, 768x432, 29.97 tbr, 90k tbn, 59.94 tbc
and use the tbn value.
Be aware that your ffprobe command selects the first video stream, but your ffmpeg command leaves it to ffmpeg to select streams, which may not select the same streams.
I'm trying to generate h.264 video from raw 2-byte gray video (14-bit range encoded in 16-bit values). I can do something like:
ffmpeg -f rawvideo -pix_fmt gray16le -s:v 1280x720 -r 60 -i input.raw -c:v libx264 output.mp4
And I get video but it's pretty dark, not sure if it's clipping, doing a linear remap, or storing the 16-bit data and VLC is doing the remap. ffprobe is reporting Video: h264 (High 4:4:4 Predictive) (avc1 / 0x31637661), yuvj444p(pc), 1280x720, 108 kb/s, 60 fps, 60 tbr, 15360 tbn, 120 tbc
I was figuring I'd use the tonemap filter to make a better mapping. I added a filter before the output file with -vf.
tonemap=hable errors Impossible to convert between the formats supported by the filter 'graph 0 input from stream 0:0' and the filter 'auto_scaler_0'
zscale=transfer=linear,tonemap=hable errors Impossible to convert between the formats supported by the filter 'Parsed_tonemap_1' and the filter 'auto_scaler_1'
zscale=transfer=linear,tonemap=hable,zscale=transfer=bt709,format=yuvj444p errors code 3074: no path between colorspaces
I'm not sure where to proceed from here...
I am streaming a static png file with ffmpeg and it uses basically all my CPU. It seems a bit greedy to me, and even though I limited the fps on the input and output size, I am seeing a huge fps printed out.
w:\ffmpeg\bin>ffmpeg.exe -loop 1 -framerate 1 -i w:\colorbar2.png -r 10 -vcodec libx264 -pix_fmt yuv420p -r 10 -f mpegts udp://127.0.0.1:10001?pkt_size=1316
ffmpeg version N-68778-g5c7227b Copyright (c) 2000-2014 the FFmpeg developers
built on Dec 29 2014 22:12:54 with gcc 4.9.2 (GCC)
Input #0, png_pipe, from 'w:\colorbar2.png':
Duration: N/A, bitrate: N/A
Stream #0:0: Video: png, pal8, 320x240 [SAR 3779:3779 DAR 4:3], 1 fps, 1 tbr, 1 tbn, 1 tbc
[libx264 # 00000000002fb320] using SAR=1/1
[libx264 # 00000000002fb320] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
[libx264 # 00000000002fb320] profile High, level 1.2
Output #0, mpegts, to 'udp://127.0.0.1:10001?pkt_size=1316':
Metadata:
encoder : Lavf56.16.102
Stream #0:0: Video: h264 (libx264), yuv420p, 320x240 [SAR 1:1 DAR 4:3], q=-1--1, 10 fps, 90k tbn, 10 tbc
Metadata:
encoder : Lavc56.19.100 libx264
Stream mapping:
Stream #0:0 -> #0:0 (png (native) -> h264 (libx264))
Press [q] to stop, [?] for help
frame=561310 fps=579 q=25.0 size= 144960kB time=15:35:25.80 bitrate= 21.2kbits/s dup=505179 drop=0
As you can see the frame counter goes up quickly and fps=579 is reported on the last line. I am confused now, what does that fps mean, if above the low frame per secs are also mentioned (output 10fps, input 1 fps)
What am I doing wrong and how could I reduce CPU load more given that it's a static file that is being streamed.
Thanks!
ffmpeg attempts to decode and encode as fast as it can. Just because you set the output to be 10 frames per second does not mean that it will (de|en)code realtime at 10 frames per second.
Try the -re input option. From the ffmpeg cli-tool documentation:
Read input at native frame rate. Mainly used to simulate a grab device
or live input stream (e.g. when reading from a file). Should not be
used with actual grab devices or live input streams (where it can
cause packet loss). By default ffmpeg attempts to read the input(s)
as fast as possible. This option will slow down the reading of the
input(s) to the native frame rate of the input(s). It is useful for
real-time output (e.g. live streaming).
Example:
ffmpeg.exe -re -loop 1 -framerate 10 -i w:\colorbar2.png -c:v libx264 \
-tune stillimage -pix_fmt yuv420p -f mpegts udp://127.0.0.1:10001?pkt_size=1316
i am trying to convert some different video formats to flv using ffmpeg. But it seems that only some videos go through.
ffmpeg -i /var/www/tmp/91640.avi -ar 22050 -ab 32 -f flv /var/www/videos/91640.flv
here is some debug info:
Seems stream 0 codec frame rate differs from container frame rate: 23.98 (65535/2733) -> 23.98 (5000000/208541)
Input #0, avi, from '/var/www/tmp/91640.avi':
Duration: 00:01:12.82, start: 0.000000, bitrate: 5022 kb/s
Stream #0.0: Video: mpeg4, yuv420p, 1280x528 [PAR 1:1 DAR 80:33], 23.98 tbr, 23.98 tbn, 23.98 tbc
Stream #0.1: Audio: ac3, 48000 Hz, 5.1, s16, 448 kb/s
WARNING: The bitrate parameter is set too low. It takes bits/s as argument, not kbits/s
Output #0, flv, to '/var/www/videos/91640.flv':
Stream #0.0: Video: flv, yuv420p, 1280x528 [PAR 1:1 DAR 80:33], q=2-31, 200 kb/s, 90k tbn, 23.98 tbc
Stream #0.1: Audio: adpcm_swf, 22050 Hz, 5.1, s16, 0 kb/s
Stream mapping:
Stream #0.0 -> #0.0
Stream #0.1 -> #0.1
Error while opening codec for output stream #0.1 - maybe incorrect parameters such as bit_rate, rate, width or height
also, if i try to grab one frame ad convert it to jpeg i get an error as well
ffmpeg -i /var/www/tmp/91640.avi -an -ss 00:00:03 -t 00:00:01 -r 1 -y /var/www/videos/91640.jpg
debug info
...
[mpeg4 # 0x1d7d810]Invalid and inefficient vfw-avi packed B frames detected
av_interleaved_write_frame(): I/O error occurred
Usually that means that input file is truncated and/or corrupted.
im thinking that the image fails because the video conversion failed in the first place, not sure though
any ideas what goes wrong?
Bits, not kbits
From your console output:
WARNING: The bitrate parameter is set too low. It takes bits/s as argument, not kbits/s
Use 32k, not just 32.
Only stereo or mono is supported
The encoder adpcm_swf ony supports mono or stereo, so add -ac 2 as an output option. The console output would have suggested this if you were using a recent ffmpeg build.
Use -vframes 1 for single image outputs
Instead of -t 00:00:01 -r 1 use -vframes 1.
A better encoder
Instead of using the encoders flv and adpcm_swf, I recommend libx264 and libmp3lame:
ffmpeg -i input -vcodec libx264 -preset medium -crf 23 -acodec libmp3lame -ar 44100 -q:a 5 output.flv
-preset – Controls the encoding speed to compression ratio. Use the slowest preset you have patience for: ultrafast,superfast, veryfast, faster, fast, medium, slow, slower, veryslow.
-crf – Constant Rate Factor. A lower value is a higher quality. Range is 0-51 for this encoder. 0 is lossless, 18 is roughly "visually lossless", 23 is default, and 51 is worst quality. Use the highest value that still gives an acceptable quality.
-q:a – Audio quality for libmp3lame. Range is 0-9 for this encoder. A lower value is a higher quality.
Also see
FFmpeg and x264 Encoding Guide
Encoding VBR (Variable Bit Rate) mp3 audio