I tried to encode a series of png files into a HEVC codec video with ffmpeg (version modern enough) in these parameters:
ffmpeg -hwaccel cuda -r 60000/1001 -f image2 -i output%05d.png -c:v hevc_nvenc -preset slow -profile:v main10 -level 6.2 -pix_fmt yuv444p16le output.mkv
Then I got a mkv video file and I found that it cannot be decoded with DXV2 but with LAV Decoder, which takes up really lot of my CPU. However, I also found that the other HEVC codec video file, which has similar property of my video, can be decoded with DXV2 Decoder using my GPU. Could anyone help me find a way that, make a HEVC codec video can be decoded using GPU? Thanks a lot.
I'm going to guess your issue is with -pix_fmt yuv444p16le. I don't know what format your PNG is, but it's typically 8-bits/component RGB. You're upsampling it to 16-bits/component 4:4:4 YUV and then using that as input to the HEVC encoder at whatever is its closest-supported neighbour (10-bits/component YUV-4:4:4, according to comments in the source code).
4:4:4 is sometimes not supported by hardware decoders (see here). Also see this quote from here:
Many existing end-client devices/GPUs/SoC support hardware decode 4:2:0 but not 4:4:4 and whilst newer hardware and GPUs are introducing 4:4:4 decoders many existing devices only support 4:2:0 decode.
I would try -pix_fmt p010le, which should result in YUV-4:2:0 10-bits/component, which should work fine with all hardware decoders.
Related
Several questions regarding hardware acceleration with ffmpeg (which I think is NOT same question as this one):
How can I tell if my version of ffmpeg support hardware acceleration or not, and what are the acceleration features it support?
From the output of an transcoding process, can I tell if ffmpeg is using hardware acceleration or not?
I am using ffmpeg to pull H265 video from camera, and convert it to H264 HLS chunks. The problem now is the transcoding process is too CPU intensive. I hope to use hardware acceleration. Due to the application, I cannot use expensive GPUs such as NVidia cuda platform. If using only Intel HD graphics comes with CPU, can I significantly lower CPU usage when transcoding H265 => H264? What is the estimated performance gain in %?
Thanks.
If your CPU support for example Intel Quick Sync than you can significantly reduce the CPU load using hardware acceleration (in my test case below it was from 90% using libx264 to 20% using h264_qsv). And after enabling the hardware accelerated decoding it reduced the CPU load from 20% to 4%. So in summary from 90% to 4%. But the load depends also on other things like bitrate, resolution and CPU/hardware.
Encoding
First of all you need to know, what encoders have been enabled in your FFmpeg version. To check this you can simply execute
ffmpeg -codecs
In this this you should find your target codec (h264):
[...]
DEV.LS h264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (decoders: h264 h264_qsv h264_cuvid ) (encoders: libx264 libx264rgb h264_amf h264_mf h264_nvenc h264_qsv )
[...]
Take your attention on the "encoders". Some are software-, some are hardware-encoders. This list is dependent to your OS and the enabled encoders during FFmpeg compilation. To get a short description of each encoder you can execute:
ffmpeg -encoders
[...]
V....D libx264 libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (codec h264)
V....D libx264rgb libx264 H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 RGB (codec h264)
V....D h264_amf AMD AMF H.264 Encoder (codec h264)
V....D h264_mf H264 via MediaFoundation (codec h264)
V....D h264_nvenc NVIDIA NVENC H.264 encoder (codec h264)
V..... h264_qsv H.264 / AVC / MPEG-4 AVC / MPEG-4 part 10 (Intel Quick Sync Video acceleration) (codec h264)
[...]
Note: not all encoders might actually work. E.g. if you have no NVIDIA graphics card the h264_nvenc will not work.
Here on my Windows machine I will choose the "h264_qsv" (Intel Quick Sync Video acceleration) encoder. On macOS you want to use "h264_videotoolbox":
ffmpeg -i mysource -c:v h264_qsv -c:a copy output.mp4
Pay attention on the output quality: libx264 might have a better quality by default than a hardware accelerated encoder. So make sure that you finetune the encoding using e.g. a defined bitrate (-b:v 6M for 6MBit).
This encodes the input using the h264 encoder of Intel Quick Sync (because of the -c:v option). Please find more details on
https://trac.ffmpeg.org/wiki/HWAccelIntro
Decoding
Using the option "-hwaccel auto" before the inputs (-i) tries to use hardware accelerated decoding as well:
ffmpeg -hwaccel auto -i mysource .....
If a hardware decoder is available it is automatically used. If not then FFmpeg falls back to the software decoder. Check the console output to see what happens:
[hevc # 00000176c91d0fc0] Using auto hwaccel type dxva2 with new default device.
I'm not sure if
x264/5
use CPU and if h264/5 use GPU and also if h265 is basically HEVC_NVENC for NVIDIA GPU acceleration. So, if you could give me more info about these encoding types it would be great. I understood that, summing up a lot, x26* use CPU and are slower but more accurate while h26* are the opposite but h265 is the most recent and optimal trade off.
Furthermore, I was trying to convert a video using GPU acceleration and my question is:
Does the following command tell to the GPU to use h265 to encode a video holding the same audio and at upgrading it at its maximum video quality? Furthermore, are there other ways to express the same command?
ffmpeg.exe -hwaccel_output_format cuda -i "input" -c:v hevc_nvenc -preset medium -rc constqp -qp 0 -c:a copy "output"
H.264 and H.265 are video codecs for compressing image frames to video.
H.264 is a very widely supported standard that's been around a long time. It provides good efficiency. H.265 is a newer codec which can reduce your bandwidth requirements for a given quality by around a quarter to a third, for most videos. (Or, you can increase quality for the same bandwidth.) H.265 is a patent-encrusted cluster (even more than H.264), and there are currently three groups who claim you must license with them to use it.
I'm not sure if x264/5 use CPU and if h264/5 use GPU
x264 and x265 are open source encoder implementations of the H.264 and H.265 codecs. They run on the CPU.
and also if h265 is basically HEVC_NVENC for NVIDIA GPU acceleration
Yes, that's basically correct. GPUs with NVENC support can encode H.265 video.
Does the following command tell to the GPU to use h265 to encode a video holding the same audio and at upgrading it at its maximum video quality?...
No. Maximum quality would actually be lossless encoding, which is apparently supported.
I'm using FFmpeg with Directshow input. The output is a series of single JPEG images. FFmpeg itself maps the stream to mjpeg and uses image2 for the output.
Is it possible to increase performance by using the Intel QuickSync Video (QSV) hardware acceleration for this process? The FFmpeg QuickSync Wiki actually lists JPEG encoding since Braswell.
This is what I tried so far:
ffmpeg -init_hw_device qsv=hw -filter_hw_device hw -f dshow -video_size 3840x2160 -framerate 25 -i "video=My Webcam" -vf hwupload=extra_hw_frames=64,format=qsv -vcodec mjpeg_qsv "C:\out\%d.jpg"
The command works, images are generated - but the GPU load seems to be the same as without any qsv options..?
Thanks!
I compared it for a h264 encoded video. First with the QSV jpeg encoder:
ffmpeg -c:v h264_qsv -i myvideo.mkv -vcodec mjpeg_qsv images/%06d.jpg
and afterwards without mjpeg_qsv:
ffmpeg -c:v h264_qsv -i myvideo.mkv images/%06d.jpg
The example is minimalistic and can be improved in many ways. In the picture you can see the load of the GPU. In the red box with mjpeg_qsv and the blue box without mjpeg_qsv. The execution time was also better with mjpeg_qsv, speed=3.34x vs speed=1.84x. Since you are using your webcam as an source, your pipeline is limited by the frame rate of the live video. So, your hardware will only process 25 frames per second (-framerate 25). Depending on the resolution and your hardware, this might be an easy job for your GPU, in both cases. Also, make sure you look for the Intel GPU and not your Nvidia/AMD GPU. If you still have the impression that you can't see a performance gain, please let us know both of your commands for comparison.
When extracting Audio streams using ffmpeg from containers such as MP4, how does ffmpeg increase bitrate, if it is higher than the source bitrate?
An example might be ffmpeg -i video.mp4 -f mp3 -ab 256000 -vn music.mp3. What does ffmpeg do if the incoming bitrate is 128k? Does it interpolate or default to 128k on the output music.mp3? I know this seems like not a so-called "programming question" but ffmpeg forum says it is going out of business and no one will reply to posts there.
During transcoding, ffmpeg (or any transcoder) decodes the input into an uncompressed version; for audio, that's PCM. The encoder compresses this PCM data. It has no idea of, or interaction with, the original source representation.
If no bitrate is specified, ffmpeg will use the default rate control mode and bitrate of the encoder. For MP3 or AAC, that's typically 128 kbps for a stereo output . Although it can be lower, like 96 kbps for Opus. Encoders typically adjust based on no. of output channels. So for a 6-ch output, it may be 320 kbps. If a bitrate is specified, that's used unless the value is invalid (beyond the encoder's range). In which case, the encoder will fallback on its default bitrate selection.
I'm trying to perform a real-time encoding of a video using HEVC with ffmpeg. I'm able to achieve the required performance when running the x265 encoder separately without the support of ffmpeg. This way the my system can perform the encoding at 30 frames per second. However, my requirement is to create a MPEG-TS stream with the encoded content and therefore, the video is encoded with the ffmpeg as follows:
ffmpeg -s:v 1280x720 -i input.yuv -c:v libx265 -x265-params crf=20:keyint=25:fps=25:preset=ultrafast -f mpegts out.ts
Strangely, the performance of the encoding is reduced drastically and I'm only able to achieve an encoding performance of just 10 frames per second.
Is this a problem of the multiplexing process within ffmpeg?. Could someone please help me to resolve this issue.
Thanks.
This is can be a reason q factor in FFmpeg. You need to compare q value of FFmpeg and x265 bin. This is my guess.