how to use ffmpeg with mjpeg_vaapi encoder (with hardware acceleration)? - ffmpeg

Question

When I use ffmpeg with mjpeg encoder without hardware acceleration, everything works fine. I use following command:
ffmpeg -nostdin -rtsp_transport udp_multicast -allowed_media_types 'video' -i 'rtsp://MYIP' -ss 00:00:0.00 -f image2pipe -q:v 24 -vf scale=-2:480,format=yuv420p -c:v mjpeg -vframes 1 pipe:1
But, when I try to use mjpeg with hardware acceleration, I always get error:
ffmpeg -nostdin -rtsp_transport udp_multicast -allowed_media_types 'video' -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i 'rtsp://MYIP' -ss 00:00:0.00 -f image2pipe -q:v 24 -vf scale=-2:480,format=yuv420p -c:v mjpeg_vaapi -vframes 1 pipe:1
ffmpeg version N-91514-gc51e0cd Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609
configuration: --pkg-config-flags=--static
libavutil 56. 18.102 / 56. 18.102
libavcodec 58. 21.106 / 58. 21.106
libavformat 58. 17.101 / 58. 17.101
libavdevice 58. 4.101 / 58. 4.101
libavfilter 7. 26.100 / 7. 26.100
libswscale 5. 2.100 / 5. 2.100
libswresample 3. 2.100 / 3. 2.100
libva info: VA-API version 0.39.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_39
libva info: va_openDriver() returns 0
Input #0, rtsp, from 'rtsp://MYIP':
Metadata:
title : Session streamed with GStreamer
comment : rtsp-server
Duration: N/A, start: 0.180600, bitrate: N/A
Stream #0:0: Video: h264 (Main), yuvj420p(pc, progressive), 800x600, 50 fps, 50 tbr, 90k tbn, 100 tbc
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> mjpeg (mjpeg_vaapi))
Impossible to convert between the formats supported by the filter 'graph 0 input from stream 0:0' and the filter 'auto_scaler_0'
Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented
Error while processing the decoded data for stream #0:0
Conversion failed!
I tried with some additional options but it still keeps failing:
ffmpeg -nostdin -rtsp_transport udp_multicast -allowed_media_types 'video' -hwaccel vaapi -vaapi_device /dev/dri/renderD128 -hwaccel_output_format vaapi -i 'rtsp://MYIP' -ss 00:00:0.00 -f image2pipe -q:v 24 -vf "scale_vaapi=w=800:h=600" -b:v 700k -r 25 -vcodec mjpeg_vaapi -vframes 1 pipe:1
ffmpeg version N-91514-gc51e0cd Copyright (c) 2000-2018 the FFmpeg developers
built with gcc 5.4.0 (Ubuntu 5.4.0-6ubuntu1~16.04.10) 20160609
configuration: --pkg-config-flags=--static
libavutil 56. 18.102 / 56. 18.102
libavcodec 58. 21.106 / 58. 21.106
libavformat 58. 17.101 / 58. 17.101
libavdevice 58. 4.101 / 58. 4.101
libavfilter 7. 26.100 / 7. 26.100
libswscale 5. 2.100 / 5. 2.100
libswresample 3. 2.100 / 3. 2.100
libva info: VA-API version 0.39.0
libva info: va_getDriverName() returns 0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/i965_drv_video.so
libva info: Found init function __vaDriverInit_0_39
libva info: va_openDriver() returns 0
Input #0, rtsp, from 'rtsp://MYIP':
Metadata:
title : Session streamed with GStreamer
comment : rtsp-server
Duration: N/A, start: 0.179933, bitrate: N/A
Stream #0:0: Video: h264 (Main), yuvj420p(pc, progressive), 800x600, 50 fps, 50 tbr, 90k tbn, 100 tbc
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> mjpeg (mjpeg_vaapi))
[mjpeg_vaapi # 0x374cd80] Encoding entrypoint not found (12 / 7).
Error initializing output stream 0:0 -- Error while opening encoder for output stream #0:0 - maybe incorrect parameters such as bit_rate, rate, width or height
Conversion failed!
Can please anyone suggest right options to make it work?

Answer 1

so the problem was that my Intel CPU platform was old (Broadwell) and JPEG encoding is supported starting Braswell. Please see all supported codecs by platform on the page
After running the same command on the machine with Skylake CPU on it, it work from the first try :)

Answer 2

When you trying to leverage Intel's hardware encoding, you need to check TWO things
rather your particular intel support the encoder you want
rather your ffmpeg binary fetched from somewhere compiled with the hardware acceleration library or not.
To find out what kind of encoder your Intel support, you simply type:
vainfo|grep -i enc
it will list the encoder which your processor support. In my case, for example, it yields the following:
libva info: VA-API version 1.13.0
libva info: Trying to open /usr/lib/x86_64-linux-gnu/dri/iHD_drv_video.so
libva info: Found init function __vaDriverInit_1_13
libva info: va_openDriver() returns 0
VAProfileMPEG2Simple : VAEntrypointEncSlice
VAProfileMPEG2Main : VAEntrypointEncSlice
VAProfileH264Main : VAEntrypointEncSlice
VAProfileH264Main : VAEntrypointEncSliceLP
VAProfileH264High : VAEntrypointEncSlice
VAProfileH264High : VAEntrypointEncSliceLP
VAProfileJPEGBaseline : VAEntrypointEncPicture
VAProfileH264ConstrainedBaseline: VAEntrypointEncSlice
VAProfileH264ConstrainedBaseline: VAEntrypointEncSliceLP
VAProfileHEVCMain : VAEntrypointEncSlice
in my case, my intel cpu does have mjpeg encoding support.
if you can't find anything resembles mjpeg, then, you can pretty much give up the idea of using intel's hardware acceleration to accomplish what you want on that machine.
The next thing you need is to find out rather your ffmpeg binary support Intel's vaapi at the first place. First is to type:
ffmpeg -hide_banner -hwaccels
if one of those line has "vaapi," then, it means the ffmpeg compiled with Intel's hardware acceleration library. If your vainfo command has what you want but the above command doesn't display vaapi, then, you need to get another copy of ffmpeg binary. either from a different source, or compile it on your own.
Next, is to list all the ffmpeg encoder that actually uses vaapi, and hoping one of them resembles mjpeg:
ffmpeg -hide_banner -encoders|grep -i vaapi
In my case, I do have it:
V....D h264_vaapi H.264/AVC (VAAPI) (codec h264)
V....D hevc_vaapi H.265/HEVC (VAAPI) (codec hevc)
V....D mjpeg_vaapi MJPEG (VAAPI) (codec mjpeg)
V....D mpeg2_vaapi MPEG-2 (VAAPI) (codec mpeg2video)
V....D vp8_vaapi VP8 (VAAPI) (codec vp8)
V....D vp9_vaapi VP9 (VAAPI) (codec vp9)
So, in my case, the encoder is there.
if both vainfo and ffmpeg -hwaccels yield what you need but this last step doesn't, then, you need to get another ffmpeg binary from somewhere.
good luck.