I know that one can extract the motion vectors from an h264 encoded via by first setting the flag
av_dict_set(&opts, "flags2", "+export_mvs", 0);
then you can query the side-data for the motion vectors by doing this
sd = av_frame_get_side_data(frame, AV_FRAME_DATA_MOTION_VECTORS);
When I looked online to see if you can do something similar with HEVC encoded videos, I wasn't able to find any information. All I found was this by the definition of "AV_FRAME_DATA_MOTION_VECTORS"
Motion vectors exported by some codecs (on demand through the
export_mvs flag set in the libavcodec AVCodecContext flags2 option).
The data is the AVMotionVector struct defined in
libavutil/motion_vector.h.
but there was no information on exactly which codecs export this motion vector information. How would I go about finding this out?
If I'm not mistaken h264 is the only codec to print Motion Estimation Vectors.
I would suggest trying out the video filter mestimate.
Also, if you want to have a better ideia what's going on in ffmpeg, check the function ff_print_debug_info2 in libavcodec/mpegvideo.c
Related
I'm want to extract the motion vector information from the encoded stream without performing a full decode of the frame. I'm using ffmpeg library and aware of extracting the information using the av_frame_get_side_data. Unfortunately, it does a full decode of the frame before providing the AV_FRAME_DATA_MOTION_VECTORS. My hunch is it should be possible to parse the h264 to extract out the motion vectors only, ignoring all the other data. Any direction/pointers will be very helpful!
I want to change gop dynamic to adapt to player. Smaller gop help for reduce first screen cost; Big gop help to reduce bitrate to save bandwidth.
I found nvenc has a function which can solve the problem above.
https://docs.nvidia.com/video-technologies/video-codec-sdk/nvenc-video-encoder-api-prog-guide/#forcing-current-frame-to-be-encoded-as-intra-frame
But I'm using ffmpeg. Is there any similar function in ffmpeg?
If the AVFrame->pict_type is set to AV_PICTURE_TYPE_I, then the NVENC encoders, by default, will encode it as an intra frame. They also have a private option to force it as an IDR frame.
For the latter, add before opening the encoder.
av_opt_set(avctx->priv_data, "forced_idr", "1", 0);
Since what ffmpeg does generally is read either an audio / image / video file of a given Codec & then converts it to a different Codec, it must have at some point hold to raw values of the media files, which:
for Audio the raw Samples (2*44100 Samples) in case of Stereo Audio as int / float
rgba pixel data for images (as int8 array)
for video, array of images & linked Audio streams
How can I essentially just read those raw values & get them in Memory / on Disk in lets say C++ / Python / Java?
best regards
ffmpeg is just a command line tool. The libraries behind the scene are part of the Libav* family. i.e. libavformt, libavcodec, libavtuil, swsscale, swresample, etc.
You can use those libraries directly in C or C++, or use some soft of FFI in other languages. (you can also pipe some raw formats such as y4m)
Going from a file name to a frame buffer will take a little more code than just "open()" But there are many tutorials online, and other stackoverflow questions that answer that.
Note:
rgba pixel data for images (as int8 array)
RGBa is not very common format for video. It's usually YUV, and uasually uses sub sampling for the chroma planes. Its also usually planner, so instead of a int8 array its a array of pointers pointing to several int8 arrays
i'm using the HEVC codec with ffmpeg(version 2.7.2 ). Since I'm not able to write a one-line command in order to read/save the quantized transformed coefficients for each CU and frame, I would like to know if anyone can help me modify the original files in the libavcodec library?
No CLI option available to save the quantized coefficients. If you need to modify the source code for this purpose, below function is the right place to start with
hls_transform_unit() # libavcodec\hevc.c
I need to extract information about motion vectors and DC coefficients from an MPEG4 video. I've searched relevant sites and topics and I found that a good solution is to work with the code of ffmpeg codec. Especially the ff_print_debug_info function in libavcodec/mpegvideo.c calculates relevant information.
However, I'm new on the C/C++ field and if there is any example code that describes or explains how to extract MVs and DC coefficients that would be very helpful.
In the more recent version I use (FFmpeg 0.10.2 from http://ffmpeg.org/download.html) there is another file mpeg4videodec.c. Is there any chance to retrieve the needed information by this code?
Check out this piece of code: http://www.princeton.edu/~jiasic/cos435/motion_vector.c
It gives the basic idea of accessing motion vectors withing ffmpeg-decoded frame.
As for DC coefficients, I don't see any other way than to inject your own code into the decoder to dump coefficients as you decode.