ffmpeg libx264 encoder different machines different output - ffmpeg

I'm new to development with ffmpeg and I was wondering if it is possible to get the same bit exact output by running ffmpeg with the libx264 encoder on different machines (compiled using different compilers) with the same input file and same configuration options?
If it is possible, are there additional configuration options that need to be set when compiling libx264/ffmpeg? If it is not possible, why?
Below is the output from my running two versions off ffmpeg with libx264 that yield different MD5 hashes of the outputs.
ffmpeg installed via homebrew on OSX
ffmpeg version 3.4 Copyright (c) 2000-2017 the FFmpeg developers
built with Apple LLVM version 9.0.0 (clang-900.0.38)
configuration: --prefix=/usr/local/Cellar/ffmpeg/3.4 --enable-shared --enable-pthreads --enable-version3 --enable-hardcoded-tables --enable-avresample --cc=clang --host-cflags= --host-ldflags= --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --disable-lzma --enable-nonfree
ffmpeg on Alpine Linux (in a Docker container based on https://github.com/jrottenberg/ffmpeg/blob/master/docker-images/3.4/alpine/Dockerfile)
ffmpeg version 3.4 Copyright (c) 2000-2017 the FFmpeg developers
built with gcc 6.2.1 (Alpine 6.2.1) 20160822
configuration: --enable-shared --enable-pthreads --enable-version3 --enable-hardcoded-tables --enable-avresample --enable-gpl --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libxvid --disable-lzma --enable-nonfree --prefix=/opt/ffmpeg
Test run with ffmpeg on OSX
ffmpeg -i seg.ts -c:v libx264 -s 1280x720 -minrate 6000k -maxrate 6000k -bufsize 6000k -r 60 -threads 1 -f md5 -
Output
MD5=b7c84c0bae5da6749e389a5b69d88582
Test run with ffmpeg on Alpine Linux (in a Docker container based on https://github.com/jrottenberg/ffmpeg/blob/master/docker-images/3.4/alpine/Dockerfile)
docker run --rm -v /code/:/tmp/workdir -w=/tmp/workdir ffmpeg -i seg.ts -c:v libx264 -s 1280x720 -minrate 6000k -maxrate 6000k -bufsize 6000k -r 60 -threads 1 -f md5 -
Output
MD5=c46af8fbdbbb2bfbb9f7042ab28accfc

If you use different builds of libx264 on different CPUs nobody can guarantee you bit identical output. With same libx264 build on different CPUs (different instruction set extensions support) you also can get different output but can try to fix it by using --cpu-independent param (for ffmpeg through -x264-params) during encoding (on both).
As for why it can be not possible with different libx264 builds is because different compilers can generate different float-math code which will differ in rounding errors (use x87 or sse) and so nothing is guaranteed. You can try to minimize probability by disabling AQ, MBTree or using CQP ratecontrol but that is all.
UPDATE. Also don't forget about using same threads number on different CPUs instead of auto.

I ran into this issue when testing on docker machines running on i7 vs i9s. At first I tried disabling different cpu capabilities in https://github.com/FFmpeg/FFmpeg/blob/master/configure
But then I noticed that both machines were using the same cpu capabilities. I saw that from the logs outputted by running the command. So then I saw in the logs that they had different thread and lookahead_thread counts. So I updated by specifying the counts with this param -x264-params threads=6:lookahead_threads=1
See this post https://streaminglearningcenter.com/blogs/ffmpeg-command-threads-how-it-affects-quality-and-performance.html

Related

ffmpeg build on mac with videotoolbox enabled becomes unportable

If i configure ffmpeg this way:
./configure --disable-everything --enable-static --disable-shared \
--enable-gpl --enable-nonfree --enable-encoder=h264_videotoolbox,aac \
--enable-muxer=mp4 --enable-protocol=file --enable-libfdk-aac
--enable-videotoolbox --disable-autodetect
it works for my purposes (allows to encode h264 video with aac audio on Mac's videotoolbox - an Apple QSV toolkit), but if i send it to any other computer except the one it was built on, it fails with something like this:
dyld: Symbol not found: _kCVImageBufferTransferFunction_ITU_R_2100_HLG
Referenced from: /Users/admin/Downloads/./ffmpeg
Expected in: /System/Library/Frameworks/CoreVideo.framework/Versions/A/CoreVideo
in /Users/admin/Downloads/./ffmpeg
Abort trap: 6
if i rebuild it this way:
./configure --disable-everything --enable-static --disable-shared
--enable-gpl --enable-nonfree --enable-encoder=aac
--enable-muxer=mp4 --enable-protocol=file --enable-libfdk-aac
--disable-autodetect
so with everything else but videotoolbox removed, it runs successfully on any other computer, so apparently ffmpeg needs to carry along something it doesn't, for videotoolbox to work...
i am actually building a C++ app with ffmpeg's static libraries, but explaining what i do there will be a very long story and error message produced is exactly the same if i run it on different machines, so i better illustrate it on example of ffmpeg console utility itself.
what are the configure switches i need to do to make the ffmpeg build portable please?
Problem turned out to be my macos version (10.14), the API mentioned is since 10.13, so it didn't work on earlier version i tried. Fixed by rebuilding ffmpeg on 10.10.

https support for ffmpeg centos?

I have installed ffmpeg on centos.But when I feed url with https like
ffmpeg -i https://s3-us-west-2.amazonaws.com/bucket/check.mp4 video.mp4
Error come
https protocol not found, recompile FFmpeg with openssl, gnutls, or securetransport enabled.
I know i have to enable this --enable-openssl,but when i am doing like this
PKG_CONFIG_PATH="/ffmpeg_build/lib/pkgconfig"
./configure --prefix="$HOME/ffmpeg_build" --extra-cflags="-I$HOME/ffmpeg_build/include" --extra-ldflags="-L$HOME/ffmpeg_build/lib -ldl" --bindir="$HOME/bin" --pkg-config-flags="--static" --enable-gpl --enable-nonfree --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-openssl
It give me error like:
ERROR: opus not found
What should I do to enable https? Please help
The guide FFmpeg Wiki: Compile ffmpeg on CentOS halfelf referred to just got some cleanups, so try again. It will probably just be easiest to start over.
Install gnutls-devel or openssl-devel.
Remove old junk: rm -rf ~/ffmpeg_build ~/ffmpeg_sources ~/bin/{ffmpeg,ffprobe,ffserver,lame,vsyasm,x264,yasm,ytasm}
Re-run guide.
Stop at the FFmpeg section, add --enable-gnutls or --enable-openssl to the ./configure line, then continue following the guide.
Or forget compiling and just download a static build of ffmpeg: it has HTTPS support.

How to install libtheora for MacOSX (ffmpeg)

My ffmpeg version is missing 'libtheora' codec and I downloaded this version of 'libtheora' to make it work. http://www.theora.org/downloads/
The problem is that after extracting the zip file, I dont know how to install it. The README file does not give a lot of details. Do I just run make?
Anyone that did this before?
It is very simple with Homebrew:
brew install ffmpeg2theora
But actually it does not allow you to convert to OGG/OGV format after this. What actually helped me to convert MP4 to OGV format in the end was compiling ffmpeg with libtheora support. Detailed explanation could be found here ffmpeg Mac OSX compilation guide. And after this I finally was able to run the following command:
ffmpeg -i input.mkv -codec:v libtheora -qscale:v 7 -codec:a libvorbis -qscale:a 5 output.ogv
You can install libtheora for ffmpeg with brew
brew install theora
To use it in ffmpeg you should recompile ffmpeg. Source of solution: https://trac.ffmpeg.org/wiki/CompilationGuide/MacOSX
brew install automake fdk-aac git lame libass libtool libvorbis libvpx opus sdl shtool texi2html theora wget x264 xvid yasm
git clone git://source.ffmpeg.org/ffmpeg.git ffmpeg
cd ffmpeg
./configure --prefix=/usr/local --enable-gpl --enable-nonfree --enable-libass --enable-libfdk-aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libxvid
make && sudo make install
as for me it works like magic.
after reading a long mailinglist and getting sad because people are rude i finally solved this problem for me. it seems to be necessary to specify the lib paths with --extra-ldflags and --extra-cflags. so in my case it looked like this:
./configure --prefix=/usr/local --enable-gpl --enable-nonfree \
--enable-libass --enable-libfdk-aac --enable-libfreetype \
--enable-libopus --enable-libtheora --enable-libvorbis \
--enable-libvpx --enable-libx264 --enable-libxvid
--extra-ldflags="-L/usr/local/Cellar/lame/3.99.5/lib \
-L/usr/local/Cellar/libogg/1.3.2/lib \
-L/usr/local/Cellar/theora/1.1.1/lib \
-L/usr/local/Cellar/libvorbis/1.3.5/lib \
-L/usr/local/Cellar/xvid/1.3.4/lib" \
--extra-cflags="-I/usr/local/Cellar/lame/3.99.5/include \
-I/usr/local/Cellar/libogg/1.3.2/include \
-I/usr/local/Cellar/theora/1.1.1/include \
-I/usr/local/Cellar/libvorbis/1.3.5/include \
-I/usr/local/Cellar/xvid/1.3.4/include"
depending on your required libs and versions this may be different of course.
Have you tried Macports (http://www.macports.org/) to install ffmped or libtheora?
If you don't have it ...download an install it.
then go into your console and execute this:
sudo port install libtheora

FFMPEG, pixelated frames, and webm format

It seems like every time I split a .webm video, half or more of the frames are pixelated and distorted.
My configuration looks like the following:
FFmpeg version 0.6.6-4:0.6.6-0ubuntu0.11.04.1, Copyright (c) 2000-2010 the Libav developers
built on Jun 12 2012 16:35:16 with gcc 4.5.2
configuration: --extra-version=4:0.6.6-0ubuntu0.11.04.1 --prefix=/usr --enable-avfilter --enable-avfilter-lavf --enable-vdpau --enable-bzlib --enable-libgsm --enable-libschroedinger --enable-libspeex --enable-libtheora --enable-libvorbis --enable-pthreads --enable-zlib --enable-libvpx --disable-stripping --enable-runtime-cpudetect --enable-vaapi --enable-gpl --enable-postproc --enable-swscale --enable-x11grab --enable-libdc1394 --enable-shared --disable-static
Does anyone have any experience with something similar to this? When I play the same videos with a standard ffmpeg player on the same system, they look just fine.
If you're just splitting a .webm file into 2 .webm files, then you need to add -vcodec copy -acodec copy to your command line, i.e. something like
ffmpeg -i input.webm -vcodec copy -acodec copy -ss 10 -t 10 output.webm
Otherwise ffmpeg will also transcode the video, using the default bitrate, which is low, and this causes the pixelation. Also, it makes the splitting much, much slower.

how to compile ffmpeg with librtmp in macos?

I've tried installing ffmpeg in mac lion through homebrew like this:
brew install --use-clang ffmpeg --with-tools --with-ffplay --enable-librtmp
but the compile flags ended up as
configuration: --disable-debug --prefix=/usr/local/Cellar/ffmpeg/0.6.2 --enable-shared --enable-pthreads --enable-nonfree --enable-gpl --disable-indev=jack --enable-libx264 --enable-libfaac --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libxvid --enable-libfaad
What's the correct way to install ffmpeg with librtmp? I also want to run libavfilters like blackdetect.
Doing a direct install always failed with RTMP_Socket() missing so I was hoping for a homebrew solution.
You can include librtmp in the build by adding the --with-rtmpdump flag (after installing RTMPDump). These are the commands I used:
brew install rtmpdump
brew install ffmpeg --with-x265 --with-rtmpdump
Now I have rtmp, rtmpe, rtmps, rtmpt and rtmpte as available protocols! :)
Finally able to compile! Think the key might have been running
export PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/local/lib/pkgconfig
just before configure but not sure since i was getting different errors according to the flags. The script I ended up running (modified from other sources) is at github: https://gist.github.com/2863964

Resources