I have, as test cases, two mp4 videos: they seem to be encoded the same, but one works in Firefox and one doesn't:
bad.mp4 (doesn't work) view download
good.mp4 (does work) view download
(NOTE - I thought that the "view" links above would open them using the browser's built-in player, but it looks like they use Dropbox's video player, which DOES work in firefox. So, to see it in firefox, download it and then drag it into Firefox or right click and choose Firefox from "open with" - thanks!)
They both play in Chrome.
There is a long discussion on the Mozilla bug tracking site about a related issue:
https://bugzilla.mozilla.org/show_bug.cgi?id=1130450
So, I need to answer three questions:
1) What is the problem with bad.mp4?
2) How can I detect, programmatically (ie at the linux command line) which videos, of a library of videos, are going to have this problem?
3) How can I fix it, programmatically (eg with FFMPEG).
One thing I noticed was that in the good one, the video is on stream 0 and the audio stream 1, while on the bad one it's the other way round. I tried swapping the streams but it didn't seem to help.
EDIT: re the Baseline profile
It was suggested that FF might not like Baseline profile videos. I was able to identify some baseline profile videos that work and some that don't, so I think that's not the key problem. Here are a couple more samples:
Baseline_bad.mp4 (doesn't work) view download
Baseline_good.mp4 (does work) view download
First of all sorting mp4 tracks has nothing to do with that. Second, on Win10 with Firefox version 52.5.3 your bad.mp4 works just fine. However on MacOS using Firefox 69.03 bad.mp4 also plays but with a lot of artefacts.
The H.264 bitstream seems to be ok. I demultiplexed the bad mp4 with MP4Box MP4Box -raw 2 bad.mp4 which gives bad_track2.h264 and I decoded it with the reference software. Decoded just fine, without any problems, so the bitstream seems to be not a problem.
So it has to be the packaging into mp4. And indeed, if I package the bitstream back again using MP4Box -add bad_track2.h264 repackage.mp4, it plays just great. So this is a good news, since you don't have to re-encode (transcode) your videos, which will save you a lot of time.
Of course transcoding also solves it ffmpeg -i bad.mp4 transcode.mp4 in case you don't want to do it with MP4Box.
To answer 2 and 3. not 100 % sure if this is the same for all of your videos but the difference between good.mp4 and bad.mp4 is the compatible_brands which you can get with ffprobe. you can write a script which finds the videos with the same brand as in bad.mp4 and re-packages each of them using ffprobe, MP4Box.
I hope this helps.
Edit
To address the issue raised in #llogan's answer, regarding the Baseline AVC profile. IMHO this is very unlikely that the Baseline Profile is not supported, however can not be 100% sure about it since it may depend on the platform on which the Firefox is running.
From Mozilla's Web Video codec guide:
1: Firefox support for AVC is dependent upon the operating system's built-in or preinstalled codecs for AVC and its container in order to avoid patent concerns.
What is the problem with bad.mp4?
It's not the stream order because you can switch the order in good.mp4 and it still plays: ffmpeg -i good.mp4 -map 0:a -map 0:v -c copy switched.mp4
AUD NAL units
bad.mp4 may contain AUD NAL units Firefox doesn't like.
Baseline profile
Perhaps whatever decoder your Firefox is using doesn't like the Baseline H.264 profile. This is not the same as the far more common Constrained Baseline profile which is what good.mp4 is.
I'm not sure what version of Firefox you're using and what OS you are on so it is a guess.
How can I detect this problem?
Use ffprobe to determine the profile:
ffprobe -v error -select_streams v -of csv=p=0 -show_entries stream=profile input.mp4
If the issue is due to the AUD NAL units, and all of the Baseline videos have this issue, then this can still be useful. I didn't take the time to investigate further regarding the AUD NAL units so maybe it's a red herring.
How can I fix it?
If the AUD NAL units are the issue
Remux using the h264_metadata bitstream filter:
ffmpeg -i input.mp4 -bsf:v h264_metadata=aud=remove -map 0 -c copy -movflags +faststart output.mp4
This is fast because it avoids re-encoding.
You'll need FFmpeg 3.4 or later to use this.
See note below regarding -movflags +faststart.
If Baseline profile is the issue
You'll have to re-encode the video to change the profile:
ffmpeg -i input.mp4 -c:v libx264 -c:a copy -movflags +faststart output.mp4
See FFmpeg Wiki: H.264 for more encoding options.
I added -movflags +faststart because it seems like you'll be presenting these videos via progressive download playback. This option will allow the video to begin playing while it is still downloading.
I got an audio engine, which generates aac files. I want to mux this audio with some video. I'm using ffmpeg libraries to do just that - meaning, after the audio file is ready, I read it and mux it.
Now - for performance reasons, I don't want to wait until the audio engine completes the audio generation, I want the muxer to start reading the audio while it is being generated.
Can I achieve that using the FFMpeg libraries?
Which approach should I take?
Couldn't find any examples doing that
I am trying to implement a media player plugin in chrome using ffmpeg. This player should support rtsp,h264, MJPEG formats.
My question is, which decoders/encoders need to be enabled in ffmpeg build script?
Regards,
George
i am trying to use ffmpeg as a live transcoder to transcode tv channels from udp input to rtmp output to a wowza server.
i have 2 kinds of input channels in 1st kind the input audio is mp2 and in the second kind the input audio is acc_latm.
my problem is when i transcode the mp2 channels everything is fine but when i try to transcode the aac channel the audio is muted after few hours. but the video is fine.
the output codecs are : libx264 for video and faac or fdk-aac for audio output
i tried both aac encoders but it did not change.
i think it is the ffmpeg aac decoder's problem. but i cannot fix this.
i need a way to detect the problem online and restart the ffmpeg. or change the ffmpeg decoder codec.
please help.
thanks.
Yeah, ffmpeg is not guaranteed stable. Zoneminder used to detect crashes and restart processes when that happens. You may look at their code although IIRC they were only looking for video.
I think it would be simpler if you can enable some level of verbosity or debugging (-v loglevel) and see what messages indicated a crash (use grep to detect and some script to restart). That would be most efficient.
Another thing comes to mind is use ffmpeg/avconv to extract your resulting audio track and monitor it for some pattern in the file. Or play the resulting file and use an alsa device piping to a script. But it is under question if you would be able to reliably detect broken from legitimate silence. Much less efficient as well. Let me know if you can't figure the alsa device setup if you go that route, I don't have it handy right now.
How to convert PCM to flac using only ffmpeg API, but not the ffmpeg binary? I am looking for the full explanation of the process, but not only a solution
The best way is to look at the source of the ffmpeg binary itself, it uses the ffmpeg APIs itself. There are also code samples there. Here's the ffmpeg source code.