i am trying to use ffmpeg as a live transcoder to transcode tv channels from udp input to rtmp output to a wowza server.
i have 2 kinds of input channels in 1st kind the input audio is mp2 and in the second kind the input audio is acc_latm.
my problem is when i transcode the mp2 channels everything is fine but when i try to transcode the aac channel the audio is muted after few hours. but the video is fine.
the output codecs are : libx264 for video and faac or fdk-aac for audio output
i tried both aac encoders but it did not change.
i think it is the ffmpeg aac decoder's problem. but i cannot fix this.
i need a way to detect the problem online and restart the ffmpeg. or change the ffmpeg decoder codec.
please help.
thanks.
Yeah, ffmpeg is not guaranteed stable. Zoneminder used to detect crashes and restart processes when that happens. You may look at their code although IIRC they were only looking for video.
I think it would be simpler if you can enable some level of verbosity or debugging (-v loglevel) and see what messages indicated a crash (use grep to detect and some script to restart). That would be most efficient.
Another thing comes to mind is use ffmpeg/avconv to extract your resulting audio track and monitor it for some pattern in the file. Or play the resulting file and use an alsa device piping to a script. But it is under question if you would be able to reliably detect broken from legitimate silence. Much less efficient as well. Let me know if you can't figure the alsa device setup if you go that route, I don't have it handy right now.
Related
I have, as test cases, two mp4 videos: they seem to be encoded the same, but one works in Firefox and one doesn't:
bad.mp4 (doesn't work) view download
good.mp4 (does work) view download
(NOTE - I thought that the "view" links above would open them using the browser's built-in player, but it looks like they use Dropbox's video player, which DOES work in firefox. So, to see it in firefox, download it and then drag it into Firefox or right click and choose Firefox from "open with" - thanks!)
They both play in Chrome.
There is a long discussion on the Mozilla bug tracking site about a related issue:
https://bugzilla.mozilla.org/show_bug.cgi?id=1130450
So, I need to answer three questions:
1) What is the problem with bad.mp4?
2) How can I detect, programmatically (ie at the linux command line) which videos, of a library of videos, are going to have this problem?
3) How can I fix it, programmatically (eg with FFMPEG).
One thing I noticed was that in the good one, the video is on stream 0 and the audio stream 1, while on the bad one it's the other way round. I tried swapping the streams but it didn't seem to help.
EDIT: re the Baseline profile
It was suggested that FF might not like Baseline profile videos. I was able to identify some baseline profile videos that work and some that don't, so I think that's not the key problem. Here are a couple more samples:
Baseline_bad.mp4 (doesn't work) view download
Baseline_good.mp4 (does work) view download
First of all sorting mp4 tracks has nothing to do with that. Second, on Win10 with Firefox version 52.5.3 your bad.mp4 works just fine. However on MacOS using Firefox 69.03 bad.mp4 also plays but with a lot of artefacts.
The H.264 bitstream seems to be ok. I demultiplexed the bad mp4 with MP4Box MP4Box -raw 2 bad.mp4 which gives bad_track2.h264 and I decoded it with the reference software. Decoded just fine, without any problems, so the bitstream seems to be not a problem.
So it has to be the packaging into mp4. And indeed, if I package the bitstream back again using MP4Box -add bad_track2.h264 repackage.mp4, it plays just great. So this is a good news, since you don't have to re-encode (transcode) your videos, which will save you a lot of time.
Of course transcoding also solves it ffmpeg -i bad.mp4 transcode.mp4 in case you don't want to do it with MP4Box.
To answer 2 and 3. not 100 % sure if this is the same for all of your videos but the difference between good.mp4 and bad.mp4 is the compatible_brands which you can get with ffprobe. you can write a script which finds the videos with the same brand as in bad.mp4 and re-packages each of them using ffprobe, MP4Box.
I hope this helps.
Edit
To address the issue raised in #llogan's answer, regarding the Baseline AVC profile. IMHO this is very unlikely that the Baseline Profile is not supported, however can not be 100% sure about it since it may depend on the platform on which the Firefox is running.
From Mozilla's Web Video codec guide:
1: Firefox support for AVC is dependent upon the operating system's built-in or preinstalled codecs for AVC and its container in order to avoid patent concerns.
What is the problem with bad.mp4?
It's not the stream order because you can switch the order in good.mp4 and it still plays: ffmpeg -i good.mp4 -map 0:a -map 0:v -c copy switched.mp4
AUD NAL units
bad.mp4 may contain AUD NAL units Firefox doesn't like.
Baseline profile
Perhaps whatever decoder your Firefox is using doesn't like the Baseline H.264 profile. This is not the same as the far more common Constrained Baseline profile which is what good.mp4 is.
I'm not sure what version of Firefox you're using and what OS you are on so it is a guess.
How can I detect this problem?
Use ffprobe to determine the profile:
ffprobe -v error -select_streams v -of csv=p=0 -show_entries stream=profile input.mp4
If the issue is due to the AUD NAL units, and all of the Baseline videos have this issue, then this can still be useful. I didn't take the time to investigate further regarding the AUD NAL units so maybe it's a red herring.
How can I fix it?
If the AUD NAL units are the issue
Remux using the h264_metadata bitstream filter:
ffmpeg -i input.mp4 -bsf:v h264_metadata=aud=remove -map 0 -c copy -movflags +faststart output.mp4
This is fast because it avoids re-encoding.
You'll need FFmpeg 3.4 or later to use this.
See note below regarding -movflags +faststart.
If Baseline profile is the issue
You'll have to re-encode the video to change the profile:
ffmpeg -i input.mp4 -c:v libx264 -c:a copy -movflags +faststart output.mp4
See FFmpeg Wiki: H.264 for more encoding options.
I added -movflags +faststart because it seems like you'll be presenting these videos via progressive download playback. This option will allow the video to begin playing while it is still downloading.
I got an audio engine, which generates aac files. I want to mux this audio with some video. I'm using ffmpeg libraries to do just that - meaning, after the audio file is ready, I read it and mux it.
Now - for performance reasons, I don't want to wait until the audio engine completes the audio generation, I want the muxer to start reading the audio while it is being generated.
Can I achieve that using the FFMpeg libraries?
Which approach should I take?
Couldn't find any examples doing that
I coded an application to receive RTP packets via TCP (no packets are lost) from a hardware camera and dump its H264 packets to a file so I could play the video using MPlayer or VLC. This is already working and I pretty much did the steps described here. The commands to play the video are mplayer -fps 24 -demuxer h264es foobar.h264 and vlc foobar.h264.
The issue is now when I play the video. The camera changes the FPS frequently and because I drop the RTP info when writing the H264 file, the timestamp of each frame is lost. My question is: what do I have to do to fix the frame frequency? Should I create empty/blank P-frames (if that is possible)? If so, how would I do it?
Any solution using Linux compatible tools or libraries (like ffmpeg, libx264, libavcodec) using shell, C/C++ or Python is very much welcome.
PS: I have almost no experience with video encoding and RTP.
There is no timing information in a raw h.264 stream. The stream needs to be put into a container such as MP4 or FLV where you can tag each frame with a PTS/DTS.
I compiled VLC from the source code and it works well. When I do ". / Vlc" vlc runs. I also compiled ffmpeg from source and it works well too. When I do "ffmpeg-i-f toto.flv mp3-vn-acodec copy new_toto.mp3", the file named "new_toto.mp3" is generated.
What I cannot understand is how VLC and ffmpeg work together? What part of VLC code uses ffmpeg code?
Is there a ffmpeg command to play (read) a video?
VLC does not embed FFmpeg as a sort of slave binary used internally, as you possibly could suppose. Instead, both FFmpeg and VLC are using libavcodec, a library which implement what ffmpeg exposes through its command line interface. So, FFmpeg supports VLC through supplying its libavcodec library to be one of the essential components on which the VLC is built as a player app.
Is there a ffmpeg command to play (read) a video?
ffplay video.mp4
ffplay is however a very stripped down video viewer, mostly only suitable for debugging video things and not for daily usage.
ffplay may be more accurate than VLC for some quirky formats, for example it can handle very low FPS video where VLC 2.2.4 could not: How to resize a picture using ffmpeg's sws_scale()?
Tested in Ubuntu 16.04, ffmpeg 2.8 with this minimal procedurally generated videos from: How to resize a picture using ffmpeg's sws_scale()?
We have an encoding process in place using ffmpeg on Mac OS X. This process will take a source video and a couple files from that: .m3u8 video, .mp4 video and .mp3 audio file.
By default we've used the video from our m3u8 process which is a h264 (via libx264) video with AAC (via libfaac) audio.
We are mostly using these videos on mobile devices (hence the m3u8 files) but we also use the .mp4 files for Android, Windows Phone, etc. More and more we also need to offer these same videos on the web via either a flash player or HTML5 player.
Therefore, we'd like to have the best audio/video codec combo for all these uses ... where I'm confused is to what is "standard" for a .mp4 file?
If the .mp4 uses the mp3 codec then it plays just fine everywhere but QuickTime, in QuickTime the video plays but there is not audio (works just fine in VLC player tho.)
I've been told it's due to how QuickTime uses file extensions to assume information about the video instead of trying to actually get the codec data from the file? This does make some sense, if we encode the same file but use AAC for the audio codec then it works just fine in QuickTime.
So --- what's the "correct" or "ideal" audio/video codec combo --- is it best and safe to use AAC (i.e. will it work on a broad range of devices) even though it's not a "free" codec?
Compatibility issues set aside, you might want to take into account the quality loss induced by transcoding from mp3 to aac. As your original audio is mp3 encoded, you will have a better end result by remuxing the mp3 bitstream instead of transcoding to aac (even if aac is better than mp3 at a given bitrate).
If your workflow requires lowering the bitrate of the original mp3, then my point looses relevance.
AAC is preferred because many modern devices such as tablets and mobile phones might have hardware decoders for it which will result in better battery and lower CPU consumption. Same applies to h264.
In recent years a growing number of devices have added hardware acceleration for the (HEVC) H.265 codec, but that's probably about ~half of the mobile market at best so not yet a default.
http://en.wikipedia.org/wiki/HTML5_video - scroll down for a compatibility table. Seems like either MP3 or AAC work well with H.264
Definitely use the AAC codec, as you said, it works on a wider range of devices and it is better for use in HTML5