I am running a robot that uses fmpeg to send straming video to letsrobot.tv You can see my bot on the website called patton II. I want to overlay a video HUD on the stream.
I have found a link explaining how to do this, however I do not know how to do it with a streaming video as input instead of a single image file.
This is the command that is currently being used to stream the video:
overlayCommand = '-vf dynoverlay=overlayfile=/home/pi/runmyrobot/images/hud.png:check_interval=500'
videoCommandLine = '/usr/local/bin/ffmpeg -f v4l2 -framerate 25 -video_size 640x480 -i /dev/video%s %s -f mpegts -codec:v mpeg1video -s 640x480 -b:v %dk -bf 0 -muxdelay 0.001 %s http://%s:%s/hello/640/480/' % (deviceAnswer, rotationOption, args.kbps, overlayCommand, server, videoPort)
audioCommandLine = '/usr/local/bin/ffmpeg -f alsa -ar 44100 -i hw:1 -ac 2 -f mpegts -codec:a mp2 -b:a 32k -muxdelay 0.001 http://%s:%s/hello/640/480/' % (server, audioPort)
You already have one input, which is the webcam video:
-f v4l2 -framerate 25 -video_size 640x480 -i /dev/video%s
You want to overlay another video, so you have to add a second input, which is your HUD stream. I'm assuming that you already have a stream that's being generated on the fly:
-i /path/to/hud/stream
Then, add a complex filter that overlays one over the other:
-filter_complex "[0:v][1:v]overlay[out]"
After the filter, add a -map "[out]" option to tell ffmpeg to use the generated video as output, and add your remaining options as usual. So, in sum:
/usr/local/bin/ffmpeg -f v4l2 -framerate 25 -video_size 640x480 -i /dev/video%s \
-i /path/to/hud/stream \
-filter_complex "[0:v][1:v]overlay[out]" -map "[out]" \
-f mpegts -codec:v mpeg1video -s 640x480 -b:v %dk -bf 0 \
-muxdelay 0.001 %s http://%s:%s/hello/640/480/
Obviously, without knowing more, this is the most generic advice I can give you.
Some general tips:
Make sure that the HUD stream is the same resolution as the webcam video, where the elements are placed where you want them. Or use the overlay filter's x and y options to move the HUD.
Your HUD stream should have a transparency layer. Not all codecs and container formats support that.
You're using -codec:v mpeg1video, which is MPEG-1 video. It's quite resource-efficient but otherwise low in quality. You may want to choose a better codec, but it depends on your device capabilities (e.g., at least MPEG-2 with mpeg2, or MPEG-4 Part 10 with mpeg4, or H.264 with libx264).
Related
Trying to add a timestamp for ffmpeg for capture card recoridngs but its not working at all.
ffmpeg -f dshow -rtbufsize 100M -i video="Game Capture HD60 S (Video) (#01)":audio="Game Capture HD60 S (Audio) (#01)" -c:v mpeg2video -q:v 0 -r 29.97 -c:a aac -b:a 192k -threads 12 -flags +ilme+ildct -top 1 -strftime 1 "file-%Y%m%d-%%04d.ts"
The output file is basically just "file-%Y%m%d-%%04d.ts", no dates or times. Any help?
strftime is not a generic option for all muxers; it is specific to a few.
However, you can work around this by using the segment muxer. The segment muxer is meant for when the output needs to be split across, umm.. segments. However, by setting a very large segment time, it effectively will write a single file.
So,
ffmpeg -f dshow -rtbufsize 100M -i video="Game Capture HD60 S (Video) (#01)":audio="Game Capture HD60 S (Audio) (#01)" -c:v mpeg2video -q:v 0 -r 30000/1001 -c:a aac -b:a 192k -threads 12 -flags +ilme+ildct -top 1 -f segment -segment_time 99999 -strftime 1 "file-%Y%m%d-%%04d.ts"
(Set r value to exact representation)
I'm trying to create a website to stream some videos. For each video, I extract video, audio and subtitles in 3 different folders. It happens that a video has multiple audio tracks and multiple subtitles. I did a lot of research and I don't know how to add all of them in the manifest. Right now, I use this command:
ffmpeg -f webm_dash_manifest \
-i video1.mp4 -f webm_dash_manifest \
-i video2.mp4 -f webm_dash_manifest \
-i audio1.webm -f webm_dash_manifest \
-i audio2.webm -f webm_dash_manifest \
-i subtitles.vtt \
-c copy -map 0 -map 1 -map 2 -map 3 \
-f webm_dash_manifest -adaptation_sets "id=0,streams=v id=1,streams=a" manifest.mpd
My two videos have different resolutions and bitrates, and it works perfectly. But I don't get any subtitles and my two audio tracks are considered like one same audio track which has two different bitrates (just like videos). I think I should have many adaptation_sets, but I don't know how to create them.
How can I create that manifest the right way?
After a few days, I found the solution.
My goal is to convert a video into mpeg-dash content which is really great for streaming.
I will encode video to h264, audio to aac, and subtitles to webvtt.
It's good settings for a large browser compatibility.
vp9 is really nice too but too long to encode for me.
Tools required:
ffmpeg: https://www.ffmpeg.org/download.html
mp4dash & mp4fragment: https://www.bento4.com/downloads/
Let's suppose we have a 1080p video file "video.mkv" with these streams:
0: video stream
1: audio stream, it language
2: audio stream, en langugage
3: subtitle stream, it language
4: subtitle stream, en language
1. Extracting differents streams
1.1 Video
I extract and transcode video stream to differents resolutions and bitrates:
ffmpeg -i video.mkv -an -sn -c:0 libx264 -x264opts 'keyint=24:min-keyint=24:no-scenecut' -b:v 5300k -maxrate 5300k -bufsize 2650k -vf 'scale=-1:1080' tmp/video/video-1080.mp4
ffmpeg -i video.mkv -an -sn -c:0 libx264 -x264opts 'keyint=24:min-keyint=24:no-scenecut' -b:v 2400k -maxrate 2400k -bufsize 1200k -vf 'scale=-1:720' tmp/video/video-720.mp4
ffmpeg -i video.mkv -an -sn -c:0 libx264 -x264opts 'keyint=24:min-keyint=24:no-scenecut' -b:v 600k -maxrate 600k -bufsize 300k -vf 'scale=-1:360' tmp/video/video-360.mp4
1.2 Audio
ffmpeg -i video.mkv -map 0:1 -ac 2 -ab 192k -vn -sn tmp/audio/audio-it.mp4
ffmpeg -i video.mkv -map 0:2 -ac 2 -ab 192k -vn -sn tmp/audio/audio-en.mp4
1.3 Subtitle
ffmpeg -i video.mkv -map 0:3 -vn -an tmp/subtitle/subtitle-it.vtt
ffmpeg -i video.mkv -map 0:4 -vn -an tmp/subtitle/subtitle-en.vtt
You can use the "-loglevel warning" option to see less informations.
2. Fragment video and audio
2.1 Video
mp4fragment tmp/video/video-1080.mp4 tmp/video/f-video-1080.mp4
mp4fragment tmp/video/video-720.mp4 tmp/video/f-video-720.mp4
mp4fragment tmp/video/video-360.mp4 tmp/video/f-video-360.mp4
2.2 Audio
mp4fragment tmp/audio/audio-it.mp4 tmp/audio/f-audio-it.mp4
mp4fragment tmp/audio/audio-en.mp4 tmp/audio/f-audio-en.mp4
3. Split files and create the dash manifest
mp4dash --mpd-name=manifest.mpd tmp/video/f-video-1080.mp4 tmp/video/f-video-720.mp4 tmp/video/f-video-360.mp4 tmp/audio/f-audio-it.mp4 tmp/audio/f-audio-en.mp4 \[+format=webvtt,+language=it\]tmp/subtitle/subtitle-it.vtt \[+format=webvtt,+language=en\]tmp/subtitle/subtitle-en.vtt
You can now delete the tmp folder
rm -rf tmp
(and your source file if you don't need it anymore)
You have now your mpeg-dash content which can be streamed. You have to serve your files (allow cors and enable byte range request).
I use angular and rx-player as player. I can switch language, subtitles and the video quality is adaptative to the client's bandwidth !
Rx-player: https://github.com/canalplus/rx-player
I'm successfully streaming silent video with music added from my Raspberry Pi (Raspbian) to YouTube via ffmpeg, with the help of this GitHub gist and this post:
raspivid -o - -t 0 -vf -hf -w 1280 -h 720 -fps 25 -b 4000000 | \
ffmpeg -i music.wav \
-f h264 -i - -vcodec copy -acodec aac -ab 128k -g 50 -strict experimental \
-f flv rtmp://a.rtmp.youtube.com/live2/STREAMKEY
The last step of my project to add a transparent, full width/height png overlay to the video (1280x720 size in my case). I've seen a few related answers such as this one and this one.
With the added complexity of piping in a camera feed, mixing in an audio source and outputting to a video stream, I haven't succeeded in adding the image overlay. Where/how would I add a transparent image overlay in the example above?
The ffmpeg part will be
ffmpeg -i music.wav \
-f h264 -i - -i overlay.png
-filter_complex "[1][2]overlay"
-vcodec libx264 -preset ultrafast -tune zerolatency -acodec aac -ab 128k -g 50 -strict experimental \
-f flv rtmp://a.rtmp.youtube.com/live2/STREAMKEY
Since you're altering the video contents, copy can't be used, and the video has to be re-encoded.
I'm trying to capture video from a webcam, and I find that when I use the -vcodec copy option, it works really well (far better than any other software I've tried). However, I'd like my files to be a bit smaller, and it seems that every attempt I make to compress the video leads to extremely jumpy video. If, for example, I switch the output vcodec to mjpeg, it changes from reporting 15 fps to reporting between 3 and 4 fps. Am I doing something wrong?? Here is the call with -vcodec copy:
ffmpeg -y -f dshow -vcodec mjpeg -s 1184x656 -framerate 25 -i video="HD 720P Webcam" -vcodec copy test.avi
-- which gets me 15 fps. But if I change to mjpeg, I get only 3-4 fps:
ffmpeg -y -f dshow -vcodec mjpeg -s 1184x656 -framerate 25 -i video="HD 720P Webcam" -vcodec mjpeg test.avi
Experimental attempts to put -framerate 25 or -r 25 before test.avi also does nothing to help the situation. I'm not getting any smoother video when experimenting with mpeg4 or libx264 either. Only the copy option gives me smooth video (btw I'm filming my hands playing a piano, so there is a lot of fast motion in the videos).
Help!!!! And thank you...
I don't understand why the framerate drops so much, but you could try a 2 pass approach where you first record it using -vcodec copy (as you pasted in the question)
ffmpeg -y -f dshow -vcodec mjpeg -s 1184x656 -framerate 25 -i video="HD 720P Webcam" -vcodec copy test.avi
Then transcode it into mjpeg once it's done (something like this):
ffmpeg -i test.avi -vcodec mjpeg test.mjpeg
note: I haven't actually tested any of the above command lines.
Sounds like your webcam is outputting a variable frame rate stream. Try the below on one of your copy captured files.
ffmpeg -i test.avi -vcodec libx264 -r 30 test.mp4
(You should avoid capturing to AVI, use MKV instead)
When I encode videos by FFMpeg I would like to put a jpg image before the very first video frame, because when I embed the video on a webpage with "video" html5 tag, it shows the very first picture as a splash image. Alternatively I want to encode an image to an 1 frame video and concatenate it to my encoded video. I don't want to use the "poster" property of the "video" html5 element.
You can use the concat filter to do that. The exact command depends on how long you want your splash screen to be. I am pretty sure you don't want an 1-frame splash screen, which is about 1/25 to 1/30 seconds, depending on the video ;)
The Answer
First, you need to get the frame rate of the video. Try ffmpeg -i INPUT and find the tbr value. E.g.
$ ffmpeg -i a.mkv
ffmpeg version N-62860-g9173602 Copyright (c) 2000-2014 the FFmpeg developers
built on Apr 30 2014 21:42:15 with gcc 4.8 (Ubuntu 4.8.2-19ubuntu1)
[...]
Input #0, matroska,webm, from 'a.mkv':
Metadata:
ENCODER : Lavf55.37.101
Duration: 00:00:10.08, start: 0.080000, bitrate: 23 kb/s
Stream #0:0: Video: h264 (High 4:4:4 Predictive), yuv444p, 320x240 [SAR 1:1 DAR 4:3], 25 fps, 25 tbr, 1k tbn, 50 tbc (default)
At least one output file must be specified
In the above example, it shows 25 tbr. Remember this number.
Second, you need to concatenate the image with the video. Try this command:
ffmpeg -loop 1 -framerate FPS -t SECONDS -i IMAGE \
-t SECONDS -f lavfi -i aevalsrc=0 \
-i INPUTVIDEO \
-filter_complex '[0:0] [1:0] [2:0] [2:1] concat=n=2:v=1:a=1' \
[OPTIONS] OUTPUT
If your video doesn't have audio, try this:
ffmpeg -loop 1 -framerate FPS -t SECONDS -i IMAGE \
-i INPUTVIDEO \
-filter_complex '[0:0] [1:0] concat=n=2:v=1:a=0' \
[OPTIONS] OUTPUT
FPS = tbr value got from step 1
SECONDS = duration you want the image to be shown.
IMAGE = the image name
INPUTVIDEO = the original video name
[OPTIONS] = optional encoding parameters (such as -vcodec libx264 or -b:a 160k)
OUTPUT = the output video file name
How Does This Work?
Let's split the command line I used:
-loop 1 -framerate FPS -t SECONDS -i IMAGE: this basically means: open the image, and loop over it to make it a video with SECONDS seconds with FPS frames per second. The reason you need it to have the same FPS as the input video is because the concat filter we will use later has a restriction on it.
-t SECONDS -f lavfi -i aevalsrc=0: this means: generate silence for SECONDS (0 means silence). You need silence to fill up the time for the splash image. This isn't needed if the original video doesn't have audio.
-i INPUTVIDEO: open the video itself.
-filter_complex '[0:0] [1:0] [2:0] [2:1] concat=n=2:v=1:a=1': this is the best part. You open file 0 stream 0 (the image-video), file 1 stream 0 (the silence audio), file 2 streams 0 and 1 (the real input audio and video), and concatenate them together. The options n, v, and a mean that there are 2 segments, 1 output video, and 1 output audio.
[OPTIONS] OUTPUT: this just means to encode the video to the output file name. If you are using HTML5 streaming, you'd probably want to use -c:v libx264 -crf 23 -c:a libfdk_aac (or -c:a libfaac) -b:a 128k for H.264 video and AAC audio.
Further information
You can check out the documentation for the image2 demuxer which is the core of the magic behind -loop 1.
Documentation for concat filter is also helpful.
Another good source of information is the FFmpeg wiki on concatenation.
The answer above works for me but in my case it took too much time to execute (perhaps because it re-encodes the entire video). I found another solution that's much faster. The basic idea is:
Create a "video" that only has the image.
Concatenate the above video with the original one, without re-encoding.
Create a video that only has the image:
ffmpeg -loop 1 -framerate 30 -i image.jpg -c:v libx264 -t 3 -pix_fmt yuv420p image.mp4
Note the -framerate 30 option. It has to be the same with the main video. Also, the image should have the same dimension with the main video. The -t 3 specifies the length of the video in seconds.
Convert the videos to MPEG-2 transport stream
According to the ffmpeg official documentation, only certain files can be concatenated using the concat protocal, this includes the MPEG-2 transport streams. And since we have 2 MP4 videos, they can be losslessly converted to MPEG-2 TS:
ffmpeg -i image.mp4 -c copy -bsf:v h264_mp4toannexb -f mpegts image.ts
and for the main video:
ffmpeg -i video.mp4 -c copy -bsf:v h264_mp4toannexb -f mpegts video.ts
Concatenate the MPEG-2 TS files
Now use the following command to concatenate the above intermediate files:
ffmpeg -i "concat:image.ts|video.ts" -c copy -bsf:a aac_adtstoasc output.mp4
Although there are 4 commands to run, combined they're still much faster then re-encoding the entire video.
My solution. It sets an image with duration of 5 sec before the video along with aligning video to be 1280x720. Image should have 16/9 aspect ratio.
ffmpeg -i video.mp4 -i image.png -filter_complex '
color=c=black:size=1280x720 [temp]; \
[temp][1:v] overlay=x=0:y=0:enable='between(t,0,5)' [temp]; \
[0:v] setpts=PTS+5/TB, scale=1280x720:force_original_aspect_ratio=decrease, pad=1280:720:-1:-1:color=black [v:0]; \
[temp][v:0] overlay=x=0:y=0:shortest=1:enable='gt(t,5)' [v]; \
[0:a] asetpts=PTS+5/TB [a]'
-map [v] -map [a] -preset veryfast output.mp4