I wanna draw video overlays like lines, circles, rectangles and print texts on an AVFrame by using the ffmpeg's functions, not command line utility.
Is there any one knows how to do it?
Do you have the source code for this function?
I have asked this question when I started to work on video streaming issues.
Since an AVFrame is a decoded video data, adding overlays on video data means transcoding it. I add overlays while rendering. I used SDL library to render AvFrames and add overlays.
Related
I am stucked by a video processing feature, Specifically, upload an image and then generate a video based on various video templates.
Here are the video templates:
http://video-static.biku8.com/data/video/template/3286012076458048/7437ab55-2e83-4a36-9046-5708fcddf4c1.mp4
http://video-static.biku8.com/data/video/template/3274256089907264/ae8fa3f7-6c9c-45ca-810f-48db92cc14cb.mp4
http://video-static.biku8.com/data/video/template/3213894231425088/bf107d439b9043a58c1ea0ba26f811db_template.mp4
...
As shown in the video templates above, I just need to upload a photo to generate a great video.
My question
What is the specific idea for implementing this video?
Which third-party libraries are needed? (ffmpeg, opencv)
PS: I am using dlib and opencv for face recognition. I can generate face image, but I don't know how to insert face image into the correct position of these template videos.
I would suggest you to follow the below 3 steps
Load the template video by opencv, you can access the video frame by frame
Modify each frame, one by one.
Save frame to video stream writer
Regarding step 2, actually, you must copy the uploaded image to the each frame by a mask (the pixel from source image would be copied to destination image if its coordinate on the mask is non-black). The mask could be defined by a list of points OR by an image. You should pre-define a mask for each frame in a file. Then load the mask for each frame and copy.
How to read video, save video OpenCV read-write Video
How to insert image to another image Copy non rectangular ROI
Generating videos like them are all not easy tasks. I recommend to use Adobe After Effects or other video creating software (with some scripts and actions) if you don't need to generate it by a single program or program language.
Then, I answer them below when you need to generate it by programatically.
For the first one, you should recognize faces and bones. So you should use OpenCV. ( I recommend to use tools like OpenFrameworks or TouchDesigner and so on. )
For the second one, I don't know what you exactly want, but if you want to recognize the position of the bottle dynamically, you have to use deep learning or other way to detect it. Then you may need TensorFlow or OpenCV. ( If you just want to merge layers, you can use ffmpeg etc. )
For the last one, you should split the video frame into the boxes, then you have to control. I think there are many ways to implement this. I may use OpenFrameworks, TouchDesigner, vvvv, or Processing.
I think using ffmpeg for them is not recommended. This tool is not the best for generating complicated video. But ffmpeg will do good, for example if you just merge two videos with alpha.
How can I overlay an image onto a video without changing the video file?
I have many videos and I want to be able to open them and overlay a ruler onto them and then measure the distance an individual moved visually. All I want is to play a video and then to open up an image with some transparency and position the image over the video. This way i would be able to look at the video and see how far the individual moved.
I would like to do this without having to embed the image like a watermark, because that is computationally expensive. I would need to copy the video, embed it with the ruler and then watch the video, then delete that video file. This seems unnecessary. I would like to just watch the video and have a transparent image over it while I a watching.
Is there a program that does this all together?
Alternatively, is there a program which I can use to open an image and make it transparent and then move it over the video that is playing?
Note: I am using Windows.
It sounds form your requirements that simply overlaying a separate image layer over the video will meet your needs.
Implementing this approach will depend on the video player client you are using, but you could implement an HTML5 based solution and play the videos locally with this (or even from a URL on the web if you have them there).
There is a nice answer with a working fiddle which shows how to do this with HTML5 here: https://stackoverflow.com/a/31175193/334402
One thing to note - you have not mentioned scale in your question. If you need to measure how far the person has moved in real distance, rather than in just cm's across the video screen, then you will need to somehow work out the scale of the video. This makes things considerably harder as the video may zoom in and out during the sequence you want to measure, so you would need some reference to calculate the scale for each frame. One approach would be to use the individual as a reference, assuming they are in all the frames you are interested in.
What about using good old VLC for that?
Open VLC go to Tools→Effects and Filters→Video Effects→Overlay and select Add logo checkbox:
Then, add your transparent overlay image and play any video with VLC. The output looks like this:
I have a series of png's that have an alpha channel as a background. Each file is named like file_name.0001.png and so on, in subsequent order. I'd like to join these png's into a video with ffmpeg and maintain the transparency.
I've tried a couple of things but I suspect I'm running into a codec issue. When I run ffmpeg, the video is created but the background is black.
If it makes a difference, I'm wanting to use the video in Microsoft Powerpoint. Thanks!
Edit
The suggested duplicate is very close to what I was looking for, thank you! The only reason it's not a complete solution is none of the options presented in the other thread work well with Microsoft Powerpoint. None of the codecs used in the suggested solution play well with Powerpoint. This is not the fault of ffmpeg, but of Powerpoint.
Though ffmpeg doesn't seem to be able to do what I need, I found that imagemagick did the trick. I was able to create a gif from the images and the alpha channel was preserved. I used the following:
convert -dispose 3 -coalesce images.*.png gif_file_name.gif
The -dispose 3 is critical as it tells imagemagick to clear the image prior to overlay, otherwise, you can see each image overlaid on each other (since they have the transparent background).
I couldn't get ffmpeg to create a video that preserved the alpha channel and was Powerpoint friendly (not the fault of ffmpeg). Though ffmpeg doesn't seem to be able to do what I need, I found that imagemagick did the trick. I was able to create a gif from the images and the alpha channel was preserved. I used the following:
convert -dispose 3 -coalesce images.*.png gif_file_name.gif
The -dispose 3 is critical as it tells imagemagick to clear the image prior to overlay, otherwise, you can see each image overlaid on each other (since they have the transparent background).
Is it possible to give an end user the ability to save as a single file an animation created client side with HTML5 Canvas - other than saving the entire HTML of the page?
There are plenty of tutorials on how to save as PNG, but the animation is lost in these cases.
There is not an "easy" way to do this.
Here is a similar question...
Grabbing each frame of an HTML5 canvas
You could follow this approach and grab the frames and submit these to a server-side script to compile them into an animated GIF.
Another option, though non-trival, would be to implement a pure JavaScript GIF encoder. The GIF specification can be found here, http://graphcomp.com/info/specs/gif89a.txt
If the color table stays the same in each frame, you could probably splice together the frames from already encoded GIF urls without having to compress the pixel level data. You would still need to decode the base64 stream from Canvas.toDataURL.
You could use mjbuilder, it's a library that alllows you to save canvas frames into a mpeg file. But it has issues and it only works on Firefox.
http://ushiroad.com/mjpeg/
I am working on an application and I have a problem I just cant seem to find a solution for. The application is written in vc++. What I need to do is display a YUV video feed with text on top of it.
Right now it works correctly by drawing the text in the OnPaint method using GDI and the video on a DirectDraw overlay. I need to get rid of the overlay because it causes to many problems. It wont work on some video cards, vista, 7, etc.
I cant figure out a way to complete the same thing in a more compatible way. I can draw the video using DirectDraw with a back buffer and copy it to the primary buffer just fine. The issue here is that the text being drawn in GDI flashes because of the amount of times the video is refreshed. I would really like to keep the code to draw the text intact if possible since it works well.
Is there a way to draw the text directly to a DirectDraw buffer or memory buffer or something and then blt it to the back buffer? Should I be looking at another method all together? The two important OS's are XP and 7. If anyone has any ideas just let me know and I will test them out. Thanks.
Try to look into DirectShow and the Ticker sample on microsoft.com:
DirectShow Ticker sample
This sample uses the Video Mixing Renderer to blend video and text. It uses the IVMRMixerBitmap9 interface to blend text onto the bottom portion of the video window.
DirectShow is for building filter graphs for playing back audio or video streams an adding different filters for different effects and manipulation of video and audio samples.
Instead of using the Video Mixing Renderer of DirectShow, you can also use the ISampleGrabber interface. The advantage is, that it is a filter which can be used with other renderers as well, for example when not showing the video on the screen but streaming it over network or dumping it to a file.