I am working on ffmpeg and trying to add a audio stream on the fly. I am using AudioQueues and I get raw audio buffer. I am encoding audio with linear PCM and hence the audio I get will be of raw format, which I know ffmpeg does accept it. But I cannot figure out how. I have looked into AVStream, where in we have to create a new stream for this audio channel but how do I encode it to a video which is already initialized in another AVStream structure.
Overall, I would like to have an idea of the architecture of ffmpeg. I found it difficult to work since it is least documented. Any pointers or details are appreciated.
Thanks and Regards,
Raj Pawan G
If you want to use java, you'll find a much better documented API wrapper for FFmpeg with Xuggler.
That said, FFmpeg can support Raw PCM bu tnot all containers can contain it. use the PCM codecs (see avcodec.h) and find the one that has the right size and attributes you want. To add the audio to the same container find a AVFormatContext object that you use for your existing video stream, and use av_new_stream(...) to add a new stream. Then attach your PCM encoder and 'encode' to that and write resulting packets. See output_example.c in the FFmpeg for examples of this API in action.
Related
I wanna use ffmpeg to convert yuv raw video file into ts stream video file.So I do this in my code:
avcodec_find_encoder(AV_CODEC_ID_MPEG2TS);
But when I run it ,it occurs that:
[NULL # 0x8832020] No codec provided to avcodec_open2()
I change the "AV_CODEC_ID_MPEG2TS" into "AV_CODEC_ID_MPEG2VIDEO", it works well ,and generate a mpg file running well too.So I wanna ask why I cannot use "AV_CODEC_ID_MPEG2TS"?
I'm also looking for streaming a file with ffmpeg so I'm not sure about that but it is what I understand....
Mpeg TS (Transport Stream) is not a codec, it is an encapsulation method, so you have to encode the stream with some code (I'm not sure if you can chose any codec) and then you can encapsulate it with mpeg ts before transmit over the network.
If you don't need to transmit the stream over the network maybe you don't need mpeg ts.
I hope this will helpful....!
Look here: ffmpeg doxygen
I have an application wherein I have H.264 frames from an RTSP stream stored in a proprietary database. I need to be able to present a frame to the H.264 decoder (frames in sequence, of course) and get back the decoded frame (bitmap, whatever) output. I cannot use the traditional DirectShow streams because I don't have a stream. Is there any codec can be used in this manner? Later I will need to go the other way as well (given bitmaps or other format images, create an H.264 stream). Any help you can give would be greatly appreciated.
Create a DirectShow Source Filter that assembles the h264 stream from database, then you can pass it to standard DirectShow H264 decoder. Look into DirectShow samples for example source code.
As Isso mentioned already, you can push the H.264 data into DirectShow pipeline and have the frame decoded. Additionally to this, there is H.264 Video Decoder MFT (Windows 7 and more recent only) which might be an easier way to use the decoder and to apply it to an individual "frame". You can use other decoders as well, such as FFmpeg/libavcodec however you would still need to interface to the decoders typically designed for stream processing.
I have an application that sends raw h264 NALUs as generated from encoding on the fly using x264 x264_encoder_encode. I am getting them through plain TCP so I am not missing any frames.
I need to be able to decode such a stream in the client using Hardware Acceleration in Windows (DXVA2). I have been struggling to find a way to get this to work using FFMPEG. Perhaps it may be easier to try Media Foundation or DirectShow, but they won't take raw H264.
I either need to:
Change the code from the server application to give back an mp4 stream. I am not that experienced with x264. I was able to get raw H264 by calling x264_encoder_encode, by following the answer to this question: How does one encode a series of images into H264 using the x264 C API? How can I go from this to something that is wrapped in MP4 while still being able to stream it in realtime
I could at the receiver wrap it with mp4 headers and feed it into something that can play it using DXVA. I wouldn't know how to do this
I could find another way to accelerate it using DXVA with FFMPEG or something else that takes it in raw format.
An important restriction is that I need to be able to pre-process each decoded frame before displaying it. Any solution that does decoding and displaying in a single step would not work for me
I would be fine with either solution
I believe you should be able to use H.264 packets off the wire with Media Foundation. there's an example on page 298 of this book http://www.docstoc.com/docs/109589628/Developing-Microsoft-Media-Foundation-Applications# that use a HTTP stream with Media Foundation.
I'm only learning Media Foundation myself and am trying to do a similar thing to you, in my case I want to use H.264 payloads from an RTP packet, and from my understanding that will require a custom IMFSourceReader. Accessing the decoded frames should also be possible from what I've read since there seems to be complete flexibility in chaining components together into topologies.
I want to stream video (no audio) from a server to a client. I will encode the video using libx264 and decode it with ffmpeg. I plan to use fixed settings (at the very least they will be known in advance by both the client and the server). I was wondering if I can avoid wrapping the compressed video in a container format (like mp4 or mkv).
Right now I am able to encode my frames using x264_encoder_encode. I get a compressed frame back, and I can do that for every frame. What extra information (if anything at all) do I need to send to the client so that ffmpeg can decode the compressed frames, and more importantly how can I obtain it with libx264. I assume I may need to generate NAL information (x264_nal_encode?). Having an idea of what is the minimum necessary to get the video across, and how to put the pieces together would be really helpful.
I found out that the minimum amount of information are the NAL units from each frame, this will give me a raw h264 stream. If I were to write this to a file, I could watchit using VLC if adding a .h264
I can also open such a file using ffmpeg, but if I want to stream it, then it makes more sense to use RTSP, and a good open source library for that is Live555: http://www.live555.com/liveMedia/
In their FAQ they mention how to send the output from your encoder to live555, and there is source for both a client and a server. I have yet to finish coding this, but it seems like a reasonable solution
Problem:
I have to save live video streams data which come as an RTP packets from RTSP Server.
The data come in two formats : MPEG4 and h264.
I do not want to encode/decode input stream.
Just write to a file which is playable with proper codecs.
Any advice?
Best Wishes
History:
My Solutions and their problems:
Firt attempt: FFmpeg
I use FFmpeg libary to get audio and video rtp packets.
But in order to write packets i have to use av_write_frame :
which seems that decode /encode takes place.
Also, when i give output format as mp4 ( av_guess_format("mp4", NULL, NULL)
the output file is unplayable.
[ any way ffmpeg has bad doc. hard to find what is wrong]
Second attempth: DirectShow
Then i decide to use DirectShow. I found a RTSP Source Filter.
Then a Mux and File Writer.
Cretae Single graph:
RTSP Source --> MPEG MUX ---> File Writer
It worked but the problem is that the output file is not playable
if graph is not stoped. If something happens, graph crashed for example
the output file is not playable
Also i can able to write H264 data, but the video is completely unplayable.
The MP4 file format has an index that is required for correct playback, and the index can only be created once you've finished recording. So any solution using MP4 container files (and other indexed files) is going to suffer from the same problem. You need to stop the recording to finalise the file, or it will not be playable.
One solution that might help is to break the graph up into two parts, so that you can keep recording to a new file while stopping the current one. There's an example of this at www.gdcl.co.uk/gmfbridge.
If you try the GDCL MP4 multiplexor, and you are having problems with H264 streams, see the related question GDCL Mpeg-4 Multiplexor Problem