I'm using libavcodec to encode a stream to h264 and libavformat to store it in an mp4. The resulting container has an invalid header that can be played in VLC, but not any other player.
I've found that using the mp4 container and the "mpeg4" codec produces a valid mp4 file, but using libx265 (HEVC) or the libx264 codec produces invalid mp4s.
I can use ffmpeg -i invalid.mp4 -vcodec copy valid.mp4 and I get a file of almost the exact same size, but in a valid container.
Examples of these files are here: Broken file and
Repaied file [use the download links in the upper right to examine]
I used a hex editor to see the differences in the headers of the two files and the invalid one is 1 byte smaller than the valid one.
The code I'm using to open the container and codec and to write the header is here:
AVOutputFormat *container_format;
AVFormatContext *container_format_context;
AVStream *video_stream;
int ret;
/* allocate the output media context */
avformat_alloc_output_context2(&container_format_context, NULL, NULL, out_file);
if (!container_format_context) {
log(INFO, "Unable to determine container format from filename, exiting\n");
exit(1);
}
else {
log(INFO, "Using container %s\n", container_format_context->oformat->name);
}
if (!container_format_context) {
log(ERROR, "Could not build container format context. Encoding failed.");
exit(1);
}
container_format = container_format_context->oformat;
/* Pull codec based on name */
AVCodec* codec = avcodec_find_encoder_by_name(codec_name);
if (codec == NULL) {
log(ERROR, "Failed to locate codec \"%s\".",
codec_name);
exit(1);
}
/* create stream */
video_stream = NULL;
video_stream = avformat_new_stream(container_format_context, codec);
if (!video_stream) {
log(ERROR, "Could not allocate encoder stream. Cannot continue.\n");
exit(1);
}
video_stream->id = container_format_context->nb_streams - 1;
video_stream->time_base = video_stream->codec->time_base = (AVRational) { 1, 25};
av_dump_format(container_format_context, 0, out_file, 1);
/* Retrieve encoding context */
AVCodecContext* avcodec_context = video_stream->codec;
if (avcodec_context == NULL) {
log(ERROR, "Failed to allocate context for "
"codec \"%s\".", codec_name);
exit(1);
}
/* Init context with encoding parameters */
avcodec_context->bit_rate = bitrate;
avcodec_context->width = width;
avcodec_context->height = height;
avcodec_context->gop_size = 10;
avcodec_context->max_b_frames = 1;
avcodec_context->qmax = 31;
avcodec_context->qmin = 2;
avcodec_context->pix_fmt = AV_PIX_FMT_YUV420P;
av_dump_format(container_format_context, 0, out_file, 1);
/* Open codec for use */
if (avcodec_open2(avcodec_context, codec, NULL) < 0) {
log(ERROR, "Failed to open codec \"%s\".", codec_name);
exit(1);
}
/* Allocate corresponding frame */
AVFrame* frame = av_frame_alloc();
if (frame == NULL) {
exit(1);
}
/* Copy necessary data for frame from avcodec_context */
frame->format = avcodec_context->pix_fmt;
frame->width = avcodec_context->width;
frame->height = avcodec_context->height;
/* Allocate actual backing data for frame */
if (av_image_alloc(frame->data, frame->linesize, frame->width,
frame->height, frame->format, 32) < 0) {
exit(1);
}
/* open the output file, if the container needs it */
if (!(container_format->flags & AVFMT_NOFILE)) {
ret = avio_open(&container_format_context->pb, out_file, AVIO_FLAG_WRITE);
if (ret < 0) {
log(ERROR, "Error occurred while opening output file: %s\n",
av_err2str(ret));
exit(1);
}
}
/* write the stream header, if needed */
ret = avformat_write_header(container_format_context, NULL);
if (ret < 0) {
log(ERROR, "Error occurred while writing output file header: %s\n",
av_err2str(ret));
}
The code to encode a frame is here:
/* Init video packet */
AVPacket packet;
av_init_packet(&packet);
/* Request that encoder allocate data for packet */
packet.data = NULL;
packet.size = 0;
/* Write frame to video */
int got_data;
if (avcodec_encode_video2(avcontext, &packet, frame, &got_data) < 0) {
log(WARNING, "Error encoding frame #%" PRId64,
video_struct->next_pts);
return -1;
}
/* Write corresponding data to file */
if (got_data) {
if (packet.pts != AV_NOPTS_VALUE) {
packet.pts = av_rescale_q(packet.pts, video_struct->output_stream->codec->time_base, video_struct->output_stream->time_base);
}
if (packet.dts != AV_NOPTS_VALUE) {
packet.dts = av_rescale_q(packet.dts, video_struct->output_stream->codec->time_base, video_struct->output_stream->time_base);
}
write_packet(video_struct, &packet, packet.size);
av_packet_unref(&packet);
}
And the code to write the packet to the video stream:
static int write_packet(video_struct* video, void* data, int size) {
int ret;
/* use AVStream is not null, otherwise write to output fd */
AVPacket *pkt = (AVPacket*) data;
pkt->stream_index = video->output_stream->index;
ret = av_interleaved_write_frame(video->container_format_context, pkt);
if (ret != 0) {
return -1;
}
/* Data was written successfully */
return ret;
}
Solved this issue. The problem was that I wasn't assigning global headers to the container if the container required it. While assigning properties like height, width, bit rate and so forth to the avcodec_context, I added
if (container_format_context->oformat->flags & AVFMT_GLOBALHEADER) {
avcodec_context->flags |= CODEC_FLAG_GLOBAL_HEADER;
}
which seems to have fixed the issue.
Related
I have a binary file with raw h264 data which is arranged like that
NAL(SPS), NAL(PPS), NAL(Frame), NAL(SPS), NAL(PPS)....
and i want to mux it (without encode) into a mp4 container.
The muxing.c in the ffmpeg example do the encoding of yuv data, but it is different from my case, and i have no ideas how to change the example to do what i want to do...
I knew the commaned ffmpeg -i h264file -c copy h264.mp4 can do what i want to do, but i have to do it in my program, so i need to know how to use the ffmpeg api to do the same thing, but so far, i cannot find any simple example to do it. Is there anyone has hint on how to do it?? Thanks
Updated, i have write the code as below from the reference, it seems can create the mp4 but the time is not correct, it lost the frame rate information and the time information, it play very fast.
av_register_all();
int ret;
AVDictionary *opt = NULL;
//bool is264 = true;
const char * inputFileName = "input.264";
const char * outputFileName = "output.mp4";
AVFormatContext *ic = avformat_alloc_context();
if((ret = avformat_open_input(&ic, inputFileName, NULL, NULL)) < 0)
return -1;//
// Get format info (retrieve stream information)
if ((ret = avformat_find_stream_info(ic, NULL)) < 0)
return ret; // Couldn't find stream information
for (int i = 0; i < ic->nb_streams; i++)
{
AVStream *stream;
AVCodecContext *codec_ctx;
stream = ic->streams[i];
codec_ctx = stream->codec;
/* Reencode video & audio and remux subtitles etc. */
if (codec_ctx->codec_type == AVMEDIA_TYPE_VIDEO
|| codec_ctx->codec_type == AVMEDIA_TYPE_AUDIO) {
/* Open decoder */
ret = avcodec_open2(codec_ctx, avcodec_find_decoder(codec_ctx->codec_id), NULL);
if (ret < 0) {
//av_log(NULL, AV_LOG_ERROR, "Failed to open decoder for stream #%u\n", i);
return ret;
}
}
}
// Dump information about file onto standard error
av_dump_format(ic, 0, inputFileName, 0);
AVFormatContext *oc;
avformat_alloc_output_context2(&oc, NULL, NULL, outputFileName);
if (!oc) {
//printf("Could not deduce output format from file extension: using MPEG.\n");
//avformat_alloc_output_context2(&oc, NULL, "mpeg", outputFileName);
return -1;
}
AVStream *ist = ic->streams[0];
AVCodec *out_vid_codec = avcodec_find_encoder(oc->oformat->video_codec);
if (NULL == out_vid_codec)
return -1; // Couldn't find video encoder
AVStream *out_vid_strm = avformat_new_stream(oc, out_vid_codec);
if (NULL == out_vid_strm)
return -1; // Couldn't output video stream
ret = avcodec_copy_context(out_vid_strm->codec, ist->codec);
if (ret < 0)
return ret; // Failed to copy context
ret = avio_open(&oc->pb, outputFileName, AVIO_FLAG_WRITE);
ret = avformat_write_header(oc, NULL);
AVPacket pkt;
while(1)
{
AVStream *in_stream, *out_stream;
ret = av_read_frame(ic, &pkt);
if (ret < 0)
break;
pkt.stream_index = 0;
in_stream = ic->streams[pkt.stream_index];
out_stream = oc->streams[pkt.stream_index];
pkt.pts = av_rescale_q_rnd(pkt.pts, in_stream->time_base, out_stream->time_base, (AVRounding)(AV_ROUND_NEAR_INF | AV_ROUND_PASS_MINMAX));
pkt.dts = av_rescale_q_rnd(pkt.dts, in_stream->time_base, out_stream->time_base, (AVRounding)(AV_ROUND_NEAR_INF | AV_ROUND_PASS_MINMAX));
pkt.duration = av_rescale_q(pkt.duration, in_stream->time_base, out_stream->time_base);
pkt.pos = -1;
//log_packet(ofmt_ctx, &pkt, "out");
ret = av_interleaved_write_frame(oc, &pkt);
if (ret < 0) {
fprintf(stderr, "Error muxing packet\n");
break;
}
av_packet_unref(&pkt);
}
av_write_trailer(oc);
I have a problem with decoding of gray images encoded with FFV1 codec.
I successfully encode 16 bit gray image (with avcodec_receive_packet(...) function) and save AvPacket data to file. Then I read this data from file and try to decode (with avcodec_decode_video2 or avcodec_send_packet/avcodec_receive_frame) with no success:
when I try to decode packet with avcodec_decode_video2 function I get an error "Access violation occurred, unable to write location 0x0000000000000000".
when I try to decode packet with avcodec_send_packet/avcodec_receive_frame functions I get an error "chroma shift parameters 7 0 are invalid".
I compared packets after encoding and before decoding and all fields and values seems to be the same. I even try to decode packet just after avcodec_receive_packet (encoding function), however with the same error.
I use the 4.0 version of ffmpeg and the program is based on decode_video.c and encode_video.c examples.
When I use containers (eg. avi) to support read/write encoded images from file (based on demuxing_decoding.c and muxing.c examples) I successfully encode and decode frames with FFV1. However I cannot use containers, because I want to encode frames with different resolutions and mix few video sources together. Additionally the compression level is significantly lower (falls from 2.9 to 2.2) for few hundred of images, what is also very surprising.
So my question is how to correctly save/read (from binary file not container) and prepare AVPacker for decoding with FFV1.
Any help is greatly appreciated.
The decoding code:
extern "C" {
#include <libavcodec/avcodec.h>
#include <libavutil/opt.h>
#include <libavutil/imgutils.h>
}
#pragma warning(disable: 4996)
#define INBUF_SIZE 4096
#define FF_INPUT_BUFFER_PADDING_SIZE 64
uint8_t endcode[4];
AVCodecContext *c, c2;
AVCodec *codec;
int i, ret, x, y;
AVFrame *frame;
AVPacket *pkt, *pkt_temp;
FILE *encodedVideoFile;
AVDictionary *opts = NULL;
uint8_t *video_dst_data[4] = { NULL };
int video_dst_linesize[4];
int imageSize;
uint8_t inbuf[INBUF_SIZE + AV_INPUT_BUFFER_PADDING_SIZE];
/* flush the encoder */
frame = NULL;
encode();
/* add sequence end code to have a real MPEG file */
//fwrite(endcode, 1, sizeof(endcode), encodedVideoFile);
fclose(encodedVideoFile);
avcodec_free_context(&c);
av_frame_free(&frame);
av_packet_free(&pkt);
}
void initDecoding(const char *filename)
{
/* set end of buffer to 0 (this ensures that no overreading happens for damaged MPEG streams) */
memset(inbuf + INBUF_SIZE, 0, AV_INPUT_BUFFER_PADDING_SIZE);
/* find the MPEG-1 video decoder */
codec = avcodec_find_decoder(AV_CODEC_ID_FFV1);
if (!codec) {
fprintf(stderr, "Codec not found\n");
exit(1);
}
c = avcodec_alloc_context3(codec);
if (!c) {
fprintf(stderr, "Could not allocate video codec context\n");
exit(1);
}
/* resolution must be a multiple of two */
c->width = 1280;
c->height = 484;
/* frames per second */
c->time_base.den = 1;
c->time_base.num = 10;
c->bits_per_raw_sample = 16;
c->framerate.den = 10;
c->framerate.num = 1;
c->pix_fmt = AV_PIX_FMT_GRAY16;
//Version of FFV1 codec
c->level = 3;
/* Init the decoders, with or without reference counting */
av_dict_set(&opts, "refcounted_frames", 0 ? "1" : "0", 0);
if ((ret = avcodec_open2(c, codec, &opts)) < 0) {
return;
}
if (avcodec_open2(c, codec, NULL) < 0) {
fprintf(stderr, "Could not open codec\n");
exit(1);
}
ret = av_image_alloc(video_dst_data, video_dst_linesize,
c->width, c->height, c->pix_fmt, 4);
if (ret < 0) {
fprintf(stderr, "Could not allocate raw video buffer\n");
}
encodedVideoFile = fopen(filename, "rb");
if (!encodedVideoFile) {
fprintf(stderr, "Could not open %s\n", filename);
exit(1);
}
frame = av_frame_alloc();
if (!frame) {
fprintf(stderr, "Could not allocate video frame\n");
exit(1);
}
frame->format = c->pix_fmt;
frame->width = c->width;
frame->height = c->height;
ret = av_frame_get_buffer(frame, 32);
if (ret < 0) {
fprintf(stderr, "Could not allocate the video frame data\n");
exit(1);
}
/* make sure the frame data is writable */
ret = av_frame_make_writable(frame);
if (ret < 0)
exit(1);
}
void closeDecoding()
{
fclose(encodedVideoFile);
av_parser_close(parser);
avcodec_free_context(&c);
av_frame_free(&frame);
av_packet_free(&pkt);
}
void decodePacket()
{
size_t data_size;
int *got_frame = 0;
read_packt_from_file(pkt, encodedVideoFile);
ret = av_frame_is_writable(frame);
//First decoding function
/*ret = avcodec_decode_video2(c, frame, got_frame, pkt);
if (ret < 0) {
fprintf(stderr, "Error decoding video frame (%s)\n");
}*/
ret = avcodec_send_packet(c, pkt);
if (ret < 0) {
fprintf(stderr, "Error sending a packet for decoding\n");
exit(1);
}
while (ret >= 0) {
ret = avcodec_receive_frame(c, frame);
if (ret == AVERROR(EAGAIN) || ret == AVERROR_EOF)
return;
else if (ret < 0) {
fprintf(stderr, "Error during decoding\n");
exit(1);
}
printf("saving frame %3d\n", c->frame_number);
fflush(stdout);
}
}
size_t read_packt_from_file(AVPacket *packet, FILE *file)
{
size_t ret = 0;
int size;
uint8_t * data;
//av_packet_from_data
ret = fread(packet, sizeof(AVPacket), 1, file);
size = packet->size;
data = new uint8_t[size];
ret = fread(data, size, 1, file);
av_new_packet(packet, size);
av_packet_from_data(packet, data, size);
return ret;
}
//To write encoded AVPacket
size_t write_packt_to_file(AVPacket *packet, FILE *file)
{
size_t ret = 0;
ret = fwrite(packet, sizeof(AVPacket), 1, file);
ret = fwrite(packet->data, packet->size, 1, file);
if (packet->buf) {
fwrite(packet->buf->data, packet->buf->size, 1, file);
}
fflush(file);
return ret;
}
I use ffmpeg to decode h264 and transcode to jpeg.
I used to use the 2.8 version, now change to 4.0.
The new version avcodec_decode_video2 is deprecated, so i use the new API (send, receive).
I found that the new version ffmpeg is much slower than the old one when decoding h264.
why is that? How to reach the original speed?
Here's the decoder init codes:
version 2.8
//2.8
if (avformat_open_input(&format_, path, NULL, &options) != 0) {
printf("FileMediaBase: open input failed\n");
return -1;
}
if (avformat_find_stream_info(format_, NULL) < 0) {
printf("FileMediaBase: avformat_find_stream_info failed\n");
avformat_close_input(&format_);
return -1;
}
AVCodecContext *code_ctx = 0;
AVCodec *codec = 0;
// Find the first audio and video stream
for (int i = 0; i < format_->nb_streams; i++) {
code_ctx = format_->streams[i]->codec;
codec = avcodec_find_decoder(code_ctx->codec_id);
if (code_ctx->codec_type == AVMEDIA_TYPE_VIDEO) {
vcode_ctx_ = code_ctx;
vstream_ = format_->streams[i];
}else if (code_ctx->codec_type == AVMEDIA_TYPE_AUDIO) {
acode_ctx_ = code_ctx;
astream_ = format_->streams[i];
}
// Open the codec
if (!codec || (avcodec_open2(code_ctx, codec, NULL) < 0)) {
printf("avcodec_open2() failed\n");
avformat_close_input(&format_);
return -1;
}
version 4.0
if (avformat_open_input(&format_, path, NULL, &options) != 0) {
printf("FileMediaBase: open input failed\n");
return -1;
}
if (avformat_find_stream_info(format_, NULL) < 0) {
printf("FileMediaBase: avformat_find_stream_info failed\n");
avformat_close_input(&format_);
return -1;
}
AVStream *st;
AVCodec *dec = NULL;
AVDictionary *opts = NULL;
ret = av_find_best_stream(format_, AVMEDIA_TYPE_VIDEO, -1, -1, NULL, 0);
if (ret < 0) {
return ret;
}
else {
stream_index = ret;
st = format_->streams[stream_index];
/* find decoder for the stream */
dec = avcodec_find_decoder(st->codecpar->codec_id);
if (!dec) {
fprintf(stderr, "Failed to find %s codec\n",
av_get_media_type_string(type));
return AVERROR(EINVAL);
}
/* Allocate a codec context for the decoder */
*dec_ctx = avcodec_alloc_context3(dec);
if (!*dec_ctx) {
fprintf(stderr, "Failed to allocate the %s codec context\n",
av_get_media_type_string(type));
return AVERROR(ENOMEM);
}
/* Copy codec parameters from input stream to output codec context */
if ((ret = avcodec_parameters_to_context(*dec_ctx, st->codecpar)) < 0) {
fprintf(stderr, "Failed to copy %s codec parameters to decoder context\n",
av_get_media_type_string(type));
avcodec_free_context(dec_ctx);
return ret;
}
/*if (dec->id == AV_CODEC_ID_H264) {
(*dec_ctx)->flags |= AV_CODEC_FLAG_TRUNCATED;
}*/
/* Init the decoders, with or without reference counting */
//av_dict_set(&opts, "refcounted_frames", refcount ? "1" : "0", 0);
if ((ret = avcodec_open2(*dec_ctx, dec, &opts)) < 0) {
fprintf(stderr, "Failed to open %s codec\n",
av_get_media_type_string(type));
avcodec_free_context(dec_ctx);
return ret;
}
I print out the time spent on some API
I've solved this problem, change the number of decoding threads(default is 1 in version 4.0) to 6(default is 6 in version 2.8).
(*dec_ctx)->thread_count = 6;
I have faced the same issue when decoding h264 video stream using ffmpeg 2.8.14 and 4.0.1 versions.
In both cases when accessing the codec context after avformat_find_stream_info()
AVFormatContext *pFormatCtx;
AVCodecContext* pCodecCtxOrig;
...
avformat_find_stream_info(pFormatCtx, NULL)
...
for (uint i = 0; i < pFormatCtx->nb_streams; i++)
{
if (pFormatCtx->streams[i]->codec->codec_type == AVMEDIA_TYPE_VIDEO)
{
pCodecCtxOrig = pFormatCtx->streams[i]->codec;
...
}
}
then the pCodecCtxOrig::thread_count is actually set to 0, which means that ffmpeg library finds suitable thread_count by itself based on CPU cores when opening codec context with avcodec_open2(). In my case the thread count was set to 5.
In case of 2.8.14 the pCodecCtxOrig::thread_count is set to 0 and when doing a copy from it
AVCodecContext* pCodecCtxOrig;
AVCodecContext* pCodecCtx;
....
avcodec_copy_context(pCodecCtx, pCodecCtxOrig)
the thread_count is copied as well.
In case of 4.0.1 the avcodec_copy_context() is deprecated and the avcodec_parameters_to_context() should be used instead. The coping struct AVCodecParameters is not containing thread_count thus the info gets lost. Thread count remains the default value 1, which is set by avcodec_alloc_context3. To overcome the problem set thread_count to 0 before opening the codec context.
I am trying to decode opus using libavcodec. I am able to do it using libopus library alone. But I am trying to acheive same using libavcodec. I am trying to figure it out Why its not working in my case. I have an rtp stream and trying to decode it. The result in decoded packet is same as input. Decoded frame normally contain pcm values instead of that Im receving opus frame that actually I send. Please help me.
av_register_all();
avcodec_register_all();
AVCodec *codec;
AVCodecContext *c = NULL;
AVPacket avpkt;
AVFrame *decoded_frame = NULL;
av_init_packet(&avpkt);
codec = avcodec_find_decoder(AV_CODEC_ID_OPUS);
if (!codec) {
printf("Codec not found\n");
exit(1);
}
c = avcodec_alloc_context3(codec);
if (!c) {
printf("Could not allocate audio codec context\n");
exit(1);
}
/* put sample parameters */
c->sample_rate = 48000;
c->request_sample_fmt = AV_SAMPLE_FMT_FLT;
c->channels = 2;
/* open it */
if (avcodec_open2(c, codec, NULL) < 0) {
printf("Could not open codec\n");
exit(1);
}
AVPacket avpkt;
AVFrame *decoded_frame = NULL;
av_init_packet(&avpkt);
avpkt.data = Buffer; // Buffer is packet data here
avpkt.size = len; // length of the packet
int i, ch;
if (!decoded_frame) {
if (!(decoded_frame = av_frame_alloc())) {
RELAY_SERVER_PRINT("Could not allocate audio frame\n");
exit(1);
}
}
int ret;
int got_frame = 0;
ret = avcodec_decode_audio4(c, decoded_frame, &got_frame, &avpkt);
if (ret < 0) {
fprintf(stderr, "Error decoding audio frame (%s)\n", av_err2str(ret));
return ret;
}
printf("length %i\n", decoded_frame->pkt_size);
I had the same problem. My stream was encoded with 8kHz and ffmpeg is always initializing libopus with 48kHz (hard-coded).
See ffmpeg code snippet:
static av_cold int libopus_decode_init(AVCodecContext *avc)
{
(...)
avc->sample_rate = 48000;
avc->sample_fmt = avc->request_sample_fmt == AV_SAMPLE_FMT_FLT ?
AV_SAMPLE_FMT_FLT : AV_SAMPLE_FMT_S16;
(...)
}
I've replaced that by:
if (avc->sample_rate == 0)
avc->sample_rate = 48000;
and decoding works now. I wonder if this decoder supports dynamic bitrate changes.
The length of the raw frame has to be calculated by:
int frame_size = decoded_frame->nb_samples * av_get_bytes_per_sample(decoded_frame->sample_fmt);
I have a program which is supposed to demux input mpeg-ts, transcode the mpeg2 into h264 and then mux the audio alongside the transcoded video. When I open the resulting muxed file with VLC I neither get audio nor video. Here is the relevant code.
My main worker loop is as follows:
void
*writer_thread(void *thread_ctx) {
struct transcoder_ctx_t *ctx = (struct transcoder_ctx_t *) thread_ctx;
AVStream *video_stream = NULL, *audio_stream = NULL;
AVFormatContext *output_context = init_output_context(ctx, &video_stream, &audio_stream);
struct mux_state_t mux_state = {0};
//from omxtx
mux_state.pts_offset = av_rescale_q(ctx->input_context->start_time, AV_TIME_BASE_Q, output_context->streams[ctx->video_stream_index]->time_base);
//write stream header if any
avformat_write_header(output_context, NULL);
//do not start doing anything until we get an encoded packet
pthread_mutex_lock(&ctx->pipeline.video_encode.is_running_mutex);
while (!ctx->pipeline.video_encode.is_running) {
pthread_cond_wait(&ctx->pipeline.video_encode.is_running_cv, &ctx->pipeline.video_encode.is_running_mutex);
}
while (!ctx->pipeline.video_encode.eos || !ctx->processed_audio_queue->queue_finished) {
//FIXME a memory barrier is required here so that we don't race
//on above variables
//fill a buffer with video data
OERR(OMX_FillThisBuffer(ctx->pipeline.video_encode.h, omx_get_next_output_buffer(&ctx->pipeline.video_encode)));
write_audio_frame(output_context, audio_stream, ctx); //write full audio frame
//FIXME no guarantee that we have a full frame per packet?
write_video_frame(output_context, video_stream, ctx, &mux_state); //write full video frame
//encoded_video_queue is being filled by the previous command
}
av_write_trailer(output_context);
//free all the resources
avcodec_close(video_stream->codec);
avcodec_close(audio_stream->codec);
/* Free the streams. */
for (int i = 0; i < output_context->nb_streams; i++) {
av_freep(&output_context->streams[i]->codec);
av_freep(&output_context->streams[i]);
}
if (!(output_context->oformat->flags & AVFMT_NOFILE)) {
/* Close the output file. */
avio_close(output_context->pb);
}
/* free the stream */
av_free(output_context);
free(mux_state.pps);
free(mux_state.sps);
}
The code for initialising libav output context is this:
static
AVFormatContext *
init_output_context(const struct transcoder_ctx_t *ctx, AVStream **video_stream, AVStream **audio_stream) {
AVFormatContext *oc;
AVOutputFormat *fmt;
AVStream *input_stream, *output_stream;
AVCodec *c;
AVCodecContext *cc;
int audio_copied = 0; //copy just 1 stream
fmt = av_guess_format("mpegts", NULL, NULL);
if (!fmt) {
fprintf(stderr, "[DEBUG] Error guessing format, dying\n");
exit(199);
}
oc = avformat_alloc_context();
if (!oc) {
fprintf(stderr, "[DEBUG] Error allocating context, dying\n");
exit(200);
}
oc->oformat = fmt;
snprintf(oc->filename, sizeof(oc->filename), "%s", ctx->output_filename);
oc->debug = 1;
oc->start_time_realtime = ctx->input_context->start_time;
oc->start_time = ctx->input_context->start_time;
oc->duration = 0;
oc->bit_rate = 0;
for (int i = 0; i < ctx->input_context->nb_streams; i++) {
input_stream = ctx->input_context->streams[i];
output_stream = NULL;
if (input_stream->index == ctx->video_stream_index) {
//copy stuff from input video index
c = avcodec_find_encoder(CODEC_ID_H264);
output_stream = avformat_new_stream(oc, c);
*video_stream = output_stream;
cc = output_stream->codec;
cc->width = input_stream->codec->width;
cc->height = input_stream->codec->height;
cc->codec_id = CODEC_ID_H264;
cc->codec_type = AVMEDIA_TYPE_VIDEO;
cc->bit_rate = ENCODED_BITRATE;
cc->time_base = input_stream->codec->time_base;
output_stream->avg_frame_rate = input_stream->avg_frame_rate;
output_stream->r_frame_rate = input_stream->r_frame_rate;
output_stream->start_time = AV_NOPTS_VALUE;
} else if ((input_stream->codec->codec_type == AVMEDIA_TYPE_AUDIO) && !audio_copied) {
/* i care only about audio */
c = avcodec_find_encoder(input_stream->codec->codec_id);
output_stream = avformat_new_stream(oc, c);
*audio_stream = output_stream;
avcodec_copy_context(output_stream->codec, input_stream->codec);
/* Apparently fixes a crash on .mkvs with attachments: */
av_dict_copy(&output_stream->metadata, input_stream->metadata, 0);
/* Reset the codec tag so as not to cause problems with output format */
output_stream->codec->codec_tag = 0;
audio_copied = 1;
}
}
for (int i = 0; i < oc->nb_streams; i++) {
if (oc->oformat->flags & AVFMT_GLOBALHEADER)
oc->streams[i]->codec->flags |= CODEC_FLAG_GLOBAL_HEADER;
if (oc->streams[i]->codec->sample_rate == 0)
oc->streams[i]->codec->sample_rate = 48000; /* ish */
}
if (!(fmt->flags & AVFMT_NOFILE)) {
fprintf(stderr, "[DEBUG] AVFMT_NOFILE set, allocating output container\n");
if (avio_open(&oc->pb, ctx->output_filename, AVIO_FLAG_WRITE) < 0) {
fprintf(stderr, "[DEBUG] error creating the output context\n");
exit(1);
}
}
return oc;
}
Finally this is the code for writing audio:
static
void
write_audio_frame(AVFormatContext *oc, AVStream *st, struct transcoder_ctx_t *ctx) {
AVPacket pkt = {0}; // data and size must be 0;
struct packet_t *source_audio;
av_init_packet(&pkt);
if (!(source_audio = packet_queue_get_next_item_asynch(ctx->processed_audio_queue))) {
return;
}
pkt.stream_index = st->index;
pkt.size = source_audio->data_length;
pkt.data = source_audio->data;
pkt.pts = source_audio->PTS;
pkt.dts = source_audio->DTS;
pkt.duration = source_audio->duration;
pkt.destruct = avpacket_destruct;
/* Write the compressed frame to the media file. */
if (av_interleaved_write_frame(oc, &pkt) != 0) {
fprintf(stderr, "[DEBUG] Error while writing audio frame\n");
}
packet_queue_free_packet(source_audio, 0);
}
A resulting mpeg4 file can be obtained from here: http://87.120.131.41/dl/mpeg4.h264
I have ommited the write_video_frame code since it is a lot more complicated and I might be making something wrong there as I'm doing timebase conversation etc. For audio however I'm doing 1:1 copy. Each packet_t packet contains data from av_read_frame from the input mpegts container. In the worst case I'd expect that my audio is working and not my video. However I cannot get either of those to work. Seems the documentation is rather vague on making things like that - I've tried both libav and ffmpeg irc channels to no avail. Any information regarding how I can debug the issue will be greatly appreciated.
When different containers yield different results in libav it is almost always a timebase issue. All containers have a time_base that they like, and some will accept custom values... sometimes.
You must rescale the time base before putting it in the container. Generally tinkering with the mux state struct isn't something you want to do and I think what you did there doesn't do what you think. Try printing out all of the timebases to find out what they are.
Each frame you must recalculate PTS at least. If you do it before you call encode the encoder will produce the proper DTS. Do the same for the audio, but generally set the DTS it to AV_NO_PTS and sometimes you can get away with setting the audio PTS to that as well. To rescale easily use the av_rescale(...) functions.
Be careful assuming that you have MPEG-2 data in a MPEG-TS container, that is not always true.