It seems that MediaSource and Progressive playback use the different demuxer. ChunkDemuxer is used for MediaSource, ShellDemuxer is used for Progressive playback.
In ShellParser.cpp implementation:
PipelineStatus ShellParser::Construct(
scoped_refptr<ShellDataSourceReader> reader,
scoped_refptr<ShellParser>* parser,
const scoped_refptr<MediaLog>& media_log) {
DCHECK(parser);
DCHECK(media_log);
*parser = NULL;
// download first 16 bytes of stream to determine file type and extract basic
// container-specific stream configuration information
uint8 header[kInitialHeaderSize];
int bytes_read = reader->BlockingRead(0, kInitialHeaderSize, header);
if (bytes_read != kInitialHeaderSize) {
return DEMUXER_ERROR_COULD_NOT_PARSE;
}
// attempt to construct mp4 parser from this header
return ShellMP4Parser::Construct(reader, header, parser, media_log);
}
It seems that Cobalt can only demux MP4 container(Only ShellMP4Parser) for progressive playback.
Is it known status for Cobalt ?how can we support webm progressive playback on the device?
Cobalt will not support WebM/VP9 progressive playback. We changed the Progressive Conformance test to change VP9 to H264. This will be pushed soon.
https://github.com/youtube/js_mse_eme/commit/d7767e13be7ed8b8bdb2efda39337a4a2fb121ba
Related
I use h264_qsv to encode video, and it works, the video can be translated to be h264.
But how to force it to generate IDR frame? I use the following code:
encodeFrame->pict_type = AV_PICTURE_TYPE_I;
int ret = avcodec_send_frame(m_ctx, encodeFrame);
but no idr frame was created.
The QSV encoders have a private option to enable IDR frames when the frame is signaled as a I-frame.
Add before opening the encoder.
av_opt_set(m_ctx->priv_data, "forced_idr", "1", 0);
I'm trying to change the max buffer length on my video streaming in clappr video player.
I know that in HLS format the way to do it is like this:
player = new Clappr.Player({
playback: {
hlsjsConfig: {
maxMaxBufferLength: 30
}}})
And it's realy working for HLS videos,
I'm looking for equivalent way to do it with MPEG-dash foramt
How are you playing DASH in Clappr?
If you are using Shaka, https://github.com/clappr/dash-shaka-playback, set it up as shown at https://github.com/clappr/dash-shaka-playback, using the buffer settings you require as described at https://github.com/google/shaka-player/blob/master/docs/tutorials/network-and-buffering-config.md#buffering-configuration
Eg:
player = new Clappr.Player({
source: '//storage.googleapis.com/shaka-demo-assets/angel-one/dash.mpd',
plugins: [DashShakaPlayback],
shakaConfiguration: {
preferredAudioLanguage: 'pt-BR',
streaming: {
bufferingGoal: 30,
rebufferingGoal: 15,
bufferBehind: 60
}
}
});
I'm trying to use libavcodec library in FFMpeg to decode then re-encode a h264 video.
I have the decoding part working (rendes to an SDL window fine) but when I try to re-encode the frames I get bad data in the re-encoded videos samples.
Here is a cut down code snippet of my encode logic.
EncodeResponse H264Codec::EncodeFrame(AVFrame* pFrame, StreamCodecContainer* pStreamCodecContainer, AVPacket* pPacket)
{
int result = 0;
result = avcodec_send_frame(pStreamCodecContainer->pEncodingCodecContext, pFrame);
if(result < 0)
{
return EncodeResponse::Fail;
}
while (result >= 0)
{
result = avcodec_receive_packet(pStreamCodecContainer->pEncodingCodecContext, pPacket);
// If the encoder needs more frames to create a packed then return and wait for
// method to be called again upon a new frame been present.
// Else check if we have failed to encode for some reason.
// Else a packet has successfully been returned, then write it to the file.
if (result == AVERROR(EAGAIN) || result == AVERROR_EOF)
{
// Higher level logic, dedcodes next frame from source
// video then calls this method again.
return EncodeResponse::SendNextFrame;
}
else if (result < 0)
{
return EncodeResponse::Fail;
}
else
{
// Prepare packet for muxing.
if (pStreamCodecContainer->codecType == AVMEDIA_TYPE_VIDEO)
{
av_packet_rescale_ts(m_pPacket, pStreamCodecContainer->pEncodingCodecContext->time_base,
m_pDecodingFormatContext->streams[pStreamCodecContainer->streamIndex]->time_base);
}
m_pPacket->stream_index = pStreamCodecContainer->streamIndex;
int result = av_interleaved_write_frame(m_pEncodingFormatContext, m_pPacket);
av_packet_unref(m_pPacket);
}
}
return EncodeResponse::EncoderEndOfFile;
}
Strange behaviour I notice is that before I get the first packet from avcodec_receive_packet I have to send 50+ frames to avcodec_send_frame.
I built a debug build of FFMpeg and stepping into the code I notice that AVERROR(EAGAIN) is returned by avcodec_receive_packet because of the following in x264encoder::encode in encoder.c
if( h->frames.i_input <= h->frames.i_delay + 1 - h->i_thread_frames )
{
/* Nothing yet to encode, waiting for filling of buffers */
pic_out->i_type = X264_TYPE_AUTO;
return 0;
}
For some reason my code-context (h) never has any frames. I have spent a long time trying to debug ffmpeg and to determine what I'm doing wrong. But have reached the limit of my video codec knowledge (which is little).
I'm testing this with a video that has no audio to reduce complication.
I have created a cut down version of my application and provided a self contained (with ffmpeg and SDL built dependencies) project. Hopefully this can help anyone-one willing to help me :).
Project Link
https://github.com/maxhap/video-codec
After looking into encoder initialisation I found that I have to set the codec AV_CODEC_FLAG_GLOBAL_HEADER before calling avcodec_open2
pStreamCodecContainer->pEncodingCodecContext->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;
This change led to the re-encoded moov box looking much heathier (used MP4Box.js to parse it). However, the video still does not play correctly, the output video has grey frames at the start when played in VLC and won't play in other players.
I have since tried creating an encoding context via the sample code, rather than using my decoding codec parameters. This led to fixing the bad/data or encoding issue. However, my DTS times are scaling to huge numbers
Here is my new codec init
if (pStreamCodecContainer->codecType == AVMEDIA_TYPE_VIDEO)
{
pStreamCodecContainer->pEncodingCodecContext->height = pStreamCodecContainer->pDecodingCodecContext->height;
pStreamCodecContainer->pEncodingCodecContext->width = pStreamCodecContainer->pDecodingCodecContext->width;
pStreamCodecContainer->pEncodingCodecContext->sample_aspect_ratio = pStreamCodecContainer->pDecodingCodecContext->sample_aspect_ratio;
/* take first format from list of supported formats */
if (pStreamCodecContainer->pEncodingCodec->pix_fmts)
{
pStreamCodecContainer->pEncodingCodecContext->pix_fmt = pStreamCodecContainer->pEncodingCodec->pix_fmts[0];
}
else
{
pStreamCodecContainer->pEncodingCodecContext->pix_fmt = pStreamCodecContainer->pDecodingCodecContext->pix_fmt;
}
/* video time_base can be set to whatever is handy and supported by encoder */
pStreamCodecContainer->pEncodingCodecContext->time_base = av_inv_q(pStreamCodecContainer->pDecodingCodecContext->framerate);
pStreamCodecContainer->pEncodingCodecContext->sample_aspect_ratio = pStreamCodecContainer->pDecodingCodecContext->sample_aspect_ratio;
}
else
{
pStreamCodecContainer->pEncodingCodecContext->channel_layout = pStreamCodecContainer->pDecodingCodecContext->channel_layout;
pStreamCodecContainer->pEncodingCodecContext->channels =
av_get_channel_layout_nb_channels(pStreamCodecContainer->pEncodingCodecContext->channel_layout);
/* take first format from list of supported formats */
pStreamCodecContainer->pEncodingCodecContext->sample_fmt = pStreamCodecContainer->pEncodingCodec->sample_fmts[0];
pStreamCodecContainer->pEncodingCodecContext->time_base = AVRational{ 1, pStreamCodecContainer->pEncodingCodecContext->sample_rate };
}
Any ideas why my DTS time is re-scaling incorrectly?
I managed to fix the DTS scalling by using the time_base value directly from the decoding streams.
So
pStreamCodecContainer->pEncodingCodecContext->time_base = m_pDecodingFormatContext->streams[pStreamCodecContainer->streamIndex]->time_base
Instead of
pStreamCodecContainer->pEncodingCodecContext->time_base = av_inv_q(pStreamCodecContainer->pDecodingCodecContext->framerate);
I will create an answer based on all my finding.
To fix the initial problem of a corrupted moov box I had to add the AV_CODEC_FLAG_GLOBAL_HEADER flag to the encoding codec context before calling avcodec_open2.
encCodecContext->flags |= AV_CODEC_FLAG_GLOBAL_HEADER;
The next issue was badly scaled DTS values in the encoded package, this was causing a side effect of the final mp4 duration being in the hundreds of hours long. To fix this I had to change the encoding codec context timebase to be that of the decoding context streams timebase. This is different than using av_inv_q(framerate) as suggested in the avcodec transcoding example.
encCodecContext->time_base = decCodecFormatContext->streams[streamIndex]->time_base;
I can get raw h.264 frames from a camera. (it does NOT contain any network headers, for example rtsp, http).
They are h.264 raw data.
And I push these data to a queue frame by frame.
I googled many ffmpeg example which uses avformat_open_input() with either local file path or network path.
And I can see the video while I save the frames to a file and using avformat_open_input().
My problem is that I want to decode the frames realtime, not after it is saved as a file.
Does anyone have any idea on this?
Thanks!
You do not need avformat, you need avcodec. avformat is for parsing containers and protocols. avcodec is for encoding and decoding elementary streams (what you already have).
AVPacket avpkt; int err, frame_decoded = 0;
AVCodec *codec = avcodec_find_decoder ( AV_CODEC_ID_H264 );
AVCodecContext *codecCtx = avcodec_alloc_context3 ( codec );
avcodec_open2 ( codecCtx, codec, NULL );
// Set avpkt data and size here
err = avcodec_decode_video2 ( codecCtx, avframe, &frame_decoded, &avpkt );
I am using the following code snippets to record screen, and in most situations recorded wmv file is clear enough, but for some part of video it is not very clear (grey color for some parts). What I record is ppt with full screen mode. I am using Windows Media Encoder 9.
Here is my code snippet,
IWMEncSourceGroup SrcGrp;
IWMEncSourceGroupCollection SrcGrpColl;
SrcGrpColl = encoder.SourceGroupCollection;
SrcGrp = (IWMEncSourceGroup)SrcGrpColl.Add("SG_1");
IWMEncVideoSource2 SrcVid;
IWMEncSource SrcAud;
SrcVid = (IWMEncVideoSource2)SrcGrp.AddSource(WMENC_SOURCE_TYPE.WMENC_VIDEO);
SrcAud = SrcGrp.AddSource(WMENC_SOURCE_TYPE.WMENC_AUDIO);
SrcVid.SetInput("ScreenCap://ScreenCapture1", "", "");
SrcAud.SetInput("Device://Default_Audio_Device", "", "");
// Specify a file object in which to save encoded content.
IWMEncFile File = encoder.File;
string CurrentFileName = Guid.NewGuid().ToString();
File.LocalFileName = CurrentFileName;
CurrentFileName = File.LocalFileName;
// Choose a profile from the collection.
IWMEncProfileCollection ProColl = encoder.ProfileCollection;
IWMEncProfile Pro;
for (int i = 0; i < ProColl.Count; i++)
{
Pro = ProColl.Item(i);
if (Pro.Name == "Screen Video/Audio High (CBR)")
{
SrcGrp.set_Profile(Pro);
break;
}
}
encoder.Start();
thanks in advance,
George
I would guess that it's a problem with your encoder profile or settings, and not a problem with the code. If you're using the default "Screen Video/Audio High (CBR)" profile in WME9, it's using a video bitrate of 250Kbps, which is pretty low. I'd suggest creating a custom profile in the Windows Media Encoder Profile Editor Utility. Something like this:
awesomesc.prx
Name: Awesome Screen Profile
Audio: WMA 9.2 CBR (32kbps, 44kHz, mono CBR)
Video: WMV 9 Screen Quality VBR (Video size Same as video input, Frame rate 10fps, Key frame interval 3sec, Video quality 90)
Then just change the code to match the custom profile's name.
if (Pro.Name == "Awesome Screen Profile")
The encoder settings would take a much longer post to go through, but if you have not changed them from the defaults, you should be OK.
The Quality-based VBR algorithm can be pretty amazing, and will likely produce a surprisingly low average bitrate, but if VBR won't work for your needs, you can use the Windows Media Encoder Profile Editor utility to import the schia.prx profile that you're using and tweak the settings to find a higher CBR bitrate that produces acceptable quality.
"Screen Video/Audio Medium (CBR)"
it solved my problem