I am getting the following errors when decoding H.264 frames received from the remote end of a H.264 based SIP video call. Appreciate any help in understanding there errors.
non-existing PPS 0 referenced
decode_slice_header error
non-existing PPS 0 referenced
decode_slice_header error
no frame!
non-existing PPS 0 referenced
decode_slice_header error
non-existing PPS 0 referenced
decode_slice_header error
no frame!
That just means that ffmpeg has not seen a keyframe yet, which carries SPS and PPS information. SPS and PPS are crucial in decoding an incoming frame/slice. Keyframes are sent periodically (i.e. every 5-10 seconds or more); so if it turns out that you joined a stream before the keyframe arrived; you will see this warning for every frame until a keyframe shows up.
As soon as the keyframe shows up from the wire, ffmpeg will have enough information to decode that frame (and any subsequent frames until the next keyframe), so those warnings will go away.
You need to add frames sps and pps information. ffmpeg needs these frames to decode. You can find these values in the SDP file.
In the SDP file, you should look for NAL units, you can see something like that:
z0IAHukCwS1xIADbugAzf5GdyGQl, aM4xUg
these values, base64 encoded, should be converted to binary. I am using wireshark and wireshark converts these values automatically for you. After that you have sps and pps values.
Now you have to add these NAL blocks before the data frame:
00 00 00 01 sps 00 00 00 01 pps 00 00 00 01 data
for h264 these block is what I have been using to decode.
To decode a frame or a slice, sliceHeader is decoded, which refers to a PPS or "Picture Parameter Set". It has information regarding the specifics of the frame like width, height etc.
I guess your data is coming through a streaming input channel, in which case SPS and PPS would have been sent earlier in the stream.
You may have to concatenate the same to your stream.
Related
I need to decode an H264 stream that comes from a live DVR camera.
To facilitate the example, I stored the RAW stream from the DVR camera in the following file (test.h264): http://f.zins.com.br/test.h264
To decode the live stream, I followed the following ffmpeg example: https://github.com/FFmpeg/FFmpeg/blob/master/doc/examples/decode_video.c
If I open the .h264 test with VLC, the images look perfect.
If you decode the .h264 test with ffmpeg using avformat_open_input and avformat_find_stream_info, the images also look perfect.
But if I decode using the https://github.com/FFmpeg/FFmpeg/blob/master/doc/examples/decode_video.c example, the images are all distorted. I think this happens because along with the H264 stream can have audio together.
Enabling the debugging of ffmpeg, it shows a lot of the following errors:
[h264 # 092a9b00] Invalid NAL unit 0, skipping.
[h264 # 092a9b00] Invalid NAL unit 0, skipping.
[h264 # 092a9b00] Invalid NAL unit 0, skipping.
[h264 # 092a9b00] error while decoding MB 16 1, bytestream -28
[h264 # 092a9b00] Invalid NAL unit 8, skipping.
[h264 # 092a9b00] Invalid NAL unit 8, skipping.
[h264 # 092a9b00] Invalid NAL unit 8, skipping.
Is there a way for me to only filter the video and ignore the audio from a live stream?
Otherwise, is there any solution to decode the test.h264 using the decode_video.c example without distorting the frames?
The distorted frame sometimes looks like the image below, and sometimes it gets almost all gray.
The stream has mpeg wrapper around raw h264 packets, and we need to demux them first. If you cannot provide a URL with some protocol supported by ffmpeg (e.g. udp://) like, you should build custom AVIOContext for your live stream and pass it to
avformat_open_input(&fmt_ctx, NULL, NULL, NULL)
similar to this example.
Now you can start the usual demuxer loop with
av_read_frame(fmt_ctx, &pkt)
I wrote a RTP server to receive the RTP packets which are sent by command ffmpeg -i test.mp4 rtp rtp://ip:port (client) and the server could get the nal type 24 (STAP-A).
And I want to use the server to retrieve the spa and pps from the first nal(type 24) instead of info from ffmpeg command.
Is it possible SPS and PPS would be aggregated in one nal ?
for example
[RTP header][nal header(type 24)][nal1 header][nal1 size][nal1 payload][nal2 header][nal2 size][nal2 payload]...
thanks
It's highly likely that the STAP-A consists of the SPS and PPS: these NAL units are usually at the beginning of the stream, small and can be aggregated into a STAP A. If the IDR is small enough, it might also be part of the STAP, but usually this is to big and will be sent separately.
The best thing to verify this is to split the STAP-A into the original NAL units (See RFC6184) and check for types 7 (SPS) and 8 (PPS).
I am trying to decode H264 video frames received through RTSP streaming.
I followed this post: How to process raw UDP packets so that they can be decoded by a decoder filter in a directshow source filter
I was able to identify start of the frame & end of the frame in RTP packets and reconstructed my Video Frame.
But I didnt receive any SPS,PPS data from my RTSP session. I looked for string "sprop-parameter-sets" in my SDP(Session Description Protocol) and there was none.
Reconstructing Video Frame from RTP Packets:
Payload in the first RTP Packet goes like this : "1c 80 00 00 01 61 9a 03 03 6a 59 ff 97 e0 a9 f6"
This says that its a fragmented data("1C") and start of the frame("80"). I copied the rest of the payload data(except the first 2 bytes "1C 80").
Following RTP Packets have the "Payload" start with "1C 00" which is continuation of the frame data. I kept adding payload data(except the first 2 bytes "1C 00") into the byte buffer for all the following RTP Packets.
When I get the RTP packet with payload starts with "1C 40", which is end of the frame, I copied the rest of the payload data(except the first 2 bytes "1C 40") of that RTP Packet into the byte buffer.
Thus I reconstructed the Video Frame into the byte buffer.
Then I prepended 4 bytes [0x00, 0x00 , 0x00, 0x01] to the byte buffer before sending to the decoder, because I didnt receive any SPS, PPS NAL bytes.
When I send this byte buffer to the decoder, decoder fails when it tries to initialize sws Context.
Am I sending the NAL bytes and video frame data correctly?
How do I convert h264 avcc stream to SampleBuffer in iOS 8
Need the sample buffer to perform decompression session
It has to be in Mpeg-4 format. You will need an algorithm to read the raw byte stream looking for NAL separator codes (usually 00000001) and then look for the NAL unit code in the next byte.
Typical NAL Unit codes are 67 (PPS), 68 (SPS), 41 (P Frame) and 65 (I Frame)
Generally the first will be the SPS, capture these bytes in an NSMutable Data, then capture the PPS in another. use CMVideoFormatDescriptionCreateFromH264ParameterSets with the SPS and PPS data to create a CMFormatDescription.
Use the CMFormatDescription to create a VTDecompressionSession using VTDecompressionSessionCreate.
When you have these, capture the NALUnit payload into a CMBlockBuffer making sure to replace the separator code with a 4 byte length code (the length of the NalUnit including the unit code) and then using this and the Format Description you created before, create a CMSampleBuffer.
You now have everything you need to use VTDecompressionSessionDecodeFrame.
I am trying to stream data encoded using FFMPEg using live555. I have a custom framesource that sends the data to sink but I am unable to figure out how to set SPS and PPS in the framer. I understand that extradata contains this information but I saw only SPS in that. Does extradata changes while encoding by FFMPeg? If yes how and when we need to update this information in live555 framer.
Does anyone have a working sample using FFMpeg and live555 to stream H264
Live555 is simply a streaming tool, it does not do any encoding.
The SPS and PPS are NAL units within the encoded H264 stream (or the output from your FFMPEG implementation) (see some info here: http://www.cardinalpeak.com/blog/the-h-264-sequence-parameter-set/).
If you want to change the SPS or PPS information you need to do it in FFMPEG.
Examples of FFMPEG and Live555 working together to stream MPG2 and H264 streams are here:
https://github.com/alm865/FFMPEG-Live555-H264-H265-Streamer/
As for streaming a H264 stream, you need to break the output from FFMPEG into NAL units before you send if off to the discrete framer for it to work correctly. You must also strip the leading and trailing NAL bits from the packet (i.e. remove the NAL identifier 0x00 0x00 0x00 0x01).
Live555 will automatically read these in and update as necessary.