I recently collected video data where the video was generated as image sequences. However, between different video of the same length, different numbers of frames were acquired, which made me think that the image sequence have varied frame rates between videos. So my question is how do I convert this image sequence back to video with accurate duration between frames. Is there a way to get that information from the date and time it was created using a code? I know ffmpeg seems to be the tools many people use.
I am not sure where to start. I am not very familiar with coding, so already have trouble executing the correct codes.
I'm trying to extract raw streams from devices and files using ffmpeg. I notice the crucial frame information (Video: width, height, pixel format, color space, Audio: sample format) is stored both in the AVCodecContext and in the AVFrame. This means I can access it prior to the stream playing and I can access it for every frame.
How much do I need to account for these values changing frame-to-frame? I found https://ffmpeg.org/doxygen/trunk/demuxing__decoding_8c_source.html#l00081 which indicates that at least width, height, and pixel format may change frame to frame.
Will the color space and sample format also change frame to frame?
Will these changes be temporary (a single frame) or lasting (a significant block of frames) and is there any way to predict for this stream which behavior will occur?
Is there a way to find the most descriptive attributes that this stream is possible of producing, such that I can scale all the lower-quality frames up, but not offer a result that is mindlessly higher-quality than the source, even if this is a device or a network stream where I cannot play all the frames in advance?
The fundamental question is: how do I resolve the flexibility of this API with the restriction that raw streams (my output) do not have any way of specifying a change of stream attributes mid-stream. I imagine I will need to either predict the most descriptive attributes to give the stream, or offer a new stream when the attributes change. Which choice to make depends on whether these values will change rapidly or stay relatively stable.
So, to add to what #szatmary says, the typical use case for stream parameter changes is adaptive streaming:
imagine you're watching youtube on a laptop with various methods of internet connectivity, and suddenly bandwidth decreases. Your stream will automatically switch to a lower bandwidth. FFmpeg (which is used by Chrome) needs to support this.
alternatively, imagine a similar scenario in a rtc video chat.
The reason FFmpeg does what it does is because the API is essentially trying to accommodate to the common denominator. Videos shot on a phone won't ever change resolution. Neither will most videos exported from video editing software. Even videos from youtube-dl will typically not switch resolution, this is a client-side decision, and youtube-dl simply won't do that. So what should you do? I'd just use the stream information from the first frame(s) and rescale all subsequent frames to that resolution. This will work for 99.99% for the cases. Whether you want to accommodate your service to this remaining 0.01% depends on what type of videos you think people will upload and whether resolution changes make any sense in that context.
Does colorspace change? They could (theoretically) in software that mixes screen recording with video fragments, but it's highly unlikely (in practice). Sample format changes as often as video resolution: quite often in the adaptive scenario, but whether you care depends on your service and types of videos you expect to get.
Usually not often, or ever. However, this is based on the codec and are options chosen at encode time. I pass the decoded frames through swscale just in case.
I need to know what would be the most favorable approach for streaming screen content and controlling remote computer (mouse, keyboard). (I would like to build something like "one-click teamviewer")
So my main question is about picking the video compression method for such requirements:
Most information between subsequent frames stays the same
Color depth can be degraded, but the details (text) must remain sharp
It should work on low-end bandwidths: 512k and below
Frames can be dropped
As a general rule of thumb when is it appropriate to make a gif interlaced, a png interlaced and a jpeg progressive?
Especially when publishing the image on the web.
JPEG: YES — use progressive scan. It makes files smaller (each pass gets its own Huffman table), and partial rendering looks quite good.
GIF: NO — it's unlikely to make the file smaller, partial rendering is poor, and it's pointless for animGIFs. It's best not to use GIF at all (yes, even for anims).
PNG: NO — it hurts compression (as data from each pass is statistically quite different). If the image is large, use high-quality JPEG or lossy PNG if possible, as these may load quicker than a pixelated preview of a large lossless PNG.
ImageOptim will automatically change progressive/interlaced formats when it makes files smaller.
Disclaimers for nitpickers:
In case of small and medium-sized images the progressive preview of each image is not going to be visible long enough for the user to appreciate it. Some browsers don't even bother rendering anything until the whole file is downloaded, so it's better to focus on saving bandwidth to get the whole page loaded ASAP.
Non-progressive JPEG is a bit more efficient when the files are tiny (small thumbnails), but then the savings are tiny, too.
iOS Safari has a higher maximum allowed image size for baseline JPEG than progressive, but the right solution there is to serve images at sizes reasonable for mobile in the first place.
My general rule of thumb: don't ever use interlacing. Interlaced formats typically occupy more space, have (slightly) more complexity and less support in decoders, and the alleged advantages for the user experience are at least debatable. Some arguments for PNG, and in general.
Some people like interlaced or "progressive" images, which load
gradually. The theory behind these formats is that the user can at
least look at a fuzzy full-size proxy for the image while all the bits
are loading. In practice, the user is forced to look at a fuzzy
full-size proxy for the image while all the bits are loading. Is it
done? Well, it looks kind of fuzzy. Oh wait, the top of the image
seems to be getting a little more detail. Maybe it is done now. It is
still kind of fuzzy, though. Maybe the photographer wasn't using a
tripod. Oh wait, it seems to be clearing up now ...
Interlaced images are slightly less efficient, but show up after shorter delay on the client side when transported over the network. IMHO they should be used when the expected download time for the image is long enough to be perceived by the user (say, above 1 second). The difference in file size is really quite small, so it's better to be too-cautious and use interlacing too much rather than too little.
In common broadband internet as of 2012, I'd just use it for every image > 100kb.
These points must be useful.
Interlacing (more generally, progressive display) is a method of displaying images on a monitor.
When to use it? Your decision should be base on these factors:
•> Non-interlaced images are smaller than interlaced images.
•> Interlaced images cause less flickering than non-interlaced ones
•> Interlaced images are much more easily view-able.
The interlace lets you see the picture before all the data has been transmitted (makes them appear faster and better-looking) and gives you the "feeling" that it is being downloaded faster.
TIP: Interlacing is not recommended for small images but is a must if
the viewer uses a slow connection
This is just a copy from Y answers i thought could help to understand.
Original answer could be find at: https://answers.yahoo.com/question/index?qid=20090211121956AAz7Xz8
Just to throw my twopenneth into the argument: Interlacing was introduced years ago when internet speeds were slow, the idea being that the image would present itself in a gradually more defined manner, still giving an overall look and feel to an image without having to wait for the entire thing to load.
Interlacing, today, is basically unnecessary and should be used based on the overall size of the image being transferred.
Progressive scans on JPEG images images do provide a more refined image while attempting to reduce the overall file size (i.e. is an actual compression mode rather than a streaming method for the bits making up the image).
PNGs use a more complex algorithm than GIF.
There is an interesting related post on webmasters
https://webmasters.stackexchange.com/questions/574/progressive-jpeg-why-do-many-web-sites-avoid-rendering-jpegs-that-way-pros
Untimately it depends on how they are going to be used.
The post suggests that there is limited - genuine - support for progressive images. And sometimes they may cause issues with plugins which don't support the progressive format.
Hope that helps.
I'm new to Video technology, so any feedback (such as if I've underspecified the problem) would be greatly appreciated.
I need to display an animation (currently composed of about a thousand PNGs) on Windows and am trying to determine the best video codec or parameters for the job.
Video playback must be smooth at 30 fps
Output display is 1920x1080 on a secondary monitor
Quality does not matter (within limits)
Will be displaying alpha blended animation on top, so no DXVA
Must run on older hardware (Core Duo 4400 + nVidia 9800)
Currently using DirectShow to display the video.
Question:
Is it easier on the CPU to shrink the source to 1/2 size (or even 1/4) and have the CPU stretch it at run time?
Is there a video codec that is easier on the CPU than others?
Are there parameters for video codecs that mean less decompression is required? (The video will be stored on the HD, so size doesn't matter except as it impacts program performance).
So far:
- H.264 from ffmpeg defaults produces terrible tearing and some stuttering.
- Uncompressed video from VirtualDub produces massive stuttering.
There are so many different degrees of freedom to this problem, I'm flailing. Any suggestions by readers would be much appreciated. Thank you.
MJPEG should work. I used it for 1080i60 some 3 years back, and the playback was never an issue. Even encoding worked on-the-fly with a machine of quite similar performance to what you describe.
File size will be about 10MB/s for good quality video.
Shrinking the video will help, because if you are drawing the video to screen using e.g. DirectX, you can use the GPU to stretch it.