Trying to play a 3840x2160 video recorded by an iPhone 7 (#30fps), I get frequent pauses -- in the video, music keeps playing.
This happens both in firefox and when ffplay is invoked to play the file directly -- from command-line. The CPU is a dual E6700 #3.20GHz -- not super fast, but it should be able to play smoothly, shouldn't it? Video is Intel's "series 4" integrated chipset -- again, not a speed-daemon, but it should be adequate... Support for Intel's VA API is included.
I build ffmpeg-4.1 from source using FreeBSD port. As you can see, the port has a maddening amount of options -- including several different ones for the 264-codec.
Any suggestions for improving the decoding speed to the point, where it is watchable by a human? Thank you!
Related
In Ant Media Server after recording stream on Windows using API, the VOD plays fine on Windows. But when playing the same VOD on macOS using Quick Time Player v10.5, the video freezes after some seconds and audio continues.
VODs playback with Quick Time Player is fine for recordings made on macOS.
How can I overcome this and is it an expected behaviour!
TL;DR;
Transcode the video with ffmpeg after recording or add at least one adaptive bitrate on the Ant Media Server side.
This is a known issue in quick time player. This problem also exists for MacOS/iOS and Safari. Let me tell the cause of the problem and offer a solution.
Problem:
The resolution may be changed in WebRTC sessions according to the network conditions so that the resolution of the recording is being changed to lower or higher resolution.
Most of the players and browsers can handle that. On the other hand, Safari and Quick Time Player cannot handle resolution changes and the problem you mention appear.
Solution:
Transcoding the stream into a specific resolution with ffmpeg or using adaptive bitrate on the server side resolves this issue. Typical ffmpeg command is sufficient
ffmpeg -i INPUT.mp4 OUTPUT.mp4
A. Oguz antmedia.io
I developed an application in C that can display 4 videos, and 1 sound file in the background.
The video uses the WMP object in the C++ class provided by Microsoft in the WMP SDK. The audio uses Windows' MCI (Media Control Interface) which is sent command strings.
To be able to play lots of different formats, I installed windows.7.codec.pack.
I experience a problem that when playing more than 3 media files (video or audio), the media stall. A video to be started constantly cycles from state 3 (Playing) to state 9 (Preparing new media) to state 10 (Ready to begin playing - without anything happening), and so on and so on. This is seen as a flicker of the video (state 3) followed by a few seconds of nothing (black, or desktop background, state 9). Once a video has started, it plays fine until the end.
Decreasing the number of media files to play to 2 videos and 1 audio lets it play fine; increasing to 3 video or more and 1 audio and this happens. Task Manager shows a CPU load of less than 25%, so CPU cannot be the problem.
It seems that MCI and WMP share stuff in the background because not only do videos stall, also audio stops without reporting errors (querying MCI returns that it is playing, but there is no sound).
I upgraded to windows.7.codec.pack.v4.2.6. This had a terrible performance
I reverted to windows.7.codec.pack.v4.1.6. This has a much better performance but still not perfect.
My question(s):
Is there any way I can configure Windows or the codec package to seamlessly handle 5 streams?
How can I report this problem to the developer?
Are there other codec packages that do not have this problem?
Any other suggestion?
System info: Intel i7-3520M X64 dual core at 2.9Ghz with 8GB physical memory and NVIDIA Quatro K1000M display adapter.
I think I found the solution.
After playing a video, I called the Player's Close() method. The documentation says:
The close method releases Windows Media Player resources.
Remarks
This method closes the current digital media file, not the Player itself.
Indeed I wanted to release the media file so it would no longer be busy in the file system. However, it seems that more resources were released than just the media file. As a consequence, for a next media file to be played, the player had to allocate resources again. It seems that turned out to be a bottle neck.
No longer calling Close() but just giving it the URL (filename) of the next media file to play now solved the problem. (I still have to give some retries sometimes but the general performance is now very well acceptable.)
The media file is released when the next media file starts playing.
I wish to simultaneously play sounds through up to 12 mono speakers.
I could connect these to my MacBook using 6 USB soundcards, and use the left and right channel of each.
But how can I get the MacBook to play sound out of speaker #5, for example?
PS If anyone can see a smarter way to wire up 12 speakers to a MacBook, please do say!
You can setup an Aggregate Device (Audio Midi Setup > Create Aggregate Device), which allows the ability to combine multiple devices of the same model, or to combine multiple inputs and outputs for apps that don't support separate input and output devices. This Apple guide shows how it works and is surprisingly quite easy to setup.
Another way to route audio to multiple channels and outputs (up to 64) is with the free app/plug-in Soundflower. You can download a compiled version, or compile the source code if you want/need to specifically do something that the current compiled version might not.
I want to create a virtual audio device that gets audio data from the default output (which is an output IOAudioStream) and converts it to an input IOAudioStream.
I went through most of the examples I could find, however they only implement a feature to copy the output IOAudioStream to the input one at most. That means it only converts the audio to an input stream if the audio device is selected as output.
This should be possible, since ScreenFlow allows recording of computer audio by installing a kext that creates a virtual driver.
How can I access the audio data from the default output and send it to my virtual driver?
Take a look at the open-source WavTap, which is a simplified fork of the open-source SoundFlower virtual sound card driver. It is a .kext that I believe does substantially what you want.
For reference, here is how some popular commercial closed-source options work:
Rogue Amoeba's Audio Hijack Pro
-Captures system audio via code based off of the open-source SoundFlower .kext
-Captures an application's audio by substituting a "patch" framework for the normal CoreAudio.framework when launching the application
-Captures an already-running application's audio with the help of the haxie "Application Enhancer" (APE) from Unsanity
These features are branded as their "Instant On" feature (InstantOn.kext).
Ambrosia Software's WireTap Studio
-Captures system audio and application audio via an in-house developed .kext
Telestream's ScreenFlow
-Captures system audio via an in-house developed .kext. (Version 2.x uses varaudio.kext; Version 3.x uses TelestreamAudio.kext)
Macsome's Audio Recorder
-Unknown method
Araelium Group's Screenflick
-Captures system audio using the SoundFlower .kext
UPDATE #1
After reading the author's comments, it appears the underlying goal is to be able to capture the system sound without publishing the virtual audio driver as a device (that would appear in the System Preference's list) and without changing the current default output device (or at least the appearance that the device has changed).
SoundFlower: Adds a sound device to the list upon installation
WavTap: Adds a sound device to the list upon installation; auto-selects the device when the WavTap application is started; auto-deselects the device when the application is shutdown and reselects the previous device
Audio Hijack Pro: Adds a sound device only when audio capture of the default system sound is selected; removes the sound device when audio capture is no longer selected and reselects the previous device
WireTap Studio: Unknown
ScreenFlow: Captures the system sound without changing the current default output device and without publishing the virtual audio driver as a device
UPDATE #2
A quote from Jeff Moore, a CoreAudio Apple engineer, in reference to applications such as WireTap and Audio Hijack Pro:
"There are no APIs on the system that will give you the output of any specific app or the whole mix going to the hardware...[Capturing System Sound] isn't supported by the System and those folks had to be clever. There's nothing stopping you from doing the same thing except how willing you are to get your hands dirty.
The fact is, Mac OS X's audio system was designed first and foremost for performance. This lead us to a design where it is not easy to support the functionality you want without imposing performance penalties. So, we have opted for better performance at the cost of not being able to provide this feature."
If you want to read more on the subject, check out these threads on the CoreAudio API mailing list:
"WireTap, CoreAudio's API, and system capture, and kexts..."
"Another question on capturing audio played back by a software"
"Capturing currently played audio using CoreAudio on Mac"
"'audio hijack'"
"monitoring system audio output like wire tap"
"Capturing audio output to a file"
"Mirroring Audio Output"
"Recording system audio"
Relevant SO Questions:
Hide Audio device using codeless kext
So long story short, you're not likely to find examples from Apple that accomplish this, and you're not likely to find open source code that accomplishes this either, unless someone is feeling very generous. It appears to be too valuable of information.
After additional research, here are some theoretical techniques I came up with that might allow you to accomplish your goal:
Similar to Prosoft Engineering's Hear product, you could create a HAL plugin (user-mode virtual driver) rather than a .kext (kernel-mode virtual driver). Apple has a sample HAL plugin called "SampleHardwarePlugIn" and PulseAudio has one as well. However, with his method I don't think you get access to a pre-mixed system sound stream. You would have to gather up all streams from the various applications (which must use CoreAudio to play sound) and mix them together for pseudo system sound capture.
Create a virtual audio device that is hidden [1][2] from user interaction. When the user wishes to capture the default sound, programmatically create an aggregate device that includes your hidden virtual device and the current default sound device. Temporarily set this aggregate device as the default output. In this manner, you are able to both capture the default sound and hear it.
Side Note: If Mac OS X allows for a hidden device to also be set as the default output device, what would System Preferences show as the selected device? If it instead shows the secondary output device as selected, then you have the added allusion that nothing has changed.
A newer open-source virtual audio device that works with the latest versions of MacOS is BlackHole - it supports multiple audio channels and sampling rates.
It can be used as an audio sink and/or source. It's also handy as part of an aggregate audio device so audio can be heard and re-routed - e.g. using the MacOS Audio MIDI Setup app
I'm using ManagedMEdiaHelpers project as a base to a background audio streaming project.
The audio output is fine but sometimes is takes 1 to 6 seconds to start playing. During that time the device sends some strange noises similar to hiccups or scratching.
The mp3 I'm trying to stream have the following properies:
Bitrate: 320000
Sampling Rate: 44100
What are the possible causes to receive that kind of noises on the start of the stream when the rest of the mp3 plays just fine?
More info
I noticed that during the hiccups the fps count was below 20 so I tried to profile the application.
I got the following message during the hiccups period:
Waring :Very high CPU usage by system threads: System and other applications
are using 65,02% of the CPU. This CPU usage may be caused by other
tasks that are running on the system or they may be caused by system
operations that are triggered by a user application. Ensure that no
other tasks are running on the system during profiling.
It was an hardware limitation of HTC Radar.
I Just tried the same code on an Samsung OMNIA 7 and the stream is just perfect. Also there's no penalty on the fps count so I think that on this device there's no CPU hogging.
Strangely gsmarena says that both devices have the same CPU.