Get QuickTime metadata: codecs, bitrates, dimensions - cocoa

I'm finding it difficult to determine how to extract the following information from a QuickTime movie, either using QTKit or the older QuickTime APIs in OS X, targeting 10.5+:
Video and audio codecs used (e.g. "H.264")
Video and audio bitrates (e.g. 64 kbps)
Dimensions
The specific problems I've encountered are:
1) The only means to the video and audio codec names that I've found involve the use of ImageDescriptionHandle and SoundDescriptionHandle, both of which appear to require the Carbon-only methods NewHandleClear and DisposeHandle, as well as requiring the 32-bit only Media object. Is there a more modern method that doesn't require the Carbon framework and is 64-bit compatible?
2) For the bitrate, I'm getting the GetMediaDataSizeTime64 and dividing by the track duration in seconds. However, in the case of one audio track, that method returns a value of 128 kbps, but calling QTSoundDescriptionGetProperty with the audio track media and the kQTAudioPropertyID_FormatString param returns a string of "64 kbps". Why would those two values be different? Is there a better way to calculate a track's bitrate?
3) Dimensions returned by [QTMovie movieAttributes] objectForKey:QTMovieNaturalSizeAttribute] or by [QTTrack attributeForKey:QTTrackDimensionsAttribute] are incorrect for one particular movie. The size returned is 720 x 480, but the actual view size in QuickTime Player is 640 x 480. Player's info window shows a size string of "720 x 480 (640 x 480)". Is there a better way to determine the actual movie dimensions?
Thanks in advance!

This metadata can be obtained from the [movie tracks] QTTrack* objects.
1) Enumerating through the tracks you can find the video and audio tracks.
QTMedia* media = [track media];
if ([media hasCharacteristic:QTMediaCharacteristicVisual])
{
// video track
}
if ([media hasCharacteristic:QTMediaCharacteristicAudio])
{
// audio track
}
The information about codecs:
NSString* summary = [track attributeForKey:QTTrackFormatSummaryAttribute];
2) To calculate the movie's bitrate you need to calculate the total data size of all tracks and divide it on the movie duration.
Enumerating through the tracks get the data size of each track:
QTMedia* media = [track media];
Track quicktimeTrack = [track quickTimeTrack];
TimeValue startTime = 0;
TimeValue duration = GetTrackDuration(quicktimeTrack);
long trackDataSize = GetTrackDataSize(quicktimeTrack, startTime, duration);
3) To get the movie's dimensions
NSSize movieSize = [(NSValue*)[[movie movieAttributes] objectForKey:QTMovieNaturalSizeAttribute] sizeValue];
However, the actual dimensions of the video track may be different:
Fixed width = 0;
Fixed height = 0;
GetTrackDimensions(videoTrack, &width, &height);

Related

How to get the video's duration or number of frames in ffmpeg filter

I'm writing a filter in ffmpeg and I have to get the duration or number of frames in filter_frame function. But the parameters of "filter_frame" function only consist of AVFilterLink and AVFrame:
static int filter_frame(AVFilterLink* link, AVFrame* in) {
AVFilterContext* avctx = link->dst;
ExtractContext* privCtx = (ExtractContext*)avctx->priv;
AVFilterLink* outlink = avctx->outputs[0];
//here I want to get the duration of total number of frames:
}
I can't find attributes which is related to duration of frame numbers in either class. I have googles but all answers are about getting duration or frame numbers using ffmpeg.exe, which don't meet my requirements because I have to get them in self-written filter in FFMPEG. Thanks for your help!

Play a PCM stream sampled at 16 kHz

I get a input frame stream through a socket, it is a mono 32-bit IEEE floating point PCM stream sampled at 16 kHz.
I get this with the following code : audio File sample
With Audacity i can visualize this and i see a regular cuts between my audio flux:
var audioCtx = new(window.AudioContext || window.webkitAudioContext)();
var audioBuffer = audioCtx.createBuffer(1, 256, 16000);
var BufferfloatArray;
var source = audioCtx.createBufferSource();
source.buffer = audioBuffer;
var gainNode = audioCtx.createGain();
gainNode.gain.value = 0.1;
gainNode.connect(audioCtx.destination);
source.connect(gainNode);
source.start(0);
socket.on('audioFrame', function(raw) {
var context = audioCtx;
BufferfloatArray = new Float32Array(raw);
var src = context.createBufferSource();
audioBuffer.getChannelData(0).set(BufferfloatArray);
src.buffer = audioBuffer;
src.connect(gainNode);
src.start(0);
});
I think it is because of the sample rate of my raw buffer (16000) is different of the sample rate of my Audio context (44100), what do you think ?
This is not a sample rate problem, because the AudioBufferSourceNode resamples the audio to the AudioContext's rate when playing.
What you should do here, is to have a little queue of buffers you feed using the network, and then, play your buffers normally like you do, but from the buffer queue, taking extra care to schedule them (using the first parameter of the start method of the AudioBufferSourceNode) at the right time, so that the end of the previous buffer is exactly the start of the next one. You can use the AudioBuffer.duration parameter to achieve this (duration is in seconds).

In Cocoa, producing a tone at given frequency for given duration [duplicate]

I want to play Beep sound in my Mac Os X and specify duration and frequency. On Windows it can be done by using Beep function (Console.Beep in .Net). Is there anything equivalent in Mac? I am aware of NSBeep but it does not take any parameters.
On the Mac, the system alert sound is a sampled (prerecorded) sound that the user chooses. It often sounds nothing like a beep—it may be a honk, thunk, blare, or other sound that can't be as a simple constant waveform of fixed shape, frequency, and amplitude. It can even be a recording of the user's voice, or a clip from a TV show or movie or game or song.
It also does not need to be only a sound. One of the accessibility options is to flash the screen when an alert sound plays; this happens automatically when you play the alert sound (or a custom alert sound), but not when you play a sound through regular sound-playing APIs such as NSSound.
As such, there's no simple way to play a custom beep of a specified and constant shape, frequency, and amplitude. Any such beep would differ from the user's selected alert sound and may not be perceptible to the user at all.
To play the alert sound on the Mac, use NSBeep or the slightly more complicated AudioServicesPlayAlertSound. The latter allows you to use custom sounds, but even these must be prerecorded, or at least generated by your app in advance using more Core Audio code than is worth writing.
I recommend using NSBeep. It's one line of code to respect the user's choices.
PortAudio has cross platform C code for doing this here: https://subversion.assembla.com/svn/portaudio/portaudio/trunk/examples/paex_sine.c
That particular sample generates tones on the left and right speaker, but doesn't show how the frequencies are calculated. For that, you can use the formula in this code: Is there an library in Java for emitting a certain frequency constantly?
I needed a similar functionality for an app. I ended up writing a small, reusable class to handle this for me.
Source on GitHub
A reusable class for generating simple sine waveform audio tones with specified frequency and amplitude. Can play continuously or for a specified duration.
The interface is fairly straightforward and is shown below:
#interface TGSineWaveToneGenerator : NSObject
{
AudioComponentInstance toneUnit;
#public
double frequency;
double amplitude;
double sampleRate;
double theta;
}
- (id)initWithFrequency:(double)hertz amplitude:(double)volume;
- (void)playForDuration:(float)time;
- (void)play;
- (void)stop;
#end
Here's a way of doing this with the newer AVAudioEngine/AVAudioNode APIs, and Swift:
import AVFoundation
import Accelerate
// Specify the audio format we're going to use
let sampleRateHz = 44100
let numChannels = 1
let pcmFormat = AVAudioFormat(standardFormatWithSampleRate: Double(sampleRateHz), channels: UInt32(numChannels))
let noteFrequencyHz = 440
let noteDuration: NSTimeInterval = 1
// Create a buffer for the audio data
let numSamples = UInt32(noteDuration * Double(sampleRateHz))
let buffer = AVAudioPCMBuffer(PCMFormat: pcmFormat, frameCapacity: numSamples)
buffer.frameLength = numSamples // the buffer will be completely full
// The "standard format" is deinterleaved float, so we can assume the stride is 1.
assert(buffer.stride == 1)
for channelBuffer in UnsafeBufferPointer(start: buffer.floatChannelData, count: numChannels) {
// Generate a sine wave with the specified frequency and duration
var length = Int32(numSamples)
var dc: Float = 0
var multiplier: Float = 2*Float(M_PI)*Float(noteFrequencyHz)/Float(sampleRateHz)
vDSP_vramp(&dc, &multiplier, channelBuffer, buffer.stride, UInt(numSamples))
vvsinf(channelBuffer, channelBuffer, &length)
}
// Hook up a player and play the buffer, then exit
let engine = AVAudioEngine()
let player = AVAudioPlayerNode()
engine.attachNode(player)
engine.connect(player, to: engine.mainMixerNode, format: pcmFormat)
try! engine.start()
player.scheduleBuffer(buffer, completionHandler: { exit(1) })
player.play()
NSRunLoop.mainRunLoop().run() // Keep running in a playground

Beep with custom frequency and duration

I want to play Beep sound in my Mac Os X and specify duration and frequency. On Windows it can be done by using Beep function (Console.Beep in .Net). Is there anything equivalent in Mac? I am aware of NSBeep but it does not take any parameters.
On the Mac, the system alert sound is a sampled (prerecorded) sound that the user chooses. It often sounds nothing like a beep—it may be a honk, thunk, blare, or other sound that can't be as a simple constant waveform of fixed shape, frequency, and amplitude. It can even be a recording of the user's voice, or a clip from a TV show or movie or game or song.
It also does not need to be only a sound. One of the accessibility options is to flash the screen when an alert sound plays; this happens automatically when you play the alert sound (or a custom alert sound), but not when you play a sound through regular sound-playing APIs such as NSSound.
As such, there's no simple way to play a custom beep of a specified and constant shape, frequency, and amplitude. Any such beep would differ from the user's selected alert sound and may not be perceptible to the user at all.
To play the alert sound on the Mac, use NSBeep or the slightly more complicated AudioServicesPlayAlertSound. The latter allows you to use custom sounds, but even these must be prerecorded, or at least generated by your app in advance using more Core Audio code than is worth writing.
I recommend using NSBeep. It's one line of code to respect the user's choices.
PortAudio has cross platform C code for doing this here: https://subversion.assembla.com/svn/portaudio/portaudio/trunk/examples/paex_sine.c
That particular sample generates tones on the left and right speaker, but doesn't show how the frequencies are calculated. For that, you can use the formula in this code: Is there an library in Java for emitting a certain frequency constantly?
I needed a similar functionality for an app. I ended up writing a small, reusable class to handle this for me.
Source on GitHub
A reusable class for generating simple sine waveform audio tones with specified frequency and amplitude. Can play continuously or for a specified duration.
The interface is fairly straightforward and is shown below:
#interface TGSineWaveToneGenerator : NSObject
{
AudioComponentInstance toneUnit;
#public
double frequency;
double amplitude;
double sampleRate;
double theta;
}
- (id)initWithFrequency:(double)hertz amplitude:(double)volume;
- (void)playForDuration:(float)time;
- (void)play;
- (void)stop;
#end
Here's a way of doing this with the newer AVAudioEngine/AVAudioNode APIs, and Swift:
import AVFoundation
import Accelerate
// Specify the audio format we're going to use
let sampleRateHz = 44100
let numChannels = 1
let pcmFormat = AVAudioFormat(standardFormatWithSampleRate: Double(sampleRateHz), channels: UInt32(numChannels))
let noteFrequencyHz = 440
let noteDuration: NSTimeInterval = 1
// Create a buffer for the audio data
let numSamples = UInt32(noteDuration * Double(sampleRateHz))
let buffer = AVAudioPCMBuffer(PCMFormat: pcmFormat, frameCapacity: numSamples)
buffer.frameLength = numSamples // the buffer will be completely full
// The "standard format" is deinterleaved float, so we can assume the stride is 1.
assert(buffer.stride == 1)
for channelBuffer in UnsafeBufferPointer(start: buffer.floatChannelData, count: numChannels) {
// Generate a sine wave with the specified frequency and duration
var length = Int32(numSamples)
var dc: Float = 0
var multiplier: Float = 2*Float(M_PI)*Float(noteFrequencyHz)/Float(sampleRateHz)
vDSP_vramp(&dc, &multiplier, channelBuffer, buffer.stride, UInt(numSamples))
vvsinf(channelBuffer, channelBuffer, &length)
}
// Hook up a player and play the buffer, then exit
let engine = AVAudioEngine()
let player = AVAudioPlayerNode()
engine.attachNode(player)
engine.connect(player, to: engine.mainMixerNode, format: pcmFormat)
try! engine.start()
player.scheduleBuffer(buffer, completionHandler: { exit(1) })
player.play()
NSRunLoop.mainRunLoop().run() // Keep running in a playground

how to improve video quality?

I am using the following code snippets to record screen, and in most situations recorded wmv file is clear enough, but for some part of video it is not very clear (grey color for some parts). What I record is ppt with full screen mode. I am using Windows Media Encoder 9.
Here is my code snippet,
IWMEncSourceGroup SrcGrp;
IWMEncSourceGroupCollection SrcGrpColl;
SrcGrpColl = encoder.SourceGroupCollection;
SrcGrp = (IWMEncSourceGroup)SrcGrpColl.Add("SG_1");
IWMEncVideoSource2 SrcVid;
IWMEncSource SrcAud;
SrcVid = (IWMEncVideoSource2)SrcGrp.AddSource(WMENC_SOURCE_TYPE.WMENC_VIDEO);
SrcAud = SrcGrp.AddSource(WMENC_SOURCE_TYPE.WMENC_AUDIO);
SrcVid.SetInput("ScreenCap://ScreenCapture1", "", "");
SrcAud.SetInput("Device://Default_Audio_Device", "", "");
// Specify a file object in which to save encoded content.
IWMEncFile File = encoder.File;
string CurrentFileName = Guid.NewGuid().ToString();
File.LocalFileName = CurrentFileName;
CurrentFileName = File.LocalFileName;
// Choose a profile from the collection.
IWMEncProfileCollection ProColl = encoder.ProfileCollection;
IWMEncProfile Pro;
for (int i = 0; i < ProColl.Count; i++)
{
Pro = ProColl.Item(i);
if (Pro.Name == "Screen Video/Audio High (CBR)")
{
SrcGrp.set_Profile(Pro);
break;
}
}
encoder.Start();
thanks in advance,
George
I would guess that it's a problem with your encoder profile or settings, and not a problem with the code. If you're using the default "Screen Video/Audio High (CBR)" profile in WME9, it's using a video bitrate of 250Kbps, which is pretty low. I'd suggest creating a custom profile in the Windows Media Encoder Profile Editor Utility. Something like this:
awesomesc.prx
Name: Awesome Screen Profile
Audio: WMA 9.2 CBR (32kbps, 44kHz, mono CBR)
Video: WMV 9 Screen Quality VBR (Video size Same as video input, Frame rate 10fps, Key frame interval 3sec, Video quality 90)
Then just change the code to match the custom profile's name.
if (Pro.Name == "Awesome Screen Profile")
The encoder settings would take a much longer post to go through, but if you have not changed them from the defaults, you should be OK.
The Quality-based VBR algorithm can be pretty amazing, and will likely produce a surprisingly low average bitrate, but if VBR won't work for your needs, you can use the Windows Media Encoder Profile Editor utility to import the schia.prx profile that you're using and tweak the settings to find a higher CBR bitrate that produces acceptable quality.
"Screen Video/Audio Medium (CBR)"
it solved my problem

Resources