Firefox is limited in its audio resampling ability for audio mediastreams. If the input media stream's sample rate is not the same as the AudioCotext's, then it complains :
DOMException: AudioContext.createMediaStreamSource: Connecting AudioNodes from AudioContexts with different sample-rate is currently not supported.
For example if we get an audio stream like so :
navigator.mediaDevices.getUserMedia(constraints).then(stream => {
let context = new (window.AudioContext || window.webkitAudioContext)({sampleRate : 48000});
let audioInput = this.context.createMediaStreamSource(stream);
});
Firefox will complain about mismatching sample rates - if they are different between the audio context and the hardware device's settings in the audio subsystem.
I can't find a way to get the sample rate from the audio track in the stream. I've tried :
let tracks = stream.getAudioTracks();
let settings = tracks[0].getSettings();
let constraints = tracks[0].getConstraints();
But none of these objects have the streams's sampleRate in them.
Is there another way to enquire an audio track's/stream's sample rate ?
Related
Since firefox 37 I cannot add volume control to the input(microphone), i get the error :
IndexSizeError: Index or size is negative or greater than the allowed amount
It works fine on Chrome.
Here is the code sample :
var audioContext = new (window.AudioContext || window.webkitAudioContext)(); // define audio context
var microphone = audioContext.createMediaStreamDestination();
var gain = audioContext.createGain();
var speaker = audioContext.createMediaStreamDestination(gain);
gain.gain.value = 1;
microphone.connect(gain);
gain.connect(speaker);
The error is thrown here :
microphone.connect(gain);
weirdly it works on firefox nightly.
This error is similar to this stackoverflow :link
Related link :
link on StackOverflow
Shouldn't you use this for microphone?
var microphone = audioContext.createMediaStreamSource();
instead of this
var microphone = audioContext.createMediaStreamDestination();
A microphone is not a destination. It is a source.
Firstly I think it should be
var microphone = audioContext.createMediaStreamSource(stream);
Here stream is the microphone audio stream. Find more info here.
Also check out this demo with elaboration here. It is similar to what you are trying. Replace createMediaElementSource with createMediaStreamSource will work.
I get a input frame stream through a socket, it is a mono 32-bit IEEE floating point PCM stream sampled at 16 kHz.
I get this with the following code : audio File sample
With Audacity i can visualize this and i see a regular cuts between my audio flux:
var audioCtx = new(window.AudioContext || window.webkitAudioContext)();
var audioBuffer = audioCtx.createBuffer(1, 256, 16000);
var BufferfloatArray;
var source = audioCtx.createBufferSource();
source.buffer = audioBuffer;
var gainNode = audioCtx.createGain();
gainNode.gain.value = 0.1;
gainNode.connect(audioCtx.destination);
source.connect(gainNode);
source.start(0);
socket.on('audioFrame', function(raw) {
var context = audioCtx;
BufferfloatArray = new Float32Array(raw);
var src = context.createBufferSource();
audioBuffer.getChannelData(0).set(BufferfloatArray);
src.buffer = audioBuffer;
src.connect(gainNode);
src.start(0);
});
I think it is because of the sample rate of my raw buffer (16000) is different of the sample rate of my Audio context (44100), what do you think ?
This is not a sample rate problem, because the AudioBufferSourceNode resamples the audio to the AudioContext's rate when playing.
What you should do here, is to have a little queue of buffers you feed using the network, and then, play your buffers normally like you do, but from the buffer queue, taking extra care to schedule them (using the first parameter of the start method of the AudioBufferSourceNode) at the right time, so that the end of the previous buffer is exactly the start of the next one. You can use the AudioBuffer.duration parameter to achieve this (duration is in seconds).
I'm finding it difficult to determine how to extract the following information from a QuickTime movie, either using QTKit or the older QuickTime APIs in OS X, targeting 10.5+:
Video and audio codecs used (e.g. "H.264")
Video and audio bitrates (e.g. 64 kbps)
Dimensions
The specific problems I've encountered are:
1) The only means to the video and audio codec names that I've found involve the use of ImageDescriptionHandle and SoundDescriptionHandle, both of which appear to require the Carbon-only methods NewHandleClear and DisposeHandle, as well as requiring the 32-bit only Media object. Is there a more modern method that doesn't require the Carbon framework and is 64-bit compatible?
2) For the bitrate, I'm getting the GetMediaDataSizeTime64 and dividing by the track duration in seconds. However, in the case of one audio track, that method returns a value of 128 kbps, but calling QTSoundDescriptionGetProperty with the audio track media and the kQTAudioPropertyID_FormatString param returns a string of "64 kbps". Why would those two values be different? Is there a better way to calculate a track's bitrate?
3) Dimensions returned by [QTMovie movieAttributes] objectForKey:QTMovieNaturalSizeAttribute] or by [QTTrack attributeForKey:QTTrackDimensionsAttribute] are incorrect for one particular movie. The size returned is 720 x 480, but the actual view size in QuickTime Player is 640 x 480. Player's info window shows a size string of "720 x 480 (640 x 480)". Is there a better way to determine the actual movie dimensions?
Thanks in advance!
This metadata can be obtained from the [movie tracks] QTTrack* objects.
1) Enumerating through the tracks you can find the video and audio tracks.
QTMedia* media = [track media];
if ([media hasCharacteristic:QTMediaCharacteristicVisual])
{
// video track
}
if ([media hasCharacteristic:QTMediaCharacteristicAudio])
{
// audio track
}
The information about codecs:
NSString* summary = [track attributeForKey:QTTrackFormatSummaryAttribute];
2) To calculate the movie's bitrate you need to calculate the total data size of all tracks and divide it on the movie duration.
Enumerating through the tracks get the data size of each track:
QTMedia* media = [track media];
Track quicktimeTrack = [track quickTimeTrack];
TimeValue startTime = 0;
TimeValue duration = GetTrackDuration(quicktimeTrack);
long trackDataSize = GetTrackDataSize(quicktimeTrack, startTime, duration);
3) To get the movie's dimensions
NSSize movieSize = [(NSValue*)[[movie movieAttributes] objectForKey:QTMovieNaturalSizeAttribute] sizeValue];
However, the actual dimensions of the video track may be different:
Fixed width = 0;
Fixed height = 0;
GetTrackDimensions(videoTrack, &width, &height);
This is supposed to be possible on Mac OS X by overwriting the sample rate in the AudioStreamBasicDescription then create a new output queue.
I've been able to retrieve the default sample rate and write a new one (ie. replace 44100 with 48000) but this is not resulting in any pitch change in the output signal.
err = AudioFileGetProperty(mAudioFile, kAudioFilePropertyDataFormat, &size, &mDataFormat);
if (err != noErr)
NSLog(#"Couldn't determine the audio file format");
Float64 mySampleRate = mDataFormat.mSampleRate; //the initial rate
if (inRate != 1) {
//write a new value
mDataFormat.mSampleRate = inRate;
//then
err = AudioQueueNewOutput etc.
Any suggestions would be greatly appreciated.
Changing the sample rate doesn't change the pitch of the audio. You may perceive that something playing back faster has a higher pitch. However that's perception rather than reality.
To change pitch, you'll need to process the audio data through a Digital Signal Processing (DSP) library. Alternatively, take a look at running it through an AudioUnit:
Audio Unit Programming Guide
I am using the following code snippets to record screen, and in most situations recorded wmv file is clear enough, but for some part of video it is not very clear (grey color for some parts). What I record is ppt with full screen mode. I am using Windows Media Encoder 9.
Here is my code snippet,
IWMEncSourceGroup SrcGrp;
IWMEncSourceGroupCollection SrcGrpColl;
SrcGrpColl = encoder.SourceGroupCollection;
SrcGrp = (IWMEncSourceGroup)SrcGrpColl.Add("SG_1");
IWMEncVideoSource2 SrcVid;
IWMEncSource SrcAud;
SrcVid = (IWMEncVideoSource2)SrcGrp.AddSource(WMENC_SOURCE_TYPE.WMENC_VIDEO);
SrcAud = SrcGrp.AddSource(WMENC_SOURCE_TYPE.WMENC_AUDIO);
SrcVid.SetInput("ScreenCap://ScreenCapture1", "", "");
SrcAud.SetInput("Device://Default_Audio_Device", "", "");
// Specify a file object in which to save encoded content.
IWMEncFile File = encoder.File;
string CurrentFileName = Guid.NewGuid().ToString();
File.LocalFileName = CurrentFileName;
CurrentFileName = File.LocalFileName;
// Choose a profile from the collection.
IWMEncProfileCollection ProColl = encoder.ProfileCollection;
IWMEncProfile Pro;
for (int i = 0; i < ProColl.Count; i++)
{
Pro = ProColl.Item(i);
if (Pro.Name == "Screen Video/Audio High (CBR)")
{
SrcGrp.set_Profile(Pro);
break;
}
}
encoder.Start();
thanks in advance,
George
I would guess that it's a problem with your encoder profile or settings, and not a problem with the code. If you're using the default "Screen Video/Audio High (CBR)" profile in WME9, it's using a video bitrate of 250Kbps, which is pretty low. I'd suggest creating a custom profile in the Windows Media Encoder Profile Editor Utility. Something like this:
awesomesc.prx
Name: Awesome Screen Profile
Audio: WMA 9.2 CBR (32kbps, 44kHz, mono CBR)
Video: WMV 9 Screen Quality VBR (Video size Same as video input, Frame rate 10fps, Key frame interval 3sec, Video quality 90)
Then just change the code to match the custom profile's name.
if (Pro.Name == "Awesome Screen Profile")
The encoder settings would take a much longer post to go through, but if you have not changed them from the defaults, you should be OK.
The Quality-based VBR algorithm can be pretty amazing, and will likely produce a surprisingly low average bitrate, but if VBR won't work for your needs, you can use the Windows Media Encoder Profile Editor utility to import the schia.prx profile that you're using and tweak the settings to find a higher CBR bitrate that produces acceptable quality.
"Screen Video/Audio Medium (CBR)"
it solved my problem