I'm developing an app like Talking Tom
I tried to record audio, save to file then play with Mediaplayer in android but there is a bit delay to save the file then play. It is not smooth like Talking Tom
I saw that both MediaRecorder & MediaPlayer accept argument FileDescriptor in setOutputFile & setDataSource.
Is there any way to record then play without save to file ?
Do SoundPool & Mediaplayer can sound with byte array like byte [] buffer ?
Please help!
AudioRecord has the ability to record to a buffer, which can be played without saving to a file. MediaRecorder, which can also record videos, cannot.
http://developer.android.com/reference/android/media/AudioRecord.html
Two previous related questions answered on stackoverflow:
AudioRecord - how to get data in to buffer?
Android AudioRecord to File then use AudioTrack for Playback
I don't know if soundpool can be used with an array. The samples I seen and follow create multiple soundpools:
soundPool1 = new SoundPool(3, AudioManager.STREAM_MUSIC,0);
sound1 = soundPool1.load(getApplication(), R.raw.basskickdrum,1);
soundPoolA = new SoundPool(3, AudioManager.STREAM_MUSIC,0);
soundA = soundPoolA.load(getApplication(),R.raw.closedhighhat,1);
soundPool2 = new SoundPool(3, AudioManager.STREAM_MUSIC,0);
sound2 = soundPool2.load(getApplication(),R.raw.snare2,1);
Related
For Dual Playback Demo page
<video class="primary"></video>
<video class="secondary" muted autoplay loop src="//storage.googleapis.com/ytlr-cert.appspot.com/test-materials/media/manual/dual_video/secondary-video.mp4"
maximum-capabilities="width=432; height=240; framerate=15;"></video>
How to distinguish between primary video and secondary video while StarboardPlayer is being created?
In StarboardPlayer constructor
StarboardPlayer::StarboardPlayer(
... [parameters skipped]
const std::string& max_video_capabilities)
max_video_capabilities string parameter is always empty on both primary and secondary, the page mentioned above doesn't use HTMLVideoElement.setMaxVideoCapabilities() to set max_video_capabilities parameter and following attribute in secondary video also not handled by HTMLMediaElement
maximum-capabilities="width=432; height=240; framerate=15;"
For resource limited system, minor resource allocated for secondary video during playback construction. For example, our system has only one resource which can be used in "punch out" output mode for primary video, so "decode to texture" output mode used in secondary video. After playback constructed, it seems not possible to change output mode dynamically? And memory resource also limited for secondary video.
So it's very important to distinguish between primary video and secondary video on resource limited embedded system, but it seems not possible in starboard layer and I don't want to modify code outside of starboard.
Any suggestions?
I noticed following code was added to dual_video.js. Now starboard layer can use max_video_capabilities string to determine whether it is secondary video or not.
Thank you :)
var secondaryVideo = document.querySelector('video.secondary');
secondaryVideo.setMaxVideoCapabilities('width=432; height=240; framerate=15;');
secondaryVideo.loop = true;
secondaryVideo.autoplay = true;
secondaryVideo.muted = true;
secondaryVideo.src = 'https://storage.googleapis.com/ytlr-cert.appspot.com/test-materials/media/manual/dual_video/secondary-video.mp4';
I'm currently implementing offline streaming with FairPlay streaming. Therefor I'm downloading streams using an AVAssetDownloadTask.
I want to give the users feedback about the size of the download which starts to begin:
Are you sure you want to download this stream? It will take 2.4GB to download and you currently have 14GB of space left
I've checking properties like countOfBytesReceived and countOfBytesExpectedToReceive but these wont give back correct values.
let headRequest = NSMutableURLRequest(URL: asset.streamURL)
headRequest.HTTPMethod = "HEAD"
let sizeTask = NSURLSession.sharedSession().dataTaskWithRequest(headRequest) { (data, response, error) in
print("Expected size is \(response?.expectedContentLength)")
}.resume()
prints a size of 2464, where at the end the size is 3GB.
During the download I logged above properties:
func URLSession(session: NSURLSession, assetDownloadTask: AVAssetDownloadTask, didLoadTimeRange timeRange: CMTimeRange, totalTimeRangesLoaded loadedTimeRanges: [NSValue], timeRangeExpectedToLoad: CMTimeRange) {
print("Downloaded \( convertFileSizeToMegabyte(Float(assetDownloadTask.countOfBytesReceived)))/\(convertFileSizeToMegabyte(Float(assetDownloadTask.countOfBytesExpectedToReceive))) MB")
}
But these stay at zero:
Downloaded 0.0/0.0 MB
HLS streams are actually a collection of files known as manifests and transport streams. Manifests usually contain a text listing of sub-manifests (each one corresponding to a different bitrate), and these sub-manifests contains a list of transport streams that contain the actual movie data.
In your code, when you download the HLS URL, you're actually downloading just the master manifest, and that's typically a few thousand bytes. If you want to copy the entire stream, you'll need to parse all the manifests, replicate the folder structure of the original stream, and grab the transport segments too (these are usually in 10-second segments, so there can be hundreds of these). You may need to rewrite URLs if the manifests are specified with absolute URLs as well.
To compute the size of each stream, you could multiply the bitrate (listed in the master manifest) by the duration of the stream; that might be a good enough estimate for download purposes.
A better answer here, since you're using the AVAssetDownloadTask in the context of offline FairPlay, is to implement the AVAssetDownloadDelegate. One of the methods in that protocol gives you the progress you're looking for:
URLSession:assetDownloadTask:didLoadTimeRange:totalTimeRangesLoaded:timeRangeExpectedToLoad:
Here's WWDC 2016 Session 504 showing this delegate in action.
There are a lot of details related to offline playback with FairPlay, so it's a good idea to go through that video very carefully.
I haven't worked with this API personally, but I am at least somewhat familiar with HTTP Live Streaming. With that knowledge, I think I know why you're not able to get the info you're looking for.
The HLS protocol is designed for handling live streaming as well as streaming of fixed-length assets. It does this by dicing up the media into in what are typically about ten-second chunks, IIRC, and listing the URLs for those chunks in a playlist file at a specific URL.
If the playlist does not change, then you can download the playlist, calculate the number of files, get the length of the first file, and multiply that by the number of files, and you'll get a crude approximation, which you can replace with an exact value when you start retrieving the last chunk.
However there is no guarantee that the playlist will not change. With HLS, the playlist can potentially change every ten seconds, by removing the oldest segments (or not) and adding new segments at the end. In this way, HLS supports streaming of live broadcasts that have no end at all. In that context, the notion of the download having a size is nonsensical.
To make matters worse, 2464 is probably the size of the playlist file, not the size of the first asset in it, which is to say that it tells you nothing unless that subclass's didReceiveResponse: method works, in which case you might be able to obtain the length of each segment by reading the Content-Length header as it fetches it. And even if it does work normally, you probably still can't obtain the number of segments from this API (and there's also no guarantee that all the segments will be precisely the same length, though they should be pretty close).
I suspect that to obtain the information you want, even for a non-live asset, you would probably have to fetch the playlist, parse it yourself, and perform a series of HEAD requests for each of the media segment URLs listed in it.
Fortunately, the HLS specification is a publicly available standard, so if you want to go down that path, there are RFCs you can read to learn about the structure of the playlist file. And AFAIK, the playlist itself isn't encrypted with any DRM or anything, so it should be possible to do so even though the actual decryption portion of the API isn't public (AFAIK).
This is my C#/Xamarin code to compute the final download size. It is most likely imperfect, especially with the new codecs supported with iOS11, but you should get the idea.
private static async Task<long> GetFullVideoBitrate(string manifestUrl)
{
string bandwidthPattern = "#EXT-X-STREAM-INF:.*(BANDWIDTH=(?<bitrate>\\d+)).*";
string videoPattern = "^" + bandwidthPattern + "(RESOLUTION=(?<width>\\d+)x(?<height>\\d+)).*CODECS=\".*avc1.*\".*$";
string audioPattern = "^(?!.*RESOLUTION)" + bandwidthPattern + "CODECS=\".*mp4a.*\".*$";
HttpClient manifestClient = new HttpClient();
Regex videoInfoRegex = new Regex(videoPattern, RegexOptions.Multiline);
Regex audioInfoRegex = new Regex(audioPattern, RegexOptions.Multiline);
string manifestData = await manifestClient.GetStringAsync(manifestUrl);
MatchCollection videoMatches = videoInfoRegex.Matches(manifestData);
MatchCollection audioMatches = audioInfoRegex.Matches(manifestData);
List<long> videoBitrates = new List<long>();
List<long> audioBitrates = new List<long>();
foreach (Match match in videoMatches)
{
long bitrate;
if (long.TryParse(match.Groups["bitrate"]
.Value,
out bitrate))
{
videoBitrates.Add(bitrate);
}
}
foreach (Match match in audioMatches)
{
long bitrate;
if (long.TryParse(match.Groups["bitrate"]
.Value,
out bitrate))
{
audioBitrates.Add(bitrate);
}
}
if (videoBitrates.Any() && audioBitrates.Any())
{
IEnumerable<long> availableBitrate = videoBitrates.Where(b => b >= Settings.VideoQuality.ToBitRate());
long videoBitrateSelected = availableBitrate.Any() ? availableBitrate.First() : videoBitrates.Max();
long totalAudioBitrate = audioBitrates.Sum();
return videoBitrateSelected + totalAudioBitrate;
}
return 0;
}
I have a cross platform (Xamrin) app that does some classic Bluetooth communication and is working absolutely fine on iOS8. However, after re-building and running it on iOS9 I can't get the NSInputStream to ever have "HasBytesAvailable"= true. Note: I followed all the instructions from Xamarin's website.
I tried both assigning a delegate to the InputStream and waiting on the NSRunLoop but the stream never seems to have bytes available. The event only fires (on iOS9) when the Input stream is opened (on iOS8 it fires as expected).
Here is a snippet of the code that does the reading successfully on iOS8 (delegate method):
EAsession.InputStream.Delegate = new Foo();
EAsession.InputStream.Schedule(NSRunLoop.Current,NSRunLoop.NSDefaultRunLoopMode);
EAsession.InputStream.Open();
(NSRunLoop.Current).RunUntil(NSDate.FromTimeIntervalSinceNow(2));
Where Foo is a class that implements: NSObject, INSStreamDelegate
public class Foo :NSObject, INSStreamDelegate
{
[Export("stream:handleEvent:")]
public void HandleEvent(Foundation.NSStream theStream, Foundation.NSStreamEvent streamEvent)
{
//Code to read bytes here
}
To make sure there really are bytes sent to the iPhone5 I modified the external Bluetooth device to just echo any bytes received.
Using either method (delegate or waiting on NSRunLoop) on iOS8, the echo arrives immediately. However when I change the target device to the iOS9 I can wait forever and HasBytesAvailable would always be false.
I even tried reading regardless of HasBytesAvailable being false, but nothing is being read (no big surprise there I guess).
Moreover, I tried building using both Xcode6.4 and Xcode 7 with same result.
At the moment I am out of ideas so any help would be greatly appreciated!
Thanks in advance!
EDIT:
I contacted Xamarin and I am writing a test app for them to test whether it is an Apple issue or Xamarin issue.
Also, see the comment in this link about Bluetooth... perhaps related?
So I finally solved it,
I am not entirely sure what was the problem eventually as I did a few things:
Update to Xamarin beta stream ( 5.9.7 build 12).
Update to the latest iOS (9.0.2)
Use Xcode 7.0
And change where I called the delegate, open stream and schedule.
The change to the code was that instead of handling the opening, scheduling and delegate assignment in the receiving and sending methods separately, I did it at the same place. Namely, when creating the EASession I assigned all of the above like so:
s = new EASession(device, protocol);
output_stream = s.OutputStream;
input_stream = s.InputStream;
output_stream.Delegate=this;
output_stream.Schedule(NSRunLoop.Current, NSRunLoop.NSDefaultRunLoopMode);
output_stream.Open();
input_stream.Delegate = this;
input_stream.Schedule(NSRunLoop.Current, NSRunLoop.NSDefaultRunLoopMode);
input_stream.Open();
At the moment I am not unscheduling the input and output streams and I am not sure it this is the right thing to do. I'll try and keep this updated as I move from working to a nice looking code that works...
Unlike what Bluecode describes I definitely needed to use my input stream and I was always opening it; so I am not sure if it was the same problem. I am still unsure what was the solution to the problem as it can be either one or a combination of the above. I might experiment a bit more into this later on when I have more time to see which one was the single solution? If any.
Hope this helps any.
Cheers
I was able to solve this problem by opening an InputStream to the EAAccessory that I had no intention of using. My guess is that both an Input and Output streams are required as part of the new iOS9 Bluetooth contract. This is the relevant update to my working code:
accessory.Delegate = new EAAccessoryDelegateHandler ();
if (PrinterSession == null)
{
PrinterSession = new EASession (accessory, PrintProtocol);
}
var mysession = PrinterSession;
mysession.InputStream.Open();
mysession.OutputStream.Delegate = this;
mysession.OutputStream.Schedule (NSRunLoop.Current, NSRunLoop.NSDefaultRunLoopMode);
mysession.OutputStream.Open ();
Then when I close my OutputStream I close the unused InputStream immediately afterwards.
I'm using the C++ code given here. But the shared speech recognition used here runs its own commands such as move,minimize,delete. I need to create this without invoking the MS speech recognition program.
hr = cpEngine.CoCreateInstance(CLSID_SpSharedRecognizer);
this line above creates the shared instance.
I tried to use CLSID_SpInprocRecognizer instead but can not get it right. I'm new to this.
Is there a way to do this?
I met the same issue here, and spent lot of time trying to find an answer. Luckily, I've got the solution by following the steps:
Do use the in-process recognizer, if you want to get rid of the MS speech recognition program
hr = cpRecognizer.CoCreateInstance(CLSID_SpInprocRecognizer);
2.In-process recognizer doesn't have default input sources or recognition engines set up, and you need to set them to get the in-process recognizer to listen.
CComPtr<ISpObjectToken> cpObjectToken;
CComPtr<ISpAudio> cpAudio;
// Get the default audio input token.
hr = SpGetDefaultTokenFromCategoryId(SPCAT_AUDIOIN, &cpObjectToken);
// Set the audio input to our token.
hr = cpRecognizer->SetInput(cpObjectToken, TRUE);
// Set up the inproc recognizer audio input with an audio input object.
// Create the default audio input object.
hr = SpCreateDefaultObjectFromCategoryId(SPCAT_AUDIOIN, &cpAudio);
// Set the audio input to our object.
hr = cpRecognizer->SetInput(cpAudio, TRUE);
3.specifies the particular speech recognition engine to be used. If not specified, it will use the default one. If it's not called, it still use the default one(I commend out this line, still works fine).
hr = cpRecognizer->SetRecognizer(NULL);
That's it! It opens a default U.S. English recognition engine, and picks up my command pretty quick.
reference:
http://stackoverflow.com/questions/18448394/inproc-speech-recognition-engine-in-python
http://msdn.microsoft.com/en-us/library/ms718864%28v=vs.85%29.aspx
http://msdn.microsoft.com/en-us/library/ms718866%28v=vs.85%29.aspx
Is it possible to obtain raw microphone data on the Windows Phone 7 to measure the ambient sound? If so, how can I do this?
not sure if you are familiar with Dealing with micrphone in WP7 or not
but you can check this blog post for a full detailed tutorial :
you initialize a buffer with the size of the data :
_buffer = new byte[_microphone.GetSampleSizeInBytes(_microphone.BufferDuration)];
and whenever the Microphone data is ready in this event handler get the data from the microphone add it to the Buffer then you can do anything using it :
private void MicrophoneBufferReady(object sender, EventArgs e)
{
_microphone.GetData(_buffer);
//Do something with the data
}
you'll need to include and everything is stated in the tutorial in Details
using Microsoft.Xna.Framework.Audio;
All you need is here:
http://library.developer.nokia.com/Community/Wiki/How_to_access_and_manage_the_Microphone_raw_data_in_WP7