Everytime MediaPlayer.Play() is executed from the UI thread the UI freezes for a significant amount of time. I don't think you can do anything about the time it takes to start playing the SongCollection but at least the UI should stay responsive.
Running MediaPlayer.Play() from another thread obviously doesn't work.
The MediaPlayer is a component from the Xna Namespace. If you are using this feature in a game, you are most certain running a GameLoop to prevent this freeze from happening: GameLoop
If you use this component in an App, you can simulate this behavior your own
public MainPage()
{ InitializeComponent();
// Timer to simulate the XNA Game Studio game loop (Microphone is from XNA Game Studio)
DispatcherTimer dt = new DispatcherTimer();
dt.Interval = TimeSpan.FromMilliseconds(33);
dt.Tick += delegate { try { FrameworkDispatcher.Update(); } catch { } };
dt.Start();
}
(see complete sample on how to run a microphone outside a gameloop: msdn)
Related
This question is about running a non-blocking, high-performance activity in nativescript that is needed for the simple task of reading and saving raw audio from the microphone by directly accessing the hardware through the native Android API. I believe I have brought the nativescript framework to the edge of its capabilities, and I need experts' help.
I'm building a WAV audio recorder in Nativescript Android. Native implementation is described here (relevant code below).
In short, this can be done by reading audio steam from an android.media.AudioRecord buffer, and then writing the buffer to a file in a separate thread, as described:
Native Android implementation
startRecording() is triggered by a button press, and starts a new Thread that runs writeAudioDataToFile():
private void startRecording() {
// ... init Recorder
recorder.startRecording();
isRecording = true;
recordingThread = new Thread(new Runnable() {
#Override
public void run() {
writeAudioDataToFile();
}
}, "AudioRecorder Thread");
recordingThread.start();
}
Recording is stopped by setting isRecording to false (stopRecording() is triggered by a button press):
private void stopRecording() {
isRecording = false;
recorder.stop();
recorder.release();
recordingThread = null;
}
Reading and saving buffer is stopped if isRecording = false:
private void writeAudioDataToFile() {
// ... init file and buffer
ByteArrayOutputStream recData = new ByteArrayOutputStream();
DataOutputStream dos = new DataOutputStream(recData);
int read = 0;
while(isRecording) {
read = recorder.read(data, 0, bufferSize);
for(int i = 0; i < bufferReadResult; i++) {
dos.writeShort(buffer[i]);
}
}
}
My Nativescript javascript implementation:
I wrote a nativescript typescript code that does the same as the native Android code above. The problem #1 I faced was that I can't run while(isRecording) because the javascript thread would be busy running inside the while loop, and would never be able to catch button clicks to run stopRecording().
I tried to solve problem #1 by using setInterval for asynchronous execution, like this:
startRecording() is triggered by a button press, and sets a time interval of 10ms that executes writeAudioDataToFile():
startRecording() {
this.audioRecord.startRecording();
this.audioBufferSavingTimer = setInterval(() => this.writeAudioDataToFile(), 10);
}
writeAudioDataToFile() callbacks are queued up every 10ms:
writeAudioDataToFile() {
let bufferReadResult = this.audioRecord.read(
this.buffer,
0,
this.minBufferSize / 4
);
for (let i = 0; i < bufferReadResult; i++) {
dos.writeShort(buffer[i]);
}
}
Recording is stopped by clearing the time interval (stopRecording() is triggered by button press):
stopRecording() {
clearInterval(this.audioBufferSavingTimer);
this.audioRecord.stop();
this.audioRecord.release();
}
Problem #2: While this works well, in many cases it makes the UI freeze for 1-10 seconds (for example after clicking a button to stop recording).
I tried to change the time interval that executes writeAudioDataToFile() from 10ms to 0ms and up to 1000ms (while having a very big buffer), but then the UI freezes were longer and, and I experienced loss in the saved data (buffered data that was not saved to the file).
I tried to offload this operation to a separate Thread by using a nativescript worker thread as described here, where startRecording() and stopRecording() are called by messages sent to the thread like this:
global.onmessage = function(msg) {
if (msg.data === 'startRecording') {
startRecording();
} else if (msg.data === 'stopRecording') {
stopRecording();
}
}
This solved the UI problem, but created problem #3: The recorder stop was not executed on time (i.e. recording stops 10 to 50 seconds after the 'stopRecording' msg.data is received by the worker thread). I tried to use different time intervals in the setInterval inside the worker thread (0ms to 1000ms) but that didn't solve the problem and even made stopRecording() be executed with greater delays.
Does anyone have an idea of how to perform such a non-blocking high-performance recording activity in nativescript/javascript?
Is there a better approach to solve problem #1 (javascript asynchronous execution) that I described above?
Thanks
I would keep the complete Java implementation in actual Java, you can do this by creating a java file in your plugin folder:
platforms/android/java, so maybe something like:
platforms/android/java/org/nativescript/AudioRecord.java
In there you can do everything threaded, so you won't be troubled by the UI being blocked. You can call the Java methods directly from NativeScript for starting and stopping the recording. When you build your project, the Java file will automatically be compiled and included.
You can generate typings from your Java class by grabbing classes.jar from the generated .aar file of your plugin ({plugin_name}.aar) and generate type declarations for it: https://docs.nativescript.org/core-concepts/android-runtime/metadata/generating-typescript-declarations
This way you have all the method/class/type information available in your editor.
I am having a difficult time finding documentation on background tasks support for Xamarin.Forms.
Does Xamarin.Forms provide support for periodic background tasks?
I need to implement this for both Windows Phone 10 and Android.
XF has no implementation for background tasks. You will need to implement these natively. Below are examples on how to do it for each type of project.
UWP
https://github.com/Microsoft/Windows-universal-samples/tree/master/Samples/BackgroundTask
Android
https://developer.xamarin.com/guides/android/application_fundamentals/backgrounding/part_2_android_services/
WinRT
https://visualstudiomagazine.com/articles/2013/05/01/background-tasks-in-windows-store-apps.aspx
iOS
Just for those that want iOS as well.
https://developer.xamarin.com/guides/ios/application_fundamentals/backgrounding/part_3_ios_backgrounding_techniques/
Xamarin.Forms
Going into more detail for each section is https://xamarinhelp.com/xamarin-background-tasks/
Yeas, but it depends what you need to do.
You can for example use System.Threading.Timer (.net class) is Activity/Service
private System.Threading.Timer timer;
In Activity OnCreate
TimeSpan timerTime = new TimeSpan(0, 0, 0, 0, 1000);
timer = new System.Threading.Timer(new System.Threading.TimerCallback(OnTimerFired), null, timerTime, timerTime);
In Activity OnDestroy
if (timer != null)
{
timer.Change(System.Threading.Timeout.Infinite, System.Threading.Timeout.Infinite);
timer.Dispose();
timer = null;
}
private void OnTimerFired(object state)
{
Do some periodic job
}
I use Xamarin.Forms.Device.StartTimer Method, it starts a recurring timer using the device clock capabilities. While the callback returns true, the timer will keep recurring.
http://developer.xamarin.com/api/member/Xamarin.Forms.Device.StartTimer/
I am building a Windows 8.1 Universal app, and at the minute specifically I am working on the Windows Phone side of it. The app is an RT app built using the MVVM pattern, and I am using MVVM Light.
The app is for a financial institunion and exposes payment information. As such, there is a security requirement to log the user out of the app if there is no activity after a period of 2 minutes.
I've searched high and low for a solution to this problem. The only one which really works is to set up gestures in the view, and use a DispatchTimer, which expires and redirects to the app lock screen (not the built-in Windows one) after 2 minutes if no gestures have been detected. A gesture resets the timer. Here is some sample code for the gesture setup:
public MainPage()
{
this.InitializeComponent();
//Register the gestures, for the app timeout
this.PointerPressed += MainPage_PointerPressed;
this.PointerMoved += MainPage_PointerMoved;
this.PointerReleased += MainPage_PointerReleased;
gr.CrossSliding += Gr_CrossSliding;
gr.Dragging += Gr_Dragging;
gr.Holding += Gr_Holding;
gr.ManipulationCompleted += Gr_ManipulationCompleted;
gr.ManipulationInertiaStarting += Gr_ManipulationInertiaStarting;
gr.ManipulationStarted += Gr_ManipulationStarted;
gr.ManipulationUpdated += Gr_ManipulationUpdated;
gr.RightTapped += Gr_RightTapped;
gr.Tapped += Gr_Tapped;
gr.GestureSettings = Windows.UI.Input.GestureSettings.ManipulationRotate | Windows.UI.Input.GestureSettings.ManipulationTranslateX | Windows.UI.Input.GestureSettings.ManipulationTranslateY |
Windows.UI.Input.GestureSettings.ManipulationScale | Windows.UI.Input.GestureSettings.ManipulationRotateInertia | Windows.UI.Input.GestureSettings.ManipulationScaleInertia |
Windows.UI.Input.GestureSettings.ManipulationTranslateInertia | Windows.UI.Input.GestureSettings.Tap;
InitializeTimer();
}
There are methods for receiving gesture inputs on the page, like this:
private void Gr_Tapped(Windows.UI.Input.GestureRecognizer sender, Windows.UI.Input.TappedEventArgs args)
{
ResetTimer();
}
And here is my timer code:
private DispatcherTimer _timer2 { get; set; }
int ticks = 0;
private void InitializeTimer()
{
_timer2 = new DispatcherTimer();
_timer2.Tick += timer2_Tick;
_timer2.Interval = new TimeSpan(00, 0, 1);
bool enabled = _timer2.IsEnabled;
_timer2.Start();
}
void timer2_Tick(object sender, object e)
{
ticks++;
if (ticks >= 120)
{
Timeout();
}
}
private void Timeout()
{
_timer2.Stop();
_timer2 = null;
Frame.Navigate(typeof(LockView));
}
private void ResetTimer()
{
_timer2.Stop();
_timer2.Start();
ticks = 0;
}
This solution is far from perfect. I will have to use it if I can't find a better one, but there are 4 main problems with it;
It breaks the MVVM pattern. All this code has to go in the code behind of the View. I have 7 views which are accessible after the user logs in, so there will be a lot of repitition
It doesn't capture all gestures. For example is seems to ignore when scrolling up and down a listbox full of items
I have to call the ResetTimer() method or _timer2.Stop() from anywhere in the view that might be considered a gesture (even just before navigating away from the view)
It's just a messy and awkward solution
Does anyone know of a better way of achieving this? Surely there's a more elegant solution, maybe something at a higher level, rather than trying to catch every last button press and gesture.
Also, it would be useful if the app could detect whether it was running in the foreground or the background, and auto-navigates to the lock screen if the user navigates away from the app then goes back to it later.
Thanks in advance for any suggestions...
I have an XNA arcade game which runs over Silverlight environment. The game has a few sound clips which are played in random order as background music.
As stated in http://msdn.microsoft.com/en-us/library/microsoft.xna.framework.media.songcollection.aspx, a developer cannot have full control over Media Player. Particularly, developers cannot create their song collections or add songs to the playback queue. There is a recommendation to play songs one at a time by calling MediaPlayer.Play().
That’s exactly what I'm doing but I experience a performance flaw each time when another song starts playing. The game hangs on for a moment when I call MediaPlayer.Play() despite all sound clips are loaded during game initialization, not in runtime. This happens only on some devices (e.g. HTC Mozart). In contrast, if I disable game sounds and play the same clips in phone’s music player while running the game, there are no performance issues during song changes. I also don’t experience performance problems if we play the clips using SoundEffect class. However, I'm strongly willing to use MediaPlayer for background sound purposes due to 2 reasons:
- SoundEffect doesn’t issue notification when playback is completed
- SoundEffect doesn’t seem to work with .mp3 files, and using .wav files is very expensive
I've also run profiling tests which confirmed that the poor performance time frame starts in a few milliseconds after MediaPlayer.Play() and continues during about 0,4 seconds. During this time my game doesn't execute any heavy-weight operations, just regular game timer's Update() function.
Here is my code snippets:
public void PlayBackgroundMusic(Song song)
{
if ((!(App.Current as App).AppModel.SoundDisabled) && (song != null))
{
if (MediaPlayer.State != MediaState.Stopped)
{
StopBackgroundMusic();
}
MediaPlayer.Play(song);
}
}
public void StopBackgroundMusic()
{
MediaPlayer.Stop();
}
and the handler:
private void OnMediaStateChanged(object sender, EventArgs e)
{
if (MediaPlayer.State != MediaState.Playing)
{
if (!AppModel.SoundDisabled)
{
int index = soundRandomizer.Next(0, sounds.Length - 1);
PlayBackgroundMusic(sounds[index]);
}
}
}
Are there any suggestions?
After all, I found a solution which I'm satisfied with. It eliminates jiggling almost completely. Its idea is to use every MediaPlayer API in a separate thread obtained from thread pool. I'm not aware how it fixes the problem but this really works for me:
public void PlayBackgroundMusic(Song song)
{
if ((!(App.Current as App).AppModel.SoundDisabled) && (song != null))
{
if (MediaPlayer.State != MediaState.Stopped)
{
StopBackgroundMusic();
}
ThreadPool.QueueUserWorkItem((o) =>
{
MediaPlayer.Play(song);
}
}
}
public void StopBackgroundMusic()
{
ThreadPool.QueueUserWorkItem((o) =>
{
MediaPlayer.Stop();
}
}
Is there a easy way to perform a method after a given delay like in iOS out of the box?
On iPhone I would do this:
[self performSelector:#selector(connectSensor) withObject:nil afterDelay:2.5];
It will then schedule the method connectSensor on the main thread (UI thread) to be executed after 2,5 seconds. And because it is automatically scheduled on the main thread, you don't have to worry about cross thread issues. (There is also a performSelectorOnBackground version)
So how would I do this properly in WP7?
Currently I'm accomplishing this with a timer, but I'm not sure if this is a good solution.
private Timer timer;
private void DoSomethingAfterDaly()
{
// ... do something here
timer = new Timer( (o) => Deployment.Current.Dispatcher.BeginInvoke(() => NavigationService.GoBack()), null, 2500, Timeout.Infinite);
}
How could this be encapsulated into an extension method so I can just call this.Perform(MyMethod, null, 2500); ?
You can use a BackgroundWorker like so:
private void Perform(Action myMethod, int delayInMilliseconds)
{
BackgroundWorker worker = new BackgroundWorker();
worker.DoWork += (s, e) => Thread.Sleep(delayInMilliseconds);
worker.RunWorkerCompleted += (s, e) => myMethod.Invoke();
worker.RunWorkerAsync();
}
The call into this method would look like this:
this.Perform(() => MyMethod(), 2500);
The background worker will run the sleep on a thread off of the UI thread so your application is free to do other things while the delay is occurring.
You can use the Reactive Extensions for WP7 to observe on a timer:
Observable
.Timer(TimeSpan.FromMilliseconds(2500))
.SubscribeOnDispatcher()
.Subscribe(_ =>
{
NavigationService.GoBack();
});
Given the brevity of this code, I don't think you'd gain much by creating an extension method for it :) For more information about the Reactive Extensions for WP7, take a look at this MSDN page
.