Poor performance on song playback start in WP7 - performance

I have an XNA arcade game which runs over Silverlight environment. The game has a few sound clips which are played in random order as background music.
As stated in http://msdn.microsoft.com/en-us/library/microsoft.xna.framework.media.songcollection.aspx, a developer cannot have full control over Media Player. Particularly, developers cannot create their song collections or add songs to the playback queue. There is a recommendation to play songs one at a time by calling MediaPlayer.Play().
That’s exactly what I'm doing but I experience a performance flaw each time when another song starts playing. The game hangs on for a moment when I call MediaPlayer.Play() despite all sound clips are loaded during game initialization, not in runtime. This happens only on some devices (e.g. HTC Mozart). In contrast, if I disable game sounds and play the same clips in phone’s music player while running the game, there are no performance issues during song changes. I also don’t experience performance problems if we play the clips using SoundEffect class. However, I'm strongly willing to use MediaPlayer for background sound purposes due to 2 reasons:
- SoundEffect doesn’t issue notification when playback is completed
- SoundEffect doesn’t seem to work with .mp3 files, and using .wav files is very expensive
I've also run profiling tests which confirmed that the poor performance time frame starts in a few milliseconds after MediaPlayer.Play() and continues during about 0,4 seconds. During this time my game doesn't execute any heavy-weight operations, just regular game timer's Update() function.
Here is my code snippets:
public void PlayBackgroundMusic(Song song)
{
if ((!(App.Current as App).AppModel.SoundDisabled) && (song != null))
{
if (MediaPlayer.State != MediaState.Stopped)
{
StopBackgroundMusic();
}
MediaPlayer.Play(song);
}
}
public void StopBackgroundMusic()
{
MediaPlayer.Stop();
}
and the handler:
private void OnMediaStateChanged(object sender, EventArgs e)
{
if (MediaPlayer.State != MediaState.Playing)
{
if (!AppModel.SoundDisabled)
{
int index = soundRandomizer.Next(0, sounds.Length - 1);
PlayBackgroundMusic(sounds[index]);
}
}
}
Are there any suggestions?

After all, I found a solution which I'm satisfied with. It eliminates jiggling almost completely. Its idea is to use every MediaPlayer API in a separate thread obtained from thread pool. I'm not aware how it fixes the problem but this really works for me:
public void PlayBackgroundMusic(Song song)
{
if ((!(App.Current as App).AppModel.SoundDisabled) && (song != null))
{
if (MediaPlayer.State != MediaState.Stopped)
{
StopBackgroundMusic();
}
ThreadPool.QueueUserWorkItem((o) =>
{
MediaPlayer.Play(song);
}
}
}
public void StopBackgroundMusic()
{
ThreadPool.QueueUserWorkItem((o) =>
{
MediaPlayer.Stop();
}
}

Related

Framerate independent event in Unity3D

I use the libpd4unity package to communicate with Pure Data. I receive a bang from Pure Data with LibPD.Bang. On a bang event I play sound by FMOD.
Problem is, that I receive bangs frequently, for example once every 500 ms but event doesn't trigger in specific length of frame. Usually length change 1 frame less or more.
Is there a solution for this problem? For example a framerate independent event? I want to know if event (delegate) in Unity3D is framerate independent or not.
Because there is tempo for playing each sound and just 1 frame ruins rhythm.
I need to sync sounds for playing by each separate bang.
Regarding your question on whether delegates are dependent or independent from Unity's framerate, there's no straight answer. It depends on how your delegates are called. Are they called from a thread? Are they executed in a thread?
Coroutines are not framerate independent, they are executed in Unity's loop.
The following script should shine a light on the difference between handling delegates in coroutines and in threads.
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using System.Threading;
public class DelegatesAndFramerate : MonoBehaviour {
delegate void MyDelegate();
MyDelegate myDelegate1; // done with coroutines
MyDelegate myDelegate2; // done with threads
Thread thread;
bool threadDone = false;
private int frameCount = 0;
private int delegate1CallCount = 0;
private int delegate2CallCount = 0;
private int callerLoopsCount_coroutine = 0;
private int callerLoopsCount_thread = 0;
void Start () {
myDelegate1 += Elab1;
myDelegate2 += Elab2;
StartCoroutine(CallerCoroutine());
thread = new Thread(new ThreadStart(CallerThread));
thread.Start();
}
void Update()
{
frameCount++;
}
void Elab1()
{
delegate1CallCount++;
}
void Elab2()
{
delegate2CallCount++;
}
IEnumerator CallerCoroutine()
{
while(true)
{
callerLoopsCount_coroutine++;
myDelegate1();
yield return null;
}
}
void CallerThread()
{
while(!threadDone)
{
callerLoopsCount_thread++;
myDelegate2();
}
}
void OnDestroy()
{
Debug.Log("Frame Count: " + frameCount);
Debug.Log("Delegate Call Count (Coroutine): " + delegate1CallCount);
Debug.Log("Delegate Call Count (Thread): " + delegate2CallCount);
Debug.Log("Caller Loops Count (Coroutine): " + callerLoopsCount_coroutine);
Debug.Log("Caller Loops Count (Thread): " + callerLoopsCount_thread);
threadDone = true;
thread.Join();
}
}
If you attach it to a GameObject and let Unity play for some seconds you'll see that the times the delegate was called from a coroutine is equal to the number of executed frames whilst the times the delegate was called from the thread will be way bigger.
I have experience in interfacing softwares similar to Pure Data and I think what you need is a (rather typical) thread with all your delegates there, create a queue of commands for Unity and digest it in Unity's Update.
Not knowing libPD in the specific this might not be the best practice for the case but it is a widely used approach. Basically the producer-consumer pattern.
Basing on the example GUITextScript.cs, libPD only requires you to subscribe to the right delegates. You don't have control on when these are executed, the library has; so if you keep having this issue it's worth submitting a bug report to the developers I guess.

How to Poll For Controller Input In UWP App

I'm unsure about the best practice for obtaining and updating input received from a controller monitored using the GamePad class in UWP.
I've seen a couple of examples of people using Dispatch Timers and async loops inside the GamePadAdded event. In Win32 applications, I would have handled input in the WinMain update/message loop, but in UWP apps I don't know of anything similar.
Is there a loop in UWP apps that input should be collected/handled like in Win32 apps? What is the recommended protocol for polling for input from a input device (nominally a Xbox One controller)?
I'm happy to read more about UWP app development but I'm unsure of any guides that reference something like this.
Edit: It would be productive if, instead of downvoting and moving on, you shared thoughts on why this question deserved a downvote.
I've seen a couple of examples of people using Dispatch Timers and async loops inside the GamePadAdded event
This is the right way in UWP app to read Gamepad data. A little suggestion is, move the loop reading part on UI thread if you need to update UI frequently. See the solution in this blog
Is there a loop in UWP apps that input should be collected/handled like in Win32 apps
You may make a wrapper with custom event, see the open source implementation: XBoxGamepad
public class XBoxGamepad
{
private List<Gamepad> _controllers = new List<Gamepad>();
private bool _running = true;
Task backgroundWorkTask;
public event EventHandler<GamepadButtons> OnXBoxGamepadButtonPressA;
//omitted......
public XBoxGamepad()
{
Gamepad.GamepadAdded += Gamepad_GamepadAdded;
Gamepad.GamepadRemoved += Gamepad_GamepadRemoved;
backgroundWorkTask = Task.Run(() => PollGamepad());
}
//omitted......
private void Start()
{
_running = true;
}
public void Stop()
{
_running = false;
}
public async Task PollGamepad()
{
while (true)
{
if (_running)
{
foreach (Gamepad controller in _controllers)
{
if (controller.GetCurrentReading().Buttons == GamepadButtons.A)
{
OnXBoxGamepadButtonPressA(controller, controller.GetCurrentReading().Buttons);
}
//omitted......
}
}
await Task.Delay(50);
}
}
private void Gamepad_GamepadRemoved(object sender, Gamepad e)
{
_controllers.Remove(e);
}
private void Gamepad_GamepadAdded(object sender, Gamepad e)
{
_controllers.Add(e);
}
}

Play a particle system when Time.timeScale = 0?

I have a particle system explosion that I want to play when the player dies. The idea that is that everything else pauses but the explosion continues, like the Pacman animation when you die (everything freezes and the pacman death animation plays).
Trouble is, it won't work. I know Time.timeScale = 0 pauses everything, and I've tried using this script to combat that, but it doesn't seem to be working:
void Update()
{
if (Time.timeScale < 0.01f)
{
particleSystem.Simulate(Time.deltaTime, true, false);
}
}
I have also tried this, but it doesn't work either:
private ParticleSystem pSystem;
public void Awake()
{
pSystem = gameObject.GetComponent<ParticleSystem>();
}
public void Play()
{
pSystem.Simulate(Time.unscaledDeltaTime,true,true);
}
public void Update()
{
pSystem.Simulate(Time.unscaledDeltaTime,true,false);
}
I have tried this code with a script that is attached to my explosion particle system prefab, which is instantiated at the player's position when you die.
Thanks!
UPDATE:
Turns out I had tried using particleSystem.Simulate(Time.unscaledDeltaTime, true, false);, but I was calling it in FixedUpdate() rather than Update().
However, the Asteroid Base post below posted by Garfty is really interesting and is probably worth doing in the long run!
One way you could do it is by using Time.unscaledDeltaTime
Another way you could approach something like this is by creating your own time manager, but it requires some discipline to stick to. The people at Asteroid Base wrote a nice article on something like this here.
I hope this helps!

Background Audio Playback Agent terminates during playback Windows Phone 7.5

So I have a problem with the Background Audio Playback agent in WP 7.5, where I think it is getting terminated by the OS randomly during playback.
I have an app which implements a BAP agent which plays a number of mp3 files based the selected chapter in UI. Each chapter has multiple verses, these verses have an associated mp3 file in isolated storage.
Once a chapter has been selected in the UI and the user presses the play button BackgroundAudio.Instance.Play() is called and the first verse (mp3 file) for that chapter is loaded as an AudioTrack. When the track has ended the next track is loaded in the OnPlayStateChanged event method under the TrackEnded state.
I also have some logic in the TrackEnded which checks if the end of the chapter has been reached (i.e. the last mp3 file for the current chapter has been played) and if so the first mp3 file for the next chapter will retrieved.
Now all the above works fine when using the Windows Phone 7 Emulator (both 512Mb and 256Mb Emulators), the mp3 files are played correctly and when the end of a chapter has been reached the next mp3 file for the next chapter is correctly loaded and played.
The problem I encounter is when I deploy this app to a Win 8 Device (Lumia 920), the audio starts playing fine and the suddenly and seemingly randomly the audio stops! No error messages, the app does NOT crash, just the Audio Stops playing. Also when I click on the UVC buttons on the device NO AudioTrack info is displayed as is the case during audio playback or audio has been paused (just the volume info shows).
I have no idea what’s going on, I think the OS may be terminating the Background Audio Playback agent but I have no idea why (I don’t think I am reaching any of the memory limitations but I can not confirm this as I don’t know how to check if I am).
Any advice/help will be appreciated.
Thanks
Update 14/01/14
To confirm that the memory limit of 15Mb(WP7) and 20Mb(WP8) was not being reached by my BAP I implemented some code which logged the current memory usage of the BAP at various stages through out its execution.
The memory usage does not reach anywhere near the limits imposed by the OS on the BAP, the peak i reach is 7Mb the issue I described above is still occurring, I can see from the log that the next track has already been set but the state Trackready is never hit, also no exceptions/errors are thrown. This has really stumped me!
Update 15/01/14
Below is an example of how I have implemented the BAP:
public AudioPlayer()
{
if (!_classInitialized)
{
_classInitialized = true;
// Subscribe to the managed exception handler
Deployment.Current.Dispatcher.BeginInvoke(delegate
{
Application.Current.UnhandledException += AudioPlayer_UnhandledException;
});
lastPlayedVerse = currentVerseNumber;
}
}
/// Code to execute on Unhandled Exceptions
private void AudioPlayer_UnhandledException(object sender, ApplicationUnhandledExceptionEventArgs e)
{
//Helper class to help log any exceptions
IsolatedStore.WriteToIS("unhandeled Ex: " + e.ExceptionObject.Message, IsolatedStore.MemLogFileName);
if (System.Diagnostics.Debugger.IsAttached)
{
// An unhandled exception has occurred; break into the debugger
System.Diagnostics.Debugger.Break();
}
}
protected override void OnError(BackgroundAudioPlayer player, AudioTrack track, Exception error, bool isFatal)
{
//Helper class to help log any exceptions
IsolatedStore.WriteToIS("OnError Called: " + error.Message, IsolatedStore.MemLogFileName);
if (isFatal)
{
Abort();
}
else
{
NotifyComplete();
}
}
protected override void OnPlayStateChanged(BackgroundAudioPlayer player, AudioTrack track, PlayState playState)
{
switch (playState)
{
case PlayState.TrackEnded:
track = null;
IsolatedStore.AppendToFileIS(string.Format("Track Ended::Time: {0}",DateTime.Now.ToLongTimeString()), IsolatedStore.MemLogFileName);
#region Track Ended logic
//IN here I have some logic to determine what the next track should be and then I call a function that returns an AudioTrack
player.Track = GetNextTrack(); //this method returns an AudioTrack
#endregion
break;
case PlayState.TrackReady:
IsolatedStore.AppendToFileIS(string.Format("Track Ready::Time: {0}, Track: {1}", DateTime.Now.ToLongTimeString(),track.Title), IsolatedStore.MemLogFileName);
//Put this try catch in here becoz i thought that this is where the issue was (not sure if its needed as any exception should be caught by the AudioPlayer_UnhandledException function.
try
{
player.Play();
}
catch (Exception ex)
{
IsolatedStore.AppendToFileIS(string.Format("Track Ready play exception: {0}", ex.Message), IsolatedStore.MemLogFileName);
}
break;
}
NotifyComplete();
}
protected override void OnUserAction(BackgroundAudioPlayer player, AudioTrack track, UserAction action, object param)
{
switch (action)
{
case UserAction.Play:
if (player.PlayerState != PlayState.Playing)
{
IsolatedStore.AppendToFileIS(string.Format("UA-PLAY::Time: {0}, Track: {1}", DateTime.Now.ToLongTimeString(),track.Title), IsolatedStore.MemLogFileName);
player.Play();
}
break;
}
NotifyComplete();
}
private AudioTrack GetNextTrack(int audioType2Get, string filePath, int verserNum, bool incrementTrackCount)
{
#region Memusage
//Code to log the memory usage
long currMemUsage = (long)DeviceExtendedProperties.GetValue("ApplicationCurrentMemoryUsage");
currMemUsage = (currMemUsage / 1024) / 1024;
long peakMemUsage = (long)DeviceExtendedProperties.GetValue("ApplicationPeakMemoryUsage");
peakMemUsage = (peakMemUsage / 1024) / 1024;
IsolatedStore.AppendToFileIS(string.Format("Getting Track-Time: {0}, Curr:{1}, Track: {2}", DateTime.Now.ToLongTimeString(), currMemUsage, verserNum), IsolatedStore.MemLogFileName);
#endregion
AudioTrack track = null;
#region AudioTrack Set region
//Some logic to return the AudioTrack
#endregion
}
Update 24/01/2014 ISSUE RESOLVED
I finally got some time to get around to attempting what #Soonts recommended in the answer I marked as the answer, and firstly I'm using a WP8 device so I skipped the first setp that he mentioned, next i did as was mentioned in step 2 and again the max memory usage was only 8Mb.
Then a few days back there was an update for my WP8 device (WP8 Update 3), after I installed this update I attempted to reproduce the problem and guess what! The issue DOES NOT OCCUR ANY MORE!, I had my audio on continuous play for over an hour with no issues! also the memory usage was stable at around 8Mb. So it looks like there may have been a silent update to the BG Audio.
The reason why I marked #snoots answer as the answer was because he mentioned in that answer that the issue may be fixed by a silent update.
This may happen on unhandled exceptions. Subscribe for Application.Current.UnhandledException (and if you're using async-await for TaskScheduler.UnobservedTaskException) and log them somewhere. Also, override OnError method of your agent, and log.
This may happen if forget to call BackgroundAgent.NotifyComplete() after you've finished processing the requests (i.e. for player agent, the OnPlayStateChanged and OnUserAction). In this case, the OS consludes you was unable to process the request in a timely manner, and terminates the BAP process.
RAM issues, but you've figured it out.
P.S. Here's the relevant part of my Sky.fm player application. It doesn't play local MP3s , instead it streams the music from the Internets, however the player agent code should be more or less the same.
/// <summary>This class wraps AudioPlayerAgent API into the async-friendly abstract class.</summary>
/// <remarks>Logging and exception handling are added, as well.</remarks>
public abstract class PlayerAgentAsync: AudioPlayerAgent
{
static PlayerAgentAsync()
{
UnhandledExceptionHandler.subscribe();
}
public PlayerAgentAsync()
{
Logger.info( "constructed" );
}
protected override void OnError( BackgroundAudioPlayer player, AudioTrack track, Exception ex, bool isFatal )
{
if( isFatal )
{
BackgroundErrorNotifier.addError( ex );
ex.log();
Abort();
}
else
{
ex.logWarning();
try
{
// Force the track to stop
// http://blogs.msdn.com/b/wpukcoe/archive/2012/02/11/background-audio-in-windows-phone-7-5-part-3.aspx
player.Track = null;
}
catch (System.Exception ex2)
{
ex2.logWarning( "Exception while trying to stop da playa" );
}
NotifyComplete();
}
}
/// <summary>Called when the play state changes, except for the error state.</summary>
protected override async void OnPlayStateChanged( BackgroundAudioPlayer player, AudioTrack track, PlayState playState )
{
Logger.info( "new playState = {0}", playState.ToString() );
try
{
await this.playStateChangedAsync( player, track, playState ).ConfigureAwait( false );
NotifyComplete();
}
catch (System.Exception ex)
{
this.onException( ex );
}
}
/// <summary>Called when the user requests an action using some application-provided UI or the Universal Volume Control (UVC) and the application has requested notification of the action.</summary>
protected override async void OnUserAction( BackgroundAudioPlayer player, AudioTrack track, UserAction action, object param )
{
Logger.info( "action = {0};", action.ToString() );
try
{
await this.userActionAsync( player, track, action, param ).ConfigureAwait( false );
NotifyComplete();
}
catch( System.Exception ex )
{
this.onException( ex );
}
}
private void onException( Exception ex )
{
if( ex.shouldBeIgnored() )
{
ex.logWarning();
this.NotifyComplete();
return;
}
ex.log();
BackgroundErrorNotifier.addError( ex );
this.Abort();
}
protected override void OnCancel()
{
Logger.trace();
base.OnCancel();
}
/// <summary>Handle OnPlayStateChanged asyncronously.</summary>
/// <param name="player">The Microsoft.Phone.BackgroundAudio.BackgroundAudioPlayer.</param>
/// <param name="track">The track playing at the time that the play state changed.</param>
/// <param name="playState">The new state of the player.</param>
protected abstract Task playStateChangedAsync( BackgroundAudioPlayer player, AudioTrack track, PlayState playState );
/// <summary>Handle OnUserAction asyncronously</summary>
/// <param name="player">The Microsoft.Phone.BackgroundAudio.BackgroundAudioPlayer.</param>
/// <param name="track">The track playing at the time of the user action.</param>
/// <param name="action">The action that the user has requested.</param>
/// <param name="param">The data associated with the requested action.</param>
protected abstract Task userActionAsync( BackgroundAudioPlayer player, AudioTrack track, UserAction action, object param );
}
If I were you, I would probably do the following, in that order:
Ensure you have the latest OS updates. They have silently fixed some BAP-related memory issues in their 7.8 update. If you're not using the latest OS, upgrade and try to reproduce.
Place breakpoints in every place of your GUI process code that interacts with BAP, to make sure you’re not mistakenly calling e.g. BackgroundAudioPlayer.Stop() or BackgroundAudioPlayer.Track=null, try to reproduce.
Instead of “logging current memory usage of the BAP at various stages”, in static constructor of some agent class, create a thread that, in the endless loop, appends to some CSV file in the isolated storage, logging current timestamp + current memory , then sleeps for 500-1000 milliseconds. Reproduce the problem, use ISETool.exe to download the log, use Excel to build the RAM usage graph…
If (3) won’t show RAM usage approaching the limit, Implement tracing to either isolated storage or network (BTW, I’ve implemented a SysLog RFC 5426 client in my WP7 code, and installed free software called “SysRose Syslog Desktop” on my PC), then try to do printf-style debugging.

User consent to stop the playing audio

Windows Phone 7.5 / Silverlight app
If user is playing music / radio on their phone and they try to launch my application, I want to give user an option to stop the currently playing option.
Working fine:
The message popup shows up fine. When I select Cancel, the popup closes, the music keeps playing and my app starts/works as normal.
Issue:
If I select Ok i.e. to stop the currently playing music on phone, the music stops but at the same time my app also exits.
Any ideas what I am doing wrong here?
Here is the code I am using. I call this method on launching:
private void CheckAudio()
{
if (FMRadio.Instance.PowerMode == RadioPowerMode.On)
{
MessageBoxResult Choice;
Choice = MessageBox.Show("For better user experience with this application it is recommended you stop other audio applications. Do you want to stop the radio?", "Radio is currently playing!", MessageBoxButton.OKCancel);
if (Choice == MessageBoxResult.OK)
{
FMRadio.Instance.PowerMode = RadioPowerMode.Off;
}
}
if (MediaPlayer.State == MediaState.Playing)
{
MessageBoxResult Choice;
Choice = MessageBox.Show("For better user experience with this application it is recommended you stop other audio/video applications. Do you want to stop the MediaPlayer?", "MediaPlayer is currently playing!", MessageBoxButton.OKCancel);
if (Choice == MessageBoxResult.OK)
{
MediaPlayer.Stop();
}
}
}
Update:
I posted my solution below. Do let me know if I am doing anything wrong.
I found the following error was being thrown:
FrameworkDispatcher.Update has not been called. Regular
FrameworkDispatcher. Update calls are necessary for fire and forget
sound effects and framework events to function correctly.
So I added this code and now it is working fine. Now upon clicking OK, the music player stops and my app launches fine. I call the SetupTimer method from InitializeComponent in App.xaml.cs
private GameTimer gameTimer;
private void SetupTimer()
{
gameTimer = new GameTimer();
gameTimer.UpdateInterval = TimeSpan.FromMilliseconds(33);
// Call FrameworkDispatcher.Update to update the XNA Framework internals.
gameTimer.Update += new EventHandler<GameTimerEventArgs>(gameTimer_Update); //delegate { try { FrameworkDispatcher.Update(); } catch { } };
// Start the GameTimer running.
gameTimer.Start();
// Prime the pump or we'll get an exception.
FrameworkDispatcher.Update();
}
void gameTimer_Update(object sender, GameTimerEventArgs e)
{
try { FrameworkDispatcher.Update(); }
catch { }
}
If anybody sees any problem/issue with the above please do let me know. Thanks.

Resources