I want to get a RTMP link and stream the video at my unity project but, and it works fine.... just 4 frames long cause after that it freezes, but audio stills playing.
For this code i´m using VSCode,Unity3D and the public example that the creators have on their github.
// This is the start, where i initialize things of course hahauhsuhs
void Start()
{
Core.Initialize(Application.dataPath);
_libVLC = new LibVLC();
PlayPause();
}
//Then here we have the method play and Pause, where we play Media Player and give him the media URL that works for a few frames.
private void PlayPause()
{
if (_mediaPlayer == null)
{
_mediaPlayer = new MediaPlayer(_libVLC);
}
if (_mediaPlayer.IsPlaying)
{
_mediaPlayer.Pause();
}
else
{
_isPlaying = true;
if (_mediaPlayer.Media == null)
{
// playing remote media
_mediaPlayer.Media = new Media(_libVLC, new Uri(URL));
}
_mediaPlayer.Play();
}
}
// This method will be execute every frame and do some crazy stuff that i can´t explane right now.
private void Update()
{
//A few checks before executing video
if (!_isPlaying) return;
if (URL.Equals(null)) URL = "http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4"; //if URL is null give the BigChungus Bunny video
//Execute Video
if (_tex == null)
{
// If received size is not null, it and scale the texture
uint i_videoHeight = 0;
uint i_videoWidth = 0;
_mediaPlayer.Size(0, ref i_videoWidth, ref i_videoHeight);
var texptr = _mediaPlayer.GetTexture(out bool updated);
if (i_videoWidth != 0 && i_videoHeight != 0 && updated && texptr != IntPtr.Zero)
{
Debug.Log("Creating texture with height " + i_videoHeight + " and width " + i_videoWidth);
_tex = Texture2D.CreateExternalTexture((int)i_videoWidth,
(int)i_videoHeight,
TextureFormat.RGBA32,
false,
true,
texptr);
RenderSettings.skybox.mainTexture = _tex;
}
}
else if (_tex != null)
{
var texptr = _mediaPlayer.GetTexture(out bool updated);
if (updated)
{
_tex.UpdateExternalTexture(texptr);
}
}
}
this is my first real question on stackover flow so this post can have some issues, I´m open for suggestion to improve it.
I found that the problem it´s with my stream server latency, and not with my unity project.
But still... there is a way to persist the video to play with high latency?
Related
I'm using the following Xamarin tutorial https://developer.xamarin.com/recipes/android/media/video/record_video/
I can successfully record video and audio however the quality is not very good. Can anyone suggest/explain how I can increase the quality please?
I know the device can record in higher quality because the native camera app record in much higher quality.
EDIT here is my code so far
protected override void OnCreate(Bundle savedInstanceState)
{
base.OnCreate(savedInstanceState);
// Set our view from the "main" layout resource
SetContentView(Resource.Layout.RecordVideo);
var record = FindViewById<Button>(Resource.Id.Record);
var stop = FindViewById<Button>(Resource.Id.Stop);
var play = FindViewById<Button>(Resource.Id.Play);
var video = FindViewById<VideoView>(Resource.Id.SampleVideoView);
var videoPlayback = FindViewById<VideoView>(Resource.Id.PlaybackVideoView);
string path = Android.OS.Environment.ExternalStorageDirectory.AbsolutePath + "/test.mp4";
if (Camera.NumberOfCameras < 2)
{
Toast.MakeText(this, "Front camera missing", ToastLength.Long).Show();
return;
}
video.Visibility = ViewStates.Visible;
videoPlayback.Visibility = ViewStates.Gone;
_camera = Camera.Open(1);
_camera.SetDisplayOrientation(90);
_camera.Unlock();
recorder = new MediaRecorder();
recorder.SetCamera(_camera);
recorder.SetAudioSource(AudioSource.Mic);
recorder.SetVideoSource(VideoSource.Camera);
recorder.SetOutputFormat(OutputFormat.Default);
recorder.SetAudioEncoder(AudioEncoder.Default);
recorder.SetVideoEncoder(VideoEncoder.Default);
//var cameraProfile = CamcorderProfile.Get(CamcorderQuality.HighSpeed1080p);
// recorder.SetProfile(cameraProfile);
recorder.SetOutputFile(path);
recorder.SetOrientationHint(270);
recorder.SetPreviewDisplay(video.Holder.Surface);
record.Click += delegate
{
recorder.Prepare();
recorder.Start();
};
stop.Click += delegate
{
if (recorder != null)
{
video.Visibility = ViewStates.Gone;
videoPlayback.Visibility = ViewStates.Visible;
recorder.Stop();
recorder.Release();
}
};
play.Click += delegate
{
video.Visibility = ViewStates.Gone;
videoPlayback.Visibility = ViewStates.Visible;
var uri = Android.Net.Uri.Parse(path);
videoPlayback.SetVideoURI(uri);
videoPlayback.Start();
};
}
I don't see the example specifying the CamcorderProfile anywhere so you might want to start from that. It's possible that the default framerate, bitrate and video frame size are lower than you'd expect. I'm not an a computer right now but try to set the profile to for example QUALITY_1080p using the SetProfile method in MediaRecorder.
You need to set the profile after setting the video and audio sources but before calling SetOutputFile method.
I am developing an application to play online radio via streaming. I have used MediaElement. But the problem is the player doesn't play in background. I mean as soon as I click on "start" or "back" button on the phone, the streaming as well as the audio stops. I have not tested it on any device, so please inform me if it does happen in simulator but not device. Here is my code..
private void Play()
{
if (mediaElement == null || mediaElement.CurrentState != MediaElementState.Playing)
{
if (SystemTray.ProgressIndicator == null)
SystemTray.ProgressIndicator = new ProgressIndicator();
SystemTray.ProgressIndicator.IsIndeterminate = true;
SystemTray.ProgressIndicator.IsVisible = true;
SystemTray.ProgressIndicator.Text = "Connecting to *********...";
mediaStream = new ********.RadioStream(uri);
mediaStream.StreamSetupComplete += (o, e) =>
{
Dispatcher.BeginInvoke(() =>
{
if (mediaElement != null)
{
LayoutRoot.Children.Remove(mediaElement);
}
mediaElement = new MediaElement();
mediaElement.Volume = 1.0;
LayoutRoot.Children.Add(mediaElement);
mediaElement.SetSource(mediaStream);
SystemTray.ProgressIndicator.IsVisible = false;
});
};
}
}
I want to know the steps to enable this to play in background. Atleast when the user presses "start" button, the audio streaming should not stop.
Also one more problem I have is I have added an ApplicationBarMenu in which I have an "Exit" button. As soon as the user clicks this button the streaming should stop and application should close itself. I am unable to close the application programmatically. Code is give below..
void exit_Click(object sender, EventArgs e)
{
if (playing)
{
MessageBoxResult Choice;
Choice = MessageBox.Show("Media is currently playing, do you want to stop it?", "Stop Player", MessageBoxButton.OKCancel);
if (Choice == MessageBoxResult.OK)
{
ImageBrush brush = new ImageBrush();
brush.ImageSource = new BitmapImage(new Uri(#"Images/play.png", UriKind.Relative));
play.Background = brush;
Stop();
playing = false;
try
{
// if (NavigationService.CanGoBack)
// {
// while (NavigationService.RemoveBackEntry() != null)
// {
// NavigationService.RemoveBackEntry();
// }
// }
}
catch
{
}
}
else
{
}
}
}
Please help me with the proper code. Even if there is any other way to stream media in background other than MediaElement, please suggest that too..
Hoping a reply soon. Thanks to all in advance.
You must use BackgroundAudioPlayer for this.
You should take a look at Microsoft.Phone.BackgroundAudio Namespace too.
I'm aware there are a lot of questions on this topic, and I've looked through most of them as well as Googling to help me solve this problem, to no avail.
What I'd like to do is post the relevant sections of my code that produce bitmaps and render bitmaps onto PictureBoxes in my UI, and I would like to know if anyone can spot what is specifically causing this error, and can suggest how to avoid or bypass it.
I'll start with the relevant bits (3) in my VideoRenderer class:
The timer event that continously calls MoveFrameToBitmap while video is running:
private void TimerTick(object sender, EventArgs e)
{
if (frameTransport.IsNewFrameAvailable())
{
if (frameTransport.GetFrame())
{
if (MoveFrameToBitmap())
{
double msSinceLastFrame = (Int32)DateTime.Now.Subtract(lastFrameTimestamp).TotalMilliseconds;
fps = 1000 / msSinceLastFrame;
lastFrameTimestamp = DateTime.Now;
}
}
else
{
if (frameTransport.channelKeyBufferBufidMismatch)
{
needsRestart = true;
}
}
}
}
MoveFrameToBitmap, which marshals in a video frame from FrameTransport, creates a bitmap if successful, clones it and queues the frame:
internal bool MoveFrameToBitmap()
{
bool result = false;
try
{
if (frameTransport.bitmapDataSize == 0)
{
return false;
}
bool ResolutionHasChanged = ((videoWidth != frameTransport.width) | (videoHeight != frameTransport.height));
videoHeight = frameTransport.height;
videoWidth = frameTransport.width;
Bitmap bitmap = new System.Drawing.Bitmap(videoWidth, videoHeight);
Rectangle rectangle = new System.Drawing.Rectangle(0, 0, videoWidth, videoHeight);
BitmapData bitmapData = new System.Drawing.Imaging.BitmapData();
bitmapData.Width = videoWidth;
bitmapData.Height = videoHeight;
bitmapData.PixelFormat = PixelFormat.Format24bppRgb;
bitmap.LockBits(rectangle, ImageLockMode.ReadWrite, PixelFormat.Format24bppRgb, bitmapData);
Marshal.Copy(frameTransport.bitmapData, 0, bitmapData.Scan0, frameTransport.bitmapDataSize);
lock (frameQueueLock)
{
if (frameQueue.Count == 0)
{
frameQueue.Enqueue(bitmap.Clone());
}
}
bitmap.UnlockBits(bitmapData);
if (ResolutionHasChanged) skypeRef.events.FireOnVideoResolutionChanged(this, new RootEvents.OnVideoResolutionChangedArgs(videoWidth, videoHeight));
bitmap.Dispose();
result = true;
}
catch (Exception) { }
GC.Collect();
return result;
}
The property that exposes the queued frame, which can be safely accessed even when a frame is not currently queued:
public Bitmap QueuedFrame
{
get
{
try
{
lock (frameQueueLock)
{
return frameQueue.Dequeue() as Bitmap;
}
}
catch (Exception)
{
return null;
}
}
}
That's all for VideoRenderer. Now I'll show the relevant property of the static MyVideo class, which encapsulates, controls and returns frames from two video renderers. Here is the property which exposes the queued frame of the first renderer (every call to videoPreviewRenderer is locked with VPR_Lock):
public static Bitmap QueuedVideoPreviewFrame
{
get
{
lock (VPR_Lock) { return videoPreviewRenderer.QueuedFrame; }
}
}
The property for the second renderer is the same, except with its own lock object.
Moving on, here is the thread and its call in my UI that accesses the two queued frame properties in MyVideo and renders frames to the two PictureBoxes:
ThreadStart renderVCONF3501VideoThreadStart = new ThreadStart(new Action(() =>
{
while (MyAccount.IsLoggedIn)
{
if (MyVideo.VideoPreviewIsRendering)
{
if (MyVideo.VideoPreviewRenderer.NeedsRestart)
{
MyVideo.VideoPreviewRenderer.Stop();
MyVideo.VideoPreviewRenderer.Start();
MyVideo.VideoPreviewRenderer.NeedsRestart = false;
}
else
{
try
{
Bitmap newVideoPreviewFrame = MyVideo.QueuedVideoPreviewFrame;
if (newVideoPreviewFrame != null)
{
lock (VCONF3501_VPI_Lock)
{
VCONF3501_VideoPreview.Image = newVideoPreviewFrame;
}
}
}
catch (Exception) { continue; }
}
}
else
{
lock (VCONF3501_VPI_Lock)
{
VCONF3501_VideoPreview.Image = null;
}
}
if (MyVideo.LiveSessionParticipantVideoIsRendering)
{
if (MyVideo.LiveSessionParticipantVideoRenderer.NeedsRestart)
{
MyVideo.LiveSessionParticipantVideoRenderer.Stop();
MyVideo.LiveSessionParticipantVideoRenderer.Start();
MyVideo.LiveSessionParticipantVideoRenderer.NeedsRestart = false;
}
else
{
try
{
Bitmap newLiveSessionParticipantVideoFrame = MyVideo.QueuedLiveSessionParticipantVideoFrame;
if (newLiveSessionParticipantVideoFrame != null)
{
lock (VCONF3501_LSPVI_Lock)
{
VCONF3501_Video.Image = newLiveSessionParticipantVideoFrame;
}
}
}
catch (Exception) { continue; }
}
}
else
{
lock (VCONF3501_LSPVI_Lock)
{
VCONF3501_Video.Image = null;
}
}
GC.Collect();
}
}));
new Thread(renderVCONF3501VideoThreadStart).Start();
The GC.Collect() calls are to force bitmap memory release, as there was a memory leak (and still might be one--the cloned bitmaps aren't being manually disposed of and I'm not sure where to, at this point).
Where is the InvalidOperationException in System.Drawing, which causes a red cross to be drawn to the PictureBox coming from, what am I doing wrong in terms of locking and access, and how can I avoid/bypass this error?
I am trying to bypass it with the catch exception and continue logic in the thread, and that I have confirmed works . . . sometimes. At other times, the failed draw attempt seems to complete too far and draws the red cross anyway, and after that point, the PictureBox is thoroughly unresponsive and new frames cannot be drawn to it, even when the video is still running fine.
Perhaps there is a way to refresh the PictureBox so that it accepts new frames?
I was having a problem with the red cross, then I find this and it help me, I hope it helps you too:
WinForms controls and the red X
Hello everyone I have been following this tutorial here http://www.gogo-robot.com/2011/05/30/xna-skinned-model-animations/ and so far its great got the animations playing and everything, but now I want to expand it and stop the continuous loops say for instance i press the a key to make the model jump when i release the a key i want him to stop jumping but if i hold the a key i want him to keep jumping. Here what i have tried so far
and none of it works.
I am stumped here on how to do this thanks for any help with this.
private void HandleInput(GameTime gameTime)
{
currentGamePadState = GamePad.GetState(PlayerIndex.One);
// Check for changing anims
//SkinningData skinningData = model.Tag as SkinningData;
SkinningData sd = jumper.model.Tag as SkinningData;
if (currentGamePadState.Buttons.A == ButtonState.Pressed)
{
if (jumper.animationPlayer.CurrentClip.Name != "Fire")
jumper.animationPlayer.StartClip(sd.AnimationClips["Fire"]);
}
if (currentGamePadState.Buttons.X == ButtonState.Pressed)
{
if (jumper.animationPlayer.CurrentClip.Name != "DieF")
jumper.animationPlayer.StartClip(sd.AnimationClips["DieF"]);
}
//does not work
if (currentGamePadState.Buttons.X == ButtonState.Released)
{
if (jumper.animationPlayer.CurrentClip.Name == "DieF")
jumper.animationPlayer.StartClip(sd.AnimationClips["Idel"]);
}
if (currentGamePadState.Buttons.Y == ButtonState.Pressed)
{
if (jumper.animationPlayer.CurrentClip.Name != "Idel")
jumper.animationPlayer.StartClip(sd.AnimationClips["Idle"]);
}
//does not work
if (jumper.animationPlayer.CurrentTime == jumper.animationPlayer.CurrentClip.Duration)
{
//set him back to idel
jumper.animationPlayer.StartClip(sd.AnimationClips["Idle"]);
}
I have tried these configuration with no luck in the game
// Starts playing the entirety of the given clip
public void StartClip(string clip, bool loop)
{
AnimationClip clipVal = skinningData.AnimationClips[clip];
StartClip(clip, TimeSpan.FromSeconds(0), clipVal.Duration, loop);
}
// Plays a specific portion of the given clip, from one frame
// index to another
public void StartClip(string clip, int startFrame, int endFrame, bool loop)
{
AnimationClip clipVal = skinningData.AnimationClips[clip];
StartClip(clip, clipVal.Keyframes[startFrame].Time,
clipVal.Keyframes[endFrame].Time, loop);
}
// Plays a specific portion of the given clip, from one time
// to another
public void StartClip(string clip, TimeSpan StartTime, TimeSpan EndTime, bool loop)
{
CurrentClip = skinningData.AnimationClips[clip];
currentTime = TimeSpan.FromSeconds(0);
currentKeyframe = 0;
Done = false;
this.startTime = StartTime;
this.endTime = EndTime;
this.loop = loop;
// Copy the bind pose to the bone transforms array to reset the animation
skinningData.BindPose.CopyTo(BoneTransforms, 0);
}
Can you not attach a bool on the animation clip to tell it to play only once, or an active variable that can be called.
i am trying to encode videos into 3gp format using xuggler, i somehow got it to work, work as in the program stopped throwing errors and exceptions, but the new file that is created does not have any video. Now there is no error or exception for me to work with so i have stuck a wall.
EDIT: Note the audio is working as it shud.
This is the code for the main function where the listeners are configured
IMediaReader reader = ToolFactory.makeReader("/home/hp/mms/b.flv");
IMediaWriter writer = ToolFactory.makeWriter("/home/hp/mms/xuggle/a_converted.3gp", reader);
IMediaDebugListener debugListener = ToolFactory.makeDebugListener();
writer.addListener(debugListener);
ConvertVideo convertor = new ConvertVideo(new File("/home/hp/mms/b.flv"), new File("/home/hp/mms/xuggle/a_converted.3gp"));
// convertor.addListener(writer);
reader.addListener(writer);
writer.addListener(convertor);
while (reader.readPacket() == null)
;
And this is the code for the convertor that i wrote.
public ConvertVideo(File inputFile, File outputFile)
{
this.outputFile = outputFile;
reader = ToolFactory.makeReader(inputFile.getAbsolutePath());
reader.addListener(this);
}
private IVideoResampler videoResampler = null;
private IAudioResampler audioResampler = null;
#Override
public void onAddStream(IAddStreamEvent event)
{
if (writer == null)
{
writer = ToolFactory.makeWriter(outputFile.getAbsolutePath(), reader);
}
int streamIndex = event.getStreamIndex();
IStreamCoder streamCoder = event.getSource().getContainer().getStream(streamIndex).getStreamCoder();
if (streamCoder.getCodecType() == ICodec.Type.CODEC_TYPE_AUDIO)
{
streamCoder.setFlag(IStreamCoder.Flags.FLAG_QSCALE, false);
writer.addAudioStream(streamIndex, 0, 1, 8000);
}
else if (streamCoder.getCodecType() == ICodec.Type.CODEC_TYPE_VIDEO)
{
streamCoder.setFlag(IStreamCoder.Flags.FLAG_QSCALE, false);
streamCoder.setCodec(ICodec.findEncodingCodecByName("h263"));
writer.addVideoStream(streamIndex, 0, VIDEO_WIDTH, VIDEO_HEIGHT);
}
super.onAddStream(event);
}
// //
#Override
public void onVideoPicture(IVideoPictureEvent event)
{
IVideoPicture pic = event.getPicture();
if (videoResampler == null)
{
videoResampler = IVideoResampler.make(VIDEO_WIDTH, VIDEO_HEIGHT, pic.getPixelType(), pic.getWidth(), pic.getHeight(), pic.getPixelType());
}
IVideoPicture out = IVideoPicture.make(pic.getPixelType(), VIDEO_WIDTH, VIDEO_HEIGHT);
videoResampler.resample(out, pic);
IVideoPictureEvent asc = new VideoPictureEvent(event.getSource(), out, event.getStreamIndex());
super.onVideoPicture(asc);
out.delete();
}
#Override
public void onAudioSamples(IAudioSamplesEvent event)
{
IAudioSamples samples = event.getAudioSamples();
if (audioResampler == null)
{
audioResampler = IAudioResampler.make(1, samples.getChannels(), 8000, samples.getSampleRate());
}
if (event.getAudioSamples().getNumSamples() > 0)
{
IAudioSamples out = IAudioSamples.make(samples.getNumSamples(), samples.getChannels());
audioResampler.resample(out, samples, samples.getNumSamples());
AudioSamplesEvent asc = new AudioSamplesEvent(event.getSource(), out, event.getStreamIndex());
super.onAudioSamples(asc);
out.delete();
}
}
I just cant seem to figure out where the problem is. I wud be thankful if someone wud plz point me in the right direction.
EDIT: If i see the properties of my newly encoded video, its audio properties are set and its video properties are not i.e in video properties, dimension= 0 x 0, frame rate= N/A and codec= h.263. The problem here is the 0 x 0 dimension.
well i found the answer, well not exactly the answer but a way to do what i was doing. Right now i am not quiet sure why my code was not working but the hre u can find the solution that worked for me. the author here just makes a seperate resizer class, adds it as a listener to the reader,. It has the onPictureEvent method overridden. Then he makes another class MyVideoListener and overrides the onAddStream method and adds it as a listener to the writer. and then he links the two parts by adding writer as a listener to resizer. works like a a charm.