I'm having an issue with LinqToTwitter 4.1 where having a user stream open will eventually cause the programs memory usage to balloon out of control. This does not occur when the program starts running but only after some time, normally after a day or two.
Using the ANTS Memory Profiler I find this reference chain preventing System.Byte[] from being collected. The full profiler results can be downloaded here.
Code:
private async Task<Streaming> TwitterSteam(string trackHashTags, string twitterUserIds)
{
var stream = (from strm in TwitterCtx.Streaming.WithCancellation(CloseStreamSource.Token)
where strm.Type == StreamingType.Filter &&
strm.Track == trackHashTags
&& strm.Follow == (string.IsNullOrEmpty(twitterUserIds) ? "41553192" : twitterUserIds)
select strm).StartAsync(async strm =>
{
string message = string.IsNullOrEmpty(strm.Content) ? "Keep-Alive" : strm.Content;
if (message == "Keep-Alive")
{
IsRunning = true;
}
else
{
JsonData data = JsonMapper.ToObject(message);
Status tweet = new Status(data);
LogClient.LogInfo("Received Tweet: " + tweet.Text, null, LogType.Info, null);
ConvertToMessage(tweet);
IsRunning = true;
}
}).Result.SingleOrDefault();
return stream;
}
Can anyone provide insight as to why this is occurring and how I can prevent it?
Related
In okhttp's code, connection.idleAtNs has been assigned in releaseConnectionNoEvents() method,
internal fun releaseConnectionNoEvents(): Socket? {
val connection = this.connection!!
connection.assertThreadHoldsLock()
val calls = connection.calls
val index = calls.indexOfFirst { it.get() == this#RealCall }
check(index != -1)
calls.removeAt(index)
this.connection = null
if (calls.isEmpty()) {
connection.idleAtNs = System.nanoTime()
if (connectionPool.connectionBecameIdle(connection)) {
return connection.socket()
}
}
return null
}
but why Re-assignment here
private fun pruneAndGetAllocationCount(connection: RealConnection, now: Long): Int {
connection.assertThreadHoldsLock()
val references = connection.calls
var i = 0
while (i < references.size) {
val reference = references[i]
if (reference.get() != null) {
i++
continue
}
// We've discovered a leaked call. This is an application bug.
val callReference = reference as CallReference
val message = "A connection to ${connection.route().address.url} was leaked. " +
"Did you forget to close a response body?"
Platform.get().logCloseableLeak(message, callReference.callStackTrace)
references.removeAt(i)
connection.noNewExchanges = true
// If this was the last allocation, the connection is eligible for immediate eviction.
if (references.isEmpty()) {
connection.idleAtNs = now - keepAliveDurationNs
return 0
}
}
return references.size
}
If assign a value here, it may appear that the connection is removed as soon as it becomes idle.
The pruneAndGetAllocationCount method is called as part of a cleanup task. At the point you have linked to, the code has already logged a warning about a connection leak. The code releases that connection then considders the connection to be immediately available for release.
If you have concerns about the code, consider making a test case that shows what you would propose changing the code to, it might be possible to improve it.
If it's affecting you because of leaked connections, you should instead fix the bug in the application code.
// We've discovered a leaked call. This is an application bug.
val callReference = reference as CallReference
val message = "A connection to ${connection.route().address.url} was leaked. " +
"Did you forget to close a response body?"
I want to get a RTMP link and stream the video at my unity project but, and it works fine.... just 4 frames long cause after that it freezes, but audio stills playing.
For this code i´m using VSCode,Unity3D and the public example that the creators have on their github.
// This is the start, where i initialize things of course hahauhsuhs
void Start()
{
Core.Initialize(Application.dataPath);
_libVLC = new LibVLC();
PlayPause();
}
//Then here we have the method play and Pause, where we play Media Player and give him the media URL that works for a few frames.
private void PlayPause()
{
if (_mediaPlayer == null)
{
_mediaPlayer = new MediaPlayer(_libVLC);
}
if (_mediaPlayer.IsPlaying)
{
_mediaPlayer.Pause();
}
else
{
_isPlaying = true;
if (_mediaPlayer.Media == null)
{
// playing remote media
_mediaPlayer.Media = new Media(_libVLC, new Uri(URL));
}
_mediaPlayer.Play();
}
}
// This method will be execute every frame and do some crazy stuff that i can´t explane right now.
private void Update()
{
//A few checks before executing video
if (!_isPlaying) return;
if (URL.Equals(null)) URL = "http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/BigBuckBunny.mp4"; //if URL is null give the BigChungus Bunny video
//Execute Video
if (_tex == null)
{
// If received size is not null, it and scale the texture
uint i_videoHeight = 0;
uint i_videoWidth = 0;
_mediaPlayer.Size(0, ref i_videoWidth, ref i_videoHeight);
var texptr = _mediaPlayer.GetTexture(out bool updated);
if (i_videoWidth != 0 && i_videoHeight != 0 && updated && texptr != IntPtr.Zero)
{
Debug.Log("Creating texture with height " + i_videoHeight + " and width " + i_videoWidth);
_tex = Texture2D.CreateExternalTexture((int)i_videoWidth,
(int)i_videoHeight,
TextureFormat.RGBA32,
false,
true,
texptr);
RenderSettings.skybox.mainTexture = _tex;
}
}
else if (_tex != null)
{
var texptr = _mediaPlayer.GetTexture(out bool updated);
if (updated)
{
_tex.UpdateExternalTexture(texptr);
}
}
}
this is my first real question on stackover flow so this post can have some issues, I´m open for suggestion to improve it.
I found that the problem it´s with my stream server latency, and not with my unity project.
But still... there is a way to persist the video to play with high latency?
I am using Linphone SDK in Xamarin.forms project for the sip calling. I am able to make the connection using following code:
var authInfo = Factory.Instance.CreateAuthInfo(username.Text,
null, password.Text, null, null,domain.Text);
LinphoneCore.AddAuthInfo(authInfo);
String proxyAddress ="sip:"+username.Text+"#192.168.1.180:5160";
var identity = Factory.Instance.CreateAddress(proxyAddress);
var proxyConfig = LinphoneCore.CreateProxyConfig();
identity.Username = username.Text;
identity.Domain = domain.Text;
identity.Transport = TransportType.Udp;
proxyConfig.Edit();
proxyConfig.IdentityAddress = identity;
proxyConfig.ServerAddr = domain.Text + ":5160;transport=udp";
proxyConfig.Route = domain.Text;
proxyConfig.RegisterEnabled = true;
proxyConfig.Done();
LinphoneCore.AddProxyConfig(proxyConfig);
LinphoneCore.DefaultProxyConfig = proxyConfig;
LinphoneCore.RefreshRegisters();
After Successful connection, I am using the code for placing the code.
if (LinphoneCore.CallsNb == 0)
{
string phoneCall = "sip:"+address.Text+ "#192.168.1.180:5160";
var addr = LinphoneCore.InterpretUrl(phoneCall);
LinphoneCore.InviteAddress(addr);
}
else
{
Call call = LinphoneCore.CurrentCall;
if (call.State == CallState.IncomingReceived)
{
LinphoneCore.AcceptCall(call);
}
else
{
LinphoneCore.TerminateAllCalls();
}
}
And the listener that is listening to call state changed event is as:
private void OnCall(Core lc, Call lcall, CallState state, stringmessage)
{
call_status.Text = "Call state changed: " + state;
if (lc.CallsNb > 0)
{
if (state == CallState.IncomingReceived)
{
call.Text = "Answer Call (" + lcall.RemoteAddressAsString + ")";
}
else
{
call.Text = "Terminate Call";
}
if (lcall.CurrentParams.VideoEnabled) {
video.Text = "Stop Video";
} else {
video.Text = "Start Video";
}
}
else
{
call.Text = "Start Call";
call_stats.Text = "";
}
}
The call status is giving 'Internal Server Error'. I am able to receive the calls using Linphone or X-lite Soft Phone in my code, But I am not able to place the calls. I don't know whether this issue is related to server or it is related to my code. Please suggest.
Internal Server Error (HTTP Status code 500) means that an unexpected error occurred on the server. So I would suspect the problem is rather there than with your app's code.
500 - A generic error message, given when an unexpected condition was encountered and no more specific message is suitable.
It could be that your request doesn't satisfy the expectations of the endpoint you are calling, but even then, the server should then respond with a more meaningful error, than crashing with 500.
Problem Description
I am a .NET Core developer and I have recently been asked to transcribe mp3 audio files that are approximately 20 minutes long into text. Thus, the file is about 30.5mb. The issue is that speech is sparse in this file, varying anywhere between 2 minutes between a spoken sentence or 4 minutes of length.
I've written a small service based on the google speech documentation that sends 32kb of streaming data to be processed from the file at a time. All was progressing well until I hit this error that I share below as follows:
I have searched via google-fu, google forums, and other sources and I have not encountered documentation on this error. Suffice it to say, I think this is due to the sparsity of spoken words in my file? I am wondering if there is a programmatical centric workaround?
Code
I have used some code that is a slight modification of the google .net sample for 32kb streaming. You can find it here.
public async void Run()
{
var speech = SpeechClient.Create();
var streamingCall = speech.StreamingRecognize();
// Write the initial request with the config.
await streamingCall.WriteAsync(
new StreamingRecognizeRequest()
{
StreamingConfig = new StreamingRecognitionConfig()
{
Config = new RecognitionConfig()
{
Encoding =
RecognitionConfig.Types.AudioEncoding.Flac,
SampleRateHertz = 22050,
LanguageCode = "en",
},
InterimResults = true,
}
});
// Helper Function: Print responses as they arrive.
Task printResponses = Task.Run(async () =>
{
while (await streamingCall.ResponseStream.MoveNext(
default(CancellationToken)))
{
foreach (var result in streamingCall.ResponseStream.Current.Results)
{
//foreach (var alternative in result.Alternatives)
//{
// Console.WriteLine(alternative.Transcript);
//}
if(result.IsFinal)
{
Console.WriteLine(result.Alternatives.ToString());
}
}
}
});
string filePath = "mono_1.flac";
using (FileStream fileStream = new FileStream(filePath, FileMode.Open))
{
//var buffer = new byte[32 * 1024];
var buffer = new byte[64 * 1024]; //Trying 64kb buffer
int bytesRead;
while ((bytesRead = await fileStream.ReadAsync(
buffer, 0, buffer.Length)) > 0)
{
await streamingCall.WriteAsync(
new StreamingRecognizeRequest()
{
AudioContent = Google.Protobuf.ByteString
.CopyFrom(buffer, 0, bytesRead),
});
await Task.Delay(500);
};
}
await streamingCall.WriteCompleteAsync();
await printResponses;
}//End of Run
Attempts
I've increased the stream to 64kb of streaming data to be processed and then I received the following error as can be seen below:
Which, I believe, means the actual api timed out. Which is decidely a step in the wrong direction. Has anybody encountered a problem such as mine with the Google Speech Api when dealing with a audio file with sparse speech? Is there a method in which I can filter the audio down to only spoken words progamatically and then process that? I'm open to suggestions, but my research and attempts have only lead me to further breaking my code.
There is to way for recognize audio in Google Speech API:
normal recognize
long running recognize
Your sample is uses the normal recognize, which has a limit for 15 minutes.
Try to use the long recognize method:
{
var speech = SpeechClient.Create();
var longOperation = speech.LongRunningRecognize( new RecognitionConfig()
{
Encoding = RecognitionConfig.Types.AudioEncoding.Linear16,
SampleRateHertz = 16000,
LanguageCode = "hu",
}, RecognitionAudio.FromFile( filePath ) );
longOperation = longOperation.PollUntilCompleted();
var response = longOperation.Result;
foreach ( var result in response.Results )
{
foreach ( var alternative in result.Alternatives )
{
Console.WriteLine( alternative.Transcript );
}
}
return 0;
}
I hope it helps for you.
PLEASE READ THE UPDATE #2 BELOW IF YOU ARE INTERESTED IN THIS PROBLEM ;)
Say I put this code into the JS of my extension.
var reader = {
onInputStreamReady : function(input) {
var sin = Cc["#mozilla.org/scriptableinputstream;1"]
.createInstance(Ci.nsIScriptableInputStream);
sin.init(input);
sin.available();
var request = '';
while (sin.available()) {
request = request + sin.read(512);
}
console.log('Received: ' + request);
input.asyncWait(reader,0,0,null);
}
}
var listener = {
onSocketAccepted: function(serverSocket, clientSocket) {
console.log("Accepted connection on "+clientSocket.host+":"+clientSocket.port);
input = clientSocket.openInputStream(0, 0, 0).QueryInterface(Ci.nsIAsyncInputStream);
output = clientSocket.openOutputStream(Ci.nsITransport.OPEN_BLOCKING, 0, 0);
input.asyncWait(reader,0,0,null);
}
}
var serverSocket = Cc["#mozilla.org/network/server-socket;1"].
createInstance(Ci.nsIServerSocket);
serverSocket.init(-1, true, 5);
console.log("Opened socket on " + serverSocket.port);
serverSocket.asyncListen(listener);
Then I run Firefox and connect to the socket via telnet
telnet localhost PORT
I send 5 messages and they get printed out, but when I try to send 6th message I get
firefox-bin: Fatal IO error 11 (Resource temporarily unavailable) on X server :0.0.
Even worse, when I try to put this same code into an XPCOM component (because that's where I actually need it), after I try sending a message via telnet I get
Segmentation fault
or sometimes
GLib-ERROR **: /build/buildd/glib2.0-2.24.1/glib/gmem.c:137: failed to allocate 32 bytes
aborting...
Aborted
printed to the terminal from which I launched firefox.
This is really weird stuff.. Can you spot something wrong with the code I've pasted or is smth wrong with my firefox/system or is the nsIServerSocket interface deprecated?
I'm testing with Firefox 3.6.6.
I would really appreciate some answer. Perhaps you could point me to a good example of using Sockets within an XPCOM component. I haven't seen many of those around.
UPDATE
I just realised that it used to work so now I think that my Console
component breaks it. I have no idea how this is related. But if I
don't use this component the sockets are working fine.
Here is the code of my Console component. I will try to figure out
what's wrong and why it interferes and I'll post my findings later.
Likely I'm doing something terribly wrong here to cause Segmentation
faults with my javascript =)
Voodoo..
components/Console.js:
const Cc = Components.classes;
const Ci = Components.interfaces;
const Cr = Components.results;
Console.prototype = (function() {
var win;
var initialized = false;
var ready = false;
var _log = function(m, level, location) {
if (initialized&&ready) {
var prefix = "INFO: ";
switch (level) {
case "empty":
prefix = ""
break;
case "error":
prefix = "ERORR: "
break;
case "warning":
prefix = "WARNING: "
break;
}
win.document.getElementById(location).value =
win.document.getElementById(location).value + prefix + m + "\n";
win.focus();
} else if (initialized&&!ready) {
// Now it is time to create the timer...
var timer = Components.classes["#mozilla.org/timer;1"]
.createInstance(Components.interfaces.nsITimer);
// ... and to initialize it, we want to call
event.notify() ...
// ... one time after exactly ten second.
timer.initWithCallback(
{ notify: function() { log(m); } },
10,
Components.interfaces.nsITimer.TYPE_ONE_SHOT
);
} else {
init();
log(m);
}
}
var log = function(m, level) {
_log(m, level, 'debug');
}
var poly = function(m, level) {
_log(m, "empty", 'polyml');
}
var close = function() {
win.close();
}
var setReady = function() {
ready = true;
}
var init = function() {
initialized = true;
var ww = Components.classes["#mozilla.org/embedcomp/window-
watcher;1"]
.getService(Components.interfaces.nsIWindowWatcher);
win = ww.openWindow(null, "chrome://polymlext/content/
console.xul",
"console", "chrome,centerscreen,
resizable=no", null);
win.onload = setReady;
return win;
}
return {
init: init,
log : log,
poly : poly,
}
}());
// turning Console Class into an XPCOM component
Components.utils.import("resource://gre/modules/XPCOMUtils.jsm");
function Console() {
this.wrappedJSObject = this;
}
prototype2 = {
classDescription: "A special Console for PolyML extension",
classID: Components.ID("{483aecbc-42e7-456e-b5b3-2197ea7e1fb4}"),
contractID: "#ed.ac.uk/poly/console;1",
QueryInterface: XPCOMUtils.generateQI(),
}
//add the required XPCOM glue into the Poly class
for (attr in prototype2) {
Console.prototype[attr] = prototype2[attr];
}
var components = [Console];
function NSGetModule(compMgr, fileSpec) {
return XPCOMUtils.generateModule(components);
}
I'm using this component like this:
console = Cc["#ed.ac.uk/poly/console;1"].getService().wrappedJSObject;
console.log("something");
And this breaks the sockets :-S =)
UPDATE #2
Ok, if anyone is interested in checking this thing out I would really
appreciate it + I think this is likely some kind of bug (Seg fault
from javascript shouldn't happen)
I've made a minimal version of the extension that causes the problem,
you can install it from here:
http://dl.dropbox.com/u/645579/segfault.xpi
The important part is chrome/content/main.js:
http://pastebin.com/zV0e73Na
The way my friend and me can reproduce the error is by launching the
firefox, then a new window should appear saying "Opened socket on
9999". Connect using "telnet localhost 9999" and send a few messages.
After 2-6 messages you get one of the following printed out in the
terminal where firefox was launched:
1 (most common)
Segmentation fault
2 (saw multiple times)
firefox-bin: Fatal IO error 11 (Resource temporarily unavailable) on
X
server :0.0.
3 (saw a couple of times)
GLib-ERROR **: /build/buildd/glib2.0-2.24.1/glib/gmem.c:137: failed
to
allocate 32 bytes
aborting...
Aborted
4 (saw once)
firefox-bin: ../../src/xcb_io.c:249: process_responses: Assertion
`(((long) (dpy->last_request_read) - (long) (dpy->request)) <= 0)'
failed.
Aborted
If you need any more info or could point me to where to post a bug
report :-/ I'll be glad to do that.
I know this is just one of the many bugs... but perhaps you have an
idea of what should I do differently to avoid this? I would like to
use that "console" of mine in such way.
I'll try doing it with buffer/flushing/try/catch as people are suggesting, but I wonder whether try/catch will catch the Seg fault...
This is a thread problem. The callback onInputStreamReady happened to be executed in a different thread and accessing UI / DOM is only allowed from the main thread.
Solution is really simple:
change
input.asyncWait(reader,0,0,null);
to
var tm = Cc["#mozilla.org/thread-manager;1"].getService();
input.asyncWait(reader,0,0,tm.mainThread);