I've been having problems and, after spending a week trying out all kinds of solutions and tearing my hair out, I've come here to see whether anybody could help me.
I'm working on a 3D browser plugin for the Mac (I have one that works on Windows). The only fully-hardware accelerated way to do this is to use a CAOpenGLLayer (or something that inherits from that). If a NSWindow is created and you attach the layer to that window's NSView then everything works correctly. But, for some reason, I can only get a specific number of frames (16) to render when passing the layer into the browser.
Cocoa calls my layer's drawInCGLContext for the first 16 frames. Then, for some unknown reason, it stops calling it. 16 seems like a very specific - and programmatic - number of frames and so I wondered whether anybody had any insight into why drawInCGLContext would not be called after 16 frames?
I'm reasonably sure it's not because I pass the layer into the browser - I've created a very minimal example plugin that renders a rotating quad using CAOpenGLLayer and that actually works. But the full plugin is a lot more complicated than that and I just don't know where to look anymore. I just don't know why drawInCGLContext stops being called. I've tried forcing it using CATransaction, it definitely gets sent the setNeedsDisplay message - but drawInCGLContext is never called. OpenGL doesn't report any errors either (I'm currently checking the results of all OpenGL calls). I'm confused! Help?
So, for anybody else who has this problem in future: You're trying to draw using the OpenGL context outside of the drawInCGLContext. There was a bug in the code where nearly all the drawing was happening from the correct place (drawInCGLContext) but one code path led to it rendering outside of there.
No errors are raised nor does glGetError return any problems. It just stops rendering. So if this happens to you - you're almost certainly making the same mistake I made!
Related
Edit
I implemented the TangoCameraPreview by itself in a test app and it works fine. I think this error is being caused by the specific circumstances of my app. I am using Rajawali to render some graphics in my app so that could be it, but I'm not sure...
Original
I'm using the Java API and trying to use the TangoCameraPreview class instead of building my own SurfaceView and Renderer like they do in the examples. But it looks like something is failing on the JNI level and I can't get it working. Sometimes, I'll see the video for a split second, but inevitably I always get the error:
12-30 17:20:03.544 4005-4515/com.gloppygloop W/tango_client_api: TangoErrorType TangoService_updateTexture(TangoCameraId, double*): TangoService got a negative timestamp -1.000000 from camera with id 2.
12-30 17:20:03.546 4005-4515/com.gloppygloop E/TangoCameraPreview: Error updating texture.
com.google.atap.tangoservice.TangoInvalidException
at com.google.atap.tangoservice.Tango.throwTangoExceptionIfNeeded(Tango.java:826)
at com.google.atap.tangoservice.Tango.updateTexture(Tango.java:333)
at com.google.atap.tangoservice.TangoCameraPreview.updateTexture(TangoCameraPreview.java:267)
at com.google.atap.tangoservice.TangoCameraPreview$MainRenderer.onDrawFrame(TangoCameraPreview.java:117)
at android.opengl.GLSurfaceView$GLThread.guardedRun(GLSurfaceView.java:1523)
at android.opengl.GLSurfaceView$GLThread.run(GLSurfaceView.java:1240)
I'm basically just calling the TangoCameraPreview.onFrameAvailable, whenever I have OnFrameAvailable from the OnTangoUpdateListener like they recommend in the docs. As far as I can tell, its just because the JNI method is not able to get a timestamp properly from the camera and that causes an error. I'm trying to use the Fisheye camera, but I've tried the Color camera and the RGBIR camera as well.
Found the solution, so I'll just leave it here in case anyone else gets stuck with this. The problem was that I was using another GLSurfaceView in the same context as my TangoCameraPreview. After some searching around, I found this answer which cleared things up for me. Basically, whenever you have multiple SurfaceViews in the same context you need to be conscious of the Z-order of these views. Otherwise, you'll just get strange behavior like I was experiencing. The Tango error was just misleading because I'm still getting the error even though the TangoCameraPreview is functioning as expected.
I am making a drawing on NSView using a timer that is set to update every .02 seconds. On update some physical simulation makes a step, and then Canvas!.needsDisplay = true. It works when app is in foreground (usually), but when some lags happen, simulation progresses forward despite the fact that view hasn't reflected it yet. How do I pause the timer during these times to make simulation happen only when NSView can show it? I do not want to call step_over from inside drawRect, cause it seems like a bad idea, because then it would be harder to stop the simulation.
Generally this kind of update should be done the other way around, by letting the display ask you for frames as it can display them. This is done with a CADisplayLink CVDisplayLink (this is Mac; CADisplayLink is iOS). Configure it with a method you want to be called when a frame can be drawn.
Generally you do want your simulation to keep moving forward, even if it means dropping frames occasionally. For that, you check the timestamp and use that to work out what time to use for your new frame. But if you only want to move forward when the display can show it, then just update once per call.
Note that generating at 50fps is often going to mismatch the system that's trying to draw at 60fps, so you're going to wind up missing frames occasionally. That's one of several reasons not to try to push drawing with a timer.
See also Alternative of CADisplayLink for Mac OS X. Note that trying to draw at 50fps with Core Graphics usually isn't going to give good results in any case. The right tool here in OS X is Core Animation (or SpriteKit for games on 10.10, or OpenGL for more advanced high-speed rendering). You can do very basic animations with an NSTimer (and we did for years before Core Animation came along), but it's not really a tool for complex drawing.
I am trying to create an application with an OpenGL view using MonoMac. Setting up an application and an NSOpenGLView was fairly simple...
...but for some reason I cannot get a consistent frame rate rendering OpenGL. The issue I am having is that 9 out of ten frames have perfect performance and every tenth frame or so I am getting a massive frame time spike (about 60ms-80ms for a single frame). The time of the slow frame correlates with the size of the control (and even more so using retina backing buffer).
I have been digging and have come up with nothing that works for my case.
I tried to use NSOpenGLView with CVDisplayLink and rendering on the main thread with timers and DrawRect.
I tried MonoMacGameView also both versions. Actually MonoMacGameView has consistent performance but only draws when my window does not have a background color.
I reimplemented the run loop to do my own NextEvent polling just to find out that that is not the issue...
So, my current hunch is that it has something to do with layer backed rendering in Cocoa views but I really cannot figure out what is causing this.
Any hint as to what is causing this delay?
I found a solution which produces pretty good results:
Do not use NSOpenGLView or MonoMacGameView.
Use the approach described in the example on this page: http://cocoa-mono.org/archives/366/monomac-and-opengl/
To enable retina support export the ContentScale property on the deriving class and set that depending whether you are running on retina screen or not
In conclusion, using a core animation layer is the only viable solution.
Basically I am downloading a zip file and extracting a collada file to load in the browser. This works freaking awesome in chrome but is REALLY slow with model movement from the mouse in Firefox. I cant seem to figure this out or if there's a setting I'm missing to speed up Firefox or what. The file is loaded up here
http://moneybagsnetwork.com/3d/test.htm
Its using jsunzip and three.js to load everything. I've bypassed the jsunzip and that's not the issue. I've also dumbed down the model to not use any event listeners and no lights and that didn't help one bit. Completely stumped here and the model really isn't that big :/
Here is a link to a zip of the files I'm using
http://moneybagsnetwork.com/3d/good.zip
Sorry about the multiple commented lines. I might turn things back on if this gets fixed.
I have noticed that Chrome is usually way faster and more responsive with Three.js applications than Firefox. The difference is not so much on the WebGL side, but at the plain Javascript supporting code.
Looking at your code, it seems you do some very heavy javascript stuff on your onmousemove function. This could very well cause much of the performance gap between the browsers. Mousemove is executed many many times during each and every mousemovement, so it quickly adds up to slow performance. It could also be that Firefox actually creates more mousemove events for similat cursor movements (not sure).
You could either move most of the raycasting and other stuff from mousemove to mouseclick. Alternatively, you could implement a delayed mousemove so that the function is called only maximum of X times per second, or only when the mouse has stopped. Something like this (untested but should work):
var mousemovetimer = null;
function onmousemove(event){
if (mousemovetimer) {
window.clearTimeout(mousemovetimer);
}
mousemovetimer = window.setTimeout(delayedmousemove, 100);
}
function delayedmousemove(event) {
// your original mousemove code here
}
Maybe your graphics card is in our blacklist. There is usually a note about this towards the bottom of about:support.
Cards can be blacklisted for various reasons, missing drivers / features, occasional crashes ... see:
http://www.sitepoint.com/firefox-enable-webgl-blacklisted-graphics-card/
To enable WebGL, set webgl.force-enabled to true.
To enable Layers Acceleration, set layers.acceleration.force-enabled to true
To enable Direct2D in Windows Vista/7, set gfx.direct2d.force-enabled to true
I just hit something particularly hard to debug when doing some pretty intensive rendering with Canvas2D. I use all kinds of things, from globalCompositeOperation to multiple off-screen canvases, with some drawImage magic in-between.
It works perfectly fine and smooth on :
Chrome (26) [OSX 10.7.5]
Safari (6.0.2) [OSX 10.7.5]
Firefox (Both 18 and 20 Aurora) [OSX 10.7.5]
Chrome (24) [Windows 7]
Firefox (12) [Windows 7]
Chromium (24) [Archlinux, Gnome 3]
EDIT: Added tests for Windows 7. Strangely enough, it works for FF12 (I had an old version on my dual boot) but there's a definite performance hit after upgrading to FF18. It's not as bad on Windows as it is on Linux though, and the same version on OSX works flawlessly. Regression maybe?
For some reason, on Firefox and Linux (I tried both 18 and 20 Aurora), I have bad rendering performances when dragging and rendering at the same time.
If I fire-and-forget an animation, it is on par with Chrome/Safari, but if I drag and render, I often end up only seeing the end frame after I release the drag.
Neither requestAnimationFrame nor a direct rendering on the mouse event handler work.
After profiling, the reported timings for the rendering parts are well within the range of acceptable (up to 100ms at the absolute worst), and definitely do not correspond to what I see on the screen.
I tried reducing the load by removing some stuff, ending up with reported render times under 15ms, but what I saw didn't change.
What baffles me is that it works almost everywhere else except with Firefox on Linux. Any idea as to where I should look, a bug report or a solution to my problem?
I have fully switched to Chrome on linux because of this issue. It stems from the old 2d rendering engine they are using called Cairo, which is old and out-dated. Azure was to replace this engine and they have it done basically all the platforms except linux.
http://blog.mozilla.org/joe/2011/04/26/introducing-the-azure-project/
https://bugzilla.mozilla.org/show_bug.cgi?id=781731
I think I know where you should look based on this:
If I fire-and-forget an animation, it is on par with Chrome/Safari, but if I drag and render, I often end up only seeing the end frame after I release the drag.
This is probably a double-buffering bug with Firefox on linux.
Canvas implementations have double-buffering built in. You can see it in action on any browser in a simple example like this one: http://jsfiddle.net/simonsarris/XzAjv/ (which uses a setTimeout vs extra work to illustrate that clearing does not happen right away)
The implementations try to delay all rendering by rendering it to an internal bitmap, and then all at once (at the next pause) render it to the canvas. This stops the "flickering" effect when clearing a canvas before redrawing a scene, which is good.
But it seems there's a plain old bug in the Linux Firefox. During your drag and render it seems to not be updating the canvas, probably in an attempt to buffer, but seems to be doing so when it should not be. This would explain why it works in fire-and-forget scenarios.
So I think a bug report is in order. I haven't got any linux machines lying around so I can't reproduce it and submit something myself to be certain though, sorry.
This is in reply to a comment: You could, during the mouse move, dispatch the drawing portion to a tiny timer.
For instance:
// The old way
theCanvas.addEventListener('mousemove', function() {
// if we're dragging and are redrawing
drawingCode();
}, false);
// The new way
theCanvas.addEventListener('mousemove', function() {
// if we're dragging and are redrawing
// in 1 millisecond, fire off drawing code
setTimeout(function() { drawingCode(); }, 1);
}, false);
There isn't such a method, its totally hidden. What you could do is, during mouse move, dispatch