I want to convert mouse's current X and Y coordinates into the 3D space I have drawn in the viewport. I need to do this on the OpenGL ES platform. I found following possible solutions implemented in OpenGL, but none fits what I am looking for.
I found NeHe's tutorial on doing exactly this, but in traditional OpenGL way. It uses gluUnProject.
http://nehe.gamedev.net/data/articles/article.asp?article=13
Although gluUnProject is not available in OpenGL ES, its implementation seems simple enough to port back. But before calling it, we need to call glReadPixels with GL_DEPTH_COMPONENT and that is not possible in OpenGL ES. (The reason I found in this thread: http://www.khronos.org/message_boards/viewtopic.php?f=4&t=771)
What I want to do is similar to picking, except that I don't want to select the object but I want exact coordinates so that I can recognize particular portion of the object that is currently under mouse cursor. I went through the Picking tutorials in this answer.
https://stackoverflow.com/posts/2211312/revisions
But they need glRenderMode, which I believe is absent in OpenGL ES.
If you know how to solve this problem in OpenGL ES, please let me know.
Thanks.
I think the general solution is to figure out where in world space the clicked coordinate falls, assuming the screen is a plane in the world (at the camera's location). Then you shoot a ray perpendicular to the plane, into your scene.
This requires "world-space" code to figure out which object(s) the ray intersects with; the solutions you mention as being unsuitable for OpenGL ES seem to be image-based, i.e. depend on the pixels generated when rendering the scene.
With OpenGL ES 2.0 you could use a FBO and render the depth values to a texture. Obviously, this wouldn't be exactly cheap (just a way around the restriction of glReadPixels)...
Further, since - as I understand it - you want to pick certain parts of your object you might want to do some sort of color-picking where each selectable portion of the object has an unique color (note that the Lighthouse 3D tutorial only shows the general idea behind color-picking, your implementation would probably be different). You could optimize a little by performing a ray/bounding-box intersection beforehand and only rendering the relevant candidates to the texture used for picking.
Related
I am a newbie in both OpenGL as well as Three.js, I am working on a streaming based "on-line" viewer which uses websockets to transmit points (with surface normals) from one system application to a remote web interface. Long story short, I have modified BufferGeometry and use THREE.PointsMaterial to visualize incoming data with points.
Since I already am sending points locations [xyz], color [rgb] and normals [abc] so I would love to use technique such as surface-splatting Splatting. Unfortunately due to my limited knowledge and lack of internet resources can any one guide me to implement a very basic surface splatting technique using three.js?
Question: How to draw elliptical surfaces instead of points in three.js
Any help will be highly appreciated.
it would probably work using points if you compute the point-size per point such that the whole ellipsis fits in there and use the fragment-shader to compute the area of the ellipsis based on the viewing-angle (i suppose this is what you want to do, right?).
Alternatively, you can use instancing based on a simple quad and use instance-attributes for position and orientation of the quads. In this case, you just need to render a circle into each of the quads.
I've been trying to render silhouettes on CAD models with webgl. The closest i got to the desired result was with fwidth and a dot between the normal and the eye vector. I found it difficult to control the width though.
I saw another web based viewer and it's capable of doing something like this:
I started digging through the shaders, and the most i could figure out is that this is analytical - an actual line entity is drawn and that the width is achieved by rendering a quad instead of default webgl lines. There is a bunch of logic in the shader and my best guess is that the vertex positions are simply updated on every render.
This is a procedural model, so i guess that for cones and cylinders, two lines can always be allocated, silhouette points computed, and the lines updated.
If that is the case, would it be a good idea to try and do something like this in the shader (maybe it's already happening and i didn't understand it). I can see a cylinder being written to attributes or uniforms and the points computed.
Is there an approach like this already documented somewhere?
edit 8/15/17
I have not found any papers or documented techniques about this. But it got a couple of votes.
Given that i do have information about cylinders and cones, my idea is to sample the normal of that parametric surface from the vertex, push the surface out by some factor that would cover some amount of pixels in screen space, stencil it, and draw a thick line thus clipping it with the actual shape of the surface.
The traditional shader-based method is Gooch shading. The original paper is here:
http://artis.imag.fr/~Cyril.Soler/DEA/NonPhotoRealisticRendering/Papers/p447-gooch.pdf
The old fashing OpenGL technique from Jeff Lander
Question is quite simple. On Windows you have BitBlt() to draw to the screen. On OS X you could normally use OpenGl but how do I draw the bitmap to the screen using Apples new Metal framework? I cant find anything valuable in Apples Metal references.
I'm right now using Core Graphics for the drawing part but since my bitmap is updating all the time, I feel like I should move to Metal to reduce the overhead.
The very short answer is this: First, you need to setup a standard rendering pipeline using Metal, which is a bit of work, if you don't know anything about the rendering pipeline (note: but this can be simplified by avoiding 3D stuff by rendering a quad with two triangles, just give xy coordinates for the vertices). There is a lot of sample code from Apple that shows how to setup a standard rendering pipeline, the simplest is maybe MetalImageProcessing, so you could strip that down (even though that uses a compute shader which is overcomplicated to draw in, you'd want to substitute it for standard vertex and fragment shaders).
Second, you need to learn how vertex and fragment shaders work and how to draw stuff with them, see shadertoy for this.
Just noticed the users last comment, if you have an array of bytes that represent an image then you can just make an MTLTexture out of that and render it to the screen, see Apple's example above but change the compute shader to standard vertex and fragments shaders for faster performance.
I've read the "learning webgl" tutorial, but it does not explain everything. Something like google experiments with webgl are amazing, but I've been wondering... how do you move a 3D object along a custom path to swing into the scene or create a custom transition?
webgl -> opengl in web, so how do you do that in opengl?
what you're looking for is pretty common functionality, but it is hard to find concrete examples showing how to do it.
the easiest way i have found to do it is using Apple's J3DIMath.js webgl library.
you basically want to define a "camera" perspective matrix, then move the camera along a predefined path of vertices through your 3d space. as you move along the "track" of vertices, at each draw frame you can call the function J3DIMatrix4.lookat(), passing it the position vector along the path, the direction to look at, and the "up" direction, and it will create the appearance of a moving camera.
i hope this helps!
J3DIMath.js
I am working on a project where I am to use the Kinect to track facial expressions and animate a 3D model (.ply) accordingly.
So far I have managed to track a human face with a finite number of key-points on the face. I am able to get the coordinates of each key-point at every frame.
I am not very adept with animating techniques and general concepts of Mesh deformation and would really appreciate if one could provide a library which provides a high-level API so as to do said animation using x,y coordinates of key-points.
I am aware of CUBICA but unsure whether it can be used for what I want. Please excuse me as I am not very adept with this and would appreciate any help.
I too am looking for something that would do this (with .NET if possible), so I could hook Kinect up, set the position of each joint and see the character animate - without needing to set the position and angle of each bone which quickly gets very complex when you take into account the X,Y and Z positions.
So far my research has lead me to believe I will need a 3D engine that supports inverse kinematics - if anyone else had any better advice I'd be keen to hear it.