I am working on a project that involves visualizing the output of an AI model that detects joints and returns their X, Y, and Z coordinates. I have a human model with skinned meshes and bones that has been imported into a Three.js scene. I need to apply these coordinates to the joints of the human model in order to visualize them accurately.
I have tried changing the positions of the skinned meshes directly, but this doesn't seem to work. Instead, I need to rotate the bones one by one to achieve the desired result. However, I am facing a complex inverse kinematics problem as each bone has its own coordinate system and axes, and it's challenging to extract the rotations from the coordinates.
Is there an easier or more straightforward approach to this problem that I am overlooking?
Related
I need to run some calculations on meshes using ThreeJS.
The calculation should involve the faces of the mesh, but not the triangular ones.
for example in the attached image, I'd like to consider both of the triangles of the top faces as a single face.
Is there a way to know which triangles go together?
I've seen that the geometry has a "groups" property.
https://threejs.org/docs/#api/en/core/BufferGeometry.groups
But it just says it is used to split the rendering.
Can I rely on it to determine that the vertices in the group form the "face" that I need?
Is there any other way to get it?
I am trying to create a 3D Visualization of an RC airplane in Threebox. The RC plane sends live telemetry, including:
GPS Coordinates
Gyro sensor data, showing the pitch, roll and heading of the plane.
I have now loaded a Model of an airplane in Threebox, no problems with that.
My problem comes down to the rotation of the plane. I want the plane object to represent the current orientation of the RC plane. Since I have the live telemetry from the flight controller, this should be possible.
In the Documentation, I have found this method, which seemed like exactly what i needed:
plane.setRotation({x: roll, y: pitch, z: yaw/heading})
And it basically works. I can rotate the Plane around its axes. But things get messed up when I combine the rotations.
For example: When I just update the roll axis, the Object behaves just like I want it to. However, when i change the heading of the plane by 90 degrees, the roll axis suddenly becomes the pitch axis. It seems to me, that the axes of the Plane object don't rotate with the plane itself.
I've prepared a recreation of the issue on jsfiddle. You can change the heading of the plane using the slider in the bottom right.
I've been stuck on this for days, would be super happy for any help!
There are lots of issues with your jsfiddle that prevent it from running. To isolate an issue and make it easier to test you should eliminate as many variables as possible - you're using two third-party libraries that will play a big hand in how transformations behave, particularly threebox.
I would recommend sticking with three.js's built in transformation tools unless you specifically need some lat/lng transformations, or other transformations to move between a local cartesian space and a global coordinate system. In this case, a very basic plane.setRotationFromEuler(new THREE.Euler(yaw, pitch, roll)) should do the trick. Be aware of how much order in euler rotations can affect the outcome, and that three.js uses radians for all its rotations, not degrees.
I am attempting to render a flat, dynamically created heatmap on top of a 3D model that is loaded from an OBJ (or STL).
I am currently loading and rendering an OBJ with Three.js. I have vector3 points that I am currently drawing as simple red cubes (image below). These data points are all raycasted to my OBJs mesh and are lying on the surface. The vector3 points are loaded from an external data source and will change depending on what data is being viewed/collected.
I would like to render my vector3 point data into a heatmap on the surface of my OBJ. Here are some examples illustrating the type of visual effects I am trying to achieve:
I feel like vertex coloring is the method of achieving this, but my issue is that my OBJ model does not have enough tessellation to do this. As you can see many red dots fall on each face. I am struggling to find a way to draw over my object's mesh with colors exactly where my red point data is. I was assuming I would need to convert my random vector3 points into a mesh, but cannot find a method to do so.
I've looked at the possibility of generating a texture, but 1) I do not have a UV map for my OBJs and do not see a way to programmatically generate them and 2) I am a bit lost on how I would correlate vector3 point data to UV points.
I've looked at using shaders, but my vector3 point data appears to be too large for using a shader (could be hundreds of thousands of points). I also feel it is not the right approach to render the heatmap every frame and would rather only render it once on load.
I've looked into isosurfaces with point clouds and the marching cubes algorithm, but I didn't think this was the right direction since only my data is a bit like a point cloud, and I am unsure as to how I would keep this smooth along the surface of my OBJ mesh.
Although I would prefer to keep everything in JavaScript for viewing in the browser, I am open to doing server side processing in any language/program with REST so long as it can be automated without human intervention, and pushed back to the browser for rendering.
Any suggestions or guidance is appreciated.
I'm only guessing but it seems like first you need to have UV coordinates that map every triangle to a texture. Rather than do this by hand I'd suggest using a modeling package. Most modeling packages have some way of automatically and uniformly mapping every triangle to a texture. For example in Blender
Next to put the heatmap in the texture by computing which triangles are affected by each dot (your raycasting), looking up their texture coordinates, projecting that dot into texture space and then putting the colors in that part of the texture. I'm only guessing that you need to not just do exact points but probably need to consider adjacent triangles since some heat info that hits near the edge of a triangle needs to bleed over into the adjacent triangle but that adjacent triangle might be using a completely different part of the texture.
I was wondering if there is a way to copy the rotational information of a sprite into a plane. It seems that for the sprite the rotation is set within the material and it is just one value. I tried copying the matrix of the sprite into that of the plane but had no luck.
Sprite rotation is defined in WebGL as you can read in SpritePlugin.js. Thus, if you wish to copy the rotation of an existing sprite in your scene, the easiest way may be to use quaternion. They allow better defining than x/y/z-angle defined ones (Euler rotations) as they ask for the rotation axis and the angle value. In your context, at each camera movement you modify a quaternion. You set it with quaternion.setFromAxisAngle(axis,angle). Then apply it to the plane with plane.setRotationFromQuaternion(quaternion).
Math side :
The axis would be the cross product between the vectors lastCameraPosition-sprite.position and actualCameraPosition-sprite.position,
the angle, the one between those vectors, got with the acos of their dot product divided by the product of their lengths.
For more about those maths, functions getMouseProjectionOnBall and rotateCamera in TrackBallControls are of great interest.
Hope it helped !
What I'm trying to do is to "drape" some points on a PlaneGeometry. I have the planar coordinates of the points in the same coordinate system of my plane geometry, what I need i sto get the "height" from the plane to position the points on top of it.
What's the best way to achieve it? Querying the planar mesh in Javascript would be to heavy. Should it be done (and could it be done) using the vertex shader?
EDIT
Probably using a ray caster is the right solution, something like shown in this example: http://threejs.org/examples/#webgl_geometry_terrain_raycast
EDIT2
Raycasting does the job, but it's quite slow for a lot of objects. I suppose there are more efficient ways to do that...
https://dl.dropboxusercontent.com/u/13861666/2014-01-17%2011_27_15-firenze.png