HoloLens - UI/Slider and Cursor do not intersect during gaze - user-interface

I'm trying to use UI/Slider in Unity app for HoloLens.
I used the steps described here - Unity UI on the HoloLens
So as a result I have following structure:
MainCamera properties:
SliderCanvas is using MainCamera:
Slider properties:
InteractiveMeshCursor is taken from HoloToolkit.
As a result I'm getting this picture:
When I move the head the Cursor behaves correctly - it stays in the middle of the scene. If I add other 3D objects on the Scene it also correctly changes its states so GazeManager looks like is working correctly.
However I cannot gaze at Slider because it moves with the Camera too and stays in the bottom/center of the scene where I want it to be. So in my case there is no way for them to intersect.
How can I fix this? Do I need to add an other Camera for the SliderCanvas but then how to control both cameras? I am definitely missing something and would appreciate your help.

As expected the solution turned out to be simple (I missed one step from the tutorial). For UI objects you need to set Render Mode property for the Canvas to World Space and changed the Position and Scale of the Slider. Now the Gaze is working.
SliderCanvas properties:
Slider properties:

Related

Misplaced UI made with UI in hololens emulator

I got a small problem and I can’t really find the source of it. I just created a Canvas and put a Panel in the top right corner. I got it to the right scale at 3: for the Hololens and loaded it up in the Holoemulator. The problem is that even though the panel is supposed to be in the corner, it is placed almost in the middle of the screen in the end.
Is there anything I can do to fix this or is there something I am missing?
If I need to provide more information then please tell me.
Could you double-check the [Render mode] of your Canvas? Is it set as World Space? In this render mode, the Canvas will behave as any other 3D object and will render in front of or behind other objects in the scene based on 3D placement in the scene. You need to adjust the position of the canvas from the perspective of the camera.
However, according to your post, it seems that you want to place UI elements on the screen rendered on top of the scene. If so, You should turn the [Render mode] to [Screen Space - Camera]. More information please see: Canvas

GameObject does not rotate using TwoHandManipulatable nor BoundingBoxRig using Unity and HoloLens

I have a GameObject in a Unity project for HoloLens with HoloToolKit, which I need to rotate. The object has a BoxCollider and HandDraggable script, and I'm also adding a TapToPlace component in runtime. The scripts that I've tried for rotation are TwoHandManipulatable and BoundingBoxRig, and they work to resize the object, but no matter what I do, the object does not rotate.
This is how I tried with BoundingBoxRig:
And this is how it looks with TwoHandManipulatable:
None of these works, only for resize. I would appreciate the help, thanks!
For me all I had to to to get it to work was to add the Transform of your arrow2 to the box of the Two Hand Manipulatable. Just click and hold on Transform and drag it into the box next to Host Transform. I added it to my project and it looks like this:
Also I don't think you need the Hand Draggable Script if you have the Two Hand Manipulatable because the last checkbox in the Two Hand Manupulatable enables movement with one hand if you want it.
Edit: I have an Animator on it but you don't need that. I have it because I made the model in Blender and it got added automatically.

Three.js within web worker: Simulating animation without rendering to canvas

I have a hypothetical question:
Is it possible to simulate an animation of objects without rendering it to the canvas. I just want to capture objects' position using Vector.project(camera) and present it using CSS. And THREE.DeviceOrientationControls controls how the camera "view" the simulation.
I tried commenting THREE.WebGLRenderer, but it seems that THREE.PerpectiveCamera cannot update it's MatrixWorld property. Hence, the camera seems to not move and the Vector.project(camera) gives a static value. I do this because I need to put my three.js codes within a web worker.
Do I need still need to use THREE.WebGLRenderer to have a working simulation?
UPDATE:
I checked the following:
I digged deeper into ((three.scene.getObjectByName("one")).matrixWorld.getPosition()).project(three.camera);, I inspect the following values, having the above requirement (inside web worker, no renderer), using this example:
matrix: {"elements":{"0":3.2167603969573975,"1":0,"2":0,"3":0,"4":0,"5":2.1445069313049316,"6":0,"7":0,"8":0,"9":0,"10":-1.000100016593933,"11":-1,"12":5.4684929847717285,"13":2.1445069313049316,"14":-0.2000100016593933,"15":0}}
camera.projectionMatrix: {"elements":{"0":3.2167603969573975,"1":0,"2":0,"3":0,"4":0,"5":2.1445069313049316,"6":0,"7":0,"8":0,"9":0,"10":-1.000100016593933,"11":-1,"12":0,"13":0,"14":-0.2000100016593933,"15":0}}
camera.matrixWorld: {"elements":{"0":1,"1":0,"2":0,"3":0,"4":0,"5":1,"6":0,"7":0,"8":0,"9":0,"10":1,"11":0,"12":-1.7000000476837158,"13":-1,"14":0,"15":1}}
matrix.getInverse(camera.matrixWorld): {"elements":{"0":1,"1":0,"2":0,"3":0,"4":0,"5":1,"6":0,"7":0,"8":0,"9":0,"10":1,"11":0,"12":1.7000000476837158,"13":1,"14":0,"15":1}}
matrix.multiplyMatrices(camera.projectionMatrix, matrix.getInverse(camera.matrixWorld)): {"elements":{"0":3.2167603969573975,"1":0,"2":0,"3":0,"4":0,"5":2.1445069313049316,"6":0,"7":0,"8":0,"9":0,"10":-1.000100016593933,"11":-1,"12":5.4684929847717285,"13":2.1445069313049316,"14":-0.2000100016593933,"15":0}}
But, when normal (no modification), I inspect the following:
matrix: {"elements":{"0":3.2167603969573975,"1":0,"2":0,"3":0,"4":0,"5":2.1445069313049316,"6":0,"7":0,"8":0,"9":0,"10":-1.000100016593933,"11":-1,"12":5.4684929847717285,"13":2.1445069313049316,"14":-0.2000100016593933,"15":0}}
camera.projectionMatrix: {"elements":{"0":3.2167603969573975,"1":0,"2":0,"3":0,"4":0,"5":2.1445069313049316,"6":0,"7":0,"8":0,"9":0,"10":-1.000100016593933,"11":-1,"12":0,"13":0,"14":-0.2000100016593933,"15":0}}
camera.matrixWorld: {"elements":{"0":1,"1":0,"2":0,"3":0,"4":0,"5":-2.220446049250313e-16,"6":-1,"7":0,"8":0,"9":1,"10":-2.220446049250313e-16,"11":0,"12":-1.7000000476837158,"13":-1,"14":0,"15":1}}
matrix.getInverse(camera.matrixWorld): {"elements":{"0":1,"1":0,"2":0,"3":0,"4":0,"5":-2.220446049250313e-16,"6":1,"7":0,"8":0,"9":-1,"10":-2.220446049250313e-16,"11":0,"12":1.7000000476837158,"13":-2.220446049250313e-16,"14":1,"15":1}}
matrix.multiplyMatrices(camera.projectionMatrix, matrix.getInverse(camera.matrixWorld)): {"elements":{"0":3.2167603969573975,"1":0,"2":0,"3":0,"4":0,"5":-4.761761943205948e-16,"6":-1.000100016593933,"7":-1,"8":0,"9":-2.1445069313049316,"10":2.2206681307011713e-16,"11":2.220446049250313e-16,"12":5.4684929847717285,"13":-4.761761943205948e-16,"14":-1.2001099586486816,"15":-1}}
I noticed that the camera.matrixWorld property has significant difference in both condition. I do not understand what makes the difference.
Apparently, the following lines from THREE.WebGLRenderer.render are still needed to update camera.matrixWorld property:
scene.updateMatrixWorld();
camera.updateMatrixWorld();
camera.matrixWorldInverse.getInverse(vs._3.camera.matrixWorld);

Not able to run unity 3d app properly

I have just started to learn Unity 3D. I am using this as first lesson. I created a terrain, added texture, plants and grasses, added first person controller and directional lighting. When I press play, the scene automatically starts moving along Y direction. I want the scene to move only when i give input using keyboard or mouse. How to achieve this ?
The first person controller has to be placed above the terrain completely. After doing this I get the desired effect.

camera.lookAt not called when THREE controls are being used

I am working on a program, that uses THREE.RollControls, when the user goes too far away from the center of the screen, they tend to get lost, so I am working on creating a function that reorients them, facing the center of the scene.
What I had intened to do was simply call the following:
camera.lookAt(scene.position)
However, this has no affect. From what I was reading on different stack overflow questions specifically this:
ThreeJS camera.lookAt() has no effect, is there something I'm doing wrong?
It seems like their solution was to do the camera position change using the controls, rather then changing the camera itself.
I do not believe there is any 'Target' in the Roll Controls, so I don't know how I can reset where the camera is looking at based on a THREE.Vector3() Is there a simple way to do this, or will I basically have to:
So far I have 'attempted' to do the follow:
- Calculate the difference of position of the camera with the position of the scene.
- Normalize this vector
- Subtract it from the direction forward of the camera
- use this vector in controls.forward.add(thisVector)
but this doesn't do at all what I want (probably because I have no idea what I'm doing)
Thank you in advance for your time!
Isaac
The same thing bugged me too about the RollControls but I took a different approach in solving the problem. Since the controls are in the example code (in r55) you can modify the controls, as they are not part of the core library. You can see my modifications at http://www.virtuality.gr/AGG/EaZD-WebGL/js/three.js/examples/js/controls/RollControls.js
I introduced a local variable called mouseLook because I could not use the this.mouseLook. I initialized it to false and I only make it true when there is a button press i.e. when navigating in the scene. That solved my problem.

Resources