How to make reflective materials change when camera rotates - three.js

Question

I was able to make some nice metal and glass looking materials by using Skybox Cube / environment mapping.
I have made my own controls which allow one to both orbit and move/look around like in FirstPersonControls.
The problem is, the reflections look convincing when I move around - I can see the reflections move and change accordingly to my camera movement. However when I look around (rotate the camera / change it's target), there is no change in the reflections, they are just static.
I can see the same behaviour in for example three.js/examples/webgl_materials_cubemap_escher.html - if I modify it to use FirstPersonControls, the material does not look reflective/refractive at all when I look around.
Here's how I setup the cubemaps, to be honest it's copied from some example and I don't understand all of it. But it works, except for this one issue...
createSkyBox = function(urlPrefix) {
var sceneCube = new THREE.Scene();
var path = urlPrefix;
var format = '.jpg';
var urls = [
path + 'px' + format, path + 'nx' + format,
path + 'py' + format, path + 'ny' + format,
path + 'pz' + format, path + 'nz' + format
];
var reflectionCube = THREE.ImageUtils.loadTextureCube( urls );
reflectionCube.format = THREE.RGBFormat;
var refractionCube = new THREE.Texture( reflectionCube.image, new THREE.CubeRefractionMapping() );
refractionCube.format = THREE.RGBFormat;
// Skybox
var shader = THREE.ShaderUtils.lib[ "cube" ];
shader.uniforms[ "tCube" ].value = reflectionCube;
var material = new THREE.ShaderMaterial( {
fragmentShader: shader.fragmentShader,
vertexShader: shader.vertexShader,
uniforms: shader.uniforms,
depthWrite: false,
side: THREE.BackSide
} );
var size = 8000;
mesh = new THREE.Mesh( new THREE.CubeGeometry( size, size, size ), material );
mesh.geometry.computeBoundingBox();
sceneCube.add( mesh );
this._threejs_cube_scene = sceneCube;
this._threejs_cube_mesh = mesh;
this._threejs_envmap = reflectionCube;
this._threejs_envmap_refraction = refractionCube;
this._threejs_scene.add( sceneCube );
}
And here's the way I create the material:
var material = new THREE.MeshLambertMaterial( { color: 0xff00, ambient: 0xaaaaaa, envMap: this._threejs_envmap});
I then use the material in renderer.overrideMaterial (I'm using EffectComposer, if it makes any difference)
EDIT: now that I think about it, I'm not sure.. my brain melts.. it might be how the real life works :) At least intuitively when I see the code in action, the staticness while rotating camera doesn't feel right. But maybe it's because in real life it's hard to look around (eye.lookAt()) without also moving ever so slightly (eye.position = xyz).

Answer 1

you should calculate the reflection vector in world space (inside your code for 'fragmentShader' which you don't show here). If it's in object space, or view (camera) space, it won't move naturally.
Yes, this may mean some finagling with the surface normals. To convert object space normals to world space normals, use the inverse transpose of the world matrix. You'll also need to get the view vector in worldspace coordinates in order to calculate the final worldspace reflection vector.

Answer 2

Another thing to consider that's simpler than changing the shader may be giving your camera an offset if you want it to rotate like a human head. Add it to an Object3d and set it to be offset from the Object3d's position by a small amount (an amount equivalent to the distance from the human center to the eye) then rotate the Object3d instead of the camera.
It's sort-of hard to tell what effect you want though from your description, because when you simply turn your eyeballs, a reflection doesn't change. It's the slight tilt of your head that changes it.