Three.js custom Shader with Texture - opengl-es

Question

I want to write a custom shader which manipulates my image with three.js.
For that I want to create a plane with the image as a texture. Afterwards I want to move vertices around to distort the image.
(If that an absolute wrong way to do this, please tell me).
First I have my shaders:
<script type="x-shader/x-vertex" id="vertexshader">
attribute vec2 a_texCoord;
varying vec2 v_texCoord;
void main() {
// Pass the texcoord to the fragment shader.
v_texCoord = a_texCoord;
gl_Position = projectionMatrix *
modelViewMatrix *
vec4(position,1.0);
}
</script>
<script type="x-shader/x-fragment" id="fragmentshader">
uniform sampler2D u_texture;
varying vec2 v_texCoord;
void main() {
vec4 color = texture2D(u_texture, v_texCoord);
gl_FragColor = color;
}
</script>
Where I don't really understand what the texture2D is doing, but I found that in other code fragments.
What I want with this sample: Just color the vertex (gl_FracColor) with the color from the «underlying» image (=texture).
In my code I have setup a normal three scene with a plane:
// set some camera attributes
var VIEW_ANGLE = 45,
ASPECT = window.innerWidth/window.innerHeight,
NEAR = 0.1,
FAR = 1000;
var scene = new THREE.Scene();
var camera = new THREE.PerspectiveCamera(VIEW_ANGLE, ASPECT, NEAR, FAR);
camera.position.set(0, 0, 15);
var vertShader = document.getElementById('vertexshader').innerHTML;
var fragShader = document.getElementById('fragmentshader').innerHTML;
var texloader = new THREE.TextureLoader();
var texture = texloader.load("img/color.jpeg");
var uniforms = {
u_texture: {type: 't', value: 0, texture: texture},
};
var attributes = {
a_texCoord: {type: 'v2', value: new THREE.Vector2()}
};
// create the final material
var shaderMaterial = new THREE.ShaderMaterial({
uniforms: uniforms,
vertexShader: vertShader,
fragmentShader: fragShader
});
var renderer = new THREE.WebGLRenderer();
renderer.setSize( window.innerWidth, window.innerHeight );
document.body.appendChild(renderer.domElement);
var plane = {
width: 5,
height: 5,
widthSegments: 10,
heightSegments: 15
}
var geometry = new THREE.PlaneBufferGeometry(plane.width, plane.height, plane.widthSegments, plane.heightSegments)
var material = new THREE.MeshBasicMaterial( { color: 0x00ff00 } );
var plane = new THREE.Mesh( geometry, shaderMaterial );
scene.add(plane);
plane.rotation.y += 0.2;
var render = function () {
requestAnimationFrame(render);
// plane.rotation.x += 0.1;
renderer.render(scene, camera);
};
render();
Unfortunately, after running that code I just see a black window. Although I know that if I use the material as material when creating the mesh, I can see it clearly.
So it must be the shaderMaterial or the shaders.
Questions:
do I have to define the uniform u_texture and the attribute
a_texCoord in my shader Material uniforms and attributes? And do
they have to have the exact same name?
How many vertices are there anyway? Will I get a vertices for every pixel in the image? Or is it just 4 for each corner of the plane?
What value does a_texCoord have? Nothing happens if I write:
var attributes = {
a_texCoord: {type: 'v2', value: new THREE.Vector2(1,1)}
};
Or do I have to use some mapping (built in map stuff from three)? But how would I then change vertex positions?
Could someone shed some light on that matter?

Answer 1

I got it to work by changing this:
var uniforms = {
u_texture: {type: 't', value: 0, texture: texture},
};
To this:
var uniforms = {
u_texture: {type: 't', value: texture},
};
Anyway all other questions are still open and answers highly appreciated.
(btw: why the downgrade of someone?)

Answer 2

do I have to define the uniform u_texture and the attribute a_texCoord
in my shader Material uniforms and attributes? And do they have to
have the exact same name?
Yes and yes. The uniforms are defined as part of the shader-material while the attributes haven been moved from shader-material to the BufferGeometry-class in version 72 (i'm assuming you are using an up to date version, so here is how you do this today):
var geometry = new THREE.PlaneBufferGeometry(...);
// first, create an array to hold the a_texCoord-values per vertex
var numVertices = (plane.widthSegments + 1) * (plane.heightSegments + 1);
var texCoordBuffer = new Float32Array(2 * numVertices);
// now register it as a new attribute (the 2 here indicates that there are
// two values per element (vec2))
geometry.addAttribute('a_texCoord', new THREE.BufferAttribute(texCoordBuffer, 2));
As you can see, the attribute will only work if it has the exact same name as specified in your shader-code.
I don't know exactly what you are planning to use this for, but it sounds suspiciously like you want to have the uv-coordinates. If that is the case, you can save yourself a lot of work if you have a look at the THREE.PlaneBufferGeometry-class. It already provides an attribute named uv that is probably exactly what you are looking for. So you just need to change the attribute-name in your shader-code to
attribute vec2 uv;
How many vertices are there anyway? Will I get a vertices for every
pixel in the image? Or is it just 4 for each corner of the plane?
The vertices are created according to the heightSegments and widthSegments parameters. So if you set both to 5, there will be (5 + 1) * (5 + 1) = 36 vertices (+1 because a line with only 1 segment has two vertices etc.) and 5 * 5 * 2 = 50 triangles (with 150 indices) in total.
Another thing to note is that the PlaneBufferGeometry is an indexed geometry. This means that every vertex (and every other attribute-value) is stored only once, although it is used by multiple triangles. There is a special index-attribute that contains the information which vertices are used to create which triangles.
What value does a_texCoord have? Nothing happens if I write: ...
I hope the above helps to answer that.
Or do I have to use some mapping (built in map stuff from three)?
I would suggest you use the uv attribute as described above. But you absolutely don't have to.
But how would I then change vertex positions?
There are at least two ways to do this: in the vertex-shader or via javascript. The latter can be seen here: http://codepen.io/usefulthink/pen/vKzRKr?editors=1010
(the relevant part for updating the geometry starts in line 84).