Reflect vector in 3D space - algorithm

Question

A vector should be reflected when intersecting a mesh. When applying the following formula to reflect a vector, the result is set off. I am using toxiclibs in Processing.
// Get the normal of the face that is intersected.
ReadonlyVec3D n = isect.normal;
// calculate the reflected vector b
// a is the green point in the screenshot
b = a.sub(n.scale(2 * a.dot(n)));
b = b.add(b.getNormalized());
EDIT: When taking into account to create a directional vector by subtracting the last point before the intersection with the intersection, still the reflection is off.
Vec3D id = b.sub(isect.pos);
id.normalize();
b = n.scale(2 * id.dot(n)).sub(id);

Answer 1

I had the question a while back and found a few helpful resources:
Paul Bourke's solution for Line-Plane intersection
Vector reflection on 3D Kingdoms
Here's the snippet I used then:
import toxi.geom.Vec3D;
Vec3D[] face = new Vec3D[3];
float ai = TWO_PI/3;//angle increment
float r = 300;//overall radius
float ro = 150;//random offset
Vec3D n;//normal
Ray r1;
void setup() {
size(500, 500, P3D);
for (int i = 0 ; i < 3; i++) face[i] = new Vec3D(cos(ai * i) * r + random(ro), random(-50, 50), sin(ai * i) * r + random(ro));
r1 = new Ray(new Vec3D(-100, -200, -300), new Vec3D(100, 200, 300));
}
void draw() {
background(255);
lights();
translate(width/2, height/2, -500);
rotateX(map(mouseY, 0, height, -PI, PI));
rotateY(map(mouseX, 0, width, -PI, PI));
//draw plane
beginShape(TRIANGLES);
for (Vec3D p : face) vertex(p.x, p.y, p.z);
endShape();
//normals
Vec3D c = new Vec3D();//centroid
for (Vec3D p : face) c.addSelf(p);
c.scaleSelf(1.0/3.0);
Vec3D cb = face[2].sub(face[1]);
Vec3D ab = face[0].sub(face[1]);
n = cb.cross(ab);//compute normal
n.normalize();
line(c.x, c.y, c.z, n.x, n.y, n.z);//draw normal
pushStyle();
//http://paulbourke.net/geometry/planeline/
//line to plane intersection u = N dot ( P3 - P1 ) / N dot (P2 - P1), P = P1 + u (P2-P1), where P1,P2 are on the line and P3 is a point on the plane
Vec3D P2SubP1 = r1.end.sub(r1.start);
Vec3D P3SubP1 = face[0].sub(r1.start);
float u = n.dot(P3SubP1) / n.dot(P2SubP1);
Vec3D P = r1.start.add(P2SubP1.scaleSelf(u));
strokeWeight(5);
point(P.x, P.y, P.z);//point of ray-plane intersection
//vector reflecting http://www.3dkingdoms.com/weekly/weekly.php?a=2
//R = 2*(V dot N)*N - V
//Vnew = -2*(V dot N)*N + V
//PVector V = PVector.sub(r1.start,r1.end);
Vec3D V = r1.start.sub(P);
Vec3D R = n.scaleSelf(2 * (V.dot(n))).sub(V);
strokeWeight(1);
stroke(0, 192, 0);
line(P.x, P.y, P.z, R.x, R.y, R.z);
stroke(192, 0, 0);
line(r1.start.x, r1.start.y, r1.start.z, P.x, P.y, P.z);
stroke(0, 0, 192);
line(P.x, P.y, P.z, r1.end.x, r1.end.y, r1.end.z);
popStyle();
}
void keyPressed() {
setup();
}//reset
class Ray {
Vec3D start = new Vec3D(), end = new Vec3D();
Ray(Vec3D s, Vec3D e) {
start = s ;
end = e;
}
}
Note that this is a basic proof of concept.
Toxiclibs may already provide Ray/Face classes.

Answer 2

Assuming you have the incident direction id and the normal at the point of intersection n, then the reflection rd is
rd = 2 * dot(n,id) * n - id
where all the vectors are normalized.
In your case, if b is the green point, and isect is the point of intersection then id = b - isect normalized.
So the reflection ray r (assuming it has an origin and a direction) is
r.direction = rd
r.origin = isect
You can also look at this Wikipedia article. https://en.wikipedia.org/wiki/Specular_reflection.