I am fitting a plane to a 3D point set with the least square method. I already have algorithm to do that, but I want to modify it to use weighted least square. Meaning I have a weight for each point (the bigger weight, the closer the plane should be to the point).
The current algorithm (without weight) looks like this:
Compute the sum:
for(Point3D p3d : pointCloud) {
pos = p3d.getPosition();
fSumX += pos[0];
fSumY += pos[1];
fSumZ += pos[2];
fSumXX += pos[0]*pos[0];
fSumXY += pos[0]*pos[1];
fSumXZ += pos[0]*pos[2];
fSumYY += pos[1]*pos[1];
fSumYZ += pos[1]*pos[2];
}
than make the matrices:
double[][] A = {
{fSumXX, fSumXY, fSumX},
{fSumXY, fSumYY, fSumY},
{fSumX, fSumY, pointCloud.size()}
};
double[][] B = {
{fSumXZ},
{fSumYZ},
{fSumZ}
};
than solve Ax = B and the 3 components of the solution are the coefficients of the fitted plain...
So, can you please help me how to modify this to use weights? Thanks!
Intuition
A point x on a plane defined by normal n and a point on the plane p obeys: n.(x - p) = 0. If a point y does not lie on the plane, n.(y -p) will not be equal to zero, so a useful way to define a cost is by |n.(y - p)|^2 . This is the squared distance of the point y from the plane.
With equal weights, you want to find an n that minimizes the total squared error when summing over the points:
f(n) = sum_i | n.(x_i - p) |^2
Now this assumes we know some point p that lies on the plane. We can easily compute one as the centroid, which is simply the component-wise mean of the points in the point cloud and will always lie in the least-squares plane.
Solution
Let's define a matrix M where each row is the ith point x_i minus the centroid c, we can re-write:
f(n) = | M n |^2
You should be able to convince yourself that this matrix multiplication version is the same as the sum on the previous equation.
You can then take singular value decomposition of M, and the n you want is then given by the right singular vector of M that corresponds to the smallest singular value.
To incorporate weights you simply need to define a weight w_i for each point. Calculate c as the weighted average of the points, and change sum_i | n.(x_i - c) |^2 to sum_i | w_i * n.(x_i - c) |^2, and the matrix M in a similar way. Then solve as before.
Multiply each term in each sum by the corresponding weight. For example:
fSumZ += weight * pos[2];
fSumXX += weight * pos[0]*pos[0];
Since pointCloude.size() is the sum of 1 for all points, it should be replaced with the sum of all weights.
Start from redefining the least-square error calculation. The formula tries to minimize the sum of squares of errors. Multiply the squared error by a function of two points which decreases with their distance. Then try to minimize the weighted sum of squared errors and derive the coefficients from that.
Related
I am working on a genetic algorithm. Here is how it works :
Input : a list of 2D points
Input : the degree of the curve
Output : the equation of the curve that passes through points the best way (try to minimize the sum of vertical distances from point's Ys to the curve)
The algorithm finds good equations for simple straight lines and for 2-degree equations.
But for 4 points and 3 degree equations and more, it gets more complicated. I cannot find the right combination of parameters : sometimes I have to wait 5 minutes and the curve found is still very bad. I tried modifying many parameters, from population size to number of parents selected...
Do famous combinations/theorems in GA programming can help me ?
Thank you ! :)
Based on what is given, you would need a polynomial interpolation in which, the degree of the equation is number of points minus 1.
n = (Number of points) - 1
Now having said that, let's assume you have 5 points that need to be fitted and I am going to define them in a variable:
var points = [[0,0], [2,3], [4,-1], [5,7], [6,9]]
Please be noted the array of the points have been ordered by the x values which you need to do.
Then the equation would be:
f(x) = a1*x^4 + a2*x^3 + a3*x^2 + a4*x + a5
Now based on definition (https://en.wikipedia.org/wiki/Polynomial_interpolation#Constructing_the_interpolation_polynomial), the coefficients are computed like this:
Now you need to used the referenced page to come up with the coefficient.
It is not that complicated, for the polynomial interpolation of degree n you get the following equation:
p(x) = c0 + c1 * x + c2 * x^2 + ... + cn * x^n = y
This means we need n + 1 genes for the coefficients c0 to cn.
The fitness function is the sum of all squared distances from the points to the curve, below is the formula for the squared distance. Like this a smaller value is obviously better, if you don't want that you can take the inverse (1 / sum of squared distances):
d_squared(xi, yi) = (yi - p(xi))^2
I think for faster conversion you could limit the mutation, e.g. when mutating choose a new value with 20% probability between min and max (e.g. -1000 and 1000) and with 80% probabilty a random factor between 0.8 and 1.2 with which you multiply the old value.
We need to transform a set of integers A to another set of integers B such that the sum of squares of the elements of B is equal to a certain given value M.
Since there can be multiple such transformations, we need to find the one in which the sum of the square of the difference between the corresponding elements of A and B is minimum.
Input:
A set of non-negative integers A = {a1, a2, a3 ... an}
A non-negative integer M
Output:
A set of numbers B = {b1, b2, b3 ... bn}, such that:
sumi=1->n[ bi ^ 2 ] = M
sumi=1->n[ (ai-bi) ^ 2 ] = S is minimized.
The minimum sum S.
A bit of math.
Sum (ai - bi)2 = Sum (ai2 - 2 aibi + bi2) = Sum ai2 - 2 Sum aibi + Sum bi2
The first term is constant; the last one is M (also constant), so you are seeking to maximize
Sum aibi
with the restriction Sum bi2 = M.
In other words, you need a hyperplane normal to a vector A = { ai }, tangent to a hypersphere with a radius sqrt(M). Such hyperplane passes through a point where the normal line intersects with the sphere. This point is fA with |fA| = sqrt(M):
f = sqrt(M)/sqrt(Sum ai2)
The solution to your problem is
bi = ai * sqrt(M)/sqrt(Sum ai2)
EDIT: The answers so far, including the one below, map A to a set of real numbers instead of integers. As far as I can tell there is no general fix for this because there are many values of M for which there is no integer vector satisfying the constraint. Ex: M = 2. There is no vector of integers the sum of whose squares is 2. Even if M is a sum of squares, it is a sum of a certain number of squares, so even M = 4 has no solution if A has 3 or more non-zero components. As such, there is no general mapping that satisfies the problem as stated.
Here is the version that allows B to be a vector of reals:
The answer by #user58697 is quite elegant. Here is a restatement that is, perhaps, more intuitive for those of us less used to thinking with hyper geometry:
Treat A and B as vectors. Then start the same way: sum(ai - bi)2 = sum(ai2) - 2sum(aibi) + sum(bi2)
The first term is the magnitude of the vector A squared just as the last term is the magnitude of vector B squared. Both are constant so only the middle term can change. That means we want to maximize sum(aibi) which is exactly the dot product of A and B (https://en.wikipedia.org/wiki/Dot_product). The dot product of two vectors is maximized when the angle between them is 0, which is to say when they are co-directional (that is they point in the same direction).
This means that the unit vector forms of A and B must be the same. That is:
ai/|A| = bi/|B|. Solve this for bi: bi = ai * |B| / |A|
But |B| is just sqrt(M) and A is sqrt(sum(ai2)). So, just like in user58697's version:
bi = ai * sqrt(M) / sqrt(sum(ai2))
I am working on a little puzzle-game-project. The basic idea is built around projecting multi-dimensonal data down to 2D. My only problem is how to generate the randomized scenario data. Here is the problem:
I got muliple randomized vectors v_i and a target vector t, all 2D. Now I want to randomize scalar values c_i that:
t = sum c_i v_i
Because there are more than two v_i this is a overdetermined system. I also took care that the linear combination of v_i is actual able to reach t.
How can I create (randomized) values for my c_i?
Edit: After finding this Question I can additionally state, that it is possible for me also (slightly) change the v_i.
All values are based on double
Let's say your v_i form a matrix V with 2 rows and n columns, each vector is a column. The coefficients c_i form a column vector c. Then the equation can be written in matrix form as
V×c = t
Now apply a Singular Value Decomposition to matrix V:
V = A×D×B
with A being an orthogonal 2×2 matrix, D is a 2×n matrix and B an orthogonal n×n matrix. The original equation now becomes
A×D×B×c = t
multiply this equation with the inverse of A, the inverse is the same as the transposed matrix AT:
D×B×c = AT×t
Let's introduce new symbols c'=B×c and t'=AT×t:
D×c' = t'
The solution of this equation is simple, because Matrix D looks like this:
u 0 0 0 ... // n columns
0 v 0 0 ...
The solution is
c1' = t1' / u
c2' = t2' / v
And because all the other columns of D are zero, the remaining components c3'...cn' can be chosen freely. This is the place where you can create random numbers for c3'...cn. Having vector c' you can calculate c as
c = BT×c'
with BT being the inverse/transposed of B.
Since the v_i are linearly dependent there are non trivial solutions to 0 = sum l_i v_i.
If you have n vectors you can find n-2 independent such solutions.
If you have now one solution to t = sum c_i v_i you can add any multiple of l_i to c_i and you will still have a solution: c_i' = p l_i + c_i.
For each independent solution of the homogenous problem determine a random p_j and calculate
c_i'' = c_i + sum p_j l_i_j.
For 3 points in 2D :
P1(x1,y1),
P2(x2,y2),
P3(x3,y3)
I need to find a point P(x,y), such that the maximum of the manhattan distances
max(dist(P,P1),
dist(P,P2),
dist(P,P3))
will be minimal.
Any ideas about the algorithm?
I would really prefer an exact algorithm.
There is an exact, noniterative algorithm for the problem; as Knoothe pointed out, the Manhattan distance is rotationally equivalent to the Chebyshev distance, and P is trivially computable for the Chebyshev distance as the mean of the extreme coordinates.
The points reachable from P within the Manhattan distance x form a diamond around P. Therefore, we need to find the minimum diamond that encloses all points, and its center will be P.
If we rotate the coordinate system by 45 degrees, the diamond is a square. Therefore, the problem can be reduced to finding the smallest enclosing square of the points.
The center of a smallest enclosing square can be found as the center of the smallest enclosing rectangle (which is trivially computed as the max and min of the coordinates). There is an infinite number of smallest enclosing squares, since you can shift the center along the shorter edge of the minimum rectangle and still have a minimal enclosing square. For our purposes, we can simply use the one whose center coincides with the enclosing rectangle.
So, in algorithmic form:
Rotate and scale the coordinate system by assigning x' = x/sqrt(2) - y/sqrt(2), y' = x/sqrt(2) + y/sqrt(2)
Compute x'_c = (max(x'_i) + min(x'_i))/2, y'_c = (max(y'_i) + min(y'_i))/2
Rotate back with x_c = x'_c/sqrt(2) + y'_c/sqrt(2), y_c = - x'_c/sqrt(2) + y'_c/sqrt(2)
Then x_c and y_c give the coordinates of P.
If an approximate solution is okay, you could try a simple optimization algorithm. Here's an example, in Python
import random
def opt(*points):
best, dist = (0, 0), 99999999
for i in range(10000):
new = best[0] + random.gauss(0, .5), best[1] + random.gauss(0, .5)
dist_new = max(abs(new[0] - qx) + abs(new[1] - qy) for qx, qy in points)
if dist_new < dist:
best, dist = new, dist_new
print new, dist_new
return best, dist
Explanation: We start with the point (0, 0), or any other random point, and modify it a few thousand times, each time keeping the better of the new and the previously best point. Gradually, this will approximate the optimum.
Note that simply picking the mean or median of the three points, or solving for x and y independently does not work when minimizing the maximum manhattan distance. Counter-example: Consider the points (0,0), (0,20) and (10,10), or (0,0), (0,1) and (0,100). If we pick the mean of the most separated points, this would yield (10,5) for the first example, and if we take the median this would be (0,1) for the second example, which both have a higher maximum manhattan distance than the optimum.
Update: Looks like solving for x and y independently and taking the mean of the most distant points does in fact work, provided that one does some pre- and postprocessing, as pointed out by thiton.
I have a number of 2D line segments that should all intersect at one point but don't because of noise earlier in the calculations that cannot be reduced.
Is there an algorithm to compute the best approximation to the intersection of all the line segments. Something like the point with the minimum average distance to all the line segments that doesn't necessarily lie on any of the segments?
The first comment from Amit is your answer. I'll explain why.
Let p_i be your points of intersection and c = 1/n sum(p_i). Let's show that c minimizes the average distance, d(a) between the p_i and an arbitrary point a:
d(a) = 1/n sum( |a-p_i|^2 )
What is being averaged in d(a) is, using inner product notation,
|a-p_i|^2 = <a-p_i, a-p_i> = |a|^2 + |p_i|^2 - 2<a,p_i>`
The average of <a,p_i> is just <a,c>, using the bilinear properties of dot product. So,
d(a) = |a|^2 - 2<a,c> + 1/n sum( |p_i|^2 )
And so likewise
d(c) = |c|^2 - 2<c,c> + 1/n sum( |p_i|^2 ) = -|c|^2 + 1/n sum( |p_i|^2 )
Subtracting the two
d(a) - d(c) = |a|^2 - 2<a,c> + |c|^2 = |a-c|^2
So, adding d(c) to both sides, the average distance to an arbitrary point a is
d(a) = d(c) + |a-c|^2
which since all terms are positive is minimized when |a-c|^2 is zero, in other words, when a = c.
If we have freedom to select a metric, sum of squared distances may give a simple algorithm.
We can represent square of distance to a line #i as function of point coordinates, we will get (A[i]x,x)+(b[i],x)+c[i], A[i] is a matrix 3x3, b[i] - vector, c[i] - number, (a,b) - scalar multiplication.
Their sum will be (A[sum]x,x)+(b[sum],x)+c[sum].
Minimum of such function is x=-inverse(A[sum])b[sum]/2.