How to find the inverse of a matrix? I am trying to use the Gauss elimination method. I know how to solve it by hand, but unable to understand how to code.
Guass-Jordan elimination is explained clearly here: http://www.codewithc.com/c-program-for-gauss-jordan-method/
Also here is a C++ method implementation which is more aligned to finding the inverse of the matrix: http://www.sanfoundry.com/cpp-program-implement-gauss-jordan-elimination/
Note, please attempt to understand the reasoning behind the method. If I were learning this topic, I may try to write the code from the description myself first, then only look at the coded solution if I got stuck.
Also, there are likely other implementations in other languages - if you simply do a meaningful search on Google.
Good luck!
This should have been answered a billion time but ok. First of all, I don't think the Gauss-Jordan method is the best (for performances). I assume the matrix is of fixed size (3x3) in column notation. The following code is Javascript one but easily transposable to any othe language.
Matrix.prototype.inverse = function() {
var c, l, det, ret = new Matrix();
ret._M[0][0] = (this._M[1][1] * this._M[2][2] - this._M[2][1] * this._M[1][2]);
ret._M[0][1] = -(this._M[0][1] * this._M[2][2] - this._M[2][1] * this._M[0][2]);
ret._M[0][2] = (this._M[0][1] * this._M[1][2] - this._M[1][1] * this._M[0][2]);
ret._M[1][0] = -(this._M[1][0] * this._M[2][2] - this._M[2][0] * this._M[1][2]);
ret._M[1][1] = (this._M[0][0] * this._M[2][2] - this._M[2][0] * this._M[0][2]);
ret._M[1][2] = -(this._M[0][0] * this._M[1][2] - this._M[1][0] * this._M[0][2]);
ret._M[2][0] = (this._M[1][0] * this._M[2][1] - this._M[2][0] * this._M[1][1]);
ret._M[2][1] = -(this._M[0][0] * this._M[2][1] - this._M[2][0] * this._M[0][1]);
ret._M[2][2] = (this._M[0][0] * this._M[1][1] - this._M[1][0] * this._M[0][1]);
det = this._M[0][0] * ret._M[0][0] + this._M[0][1] * ret._M[1][0] + this._M[0][2] * ret._M[2][0];
for (c = 0; c < 3; c++) {
for (l = 0; l < 3; l++) {
ret._M[c][l] = ret._M[c][l] / det;
}
}
this._M = ret._M;
};
Related
I am trying to numerically solve the Klein-Gordon equation that can be found here. To make sure I solved it correctly, I am comparing it with an analytical solution that can be found on the same link. I am using the finite difference method and Matlab. The initial spatial conditions are known, not the initial time conditions.
I start off by initializing the constants and the space-time coordinate system:
close all
clear
clc
%% Constant parameters
A = 2;
B = 3;
lambda = 2;
mu = 3;
a = 4;
b = - (lambda^2 / a^2) + mu^2;
%% Coordinate system
number_of_discrete_time_steps = 300;
t = linspace(0, 2, number_of_discrete_time_steps);
dt = t(2) - t(1);
number_of_discrete_space_steps = 100;
x = transpose( linspace(0, 1, number_of_discrete_space_steps) );
dx = x(2) - x(1);
Next, I define and plot the analitical solution:
%% Analitical solution
Wa = cos(lambda * x) * ( A * cos(mu * t) + B * sin(mu * t) );
figure('Name', 'Analitical solution');
surface(t, x, Wa, 'edgecolor', 'none');
colormap(jet(256));
colorbar;
xlabel('t');
ylabel('x');
title('Wa(x, t) - analitical solution');
The plot of the analytical solution is shown here.
In the end, I define the initial spatial conditions, execute the finite difference method algorithm and plot the solution:
%% Numerical solution
Wn = zeros(number_of_discrete_space_steps, number_of_discrete_time_steps);
Wn(1, :) = Wa(1, :);
Wn(2, :) = Wa(2, :);
for j = 2 : (number_of_discrete_time_steps - 1)
for i = 2 : (number_of_discrete_space_steps - 1)
Wn(i + 1, j) = dx^2 / a^2 ...
* ( ( Wn(i, j + 1) - 2 * Wn(i, j) + Wn(i, j - 1) ) / dt^2 + b * Wn(i - 1, j - 1) ) ...
+ 2 * Wn(i, j) - Wn(i - 1, j);
end
end
figure('Name', 'Numerical solution');
surface(t, x, Wn, 'edgecolor', 'none');
colormap(jet(256));
colorbar;
xlabel('t');
ylabel('x');
title('Wn(x, t) - numerical solution');
The plot of the numerical solution is shown here.
The two plotted graphs are not the same, which is proof that I did something wrong in the algorithm. The problem is, I can't find the errors. Please help me find them.
To summarize, please help me change the code so that the two plotted graphs become approximately the same. Thank you for your time.
The finite difference discretization of w_tt = a^2 * w_xx - b*w is
( w(i,j+1) - 2*w(i,j) + w(i,j-1) ) / dt^2
= a^2 * ( w(i+1,j) - 2*w(i,j) + w(i-1,j) ) / dx^2 - b*w(i,j)
In your order this gives the recursion equation
w(i,j+1) = dt^2 * ( (a/dx)^2 * ( w(i+1,j) - 2*w(i,j) + w(i-1,j) ) - b*w(i,j) )
+2*w(i,j) - w(i,j-1)
The stability condition is that at least a*dt/dx < 1. For the present parameters this is not satisfied, they give this ratio as 2.6. Increasing the time discretization to 1000 points is sufficient.
Next up is the boundary conditions. Besides the two leading columns for times 0 and dt one also needs to set the values at the boundaries for x=0 and x=1. Copy also them from the exact solution.
Wn(:,1:2) = Wa(:,1:2);
Wn(1,:)=Wa(1,:);
Wn(end,:)=Wa(end,:);
Then also correct the definition (and use) of b to that in the source
b = - (lambda^2 * a^2) + mu^2;
and the resulting numerical image looks identical to the analytical image in the color plot. The difference plot confirms the closeness
I need to decrease the runtime of the following piece of code written in Matlab :
dt = 0.001; dt05 = dt^0.5; length_t = 1.0e6;
%a: array containing length_t elements
y0 = [1.5 2.0 1.0];y = zeros(length_t,3);y(1,:) = y0;
for i = 1:length_t-1
dy = f(y(i,:); %call to some function
y(i+1,1) = y(i,1) + dt*dy(1) ;
y(i+1,2) = y(1,2) + a(1:i)*(y(i:-1:1,2)-y(1,2)) + dt05*dy(2) ;
y(i+1,3) = y(1,3) + a(1:i)*(y(i:-1:1,3)-y(1,3)) + dt05*dy(3) ;
end
The slowest steps are the calculations of y(i+1,2) and y(i+1,3) (because they require all the previous y(:,2:3) values). How can I speed up this code by vectorization and/or using a GPU?
EDIT: a is given by
a(1) = 0.5; a (2:length_t) = cumprod( (1-((1+a(1))./(2:length_t))) )*a(1);
and f is some function like:
function dy = f(y)
k12 = 1.0; k02 = 2.0;
dy(1) = - k12*y(1)*y(2);
dy(2) = k12*y(1) - k02*y(2);
dy(3) = (k12+k02)*(y(1)+y(2)+y(3));
dy = [dy(1) dy(2) dy(3)];
end
Note that I do NOT have DSP knowledge. I hope someone can write a better answer or correct mine.
If you can tolerate some approximations:
You can see that ratio a(i+1)/a(i) tends towards 1. This means that you can calculate a*y exactly for the first N elements (N depending on your desired accuracy), then add N+1-th element to variable AY and decrease variable AY by a magic factor depending on i. That way you can save yourself a lot of multiplications at the cost of this AY being somewhat inaccurate estimate of the actual product.
Your y(i,2) would then be somewhat like (csa = cumsum(a);):
y(i,2) = a(1:N) * y(i:-1:i-N) + AY + dt05_thingy + (1-csa(i))*y(1,2);
y(i,3) = ...
AY = AY*MF(i,N) + a(N)*y(i-N);
Magic factor would depend on N and perhaps also i. Precalculate R=a(2:end)./a(1:end-1); and use MF(N, i>N) = R(N+(i-N)/2) - so take the middle ratio for the elements you are approximating.
With d3.js we can achieve eased time out of normalized time t, typically in the range [0,1]
For example:
d3.easeCubic(0.25) = 0.0625
How can we reverse that, how can we find x given known y ?
d3.easeCubic(X) = 0.0625,
X ???
The answer here is cubic root, but still.
The problem is in reusability, ease function can change to d3.easeExpIn, or `d3.easeCircleOut, or any other, do you need to invent reverse functions on your own, or are they hidden anywhere ?
Firstly, your math is wrong. d3.easeCubic(0.25) will give you 0.0625:
var easy = d3.easeCubic(0.25);
console.log(easy);
<script src="https://d3js.org/d3.v4.min.js"></script>
Now, back to your question:
How can we reverse that, how can we find x given known y?
There is no native solution, but we can create our own function to find X given a known Y. The problem, of course, is that we have to invert the math for each specific easing... But, since you asked about d3.easeCubic, which is the same of d3.easeCubicInOut, let's try to create an inverted function for that particular easing.
First step, let's have a look at the source code:
export function cubicInOut(t) {
return ((t *= 2) <= 1 ? t * t * t : (t -= 2) * t * t + 2) / 2;
}
You can easily see that this is the correct function, giving us the same value as the first snippet:
function cubicInOut(t) {
return ((t *= 2) <= 1 ? t * t * t : (t -= 2) * t * t + 2) / 2;
}
console.log(cubicInOut(0.25))
Now, let's try to invert it.
The math here is somehow complicated, but for values less than 1, here is the function:
function inverseEaseCubic(t){
return Math.cbrt(t * 2) / 2;
}
And here is the demo. We pass 0.0625 to the function, and it returns 0.25:
function inverseEaseCubic(t){
return Math.cbrt(t * 2) / 2;
}
console.log(inverseEaseCubic(0.0625))
If you want to deal with numbers bigger than 1, this is the complete function:
function InverseEaseCubic(t){
return t <= 1 ? Math.cbrt(t * 2) / 2 : (Math.cbrt(2 * t - 2) + 2) / 2;
}
PS: In his comment, #altocumulus just reminded us that, sometimes, it's even impossible to find the value. Here is a very simple example. Suppose this function:
function exponentiation(a){
return a*a;
}
Now imagine that, when called with an unknown argument, the function returned 4. What's the argument? Can we find out? Impossible to determine, because second degree equations, like this one, have 2 roots:
console.log(exponentiation(2))//returns 4
console.log(exponentiation(-2))//also returns 4
I used the #Gerardo Furtado answer but the inverse function didn't work well so I wrote another
function cubicInOut(t) {
return ((t *= 2) <= 1 ? t * t * t : (t -= 2) * t * t + 2) / 2;
}
function inverseEaseCubic(x) {
return x < .5 ? Math.cbrt(x / 4) : (2 - Math.cbrt(2 - 2 * x)) / 2;
}
console.log(inverseEaseCubic(cubicInOut(1)) === 1);
console.log(inverseEaseCubic(cubicInOut(0.6)) === 0.6);
console.log(inverseEaseCubic(cubicInOut(0.4)) === 0.4);
console.log(inverseEaseCubic(cubicInOut(0.1)) === 0.1);
console.log(inverseEaseCubic(cubicInOut(0)) === 0);
I've implemented following Batch Gradient descednt algorithm, based on various sources I was able to find around web and in lecture notes.
This implementation isn't ideal in terms of stopping criteria, but for my sample it should work.
Inputs:
x = [1,1;1,2;1,3;1,4;1,5];
y = [1;2;3;4;5];
theta = [0;0];
Code:
tempTheta = [0;0];
for c = 1:10000,
for j = 1:2,
sum = 0;
for i = 1:5,
sum = sum + ((dot(theta', x(i, :)) - y(j)) * x(i,j));
end
sum = (sum / 5) * 0.01;
tempTheta(j) = theta(j) - sum;
end
theta = tempTheta;
end
The expected result is theta = [0;1], but my implementation always returns theta = [-3.5, 1.5].
I've tried various combinations of alpha and starting point, but without luck. Where am I making mistake?
In this line
sum = sum + ((dot(theta', x(i, :)) - y(j)) * x(i,j));
you are using a wrong index of y, it should be y(i), as j is a dimension iterator, not the sample iterator.
After the change
theta =
-1.5168e-07
1.0000e+00
I am trying to implement a two-layer perceptron with backpropagation to solve the parity problem. The network has 4 binary inputs, 4 hidden units in the first layer and 1 output in the second layer. I am using this for reference, but am having problems with convergence.
First, I will note that I am using a sigmoid function for activation, and so the derivative is (from what I understand) the sigmoid(v) * (1 - sigmoid(v)). So, that is used when calculating the delta value.
So, basically I set up the network and run for just a few epochs (go through each possible pattern -- in this case, 16 patterns of input). After the first epoch, the weights are changed slightly. After the second, the weights do not change and remain so no matter how many more epochs I run. I am using a learning rate of 0.1 and a bias of +1 for now.
The process of training the network is below in pseudocode (which I believe to be correct according to sources I've checked):
Feed Forward Step:
v = SUM[weight connecting input to hidden * input value] + bias
y = Sigmoid(v)
set hidden.values to y
v = SUM[weight connecting hidden to output * hidden value] + bias
y = Sigmoid(v)
set output value to y
Backpropagation of Output Layer:
error = desired - output.value
outputDelta = error * output.value * (1 - output.value)
Backpropagation of Hidden Layer:
for each hidden neuron h:
error = outputDelta * weight connecting h to output
hiddenDelta[i] = error * h.value * (1 - h.value)
Update Weights:
for each hidden neuron h connected to the output layer
h.weight connecting h to output = learningRate * outputDelta * h.value
for each input neuron x connected to the hidden layer
x.weight connecting x to h[i] = learningRate * hiddenDelta[i] * x.value
This process is of course looped through the epochs and the weight changes persist. So, my question is, are there any reasons that the weights remain constant after the second epoch? If necessary I can post my code, but at the moment I am hoping for something obvious that I'm overlooking. Thanks all!
EDIT: Here are the links to my code as suggested by sarnold:
MLP.java: http://codetidy.com/1903
Neuron.java: http://codetidy.com/1904
Pattern.java: http://codetidy.com/1905
input.txt: http://codetidy.com/1906
I think I spotted the problem; funny enough, what I found is visible in your high-level description, but I only found what looked odd in the code. First, the description:
for each hidden neuron h connected to the output layer
h.weight connecting h to output = learningRate * outputDelta * h.value
for each input neuron x connected to the hidden layer
x.weight connecting x to h[i] = learningRate * hiddenDelta[i] * x.value
I believe the h.weight should be updated with respect to the previous weight. Your update mechanism sets it based only on the learning rate, the output delta, and the value of the node. Similarly, the x.weight is also being set based on the learning rate, the hidden delta, and the value of the node:
/*** Weight updates ***/
// update weights connecting hidden neurons to output layer
for (i = 0; i < output.size(); i++) {
for (Neuron h : output.get(i).left) {
h.weights[i] = learningRate * outputDelta[i] * h.value;
}
}
// update weights connecting input neurons to hidden layer
for (i = 0; i < hidden.size(); i++) {
for (Neuron x : hidden.get(i).left) {
x.weights[i] = learningRate * hiddenDelta[i] * x.value;
}
}
I do not know what the correct solution is; but I have two suggestions:
Replace these lines:
h.weights[i] = learningRate * outputDelta[i] * h.value;
x.weights[i] = learningRate * hiddenDelta[i] * x.value;
with these lines:
h.weights[i] += learningRate * outputDelta[i] * h.value;
x.weights[i] += learningRate * hiddenDelta[i] * x.value;
(+= instead of =.)
Replace these lines:
h.weights[i] = learningRate * outputDelta[i] * h.value;
x.weights[i] = learningRate * hiddenDelta[i] * x.value;
with these lines:
h.weights[i] *= learningRate * outputDelta[i];
x.weights[i] *= learningRate * hiddenDelta[i];
(Ignore the value and simply scale the existing weight. The learning rate should be 1.05 instead of .05 for this change.)