I'm trying to use Taylor series to develop a numerically sound algorithm for solving a function. I've been at it for quite a while, but haven't had any luck yet. I'm not sure what I'm doing wrong.
The function is
f(x)=1 + x - sin(x)/ln(1+x) x~0
Also: why does loss of precision even occur in this function? when x is close to zero, sin(x)/ln(1+x) isn't even close to being the same number as x. I don't see where significance is even being lost.
In order to solve this, I believe that I will need to use the Taylor expansions for sin(x) and ln(1+x), which are
x - x^3/3! + x^5/5! - x^7/7! + ...
and
x - x^2/2 + x^3/3 - x^4/4 + ...
respectively. I have attempted to use like denominators to combine the x and sin(x)/ln(1+x) components, and even to combine all three, but nothing seems to work out correctly in the end. Any help is appreciated.
The loss of precision can come in because when x ~ 0, ln(1+x) is also close to 0, so you wind up dividing by a very small number. Computers aren't very good at that ;-)
If you use the Taylor series for ln(1+x) directly, it's going to be kind of a pain because you'll wind up dividing by an infinite series of terms. For cases like this, I usually prefer to just compute the Taylor series for the entire function as a whole from the definition:
f(x) = f(0) + f'(0) x + f''(0) x/2 + f'''(0) x/6 + ...
from which you'll get
f(x) = 2 + 3x/2 - x^2/4 - x^3/24 - x^4/240 - 23x^5/1440 + 31x^6/2880 ...
(I cheated and plugged it into Mathematica ;-) Like Steve says, this series doesn't converge all that quickly, although I can't think of a better method at the moment.
EDIT: I think I misread the question - if all you're trying to do is find the zeros of the function, there are definitely better ways than using a Taylor series.
As this is homework, I'm just going to try to give a few pointers in the right direction.
Solution 1
Rather than using the Talyor series approximation, try to simply use a root finding algorithm such as the Newton-Raphson method, linear interpolation, or interval bisection (or combine them even). They are very simple to implement, and with an appropiate choice of starting value(s), the root can converge to a precise value quite quickly.
Solution 2
If you really need to use the Taylor series approximation for whatever reason, then just expand the sin(x), ln(x), and whatever else. (Multiplying through by ln(x) to remove the denominator in your case will work). Then you'll need to use some sort of polynomial equation solver. If you want a reasonable degree of accuracy, you'll need to go beyond the 3rd or 4th powers I'd imagine, which means a simple analytical solution is not going to be easy. However, you may want to look into something like the Durand-Kerner method for solving general polynomials of any order. Still, if you need to use high-order terms this approach is just going to lead to complications, so I would definitely recommend solution 1.
Hope that helps...
I think you need to look at what happens to ln(x+1) as x -->0 and you will see why this function does not behave well near x = 0.
I haven't looked into this that closely, but you should be aware that some taylor series converge very, very slowly.
Just compute the Taylor series of f directly.
Maxima gives me (first 4 terms about x=0):
(%i1) f(x):=1 + x - sin(x)/log(1+x);
- sin(x)
(%o1) f(x) := 1 + x + ----------
log(1 + x)
(%i2) taylor(f(x),x,0,4);
2 3 4
x x x x
(%o2)/T/ - + -- + -- + --- + . . .
2 4 24 240
Method used in question is correct - just make sure your calculator is in radians mode.
Related
I'm trying to understand the algebra behind Big-O expressions. I have gone through several questions but still don't have a very clear idea how it's done.
When dealing with powers do we always omit the lower powers, for example:
O(10n^4-n^2-10) = O(10n^4)
What difference does it make when multiplication is involved? For example:
O(2n^3+10^2n) * O(n) = O(2n^3) ??
And finally, how do we deal with logs? For example:
O(n2) + O(5*log(n))
I think we try to get rid of all constants and lower powers. I'm not sure how logarithms are involved in the simplification and what difference a multiplication sign would do. Thank you.
Big-O expressions are more closely related to Calculus, specifically limits, than they are to algebraic concepts/rules. The easiest way I've found to think about expressions like the examples you've provided, is to start by plugging in a small number, and then a really large number, and observe how the result changes:
Expression: O(10n^4-n^2-10)
use n = 2: O(10(2^4) - 2^2 - 10)
O(10 * 16 - 4 - 10) = 146
use n = 100: O(10(100^4) - 100^2- 10)
O(10(100,000,000) - 10,000 - 10) = 999,989,990
What you can see from this, is that the n^4 term overpowers all other terms in the expression. Therefore, this algorithm would be denoted as having a run-time of O(n^4).
So yes, your assumptions are correct, that you should generally go with the highest power, drop constants, and drop order-1 terms.
Logarithms are effectively "undoing" exponentiation. Because of this, they will reduce the overall O-run-time of an algorithm. However, when they are added against exponential run-times, they generally get overruled by the larger order term. In the example you provided, if we again evaluate using real numbers:
Expression: O(n^2) + O(5*log(n))
use n=2: O(2^2) + O(5*log(2))
O(4) + O(3.4657) = 7.46
use n=100: O(100^2) + O(5*log(100))
O(10,000) + O(23.02) = 10,023
You will notice that although the logarithm term is increasing, it isn't a great gain compared to the increase in n's size. However, the n^2 term is still generating a massive increase compared to the increase in n's size. Because of this, the Big O of these expressions combined would still boil down to: O(n^2).
If you're interested in further reading about the mathematics side of this, you may want to check out this post: https://secweb.cs.odu.edu/~zeil/cs361/web/website/Lectures/algebra/page/algebra.html
Source: Google Code Jam. https://code.google.com/codejam/contest/10224486/dashboard#s=a&a=1
We're asked to calculate Prob(K successes from N trials) where each of the N trials has a known success probability of p_n.
Some Analysis and thoughts on the problem are given after the Code Jam.
They observe that evaluating all possible outcomes of your N trials would take you an exponential time in N, so instead they provide a nice "dynamic programming" style solution that's O(N^2).
Let P(p#q) = Prob(p Successes after the first q Trials)
Then observe the fact that:
Prob(p#q) = Prob(qth trial succeeds)*P(p-1#q-1) + Prob(qth trial fails)*P(p#q-1)
Now we can build up a table of P(i#j) where i<=j, for i = 1...N
That's all lovely - I follow all of this and could implement it.
Then as the last comment, they say:
In practice, in problems like this, one should store the logarithms of
probabilities instead of the actual values, which can become small
enough for floating-point precision errors to matter.
I think I broadly understand the point they're trying to make, but I specifically can't figure out how to use this suggestion.
Taking the above equation, and substuting in some lettered variables:
P = A*B + C*D
If we want to work in Log Space, then we have:
Log(P) = Log(A*B + C*D),
where we have recursively pre-computed Log(B) and Log(D), and A & B are known, easily-handled decimals.
But I don't see any way to calculate Log(P) without just doing e^(Log(B)), etc. which feels like it would defeat to point of working in log space`?
Does anyone understand in better detail what I'm supposed to be doing?
Starting from the initial relation:
P = A⋅B + C⋅D
Due to its symmetry we can assume that B is larger than D, without loss of generality.
The following processing is useful:
log(P) = log(A⋅B + C⋅D) = log(A⋅elog(B) + C⋅elog(D)) = log(elog(B)⋅(A + C⋅elog(D) - log(B))
log(P) = log(B) + log(A + C⋅elog(D) - log(B)).
This is useful because, in this case, log(B) and log(D) are both negative numbers (logarithms of some probabilities). It was assumed that B is larger than D, thus its log is closer to zero. Therefore log(D) - log(B) is still negative, but not as negative as log(D).
So now, instead of needing to perform exponentiation of log(B) and log(D) separately, we only need to perform exponentiation of log(D) - log(B), which is a mildly negative number. So the above processing leads to better numerical behavior than using logarithms and applying exponentiation in the trivial way, or, equivalently, than not using logarithms at all.
Does there exist some algorithm that allows for the creation of a mathematical model given an inclusive set?
I'm not sure I'm asking that correctly... Let me try again...
Given some input set...
int Set[] = { 1, 4, 9, 16, 25, 36 };
Does there exist an algorithm that would be able to deduce the pattern evident in the set? In this case being...
Set[x] = x^2
The only way I can think of doing something like this is some GA where the fitness is how closely the generated model matches the input set.
Edit:
I should add that my problem domain implies that the set is inclusive. Meaning, I am finding the closest possible function for the set and not using the function to extrapolate beyond the set...
The problem of curve fitting might be a reasonable place to start looking. I'm not sure if this is exactly what you're looking for - it won't really identify the pattern so much as just produce a function which follows the pattern as closely as possible.
As others have mentioned, for a simple set there can easily be infinitely many such functions, so something like this may be what you want, rather than exactly what you have described in your question.
Wikipedia seems to indicate that the Gauss-Newton algorithm or the Levenberg–Marquardt algorithm might be a good place to begin your research.
A mathematical argument explaining why, in general, this is impossible:
There are only countably many computer programs that can be written at all.
There are uncountably many infinite sequences of integers.
Therefore, there are infinitely many sequences of integers for which no possible computer program can generate those sequences.
Accordingly, this is impossible in the general case. Sorry!
Hope this helps!
If you want to know if the given data fits some polynomial function, you compute successive differences until you reach a constant. The number of differences to reach the constant is the degree of the polynomial.
x | 1 2 3 4
y | 1 4 9 16
y' | 3 5 7
y" | 2 2
Since y" is 2, y' is 2x + C1, and thus y is x2 + C1x + C2. C1 is 0, since 2×1.5 = 3. C2 is 0 because 12 = 1. So, we have y = x2.
So, the algorithm is:
Take successive differences.
If it does not converge to a constant, either resort to curve fitting, or report the data is insufficient to determine a polynomial.
If it does converge to a constant, iteratively integrate polynomial expression and evaluate the trailing constant until the degree is achieved.
How can I determine the initial guess of the equation Ax+Bsin(x)=C in terms of A,B and C ?
I am trying to solve it using Newton Raphson. A,B and C will be given during runtime.
Is there any other method more efficient than Newton Raphson for this purpose ?
The optimal initial guess is the root itself, so finding an "optimal" guess isn't really valid.
Any guess will give you a valid solution eventually as long as f'(x0) != 0 for any step, which only occurs at the zeroes of cos(x), which are k*pi + pi/2 for any integer k.
I would try x0 = C * pi, just to see if it works.
Your biggest problem, however, would be the periodic nature of your function. Newton's method will be slow (if it even works) for your function as sin(x) will shift x0 back and forth over and over.
Precaution:
In Newton's method, do you notice how f'(xn) is in the denominator? f'(x) approaches 0 infinitely many times. If your f'(x) = 0.0001 (or anywhere close to zero, which has a chance of happening), your xn+1 gets thrown really far away from xn.
Worse yet, this can happen over and over due to f'(x) being a periodic function, which means that Newton's method might never even converge for an arbitrary x0.
The simplest "good" approximation is to just assume that sin(x) is approximately zero, and so set:
x0 = C/A
Well, if A,B and C are real and different from 0, then (B+C)/A is an upper quote to the highest root and (C-B)/A is a lower quote to the lowest root, as -1 <= sin(x) <= 1. You could start with those.
Newton method can work with any guess. the problem is simple,
if there is an equation and I guessed x0=100
and the best close solution for it is x0=2
and I know the answer is 2.34*
by using any guess in the world you will eventually get to 2.34*
the method says to choose a guess because without a valid guess it will take many solutions which aren't comfortable no one wants to repeat the method 20 times
and guessing a solution is not hard
you just find a critical point-
for example, 3 is too big and 2 is too small
so the answer is between 2 and 3
but if instead guessing 2 you guess 50
you will still get to the right solution.
like I said it will just take you much longer
I tested the method by myself
I guessed 1000 to a random equation
and I knew the best guess was 4
the answer was between 4 and 5
I chose 1000 it took me much time
but after a few hours, I got down from 1000 to 4.something
if you somehow can't find a critical point you can actually put a random number equals to x0 and then eventually you will get to the right solution
no matter what number you guessed.
How would I solve:
-x^3 - x - 4 = 0
You can't use quadratic, because it is to the 3rd power, right?
I know I it should come out to ~1.3788 but I'm not sure how I would derive that.
I started with:
f(x) = x + (4/(x^2 + 1)).
Solving for 0, moving the x over to the other side, multiplying by (x^2 + 1) on both sides, I end up with:
-x(x^2 + 1) = 4,
or
-x^3 - x - 4 = 0.
Finding roots of equations by using Newton's method or Fixed point iteration
Algebraically, you want to use Cardano's method:
http://www.math.ucdavis.edu/~kkreith/tutorials/sample.lesson/cardano.html
Using this method, it's about as easy to solve as the quadratic.
Actually, this is possibly clearer:
http://en.wikipedia.org/wiki/Cubic_function#Summary
Find a root by using newton iteration (see link below). Then divide the polynomial by (x-TheRootYouFound). The result will be a quadratic formula that you can plug into your quadratic root finder of your choice.
About Newton Iteration:
http://en.wikipedia.org/wiki/Newton%27s_method
About Polynomial Division
http://en.wikipedia.org/wiki/Polynomial_long_division
This article may be interesting for you as well. It covers more robust ways to solve your problem at the expense of some additional complexity.
http://en.wikipedia.org/wiki/Root-finding_algorithm
It's a cubic function. You're correct, the quadratic formula does not apply.
You gave one root, but in general there are three.
How did you arrive at that single value? Trial and error? That's legitimate. You don't need to "derive" anything.
x^3 + a2*x^2 + a1*x + a0 = 0 can be written as (x-x1)*(x-x2)*(x-x3) = 0, where x1, x2, and x3 are the three roots. If you know that the root you cited is correct, you can divide it out and leave (x-x2)*(x-x3) = 0, which is a quadratic that you can apply the usual techniques to.
This may not help from a programming viewpoint, but from a math perspective...
Note than in this particular cubic function you need to consider imaginary numbers, because when x = i then you have a denominator that is zero (in your original equation). Also, generally speaking you shouldn't multiply or divide by variables (adding and subtracting is fine though) when you move them to the other side of the equation because you'll generally forget of about the condition where the term you multiplied or divided by is zero. Those answers need to be excluded from the solution set.
x = i is an example of an excluded solution in the above cubic. You need to evaluate your excluded solutions before you manipulate the equation at all.