So here is an algorithm that is supposed to return the polynomial value of P(x) of a given polynomial with any given x.
A[] is the coefficient array and P[] the power of x array.
(e.g. x^2 +2*x + 1 would have: A[] = {1,2,1} , P[]= {2,1,0})
Also, recPower() = O(logn)
int polynomial(int x, int A[], int P[], int l, int r)
{
if (r - l == 1)
return ( A[l] * recPower(x, P[l]) ) + ( A[r] * recPower (x, P[r]) );
int m = (l + r) / 2;
return polynomial(x, A, P, l, m) + polynomial(x, A, P, m, r);
}
How do I go about calculating this time complexity? I am perplexed due to the if statement. I have no idea what the recurrence relation will be.
Following observation might help: As soon as we have r = l + 1, we spend O(logn) time and we are done.
My answer requires good understanding of Recursion Tree. So proceed wisely.
So our aim is to find : after how many iterations will we be able to tell that we have r = l + 1?
Lets find out:
Focusing on return polynomial(x, A, P, l, m) + polynomial(x, A, P, m, r);
Let us first consider left function polynomial(x, A, P, l, m). Key thing to note is that l , remains constant , in all subsequent left function called recursively.
By left function I mean polynomial(x, A, P, l, m) and by right function I mean
polynomial(x, A, P, m, r).
For left function polynomial(x, A, P, l, m), We have:
First iteration
l = l and r = (l + r)/2
Second iteration
l = l and r = (l + (l + r)/2)/2
which means that
r = (2l + l + r)/2
Third iteration
l = l and r = (l + (l + (l + r)/2)/2)/2
which means that
r = (4l + 2l + l + r)/4
Fourth iteration
l = l and r = (l + (l + (l + (l + r)/2)/2)/2)/2
which means that
r = (8l + 4l + 2l + l + r)/8
This means in nth iteration we have:
r = (l(1 + 2 + 4 + 8 +......2^n-1) + r)/2^n
and terminating condition is r = l + 1
Solving (l(1 + 2 + 4 + 8 +......2^n-1) + r)/2^n = l + 1, we get
2^n = r - l
This means that n = log(r - l). One might say that in all subsequent calls of left function we ignored the other call, that is right function call. The reason is this:
Since in the right function call we l = m, where m is already a reduced , as we take the mean, and r = r, which is even more averaged this asymptotically wont have any effect on time complexity.
So our recursion tree will have maximum depth = log(r - l). Its true that not all levels will be fully populated, but for the sake of simplicity, we assume this in asymptotic analysis. So after reaching a depth of log(r - l), we call function recPower, which takes O(logn) time. Total nodes (assuming all levels above are full) at depth log(r - l) is 2^(log(r - l) - 1). For a single node , we take O(logn) time.
Therefore we have total time = O( logn*(2^(log(r - l) - 1)) ).
This might help:
T(#terms) = 2T(#terms/2) + a
T(2) = 2logn + b
Where a and b are constants, and #terms refer to number of terms in polynomial.
This recurrence relation can be solved using Master's Theorem or using the Recursion tree method.
Related
I am having trouble developing the recurrence for an algorithm that uses recursive Merge Sort calls for list sizes greater than m. It uses Selection Sort for list sizes less or equal to m.
Here is my pseudocode:
proc merge_and_selection (A, p, r, m) {
if (p <= r) then
q = (p + r)/2
if r - p > m then
merge_and_selection(A, p, q - 1, m)
merge_and_selection(A, q + 1, r, m)
else
selection_sort(A, p, q - 1)
selection_sort(A, q + 1, r)
end
merge(A, p, q, r)
end if
}
I think the recurrence is:
with T(2) = [m(m-1)]/2
I think more accurate formula is the following:
T(n) = 2*T(n/2) + Theta(n) for n >= m/2
T(n) = Theta(n^2) for n < m/2
I have stumbled upon a problem, which requires me to calculate the nth Tetranacci Number in O(log n).
I have seen several solutions for doing this for Fibonacci Numbers
I was looking to follow a similar procedure (Matrix Multiplication/Fast Doubling) to achieve this, but I am not sure how to do it exactly (take a 4 by 4 matrix and 1 by 4 in a similar fashion doesn't seem to work). With dynamic programming/general loops/any other basic idea, I am not able to achieve sub-linear runtime. Any help appreciated!
Matrix multiplication of course works. Here's how to derive the matrix.
What we want is to find the entries that make the equation
[a b c d] [T(n-1)] [T(n) ]
[e f g h] [T(n-2)] [T(n-1)]
[i j k l] [T(n-3)] = [T(n-2)]
[m n o p] [T(n-4)] [T(n-3)]
true for all n. Expand.
a T(n-1) + b T(n-2) + c T(n-3) + d T(n-4) = T(n)
e T(n-1) + f T(n-2) + g T(n-3) + h T(n-4) = T(n-1)
i T(n-1) + j T(n-2) + k T(n-3) + l T(n-4) = T(n-2)
m T(n-1) + n T(n-2) + o T(n-3) + p T(n-4) = T(n-3)
The obvious settings here are a = b = c = d = 1 (using the recurrence) and e = j = o = 1 and f = g = h = i = k = l = m = n = p = 0 (basic algebra).
The initial vector is
[T(3)] [1]
[T(2)] [0]
[T(1)] = [0]
[T(0)] [0]
by definition.
I have derived the Tetranacci doubling formulas from the corresponding matrix as described in the other answers. The formulas are:
T(2n) = T(n+1)*(2*T(n+2) - T(n+1)) + T(n)*(2*T(n+3) - 2*T(n+2) - 2*T(n+1) - T(n))
T(2n+1) = T(n)^2 + T(n+2)^2 + T(n+1)*(2*T(n+3) - 2*T(n+2) - T(n+1))
T(2n+2) = T(n+1)*(2*T(n) + T(n+1)) + T(n+2)*(2*T(n+3) - T(n+2))
T(2n+3) = T(n+1)^2 + T(n+3)^2 + T(n+2)*(2*T(n) + 2*T(n+1) + T(n+2))
With these, we can implement the "fast doubling" method. Here's one such implementation in Python, whose native support for arbitrary-sized integers is very convenient:
def tetranacci_by_doubling(n):
if n >= 0:
a, b, c, d = 0, 0, 0, 1 # T(0), T(1), T(2), T(3)
else: # n < 0
a, b, c, d = 1, 0, 0, 0 # T(-1), T(0), T(1), T(2)
# unroll the last iteration to avoid computing unnecessary values.
for i in reversed(range(1, abs(n).bit_length())):
w = b*(2*c - b) + a*(2*(d - c - b) - a)
x = a*a + c*c + b*(2*(d - c) - b)
y = b*(2*a + b) + c*(2*d - c)
z = b*b + d*d + c*(2*(a + b) + c)
a, b, c, d = w, x, y, z
if (n >> i) & 1 == 1:
a, b, c, d = b, c, d, a + b + c + d
if n & 1 == 0:
return b*(2*c - b) + a*(2*(d - c - b) - a) # w
else: # n & 1 == 1
return a*a + c*c + b*(2*(d - c) - b) # x
def tetranacci(n):
a, b, c, d = 0, 0, 0, 1 # T(0), T(1), T(2), T(3)
# offset by 3 to reduce excess computation for large positive `n`
n -= 3
if n >= 0:
for _ in range(+n):
a, b, c, d = b, c, d, a + b + c + d
else: # n < 0
for _ in range(-n):
a, b, c, d = d - c - b - a, a, b, c
return d
# sanity check
print(all(tetranacci_by_doubling(n) == tetranacci(n) for n in range(-1000, 1001)))
I would've liked to adjust the doubling formulas to be T(2n-3),T(2n-2),T(2n-1),T(2n) in terms of T(n-3),T(n-2),T(n-1),T(n) to slightly reduce excess computation for large n, but simplifying the shifted formulas is tedious.
Update
Swapped to an iterative version since I figured out how to make it cleanly handle negative n with minimal duplication. Originally, this was the sole advantage of the recursive version.
Incorporated a technique that's described in several papers about computing Fibonacci & Lucas numbers--which is to perform the final doubling step manually after the loop to avoid computing extra unneeded values. This results in about ~40%-50% speed-up for large n (>= 10^6)! This optimization could also be applied to the recursive version, as well.
The speed-up due to the unrolling of the last iteration is pretty interesting. It suggests that nearly half of the computational work is done in the final step. This kind of makes sense, since the number of digits in T(n) (and therefore the cost of arithmetic) approximately doubles when n doubles, and we know that 2^n ~= 2^0 + 2^1 + ... + 2^(n-1). Applying the optimization to similar Fibonacci/Lucas doubling algorithms produces a similar speed-up of ~40%--although, if you're computing Fibonacci/etc. modulo some 64-bit M, I suspect this optimization isn't as valuable.
From the OEIS, this is the (1,4) entry of the nth power of
1 1 0 0
1 0 1 0
1 0 0 1
1 0 0 0
To compute the nth power of that matrix in O(log n) operations, you can use exponentiation by squaring. There might be a slightly simpler recurrence, but you should be able to implement the general technique.
Minimum standard random number generator.
I am reading about minimum standard number generator as below
Given a random integer xn, the next random integer in a random
sequence is given by computing xn+1 = a xn (mod m), where a = 7 ^ 5 =
16807 and m = 2^31 − 1 = 2147483647; as a check on your
implementation, if x0 = 1, then x10000 = 1043618065.
Park and Miller chose m as the largest Mersenne prime less than 2^32;
the smallest primitive root of m is 7, and since 5 is also prime, 7^5
is also a primitive root, hence their choice of a. Because a is a
primitive root of m, all values in the range 1 to m − 1 inclusive will
be generated before any repeat, so the random number generator has full
period. The multiplier a = 16807 has been shown to have good
randomness properties.
Subsequent to their original paper, Park and Miller recommended 48271
as an improvement, and some people use 69621, but we’ll continue to
use 16807. The easiest way to implement that is obvious: just
multiply a by the current value of x and compute the modulus.
But that may cause overflow in the intermediate multiplication,
rendering the results incorrect.
A trick of Linus Schrage allows that multiplication to be done without
overflow: Compute q = ⌊m / a⌋ and r = m (mod a) so that m = a q + r.
Then
a new x can be computed by hi = ⌊x / q⌋, lo = x (mod q), x = a · lo −
r · hi, then adding m to x if x ≤ 0.
My question is how author has computed new x in terms of as hi = floor(x/q) and lo = x(modq). I am looking for steps here. Kindly expain.
Let's simplify the notation. Set H = hi and L = lo. We have m = a * q + r. A simple calculation shows that q = 127773 and r = 2836. We observe that a < q.
Now let x_{n} be given and calculate H = x_{n} / q and L = x_{n} % q. So, x_{n} = q * H + L with L < q.
By definition x_{n+1} = a * x_{n} mod m. Calculating the right-hand-side (before reduction mod m) we get a * x_{n} = a * (q * H + L) = a * q * H + a * L = (m - r) * H + a * L = m * H - r * H + a * L.
Now let's consider r * H. Clearly 0 <= r * H < a * (x_{n} / q). As x_{n} < m and as observed above a < q, a * (x_{n} / q) < m. In particular, it doesn't overflow.
Likewise 0 < a * L < a * q < m. So, again no overflow.
We conclude that x_{n+1} = m * H - r * H + a * L. Reducing the latter modulo m we get x_{n+1} = -r * H + a * L with neither of the two right-hand-side expressions overflowing m. If the sum is negative we add m and we are done.
I want to find a fast algorithm to evaluate an expression like the following, where P is prime.
A ^ B ^ C ^ D ^ E mod P
Example:
(9 ^ (3 ^ (15 ^ (3 ^ 15)))) mod 65537 = 16134
The problem is the intermediate results can grow much too large to handle.
Basically the problem reduces to computing a^T mod m for given a, m and a term T that is ridiulously huge. However, we are able to evaluate T mod n with a given modulus n much faster than T . So we ask: "Is there an integer n, such that a^(T mod n) mod m = a^T mod m?"
Now if a and m are coprime, we know that n = phi(m) fulfills our condition according to Euler's theorem:
a^T (mod m)
= a^((T mod phi(m)) + k * phi(m)) (mod m) (for some k)
= a^(T mod phi(m)) * a^(k * phi(m)) (mod m)
= a^(T mod phi(m)) * (a^phi(m))^k (mod m)
= a^(T mod phi(m)) * 1^k (mod m)
= a^(T mod phi(m)) (mod m)
If we can compute phi(m) (which is easy to do for example in O(m^(1/2)) or if we know the prime factorization of m), we have reduced the problem to computing T mod phi(m) and a simple modular exponentiation.
What if a and m are not coprime? The situation is not as pleasant as before, since there might not be a valid n with the property a^T mod m = a^(T mod n) mod m for all T. However, we can show that the sequence a^k mod m for k = 0, 1, 2, ... enters a cycle after some point, that is there exist x and C with x, C < m, such that a^y = a^(y + C) for all y >= x.
Example: For a = 2, m = 12, we get the sequence 2^0, 2^1, ... = 1, 2, 4, 8, 4, 8, ... (mod 12). We can see the cycle with parameters x = 2 and C = 2.
We can find the cycle length via brute-force, by computing the sequence elements a^0, a^1, ... until we find two indices X < Y with a^X = a^Y. Now we set x = X and C = Y - X. This gives us an algorithm with O(m) exponentiations per recursion.
What if we want to do better? Thanks to Jyrki Lahtonen from Math Exchange for providing the essentials for the following algorithm!
Let's evaluate the sequence d_k = gcd(a^k, m) until we find an x with d_x = d_{x+1}. This will take at most log(m) GCD computations, because x is bounded by the highest exponent in the prime factorization of m. Let C = phi(m / d_x). We can now prove that a^{k + C} = a^k for all k >= x, so we have found the cycle parameters in O(m^(1/2)) time.
Let's assume we have found x and C and want to compute a^T mod m now.
If T < x, the task is trivial to perform with simple modular exponentiation. Otherwise, we have T >= x and can thus make use of the cycle:
a^T (mod m)
= a^(x + ((T - x) mod C)) (mod m)
= a^(x + (-x mod C) + (T mod C) + k*C) (mod m) (for some k)
= a^(x + (-x mod C) + k*C) * a^(T mod C) (mod m)
= a^(x + (-x mod C)) * a^(T mod C) (mod m)
Again, we have reduced the problem to a subproblem of the same form ("compute T mod C") and two simple modular exponentiations.
Since the modulus is reduced by at least 1 in every iteration, we get a pretty weak bound of O(P^(1/2) * min (P, n)) for the runtime of this algorithm, where n is the height of the stack. In practice we should get a lot better, since the moduli are expected to decrease exponentially. Of course this argument is a bit hand-wavy, maybe some more mathematically-inclined person can improve on it.
There are a few edge cases to consider that actually make your life a bit easier: you can stop immediately if m = 1 (the result is 0 in this case) or if a is a multiple of m (the result is 0 as well in this case).
EDIT: It can be shown that x = C = phi(m) is valid, so as a quick and dirty solution we can use the formula
a^T = a^(phi(m) + T mod phi(m)) (mod m)
for T >= phi(m) or even T >= log_2(m).
I have this problem that I can't solve.. what is the complexity of this foo algorithm?
int foo(char A[], int n, int m){
int i, a=0;
if (n>=m)
return 0;
for(i=n;i<m;i++)
a+=A[i]
return a + foo(A, n*2, m/2);
}
the foo function is called by:
foo(A,1,strlen(A));
so.. I guess it's log(n) * something for the internal for loop.. which I'm not sure if it's log(n) or what..
Could it be theta of log^2(n)?
This is a great application of the master theorem:
Rewrite in terms of n and X = m-n:
int foo(char A[], int n, int X){
int i, a=0;
if (X < 0) return 0;
for(i=0;i<X;i++)
a+=A[i+n]
return a + foo(A, n*2, (X-3n)/2);
}
So the complexity is
T(X, n) = X + T((X - 3n)/2, n*2)
Noting that the penalty increases with X and decreases with n,
T(X, n) < X + T(X/2, n)
So we can consider the complexity
U(X) = X + U(X/2)
and plug this into master theorem to find U(X) = O(X) --> complexity is O(m-n)
I'm not sure if there's a 'quick and dirty' way, but you can use old good math. No fancy theorems, just simple equations.
On k-th level of recursion (k starts from zero), a loop will have ~ n/(2^k) - 2^k iterations. Therefore, the total amount of loop iterations will be S = sum(n/2^i) - sum(2^i) for 0 <= i <= l, where l is the depth of recursion.
The l will be approximately log(2, n)/2 (prove it).
Transforming each part in formula for S separately, we get.
S = (1 + 2 + .. + 2^l)*n/2^l - (2^(l + 1) - 1) ~= 2*n - 2^(l + 1) ~= 2*n - sqrt(n)
Since each other statement except loop will be repeated only l times and we know that l ~= log(2, n), it won't affect complexity.
So, in the end we get O(n).