I was searching the algorithm for finding the composition of 2 linear functions n times (where n can be as large as 10^18) in O(log n) time. I just got a pdf containing polynomial composition of 2 functions with large degrees using divide and conquer algorithm.
I was wondering whether my problem for composition of linear function n times can also be solved using divide and conquer algorithm in O(log n) complexity?
If yes, please explain the algorithm.
Thanks, in advance.
EDIT 1: The composition of function f(x) n times is fofof...n-times. Here the function is to be composed to itself n times. There are no 2 functions.
You can represent the application of a linear function f(x) = ax + b as a 2-by-2 matrix multiplied by the vector (x, 1).
(f(x)) = ( a b ) (x)
( 1 ) ( 0 1 ) (1)
Applying f n times to x is multiplying the matrix n times to (x, 1), or equivalently, multiplying the matrix raised to the power of n to (x, 1).
(f^n(x)) = ( a b )^n (x)
( 1 ) ( 0 1 ) (1)
You can compute the matrix power using exponentiation by squaring.
This works whether you're working over the real numbers, the integers, or the integers modulo some number M.
Related
I'm taking an algorithms class and I repeatedly have trouble when I'm asked to analyze the runtime of code when there is a line with multiplication or division. How can I find big-theta of multiplying an n digit number with an m digit number (where n>m)? Is it the same as multiplying two n digit numbers?
For example, right now I'm attempting to analyze the following line of code:
return n*count/100
where count is at most 100. Is the asymptotic complexity of this any different from n*n/100? or n*n/n?
You can always look up here Computational complexity of mathematical operations.
In your complexity of n*count/100 is O(length(n)) as 100 is a constant and length(count) is at most 3.
In general multiplication of two numbers n and m digits length, takes O(nm), the same time required for division. Here i assume we are talking about long division. There are many sophisticated algorithms which will beat this complexity.
To make things clearer i will provide an example. Suppose you have three numbers:
A - n digits length
B - m digits length
C - p digits length
Find complexity of the following formula:
A * B / C
Multiply first. Complexity of A * B it is O(nm) and as result we have number D, which is n+m digits length. Now consider D / C, here complexity is O((n+m)p), where overall complexity is sum of the two O(nm + (n+m)p) = O(m(n+p) + np).
Divide first. So, we divide B / C, complexity is O(mp) and we have m digits number E. Now we calculate A * E, here complexity is O(nm). Again overall complexity is O(mp + nm) = O(m(n+p)).
From the analysis you can see that it is beneficial to divide first. Of course in real life situation you would account for numerical stability as well.
From Modern Computer Arithmetic:
Assume the larger operand has size
m, and the smaller has size n ≤ m, and denote by M(m,n) the corresponding
multiplication cost.
When m is an exact multiple of n, say m = kn, a trivial strategy is to cut the
larger operand into k pieces, giving M(kn,n) = kM(n) + O(kn).
Suppose m ≥ n and n is large. To use an evaluation-interpolation scheme,
we need to evaluate the product at m + n points, whereas balanced k by k
multiplication needs 2k points. Taking k ≈ (m+n)/2, we see that M(m,n) ≤ M((m + n)/2)(1 + o(1)) as n → ∞. On the other hand, from the discussion
above, we have M(m,n) ≤ ⌈m/n⌉M(n)(1 + o(1)).
given the following problem from CLRS algo book.
For each function f (n) and time t in the following table, determine
the largest size n of a problem that can be solved in time t, assuming
that the algorithm to solve the problem takes f(n) microseconds.
how can one calculate n for f(n)=nlog(n) when time is 1 second?
how can one calculate n for f(n)=n! when time is 1 second?
It is mentioned that the algorithm takes f(n) microseconds. Then, one may consider that algorithm to consist of f(n) steps each of which takes 1 microsecond.
The questions given state that relevant f(n) values are bound by 1 second. (i.e. 106 microseconds) Then, since you are looking for the largest n possible to fulfill those conditions, your questions boil down to the inequalities given below.
1) f(n) = nlog(n) <= 106
2) f(n) = n! <= 106
The rest, I believe, is mainly juggling with algebra and logarithmic equations to find the relevant values.
In first case, You can refer to Example of newtons method to calculate cube root Newton’s method to approximate the roots or Lambert W Function. It may help to calculate value of n. As per the my findings mostly there is no other analytical approach can help.
In second case, python script can help to calculate n with manual approch.
def calFact(n):
if(n == 0 or n==1):
return n
return n*calFact(n-1)
nVal=1
while(calFact(nVal)<1000000): # f(n) = n! * 10^-6 sec
nVal=nVal+1 # 10^6 = n!
print(nVal)
So in this case we are trying to find out n such that n! is equal to or near to 10^6.
I want to calculate value of
F(N) = (F(N-1) * [((N-R+1)^(N-R+1))/(R^R)]) mod M for given values of N,R and M.
Here A^B shows A power B and NOT any Bitwise operation
Here M need not to be prime.How to approach this question?Please help because if M was prime that it would not have beed so difficult to find inverse of R^R mod M.
But as M can be any value ranging from 1 to 10^9.I am not able to tackle this problem.
N can be between 1 and 10^5 and R is less than or equal to N.
Assuming you know somehow that the result of the division is an integer:
As N and R are small, you can do this by computing the prime factorisation of N-R+1 and R.
If we know that R=p^a...q^b then R^R = p^(Ra)...q^(Rb).
Similarly you can work out the power of each prime in (N-R+1)^(N-R+1).
Subtract the powers of the primes in R^R from the powers of the primes in (N-R+1)^(N-R+1) gives you the powers of each prime in the result.
You can then use a standard binary exponentiation routine to compute the result modulo M with no inverses required.
How to find nth tribonacci number with matrix multiplication method if the initial values are some arbitrary numbers say 1, 2 3 i.e T(1) = 1, T(2) =2 and T(3) = 3.
If T(n) = T(n-1) + T(n-2) + T(n-3) then how to find T(n) if n is very very large, I would appreciate if anyone can explain with matrix multiplication method. How to construct initial matrix.
The matrix multiplication method involves using the matrix recurrence relation.
For the Fibonacci series, we can define a vector of length 2 to represent adjacent Fibonacci numbers. Using this vector, we can define a recurrence relation with a matrix multiplication:
Similarly, the Tribonacci series recurrence relation can be written in this way:
The only difference is that the vector and matrix sizes are different.
Now, to calculate a large Tribonacci number, we just apply the matrix multiplication n times, and we get:
The matrix to the power of n (Mn) can be efficiently calculated, because we can use an exponentiation algorithm.
Many efficient exponentiation algorithms for scalars are described by Wikipedia in Exponentiation by Squaring. We can use the same idea for matrix exponentiation.
I will describe a simple way to do this. First we write n as a binary number, eg:
n = 37 = 100101
Then, calculate M to each power of 2 by squaring the previous power of 2: M1, M2 = M1M1, M4 = M2M2, M8 = M4M4, M16 = M8M8, M32 = M16M16, ...
And finally, multiply the powers of M corresponding to the binary digits of n. In this case, Mn = M1M4M32.
After calculating that, we can multiply the matrix with the Tribonacci vector for the first 3 values, ie.
Because the matrices have fixed size, each matrix multiplication takes constant time. We must do O(log n) matrix multiplications. Thus, we can calculate the nth Tribonacci number in O(log n) time.
Compare this to the normal dynamic programming method, where it takes O(n) time, by calculating each Tribonacci number up to the nth Tribonacci number (ie. for (i = 3 to n) {T[i] = T[i-1]+T[i-2]+T[i-3];} return T[n];).
I will assume that you know how to code up matrix multiplication in the language of your choice.
Consider:
| a1 b1 c1 |
[f(n) f(n - 1) f(n - 2)] * | a2 b2 c2 | = [f(n + 1) f(n) f(n - 1)]
| a3 b3 c3 |
Find the unknowns in the matrix based on that and that will be the matrix you want.
The answer in this case is:
1 1 0
1 0 1
1 0 0
The method is general however, it works even if you sum k previous terms, even if they have constants in front of them etc.
How large a system is it reasonable to attempt to do a linear regression on?
Specifically: I have a system with ~300K sample points and ~1200 linear terms. Is this computationally feasible?
The linear regression is computed as (X'X)^-1 X'Y.
If X is an (n x k) matrix:
(X' X) takes O(n*k^2) time and produces a (k x k) matrix
The matrix inversion of a (k x k) matrix takes O(k^3) time
(X' Y) takes O(n*k^2) time and produces a (k x k) matrix
The final matrix multiplication of two (k x k) matrices takes O(k^3) time
So the Big-O running time is O(k^2*(n + k)).
See also: http://en.wikipedia.org/wiki/Computational_complexity_of_mathematical_operations#Matrix_algebra
If you get fancy it looks like you can get the time down to O(k^2*(n+k^0.376)) with the Coppersmith–Winograd algorithm.
You can express this as a matrix equation:
where the matrix is 300K rows and 1200 columns, the coefficient vector is 1200x1, and the RHS vector is 1200x1.
If you multiply both sides by the transpose of the matrix , you have a system of equations for the unknowns that's 1200x1200. You can use LU decomposition or any other algorithm you like to solve for the coefficients. (This is what least squares is doing.)
So the Big-O behavior is something like O(mmn), where m = 300K and n = 1200. You'd account for the transpose, the matrix multiplication, the LU decomposition, and the forward-back substitution to get the coefficients.
The linear regression is computed as (X'X)^-1 X'y.
As far as I learned, y is a vector of results (or in other words: dependant variables).
Therefore, if X is an (n × m) matrix and y is an (n × 1) matrix:
The transposing of a (n × m) matrix takes O(n⋅m) time and produces a (m × n) matrix
(X' X) takes O(n⋅m²) time and produces a (m × m) matrix
The matrix inversion of a (m × m) matrix takes O(m³) time
(X' y) takes O(n⋅m) time and produces a (m × 1) matrix
The final matrix multiplication of a (m × m) and a (m x 1) matrices takes O(m²) time
So the Big-O running time is O(n⋅m + n⋅m² + m³ + n⋅m + m²).
Now, we know that:
m² ≤ m³
n⋅m ≤ n⋅m²
so asymptotically, the actual Big-O running time is O(n⋅m² + m³) = O(m²(n + m)).
And that's what we have from
http://en.wikipedia.org/wiki/Computational_complexity_of_mathematical_operations#Matrix_algebra
But, we know that there's a significant difference between the case n → ∞ and m → ∞.
https://en.wikipedia.org/wiki/Big_O_notation#Multiple_variables
So which one should we choose? Obviously it's the number of observations which is more likely to grow, rather than the number of attributes.
So my conclusion is that if we assume that the number of attributes remains constant, we can ignore the m terms and that's a relief because the time complexity of a multivariate linear regression becomes a mere linear O(n). On the other hand, we can expect our computing time explodes by a large value when the number of attributes increase substantially.
The linear regression of closed-form model is computed as follow:
derivative of
RSS(W) = -2H^t (y-HW)
So, we solve for
-2H^t (y-HW) = 0
Then, the W value is
W = (H^t H)^-1 H^2 y
where:
W: is the vector of expected weights
H: is the features matrix N*D where N is the number of observations, and D is the number of features
y: is the actual value
Then, the complexity of
H^t H is n D^2
The complexity of the transpose is D^3
So, The complexity of
(H^t H)^-1 is n * D^2 + D^3