I want to compare three algorithms in case of steps of calculation, but I'm not very familiar with the O notation. The steps of calculation for each algorithm depend on three parameters (x,y,z):
algorithm 1) number_of_steps = x^2 * y * z - xyz
algorithm 2) number_of_steps = x^2 * y^2 * z - xyz
algorithm 3) number_of_steps = x^2 * y^2 * z^2 - xyz
There is not a huge difference between x, y and z - maybe one of the value is twice or triple another one, but not more.
How would it look in o notation? There are some solutions I can think about:
O1(n) vs. O2(n*y) vs. O3(n*y*z)
or
O1(n^4) vs. O2(n^5) vs. O3(n^6) - if x, y and z are nearly equal
or just
O1(x^2 * y * z) vs. O2(x^2 * y^2 * z) vs. O3(x^2 * y^2 * z^2)
Which is correct?
If max(x,y,z)/min(x,y,z) <= 3 (or less than any fixed number), then your complexities are
Algorithm 1: O(x^4) = O(y^4) = O(z^4)
Algorithm 2: O(x^5) = O(y^5) = O(z^5)
Algorithm 3: O(x^6) = O(y^6) = O(z^6)
If all you know is that x,y,z are at least 2, you can write
Algorithm 1: O(x^2 * y * z)
Algorithm 2: O(x^2 * y^2 * z)
Algorithm 3: O(x^2 * y^2 * z^2)
Of course you could always write them out in full like O(x^2 * y * z - xyz), but the simpler versions are better when possible.
Related
need some help with question from Data Structure course:
I was given this recursive function of mergesort (pseudo code):
Mergesort_1/3(A, p, r)
if p < r
then q = (p+(r-p)/3) // round the result down
Mergesort_1/3 (A,p,q)
Mergesort_1/3 (A,q+1,r)
Merge(A,p,q,r)
and these are the questions:
Let T(n) be the worst case running time of Mergesort _1/3. Write the recursive function for T(n). Give a short explanation.
Prove that T(n)=Ω(nlogn)
The gist of classic mergesort is the following recursion:
Split an unordered list in half.
It suffices to compute the limit offsets of the partial lists.
Apply the mergesort to each of the partial lists.
After this step, the partial lists will be sorted.
Merge the sorted partial lists by inspecting the elements of both lists in their respective order, coyping from and progressing in the list with the smaller element.
Let TC(n) be the time complexity of classic mergesort. The aforementioned steps take O(1) (*), 2*O(TC(ceil(n/2))), and O(n), respectively. This lends to the recursion TC(n) = cc_0 + cc_1 * n + 2 * TC(ceil(n/2)).
Consider the generalized mergesort where lists are split unevenly, though always with the same ratio. The complexity of splitting and merging remains the same, lending to the recursion TG(n) = ca_0 + ca_1 * n + TG(1/a * n + 1) + TG((a-1)/a * n + 1)) for the generalized mergesort (using TG(x+1) instead of TG(ceil(x)); ca_0, ca_1 being the constants hidden in the O(_) notation; a=3 for mergesort_1/3).
This recurrence can be solved using the Akra-Bazzi-Method.
To this end, the recurrence needs to be written as
TG(x) = g(x) + \sum_i=1..k ( a_i * TG(b_i * x + h_i(x) ) ; x >= x_0.
with
a_i, b_i const.
a_i > 0
0 < b_i < 1
|g(x)| = O(x^c); c const.
|h(x)| = O(x / (log(x))^2)
which can be done by setting ...
k = 2
a_1 = 1
a_2 = 1
b_1 = 1/a
b_2 = (a-1)/a
g(x) = ca_0 + ca_1 * x
h_1(x) = 1
h_2(x) = 1
x_0 = 2
-> TG(x) = ca_0 + ca_1 * n + 1 * TG(1/a * x + 1 ) + 1 * TG((a-1)/a * x + 1 ) ; x >= x_0.
The Akra-Bazzi theorem requires the exponent p be found in \sum_i=1..k ( a_i * (b_i)^p ) = 1. Then the following holds:
TG(x) = Θ ( x^p \integral_(1, x) ( g(u) / (u^(p+1)) du ) )
Specifically,
a_1 * b_1^p + a_2 * b_2^p = 1
=> (1/a)^p + ((a-1)/a)^p = 1
<=> p = 1
... and thus ...
TG(x) = Θ ( x \integral_(1, x) ( (ca_0 + ca_1 * u) / u^2 du ) )
= Θ ( x [ - ca_0/u + ca_1 * log(u) ]_(1,x) )
= Θ ( - ca_0 + ca_1 * x * log(x) + ca_0 * x - ca_1 * x * log(1) )
= Θ (x * log(x))
(*) Strictly speaking this is incorrect since basic arithmetics on binary representations and memory access is O(log n). However, this makes no difference asymptotically for Θ(n log n) complexity.
Instead of two, this mergesort makes partitions the data in three equal size chunks. So you are reducing a problem of size n to three problems of size n/3. Further, when merging, you will have to go through all the three n/3 sized sorted chunks which will result in going through total n elements. Hence, the recurrence can be written as:
T(n) = 3 T(n / 3) + O(n)
Using Master Theorem: Here a = 3, b = 3 and c = 1. Logba = Log33 = 1 = c.
So the recurrence will fall in the second case and T(n) = Θ(nc * Log(n)) = Θ(n * Log(n)).
Minimum standard random number generator.
I am reading about minimum standard number generator as below
Given a random integer xn, the next random integer in a random
sequence is given by computing xn+1 = a xn (mod m), where a = 7 ^ 5 =
16807 and m = 2^31 − 1 = 2147483647; as a check on your
implementation, if x0 = 1, then x10000 = 1043618065.
Park and Miller chose m as the largest Mersenne prime less than 2^32;
the smallest primitive root of m is 7, and since 5 is also prime, 7^5
is also a primitive root, hence their choice of a. Because a is a
primitive root of m, all values in the range 1 to m − 1 inclusive will
be generated before any repeat, so the random number generator has full
period. The multiplier a = 16807 has been shown to have good
randomness properties.
Subsequent to their original paper, Park and Miller recommended 48271
as an improvement, and some people use 69621, but we’ll continue to
use 16807. The easiest way to implement that is obvious: just
multiply a by the current value of x and compute the modulus.
But that may cause overflow in the intermediate multiplication,
rendering the results incorrect.
A trick of Linus Schrage allows that multiplication to be done without
overflow: Compute q = ⌊m / a⌋ and r = m (mod a) so that m = a q + r.
Then
a new x can be computed by hi = ⌊x / q⌋, lo = x (mod q), x = a · lo −
r · hi, then adding m to x if x ≤ 0.
My question is how author has computed new x in terms of as hi = floor(x/q) and lo = x(modq). I am looking for steps here. Kindly expain.
Let's simplify the notation. Set H = hi and L = lo. We have m = a * q + r. A simple calculation shows that q = 127773 and r = 2836. We observe that a < q.
Now let x_{n} be given and calculate H = x_{n} / q and L = x_{n} % q. So, x_{n} = q * H + L with L < q.
By definition x_{n+1} = a * x_{n} mod m. Calculating the right-hand-side (before reduction mod m) we get a * x_{n} = a * (q * H + L) = a * q * H + a * L = (m - r) * H + a * L = m * H - r * H + a * L.
Now let's consider r * H. Clearly 0 <= r * H < a * (x_{n} / q). As x_{n} < m and as observed above a < q, a * (x_{n} / q) < m. In particular, it doesn't overflow.
Likewise 0 < a * L < a * q < m. So, again no overflow.
We conclude that x_{n+1} = m * H - r * H + a * L. Reducing the latter modulo m we get x_{n+1} = -r * H + a * L with neither of the two right-hand-side expressions overflowing m. If the sum is negative we add m and we are done.
I am refreshing on Master Theorem a bit and I am trying to figure out the running time of an algorithm that solves a problem of size n by recursively solving 2 subproblems of size n-1 and combine solutions in constant time.
So the formula is:
T(N) = 2T(N - 1) + O(1)
But I am not sure how can I formulate the condition of master theorem.
I mean we don't have T(N/b) so is b of the Master Theorem formula in this case b=N/(N-1)?
If yes since obviously a > b^k since k=0 and is O(N^z) where z=log2 with base of (N/N-1) how can I make sense out of this? Assuming I am right so far?
ah, enough with the hints. the solution is actually quite simple. z-transform both sides, group the terms, and then inverse z transform to get the solution.
first, look at the problem as
x[n] = a x[n-1] + c
apply z transform to both sides (there are some technicalities with respect to the ROC, but let's ignore that for now)
X(z) = (a X(z) / z) + (c z / (z-1))
solve for X(z) to get
X(z) = c z^2 / [(z - 1) * (z-a)]
now observe that this formula can be re-written as:
X(z) = r z / (z-1) + s z / (z-a)
where r = c/(1-a) and s = - a c / (1-a)
Furthermore, observe that
X(z) = P(z) + Q(z)
where P(z) = r z / (z-1) = r / (1 - (1/z)), and Q(z) = s z / (z-a) = s / (1 - a (1/z))
apply inverse z-transform to get that:
p[n] = r u[n]
and
q[n] = s exp(log(a)n) u[n]
where log denotes the natural log and u[n] is the unit (Heaviside) step function (i.e. u[n]=1 for n>=0 and u[n]=0 for n<0).
Finally, by linearity of z-transform:
x[n] = (r + s exp(log(a) n))u[n]
where r and s are as defined above.
so relabeling back to your original problem,
T(n) = a T(n-1) + c
then
T(n) = (c/(a-1))(-1+a exp(log(a) n))u[n]
where exp(x) = e^x, log(x) is the natural log of x, and u[n] is the unit step function.
What does this tell you?
Unless I made a mistake, T grows exponentially with n. This is effectively an exponentially increasing function under the reasonable assumption that a > 1. The exponent is govern by a (more specifically, the natural log of a).
One more simplification, note that exp(log(a) n) = exp(log(a))^n = a^n:
T(n) = (c/(a-1))(-1+a^(n+1))u[n]
so O(a^n) in big O notation.
And now here is the easy way:
put T(0) = 1
T(n) = a T(n-1) + c
T(1) = a * T(0) + c = a + c
T(2) = a * T(1) + c = a*a + a * c + c
T(3) = a * T(2) + c = a*a*a + a * a * c + a * c + c
....
note that this creates a pattern. specifically:
T(n) = sum(a^j c^(n-j), j=0,...,n)
put c = 1 gives
T(n) = sum(a^j, j=0,...,n)
this is geometric series, which evaluates to:
T(n) = (1-a^(n+1))/(1-a)
= (1/(1-a)) - (1/(1-a)) a^n
= (1/(a-1))(-1 + a^(n+1))
for n>=0.
Note that this formula is the same as given above for c=1 using the z-transform method. Again, O(a^n).
Don't even think about Master's Theorem. You can only use Masther's Theorem when you're given master's theorem when b > 1 from the general form T(n) = aT(n/b) + f(n).
Instead, think of it this way. You have a recursive call that decrements the size of input, n, by 1 at each recursive call. And at each recursive call, the cost is constant O(1). The input size will decrement until it reaches 1. Then you add up all the costs that you used to make the recursive calls.
How many are they? n. So this would take O(2^n).
Looks like you can't formulate this problem in terms of the Master Theorem.
A good start is to draw the recursion tree to understand the pattern, then prove it with the substitution method. You can also expand the formula a couple of times and see where it leads.
See also this question which solves 2 subproblems instead of a:
Time bound for recursive algorithm with constant combination time
May be you could think of it this way
when
n = 1, T(1) = 1
n = 2, T(2) = 2
n = 3, T(3) = 4
n = 4, T(4) = 8
n = 5, T(5) = 16
It is easy to see that this is a geometric series 1 + 2+ 4+ 8 + 16..., the sum of which is
first term (ratio^n - 1)/(ratio - 1). For this series it is
1 * (2^n - 1)/(2 - 1) = 2^n - 1.
The dominating term here is 2^n, therefore the function belongs to Theta(2^n). You could verify it by doing a lim(n->inf) [2^n / (2^n - 1)] = +ve constant.
Therefore the function belongs to Big Theta (2^n)
Is there faster algo to calculate (n! modulo m).
faster than reduction at every multiplication step.
And also
Is there faster algo to calculate (a^p modulo m) better than right-left binary method.
here is my code:
n! mod m
ans=1
for(int i=1;i<=n;i++)
ans=(ans*i)%m;
a^p mod m
result=1;
while(p>0){
if(p%2!=0)
result=(result*a)%m;
p=(p>>1);
a=(a*a)%m;
}
Now the a^n mod m is a O(logn), It's the Modular Exponentiation Algorithm.
Now for the other one, n! mod m, the algorithm you proposed is clearly O(n), So obviously the first algorithm is faster.
The standard trick for computing a^p modulo m is to use successive square. The idea is to expand p into binary, say
p = e0 * 2^0 + e1 * 2^1 + ... + en * 2^n
where (e0,e1,...,en) are binary (0 or 1) and en = 1. Then use laws of exponents to get the following expansion for a^p
a^p = a^( e0 * 2^0 + e1 * 2^1 + ... + en * 2^n )
= a^(e0 * 2^0) * a^(e1 * 2^1) * ... * a^(en * 2^n)
= (a^(2^0))^e0 * (a^(2^1))^e1 * ... * (a^(2^n))^en
Remember that each ei is either 0 or 1, so these just tell you which numbers to take. So the only computations that you need are
a, a^2, a^4, a^8, ..., a^(2^n)
You can generate this sequence by squaring the previous term. Since you want to compute the answer mod m, you should do the modular arithmetic first. This means you want to compute the following
A0 = a mod m
Ai = (Ai)^2 mod m for i>1
The answer is then
a^p mod m = A0^e0 + A1^e1 + ... + An^en
Therefore the computation takes log(p) squares and calls to mod m.
I'm not certain whether or not there is an analog for factorials, but a good place to start looking would be at Wilson's Theorem. Also, you should put in a test for m <= n, in which case n! mod m = 0.
For the first computation, you should only bother with the mod operator if ans > m:
ans=1
for(int i=1;i<=n;i++) {
ans *= i;
if (ans > m) ans %= m;
}
For the second computation, using (p & 1) != 0 will probably be a lot faster than using p%2!=0 (unless the compiler recognizes this special case and does it for you). Then the same comment applies about avoiding the % operator unless necessary.
I am going through the above topic from CLRS(CORMEN) (page 834) and I got stuck at this point.
Can anybody please explain how the following expression,
A(x)=A^{[0]}(x^2) +xA^{[1]}(x^2)
follows from,
n-1 `
Σ a_j x^j
j=0
Where,
A^{[0]} = a_0 + a_2x + a_4a^x ... a_{n-2}x^{\frac{n}{2-1}}
A^{[1]} = a_1 + a_3x + a_5a^x ... a_{n-1}x^{\frac{n}{2-1}}
The polynomial A(x) is defined as
A(x) = a_0 + a_1 x + a_2 x^2 + a_3 x^3 + ...
To start the divide-and-conquer strategy of polynomial multiplication by the FFT, CLRS introduces two new polynomials: one of the coefficients of the even-powers of x called A[0] and one of the coefficients of the odd-powers of x called A[1]
A[0](x) = a_0 + a_2 x + a_4 x^2 + ...
A[1](x) = a_1 + a_3 x + a_5 x^2 + ...
Now if we substitute x^2 into A[0] and A[1], we have
A[0](x^2) = a_0 + a_2 x^2 + a_4 x^4 + ...
A[1](x^2) = a_1 + a_3 x^2 + a_5 x^4 + ...
and if we multiply A[1](x^2) by x, we have
x A[1](x^2) = a_1 x + a_3 x^3 + a_5 x^5 + ...
Now if we add A[0](x^2) and x A[1](x^2), we have
A[0](x^2) + x A[1](x^2) = (a_0 + a_2 x^2 + a_4 x^4 + ...) + (a_1 x + a_3 x^3 + a_5 x^5 + ...)
= a_0 + a_1 x + a_2 x^2 + a_3 x^3 + ...
= A(x)
Q.E.D.
If you divvy the polynomial up into "odd exponents" and "even exponents", you'll find the annoying fact that the A[1] polynomial (the one with odd exponents) has, well odd exponents! Even exponents are easier to work with, for FFT. So, one can simply factor out a single "x" from all of the values in A[1], and move it outside of the expression.
FFT likes working with even-exponented polynomials only. Thus, when you're dividing-and-conquering, you want to turn your A[1] expression into an "even-exponented" polynomial, and recurse on that, and then multiply-back-in that x. You will see that occur in the inner loop of the actual algorithm.
Edit: I realize that your confusion may stem from the fact that they're "passing in" (x^2) as the value in the polynomial. The "x" in A[1] and A[0] are different from the x in the (x^2) expression. You'll see how that must be, as while the original polynomial A goes up to exponent N, A[1] and A[0] both only go up to exponent (N/2).
I'm not going to answer your question because I feel that previous people have answered it. What I will do is try to explain the purpose of the FFT.
First, the FFT is a way to compute the convolution between two vectors. That is, suppose x = and y= are 1xn vectors then the convolution of x and y is
\sum_{i=0} ^n {xi y{n-i}}.
You will have to accept the fact that computing that value is EXTREMELY useful in a wide range of applications.
Now consider the following.
Suppose we construct two polynomials
A(z) = x0 + x1*z + x2 *z^2 + .. + xn^z^n
B(z) = y0 + y1*z + y2 *z^2 + .. + yn^z^n
then the multiplication is
AB(z) = A(z)B(z) = \sum_{i=0} ^ n (\sum_{k=0} ^ i xk*y{i-k}) z^i
where the inside sum is clearly a convolution of different size for different values of k.
Now we can clearly compute the coefficients (convolutions) of AB in n^2 time by a brute force method.
However, we can also be much more clever. Consider the fact that any polynomial of degree n can be described uniquely by n+1 points. That is given n+1 points we can construct the unique polynomial of degree n that goes through all n+1 points. Further more consider 2 polynomials in the form of n+1 points. You can compute their product by simply multiplying the n+1 y-values and keeping the x-values to result in their product in point-form. Now given a polynomial in n+1 point-form you can find the unique polynomial that describes it in O(n) time (actually Im not sure about this, it may be O(nlogn) time but certainly not more.)
This is exactly what the FFT does. However, the points that it picks to get the n+1 points to described the polynomials A and B are VERY carefully chosen. Some of the points are indeed complex because it just so happens that you can save time in evaluating a Polynomial by considering such points. That is if you were to choose just real points instead of the carefully chosen points that the FFT uses you would need O(n^2) time to evaluate the n+1 points. If you choose the FFT you only need O(nlogn) time. And thats all there is to the FFT. Oh and there is a unique side effect to the way that the FFT chooses points. Given an n-th degree polynomial, you must choose 2^m points where m is chosen such that 2^m is the smallest power of 2 greater than or equal to n.
A(x) is broken in to even x^2, and odd x parts,
for example if A(x) = 21 x^5 + 17 x^4 + 33 x^3 + 4 x^2 + 8 x + 7
then A0 = 17 y^2 + 4 y + 7
so that A0(x^2) = 17 x^4 + 4 x^2 + 7
and A1 = 21 y^2 + 33 y + 8
so that A1(x^2) = 21 x^4 + 33 x^2 + 8
or x * A1(x^2) = 21 x^5 + 33 x^3 + 8 x
clearly, in this case, A(x) = A0(x^2) + x A1(x^2) = even + odd parts