I think this is more programming than math, so I posted here.
All the java algorithms in my question come from here.
We have an iterative and recursive merge sort. Both using the same merge function.
The professor teaching this lecture says that the critical operation for merge sort is the comparison.
So I came up with this formula for merge() based on compares:
>3n + 2
3: worst case compares through each loop.
n: amount of times loop will iterate.
2: the "test" compares.
The recursiveMergesort() has the base case compare plus the recursive calls for a total of:
>T(n/2) + 1 + 3n + 2 = T(n/2) + 3n + 3
The iterativeMergesort() simply has one loop that runs *n/2* times with a nested loop that runs n times. That leads me to this formula (but I think it's wrong):
>(n/2) * n + 3n + 2 = (n^2)/2 + 3n + 2
The books say that the recurrence formula for the recursive mergesort is
2T(n/2) + theta(n)
Which solves with the master method to
theta(NlogN)
Question 1:
How are the formulas I created simplified to
T(n/2) + theta(n)
Question 2:
Can I use any of these formulas (the ones I created, the textbook formula, or the time complexity *theta(nlogn)*) to predict number of compares when running this particular algorithm on an array size n
Question 3:
For the bonus: Is my formula for the iterative method correct?
Merge:
private static void merge(int[] a, int[] aux, int lo, int mid, int hi) {
// DK: add two tests to first verify "mid" and "hi" are in range
if (mid >= a.length) return;
if (hi > a.length) hi = a.length;
int i = lo, j = mid;
for (int k = lo; k < hi; k++) {
if (i == mid) aux[k] = a[j++];
else if (j == hi) aux[k] = a[i++];
else if (a[j] < a[i]) aux[k] = a[j++];
else aux[k] = a[i++];
}
// copy back
for (int k = lo; k < hi; k++)
a[k] = aux[k];
}
Recursive Merge sort:
public static void recursiveMergesort(int[] a, int[] aux, int lo, int hi) {
// base case
if (hi - lo <= 1) return;
// sort each half, recursively
int mid = lo + (hi - lo) / 2;
recursiveMergesort(a, aux, lo, mid);
recursiveMergesort(a, aux, mid, hi);
// merge back together
merge(a, aux, lo, mid, hi);
}
public static void recursiveMergesort(int[] a) {
int n = a.length;
int[] aux = new int[n];
recursiveMergesort(a, aux, 0, n);
}
Iterative merge sort:
public static void iterativeMergesort(int[] a) {
int[] aux = new int[a.length];
for (int blockSize=1; blockSize<a.length; blockSize*=2)
for (int start=0; start<a.length; start+=2*blockSize)
merge(a, aux, start, start+blockSize, start+2*blockSize);
}
Wow, you made it all the way down here. Thanks!
Question 1:
Where are you getting your facts? To obtain theta(nlogn) complexity you need
T(n) = a T(n/b) + f(n), where a > 1, b > 1 and f(n) = cn + d. c != 0
Note: There are additional constraints, dictated by the Master theorem
You cannot derive from reccurence relation based on T(n) > T(n/2) + 3n + 3. You probably forgot that the cost of an array of size n is the cost of the merge plus twice the cost of each part. So rather
T(n) = 2T(n/2) + 3n + 3
Question 2:
You cannot use theta, Big O or Big Omega to predict number of compares when running this particular algorithm on an array size n. Because they are asymptotic expressions. You need to solve the relation above, assuming it is correct.
For instance T(n) = 2T(n/2) + 3n + 3 has the solution
T(n) = 3n log2(n) + 1/2(c+6)n - 3, c constant
Still this is the number of comparisons of the algorithm. All optimizations and constraints of a real program are not considered.
Question 3:
Nope
Related
This question already has answers here:
Big O, how do you calculate/approximate it?
(24 answers)
Closed 1 year ago.
what is the Big O for below series
1+2+3+4+....+N
if I've to write a code for the series, it will be like
public void sum(int n){
int sum =0;
for(int i=1;i<=n;i++){
sum += i;
}
print(sum);
}
based on the above code its O(N)
Somewhere (in a udemy course) I read the order of the series is O(N square). why?
This below code has runtime O(N).
public void sum(int n){
int sum =0;
for(int i=1;i<=n;i++){
sum=+i;
}
print(sum);
}
However
O(1+2+3+...N) is O(N^2) since O(1+2+3+...N)=O(N(N+1)/2)=O(N^2).
I am guessing you are reading about the second statement and you confuse the two.
You are confusing the complexity of computing 1 + 2 + ... + N (by summing) with the result of computing it.
Consider the cost function f(N) = 1 + 2 + ... + N. That simplifies to N(N + 1)/2, and has the complexity O(N^2).
(I expect that you learned that sum in your high school maths course. They may even have even taught you how to prove it ... by induction.)
On the other hand, the algorithm
public void sum(int n){
int sum = 0;
for(int i = 1; i <= n; i++){
sum += i;
}
print(sum);
}
computes 1 + 2 + ... + N by performing N additions. When do a full analysis of the algorithm taking into account all of the computation, the cost function will be in the complexity class O(N).
But I can also compute 1 + 2 + ... + N with a simpler algorithm that makes use of our match knowledge:
public void sum(int n){
print(n * (n + 1) / 2);
}
This alternative algorithm's cost function is O(1) !!!
Lesson: don't confuse an algorithm's cost function with its result.
I want to calculate the time complexity of the following recursive algorithm, where,
n = j - i (size of array)
i ≤ k ≤ j
process(A, i, j) takes Θ(n) time
Algo(Array A[], int i, int j)
if (i<j)
k = process(A, i, j)
Algo(A, i, k)
Algo(A, k+1, j)
I came up with the following:
T(n) = Θ(n) + T(k) + T(n-k)
I'm not sure if that is correct and if it is, how to proceed from that?
Update:
Is the following correct?
The worst case value for k is i or j,
T(n) = Θ(n) + T(0) + T(n)
This recursive structure is more like quick sort whose worst-case running time is O(n^2).
algorithm quicksort(A, lo, hi) is
if lo < hi then
p := partition(A, lo, hi)
quicksort(A, lo, p – 1)
quicksort(A, p + 1, hi)
The partition algorithm takes O(N) time. For analysis please refer Quick-Sort. I can't explain any better.
I am looking for an efficient algorithm of the problem, for any N find all i and j such that N=i^j.
I can solve it of O(N^2) as follows,
for i=1 to N
{
for j=1 to N
{
if((Power(i,j)==N)
print(i,j)
}
}
I am looking for better algorithm(or program in any language)if possible.
Given that i^j=N, you can solve the equation for j by taking the log of both sides:
j log(i) = log(N) or j = log(N) / log(i). So the algorithm becomes
for i=2 to N
{
j = log(N) / log(i)
if((Power(i,j)==N)
print(i,j)
}
Note that due to rounding errors with floating point calculations, you might want to check j-1 and j+1 as well, but even so, this is an O(n) solution.
Also, you need to skip i=1 since log(1) = 0 and that would result in a divide-by-zero error. In other words, N=1 needs to be treated as a special case. Or not allowed, since the solution for N=1 is i=1 and j=any value.
As M Oehm pointed out in the comments, an even better solution is to iterate over j, and compute i with pow(n,1.0/j). That reduces the time complexity to O(logN), since the maximum value for j is log2(N).
Here is a method you can use.
Lets say you have to solve an equation:
a^b = n //b and n are known
You can find this using binary search. If, you get a condition such that,
x^b < n and (x+1)^b > n
Then, no pair (a,b) exists such that a^b = n.
If you apply this method in range for b from 1..log(n), you should get all possible pairs.
So, complexity of this method will be: O(log n * log n)
Follow these steps:
function ifPower(n,b)
min=1, max=n;
while(min<max)
mid=min + (max-min)/2
k = mid^b, l = (mid + 1)^b;
if(k == n)
return mid;
if(l == n)
return mid + 1;
if(k < n && l > n)
return -1;
if(k < n)
max = mid - 1;
else
min = mid + 2; //2 as we are already checking for mid+1
return -1;
function findAll(n)
s = log2(n)
for i in range 2 to s //starting from 2 to ignore base cases, powers 0,1...you can handle them if required
p = ifPower(n,i)
if(b != -1)
print(b,i)
Here, in the algorithm, a^b means a raised to power of b and not a xor b (its obvs, but just saying)
Consider the following function:
int testFunc(int n){
if(n < 3) return 0;
int num = 7;
for(int j = 1; j <= n; j *= 2) num++;
for(int k = n; k > 1; k--) num++;
return testFunc(n/3) + num;
}
I get that the first loop is O(logn) while the second loop gives O(n) which gives a time complexity of O(n) in total. But due to the recursive calls I thought the time complexity would be O(nlogn), but apperantly it is only O(n). Can anyone explain why?
The recursive call pretty much gives the following for the complexity(denoting the complexity for input n by T(n)):
T(n) = log(n) + n + T(n/3)
First observation as you correctly noted is that you can ignore the logarithm as it is dominated by n. Now we are only left with T(n) = n + T(n/3). Try writing this up to 0 for instance. We have:
T(n) = n + n/3 + n/9+....
You can easily prove that the above sum is always less than 2*n. In fact better limits can be proven but this one is enough to state that overall complexity is O(n).
For procedures using a recursive algorithm such as the following:
procedure T( n : size of problem ) defined as:
if n < base_case then exit
Do work of amount f(n) // In this case, the O(n) for loop
T(n/b)
T(n/b)
... a times... // In this case, b = 3, and a = 1
T(n/b)
end procedure
Applying the Master theorem to find the time complexity, the f(n) in this case is O(n) (due to the second for loop, like you said). This makes c = 1.
Now, logba = log31 = 0, making this the 3rd case of the theorem, according to which the time complexity T(n) = Θ(f(n)) = Θ(n).
I have this problem that I can't solve.. what is the complexity of this foo algorithm?
int foo(char A[], int n, int m){
int i, a=0;
if (n>=m)
return 0;
for(i=n;i<m;i++)
a+=A[i]
return a + foo(A, n*2, m/2);
}
the foo function is called by:
foo(A,1,strlen(A));
so.. I guess it's log(n) * something for the internal for loop.. which I'm not sure if it's log(n) or what..
Could it be theta of log^2(n)?
This is a great application of the master theorem:
Rewrite in terms of n and X = m-n:
int foo(char A[], int n, int X){
int i, a=0;
if (X < 0) return 0;
for(i=0;i<X;i++)
a+=A[i+n]
return a + foo(A, n*2, (X-3n)/2);
}
So the complexity is
T(X, n) = X + T((X - 3n)/2, n*2)
Noting that the penalty increases with X and decreases with n,
T(X, n) < X + T(X/2, n)
So we can consider the complexity
U(X) = X + U(X/2)
and plug this into master theorem to find U(X) = O(X) --> complexity is O(m-n)
I'm not sure if there's a 'quick and dirty' way, but you can use old good math. No fancy theorems, just simple equations.
On k-th level of recursion (k starts from zero), a loop will have ~ n/(2^k) - 2^k iterations. Therefore, the total amount of loop iterations will be S = sum(n/2^i) - sum(2^i) for 0 <= i <= l, where l is the depth of recursion.
The l will be approximately log(2, n)/2 (prove it).
Transforming each part in formula for S separately, we get.
S = (1 + 2 + .. + 2^l)*n/2^l - (2^(l + 1) - 1) ~= 2*n - 2^(l + 1) ~= 2*n - sqrt(n)
Since each other statement except loop will be repeated only l times and we know that l ~= log(2, n), it won't affect complexity.
So, in the end we get O(n).