I have a recursive algorithm like that which computes the smallest integer in the array.
ALGORITHM F_min1(A[0..n-1])
//Input: An array A[0..n-1] of real numbers
If n = 1
return A[0]
else
temp ← F_minl(A[0..n-2])
If temp ≤ A[n-1]
return temp
else
return A[n-1]
What I think that the reccurence relation should be
T(n)=T(n-1) + n
However I am not sure about the + n part. I want to be sure in which cases the recurrence is T(n)=T(n-1) + 1 and in which cases the recurrence is T(n)=T(n-1) + n.
The recurrence should be
T(1) = 1,
T(n) = T(n-1) + 1
because besides the recursive call to the smaller array, all computational effort (reading the last entry of A and doing the comparison) take constant time in unit cost measure. The algorithm can be understood as divide-and-conquer where the divide part is splitting the array into a prefix and the last entry; the conquer part, which is a comparison, cannot take more than constant time here. In total, a case where there is linear work after the recursive call does not exist.
Related
Alright I have some difficulty completely understanding recurrence relations.
So for example if I'm analyzing quick sort in the worst case using Θ-notation, giving the array an input of unsorted positive numbers;
When base case n <= 3 I sort the small array using insertionsort.
Time: O(1) or O(n^2)?
I use linear search to set the pivot as the median of all elements.
Time: Θ(n)
Partitioning left and right of pivot and performing recursion.
Time: 2 * T(n/2) I think
would the recurrence be:
T(n) = O(1) + Θ(n) + 2 * T(n/2) then?
Something tells me the base case would instead take O(n^2) time because if the input is small enough that would be the worst case.
Would that then give me the recurrence:
T(n) = O(n^2) + Θ(n) + 2 * T(n/2)?
When base case n <= 3 I sort the small array using insertionsort.
always O(1)
I use linear search to set the pivot as the median of all elements.
You may want to clarify this more, to find the median as pivot is not a simple task of doing linear search. The few ways are i) using
Quick Select Algorithm (average O(N)) or ii) sorting the subpartition
O(NlogN) iii) median-of-median algorithm O(N).
Partitioning left and right of pivot and performing recursion.
2F(N/2) + N
Putting it all together (Assuming that the pivot is always the median & that you take O(N) to find the pivot each time):
Best_Case = Worst_Case (since the pivot is always median)
F(3) = F(2) = F(1) = 1
F(n) = 2F(N/2) + 2N
= 2(2F(N/2^2) + 2(N/2)) + 2N
= 2(2)F(N/2^2) + 2N + 2N
= 2(3)F(N/2^3) + 3(2)N
= 2(LogN)F(N/N) + (2N)LogN
= O(NlogN)
I have two pseudo-code algorithms:
RandomAlgorithm(modVec[0 to n − 1])
b = 0;
for i = 1 to n do
b = 2.b + modVec[n − i];
for i = 1 to b do
modVec[i mod n] = modVec[(i + 1) mod n];
return modVec;
Second:
AnotherRecursiveAlgo(multiplyVec[1 to n])
if n ≤ 2 do
return multiplyVec[1] × multiplyVec[1];
return
multiplyVec[1] × multiplyVec[n] +
AnotherRecursiveAlgo(multiplyVec[1 to n/3]) +
AnotherRecursiveAlgo(multiplyVec[2n/3 to n]);
I need to analyse the time complexity for these algorithms:
For the first algorithm i got the first loop is in O(n),the second loop has a best case and a worst case , best case is we have O(1) the loop runs once, the worst case is we have a big n on the first loop, but i don't know how to write this idea as a time complexity cause i usually get b=sum(from 1 to n-1) of 2^n-1 . modVec[n-1] and i get stuck here.
For the second loop i just don't get how to solve the time complexity of this one, we usually have it dependant on n , so we need the formula i think.
Thanks for the help.
The first problem is a little strange, all right.
If it helps, envision modVec as an array of 1's and 0's.
In this case, the first loop converts this array to a value.
This is O(n)
For instance, (1, 1, 0, 1, 1) will yield b = 27.
Your second loop runs b times. The dominating term for the value of b is 2^(n-1), a.k.a. O(2^n). The assignment you do inside the loop is O(1).
The second loop does depend on n. Your base case is a simple multiplication, O(1). The recursion step has three terms:
simple multiplication
recur on n/3 elements
recur on n/3 elements (from 2n/3 to the end is n/3 elements)
Just as your binary partitions result in log[2] complexities, this one will result in log[3]. The base doesn't matter; the coefficient (two recursive calls) doesn't' matter. 2*O(log3) is still O(log N).
Does that push you to a solution?
First Loop
To me this boils down to the O(First-For-Loop) + O(Second-For-Loop).
O(First-For-Loop) is simple = O(n).
O(Second-For-Loop) interestingly depends on n. Therefore, to me it's can be depicted as O(f(n)), where f(n) is some function of n. Not completely sure if I understand the f(n) based on the code presented.
The answer consequently becomes O(n) + O(f(n)). This could boil down to O(n) or O(f(n)) depending upon which one is larger and more dominant (since the lower order terms don't matter in the big-O notation.
Second Loop
In this case, I see that each call to the function invokes 3 additional calls...
The first call seems to be an O(1) call. So it won't matter.
The second and the third calls seems to recurses the function.
Therefore each function call is resulting in 2 additional recursions.
Consequently , the time complexity on this would be O(2^n).
I have a question on forming recurrence relations and calculating the time complexity.
If we have a recurrence relation T(n)=2T(n/2) + c then it means that the constant amount of work c is divided into 2 parts T(n/2) + T(n/2) when drawing recursion tree.
Now consider recurrence relation of factorial which is T(n)=n*T(n-1) + c . If we follow the above method then we should divide the work c into n instances each of T(n-1) and then evaluate time complexity. However if calculate in this way then answer will O(n^n) because we will have O(n^n) recursive calls which is wrong.
So my question is why can't we use the same approach of dividing the elements into sub parts as in first case.
Let a recurrence relation be T(n) = a * T(n/b) + O(n).
This recurrence implies that there is a recursive function which:
divides the original problem into a subproblems
the size of each subproblem will be n/b if the current problem size is n
when the subproblems are trivial (too easy to solve), no recursion is needed and they are solved directly (and this process will take O(n) time).
When we say that original problem is divided into a subproblems, we mean that there are a recursive calls in the function body.
So, for instance, if the function is:
int f(int n)
{
if(n <= 1)
return n;
return f(n-1) + f(n-2);
}
we say that the problem (of size n) is divided into 2 subproblems, of sizes n-1 and n-2. The recurrence relation would be T(n) = T(n-1) + T(n-2) + c. This is because there are 2 recursive calls, and with different arguments.
But, if the function is like:
int f(int n)
{
if(n <= 2)
return n;
return n * f(n-1);
}
we say that the problem (of size n) is divided into only 1 subproblem, which is of size n-1. This is because there is only 1 recursive call.
So, the recurrence relation would be T(n) = T(n-1) + c.
If we multiply the T(n-1) with n, as would seem normal, we are conveying that there were n recursive calls made.
Remember, our main motive for forming recurrence relations is to perform (asymptotic) complexity analysis of recursive functions. Even though it would seem like n cannot be discarded from the relation as it depends on the input size, it would not serve the same purpose as it does in the function itself.
But, if you are talking about the value returned by the function, it would be f(n) = n * f(n-1). Here, we are multiplying with n because it is an actual value, that will be used in the computation.
Now, coming to the c in T(n) = T(n-1) + c; it merely suggests that when we are solving a problem of size n, we need to solve a smaller problem of size n-1 and some other constant (constant time) work like comparison, multiplication and returning values are also performed.
We can never divide "constant amount of work c" into two parts T(n/2) and T(n/2), even using recursive tree method. What we are, in fact, dividing, is the problem into two halves. The same "c" amount of work will be needed in each recursive call in each level of the recursive tree.
If there were a recurrence relation like T(n) = 2T(n/2) + O(n), where the amount of work to be done depends on the input size, then the amount of work to be done at each level will be halved at the next level, just like you described.
But, if the recurrence relation were like T(n) = T(n-1) + O(n), we would not be dividing the amount of work into two halves in the next recursion level. We would just be reducing the amount of work by one at each successive level (n-sized problem becomes n-1 at next level).
To check how the amount of work will change with recursion, apply substitution method to your recurrence relation.
I hope I have answered your question.
The question is :
UNBALANCED MERGE SORT
is a sorting algorithm, which is a modified version of
the standard MERGE SORT
algorithm. The only difference is that instead of dividing
the input into 2 equal parts in each stage, we divide it into two unequal parts – the first
2/5 of the input, and the other 3/5.
a. Write the recurrence relation for the worst case time complexity of the
UNBALANCED MERGE SORT
algorithm.
b. What is the worst case time complexity of the UNBALANCEDMERGESORT
algorithm? Solve the recurrence relation from the previous section.
So i'm thinkin the recurrence relation is : T(n) <= T(2n/5) + T(3n/5) + dn.
Not sure how to solve it.
Thanks in advance.
I like to look at it as "runs", where the ith "run" is ALL the recursive steps with depth exactly i.
In each such run, at most n elements are being processed (we will prove it soon), so the total complexity is bounded by O(n*MAX_DEPTH), now, MAX_DEPTH is logarithmic, as in each step the bigger array is size 3n/5, so at step i, the biggest array is of size 3^i/5^i * n.
Sovle the equation:
3^i/5^i * n = 1
and you will find out that i = log_a(n) - for some base a
So, let's be more formal:
Claim:
Each element is being processed by at most one recursive call at depth
i, for all values of i.
Proof:
By induction, at depth 0, all elements are processed exactly once by the first call.
Let there be some element x, and let's have a look on it at step i+1. We know (induction hypothesis) that x was processed at most once in depth i, by some recursive call. This call later invoked (or not, we claim at most once) the recursive call of depth i+1, and sent the element x to left OR to right, never to both. So at depth i+1, the element x is proccessed at most once.
Conclusion:
Since at each depth i of the recursion, each element is processed at most once, and the maximal depth of the recursion is logarithmic, we get an upper bound of O(nlogn).
We can similarly prove a lower bound of Omega(nlogn), but that is not needed, since sorting is already an Omega(nlogn) problem - so we can conclude the modified algorithm is still Theta(nlogn).
If you want to prove it with "basic arithmetics", it can also be done, by induction.
Claim: T(n) = T(3n/5) + T(2n/5) + n <= 5nlog(n) + n
It will be similar when replacing +n with +dn, I simplified it, but follow the same idea of proof with T(n) <= 5dnlogn + dn
Proof:
Base: T(1) = 1 <= 1log(1) + 1 = 1
T(n) = T(3n/5) + T(2n/5) + n
<= 5* (3n/5) log(3n/5) +3n/5 + 5*(2n/5)log(2n/5) +2n/5 + n
< 5* (3n/5) log(3n/5) + 5*(2n/5)log(3n/5) + 2n
= 5*nlog(3n/5) + 2n
= 5*nlog(n) + 5*nlog(3/5) + 2n
(**)< 5*nlog(n) - n + 2n
= 5nlog(n) + n
(**) is because log(3/5)~=-0.22, so 5nlog(3/5) < -n, and 5nlog(3/5) + 2n < n
I am trying to prove the following worst-case scenario for the Quicksort algorithm but am having some trouble. Initially, we have an array of size n, where n = ij. The idea is that at every partition step of Quicksort, you end up with two sub-arrays where one is of size i and the other is of size i(j-1). i in this case is an integer constant greater than 0. I have drawn out the recursive tree of some examples and understand why this is a worst-case scenario and that the running time will be theta(n^2). To prove this, I've used the iteration method to solve the recurrence equation:
T(n) = T(ij) = m if j = 1
T(n) = T(ij) = T(i) + T(i(j-1)) + cn if j > 1
T(i) = m
T(2i) = m + m + c*2i = 2m + 2ci
T(3i) = m + 2m + 2ci + 3ci = 3m + 5ci
So it looks like the recurrence is:
j
T(n) = jm + ci * sum k - 1
k=1
At this point, I'm a bit lost as to what to do. It looks the summation at the end will result in j^2 if expanded out, but I need to show that it somehow equals n^2. Any explanation on how to continue with this would be appreciated.
Pay attention, the quicksort algorithm worst case scenario is when you have two subproblems of size 0 and n-1. In this scenario, you have this recurrence equations for each level:
T(n) = T(n-1) + T(0) < -- at first level of tree
T(n-1) = T(n-2) + T(0) < -- at second level of tree
T(n-2) = T(n-3) + T(0) < -- at third level of tree
.
.
.
The sum of costs at each level is an arithmetic serie:
n n(n-1)
T(n) = sum k = ------ ~ n^2 (for n -> +inf)
k=1 2
It is O(n^2).
Its a problem of simple mathematics. The complexity as you have calculated correctly is
O(jm + ij^2)
what you have found out is a parameterized complextiy. The standard O(n^2) is contained in this as follows - assuming i=1 you have a standard base case so m=O(1) hence j=n therefore we get O(n^2). if you put ij=n you will get O(nm/i+n^2/i) . Now what you should remember is that m is a function of i depending upon what you will use as the base case algorithm hence m=f(i) thus you are left with O(nf(i)/i + n^2/i). Now again note that since there is no linear algorithm for general sorting hence f(i) = omega(ilogi) which will give you O(nlogi + n^2/i). So you have only one degree of freedom that is i. Check that for any value of i you cannot reduce it below nlogn which is the best bound for comparison based.
Now what I am confused is that you are doing some worst case analysis of quick sort. This is not the way its done. When you say worst case it implies you are using randomization in which case the worst case will always be when i=1 hence the worst case bound will be O(n^2). An elegant way to do this is explained in randomized algorithm book by R. Motwani and Raghavan alternatively if you are a programmer then you look at Cormen.