Can you point out to me an example of a divide and conquer algorithm that runs in CONSTANT time! I'm in a "OMG! I can not think of any such thing" kind of situation. Point me to something please. Thanks
I know that an alg that follows the following recurence: T(n) = 2T(n/2) + n would be merge sort. We're dividing the problem into 2 subproblems - each of size n/2. Then we're taking n time to conquer everything back into one sorted array.
I also know that T(n) = T(n/2) + 1 would be binary search.
But what is T(n) = 1?
For a divide-and-conquer algorithm to run in constant time, it needs to do no more than a fixed amount of work on any input. Therefore, it can make at most a fixed number of recursive calls on any input, since if the number of calls was unbounded, the total work done would not be a constant. Moreover, it needs to do a constant amount of work across all those recursive calls.
This eliminates basically any reasonable-looking recurrence relation. Anything of the form
T(n) = aT(n / b) + O(nk)
is immediately out of the question, because the number of recursive calls would grow as a function of the input n.
You could make some highly contrived divide-and-conquer algorithms that run in constant time. For example, consider this problem:
Return the first element of the input array.
This could technically be solved with divide-and-conquer by noting that
The first element of a one-element array is equal to itself.
The first element of an n-element array is the first element of the subarray of just the first element.
The recurrence is then
T(n) = T(1) + O(1)
T(1) = 1
As you can see, this is a very odd-looking recurrence, but it does work out.
I've never heard of anything like this coming up in practice, but if I think of anything I'll try to update this answer with details. (A note: I'm not expecting to ever update this answer. ^_^)
Hope this helps!
Related
Assume that the size of the input problem increases with an integer n. Let T(n) be the time complexity of a divide-and-conquer algorithm to solve this problem. Then T(n) satisfies an equation of the form:
T(n) = a T(n/b) + f (n).
Now my question is: how can a and b be unequal?
It seems that they should be equal because the number of recursive calls must be equal to b (size of a sub-problem).
In software, time is often wasted on control operations, like function calls. So usually a > b.
Also, there are situations where the problem calls for just one "recursive call" (which would then be just iteration), for example, binary search. In these cases, a < b.
I'm currently taking an algorithm analysis course. One of the questions of a quiz was to write an algorithm with the runtime T(n) = 4T(3n/4) + n^2 where the algorithm does not have to do anything significant.
I couldn't find any similar examples so I'm unsure how to proceed.
To simplify how to think about this kind of problem, just use an array of n elements to represent a problem of size n.
Then, the running time T(n) represents the algorithm run on the array.
The running time 4T(3n/4) represents the algorithm run on 3 quarters of the array four times.
The running time n^2 represents some quadratic operation on the array (for example, a bubble sort).
silly_algo (arr[], n)
if n == 0 return
for i : 1..4
silly_algo(arr, 3*n/4)
bubblesort(arr, n)
I am following a course of Algorithms and Data Structures.
Today, my professor said the complexity of the following algorithm is 2n.
I waited till the lesson was over, approached him and told him I actually believed it was an O(n) algorithm, and I did the computation to prove it, and wanted to show them to it, but he continued to say it was not, without giving me any convincing explanation.
The algorithm is recursive, and it has this complexity:
{ 1 if n=1
T(n) = {
{ 2T(n/2) otherwise
I computed it down to be a O(n), this way:
Let's expand T(n)
T(n) = 2 [2 * T(n/(2^2))]
= 2^2 * T(n/(2^2))
= 2^2 * [2 * T(n/(2^3))]
= 2^3 * T(n/(2^3))
= ...
= 2^i * T(n/(2^i)).
We stop when the term inside the T is 1, that is:
n/(2i) = 1 ==> n = 2i ==> i = log n
After the substitution, we obtain
T(n) = 2^log n * T(1)
= n * 1
= O(n).
Since this algorithm jumped out of a lesson on Merge Sort, I noted how Merge Sort, which notoriously is O(n log n) has a complexity of 2T(n/2) + Θ(n) (obviously higher than 2T(n/2)), and I asked him why is it, that an algorithm with a lower complexity, gets a higher big-O. Because, at this point, it's counter intuitive for me. He replied, words for words, "If you think that is counter-intuitive, you have serious problem in your math."
My questions are:
Is there any fallacy in my demonstration?
Wouldn't the last situation be counter-intuitive?
Yes, this is also a vent.
Proof - 1
This recurrence falls in case - 3 of Master Theorem, with
a = 2;
b = 2; and,
c = -∞
and thus Logba = 1 which is bigger than -∞. Therefore the running time is Θ(n1) = Θ(n).
Proof - 2
Intuitively, you are breaking the problem of size n into 2 problems of size n/2 and the cost to join the result of two sub-problems is 0 (i.e. there is no constant component in the recurrence).
Hence at the bottom-most level you have n problems of cost 1 each, resulting in the running time of n * O(1) which is equal to O(n).
Edit: Just to complete this answer I will also add the answers to specific questions asked by you.
Is there any fallacy in my demonstration?
No. It is correct.
Wouldn't the last situation be counter-intuitive?
Definitely it is counter-intuitive. See the Proof-2 above.
You are correct in computing the time complexity of the given relation. If we are measuring the input size in n(which we should) then your professor is wrong in claiming that the time complexity is 2^n.
You should probably discuss it with him and clear any misunderstanding that you might have.
You are clearly correct that a function T(n) which satisfies that recurrence relation is O(n). It is essentially obvious since it says that the complexity of a given problem is twice that of a problem which is half the size. You can't get much more linear than that. For example -- the complexity of searching through a list of 1000 elements with a linear search is twice that of searching through a list with 500 elements.
If your professor is also correct then perhaps you are incorrect about the complexity satisfying that recurrence. Alternatively, sometimes there is some confusion about how the input size is being measured. For example, an integer n is exponential in the number of bits needed to specify it. For example -- brute force trial division of an integer n is O(sqrt(n)) which is much better than O(n). The reason that this doesn't contradict that fact that brute force factoring is essentially worthless for e.g. cracking RSA is because for say a 256 bit key the relevant n is around 2^256.
When calculate the median, we know that if we break the input array into subgroups as five and solve it recursively, we will get O(n) complexity, but if we break the array into 3, it won't return the O(n) complexity.
Does any one know how to prove it?
It' gonna be nlg(n) .
Try to draw it's recursion tree : The total cost of each level is equal to n, and the depth of this tree is lg(n) .
Note : Actually it's depth is between log(n) base 3 and log(n) base 3/2, but since the order of logarithms in all bases are same, we can just consider it as lg(n).
It looks like the recurrence in the title is wrong, but I think the Master Theorem for solving recurrences will be handy. You can show that going from one denominator to another puts you into a different case which goes from O(n) to something worse.
for i = 0 to size(arr)
for o = i + 1 to size(arr)
do stuff here
What's the worst-time complexity of this? It's not N^2, because the second one decreases by one every i loop. It's not N, it should be bigger. N-1 + N-2 + N-3 + ... + N-N+1.
It is N ^ 2, since it's the product of two linear complexities.
(There's a reason asymptotic complexity is called asymptotic and not identical...)
See Wikipedia's explanation on the simplifications made.
Think of it like you are working with a n x n matrix. You are approximately working on half of the elements in the matrix, but O(n^2/2) is the same as O(n^2).
When you want to determine the complexity class of an algorithm, all you need is to find the fastest growing term in the complexity function of the algorithm. For example, if you have complexity function f(n)=n^2-10000*n+400, to find O(f(n)), you just have to find the "strongest" term in the function. Why? Because for n big enough, only that term dictates the behavior of the entire function. Having said that, it is easy to see that both f1(n)=n^2-n-4 and f2(n)=n^2 are in O(n^2). However, they, for the same input size n, don't run for the same amount of time.
In your algorithm, if n=size(arr), the do stuff here code will run f(n)=n+(n-1)+(n-2)+...+2+1 times. It is easy to see that f(n) represents a sum of an arithmetic series, which means f(n)=n*(n+1)/2, i.e. f(n)=0.5*n^2+0.5*n. If we assume that do stuff here is O(1), then your algorithm has O(n^2) complexity.
for i = 0 to size(arr)
I assumed that the loop ends when i becomes greater than size(arr), not equal to. However, if the latter is the case, than f(n)=0.5*n^2-0.5*n, and it is still in O(n^2). Remember that O(1),O(n),0(n^2),... are complexity classes, and that complexity functions of algorithms are functions that describe, for the input size n, how many steps there is in the algorithm.
It's n*(n-1)/2 which is equal to O(n^2).