can anyone tell me what exactly is the time complexity of this Heap's algorithm shown in wikipedia, https://en.wikipedia.org/wiki/Heap%27s_algorithm ?
I searched several websites and the answers are all vague, some of them say the time complexity is O(N!) some of them say it's O(NlogN). Which one is the correct answer? And why?
Thank you.
There are N! permutations in all and generating all of them requires Θ(N!) time and Θ(N) space. In other words, each permutation requires amortised Θ(1) time.
Those facts can be derived from the recursive algorithm presented on the Wikipedia page. In essence, the code alternates swaps and outputs so each output involves a single swap.
However, there are also call operations and loop tests. There is a single loop test before each call, so it is only necessary to count the total number of calls.
In the worst case, there will be n recursive calls before an output. But that only happens once, at the very beginning of the algorithm. A single call with argument n produces n! outputs. It does that with n recursive calls, each of which produces (n-1)! outputs, and does (n-1) recursive calls, so there are n(n-1) calls with argument n-2. And so on, so there are a total of 1 + n + n(n-1) + n(n-1)(n-2) + ... + n! calls.
That can be written as Σ0≤i≤nn!/i! or (Σ0≤i≤n1/i!)n! Or (e-1), which is approximately 1.71828 n!
I think you are confusing between the Heap's algorithm and heapsort algorithm or heap data structure. The later two have O(NlogN) complexity for sorting.
The algorithm you mentioned is for generating all permutations, and because there are N! permutations for every N-element array, the complexity is O(N!).
Related
can anyone tell me what exactly is the time complexity of this Heap's algorithm shown in wikipedia, https://en.wikipedia.org/wiki/Heap%27s_algorithm ?
I searched several websites and the answers are all vague, some of them say the time complexity is O(N!) some of them say it's O(NlogN). Which one is the correct answer? And why?
Thank you.
There are N! permutations in all and generating all of them requires Θ(N!) time and Θ(N) space. In other words, each permutation requires amortised Θ(1) time.
Those facts can be derived from the recursive algorithm presented on the Wikipedia page. In essence, the code alternates swaps and outputs so each output involves a single swap.
However, there are also call operations and loop tests. There is a single loop test before each call, so it is only necessary to count the total number of calls.
In the worst case, there will be n recursive calls before an output. But that only happens once, at the very beginning of the algorithm. A single call with argument n produces n! outputs. It does that with n recursive calls, each of which produces (n-1)! outputs, and does (n-1) recursive calls, so there are n(n-1) calls with argument n-2. And so on, so there are a total of 1 + n + n(n-1) + n(n-1)(n-2) + ... + n! calls.
That can be written as Σ0≤i≤nn!/i! or (Σ0≤i≤n1/i!)n! Or (e-1), which is approximately 1.71828 n!
I think you are confusing between the Heap's algorithm and heapsort algorithm or heap data structure. The later two have O(NlogN) complexity for sorting.
The algorithm you mentioned is for generating all permutations, and because there are N! permutations for every N-element array, the complexity is O(N!).
What I think I understand.
In the worst case the quick-sort algorithm picks the largest or smallest key in list/array to be sorted for every recursive call(if recursive implementation). I understand that the size of n will determine both the number of recursive calls and the number of comparisons(which will decrease by 1 with every step of recursion). So we have n+(n-1)+(n-2)+...+2+1 number of primitive comparison in total.
What I don't understand.
The part I don't quite grasp is how this is O(n^2)? Like I know that it is at least O(n^2) as n+(n-1)+(n-2)+...+2+1 < n^2 but how do I know it isn't say O(n*logn)? Do I have to prove that result to safely confirm it is O(n^2) or is that immediately obvious in a way I can't see? I guess in general how do I know that I have found the lowest function that represents the big O time complexity.
n+(n-1)+(n-2)+...+1 = n(n+1)/2 (you can prove it by mathematical induction), which obviously is O(n^2).
https://en.wikipedia.org/wiki/1_%2B_2_%2B_3_%2B_4_%2B_%E2%8B%AF
I'm trying to find out which is the Theta complexity of this algorithm.
(a is a list of integers)
def sttr(a):
for i in xrange(0,len(a)):
while s!=[] and a[i]>=a[s[-1]]:
s.pop()
s.append(i)
return s
On the one hand, I can say that append is being executed n (length of a array) times, so pop too and the last thing I should consider is the while condition which could be executed probably 2n times at most.
From this I can say that this algorithm is at most 4*n so it is THETA(n).
But isn't it amortised analysis?
On the other hand I can say this:
There are 2 nested cycles. The for cycle is being executed exactly n times. The while cycle could be executed at most n times since I have to remove item in each iteration. So the complexity is THETA(n*n).
I want to compute THETA but don't know which of these two options is correct. Could you give me advice?
The answer is THETA(n) and your arguments are correct.
This is not amortized analysis.
To get to amortized analysis you have to look at the inner loop. You can't easily say how fast the while will execute if you ignore the rest of the algorithm. Naive approach would be O(N) and that's correct since that's the maximum number of iterations. However, since we know that the total number of executions is O(N) (your argument) and that this will be executed N time we can say that the complexity of the inner loop is O(1) amortized.
for i = 0 to size(arr)
for o = i + 1 to size(arr)
do stuff here
What's the worst-time complexity of this? It's not N^2, because the second one decreases by one every i loop. It's not N, it should be bigger. N-1 + N-2 + N-3 + ... + N-N+1.
It is N ^ 2, since it's the product of two linear complexities.
(There's a reason asymptotic complexity is called asymptotic and not identical...)
See Wikipedia's explanation on the simplifications made.
Think of it like you are working with a n x n matrix. You are approximately working on half of the elements in the matrix, but O(n^2/2) is the same as O(n^2).
When you want to determine the complexity class of an algorithm, all you need is to find the fastest growing term in the complexity function of the algorithm. For example, if you have complexity function f(n)=n^2-10000*n+400, to find O(f(n)), you just have to find the "strongest" term in the function. Why? Because for n big enough, only that term dictates the behavior of the entire function. Having said that, it is easy to see that both f1(n)=n^2-n-4 and f2(n)=n^2 are in O(n^2). However, they, for the same input size n, don't run for the same amount of time.
In your algorithm, if n=size(arr), the do stuff here code will run f(n)=n+(n-1)+(n-2)+...+2+1 times. It is easy to see that f(n) represents a sum of an arithmetic series, which means f(n)=n*(n+1)/2, i.e. f(n)=0.5*n^2+0.5*n. If we assume that do stuff here is O(1), then your algorithm has O(n^2) complexity.
for i = 0 to size(arr)
I assumed that the loop ends when i becomes greater than size(arr), not equal to. However, if the latter is the case, than f(n)=0.5*n^2-0.5*n, and it is still in O(n^2). Remember that O(1),O(n),0(n^2),... are complexity classes, and that complexity functions of algorithms are functions that describe, for the input size n, how many steps there is in the algorithm.
It's n*(n-1)/2 which is equal to O(n^2).
Comparison based sorting is big omega of nlog(n), so we know that mergesort can't be O(n). Nevertheless, I can't find the problem with the following proof:
Proposition P(n): For a list of length n, mergesort takes O(n) time.
P(0): merge sort on the empty list just returns the empty list.
Strong induction: Assume P(1), ..., P(n-1) and try to prove P(n). We know that at each step in a recursive mergesort, two approximately "half-lists" are mergesorted and then "zipped up". The mergesorting of each half list takes, by induction, O(n/2) time. The zipping up takes O(n) time. So the algorithm has a recurrence relation of M(n) = 2M(n/2) + O(n) which is 2O(n/2) + O(n) which is O(n).
Compare the "proof" that linear search is O(1).
Linear search on an empty array is O(1).
Linear search on a nonempty array compares the first element (O(1)) and then searches the rest of the array (O(1)). O(1) + O(1) = O(1).
The problem here is that, for the induction to work, there must be one big-O constant that works both for the hypothesis and the conclusion. That's impossible here and impossible for your proof.
The "proof" only covers a single pass, it doesn't cover the log n number of passes.
The recurrence only shows the cost of a pass as compared to the cost of the previous pass. To be correct, the recurrence relation should have the cumulative cost rather than the incremental cost.
You can see where the proof falls down by viewing the sample merge sort at http://en.wikipedia.org/wiki/Merge_sort
Here is the crux: all induction steps which refer to particular values of n must refer to a particular function T(n), not to O() notation!
O(M(n)) notation is a statement about the behavior of the whole function from problem size to performance guarantee (asymptotically, as n increases without limit). The goal of your induction is to determine a performance bound T(n), which can then be simplified (by dropping constant and lower-order factors) to O(M(n)).
In particular, one problem with your proof is that you can't get from your statement purely about O() back to a statement about T(n) for a given n. O() notation allows you to ignore a constant factor for an entire function; it doesn't allow you to ignore a constant factor over and over again while constructing the same function recursively...
You can still use O() notation to simplify your proof, by demonstrating:
T(n) = F(n) + O(something less significant than F(n))
and propagating this predicate in the usual inductive way. But you need to preserve the constant factor of F(): this constant factor has direct bearing on the solution of your divide-and-conquer recurrence!