I know that the upper bound of heap sort is O(n log n), but how do I go about showing an example where it requires exactly n*logn steps? Thanks.
Related
To build a heap, it's frequently mistaken that O(n log n) is the strict upper bound, but it's actually O(n).
I want to say that popping all n elements from a heap is O(n log n), but based on what I know about complexity for building a heap, I'm hesitant, and I think it might be O(n) for the same reason building a heap is O(n).
Is this correct?
Each time we pop, we need to find a new root, and the heapify function will take O(h) time where h is the height of the binary tree. For a balanced binary tree like what a heap uses, h = log_2(n). But the number of nodes decreases one by one for each pop, so h should decrease.
So if I were to calculate the complexity I would do something like
O(log_2(n) + log_2(n-1) + ... + log_2(1))
= O(log_2(n * (n-1) * ... * (1))
= O(log_2(n^n))
= O(n log_2(n)
But is this the actual strict upper bound?
Imagine there was a way to do n pops from a heap in time O(n). That would give you a comparison-based sorting algorithm that runs in time O(n) as follows:
Build the heap in time O(n) using heapify.
Use the O(n)-time algorithm to do n pops.
However, this is impossible because all comparison-based sorting algorithms run in time Ω(n log n) on average and in the worst-case.
In fact, this argument shows that the cost of n pops must have a worst-case runtime of Ω(n log n), so that upper bound is tight in the worst case.
I have googled for lots of websites and they all say "the time complexity of clearing a heap is O(n log n)." The reason is:
Swapping the tailing node the root costs O(1).
Swapping "the new root" to suitable place costs O(level) = O(log n).
So deleting a node (the root) costs O(log n).
So deleting all n nodes costs O(n log n).
In my opinion, the answer is right but not "tight" because:
The heap (or its level) becoming smaller during deleting.
As a result, the cost of "swapping the new root to suitable place" becomes smaller.
The aforementioned reason of "O(n log n)" does not embody such change.
The time complexity of creating a heap is proved as O(n) at here.
I tend to believe that the time complexity of clearing a heap is O(n) as well because creating and clearing is very similar - both contain "swapping node to suitable position" and "change of heap size".
However, when considering O(n) time for clearing a heap, here is a contradiction:
By creating and clearing a heap, it is possible to sort an array in O(n) time.
The lower limit of time complexity of sorting is O(n log n).
I have thought about the question for a whole day but still been confused.
What on earth clearing a heap costs? Why?
As you correctly observe, the time taken is O((log n) + (log n-1) + ... + (log 2) + (log 1)). That's the same as O(log(n!)), which is the same as O(n log n) (proof in many places, but for example: What is O(log(n!)) and O(n!) and Stirling Approximation).
So you're right that the argument given for the time complexity of removing every element of a heap being O(nlog n) is wrong, but the result is still right.
Your equivalence between creating and "clearing" the heap is wrong. When you create the heap, there's a lot of slack because the heap invariant allows many choices at every level and this happens to mean that it's possible to find a valid ordering of the elements in O(n) time. When "clearing" the heap, there's no such slack (and the standard proof about comparison sorts needing at least n log n time proves that it's not possible).
I have to answer the following question:
What sorting algorithm is recommended if the first n-m part
is already sorted and the remaining part m is unsorted? Are there any algorithms that take O(n log m) comparisons? What about O(m log n) comparisons?
I just can't find the solution.
My first idea was insertion sort because O(n) for almost sorted sequence. But since we don't know the size of m the Runtime is very likely to be O(n^2) eventough the sequence is half sorted already isn't it?
Then I tought perhabs its quick sort because it takes (Sum from k=1 to n) Cavg (1-m) + Cavg (n-m) comparisons. But after ignoring the n-m part of the sequence the remaining sequence is 1-m in quicksort and not m.
Merge Sort and heap sort should have a runtime of O(m log m) for the remaining sequence m I would say.
Does anyone have an idea or can give me some advice?
Greetings
Have you tried sorting remaining part m separately as O(m log (m)) complexity (with any algorithm you like: MergeSort, HeapSort, QuickSort, ...) and then merge that part with sorted part using MergeSort (You won't even need to fully implement MergeSort - just single pass of it's inner loop body to merge two sorted sequences)?
That would result in O(m*log(m) + n + m) = O(m*log(m) + n) complexity. I don't believe it is possible to find better asymptotic complexity on single-core CPU. Although it will require additional O(n+m) memory for merging result array.
Which sort algorithm works best on mostly sorted data?
Sounds like insertion and bubble are good. You are free to implement as many as you want then test to see which is faster/fewer operations by supplying them partially sorted data.
I don't really understand why we don't just always select the median element as the pivot. This can be done in O(n) and thus results in a total run time of O(n log n).
I just assume that probably there is a large constant hidden in the O(n) for the median search.
From the Wikipedia Quicksort page:
Conversely, once we know a worst-case selection algorithm is available, we can use it to find the ideal pivot (the median) at every step of quicksort, producing a variant with worst-case O(n log n) running time. In practical implementations, however, this variant is considerably slower on average.
In other words, the cost of forcing it to be guaranteed O(n log n) is generally not worth paying. There's more information on that page, as well as on the selection algorithms page.
use randomized quicksort and you have worstcase run time of O(n log n) with very high probability.
Apparently it seems that the running time of finding the median is O(n) using randomized version of partition, but actually when the partition is again unbalanced at its extreme then the running time goes to O(n2). So you can make no improvement right from here.
But still there's a hope. If you go through "CORMEN" then you will find that finding ith order statistic can be done in linear time even in worst case scenario. The technique that is used is to use the median of median as the pivot element and then find the nedian which guarantees the linear running time in any case.
So we can use that technique in quicksort also to get O(nlgn) running time
Is there any practical algorithm for generic elements (unlike counting sort or bucket sort) that runs faster than O(n log n)?
Many people have mentioned the information-theoretic Ω(n lg n) bound on comparison sorting algorithms, which can't be broken in comparison sorts. (This earlier question explores why that's the case.)
However, there are some types of comparison sorts that, while not breaking O(n lg n) in the average case, can be shown to run faster on inputs that are already presorted to some extent. For example, Dijkstra's smoothsort runs in O(n) on already-sorted inputs with O(n lg n) worst-case behavior. One of my favorite sorts, Cartesian tree sort, provably takes optimal advantage of presortedness in a few metrics. For example, it can sort any sequence with a constant number of increasing or decreasing subsequences in time O(n), degrading gracefully to O(n lg n) in the worst case.
On the subject of non-comparison sorts, there are some famous but tricky sorting algorithms for integers that surpass O(n lg n) bynp doing clever bit-manipulation tricks. The best known integer sorting algorithm is a randomized algorithm that can sort in O(n √lg lg n), while the fastest deterministic algorithm for integer sorting runs in O(n lg lg n) time. You may have heard that radix sort works in O(n), though technically it's O(n lg U), where U is the largest value in the array to sort.
In short, no, you can't do much better than O(n lg n), but you can do marginally better if you know something about your input.
For generic elements that you can only compare and not access the internals of, it is impossible to have a sorting algorithm faster than Theta(n log n). That is because there are n! (n factorial) possible orders of the elements, and you need Theta(n log n) comparisons to distinguish all of them.
No. This is one of the few rigorous minimum bounds for algorithms we have. For a collection of n elements, there are n! different orders, so to specify a given order we need log(n!) bits. By Stirling's approximation this is approximately n log n. For each comparison we do between elements, we get essentially one bit of information (ignoring the possibility of equal elements).
For how many elements? Even though it's something like N1.2, a Shell-Metzner sort is often faster than most others up to a few thousand elements (or so).
It also depends on what you mean by "generic" and "practical". A radix sort can beat O(n log n), and it works for a fairly wide variety of data (but definitely not everything).
If your idea of practical and generic limits the algorithm to one that directly compares elements, then no -- nothing does (or ever can) be better than O(n log n). That's been proven for quite some time.