I know that we can find the 2nd largest element in an array of size N in N+log(N)-2 using a "tournament" algorithm. Now I wonder if we can find the k-th largest element using a similar "tournament".
I know there is an O(N) "selection" algorithm to find the k-th largest element. It uses Quick Select with a "good" pivot, which can be found in O(N). We can build also a heap from the array in O(N) and retrieve k element from the heap.
I wonder if there is another approach.
I believe you can make this an O(N log k) algorithm: Iterate over the array, and maintain a min-heap of the largest k elements encountered so far. So the first k elements go directly into the heap. Every subsequent element will be compared against the tip of the heap, and if it is larger, the tip will be removed from the heap and the new element inserted, which is an O(log k) operation for a heap of size k. When the algorithm is done, and the sequence had a length of at least k, then the tip of the heap will have the kth largest element, and the rest of the heap the larger elements.
This approach has inferior worst-case behaviour than the median-of-medians O(n) solution, but will be much easier to implement and yield rather good behaviour for small k. So it might be well suited for many practical applications.
Related
Is it possible to have O(1) time complexity in find the k largest or smallest numbers in an array, by making a stack class with an auxiliary data structure to track k largest/smallest in every push() and pop(). Since retrieval is O(1), return k elements in a get method
Yes, can find out Kth largest element or smallest element by O(1) complexity only if your array is in sorted order.
If you are looking for a well-known data structure you can find Max-Heap and Min-Heap useful. You can find more about that here.
Update
As you have updated your question from max and min to k largest, you can preprocess your data into a sorted array and then insert new value using the insertion sort strategy. Then, you can report the k largest value in O(k) (and if the k is constant it would be in O(1)).
you can try the below link that discuss about the finding largest number
https://www.geeksforgeeks.org/k-largestor-smallest-elements-in-an-array/
Suppose you have an unsorted array of length n. Now you want to take the k-largest elements from the array. We also know that n is much larger than k.
My naive idea is to sort the array and output the k-largest elements. That would cost O(nlog(n)). Is there a more efficient algorithm that takes advantage of the fact that n is much larger than k ?
Yes, more effective algorithms do exist.
Get k first elements, build min-heap containing these k elements.
Traverse through other elements. If current item is larger than heap top, remove top and insert current item.
After the end heap will contain k largest elements.
Complexity is O(N*logK)
Also consider QuickSelect algorithm with average time O(n) (while it has worst case up to quadratic)
I have a minimum heap with n elements and want to find k smallest numbers from this heap. What is the worst-case complexity?
Here is my approach: somewhere on the stackoverflow I read that complexity of finding i-th smallest number in a min heap is O(i). So if we would like to find n-1 smallest numbers (n is pointless, since it would be the entire heap), the total complexity would look something like this:
O(n-1)+O(n-2)+O(n-3)+…+O(2)+O(1)=O((1+n-1)*(n/2))=O(n^2)
Is this correct?
No, the time is much better than that. O(k log(n)) very easily, and O(k) if you're smart.
Finding and removing the smallest element from the heap is O(log(n)). This leads to O(k log(n)) time very easily.
BUT the result that you are thinking about is https://ac.els-cdn.com/S0890540183710308/1-s2.0-S0890540183710308-main.pdf?_tid=382a1cac-e4f7-11e7-8ac9-00000aab0f02&acdnat=1513713791_08f4df78a8821855e8ec788da063ea2f that shows how to find the size of the kth smallest number in time O(k). Now you use the fact that a heap is a binary tree and start from the root and do a recursive search for every number that you find which is smaller than that largest. Then fill out the rest of your list with copies of the k'th smallest number.
In that search you will wind up looking at up to k-1 that are at most that size, and for some of them you will look at up to 2 children that are too large to bother with, for a maximum of 3k-3 elements. This makes the whole algorithm O(k).
That link died due to bitrot. Hopefully https://www.sciencedirect.com/science/article/pii/S0890540183710308 lasts longer.
I am doubtful that it is possible to identify the kth smallest element in time O(k). The best I have seen before is an O(k log k) algorithm, which also conveniently solves your problem by identifying the k smallest elements. You can read the details in another answer on StackOverflow or on Quora.
The basic idea is to manipulate a secondary heap. Initially, this secondary heap contains only the root of the original heap. At each step, the algorithm deletes the min of the secondary heap and inserts its two original children (that is, its children from the original heap) into the secondary heap.
This algorithm has a nice property that on step i, the element it deletes from the secondary heap is the ith smallest element overall. So after k steps, the set of items which have been deleted from the secondary heap are exactly the k smallest elements. This algorithm is O(k log k) because there are O(k) deletions/insertions into a secondary heap which is upper bounded in size by O(k).
EDIT: I stand corrected! btilly's answer provides a solution in O(k) using a result from this paper.
There is a recent (2019) algorithm that finds the k smallest elements of a binary min-heap in time O(k) that uses the soft heap data structure. This is a dramatically simpler algorithm than Frederickson’s original O(k)-time heap selection algorithm. See “ Selection from Heaps, Row-Sorted Matrices, and X+Y Using Soft Heaps” by Kaplan et al.
How would you find the k smallest elements from an unsorted array using quicksort (other than just sorting and taking the k smallest elements)? Would the worst case running time be the same O(n^2)?
You could optimize quicksort, all you have to do is not run the recursive potion on the other portions of the array other than the "first" half until your partition is at position k. If you don't need your output sorted, you can stop there.
Warning: non-rigorous analysis ahead.
However, I think the worst-case time complexity will still be O(n^2). That occurs when you always pick the biggest or smallest element to be your pivot, and you devolve into bubble sort (i.e. you aren't able to pick a pivot that divides and conquers).
Another solution (if the only purpose of this collection is to pick out k min elements) is to use a min-heap of limited tree height ciel(log(k)) (or exactly k nodes). So now, for each insert into the min heap, your maximum time for insert is O(n*log(k)) and the same for removal (versus O(n*log(n)) for both in a full heapsort). This will give the array back in sorted order in linearithmic time worst-case. Same with mergesort.
Is there an in-place algorithm to arrange the k smallest integers in an array of n distinct integers with 1<=k<=n?
I believe counting sort can be modified for this, but I can't seem to figure out how?
Any help will be appreciated.
How about selection sort? It runs in place in O(n^2). Just stop after you've found k smallest elements.
Do you want to partition the array so that k smallest elements are the first k elements (not necessarily sorted order)? IF so, what you are looking for is generalized median find algorithm which runs in O(n) (Just google for median find algorithm).
If you can live with randomized algorithm that finishes in linear time with high probability then all you have to do is keep picking your pivot randomly which greatly simplifies the implementation.
You could use randomized selection to select the kth smallest integer in O(n) time, then partition on that element, and then use quicksort on the k smallest elements. This uses O(1) additional memory and runs in total time O(n + k log k).
You're looking for a selection algorithm. BFPRT will give you guaranteed worst-case O(n) performance, but it's pretty complex.