If on empty min heap we doing n arbitrary insert and delete operations, (with given location of delete in min-heap). why the amortized analysis for insert is O(1) and delete is O(log n)?
a) insert O(log n), delete O(1)
b) insert O(log n), delete O(log n)
c) insert O(1), delete O(1)
d) insert O(1), delete O(log n)
any person could clarify it for me?
Based on your question and responses to comments, I'm going to assume a binary heap.
First, the worst case for insertion is O(log n) and the worst case for removal of the smallest item is O(log n). This follows from the tree structure of the heap. That is, for a heap of n items, there are log(n) levels in the tree.
Insertion involves (logically) adding the item as the lowest right-most node in the tree and then "bubbling" it up to the required level. If the new item is smaller than the root, then it has to bubble all the way to the top--all log(n) levels. So if you insert the numbers 10, 9, 8, 7, 6, 5, 4, 3, 2, 1 into a min-heap, you'll hit the worst case for every insertion.
Removal of the smallest element involves replacing the lowest item (the root) with the last item and then "sifting" the item down to its proper position. Again, this can take up to log(n) operations.
That's the worst case. The average case is much different.
Remember that in a binary heap, half of the nodes are leafs--they have no children. So if you're inserting items in random order, half the time the item you're inserting will belong on the lowest level and there is no "bubble up" to do. So half the time your insert operation is O(1). Of the other half, half of those will belong on the second level up. And so on. The only time you actually do log(n) operations on insert is when the item you're inserting is smaller than the existing root item. It's quite possible, then, that the observed runtime behavior is that insertion is O(1). In fact that will be the behavior if you insert a sorted array into a min-heap. That is, if you were to insert the values 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 in that order.
When removing the smallest item from a min-heap, you take the last item from the heap and sift it down from the top. The "half the time" rule comes into play again, but this time it's working against you. That last item you took from the heap probably belongs down there on the lowest level. So you have to sift it all the way back down, which takes log(n) operations. Half the time you'll have do to all log(n) operations. Half of the remaining you'll need to do all but one of them, etc. And in fact the minimum number of levels you have to sift down will depend on the depth of the tree. For example, if your heap has more than three items then you know that removing the smallest item will require at least one sift-down operation because the next-lowest item is always on the second level of the tree.
It turns out, then, that in the average case insertion into a binary heap takes much less than O(log n) time. It's likely closer to O(1). And removal from a binary heap is much closer to the worst case of O(log n).
Related
For the following questions
Question 3
You are given a heap with n elements that supports Insert and Extract-Min. Which of the following tasks can you achieve in O(logn) time?
Find the median of the elements stored in the heap.
Find the fifth-smallest element stored in the heap.
Find the largest element stored in the heap.
Find the median of the elements stored in theheap.
Why is "Find the largest element stored in the heap."not correct, my understanding here is that you can use logN time to go to the bottom of the heap, and one of the element there must be the largest element.
"Find the fifth-smallest element stored in the heap." this should take constant time right, because you only need to go down 5 layers at most?
"Find the median of the elements stored in the heap. " should this take O(n) time? because we extract min for the n elements to get a sorted array, and take o(1) to find the median of it?
It depends on what the running times are of the operations insert and extract-min. In traditional heaps, both take ϴ(log n) time. However, in finger-tree-based heaps, only insert takes ϴ(log n) time, while extract-min takes O(1) time. There, you can find the fifth smallest element in O(5) = O(1) time and the median in O(n/2) = O(n) time. You can also find the largest element in O(n) time.
Why is "Find the largest element stored in the heap."not correct, my understanding here is that you can use logN time to go to the bottom of the heap, and one of the element there must be the largest element.
The lowest level of the heap contains half of the elements. More correctly, half of the elements of the heap are leaves--have no children. The largest element in the heap is one of those. Finding the largest element of the heap, then, will require that you examine n/2 items. Except that the heap only supports insert and extract-min, so you end up having to call extract-min on every element. Finding the largest element will take O(n log n) time.
"Find the fifth-smallest element stored in the heap." this should take constant time right, because you only need to go down 5 layers at most?
This can be done in log(n) time. Actually 5*log(n) because you have to call extract-min five times. But we ignore constant factors. However it's not constant time because the complexity of extract-min depends on the size of the heap.
"Find the median of the elements stored in the heap." should this take O(n) time? because we extract min for the n elements to get a sorted array, and take o(1) to find the median of it?
The median is the middle element. So you only have to remove n/2 elements from the heap. But removing an item from the heap is a log(n) operation. So the complexity is O(n/2 log n) and since we ignore constant factors in algorithmic analysis, it's O(n log n).
I know that in the case of inserting numbers 1,2,3,......,n into a initially empty min heap with the order 1,2,3,.....,n, you will just need to put them in one-by-one.
But I can't quite work out how to calculate the time complexity of two different cases: if you insert them in a reverse order (n,n-1,n-2,....,2,1) or even with other numbers with the order (1,n+1,2,n+2,3,n+3,....,n-1,2n-1,n,2n). I know that for the reverse case, you will have to move the numer inserted "along" the height of the heap (which is logn) but I am not quite sure about the remaining parts...
As you say, when you insert the numbers 0..n in-order into a min-heap, insertion is O(1) per item. Because all you have to do is append the number into the array.
When you insert in reverse order, then every item is inserted into the bottom row, and has to be sifted up through the heap to the root. Every insertion has to move up log(n) rows. So insertion is O(log n) per item.
The average, when you're inserting items at random, as discussed at some length in Argument for O(1) average-case complexity of heap insertion and the articles it links, is something like 1.6.
So there is a very strong argument that the average complexity of binary heap insertion is O(1).
In your particular case, your insertions are alternating O(1) and O(log n). So over time you have O((1+log n)/2), which is going to be O(n log n) to insert all of the items.
Hi I am new to algorithms and am quite fascinated by it.
I am trying to figure out worst case time complexity of insertion sort and it is mentioned as O(n**2). Instead, we can have the time complexity as O(N*logN).
Here is my explanation,
THe insertion sort looks at the 1st element and assumes it is sorted. Next it looks at the 2nd element and compares with the predecessor sorted sublist of 1 element and inserts it based on the comparison with elements in the predecessor sorted sublist. This process is repeated similarly.
Everywhere it is mentioned that to insert an element into the predecessor sorted sublist, basically linear search,it takes O(N) time and as we have do these operations for n elements it takes us O(N**2).
However, if we use binary insertion to insert the element into predecessor sublist it should take O(logn) time where n is the length of sublist. Basically compare the new element with the middle element of predecessor sorted sublist and if it is greater than the middle element then new element lies between the middle element and the last element of the sublist.
As we repeat the operations for n items it should take us O(N*logN). We can use binary search approach as we know the predecessor sublist is sorted.
So shouldn't the worst case time complexity be O(N*logN) instead of O(N**2).
Yes, you can find the insertion point in O(log n), but then you have to make space to insert the item. That takes O(n) time.
Consider this partially-sorted array:
[1,2,3,5,6,7,9,4]
You get to the last item, 4, and you do a binary search to locate the position where it needs to be inserted. But now you have to make space, which means moving items 9, 7, 6, and 5 down one place in the array. That's what makes insertion sort O(n^2).
Speical minHeap is a minHeap which each level is sorted from left to right.
How can I print all the n elements by order in O(n) at worst case?
The minHeap is implemented by binary heap, in which the tree is a complete binary tree (see figure).
here is the example of a special minHeap:
So the result should be: [1,3,4,5,8,10,17,18,20,22,25,30]
Question from homework.
If n is a parameter independent of the size of the heap, then under a standard comparison-based model, this is impossible. You will need additional restrictions, like more preexisting order than you've mentioned, or all elements of the heap being integers under a sufficiently low bound.
Suppose you have a heap of height k, where the root and its chain of left children have values 1, 2, 3, ... k. We can assign values >k to the k-1 right children of these nodes in any order without violating the "special minheap" condition, then assign values greater than those to fill out the rest of the heap. Printing the top 2k-1 values in this heap requires sorting k-1 values that could be in any order, which cannot be done through comparisons in less than O(k*log(k)) time.
If n is supposed to be the size of the heap, this is straightforward. The heap invariant is unnecessary; it only matters that the layers are sorted. A mergesort merging the first and second layers, then merging each successive layer into the already-merged results, will take O(n) time. The kth merge merges 2^k-1 already-merged elements with <=2^k elements from the next layer, taking O(2^k) time. There are O(log(n)) merges, and summing O(2^k) from k=1 to k=log(n) gives O(n).
Each level of the heap is in ascending order. There are log(n) levels.
We can do a merge of the levels, which is O(n log k). k in this case is the number of levels, or log(n), so we know it's possible to do this in O(n * log(log n)).
The levels have 1, 2, 4, 8, 16, etc. nodes in them. The first merge operation removes the first level, so the number of items in our merge heap becomes k-1. In the worst case, after half of the nodes have been removed, the merge heap is k-2, etc.
I don't have the math at hand, but I suspect the solution involves showing that expanding the series (i.e. keeping track of the merge heap size and multiplying by the number of nodes that go through each size heap) reduces to 2, as mentioned in the comments.
How would you find the k smallest elements from an unsorted array using quicksort (other than just sorting and taking the k smallest elements)? Would the worst case running time be the same O(n^2)?
You could optimize quicksort, all you have to do is not run the recursive potion on the other portions of the array other than the "first" half until your partition is at position k. If you don't need your output sorted, you can stop there.
Warning: non-rigorous analysis ahead.
However, I think the worst-case time complexity will still be O(n^2). That occurs when you always pick the biggest or smallest element to be your pivot, and you devolve into bubble sort (i.e. you aren't able to pick a pivot that divides and conquers).
Another solution (if the only purpose of this collection is to pick out k min elements) is to use a min-heap of limited tree height ciel(log(k)) (or exactly k nodes). So now, for each insert into the min heap, your maximum time for insert is O(n*log(k)) and the same for removal (versus O(n*log(n)) for both in a full heapsort). This will give the array back in sorted order in linearithmic time worst-case. Same with mergesort.