Is there a way to combine merge and insertion sort in such a way that the new algorithm is faster than each algorithm (merge and insertion sort) individually? I've been thinking about it for the past couple of hours but all I can come up with is once the division of the array in merge sort reaches a certain threshold (i.e. length of 10) then insertion sort kicks in and sort the array. But that won't make the algorithm faster it'll just reduce overhead. Any ideas regarding this?
Related
I am trying to understand, kinda intuitively, how the runtime for merge sort is so much better than insertion sort.
Even though we divide and conquer with merge sort, on a single CPU, each of the nodes of the merge sort execution tree will be executed serially. Is the smaller input size on every recursive call (iteration) somehow the key thing for merge sort?
Or is it the fact that since merge sort is not in-place and uses O(n) space this saves on the number of shifts we have to do in insertion sort to make space for the insertion of the smaller number.
But what about the penalty of copying the elements in left and right temporary arrays in every merge step?
Yes, that smaller input size is in large part where the speed up of mergesort comes from compared with insertion sort. The fact that mergesort uses more space is more an artifact of how it works than an inherent reason for the speedup.
Here’s one way to see this. We know that insertion sort, on average, takes time Θ(n2). Now, suppose you want to insertion sort an array of n elements. Instead, you cut the array apart into two smaller arrays of size roughly n/2 and insertion sort each of those. How long will this take? Since insertion sort has quadratic runtime, the cost of insertion sorting each half will be roughly one quarter the cost of insertion sorting the whole array ((n/2)2 = n2 / 4). Since there are two of those arrays, the total cost of sorting things this way will be roughly
2(n2 / 4) = n2 / 2,
which is half the time required to sort the original array. This gives rise to a simple sorting algorithm that’s an improvement over insertion sort:
Split the array in half.
Insertion sort each half.
Merge the two halves together.
That last step introduces linear space overhead for the merge, though you could do it with an in-place merge at a higher cost.
This algorithm, “split sort,” is about twice as a fast as insertion sort. So then you might ask - why split in halves? Why not quarters? After all, the cost of sorting one quarter of the array is about
(n/4)2 = n2 / 16,
which is sixteen times faster than sorting the original array! We could turn that into a sorting algorithm like this:
Split the array into quarters.
Insertion sort each quarter.
Merge the quarters into halves.
Merge the halves into the full array.
This will be about four times faster than insertion sort (each sort takes one sixteenth the time of the original sort, and we do four of them).
You can think of mergesort as the “limit” of this process where we never stop splitting and divide the array into the smallest units possible and then merge them all back together at the end. The speedup is based on the fact that sorting smaller arrays is inherently faster than sorting larger arrays, with the memory overhead for the merge being more of an implementation detail than an intrinsic reason for the speedup.
Another way to see that the space usage isn’t necessary for the speedup is to compare insertion sort to heapsort. Heapsort also runs in time O(n log n) but uses only O(1) auxiliary space.
Hope this helps!
Even an in place merge sort (O(1) space) is faster than insertion sort for n >= ~128 on a typical X86.
For smaller n, insertion sort is faster, due to cache and related constant factors, and because of this, most library implementations of stable sort use a hybrid of insertion sort (to create small sorted runs) and bottom up merge sort.
An example of an in place merge sort is block merge sort (grail), O(1) space, still with O(n log(n)) time complexity, but about 50% slower than a standard merge sort, and the code is complicated:
https://github.com/Mrrl/GrailSort/blob/master/GrailSort.h
But what about the penalty of copying the elements in left and right temporary arrays in every merge step?
Typical merge sort avoids copying of data by doing a one time allocation of a temporary array and then changes the direction of merge based on merge pass for bottom up merge sort, or level of recursion for top down merge sort.
I was thinking of optimizing the current sorting algorithms. To make the process faster, I thought of using threads and divide the arrays into two parts. Sort that both arrays using insertion sort simultaneously using threads and wait for both of them to complete. After that use bubble sort to merge both of that array. Do you think, using this algorithm, the sorting will be faster ?
No, that will not be faster, in the general case. Imagine, for example, that your initial array looks like this:
[9,5,7,6,8,3,2,0,4,1]
After sorting the two halves it looks like this:
[5,6,7,8,9,0,1,2,3,4]
Sorting that with bubble sort won't be appreciably faster than sorting the initial array with bubble sort. Total time elapsed with the insertion sorts plus the bubble sort almost certainly will be more than if you just sorted the initial array with a single thread.
Neither bubble sort nor insertion sort are particularly amenable to parallelization. You're better off implementing a parallel quicksort. Or, if you insist on using insertion sort in the threads, use a merge to combine the sorted sub-arrays. Of course, the merge would require O(n) additional memory.
No. Your method would not be faster than known methods of parallel sorting algorithms. Insertion sort is O(n^2), and if applied would not result in better result unless the code is more parallelised. Given I assume you have only two threads to optimize : best will be to use Merge sort with two threads, which would help in Worst Case to be O(nlogn). I dont know why you want to learn two thread parallelism, but learning from the following resource might get you the knack of parallel algorithms of sorting in multithreads :
http://www.dcc.fc.up.pt/~fds/aulas/PPD/1112/sorting.pdf
Question;
You have to sort an array of bank transactions by date. Most of them
are in order (by date), only a few are out of order.
Which sorting algorithm will you use between insertion sort, selection
sort and merge sort in order to take advantage of the fact that the
array is almost sorted?
My answer (not sure if its correct)
Assuming that N >= 5, i would go with Merge Sort since its avg. time complexity would be O(n * log n) which would be more efficient than insertion sort O(n^2). However, since multiple transactions will be on the same dates, insertion sort would be a good STABLE sorting method.
Which one is better in this case? Merge or insertion sort? Am i in the right direction?
You should pick insertion sort, not because of stability, but because it's adaptive (see here) and will outperform due to the fact the input is almost sorted to begin with.
The meaning of this "adaptive" characteristic is that elements that are already in place are processed at O(1) time, and elements very close to their sorted position can also be considered O(1) (up to some k distance).
For sorting algorithms, why can't you just cut the array in half, and just use selection sort or insertion sort on both, and put them back together, to significantly improve the speed?
You're saying that your algorithm is faster than existing sorts, for example, selection sort and insertion sort. But then, once you've split your array in half, you'd be better using your algorithm rather than selection/insertion sort to sort the halves (perhaps unless the halves are small).
This is exactly merge-sort.
You are right. This approach is followed in some sorting algorithms. For example in Merge sort which divides the array into two halves and if these two halves are small, you can apply insertion sort directly on them but if they are large then it would not be feasible as you better divide the halves too (please see the details of Merge sort) . Insertion sort/Selection sort/Bubble sort perform better when array is small or generally on nearly sorted data. If you are tackling long data then choose Merge sort/Quick sort/Redix sort.
This is an interview question that I recently found on Internet:
If you are going to implement a function which takes an integer array as input and returns the maximum, would you use bubble sort or merge sort to implement this function? What if the array size is less than 1000? What if it is greater than 1000?
This is how I think about it:
First, it is really weird to use sorting to implement the above function. You can just go through the array once and find the max one.
Second, if have to make a choice between the two, then bubble sort is better - you don't have to implement the whole bubble sort procedure but only need to do the first pass. It is better than merge sort both in time and space.
Are there any mistakes in my answer? Did I miss anything?
It's a trick question. If you just want the maximum, (or indeed, the kth value for any k, which includes finding the median), there's a perfectly good O(n) algorithm. Sorting is a waste of time. That's what they want to hear.
As you say, the algorithm for maximum is really trivial. To ace a question like this, you should have the quick-select algorithm ready, and also be able to suggest a heap datastructure in case you need to be able to mutate the list of values and always be able to produce the maximum rapidly.
I just googled the algorithms. The bubble sort wins in both situations because of the largest benefit of only having to run through it once. Merge sort can not cut any short cuts for only having to calculate the largest number. Merge takes the length of the list, finds the middle, and then all the numbers below the middle compare to the left and all above compare to the right; in oppose to creating unique pairs to compare. Meaning for every number left in the array an equal number of comparisons need to be made. In addition to that each number is compared twice so the lowest numbers of the array will most likely get eliminated in both of their comparisons. Meaning only one less number in the array after doing two comparisons in many situations. Bubble would dominate
Firstly I agree with everything you have said, but perhaps it is asking about knowing time complexity's of the algorithms and how the input size is a big factor in which will be fastest.
Bubble sort is O(n2) and Merge Sort is O(nlogn). So, on a small set it wont be that different but on a lot of data Bubble sort will be much slower.
Barring the maximum part, bubble sort is slower asymptotically, but it has a big advantage for small n in that it doesn't require the merging/creation of new arrays. In some implementations, this might make it faster in real time.
only one pass is needed , for worst case , to find maximum u just have to traverse the whole array , so bubble would be better ..
Merge sort is easy for a computer to sort the elements and it takes less time to sort than bubble sort. Best case with merge sort is n*log2n and worst case is n*log2n. With bubble sort best case is O(n) and worst case is O(n2).