We have 3 variants of Merge sort.
Top down
Bottom up
Natural
Are any of these adaptive algorithms? For instance, if an array is sorted they will take advantage of the sorted order.
According to me, no matter if an array is sorted or not, merge sort will still go in for comparisons and then merge. So, the answer is none of these are adaptive.
Is my understanding is correct?
Natural merge sort is adaptive. For example, it executes only one run through sorted array and makes N comparisons.
Both top down and bottom up sorts are not adaptive, they always make O(NlogN) operations
Merge Sort is an implementation of divide and conquer algorithm. You need to divide the whole array into sub arrays. For example
[1, 3 ,5, 6, 2, 4,1 10] is divided into [1 3 5 6] and [2 4 1 10]. [1 3 5 6] is divided into [1 3] and [5 6]. Now as both [1 3] and [5 6] are sorted swap procedure is not needed.
So there is at least a little complexity if the array is sorted
Accoring to me, no matter if an array is sorted or not, merge sort will still go in for comparisons
I don't think its possible to sort without any comparisons and save memory if the input is not already sorted or contains information regarding what sorting procidure to follow. The fastest sorting algorithm is of time complexity O(n log(n)) (there are sorting techniques like bead sorting. I am not considering it here as it is not an optimal method memory vice)
P.S quick sort does involve comparisons (but is adaptive), and in its worse case it makes about O(n^2) comparisons.
P.P.S Radix sort, counting sort and bucket sort are some examples of adaptive sort;
Related
Encountered a question like this for mergesort specifically and was wondering how does one approach a question like this for other algorithms (insertionsort,heapsort,quicksort and etc)
Is it safe to assume that the nth best/worst arrangement for any algorithm is the nth step of solving the best/worst arrangement for the same set of data?
Example:
If the worst case for mergesort with the following array of integers [1,2,3,4,5,6,7,8] is [1,5,3,7,2,6,4,8]. What is the next worst case for this array of integers?
I assumed it would be the next arrangement when solving the worst case which is [1,3,5,7,2,6,4,8]. Am I approaching such a question wrongly?
The concept of a "next-best" or "next-worst" case is not really well-defined in the first place. Neither is the concept of "the state of the array after one step", because not all algorithms modify an array in-place.
When we say the "worst case" of an algorithm, we don't mean a single input to an algorithm. For example, the array [5, 4, 3, 2, 1] is not - by itself - the worst case of the insertion sort algorithm. This array is one of the worst inputs (i.e. highest number of steps to compute) for insertion sort out of arrays of length 5, but we are very rarely interested in arrays of one specific length.
What we mean by "best case" or "worst case" is actually an infinite family of inputs, such that each member of that family is a best or worst input for its own value of n, and the family must contain inputs for arbitrarily large values of n. So, for example:
The infinite set of arrays {[1], [2, 1], [3, 2, 1], [4, 3, 2, 1], ...} is a worst case for insertion sort. For inputs from this infinite set, the asymptotic complexity of insertion sort is Θ(n2).
The infinite set of arrays {[1], [1, 2], [1, 2, 3], [1, 2, 3, 4], ...} is a best case for insertion sort. For inputs from this infinite set, the asymptotic complexity of insertion sort is Θ(n).
Note that the (larger) infinite set of all arrays which are in descending order is also a worst case for insertion sort, and the (larger) infinite set of all arrays in ascending order is also a best case. So the family is not unique, but the asymptotic complexity of the algorithm on inputs from any two "best case" (or any two "worst case") families is the same.
Now we've got that out of the way, let's think about what a "next-best" or "next-worst" case would have to mean. If the asymptotic complexity of insertion sort on some family of inputs is also Θ(n2), then that family is a worst case for insertion sort; so the asymptotic complexity of a "next-worst" case would have to be something lower than Θ(n2).
But however small a gap you choose, it is not the "next-worst":
If you choose a family where the complexity is Θ(n1.999), then it is not "next-worst" because I can find another family where the complexity is Θ(n1.9999).
If you choose a family where the complexity is Θ(n2 / log n), I can find one where it's Θ(n2 / log log n).
That is, the asymptotic complexities of different families of possible inputs for insertion sort form a dense order, for any two different complexities there is another complexity in between those two, so there is no "next" or "previous" one.
I am reading about use cases of Selection Sort, and this source says:
(selection sort is used when...) cost of writing to a memory matters like in flash memory (number of writes/swaps is O(n) as compared to O(n2) of bubble sort)
We can even see O(n^2) swaps in this example:
[1, 2, 3, 4, 5]. It's going to have 4 swaps, then 3, then 2, and 1. That is O(n^2), not O(n) swaps. Why do they say the opposite?
A selection sort has a time complexity of O(n2), but only O(n) swaps.
In each iteration i, you go over all the remaining items (in indexes i and onwards), find the right value to populate that index, and swap it there. So in total you perform O(n2) comparisons, but only O(n) swaps.
Say I'm given a bunch of number pairs, is there a good algorithm to find the pair with the largest sum of their squares besides computing all sums of squares and comparing it to the current max?
eg
input:
[3, 3]
[0, 3]
[4, 0]
[2, 4]
output:
[2, 4] (sum of squares: 20)
To use a binary search, you would need to sort the list. This raises two problems (mentioned or hinted above):
You don't have a well-ordered collection
The optimal sort is O(n log n)
Sum of two squares is simple: two data accesses for the numbers, two multiplications, and an addition. On a linear CPU, this pipelines to 3 cheap (i.e. single-cycle) operations. On a modern AI processor, this is a single-cycle dot-product.
In short, not only is the linear, brute-force approach only O(n), it's a fast O(n). Take the money and run.
How is it different if I select a randomized pivot versus just selecting the first pivot in an unordered set/list?
If the set is unordered, isnt selecting the first value in the set, random in itself? So essentially, I am trying to understand how/if randomizing promises a better worst case runtime.
I think you may be mixing up the concepts of arbitrary and random. It's arbitrary to pick the first element of the array - you could pick any element you'd like and it would work equally well - but it's not random. A random choice is one that can't be predicted in advance. An arbitrary choice is one that can be.
Let's imagine that you're using quicksort on the sorted sequence 1, 2, 3, 4, 5, 6, ..., n. If you choose the first element as a pivot, then you'll choose 1 as the pivot. All n - 1 other elements then go to the right and nothing goes to the left, and you'll recursively quicksort 2, 3, 4, 5, ..., n.
When you quicksort that range, you'll choose 2 as the pivot. Partitioning the elements then puts nothing on the left and the numbers 3, 4, 5, 6, ..., n on the right, so you'll recursively quicksort 3, 4, 5, 6, ..., n.
More generally, after k steps, you'll choose the number k as a pivot, put the numbers k+1, k+2, ..., n on the right, then recursively quicksort them.
The total work done here ends up being Θ(n2), since on the first pass (to partition 2, 3, ..., n around 1) you have to look at n-1 elements, on the second pass (to partition 3, 4, 5, ..., n around 2), you have to look at n-2 elements, etc. This means that the work done is (n-1)+(n-2)+ ... +1 = Θ(n2), quite inefficient!
Now, contrast this with randomized quicksort. In randomized quicksort, you truly choose a random element as your pivot at each step. This means that while you technically could choose the same pivots as in the deterministic case, it's very unlikely (the probability would be roughly 22 - n, which is quite low) that this will happen and trigger the worst-case behavior. You're more likely to choose pivots closer to the center of the array, and when that happens the recursion branches more evenly and thus terminates a lot faster.
The advantage of randomized quicksort is that there's no one input that will always cause it to run in time Θ(n log n) and the runtime is expected to be O(n log n). Deterministic quicksort algorithms usually have the drawback that either (1) they run in worst-case time O(n log n), but with a high constant factor, or (2) they run in worst-case time O(n2) and the sort of input that triggers this case is deterministic.
In quick sort, the pivot is always the right most index of the selected array whereas in Randomized quick sort, pivot can be any element in the array.
I'm trying to understand the main disadvantages of non-comparison-based sorting algorithms to comparison-based sorting algorithms.
Is it mainly because non-comparison-based sorting algorithms do poorly for non-fixed length inputs?
e.g. let's say I had 4 elements [1, 100000000000000000024, 3, 5]
A comparison-based sorting algorithm would solve it in 4 * log(4).
While a non-comparison-based sorting algorithm would solve it in 4 * length_of("100000000000000000024") assuming we use the keys 0 to 9 and use an algorithm such as LSD or MSD.
Or let's say we know the numbers fall between 0 and 100000000000000000024, we could have keys from 0 to 100000000000000000024, but that's extremely space inefficient as comparison-based sorting algorithm can be done in-place.
Thanks,
Waley
Non-Comparison sorting algorithms often need many assumptions about the input data (integers from small range for count sort, uniformly distributed for bucket sort, etc.).
The time complexity is also often dependent on only on the input size. For instance count sort depends on the range, as well as the radix sort.