I want to sort n strings each of length n in O(n^2) ? Is their any other solution beside radix based sorting or trie based ?
Let's suppose you try using a comparison-based sort. Comparing two strings of length n takes, in the worst case, time O(n). Consequently, in the worst case, you'd have to make only O(n) comparisons to sort the strings. However, that's impossible, since comparison-based sorting algorithms require Ω(n log n) comparisons on average. This worst-case can be realized; starting with the array x1, x2, ..., xn, you can form the strings an-1x1, an-1x2, ..., an-1xn, and comparisons will take time Θ(n) each.
That rules out comparison sorts, which leaves behind approaches that harness actual properties of strings. The approaches you listed - trie-based approaches and radix sort - form the basis for most of the algorithms along these lines (in fact, to the best of my knowledge, all string sorting algorithms are variations on these themes). There are nicely-optimized implementations of these algorithms. For example, burstsort is an optimized radix sort that's designed to be cache-friendly, and therefore has better performance than the naive algorithm.
One detail that you need to keep in mind is that the size of the alphabet from which your strings are drawn does have an impact on the runtime. Radix sort on strings of length n more properly takes time O(n2 + n|Σ|), where |Σ| is the number of different characters that the strings can be made from. Consequently, you can't necessarily use radix sort to sort n strings of length n in time O(n2) if there are way more than n characters in your alphabet.
Related
The question:
Merge k sorted arrays each with n elements into a single array of size nk in minimum time complexity. The algorithm should be a comparison-based algorithm. No assumption on the input should be made.
So I know about an algorithm that solves the problem in nklogk time complexity as mentioned here: https://www.geeksforgeeks.org/merge-k-sorted-arrays/.
Though, my question is can we sort in less than nklogk, meaning, the runtime is o(nklogk).
So I searched through the internet and found this answer:
Merge k sorted arrays of size n in O(nk) time complexity
Which claims to divide an array of size K into singletons and merge them into a single array. But this is incorrect since one can claim that he found an algorithm that solves the problem in sqrt(n)klogk which is o(nklogk) but n=1 so we sort the array in KlogK time which doesn't contradict the lower bound on sorting an array.
So how can I contradict the lower bound on sorting an array? meaning, for an array of size N which doesn't have any assumptions on the input, sorting will take at least NlogN operations.
The lower bound of n log n only applies to comparison-based sorting algorithms (heap sort, merge sort, etc.). There are, of course, sorting algorithms that have better time complexities (such as counting sort), however they are not comparison-based.
Is there any sorting algorithm with an average time complexity log(n)??
example [8,2,7,5,0,1]
sort given array with time complexity log(n)
No; this is, in fact, impossible for an arbitrary list! We can prove this fairly simply: the absolute minimum thing we must do for a sort is look at each element in the list at least once. After all, an element may belong anywhere in the sorted list; if we don't even look at an element, it's impossible for us to sort the array. This means that any sorting algorithm has a lower bound of n, and since n > log(n), a log(n) sort is impossible.
Although n is the lower bound, most sorts (like merge sort, quick sort) are n*log(n) time. In fact, while we can sort purely numerical lists in n time in some cases with radix sort, we actually have no way to, say, sort arbitrary objects like strings in less than n*log(n).
That said, there may be times when the list is not arbitrary; ex. we have a list that is entirely sorted except for one element, and we need to put that element in the list. In that case, methods like binary search tree can let you insert in log(n), but this is only possible because we are operating on a single element. Building up a tree (ie. performing n inserts) is n*log(n) time.
As #dominicm00 also mentioned the answer is no.
In general when you see an algorithm with time complexity of Log N with base 2 that means that, you are dividing the input list into 2 sets, and getting rid of one of them repeatedly. In sorting algorithm we need to put all the elements in their appropriate place, if we get rid of half of the list in each iteration, that does not correlate with sorting functionality.
The most efficient sorting algorithms have the time complexity of O(n), but with some limitations. Three most famous algorithm with complexity of O(n) are :
Counting sort with time complexity of O(n+k), while k is the maximum number in given list. Assuming n>>k, you can consider its time complexity as O(n)
Radix sort with time complexity of O(d*(n+k)), where k is maximum number of input list and d is maximum number of digits you may have in input list. Similar to counting sort assuming n>>k && n>>d => time complexity will be O(n)
Bucket sort with time complexity of O(n)
But in general due to limitation of each of these algorithms most implementation relies on O(n* log n) algorithms, such as merge sort, quick sort, and heap sort.
Also there are some sorting algorithms with time complexity of O(n^2) which are recommended for list with smaller sizes such as insertion sort, selection sort, and bubble sort.
Using a PLA it might be possible to implement counting sort for a few elements with a low range of values.
count each amount in parallel and sum using lg2(N) steps
find the offset of each element in lg2(N) steps
write the array in O(1)
Only massive parallel computation would be able to do this, general purpose CPU's would not do here unless they implement it in silicon as part of their SIMD.
Consider this problem:
A comparison-based sorting algorithm sorts an array with n items. For which fraction of n! permutations, the number of comparisons may be cn where c is a constant?
I know the best time complexity for sorting an array with arbitrary items is O(nlogn) and it doesn't depend on any order, right? So, there is no fraction that leads to cn comparisons. Please guide me if I am wrong.
This depends on the sorting algorithm you use.
Optimized Bubble Sort for example, compares all neighboring elements of an array and swaps them when the left element is larger then right one. This is repeated until no swaps where performed.
When you give Bubble Sort a sorted array it won't perform any swaps in the first iteration and thus sorts in O(n).
On the other hand, Heapsort will take O(n log n) independent of the order of the input.
Edit:
To answer your question for a given sorting algorithm, might be non-trivial. Only one out of n! permutations is sorted (assuming no duplicates for simplicity). However, for the example of bubblesort you could (starting for the sorted array) swap each pair of neighboring elements. This input will take Bubblesort two iterations which is also O(n).
Consider two k-bit numbers (in binary representation):
$$A = A_1 A_2 A_3 A_4 ... A_k $$
$$B = B_1 B_2 B_3 B_4 ... B_k $$
to compare we scan from left to right looking for an occurrence of a 0 and check opposite number if that digit is also a 0 (for both numbers) noticing that if ever such a case is found then the source of the 0 is less than the source of the 1. But what if the numbers are:
111111111111
111111111110
clearly this will require scanning the whole number and if we are told nothing about the numbers ahead of time and simply given them then:
Comparison take $O(k)$ time.
Therefore when we look at the code for a sorting method such as high-performance quick sort:
HPQuicksort(list): T(n)
check if list is sorted: if so return list
compute median: O(n) time (or technically: O(nk))
Create empty list $L_1$, $L_2$, and $L_3$ O(1) time
Scan through list O(n)
if element is less place into $L_1$ O(k)
if element is more place into $L_2$ O(k)
if element is equal place into $L_3$ O(k)
return concatenation of HP sorted $L_1$, $L_3$, $L_2$ 2 T(n/2)
Thus: T(n) = O(n) + O(nk) + 2*T(n/2) ---> T(n) = O(nklog(n))
Which means quicksort is slower than radix sort.
Why do we still use it then?
There seem to be two independent questions here:
Why do we claim that comparisons take time O(1) when analyzing sorting algorithms, when in reality they might not?
Why would we use quicksort on large integers instead of radix sort?
For (1), typically, the runtime analysis of sorting algorithms is measured in terms of the number of comparisons made rather than in terms of the total number of operations performed. For example, the famous sorting lower bound gives a lower bound in terms of number of comparisons, and the analyses of quicksort, heapsort, selection sort, etc. all work by counting comparisons. This is useful for a few reasons. First, typically, a sorting algorithm will be implemented by being given an array and some comparison function used to compare them (for example, C's qsort or Java's Arrays.sort). From the perspective of the sorting algorithm, this is a black box. Therefore, it makes sense to analyze the algorithm by trying to minimize the number of calls to the black box. Second, if we do perform our analyses of sorting algorithms by counting comparisons, it's easy to then determine the overall runtime by multiplying the number of comparisons by the cost of a comparison. For example, you correctly determined that sorting n k-bit integers will take expected time O(kn log n) using quicksort, since you can just multiply the number of comparisons by the cost of a comparison.
For your second question - why would we use quicksort on large integers instead of radix sort? - typically, you would actually use radix sort in this context, not quicksort, for the specific reason that you pointed out. Quicksort is a great sorting algorithm for sorting objects that can be compared to one another and has excellent performance, but radix sort frequently outperforms it on large arrays of large strings or integers.
Hope this helps!
Is it theoretically possible to sort an array of n integers in an amortized complexity of O(n)?
What about trying to create a worst case of O(n) complexity?
Most of the algorithms today are built on O(nlogn) average + O(n^2) worst case.
Some, while using more memory are O(nlogn) worst.
Can you with no limitation on memory usage create such an algorithm?
What if your memory is limited? how will this hurt your algorithm?
Any page on the intertubes that deals with comparison-based sorts will tell you that you cannot sort faster than O(n lg n) with comparison sorts. That is, if your sorting algorithm decides the order by comparing 2 elements against each other, you cannot do better than that. Examples include quicksort, bubblesort, mergesort.
Some algorithms, like count sort or bucket sort or radix sort do not use comparisons. Instead, they rely on the properties of the data itself, like the range of values in the data or the size of the data value.
Those algorithms might have faster complexities. Here is an example scenario:
You are sorting 10^6 integers, and each integer is between 0 and 10. Then you can just count the number of zeros, ones, twos, etc. and spit them back out in sorted order. That is how countsort works, in O(n + m) where m is the number of values your datum can take (in this case, m=11).
Another:
You are sorting 10^6 binary strings that are all at most 5 characters in length. You can use the radix sort for that: first split them into 2 buckets depending on their first character, then radix-sort them for the second character, third, fourth and fifth. As long as each step is a stable sort, you should end up with a perfectly sorted list in O(nm), where m is the number of digits or bits in your datum (in this case, m=5).
But in the general case, you cannot sort faster than O(n lg n) reliably (using a comparison sort).
I'm not quite happy with the accepted answer so far. So I'm retrying an answer:
Is it theoretically possible to sort an array of n integers in an amortized complexity of O(n)?
The answer to this question depends on the machine that would execute the sorting algorithm. If you have a random access machine, which can operate on exactly 1 bit, you can do radix sort for integers with at most k bits, which was already suggested. So you end up with complexity O(kn).
But if you are operating on a fixed size word machine with a word size of at least k bits (which all consumer computers are), the best you can achieve is O(n log n). This is because either log n < k or you could do a count sort first and then sort with a O (n log n) algorithm, which would yield the first case also.
What about trying to create a worst case of O(n) complexity?
That is not possible. A link was already given. The idea of the proof is that in order to be able to sort, you have to decide for every element to be sorted if it is larger or smaller to any other element to be sorted. By using transitivity this can be represented as a decision tree, which has n nodes and log n depth at best. So if you want to have performance better than Ω(n log n) this means removing edges from that decision tree. But if the decision tree is not complete, than how can you make sure that you have made a correct decision about some elements a and b?
Can you with no limitation on memory usage create such an algorithm?
So as from above that is not possible. And the remaining questions are therefore of no relevance.
If the integers are in a limited range then an O(n) "sort" of them would involve having a bit vector of "n" bits ... looping over the integers in question and setting the n%8 bit of offset n//8 in that byte array to true. That is an "O(n)" operation. Another loop over that bit array to list/enumerate/return/print all the set bits is, likewise, an O(n) operation. (Naturally O(2n) is reduced to O(n)).
This is a special case where n is small enough to fit within memory or in a file (with seek()) operations). It is not a general solution; but it is described in Bentley's "Programming Pearls" --- and was allegedly a practical solution to a real-world problem (involving something like a "freelist" of telephone numbers ... something like: find the first available phone number that could be issued to a new subscriber).
(Note: log(10*10) is ~24 bits to represent every possible integer up to 10 digits in length ... so there's plenty of room in 2*31 bits of a typical Unix/Linux maximum sized memory mapping).
I believe you are looking for radix sort.