I was thinking about sorting algorithm that starts with categorizing numbers by their length (i.e group with number 3 and 4, and another with 100 and 101) and then sorting them in their collective groups which i think would decrease sorting time (maybe even adding first number grouping to even further decrease the comparing time). Does that make sense? Do you have any recommendation how one would start that?
For simplicity, assume that you split the dataset in k subsets of equal size n. The sorting workload would become k.n.log(n) instead of k.n.log(k.n) = k.n.log(n) + k.log(k).n.
So from a theoretical standpoint, this makes no difference because both asymptotic complexities are O(n.log(n)). From a practical standpoint, you have to trade the gain on the number of key comparisons, to the loss due to extra element classification by length and pre/post-sort moves.
Only an actual implementation can tell you.
Related
I've been learning data structures and algorithms from a book, in which it compares time efficiency in terms of number of steps taken by various sorting algorithms. I'm confused as to what we define as one step while doing this.
So while counting no. of steps we consider the worst case scenarios. I understood how we come up with the no. of steps for bubble sort. But for selection sort, I am confused about the part where we compare every element with the current lowest value.
For example, in the worst case array, lets say 5,4,3,2,1, and lets say we are in the first pass through. When we start, 5 is the current lowest value. When we move to 4, and compare it to 5, we change the current lowest value to 4.
Why isnt this action of changing the current lowest value to 4 counted as a swap or an additional step? I mean, it is a step separate from the comparison step. The book I am referring to states that in the first passthrough, the number of comparisons are n-1 but the no. of swaps is only 1, even in worst case, for an n size array. Here they are assuming that the step of changing the current lowest value is a part of the comparison step, which I think should not be a valid assumption, since there can be an array in which you compare but don't need to change the lowest current value and hence your no. of steps eventually reduce. Point being, we cant assume that the no. of steps in the first pass through for selection sort in the worst case is n-1 (comparisons) + 1 (swap). It should be more than (n-1) + (1).
I understand that both selection sort and bubble sort lie in the same classification of time complexity as per big O methodology, but the book goes on to claim that selection sort has lesser steps than bubble sort in worst case scenarios, and I'm doubting that. This is what the book says: https://ibb.co/dxFP0
Generally in these kinds of exercises you’re interested in whether the algorithm is O(1), O(n), O(n^2) or something higher. You’re generally not interested in O(1) vs O(2) or in O(3n) vs O(5n) because for sufficiently large n only the power of n matters.
To put it another way, small differences in the complexity of each step, maybe favors of 2 or 3 or even 10, don’t matter against choosing an algorithm with a factor of n = 300 or more additional work
I've been studying sorting algorithms and had a question about the number of comparisons in each sorting algorithm.
Let's say we have a sorting algorithm (insertion sort, quicksort, anything). Then I want to count the number of comparisons using different files. These files have items that are randomized and not in order. For example, file 1 has 10 items, containing letters a to j. Then we have another file (again, 10 items) containing integers 1 to 10. Then we have another file (10 items), containing float numbers 1.1111111111 to 10.1111111111. If we want to sort these using any sorting algorithm (for the first one we sort in alphabetical order and others from smallest to largest number).
If we count the number of comparisons (in a quicksort algorithm, for example) in each file, would they be the same since we are comparing the same number of items, or does the length of the items change the number of comparisons (a vs 10.1111111)? If they are the same, is it the case for all sorting algorithms (at least the ones I mentioned) or just some? I don't think it's a hard question (sorry), but I'm over-thinking as usual. I'm guessing that they would be the same, but I'm not really. Would someone care to explain?
The number of comparisons depends on the initial state. the sorting algorithm and the specific implementation.
For example:
The implementation could make a first pass to check if the set is already sorted up or down to avoid unnecessary work or even a worst case scenario. This has a small cost but can avoid a pathological case. The number of comparisons will be very different for the same set between an implementation that does and one that does not.
Some implementation choices such as which element to select as a pivot in qsort() will greatly impact the number of comparisons for identical sets.
Even worse: to avoid quadratic worst case in qsort() that can be triggered more of less easily as described in Kernighan's paper anti qsort, one can implement qsort() to make non deterministic choices of pivot values, using some source of randomness. For such an implementation, the number of comparisons may vary, even for sorting the same set repeatedly. Note that this can produce a different order if some elements compare equal, due to qsort()s unstability.
Your teacher's question cannot be answered precisely unless you know both the initial state and the sorting algorithm specific implementation. Even best case and worst case numbers depend on the implementation details.
You are considering performance of algorithm with varies in input files. To standardize this kind of problems, scientist already gave three types of performance for every algorithm :
Best Case - Lower Bound on cost
Worst case - Upper Bound on cost
Average case - "Expected cost"
Now if you want to get number of comparison it makes with particular input then you can form your own mathematical model. But rather for standardization you can think of these three types. And another thing is, number of comparison doesn't varies with input type, but in which order data is. That means if you pass sorted input to the insertion sort, it will give you O(N) with approx N comparisons. But if it is in reverse form, then its worst case.
This is the analysis of sorting:
Reference : Princeton course
This is a homework problem.So I am looking for hints rather than the solution. Consider a set of n numbers. Each number is 'k' digits long. Suppose 'k' is much much larger and does not fit into a single word of memory. In such a scenario what is the complexity of mergesort and radix sort?
My analysis is - Asymptotic complexity doesn't depend on the underlying architectural details like the number of words a number occupies etc. May be the constant factor changes and algorithms run slower, but the overall complexity remains the same. For instance,in languages like Python that handle arbitrarily long integers, the algorithms remain the same. But a few of my friends argue that as the number of words occupied by a number 'w' grows towards infinity, the complexity does change.
Am I on the right path?
The runtime of an algorithm can indeed depend on the number of machine words making up the input. As an example, take integer multiplication. The computer can compute the product of two one-word numbers in time O(1), but it can't compute the product of two arbitrarily-sized numbers in time O(1) because the machine has to load each word into memory as part of its computation.
As a hint for radix sort versus mergesort - the mergesort algorithm makes O(n log n) comparisons between elements, but those comparisons might not take time O(1) each. How much time does it take to compare two numbers that require k machine words each? Similarly, radix sort's runtime depends on the number of digits in the number. How many rounds of radix sort do you need if you have k machine words in each number?
Hope this helps!
You're sort of correct. This is a large part of why most complexity analysis will (somewhere, at least implicitly) state that it's working with the count of some basic operations, not actual time. You generally take for granted that most (if not all) of those basic operations (e.g., comparison, swapping, math like addition or subtraction, etc.) are constant time, which lets you translate almost directly from operation count (the actual complexity) to time consumed.
To be entirely accurate, however, asymptotic complexity is (should) normally specified in terms of a count of fundamental operations, though, not actual time consumed.
Counting sort is known with linear time if we know that all elements in the array are upper bounded by a given number. If we take a general array, cant we just scan the array in linear time, to find the maximum value in the array and then to apply counting sort?
It is not enough to know the upper bound to run a counting sort: you need to have enough memory to fit all the counters.
Consider a situation when you go through an array of 64-bit integers, and find out that the largest element is 2^60. This would mean two things:
You need an O(2^60) memory, and
It is going to take O(2^60) to complete the sort.
The fact that O(2^60) is the same as O(1) is of little help here, because the constant factor is simply too large. This is very often a problem with pseudo-polynomial time algorithms.
Suppose the largest number is like 235684121.
Then you'll spend incredible amounts of RAM to keep your buckets.
I would like to mention something with #dasblinkenlight and #AlbinSunnanbo answers, your idea to scan the array in O(n) pass, to find the maximum value in the array is okay. Below is given from Wikipedia:
However, if the value of k is not already known then it may be
computed by an additional loop over the data to determine the maximum
key value that actually occurs within the data.
As the time complexity is O(n + k) and k should be under a certain limit, your found k should be small. As #dasblinkenlight mentioned, O(large_value) can't practically be converged to O(1).
Though I don't know about any major applications of Counting sort so far except used as a subroutine of Radix Sort, it can be nicely used in problems like string sorting( i.e. sort "android" to "addnoir") as here k is only 255.
How are algorithms analyzed? What makes quicksort have an O(n^2) worst-case performance while merge sort has an O(n log(n)) worst-case performance?
That's a topic for an entire semester. Ultimately we are talking about the upper bound on the number of operations that must be completed before the algorithm finishes as a function of the size of the input. We do not include the coeffecients (ie 10N vs 4N^2) because for N large enough, it doesn't matter anymore.
How to prove what the big-oh of an algorithm is can be quite difficult. It requires a formal proof and there are many techniques. Often a good adhoc way is to just count how many passes on the data the algorithm makes. For instance, if your algorithm has nested for loops, then for each of N items you must operate N times. That would generally be O(N^2).
As to merge sort, you split the data in half over and over. That takes log2(n). And for each split you make a pass on the data, which gives N log(n).
quick sort is a bit trickier because in the average case it is also n log (n). You have to imagine what happens if your partition splits the data such that every time you get only one element on one side of the partition. Then you will need to split the data n times instead of log(n) times which makes it N^2. The advantage of quicksort is that it can be done in place, and that we usually get closer to N log(n) performance.
This is introductory analysis of algorithms course material.
An operation is defined (ie, multiplication) and the analysis is performed in terms of either space or time.
This operation is counted in terms of space or time. Typically analyses are performed as Time being the dependent variable upon Input Size.
Example pseudocode:
foreach $elem in #list
op();
endfor
There will be n operations performed, where n is the size of #list. Count it yourself if you don't believe me.
To analyze quicksort and mergesort requires a decent level of what is known as mathematical sophistication. Loosely, you solve a discrete differential equation derived from the recursive relation.
Both quicksort and merge sort split the array into two, sort each part recursively, then combine the result. Quicksort splits by choosing a "pivot" element and partitioning the array into smaller or greater then the pivot. Merge sort splits arbitrarily and then merges the results in linear time. In both cases a single step is O(n), and if the array size halves each time this would give a logarithmic number of steps. So we would expect O(n log(n)).
However quicksort has a worst case where the split is always uneven so you don't get a number of steps proportional to the logarithmic of n, but a number of steps proportional to n. Merge sort splits exactly into two halves (or as close as possible) so it doesn't have this problem.
Quick sort has many variants depending on pivot selection
Let's assume we always select 1st item in the array as a pivot
If the input array is sorted then Quick sort will be only a kind of selection sort!
Because you are not really dividing the array.. you are only picking first item in each cycle
On the other hand merge sort will always divide the input array in the same manner, regardless of its content!
Also note: the best performance in divide and conquer when divisions length are -nearly- equal !
Analysing algorithms is a painstaking effort, and it is error-prone. I would compare it with a question like, how much chance do I have to get dealt two aces in a bridge game. One has to carefully consider all possibilities and must not overlook that the aces can arrive in any order.
So what one does for analysing those algorithms is going through an actual pseudo code of the algorithm and add what result a worst case situation would have. In the following I will paint with a large brush.
For quicksort one has to choose a pivot to split the set. In a case of dramatic bad luck the set splits in a set of n-1 and a set of 1 each time, for n steps, where each steps means inspecting n elements. This arrive at N^2
For merge sort one starts by splitting the sequence into in order sequences. Even in the worst case that means at most n sequences. Those can be combined two by two, then the larger sets are combined two by two etc. However those (at most) n/2 first combinations deal with extremely small subsets, and the last step deals with subsets that have about size n, but there is just one such step. This arrives at N.log(N)