Q. (1) On input sequence DECR InsertionSort is asymptotically faster than MergeSort?
Q. (2) On input sequence, INCR MergeSort is asymptotically faster than InsertionSort?
Related
I have this algorithm and I wanna analyse the time complexity but I am not sure I am correct:
n = int(input("Enter Target Value: "))
x = 1
count = 0
while n != x:
if n % 2 == 0:
n /= 2
count +=1
else:
n -= 1
count +=1
print(count)
for the while loop, n/2 will have the time complexity of O(logn) and n-1 will be O(n), so O(logn)+O(n) will still be O(logn) in the for loop. The 3 initializing will be O(1), so the run time complexity of this algo will be O(logn). Am I correct? Thanks
The outcome is correct, but the reasoning is not. The n-=1 statement will not be executed O(n) times, and O(logn)+O(n) is actually O(n), not O(logn).
Imagine n in its binary representation. Then the n-=1 operation will be executed just as many times as there are 1-bits in that representation. The n/=2 statement will be executed just as many times as there are bits in the representation, regardless of whether they are 0 or 1. This is because a 1-bit will first be converted to a 0-bit with the n-=1 operation, and then the next iteration will pick up that same bit (which has become 0) for the n/=2 operation, which actually drops that bit.
So in the worst case, all the significant bits of n are 1-bits. And then you have O(logn) executions of n-=1 operation, and O(logn) executions of n/=2. In total the loop makes 2O(logn) iterations, which gives this algorithm a O(logn) time complexity.
An array A of size n is known to be sorted except for the first k elements and last k elements where k is a constant. which of the following algorithms is best suited for sorting the array?
A) Quicksort
B) Bubble sort
C) Selection Sort
D) Insertion Sort
Given answer is D.
unable to understand how this works and also What would have been the answer if Merge sort is also given?
Let's have a look at the complexity of the algorithms:
A) Quicksort: would take worse case O(n²) average O(n log n)
B) Bubble Sort: would take O(n²)
C) Selection Sort: would take O(n²)
D) Insertion Sort: would take O(k* n) if k is constant = O(n)
So D has the best performance.
(for each of the k elements: O(log n) to find the position to insert to + O(n) to insert)
But since Quicksort is known to have a small konstant faktor and is in average O(n log n), it is most likely faster for "bigger" k values.
Extra:
E) merge sort : would take 2 * O(k log k) + O(n)
sort the k elements at the front O(k log k)
sort the k elements at the end O(k log k)
merge the 3 lists O(n)
Over all that makes for constant k O(n) so based on complexity the same as Insertion Sort.
But if you look at it with k not constant:
merge sort: O(k log k) + O(n)
Insertion Sort: O(k* n)
So insertion sort would be faster.
Arguments against merge sort:
In general merge sort is not inplace (insertion sort is), so you would need extra space or a very clever implemenattion variant that manages to do it inplace without much overhead in complexity.
Since the first K and last K elements are constant in numbers, so it really makes no sense in calculating their complexity as it will be constant.
Comparing all above given algos with their complexity:
A) Quicksort: Worst case O(n²) average O(n log n)
B) Bubble Sort: O(n²)
C) Selection Sort: O(n²)
D) Insertion Sort: O(k* n) if k=constant = O(n)
If the inversion count is O(n), then the time complexity of insertion sort is O(n). In worst case, there can be n(n-1)/2 inversions. The worst case occurs when the array is sorted in reverse order. So the worst case time complexity of insertion sort is O(n2).*
So, Quicksort is best in general but for small List Insertion sort has
a Advantage :
Insertion sort is faster for small n because Quick Sort has extra overhead from the recursive function calls. Insertion sort is also
more stable than Quick sort and requires less memory.
See Why is Insertion sort better than Quick sort for small list of elements?
This is the recurrence of the worst-case running time T(n) of the Merge-Sort procedure.
What is T?
why 2T(n/2) ?
For which operation is the O(n) ?
For simplicity, assume that n is a power of 2 so that each divide step yields two subproblems, both of size exactly n/2.
The base case occurs when n = 1.
When n ≥ 2, time for merge sort steps:
Divide: Just compute q as the average of p and r, which takes constant time i.e. Θ(1).
why 2T(n/2) ?
Conquer: Recursively solve 2 subproblems, each of size n/2, which is 2T(n/2).
For which operation is the O(n) ?
Combine: MERGE on an n-element subarray takes Θ(n) time.
Summed together they give a function that is linear in n, which is Θ(n). Therefore, the recurrence for merge sort running time is
As it is proven, comparison based sorting has complexity T(n)=nlogn. So if we have array of positive integers (not specific one-so that counting or radix sorts cannot be applied) how to determine appropriate reduction from Sort to SortPositiveNumbers (for example) in order to prove that there is polynomial and correct transformation and SortPositiveNumbers has also lower bound of nlogn ?
Any help would be appreciated.
We can reduce Sort to SortPositiveNumbers using the following pseudo-code:
Sort(A[1..n])
B[1..n]
p <- n+1
s <- 0
for k <- 1 to n
if A[k] > 0
p <- p-1
B[p] <- A[k]
else if A[k] < 0
s <- s+1
B[s] <- -A[k]
SortPositiveNumbers(B[1...s])
SortPositiveNumbers(B[p...n])
for k <- 1 to s
A[k] <- -B[s+1-k]
for k <- s+1 to p-1
A[k] <- 0
for k <-p to n
A[k] <- B[k]
Here the reduction is linear (2n), so it is polynomial and it is correct because it uses SortPositiveNumbers to sort only positive numbers from A( negatives are converted into positive) and finally the reduction again converts positive into negative and adds them into the needed index of the sorted array.
Let's a assume that you can sort positive numbers with complexity C' < nlogn. Then I can sort arbitrary arrays by spliting them into positive and negative (O(N)) converting the negatives into positives (O(N)) sorting the two arrays (O(C'), times 2 but it doesn't matter for complexity), reverse the negative array O(N) and concatenating the two arrays (O(N)). So even overall the complexity is O(N + C') (that is maximum between O(N) and O(C')). This is lower than nlogn which you have proven to be minimum complexity for sorting arbitrary arrays. This is a contradiction, so it follows that the initial assumptions (the only assumption we've made) is wrong. That is there's co algorithm with a complexity less than nlogn for sorting positive numbers.
How does a program's worst case or average case dependent on log function? How does the base of log come in play?
The log factor appears when you split your problem to k parts, of size n/k each and then "recurse" (or mimic recursion) on some of them.
A simple example is the following loop:
foo(n):
while n > 0:
n = n/2
print n
The above will print n, n/2, n/4, .... , 1 - and there are O(logn) such values.
the complexity of the above program is O(logn), since each printing requires constant amount of time, and number of values n will get along the way is O(logn)
If you are looking for "real life" examples, in quicksort (and for simplicity let's assume splitting to exactly two halves), you split the array of size n to two subarrays of size n/2, and then you recurse on both of them - and invoke the algorithm on each half.
This makes the complexity function of:
T(n) = 2T(n/2) + O(n)
From master theorem, this is in Theta(nlogn).
Similarly, on binary search - you split the problem to two parts, and recurse only on one of them:
T(n) = T(n/2) + 1
Which will be in Theta(logn)
The base is not a factor in big O complexity, because
log_k(n) = log_2(n)/log_2(k)
and log_2(k) is constant, for any constant k.