I'm trying to understand the time complexity of insertion sort. I got stuck at while loop. I'm unable to understand how many times while loop executes
InsertionSort(A)
for j = 2 to A.length
key = A[j]
i = j - 1
while i>0 and A[i]>key
A[i+1] = A[i]
i = i - 1
A[i+1] = key
I know that for loop executes n+1 times and every statement in the loop execute n times
while loop also executes n times
But, what I don't understand is "How many times statements under while loop executes for both worst and best cases?"
In the worst case, A is sorted in descending order, which means that for the j'th entry, the inner loop will run j times (give or take a "+1" or "-1"...). Happily, there is a formula for that: as Gauss famously found out spontaneously and under duress, summing up all numbers from 1 to n yields a result of n*(n+1)/2.
As we only care about complexity and not actual values, we can leave the constant and multiplicative factors off and end up with O(n^2).
Tongue-in-cheek aside, the fact that there is a loop within a loop is a strong indication for O(n^2) when the inner loop count is bounded linearly - which it is here.
Best case, with A already sorted in ascending order, the inner loop will be entered not at all, and overall complexity will be O(n).
The average case depends heavily on what your expected "unorderedness" looks like. For example, the sort will behave greatly if your list is basically always sorted already, and there are only very few, very local switchups.
Related
I have am trying to learn Big O and am confused on an algorithm I just came across. The algorithm is:
void pairs(int[] array){
for (int i=0; i < array.length; i++){
for (int j=i+1; j<array.length; j++){
System.out.println(array[i]+","+array[j]);
}
}
}
I think the first for loop is O(n) and I know the second for loop is O(1/2*n(n+1)). The answer to the problem was that the run time for the function is O(n^2). I simplified O(1/2*n(n+1)) to O(1/2*(n^2+n)). So I'm confused because I thought that you needed to multiply the two run time terms since the for loop is nested, which would give O(n) * O(1/2*(n^2+n)). I simplified this to O(1/2n^3 + 1/2n^2). From what I understand of Big O, you only keep the largest term so this would reduce to O(n^3). Can anyone help me out with where I went wrong? Not sure how the answer is O(n^2) instead of O(n^3).
When you say the inner loop is O(1/2*n(n+1)), you are actually describing the big-O complexity of both loops.
To say that the outer loop has complexity O(N) basically means its body runs N times. But for your calculation of the inner loop's complexity, you already took account of all iterations of the outer loop, because you added up the number of times the inner loop runs over all iterations of the outer loop. If you multiply again by N, you would be saying that the outer loop itself is re-run another N times.
Put another way, what your analysis shows is that the inner loop body (the System.out.println call) runs 1/2*n(n+1) times overall. That means the overall complexity of the two-loop combination is O(1/2*n(n+1)) = O(n^2). The overall complexity of the two-loop combination describes how many times the innermost code is run.
your mistake is counting the second loop as O(1/2n^2)...
first, you can clearly see it is capped to N-1 (when j = 0)
first loop is clearly N
Second loop is MAX of N-1...
threrefore, O(N^2)...
if we examine it little more,
second loop will run N-1 when i=0,
then N-2 for i=1,
and ONE single time for i=n-1,
this is 1/2n(n-1) = 1/2n^2 - 1/2n = O(n^2)
Notice this includes all iteration of the outer loop too!
I don't mean to be asking for help with something simple, but I can't seem to figure out how to answer this question.
Compute the time complexity of the following program
fragment:
sum = 0;
for i=1 to n do
for j=1 to i do
k = n*2
while k>0 do
sum=sum+1;
k = k div 2;
I recognize that what is inside the while loop takes O(1), the while loop takes O(logn), but then I don't follow how that connects to the nested for loops, since I am used to just doing nested sigma notations for for loops.
Thanks!
A formal demonstration which shows step by step the order of growth of your algorithm:
Here are some hints on to break down this function's complexity:
Look at the inner loop, where k=n*2. Lets assume n=8 so k=16, k keeps being divided by 2 until it's 0 or less (i'll assume here that rounding 0.5 yields 0). So the series describing k until the end of the loop will be 16,8,4,2,1,0. try to think what function describes the number of elements in this series, if you know the first value is k.
You have two nested for loops, the first loop just iterates n times, then second (inner) loop iterates until it reaches the number of the iteration of the first loop (represented by i), which means at first it will iterate once, then twice, and so on until n. So the number of iterations performed by the second loop can be described by the series: 1, 2, 3, ... , n. This is a very simple arithmetic progression, and the sum of this series will give you the total number of iterations of the inner for loop. This is also the number of times you call the inner while loop (which is not affected by the number of the current iteration, as k depends on n which is constant and not on i or j).
i m calculating running time for this algorithm?
Cost No Of Times
for(j=1;j<=n-1;j++){ c1 n(loop will run for n-1 times +1 for failed cond
for(i=0;i<=n-2;i++){ c2 n*(n-1) (n-1 from outer loop and n for inner
if(a[i]>a[i+1]){ c3 (n-1)*(n-1)
Swap c4 (n-1)*(n-1) {in worst case }
}
}
in worst case
T(n)= c1*n + c2*(n-1)n + c3(n-1)(n-1) + c4*(n-1)(n-1)
which is O(n^2)
In Best case:
T(n)=c1*n + c2*(n-1)n + c3(n-1)(n-1)
which is O(n^2).
BUT actually in best case bubble sort has time complexity O(n).
Can anyone explain?
Bubble Sort has O(n) time complexity in the best case because it is possible to pass an already sorted list to it.
You have to check if you did any swaps after the second nested loop. If no swaps were done, the list is sorted and there's no need to continue, so you can break the loop.
For an already-sorted list, you'd have iterated over all n elements once in this case.
your algo for implementing bubble sort is correct but not efficient,
// n is the total number of elments
do{
swp = false // swp checks whether or not any variable has been swapped
in the inner loop
for(i=0;i<=n-2;i++){
if(a[i]>a[i+1])
{
swap(a[i],a[i+1])
sw = true
}
n = n-1
}while(sw == true && n>0)
swp is a variable which checks whether there has been any swap in the inner loop or not,
if there has not been any swap this means that our array is sorted.
The best case for bubble sort is when the elements are already sorted in ascending order(in this case)
for which the inner loop just runs once but the if condition(in the inner loop) is never satisfied and swp remains false and thus we exit from the outer loop after one iteration which gives bubble sort O(n) complexity.
You can compute the number of iterations (what's inside the loop is irrelevant because it's of constant time) using Sigma Notation:
Bubble Sort with a best case running time is actually an enhanced version of this sorting algorithm.
During the first parse (outer loop), if no swap was performed, that is a decisive information that the array is sorted, and it is pointless to cover all cases.
Therefore, the outer loop would iterate once, and the inner loop would iterate n times: that's n + 1 iterations overall ==> O(n).
This is my first question and I'm pretty sure that I will receive also my first answer :P
I have to make an asymptotic analysis of an algorithm that from an array A[1...n] compute a matrix M[n][n] which contains for each M[i][j] a value given by:
M[i][j] = (A[i] + ... + A[j])/(j-i+1), if i<=j
and
M[i][j] = 0, if i>j
for i=1 to N do [N]
for j=1 to N do [N]
if(i<=j) [cost]
start=i [cost]
end=j [cost]
sum=0 [cost]
for a=start to end do [??]
sum += A[a] [cost]
M[i][j]=sum/(j-i+1 [cost]
else [cost]
M[i][j]=0 [cost]
Considering that giving the first two for loop I have to expect at least a running time of O(n^2), with the third inner for loop I will get something like O(N*N*[??]).
The third for loop executes everytime j-i+1 operations and only for i<=j.
The matrix will have the first row filled with the computed average, the second the first element equal to 0 and then the computed averages...
The final matrix will result almost half filled(but not N/2) So the value for the third loop is not [N/2]
How can I compute the running time for the innermost For and also the running time for the whole algorithm?
You could try to just calculate the number of times the inner loop statement gets executed.
You loop over i from 1 to N. Now you will only do the inner loop (with the sum) when j is larger than or equal to N so j loops from i to N. The inner most loop then consists of j-i additions. In all you get (in maple syntax)
sum(sum(j-i,j=i..N), i=1..N)= 1/6*N^3-1/6*N
You then need to take into account the assignment of M[i][j] which is done N^2 times.
The previous was the total amount of instructions. However, if you're just looking for the overal complexity then you should just see that the inner loop is dependent on N which gives you O(N^3) overall complexity.
Do note that the code could avoid the inner loop complexity by storing the A[i] sums from the start and don't try to recalculate them every time
sum = 0;
for(int i = 0; i < N; i++)
for(int j = i; j >= 0; j--)
sum++;
From what I understand, the first line is 1 operation, 2nd line is (i+1) operations, 3rd line is (i-1) operations, and 4th line is n operations. Does this mean that the running time would be 1 + (i+1)(i-1) + n? It's just these last steps that confuse me.
To analyze the algorithm you don't want to go line by line asking "how much time does this particular line contribute?" The reason is that each line doesn't execute the same number of times. For example, the innermost line is executed a whole bunch of times, compared to the first line which is run just once.
To analyze an algorithm like this, try identifying some quantity whose value is within a constant factor of the total runtime of the algorithm. In this case, that quantity would probably be "how many times does the line sum++ execute?", since if we know this value, we know the total amount of time that's spent by the two loops in the algorithm. To figure this out, let's trace out what happens with these loops. On the first iteration of the outer loop, i == 0 and so the inner loop will execute exactly once (counting down from 0 to 0). On the second iteration of the outer loop, i == 1 and the inner loop executes exactly twice (first with j == 1, once with j == 0. More generally, on the kth iteration of the outer loop, the inner loop executes k + 1 times. This means that the total number of iterations of the innermost loop is given by
1 + 2 + 3 + ... + N
This quantity can be shown to be equal to
N (N + 1) N^2 + N N^2 N
--------- = ------- = --- + ---
2 2 2 2
Of these two terms, the N^2 / 2 term is the dominant growth term, and so if we ignore its constant factors we get a runtime of O(N2).
Don't look at this answer as something you should memorize - think of all of the steps required to get to the answer. We started by finding some quantity to count, and then saw how that quantity was influenced by the execution of the loops. From this, we were able to derive a mathematical expression for that quantity, which we then simplified. Finally, we took the resulting expression and determined the dominant term, which serves as the big-O for the overall function.
Work from inside-out.
sum++
This is a single operation on it's own, as it doesn't repeat.
for(int j = i; j >= 0; j--)
This loops i+1 times. There are several operations in there, but you probably don't mean to count the number of asm instructions. So I'll assume for this question this is a multiplier of i+1. Since the loop contents is a single operation, the loop and its block perform i+1 operations.
for(int i = 0; i < N; i++)
This loops N times. So as before, this is a multiplier of N. Since the block performs i+1 operations, this loop performs N(N+1)/2 operations in total. And that's your answer! If you want to consider big-O complexity, then this simplifies to O(N2).
It's not additive: the inner loop happens once for EACH iteration of the outer loop. So it's O(n2).
By the way, this is a good example of why we use asymptotic notation for this kind of thing -- depending on the definition of "operation" the exact details of the count could vary pretty widely. (Like, is sum++ a single operation, or is it add sum to 1 giving temp; load temp to sum?) But since we know that all that can be hidden in a constant factor, it's still going to be O(n2).
No; you don't count a specific number of operations for each line and then add them up. The entire point of constructions like 'for' is to make it possible for a given line of code to run more than once. You're supposed to use thinking and logic skills to figure out how many times the line 'sum++' will run, as a function of N. Hint: it runs once for every time that the third line is encountered.
How many times is the second line encountered?
Each time the second line is encountered, the value of 'i' is set. How many times does the third line run with that value of i? Therefore, how many times will it run overall? (Hint: if I give you a different amount of money on several different occasions, how do you find out the total amount I gave you?)
Each time the third line is encountered, the fourth line happens once.
Which line happens most often? How often does it happen, in terms of N?
So guess what interest you is the sum++ and how many time you execute it.
The final stat of sum would give you that answer.
Actually your loop is just:
Sigma(n) n goes from 1 to N.
Which equal to: N*(N+1) / 2 This give you in big-o-notation O(N^2)
Also beside the name of you question there is no worst case in you algorithm.
Or you could say that the worst case is when N goes to infinity.
Using Sigma notation to represent your loops: