I would like to understand Asymptotic Analysis better since I believe I don't have solid understanding on that. I would appreciate if someone can highlight a better approach to it. Here are two examples
for (int i = 1; i <= n; i *= 2) {
for (int j = 0; j < n; j++) {
count++;
}
}
This question is from Quiz and its answer is O(n log n)
I watched a lecture of Stanford University and its example is below
for i = 1 to n
for j = i + 1 to n
if A[i] == A [j] return TRUE otherwise
return FALSE
Asymptotic Analysis for second given problem is Quadratic O(n^2)
How can I know when is O(n log n) or O(n^2) they both have nested for loop?
Any answer is highly appreciated. Thanks beforehand
The first example is O(nlogn) because the outer loop repeats log(n) times, and each of its repeat require O(n) repeats of the inner loop - so this totals in O(log(n) * n) = O(nlogn).
However, in the 2nd example - the outer loop requires O(n) iterations, and for each iteration i - it requires O(n-i) iterations of the inner loop. This means it will take n + (n-1) + (n-2) + ... + 2 + 1 total time. This is arithmetic progression, and the sum of it is in O(n^2)
There is no "easy way" to know the complexity without understanding some of what happens - complexity analysis is case dependent.
However, there are some hints that might help you, for example - if the iteration counter is multiplying each iteration - it is a strong indication a logarithm will be involved in the complexity function, like in your first example.
Related
My approach (might be incorrect) is formulaic. If there is a loop then (n+1) if there is a nested loop (n^2) if a statement then O(1). If division then log(n).
Here are some example and my reasoning to solving, not sure at all if this approach is problematic or if any of them are correct. I need help with this.
Example1:
i = n; // I think O(1) because it's a statment
while (i > 1) // I think O(n) because it's a loop
i = i/4; // O(n) because it's in a loop and log_4(n) b/c division
// I think Overall if we combine the O(n) from earlier and the log_4(n)
// Therefore, I think overall O(nlog(n))
Example2:
for (i = 1; i < n; i = i + i) // I think this is O(n+1) thus, O(n)
System.out.println("Hello World"); // O(n) because it's in a loop
// Therefore, overall I think O(n)
Example3:
for (i = 0; i < n; i = i + 1) // I think O(n+1), thus O(n)
for (j = 1; j < n; j++) // I think O(n^2) because in a nested loop
System.out.println("Hello Universe!"); // O(n^2) because in a nested
// Therefore, overall I think O(n^2)
Example4:
for (i = 1; i < (n * n + 3 * n + 17) / 4; i = i + 1) // O((20+n^3)/4) thus, O(n^3)
System.out.println("Hello Again!'); // O(n) because it's in a loop
// Therefore, overall I think O(n^3) because largest Big-O in the code
Thank you
Example1: Your result is wrong. Because the loop happens log_4(n) times and also the division takes O(1) (division by 4 just needs a bitwise shift). Thus the overall time is only O(log(n))
Example2: It's wrong too. In each iteration you duplicate the loop variable. So the loop happens O(log(n)) times. The print command takes O(1) and total time is O(log(n)).
Example3: Your answer is correct. Because you have two nested O(n) loops. Note that this loop is different from two previous examples.
Example4: I think you made a writing mistake. Does ((n * n + 3 * n + 17) / 4) equal O((20+n^3)/4)??? It is O(n^2). Thus, according to my previous explanations, the overall time is O(n^2).
I'm first asked to develop a simple sorting algorithm that sorts an array of integers in ascending order and put it to code:
int i, j;
for ( i = 0; i < n - 1; i++)
{
if(A[i] > A[i+1])
swap(A, i+1, i);
for (j = n - 2; j >0 ; j--)
if(A[j] < A[j-1])
swap(A, j-1, j);
}
Now that I have the sort function, I'm asked to do a theoretical analysis for the running time of the algorithm. It says that the answer is O(n^2) but I'm not quite sure how to prove that complexity.
What I know so far is that the 1st loop runs from 0 to n-1, (so n-1 times), and the 2nd loop from n-2 to 0, (so n-2 times).
Doing the recurrence relation:
let C(n) = the number of comparisons
for C(2) = C(n-1) + C(n-2)
= C(1) + C(0)
C(2) = 0 comparisons?
C(n) in general would then be: C(n-1) + C(n-2) comparisons?
If anyone could guide my step by step, that would be greatly appreciated.
When doing a "real" big O - time complexity analysis, you select one operation that you count, obviously the one that dominates the running time. In your case you could either choose the comparison or the swap, since worst case there will be a lot of swaps right?
Then you calculate how many times this will be evoked, scaling to input. So in your case you are quite right with your analysis, you simply do this:
C = O((n - 1)(n - 2)) = O(n^2 -3n + 2) = O(n^2)
I come up with these numbers through reasoning about the flow of data in your code. You have one outer for-loop iterating right? Inside that for-loop you have another for-loop iterating. The first for-loop iterates n - 1 times, and the second one n - 2 times. Since they are nested, the actual number of iterations are actually the multiplication of these two, because for every iteration in the outer loop, the whole inner loop runs, doing n - 2 iterations.
As you might know you always remove all but the dominating term when doing time complexity analysis.
There is a lot to add about worst-case complexity and average case, lower bounds, but this will hopefully make you grasp how to reason about big O time complexity analysis.
I've seen a lot of different techniques for actually analyzing the expression, such as your recurrence relation. However I personally prefer to just reason about the code instead. There are few algorithms which have hard upper bounds to compute, lower bounds on the other hand are in general very hard to compute.
Your analysis is correct: the outer loop makes n-1 iterations. The inner loop makes n-2.
So, for each iteration of the outer loop, you have n-2 iterations on the internal loop. Thus, the total number of steps is (n-1)(n-2) = n^2-3n+2.
The dominating term (which is what matters in big-O analysis) is n^2, so you get O(n^2) runtime.
I personally wouldn't use the recurrence method in this case. Writing the recurrence equation is usually helpful in recursive functions, but in simpler algorithms like this, sometimes it's just easier to look at the code and do some simple math.
Basically I am struggling to come to grips with operation counting and Big-O notation. I understand it is possibly one of the harder parts of computer science to understand and I have to admit I am struggling with it. Could anyone give me some help with these examples, and possibly any further help/links with Big-O?
for (i = 0; i < N; i++)
{ for (j = i; j < N; j++)
{ sequence of statements }
}
Here I would say the complexity is O(N²) - Quadratic
int m = -9
for (j = 0; j < n; j+=5)
{
if (j<m)
{
for (k = 1; k <n; k*=3)
{some code}
}
}
Here I would also say is O(N²). The first loop takes N and the second loop takes N so I would say the answer is O(N*N) which is equal to O(N²).
Any help and advice for further understanding would be great!!
The first is indeed O(n^2), as you suspected, assuming the 'sequence of statements' is O(1).
However, the second part of code is O(n), since the condition j < m is never met - and thus, the outer loop only iterates itself without actually doing nothing. The inner loop is not even reachable.
As side note, some compilers may actually optimize the second part of code to run in O(1) by just setting the end values of variables, but this is not the point of the question.
The second example is complexity O(N).
int m = -9
for (j = 0; j < n; j+=5)
{
if (j<m)
{
// this never executes; m is negative and j is positive
}
}
First example:
The inner loop executes N times when i = 0, N-1 times when i = 1, and so on...
You can just calculate the number of steps the for loops execute
(N) + (N - 1) + (N - 2) + ... + 2 + 1
steps = N(N+1)/2 = (N^2 + N) / 2
N <=> 1 |add the left to the right| => N+1
(N - 1) <=> 2 |add the left to the right| => N+1
(N - 2) <=> 3 |add the left to the right| => N+1 . . . N
What does the big-O nation means?
F(N) = O(G(N)) means that |F(N)|<= c*|G(N)| where c > 0
It means that the G(N) function is an upper bound on the grothw rate of the function. F(N) could not grow faster than G(N).
Just to throw this out here: if we assume that the j<-9 clause is a mistake and ignore it, then your second example has two nested loops. However the inner loop is actually multiplying k times 3. So this makes this inner loop O(log n). Which makes the pair of loops O(n log n). I'm not sure this is the answer, but you asked for further understanding, so... you know... maybe that's further.
Ok, also on a further note I would really suggest you to go through a single lecture in the series of introduction of algorithms. Believe me you won't need to look any further.
http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-046j-introduction-to-algorithms-sma-5503-fall-2005/video-lectures/
As I understand, the complexity of an algorithm is a maximum number of operations performed while sorting. So, the complexity of Bubble sort should be a sum of arithmmetic progression (from 1 to n-1), not n^2.
The following implementation counts number of comparisons:
public int[] sort(int[] a) {
int operationsCount = 0;
for (int i = 0; i < a.length; i++) {
for(int j = i + 1; j < a.length; j++) {
operationsCount++;
if (a[i] > a[j]) {
int temp = a[i];
a[i] = a[j];
a[j] = temp;
}
}
}
System.out.println(operationsCount);
return a;
}
The ouput for array with 10 elements is 45, so it's a sum of arithmetic progression from 1 to 9.
So why Bubble sort's complexity is n^2, not S(n-1) ?
This is because big-O notation describes the nature of the algorithm. The major term in the expansion (n-1) * (n-2) / 2 is n^2. And so as n increases all other terms become insignificant.
You are welcome to describe it more precisely, but for all intents and purposes the algorithm exhibits behaviour that is of the order n^2. That means if you graph the time complexity against n, you will see a parabolic growth curve.
Let's do a worst case analysis.
In the worst case, the if (a[i] > a[j]) test will always be true, so the next 3 lines of code will be executed in each loop step. The inner loop goes from j=i+1 to n-1, so it will execute Sum_{j=i+1}^{n-1}{k} elementary operations (where k is a constant number of operations that involve the creation of the temp variable, array indexing, and value copying). If you solve the summation, it gives a number of elementary operations that is equal to k(n-i-1). The external loop will repeat this k(n-i-1) elementary operations from i=0 to i=n-1 (ie. Sum_{i=0}^{n-1}{k(n-i-1)}). So, again, if you solve the summation you see that the final number of elementary operations is proportional to n^2. The algorithm is quadratic in the worst case.
As you are incrementing the variable operationsCount before running any code in the inner loop, we can say that k (the number of elementary operations executed inside the inner loop) in our previous analysis is 1. So, solving Sum_{i=0}^{n-1}{n-i-1} gives n^2/2 - n/2, and substituting n with 10 gives a final result of 45, just the same result that you got by running the code.
Worst case scenario:
indicates the longest running time performed by an algorithm given any input of size n
so we will consider the completely backward list for this worst-case scenario
int[] arr= new int[]{9,6,5,3,2};
Number of iteration or for loops required to completely sort it = n-1 //n - number of elements in the list
1st iteration requires (n-1) swapping + 2nd iteration requires (n-2) swapping + ……….. + (n-1)th iteration requires (n-(n-1)) swapping
i.e. (n-1) + (n-2) + ……….. +1 = n/2(a+l) //sum of AP
=n/2((n-1)+1)=n^2/2
so big O notation = O(n^2)
What is the complexity given for the following problem is O(n). Shouldn't it be
O(n^2)? That is because the outer loop is O(n) and inner is also O(n), therefore n*n = O(n^2)?
The answer sheet of this question states that the answer is O(n). How is that possible?
public static void q1d(int n) {
int count = 0;
for (int i = 0; i < n; i++) {
count++;
for (int j = 0; j < n; j++) {
count++;
}
}
}
The complexity for the following problem is O(n^2), how can you obtain that? Can someone please elaborate?
public static void q1E(int n) {
int count = 0;
for (int i = 0; i < n; i++) {
count++;
for (int j = 0; j < n/2; j++) {
count++;
}
}
}
Thanks
The first example is O(n^2), so it seems they've made a mistake. To calculate (informally) the second example, we can do n * (n/2) = (n^2)/2 = O(n^2). If this doesn't make sense, you need to go and brush up what the meaning of something being O(n^k) is.
The complexity of both code is O(n*n)
FIRST
The outer loop runs n times and the inner loop varies from 0 to n-1 times
so
total = 1 + 2 + 3 + 4 ... + n
which if you add the arithmetic progression is n * ( n + 1 ) / 2 is O(n*n)
SECOND
The outer loop runs n times and the inner loop varies from 0 to n-1/2 times
so
total = 1 + 1/2 + 3/2 + 4/2 ... + n/2
which if you add the arithmetic progression is n * ( n + 1 ) / 4 is also O(n*n)
First case is definitely O(n^2)
The second is O(n^2) as well because you omit constants when calculate big O
Your answer sheet is wrong, the first algorithm is clearly O(n^2).
Big-Oh notation is "worst case" so when calculating the Big-Oh value, we generally ignore multiplications / divisions by constants.
That being said, your second example is also O(n^2) in the worst case because, although the inner loop is "only" 1/2 n, the n is the clear bounding factor. In practice the second algorithm will be less than O(n^2) operations -- but Big-Oh is intended to be a "worst case" (ie. maximal bounding) measurement, so the exact number of operations is ignored in favor of focusing on how the algorithm behaves as n approaches infinity.
Both are O(n^2). Your answer is wrong. Or you may have written the question incorrectly.