I'm having troubles understanding how to estimate the Big-O. We've had two lectures on this topic and the only thing I undestand is to take the leading coefficient from the largest polynomial in the function and replace it with an O so it would look like O(...)
During the first lecture this was shown
int i = length;
while (i>0) {
i--;
int j = i -1;
while (j >= 0) {
if (a[i] == a[j]) {
return 1;
}
j--;
}
}
return 0;
Followed by this on the following slide
int i = length; // Counts as 1
while (i>0) { // Counts as N+1
i--; // Counts as N
int j = i -1; // Coutns as N
while (j >= 0) { // Counts as i+1
if (a[i] == a[j]) { // Counts as i
return 1;
}
j--; // Counts as i
}
}
return 0; // Counts as 1
From this, I'm wondering why
return 1;
isn't counted as a step.
Following that slide, it tells us that the
Outer Loop count is 3N+1
Inner Loop count is 3i+1 ; for all possible i from 0 to N-1
I understand that the second [while] loop will occur N times and following that, the [if] will occur i times where i is equal to N-1 since if j < 0, the second while loop will still be read but nothing else will happen after it.
The slide shows that the Total from the Inner loop is equal to
3N^2 - 1/2N
and that the Grand Total is equal to 3/2N^2 + 5/2N +3.
Wondering if anyone has time to walk me through how to acquire the functions used in Big-O estimations like in the example above; I have no idea how 3i+1 translated into 3N^2 - 1/2N as well has how the Grand Total is calculated.
I will try to explain the calculation of the complexity of your example.
First we notice, that every operation requries only constant time, written as O(1), which means, that the run time does not depend on the input.
int i = length; // O(1), executed only one time
while (i > 0) { // outer loop, condition needs O(1)
i--; // O(1)
int j = i - 1; // O(1)
while (j >= 0) { // inner loop, condition needs O(1)
if (a[i] == a[j]) { // O(1)
return 1; // first return statement
}
j--; // O(1)
}
}
return 0; // second return statement, executed only one time
The number of operations in every loop is constant, so we only have to count how often they are executed.
The outer loop runs from i = n to i = 1. For each i the inner loop is executed once and executes i constant time operations itself. In total we get
3 + Σi=0,...,n-13 + 3i + 1 = 3 + 4n + 3/2⋅(n-1)⋅n = 3/2⋅n² + 5/2⋅n + 3
(1) (2) (3) (4) (5)
Explanations:
The 3 contains the first line, the last line and the an additional execution of the outer loop condition. The condition evaluates n times to true and one time to false.
This 3 contains one evaluation of the condition of the outer loop and the first two lines in the outer loop.
The factor 3 in front of the i contains one evaluation of the inner loop condition, the evaluation of the if statement and the last line of the inner loop.
The 1 is for the additional evaluation where the inner loop condition evaluates to false.
The sum of consecutive integers 1 to n evaluates to 1/2⋅n⋅(n+1). Notice the sum here is from 0 to n-1, so it evaluates to 1/2⋅(n-1)⋅n.
The frist (inner) return statement is not counted, because if executed the algorithm terminates. But we want to calculate the maximum number of steps, the so called worst case. This case is when the algorithm terminates as late as possible.
Notice: The calculation of the steps is very precise. This is not necessary to get the big-O complexity. It would be enough to say, that each loop runs in O(n) and since they are nested the complexity has to be multiplied so you get O(n)⋅O(n) = O(n²).
Related
It looks like some sort of a partial-sort.
int n = a.length;
for(int i = 0; i < n; i++) {
while(a[i] != i) {
if(a[i] < 0 || a[i] >= n) //avoid stepping out of range
break;
if(a[i] == a[a[i]]) //avoid inf loop by duplicates
break;
int t = a[i];
a[i] = a[t];
a[t] = t;
}
}
return a;
On first look, seems like O(N^2) but when I run it seems O(N). Any ideas? Thanks in advance.
You're right that it's O(n):
To help explain this I'll make up a definition:
Reflective: An element, a[i], in an array, a, is reflective if a[i] = i.
Iterations of while loop that do result in a break:
For each value of i, you can have exactly one break that's executed within the while loop (including the while condition). As there's n values of i, this means there's n total iterations of the while loop that result in a break.
Iterations of while loop that don't result in a break:
For this part it might help to imagine our array where each element is either reflective (1), or non-reflective (0):
| 0 | 0 | 1 | 0 | 1 | 1 | 0 | 1 | 1 |
Once we have passed the break points, then we know that a[i] != a[a[i]] (ie. if we name a[i] as t, then we know that a[t] != t). And because we later assign a[t] = t, then we have changed an element of the array from non-reflective to reflective. Note that nowhere in your code do we make a reflective element non-reflective: The assignment a[i] = a[t] could result in a[i] being non-reflective, but we also know that it wasn't reflective to begin with because the while statement was true: a[i] != i.
From our visual, this means that no 1 ever changes to a 0, and yet every iteration of the while loop (that passes the break points) results in at least one 0 flipping to a 1.
Once you observe that every (non-break) iteration of the inner loop takes at least one (possibly two) non-reflective elements from the array and converts it to become permanently reflective, then we realise that the total amount of (non-break) iterations of the inner loop cannot exceed n for the entire run-time of the program.
In summary: i is iterated and checked in the for loop n times, and each does a constant amount of work, c1. There's n total iterations of the iterations of the while loop that correspond to a break, and at most n iterations that don't correspond to a break. Hence there's at most 2n iterations of the while loop in total. The work done in a single iteration of the while loop is some max constant, c2.
Hence time complexity <= c1*n + c2*2*n = O(n).
As for the function of the code, it rearranges elements to make as many of them reflective as possible: if after this function a[i] is non-reflective, then the value i isn't present in the array.
I have read many topics which people usually say that Selection sort's complexity in best case is still O(n^2). But I coundn't be convinced by those ideas.
For instance, I want to sort the array in ascending order. And this is my algorithm in Java code:
void selectionSort(int[] arr) {
int min, temp;
int length = arr.length;
for (int i = 0; i <= length - 1; i++) {
//System.out.println(i);
min = i;
for (int j = i+1; j < length ; j++) {
if (arr[j] < arr[min]) {
min = j;
}
}
if (min != i) {
temp = arr[min];
arr[min] = arr[i];
arr[i] = temp;
} else {
break;
}
}
}
I believe this is O(n) in best case, which is the input arrray is already sorted.
One additional thing I added here to the algorithm is to check if (min == i) and break the loop.
What do you think? Am I wrong?
SelectionSort clearly has an O(N²) time complexity as the loops must be executed in full. The number of comparisons is the triangular number T(N-1) in all cases, while the number of swaps is linear (in the standard version).
Avoiding a swap for an element already in place is probably a bad idea because it is effective with very low probability and executed for nothing in most cases. (Not counting that the break... breaks the algorithm.)
Both loops in selection sort will fully run n times. Lets take example of [1,2,3,4,5]
Outer loop will run 5 times, for each value of array. Then inner loop will check that whether this is minimum value in rest of the array or not.
outer value = 1 -> Inner value 2,3,4,5
outer value = 2 -> Inner value 3,4,5
outer value = 3 -> Inner value 4,5
outer value = 4 -> Inner value 5
outer value = 5 -> Inner value none
Also, in your code, this check is incorrect
else {
break;
}
say array is [1,3,2]. In first outer loop, min ==i, and it will break and wont even move to next values.
I'm trying to find the order and growth function of this for loop inside a function which takes in an array of length n > 2.
This function orders the array in ascending order. I'm trying to find the order for a worst case scenario: when the array is ordered initially in descending order and the function therefore has to iterate through the array many times to sort it.
Here is the loop:
for (int next = 1; next < array.length; next++) {
int value = array[next];
int index = next;
while (index > 0 && value < array[index - 1]) {
array[index] = array[index - 1];
index--;
}
array[index] = value;
}
I've been racking my brains trying to figure it out. Writing tests, writing tons of functions down and I get close, but never right on. How would you go through such a loop to find it's order and growth function?
Any direction would greatly be appreciated. Thank you so much.
Let n be the array length. This loop is of worst-case running time O(n^2). The simple way to see this is as follows:
When next = 1, the number of operations done by the while loop is at most 1. When next = 2, the number of operations is at most 2, and so on until next = (n - 1). We can ignore the other operations done by the for loop because they constitute lower order terms which are irrelevant to growth.
So now, the number of operations is k * (1 + 2 + 3 + 4 + ... + (n - 1)) = k*(n*(n - 1)/2) = kn^2 - kn/2 where k is a constant factor.
Therefore, the growth of the function is of order n^2.
Edit:
To address the comment, we usually do not count the total number of statements because there is no standard to do so.
For example, would you count one iteration of the follow loop as one statement (a single print statement) or two statements (the print statement and incrementing i)?
for (int i = 0; i < n; i++)
{
print(i);
}
In addition, it is frankly not a very useful metric. In most cases, we only care about the highest order term of an algorithm.
However, to answer your question, I would count the loop as performing these many statements:
2n^2 + 2n - 3.
I am trying to find the run time on each line, when the best case and worst case would occur, and the Big-O in worst and best case.
What the code i pasted does is find the the longest length of the sequence of ascending numbers in array.
For example if we had [4,5,6,9,1,2,3,4,5,6] , the longest sequence would be 6.
will the first for loop will be executed n times?
will the second for loop be executed n times?
will the if statement be executed n times?
will the statement inside the if, be executed n times?
Will the best case occur when the array is in ascending order and the worst occur when the array is in descending order?
The reason I don't believe this to be true is that, when it is sorted in ascending order the second loop will be ran all the way through. When it is sorted in descending order, the second loop will always break because the and statement does not hold true.
for (i = 0, length = 1; i < n-1; i++) {
for (i1 = i2 = k = i; k < n-1 && a[k] < a[k+1]; k++, i2++);
if (length < i2 - i1 + 1)
length = i2 - i1 + 1;
}
return length;
First, note that the variables i1 and i2 are not really needed, as i1 == i and i2 == k at each iteration of the inner loop, so we can just write:
for (i = 0, length = 1; i < n-1; i++) {
for (k = i; k < n-1 && a[k] < a[k+1]; k++);
if (length < k - i + 1)
length = k - i + 1;
}
return length;
The outer loop will execute n-1 times. No difference in worst/best case there.
The best case occurs when a is a non-increasing sequence. In that case a[k] < a[k+1] will never be true and thus the inner loop's condition will only be executed once. The return value will in that case be 1, and the time complexity O(n).
The worst case occurs when a is an ever increasing sequence. In that case a[k] < a[k+1] will always be true, and thus the inner loop's condition will iterate n-1 - i times, the loop's condition once time more when k < n-1 is false.
The if condition and the adjustment of length execute in constant time.
The nested loop's body (the if) executes in total as many times as one can make a multiset of 2 elements (represented by i and k) from a set of n-1 elements. The formula for that is (n-1)n/2, which is O(n²).
What's the big O of this?
for (int i = 1; i < n; i++) {
for (int j = 1; j < (i*i); j++) {
if (j % i == 0) {
for (int k = 0; k < j; k++) {
// Simple computation
}
}
}
}
Can't really figure it out. Inclined to say O(n^4 log(n)) but feel like i'm wrong here.
This is quite a confusing analysis, so let's break it down bit by bit to make sense of the calculations:
The outermost loop runs for n-1 iterations (since 1 ≤ i < n).
The next loop inside it makes (i² - 1) iterations for each index i of the outer loop (since 1 ≤ j < i²).
In total, this means the number of iterations for these two loops is equal to calculating the sum of (i²-1) for each 1 ≤ i < n. This is similar to computing the sum of the first n squares, and is order of magnitude of O(n³).
Note the modulo operator % takes constant time (O(1)) to compute, therefore checking the condition if (j % i == 0) for all iterations of these two loops will not affect the O(n³) runtime.
Now let's talk about the inner loop inside the conditional.
We are interested in seeing how many times (and for which values of j) this if condition evaluates to true, since this would dictate how many iterations the innermost loop will run.
Practically speaking, (j % i) will never equal 0 if j < i, so the second loop could actually be shortened to start from i rather than from 1, however this will not impact the Big-O upper bound of the algorithm.
Notice that for a given number i, (j % i == 0) if and only if i is a divisor of j. Since our range is (1 ≤ j < i²), there will be a total of (i-1) values of j for which this will be true, for any given i. If this is confusing, consider this example:
Let's assume i = 4. Then our index j would iterate through all values 1,..,15=i²,
and (j%i == 0) would be true for j = 4, 8, 12 - exactly (i - 1) values.
The innermost loop would therefore make a total of (12 + 8 + 4 = 24) iterations. Thus for a general index i, we would look for the sum: i + 2i + 3i + ... + (i-1)i to indicate the number of iterations the innermost loop would make.
And this could be generalized by calculating the sum of this arithmetic progression. The first value is i and the last value is (i-1)i, which results in a sum of (i³ - i²)/2 iterations of the k loop for every value of i. In turn, the sum of this for all values of i could be computed by calculating the sum of cubes and the sum of squares - for a total runtime of O(n⁴) iterations of the innermost loop (the k loop) for all values of i.
Thus in total, the runtime of this algorithm would be the total of both runtimes we calculated above. We checked the if statement O(n³) times and the innermost loop ran for O(n⁴), so assuming // Simple computation runs in constant time, our total runtime would come down to:
O(n³) + O(n⁴)*O(1) = O(n⁴)
Let us assume that i = 2.Then j can be [1,2,3].The "k" loop will run for j = 2 only.
Similarly for i=3,j can be[1,2,3,4,5,6,7,8].hence, k can run for j = 3,6. You can see a pattern here that for any value of i, the 'k' loop will run (i-1) times.The length of loops will be [i,2*i,3*i,....i*i].
Hence the time complexity of k loop is
=i+(2*i)+(3*i)+ ..... +(i*i)
=(i^2)(i+1)/2
Hence the final complexity will be
= (n^3)(n+3)/2