I have am trying to learn Big O and am confused on an algorithm I just came across. The algorithm is:
void pairs(int[] array){
for (int i=0; i < array.length; i++){
for (int j=i+1; j<array.length; j++){
System.out.println(array[i]+","+array[j]);
}
}
}
I think the first for loop is O(n) and I know the second for loop is O(1/2*n(n+1)). The answer to the problem was that the run time for the function is O(n^2). I simplified O(1/2*n(n+1)) to O(1/2*(n^2+n)). So I'm confused because I thought that you needed to multiply the two run time terms since the for loop is nested, which would give O(n) * O(1/2*(n^2+n)). I simplified this to O(1/2n^3 + 1/2n^2). From what I understand of Big O, you only keep the largest term so this would reduce to O(n^3). Can anyone help me out with where I went wrong? Not sure how the answer is O(n^2) instead of O(n^3).
When you say the inner loop is O(1/2*n(n+1)), you are actually describing the big-O complexity of both loops.
To say that the outer loop has complexity O(N) basically means its body runs N times. But for your calculation of the inner loop's complexity, you already took account of all iterations of the outer loop, because you added up the number of times the inner loop runs over all iterations of the outer loop. If you multiply again by N, you would be saying that the outer loop itself is re-run another N times.
Put another way, what your analysis shows is that the inner loop body (the System.out.println call) runs 1/2*n(n+1) times overall. That means the overall complexity of the two-loop combination is O(1/2*n(n+1)) = O(n^2). The overall complexity of the two-loop combination describes how many times the innermost code is run.
your mistake is counting the second loop as O(1/2n^2)...
first, you can clearly see it is capped to N-1 (when j = 0)
first loop is clearly N
Second loop is MAX of N-1...
threrefore, O(N^2)...
if we examine it little more,
second loop will run N-1 when i=0,
then N-2 for i=1,
and ONE single time for i=n-1,
this is 1/2n(n-1) = 1/2n^2 - 1/2n = O(n^2)
Notice this includes all iteration of the outer loop too!
Related
I'm trying to understand the time complexity of insertion sort. I got stuck at while loop. I'm unable to understand how many times while loop executes
InsertionSort(A)
for j = 2 to A.length
key = A[j]
i = j - 1
while i>0 and A[i]>key
A[i+1] = A[i]
i = i - 1
A[i+1] = key
I know that for loop executes n+1 times and every statement in the loop execute n times
while loop also executes n times
But, what I don't understand is "How many times statements under while loop executes for both worst and best cases?"
In the worst case, A is sorted in descending order, which means that for the j'th entry, the inner loop will run j times (give or take a "+1" or "-1"...). Happily, there is a formula for that: as Gauss famously found out spontaneously and under duress, summing up all numbers from 1 to n yields a result of n*(n+1)/2.
As we only care about complexity and not actual values, we can leave the constant and multiplicative factors off and end up with O(n^2).
Tongue-in-cheek aside, the fact that there is a loop within a loop is a strong indication for O(n^2) when the inner loop count is bounded linearly - which it is here.
Best case, with A already sorted in ascending order, the inner loop will be entered not at all, and overall complexity will be O(n).
The average case depends heavily on what your expected "unorderedness" looks like. For example, the sort will behave greatly if your list is basically always sorted already, and there are only very few, very local switchups.
I recently learned about formal Big-O analysis of algorithms; however, I don't see why these 2 algorithms, which do virtually the same thing, would have drastically different running times. The algorithms both print numbers 0 up to n. I will write them in pseudocode:
Algorithm 1:
def countUp(int n){
for(int i = 0; i <= n; i++){
print(n);
}
}
Algorithm 2:
def countUp2(int n){
for(int i = 0; i < 10; i++){
for(int j = 0; j < 10; j++){
... (continued so that this can print out all values 0 - Integer.MAX_VALUE)
for(int z = 0; z < 10; z++){
print("" + i + j + ... + k);
if(("" + i + j + k).stringToInt() == n){
quit();
}
}
}
}
}
So, the first algorithm runs in O(n) time, whereas the second algorithm (depending on the programming language) runs in something close to O(n^10). Is there anything with the code that causes this to happen, or is it simply the absurdity of my example that "breaks" the math?
In countUp, the loop hits all numbers in the range [0,n] once, thus resulting in a runtime of O(n).
In countUp2, you do somewhat the exact same thing, a bunch of times. The bounds on all your loops is 10.
Say you have 3 loop running with a bound of 10. So, outer loop does 10, inner does 10x10, innermost does 10x10x10. So, worst case your innermost loop will run 1000 times, which is essentially constant time. So, for n loops with bounds [0, 10), your runtime is 10^n which, again, can be called constant time, O(1), since it is not dependent on n for worst case analysis.
Assuming you can write enough loops and that the size of n is not a factor, then you would need a loop for every single digit of n. Number of digits in n is int(math.floor(math.log10(n))) + 1; lets call this dig. So, a more strict upper bound on the number of iterations would be 10^dig (which can be kinda reduced to O(n); proof is left to the reader as an exercise).
When analyzing the runtime of an algorithm, one key thing to look for is the loops. In algorithm 1, you have code that executes n times, making the runtime O(n). In algorithm 2, you have nested loops that each run 10 times, so you have a runtime of O(10^3). This is because your code runs the innermost loop 10 times for each run of the middle loop, which in turn runs 10 times for each run of the outermost loop. So the code runs 10x10x10 times. (This is purely an upper bound however, because your if-statement may end the algorithm before the looping is complete, depending on the value of n).
To count up to n in countUp2, then you need the same number of loops as the number of digits in n: so log(n) loops. Each loop can run 10 times, so the total number of iterations is 10^log(n) which is O(n).
The first runs in O(n log n) time, since print(n) outputs O(log n) digits.
The second program assumes an upper limit for n, so is trivially O(1). When we do complexity analysis, we assume a more abstract version of the programming language where (usually) integers are unbounded but arithmetic operations still perform in O(1). In your example you're mixing up the actual programming language (which has bounded integers) with this more abstract model (which doesn't). If you rewrite the program[*] so that is has a dynamically adjustable number of loops depending on n (so if your number n has k digits, then there's k+1 nested loops), then it does one iteration of the innermost code for each number from 0 up to the next power of 10 after n. The inner loop does O(log n) work[**] as it constructs the string, so overall this program too is O(n log n).
[*] you can't use for loops and variables to do this; you'd have to use recursion or something similar, and an array instead of the variables i, j, k, ..., z.
[**] that's assuming your programming language optimizes the addition of k length-1 strings so that it runs in O(k) time. The obvious string concatenation implementation would be O(k^2) time, meaning your second program would run in O(n(log n)^2) time.
Here's the code I've implemented in a nutshell. The for loop should have a complexity of O(n). I just can't figure out the time complexity of the inner while loop.
int x,n; // Inputted by the user.
for (int i=0; i<n; i++)
{
int done=0;
if (Condition)
{
while (done < x)
{
done++; // Based on a lot of operations
}
}
}
I can post the whole code if you want. Thanks in advance :)
Here, the complexity is measured by studying the number of times the program will run the operations of the inner loop.
Each time Condition is triggered, the inner loop runs x times. Thus the inner loop complexity is O(x).
This loop can run at most n times. This provides you an overall worst-case complexity of O(x.n).
Having additional knowledge about Condition can get you a more precise analysis. You may be able to compute average complexity for example.
As an example : let Condition be !(i & (i-1)). This is true if and only if i is either 0 or a power of 2. In this case, your loop would get run exactly E(ln2(n)) + 2 times (E(.) being the integer part function). In the end, the overall complexity knowing this becomes O(x.ln(n))
I am trying to determine why the bigO of this algorithm is m^2*n, and why the innermost loop is executing in m^2*n steps.
int m=10, n=15;
int inLoop = 0, midLoop = 0, outLoop = 0;
for(int i=1;i<=m;i++)
{
outLoop++;
for(int j=1;j<=2*i-1;j++)
{
midLoop++;
for(int k=1;k<=n;k++)
{
inLoop++;
}
}
}
System.out.println("Out Loop " + outLoop);
System.out.println("Mid Loop " + midLoop);
System.out.println("Inner Loop " + inLoop);
When I run this, I get that the inner loop runs 1500 times, the middle loop 100 times, and the outermost loop 10 times.
Before running this code I thought that this code ran the first loop m times, the second loop m^2 times, and the last loop n times, which with these values would result in the inner loop output to be 15,000.
Apparently the algorithm seems to be executing the innermost loop in m^2 * n steps as opposed to the m^3*n steps I believed it would be.
summation(2i - 1) as i starts at 1 and ends at m is:
2*summation(i) - summation(1) = 2 * (m+1)/2 * m - m = O(m^2)
This is only for the outer and middle loop
The inner loop is straight forward resulting in O(n * m^2)
I think that the idea is clear, that you count how often each loop is repeated, and that it's rather the details that are unclear. Now, in order to ease these things, you can separate these loops and conquer them separately.
For the outermost loop, it is pretty clear that it runs m times. That is, its complexity is Θ(m x), with x being the complexity of what's inside. For the innermost loop, the case is also pretty simple, it only depends on the value of n, which is constant, so its complexity is Θ(n).
The middle loop is the one that is a bit more complicated. Its complexity depends on i, but i is not constant but the loop variable of the outer loop. However, you can use the average as replacement. In this case the average is pretty simple, which you can visualize if you draw a checkboard. In the first iteration of the outer loop, i=1, so j will only take a single value 1. In the second iteration, i=2 and j=1..3. In the third, it's j=1..5 and so on. If you draw these underneath each other, you get a triangle-like shape. It has the width 1 at the top, the width 2m-1 at the bottom and a height of m. It's area is therefore ((2m-1)+1)/2=m.
Putting this together, you have complexities Θ(m) for the outer and middle loop and Θ(n) for the inner loop, making it Θ(m²n) overall.
sum = 0;
for(int i = 0; i < N; i++)
for(int j = i; j >= 0; j--)
sum++;
From what I understand, the first line is 1 operation, 2nd line is (i+1) operations, 3rd line is (i-1) operations, and 4th line is n operations. Does this mean that the running time would be 1 + (i+1)(i-1) + n? It's just these last steps that confuse me.
To analyze the algorithm you don't want to go line by line asking "how much time does this particular line contribute?" The reason is that each line doesn't execute the same number of times. For example, the innermost line is executed a whole bunch of times, compared to the first line which is run just once.
To analyze an algorithm like this, try identifying some quantity whose value is within a constant factor of the total runtime of the algorithm. In this case, that quantity would probably be "how many times does the line sum++ execute?", since if we know this value, we know the total amount of time that's spent by the two loops in the algorithm. To figure this out, let's trace out what happens with these loops. On the first iteration of the outer loop, i == 0 and so the inner loop will execute exactly once (counting down from 0 to 0). On the second iteration of the outer loop, i == 1 and the inner loop executes exactly twice (first with j == 1, once with j == 0. More generally, on the kth iteration of the outer loop, the inner loop executes k + 1 times. This means that the total number of iterations of the innermost loop is given by
1 + 2 + 3 + ... + N
This quantity can be shown to be equal to
N (N + 1) N^2 + N N^2 N
--------- = ------- = --- + ---
2 2 2 2
Of these two terms, the N^2 / 2 term is the dominant growth term, and so if we ignore its constant factors we get a runtime of O(N2).
Don't look at this answer as something you should memorize - think of all of the steps required to get to the answer. We started by finding some quantity to count, and then saw how that quantity was influenced by the execution of the loops. From this, we were able to derive a mathematical expression for that quantity, which we then simplified. Finally, we took the resulting expression and determined the dominant term, which serves as the big-O for the overall function.
Work from inside-out.
sum++
This is a single operation on it's own, as it doesn't repeat.
for(int j = i; j >= 0; j--)
This loops i+1 times. There are several operations in there, but you probably don't mean to count the number of asm instructions. So I'll assume for this question this is a multiplier of i+1. Since the loop contents is a single operation, the loop and its block perform i+1 operations.
for(int i = 0; i < N; i++)
This loops N times. So as before, this is a multiplier of N. Since the block performs i+1 operations, this loop performs N(N+1)/2 operations in total. And that's your answer! If you want to consider big-O complexity, then this simplifies to O(N2).
It's not additive: the inner loop happens once for EACH iteration of the outer loop. So it's O(n2).
By the way, this is a good example of why we use asymptotic notation for this kind of thing -- depending on the definition of "operation" the exact details of the count could vary pretty widely. (Like, is sum++ a single operation, or is it add sum to 1 giving temp; load temp to sum?) But since we know that all that can be hidden in a constant factor, it's still going to be O(n2).
No; you don't count a specific number of operations for each line and then add them up. The entire point of constructions like 'for' is to make it possible for a given line of code to run more than once. You're supposed to use thinking and logic skills to figure out how many times the line 'sum++' will run, as a function of N. Hint: it runs once for every time that the third line is encountered.
How many times is the second line encountered?
Each time the second line is encountered, the value of 'i' is set. How many times does the third line run with that value of i? Therefore, how many times will it run overall? (Hint: if I give you a different amount of money on several different occasions, how do you find out the total amount I gave you?)
Each time the third line is encountered, the fourth line happens once.
Which line happens most often? How often does it happen, in terms of N?
So guess what interest you is the sum++ and how many time you execute it.
The final stat of sum would give you that answer.
Actually your loop is just:
Sigma(n) n goes from 1 to N.
Which equal to: N*(N+1) / 2 This give you in big-o-notation O(N^2)
Also beside the name of you question there is no worst case in you algorithm.
Or you could say that the worst case is when N goes to infinity.
Using Sigma notation to represent your loops: