What would the Big O notation be for two for loops that aren't nested?
Example:
for(int i=0; i<n; i++){
System.out.println(i);
}
for(int j=0; j<n; j++){
System.out.println(j);
}
Linear
O(n) + O(n) = 2*O(n) = O(n)
It does not matter how many non nested loops do you have (if this number is a constant and does not depends on n) the complexity would be linear and would equal to the maximum number of iterations in the loop.
Technically this algorithm still operates in O(n) time.
While the number of iterations increases by 2 for each increase in n, the time taken still increases at a linear rate, thus, in O(n) time.
It would be O(2n) because you run n+n = 2n iterations.
O(2n) is essentially equivalent to O(n) as 2 is a constant.
It will be O(n) + O(n) ==> Effectively O(n) since we don't keep constant values.
Assuming a scenario each loop runs up to n
So we can say the complexity of each for loop is O(n) as each loop will run n times.
So you specified these loops are not nested in a linear scenario for first loop O(n)+ second loop O(n)+ third loop O(n) which will give you 3O(n).
Now as we are mostly concentrating on the variable part of the complexity we will exclude the constant part and will say it's O(n) for this scenario.
But in a practical scenario, I will suggest you keep in mind that the constant factor will also play a vital role so never exclude them.
For example, consider time complexity to find the smallest integer from an integer array anyone will say it's O(n) but to find second largest or smallest of the same array will be O(2n).
But most of the answers will be saying it's O(n) where actually ignoring the constant.
Consider if the array is of 10 million size then that constant can't be ignored.
Related
I have the following algorithm:
for(int i = 1; i < n; i++)
for(int j = 0; j < i; j++)
if(j % i == 0) System.out.println(i + " " + j);
This will run max(n,0) times.
Would the Big-O notation be O(n)? If not, what is it and why?
Thank you.
You haven't stated what you are trying to measure with the Big-O notation. Let's assume it's time complexity. Next we have to define what the dependent variable is against which you want to measure the complexity. A reasonable choice here is the absolute value of n (as opposed to the bit-length), since you are dealing with fixed-length ints and not arbitrary-length integers.
You are right that the println is executed O(n) times, but that's counting how often a certain line is hit, it's not measuring time complexity.
It's easy to see that the if statement is hit O(n^2) times, so we have already established that the time complexity is bounded from below by Omega(n^2). As a commenter has already noted, the if-condition is only true for j=0, so I suspect that you actually meant to write i % j instead of j % i? This matters because the time complexity of the println(i + " " + j)-statement is certainly not O(1), but O(log n) (you can't possibly print x characters in less than x steps), so at first sight there is a possibility that the overall complexity is strictly worse than O(n^2).
Assuming that you meant to write i % j we could make the simplifying assumption that the condition is always true, in which case we would obtain the upper bound O(n^2 log n), which is strictly worse than O(n^2)!
However, noting that the number of divisors of n is bounded by O(Sqrt(n)), we actually have O(n^2 + n*Sqrt(n)*log(n)). But since O(Sqrt(n) * log(n)) < O(n), this amounts to O(n^2).
You can dig deeper into number theory to find tighter bounds on the number of divisors, but that doesn't make a difference since the n^2 stays the dominating factor.
So the tightest upper bound is indeed O(n^2), but it's not as obvious as it seems at first sight.
max(n,0) would indeed be O(n). However, your algorithm is in O(n**2). Your first loop goes n times, and the second loop goes i times which is on average n/2. That makes O(n**2 / 2) = O(n**2). However, unlike the runtime of the algorithm, the amount of times println is reached is in O(n), as this happens exactly n times.
So, the answer depends on what exactly you want to measure.
I'm having a bit of trouble determining the big o of this solution I came up with on a CTCI question
The space complexity should be O(1)
but is the run time O(N)? It seems to be more towards O(N^2) because of the while loop. But the while loop does not run N times for each element inside the for loop.
public static int[] missing_two(int [] n){
for(int i=0;i<n.length;i++){
while(n[i]!=i){
int temp=n[i];
n[i]=n[n[i]];
n[temp]=temp;
}
}
}
Would 6,0,1,2,3,4,5 be an example of worst case here?
The while loop will run n times on the first index. and 0 afterwards. Therefore shouldn't this be O(2n) => O(n)?
My understanding is that big O notation is used for giving an upper bound limit or a worst case scenario. The question you have to make yourself is: Could input vector n have values such that at least in one case, for all values in input vector n, the while loop is fully executed? Then a correct Big O notation would be O(N^2), if not but close, then I guess O(N^2) would still be a better estimation than O(N), as O(N) for an upper bound would have already been exceeded.
Now that you have edited the question and given more info about it I agree with you. It is O(N)
for (int j=0,k=0; j<n; j++)
for (double m=1; m<n; m*=2)
k++;
I think it's O(n^2) but I'm not certain. I'm working on a practice problem and I have the following choices:
O(n^2)
O(2^n)
O(n!)
O(n log(n))
Hmmm... well, break it down.
It seems obvious that the outer loop is O(n). It is increasing by 1 each iteration.
The inner loop however, increases by a power of 2. Exponentials are certainly related (in fact inversely) to logarithms.
Why have you come to the O(n^2) solution? Prove it.
Its O(nlog2n). The code block runs n*log2n times.
Suppose n=16; Then the first loop runs 16 (=n) times. And the second loops runs 4(=log2n) times (m=1,2,4,8). So the inner statement k++ runs 64 times = (n*log2n) times.
lets look at the worst-case behaviour. for second loop search continues from 1, 2, 4, 8.... lets say n is 2^k for some k >= 0. in the worst-case we might end up searching until 2^k and realise we overshot the target. Now we know that target can be in 2^(k - 1) and 2^k. The number of elements in that range are 2^(k - 1) (think a second.). The number of elements that we have examined so far is O(k) which is O(logn) and for first loop it's O(n).(too simple to find out). then order of whole code will O(n(logn)).
A generic way to approach these sorts of problems is to consider the order of each loop, and because they are nested, you can multiply the "O" notations.
Some simple rules for big "O":
O(n)O(m) = O(nm)
O(n) + O(m) = O(n + m)
O(kn) = O(n) where 'k' is some constant
The 'j' loop iterates across n elements, so clearly it is O(n).
The 'm' loop iterates across log(n) elements, so it is O(log(n)).
Since the loops are nested, our final result would O(n) * O(log(n)) = O(n*log(n)).
I was reading about Big O notation. It stated,
The big O of a loop is the number of iterations of the loop into
number of statements within the loop.
Here is a code snippet,
for (int i=0 ;i<n; i++)
{
cout <<"Hello World"<<endl;
cout <<"Hello SO";
}
Now according to the definition, the Big O should be O(n*2) but it is O(n). Can anyone help me out by explaining why is that?
Thanks in adavance.
If you check the definition of the O() notation you will see that (multiplier) constants doesn't matter.
The work to be done within the loop is not 2. There are two statements, for each of them you have to do a couple of machine instructions, maybe it's 50, or 78, or whatever, but this is completely irrelevant for the asymptotic complexity calculations because they are all constants. It doesn't depend on n. It's just O(1).
O(1) = O(2) = O(c) where c is a constant.
O(n) = O(3n) = O(cn)
O(n) is used to messure the loop agains a mathematical funciton (like n^2, n^m,..).
So if you have a loop like this
for(int i = 0; i < n; i++) {
// sumfin
}
The best describing math function the loops takes is calculated with O(n) (where n is a number between 0..infinity)
If you have a loop like this
for(int i =0 ; i< n*2; i++) {
}
Means it will took O(n*2); math function = n*2
for(int i = 0; i < n; i++) {
for(int j = 0; j < n; j++) {
}
}
This loops takes O(n^2) time; math funciton = n^n
This way you can calculate how long your loop need for n 10 or 100 or 1000
This way you can build graphs for loops and such.
Big-O notation ignores constant multipliers by design (and by definition), so being O(n) and being O(2n) is exactly the same thing. We usually write O(n) because that is shorter and more familiar, but O(2n) means the same.
First, don't call it "the Big O". That is wrong and misleading. What you are really trying to find is asymptotically how many instructions will be executed as a function of n. The right way to think about O(n) is not as a function, but rather as a set of functions. More specifically:
O(n) is the set of all functions f(x) such that there exists some constant M and some number x_0 where for all x > x_0, f(x) < M x.
In other words, as n gets very large, at some point the growth of the function (for example, number of instructions) will be bounded above by a linear function with some constant coefficient.
Depending on how you count instructions that loop can execute a different number of instructions, but no matter what it will only iterate at most n times. Therefore the number of instructions is in O(n). It doesn't matter if it repeats 6n or .5n or 100000000n times, or even if it only executes a constant number of instructions! It is still in the class of functions in O(n).
To expand a bit more, the class O(n*2) = O(0.1*n) = O(n), and the class O(n) is strictly contained in the class O(n^2). As a result, that loop is also in O(2*n) (because O(2*n) = O(n)), and contained in O(n^2) (but that upper bound is not tight).
O(n) means the loops time complexity increases linearly with the number of elements.
2*n is still linear, so you say the loop is of order O(n).
However, the loop you posted is O(n) since the instructions in the loop take constant time. Two times a constant is still a constant.
The fastest growing term in your program is the loop and the rest is just the constant so we choose the fastest growing term which is the loop O(n)
In case if your program has a nested loop in it this O(n) will be ignored and your algorithm will be given O(n^2) because your nested loop has the fastest growing term.
Usually big O notation expresses the number of principal operations in a function.
In this tou're overating over n elements. So complexity is O(n).
Surely is not O(n^2), since quadratic is the complexity of those algorithms, like bubble sort which compare every element in the input with all other elements.
As you remember, bubble sort, in order to determine the right position in which to insert an element, compare every element with the others n in a list (bubbling behaviour).
At most, you can claim that you're algorithm has complexity O(2n),since it prints 2 phrases for every element in the input, but in big O notation O(n) is quiv to O(2n).
I recently had an interview and was given a small problem that I was to code up.
The problem was basically find duplicates in an array of length n, using constant space in O(n). Each element is in the range 1-(n-1) and guaranteed to be a duplicate. This is what I came up with:
public int findDuplicate(int[] vals) {
int indexSum=0;
int valSum=0;
for (int i=0; i< vals.length; i++) {
indexSum += i;
valSum += vals[i];
}
return valSum - indexSum;
}
Then we got into a discussion about the runtime of this algorithm. A sum of series from 0 -> n = (n^2 + n)/2 which is quadratic. However, isn't the algorithm O(n) time? The number of operations are bound by the length of the array right?
What am I missing? Is this algorithm O(n^2)?
The fact that the sum of the integers from 0 to n is O(n^2) is irrelevant here.
Yes you run through the loop exactly O(n) times.
The big question is, what order of complexity are you assuming on addition?
If O(1) then yeah, this is linear. Most people will assume that addition is O(1).
But iwhat if addition is actually O(b) (b is bits, and in our case b = log n)? If you are going to assume this, then this algorithm is actually O(n * log n) (adding n numbers, each needs log n bits to represent).
Again, most people assume that addition is O(1).
Algorithms researchers have standardized on the unit-cost RAM model, where words are Theta(log n) bits and operations on words are Theta(1) time. An alternative model where operations on words are Theta(log n) time is not used any more because it's ridiculous to have a RAM that can't recognize palindromes in linear time.
Your algorithm clearly runs in time O(n) and extra space O(1), since convention is for the default unit of space to be the word. Your interviewer may have been worried about overflow, but your algorithm works fine if addition and subtraction are performed modulo any number M ≥ n, as would be the case for two's complement.
tl;dr Whatever your interviewer's problem was is imaginary or rooted in an improper understanding of the conventions of theoretical computer science.
You work on each on n cells one time each. Linear time.
Yes the algorithm is linear*. The result of valSum doesn't affect the running time. Take it to extreme, the function
int f(int[] vals) {
return vals.length * vals.length;
}
gives n2 in 1 multiplication. Obviously this doesn't mean f is O(n2) ;)
(*: assuming addition is O(1))
The sum of i from i=0 to n is n*(n+1)/2 which is bounded by n^2 but that has nothing to do with running time... that's just the closed form of the summation.
The running time of your algorithm is linear, O(n), where n is the number of elements in your array (assuming the addition operation is a constant time operation, O(1)).
I hope this helps.
Hristo