The problem:
Find the complexity (big theta) of this function:
s <-- 0;
i = 2;
while (i <= n^2) do
for j <-- i to 2i[ln(i)] do
s <-- s + i - j;
end
i <-- i + 2;
end
return (s);
I got big theta (n^2(ln(n)) since the for loop will run 2i[ln(i)] - i times which would be ci[ln(i)] time. And then I multiplied it by n^2 since that is the approximate time of the while loop.
I solved it a different way and got big theta (n^4).
I'm not sure if if my either of my answers are correct, and that is based on my confusion on how to solve these problems when loops are dependent vs independent.
The most robust solution is to sum 2*(2i)*ln(i) - 2i + 1 for 2 <= i <= n^2/2. We have:
1 + 2 + ... + floor(n^2/2) = O(n^4)
Whereas:
1*ln(1) + ... + floor(n^2/2)*ln(floor(n^2/2)) = O(n^4*ln(n))
(The constant 4 doesn't have to be taken into account.)
In fact, you can even get an expression asymptotically equivalent to your sum using a comparison with an integral.
Therefore the time complexity is a O(n^4*ln(n))-O(n^4)+O(n) = O(n^4*ln(n)).
Without MathJax writing maths is a pain, here is a more detailed proof.
Related
Not sure if this is the right place for this kind of question but here it goes. Given the following code, how many basic operations are there, and how many times is each one performed. What is the big O notation for this running time. This is in MATLAB, if it matters.
total = 0;
for i = 1:n
for j = 1:n
total = total + j;
end
end
My thinking is that, for each n, the j = 1:n loop runs once. Within the j = 1:n loop there are n calculations. So for the j = 1:n loop its n^2. This runs n times within the i = 1:n loop, so the total amount of calcultions is n^3, and the big O noation is O(N^3). Is this correct?
The short answer is:
O(n^2)
The long (and simplified) answer is:
The big "O" refers to the complexity of an algorithm (in this case, your code). Your question asks "how many" loops or operations are performed, but the "O" notation gives a relative idea of the complexity of an algorithm, thus not an absolute quantity. This would totally be impractical, the idea of the O notation is to generalise a measure of the complexity so that algorithms can be compared relatively to the other, without worrying too much about how many assignments, loops, and so on are performed.
That being said, there are specific guidelines on how to compute the complexity of an algorithm. Generally:
Loops are of complexity O("n"), not matter how many iterations they perform (remember, this is an abstract measure).
Operations such as assignments, additions etc are generally approximated to O(1) (complexity of 1) because the time they take to be performed is negligible.
There are specific rules for if then else operations, but it would make things more complicated and I invite you to read some introduction material on performing algorithm complexity analysis.
Also, be careful, the "n" is not that used in your code, it is a special notation used to denote a "generic" linear complexity.
Measuring the complexity of an algorithm is a recursive operation. You start with the basic operations and move up to loops etc. So, here is a detailed (I purposely detail too much so you get an idea of how it works, but in practice you don't have to go in that level of detail):
You start of with the first instruction:
O(total = 0;) = O(1)
because it is an assignment.
Then:
O(total = total + j;) = O(total + j) + O(total = x)
where x is the result of total + j.
= O(1) + O(1)
These are basic operations, thus they have a complexity of 1.
= O(1)
Because "O" is a "greatness" indicator that considers any sum of constants as 1.
Now coming to the loop:
O(
for i = 1:n // O(n)
for j = 1:n // O(n)
total = total + j; // O(1)
end
end
)
=
O(
n * (
n * (
1
)
)
= O(n * n * 1)
= O(n^2)
If you had two loops in a row (for ... ; for .... ;), the complexity would not be O(2n), but O(n), because again, O generalises.
Hope that helps :)
Your analysis is on the right track, but you're overestimating the cost by a factor of n. In particular, look here:
Within the j = 1:n loop there are n calculations. So for the j = 1:n loop its n^2.
You are right that the j = 1:n loop does n calculations, but each individual iteration of the loop only does 1 calculation. Since the loop runs n times, the work done is O(n), not O(n2). If you then repeat the rest of your analysis from that point, you'll end up getting that the total work done is Θ(n2), a tighter bound than what you had before.
As a note - you can actually speed this up pretty significantly. Notice that the inner loop adds 1 + 2 + 3 + ... + n to the total. We know that 1 + 2 + 3 + ... + n = n(n+1)/2, so you can rewrite the code as
total = 0;
for i = 1:n
total = total + n * (n + 1) / 2;
end
But notice that now you're just adding in n copies of n * (n + 1) / 2, so you can just rewrite this as
total = n * n * (n + 1) / 2
and the whole thing takes time O(1).
I'm learning Big-O notation right now and stumbled across this small algorithm in another thread:
i = n
while (i >= 1)
{
for j = 1 to i // NOTE: i instead of n here!
{
x = x + 1
}
i = i/2
}
According to the author of the post, the complexity is Θ(n), but I can't figure out how. I think the while loop's complexity is Θ(log(n)). The for loop's complexity from what I was thinking would also be Θ(log(n)) because the number of iterations would be halved each time.
So, wouldn't the complexity of the whole thing be Θ(log(n) * log(n)), or am I doing something wrong?
Edit: the segment is in the best answer of this question: https://stackoverflow.com/questions/9556782/find-theta-notation-of-the-following-while-loop#=
Imagine for simplicity that n = 2^k. How many times x gets incremented? It easily follows this is Geometric series
2^k + 2^(k - 1) + 2^(k - 2) + ... + 1 = 2^(k + 1) - 1 = 2 * n - 1
So this part is Θ(n). Also i get's halved k = log n times and it has no asymptotic effect to Θ(n).
The value of i for each iteration of the while loop, which is also how many iterations the for loop has, are n, n/2, n/4, ..., and the overall complexity is the sum of those. That puts it at roughly 2n, which gets you your Theta(n).
I've been ripping my hair out trying to solve this:
Σ(k=0,n)3k = O(3n)
I've been looking through various things online but I still can't seem to solve it. I know it involves the formal definition of Big O, where
|f(x)| <= C*|g(x)|, x>=k
Since they are the same, I am assuming C is some value I have to find through induction to prove the original statement, and that k=0.
Thanks for your help with this.
Σ(k=0,n)3k
= 30 + 31 + ... + 3n
= (1 - 3n+1) / (1 - 3) ; sum of geometric series
= (3/2)*3n - k
<= c*3n ; for c >= 3/2
= O(3n)
Induction is not needed here; that sum is a geometric series and has closed form solution
= 1(1-3^(n + 1))/(1-3) = (3^(n + 1) - 1)/2
= (3*3^n - 1)/2
Pick C = 3/2 and F = 3/2*3^n - 1/2, G = 3^n, and this satisfies the requirement for O(3^n), but really in practice, though it might be thought informal and sloppy, you don't really worry much about an exact constant since any constant will do for satisfying Big-O.
You can rewrite it as 3n * ( 1 + 1/3 + 1/9 + ....1/3n).
There is an upper bound for that sum. Calculate the limit of that infinite series.
From there, it's easy to get a good C, eg: 2.
Refreshing up on algorithm complexity, I was looking at this example:
int x = 0;
for ( int j = 1; j <= n; j++ )
for ( int k = 1; k < 3*j; k++ )
x = x + j;
I know this loops ends up being O(n^2). I'm believing inner loop is executed 3*n times( 3(1+2+...n) ), and the outer loop executes n times. So, O(3n*n) = O(3n^2) = O(n^2).
However, the source I'm looking at expands the execution of the inner loop to: 3(1+2+3+...+n) = 3n^2/2 + 3n/2. Can anyone explain the 3n^2/2 + 3n/2 execution times?
for each J you have to execute J * 3 iterations of internal loop, so you command x=x+j will be finally executed n * 3 * (1 + 2 + 3 ... + n) times, sum of Arithmetic progression is n*(n+1)/2, so you command will be executed:
3 * n * (n+1)/2 which is equals to (3*n^2)/2 + (3*n)/2
but big O is not how much iterations will be, it is about assymptotic measure, so in expression 3*n*(n+1)/2 needs to remove consts (set them all to 0 or 1), so we have 1*n*(n+0)/1 = n^2
Small update about big O calculation for this case: to make big O from the 3n(n+1)/2, for big O you can imagine than N is infinity, so:
infinity + 1 = infinity
3*infinity = infinity
infinity/2 = infinity
infinity*infinity = infinity^2
so you after this you have N^2
The sum of integers from 1 to m is m*(m+1)/2. In the given problem, j goes from 1 to n, and k goes from 1 to 3*j. So the inner loop on k is executed 3*(1+2+3+4+5+...+n) times, with each term in that series representing one value of j. That gives 3n(n+1)/2. If you expand that, you get 3n^2/2+3n/2. The whole thing is still O(n^2), though. You don't care if your execution time is going up both quadratically and linearly, since the linear gets swamped by the quadratic.
Big O notation gives an upper bound on the asymptotic running time of an algorithm. It does not take into account the lower order terms or the constant factors. Therefore O(10n2) and O(1000n2 + 4n + 56) is still O(n2).
What you are doing is try to count the number the number of operations in your algorithm. However Big O does not say anything about the exact number of operations. It simply provides you an upper bound on the worst case running time that may occur with an unfavorable input.
The exact precision of your algorithm can be found using Sigma notation like this:
It's been empirically verified.
This is an old homework problem from my algorithms class. I have the solution to the problem, but even after repeated attempts, I fail to understand how to think in the right direction to arrive at the solution.
function h(N) {
if (N==1) return 3;
else {
sum = 1;
i = 0;
while (i < h(N-1))
sum = sum + i;
i = i + 1;
return sum;
}
}
According to me, since h(N-1) is called repeatedly in the while loop, the while loop should run as many number of times as what is returned by h(N-1). Besides that, the function call h(N-1) in the while loop will also happen that many number of times. Thus, according to me, I should get something like this:
T(N) = T(N-1)*H(N-1) + C*H(N-1) + D
where
1. T(N) is the running time,
2. T(N-1)*H(N-1) because the one recursive call to h(N-1) will take T(N-1) and since it's recomputed every time the comparison is made, it will be called H(N-1) times. (where H(N-1) is the value returned from the call)
3. and C*H(N-1) is the running time for the statements inside the while loop (since the while loop runs H(N-1) times.
I did not get a satisfactory answer from my professor and I would appreciate if someone could help me understand this.
Thank you!
Try understanding this in two steps, first consider this simpler function, where we replace the while loop with an if.
function g(N) {
if (N==1) return 3;
else {
sum = 1;
i = 0;
if(i < g(N-1))
sum = sum + i;
i = i + 1;
return sum;
}
}
Here, we get the recurrence:
G(N) = G(N-1) + O(1)
So far, so good? Here, the work to compute g(N) involves solving the smaller problem g(N-1) plus a constant amount of work.
Now, let's go back to the original function h(N). What has changed? Now, the work to compute h(N) involves solving the subproblem h(N - 1), h(N-1) times. And in each of those times (i.e. in the while loop), we do a constant amount of work. There is also another constant amount of work that is done only once in h(N), i.e. out of the while loop. So, we essentially get:
H(N) = H(N - 1) *{H(N - 1) + O(1)} + O(1)
We can rewrite the above by making the substitution T(n) = H(n) + O(1). Thus, we get:
T(N) = H(N - 1) * T(N - 1) + O(1)
Assume that in executing h(N), the value of h(N-1) is recomputed at each iteration of the loop (which is probably the case for most languages and most compilers)