I have the homework question:
Find a theta notation for the number of times the statement x = x + 1 is executed. (10 points).
i = n
while (i >= 1)
{
for j = 1 to n
{
x = x + 1
}
i = i/2
}
This is what I have done:
Ok first let’s make it easier. We will fist find the order of growth of:
while (i >= 1)
{
x = x + 1
i = i/2
}
that has order of growth O(log(n)) actually log base 2
the other inner for loop will execute n times therefore the algorithm should be of order:
O(log(n)*n)
The part where I get confused is that I am supposed to find theta notation NOT big-O. I know that theta notation is suppose to bound the function on the upper and lower limit. Will the correct answer be Theta(log(n)*n)?
I have found the answer in this link but I don't know how you get to that answer. Why they claim that the answer is Theta(n) ?
You should now prove it is also Omega(nlogn).
I won't show exactly how, since it is homework - but it is with the same principles you show O(nlogn). You need to show [unformally explnation:] that the asymptotic behavior of the function, is growing at least as fast as nlogn. [for big O you show it is growing at most at the rate of nlogn].
Remember that if a function is both O(nlogn) and Omega(nlogn), it is Theta(nlogn) [and vise versa]
p.s. Your hunch is true, it is easy to show it is not Omega(n), and thus it is not Theta(n)
p.s. 2: I think the author of the other answer confused with a different program:
i = n
while (i >= 1)
{
for j = 1 to i //NOTE: i instead of N here!
{
x = x + 1
}
i = i/2
}
The above program is indeed Theta(n), but it is different from the one you have provided.
Rephrasing your code fragment in a more formal way, so that it could be represented easily using Sigma Notation:
for (i = n; i >= 1; i = i/2 ) {
for j = 1; j <= n; j ++) {
x = x + 1; // instruction of cost 'c'
}
}
We obtain:
As #amit mentions I already have the upper limit of the function and that is Big-O which it actually is O(n*lgn). if I plot a table of that function I will get something like:
n n*lng
1 0
2 2
3 4.754887502
4 8
5 11.60964047
6 15.509775
7 19.65148445
8 24
9 28.52932501
10 33.21928095
because that is big-O then that means that the real function will be bounded by those values. In other words the real values should be less than the values at the table. for example taking a point for instace when n=9 we know that the answer should be less than or equal to 28.52932501 by looking at the table
So now we are missing to find Omega and that is the other bound. I think that the lower bound function should be Omega(n) and then we will get the table
n Omega(n)
1 1
2 2
3 3
4 4
5 5
6 6
7 7
8 8
9 9
.......
so that will be the other bound. If we take again point for example where n = 9 again then that will give us 9. that means that our real function should give us a value greater or equal to 9. based on our big-O function we also know that it should be les than or equal to 28.52932501
Related
Why is the runtime closer to O(1.6^N) and not O(2^N)? I think it has something to do with the call stack, but it's still a little unclear to me.
int Fibonacci(int n)
{
if (n <= 1)
return n;
else
return Fibonacci(n - 1) + Fibonacci(n - 2);
}
For starters, remember that big-O notation always provides an upper bound. That is, if a function is O(n), it's also O(n2), O(n3), O(2n), etc. So in that sense, you are not incorrect if you say that the runtime is O(2n). You just don't have a tight bound.
To see where the tight bound of Θ(φn) comes from, it might help to look at how many recursive calls end up getting made when evaluating Fibonacci(n). Notice that
Fibonacci(0) requires one call, the call to Fibonacci(0).
Fibonacci(1) requires one call, the call to Fibonacci(1).
Fibonacci(2) requires three calls: one to Fibonacci(2), and one each to Fibonacci(0) and Fibonacci(1).
Fibonacci(3) requires five calls: one to Fibonacci(3), then the three calls generated by invoking Fibonacci(2) and the one call generated by invoking Fibonacci(1).
Fibonacci(4) requires nine calls: one to Fibonacci(4), then five from the call to Fibonacci(3) and three from the call to Fibonacci(2).
More generally, the pattern seems to be that if you're calling Fibonacci(n) for some n ≥ 2, that the number of calls made is one (for the call itself), plus the number of calls needed to evaluate Fibonacci(n-1) and Fibonacci(n-2). If we let Ln denote the number of calls made, this means that we have that
L0 = L1 = 1
Ln+2 = 1 + Ln + Ln+1.
So now the question is how fast this sequence grows. Evaluating the first few terms gives us
1, 1, 3, 5, 9, 15, 25, 41, ...
which definitely gets bigger and bigger, but it's not clear how much bigger that is.
Something you might notice here is that Ln kinda sorta ish looks like the Fibonacci numbers. That is, it's defined in terms of the sum of the two previous terms, but it has an extra +1 term. So maybe we might want to look at the difference between Ln and Fn, since that might show us how much "faster" the L series grows. You might notice that the first two values of the L series are 1, 1 and the first two values of the Fibonacci series are 0, 1, so we'll shift things over by one term to make things line up a bit more nicely:
L(n) 1 1 3 5 9 15 25 41
F(n+1) 1 1 2 3 5 8 13 21
Diff: 0 0 1 2 4 7 12 20
And wait, hold on a second. What happens if we add one to each term of the difference? That gives us
L(n) 1 1 3 5 9 15 25 41
F(n+1) 1 1 2 3 5 8 13 21
Diff+1 1 1 2 3 5 8 13 21
Whoa! Looks like Ln - Fn+1 + 1 = Fn+1. Rearranging, we see that
Ln = 2Fn+1 - 1.
Wow! So the actual number of calls made by the Fibonacci recursive function is very closely related to the actual value returned. So we could say that the runtime of the Fibonacci function is Θ(Fn+1) and we'd be correct.
But now the question is where φ comes in. There's a lovely mathematical result called Binet's formula that says that Fn = Θ(φn). There are many ways to prove this, but they all essentially boil down to the observation that
the Fibonacci numbers seem to grow exponentially quickly;
if they grow exponentially with a base of x, then Fn+2 = Fn + Fn+1 can be rewritten as x2 = x + 1; and
φ is a solution to x2 = x + 1.
From this, we can see that since the runtime of Fibonacci is Θ(Fn+1), then the runtime is also Θ(φn+1) = Θ(φn).
The number φ = (1+sqrt(5))/2 is characterized by the two following properties:
φ >= 1
φ2 = φ + 1.
Multiplying the second equation by φ{n-1} we get
φn+1 = φn + φn-1
Since f(0) = 0, f(1) = 1 and f(n+1) = f(n) + f(n-1), using 1 and 3,
it is easy to see by induction in n that f(n) <= φn
Thus f(n) is O(φn).
A similar inductive argument shows that
f(n) >= φn-3 = φ-3φn (n >= 1)
thus f(n) = Θ(φn).
This question already has answers here:
Big O, how do you calculate/approximate it?
(24 answers)
Closed 5 years ago.
I'm studying algorithm's complexity and I'm still not able to determine the complexity of some algorithms ... Ok I'm able to figure out basic O(N) and O(N^2) loops but I'm having some difficult in routines like this one:
// What is time complexity of fun()?
int fun(int n)
{
int count = 0;
for (int i = n; i > 0; i /= 2)
for (int j = 0; j < i; j++)
count += 1;
return count;
}
Ok I know that some guys can calculate this with the eyes closed but I would love to to see a "step" by "step" how to if possible.
My first attempt to solve this would be to "simulate" an input and put the values in some sort of table, like below:
for n = 100
Step i
1 100
2 50
3 25
4 12
5 6
6 3
7 1
Ok at this point I'm assuming that this loop is O(logn), but unfortunately as I said no one solve this problem "step" by "step" so in the end I have no clue at all of what was done ....
In case of the inner loop I can build some sort of table like below:
for n = 100
Step i j
1 100 0..99
2 50 0..49
3 25 0..24
4 12 0..11
5 6 0..5
6 3 0..2
7 1 0..0
I can see that both loops are decreasing and I suppose a formula can be derived based on data above ...
Could someone clarify this problem? (The Answer is O(n))
Another simple way to probably look at it is:
Your outer loop initializes i (can be considered step/iterator) at n and divides i by 2 after every iteration. Hence, it executes the i/2 statement log2(n) times. So, a way to think about it is, your outer loop run log2(n) times. Whenever you divide a number by a base continuously till it reaches 0, you effectively do this division log number of times. Hence, outer loop is O(log-base-2 n)
Your inner loop iterates j (now the iterator or the step) from 0 to i every iteration of outer loop. i takes the maximum value of n, hence the longest run that your inner loop will have will be from 0 to n. Thus, it is O(n).
Now, your program runs like this:
Run 1: i = n, j = 0->n
Run 2: i = n/2, j = 0->n/2
Run 3: i = n/4, j = 0->n/4
.
.
.
Run x: i = n/(2^(x-1)), j = 0->[n/(2^(x-1))]
Now, runnning time always "multiplies" for nested loops, so
O(log-base-2 n)*O(n) gives O(n) for your entire code
Lets break this analysis up into a few steps.
First, start with the inner for loop. It is straightforward to see that this takes exactly i steps.
Next, think about which different values i will assume over the course of the algorithm. To start, consider the case where n is some power of 2. In this case, i starts at n, then n/2, then n/4, etc., until it reaches 1, and finally 0 and terminates. Because the inner loop takes i steps each time, then the total number of steps of fun(n) in this case is exactly n + n/2 + n/4 + ... + 1 = 2n - 1.
Lastly, convince yourself this generalizes to non-powers of 2. Given an input n, find smallest power of 2 greater than n and call it m. Clearly, n < m < 2n, so fun(n) takes less than 2m - 1 steps which is less than 4n - 1. Thus fun(n) is O(n).
This algorithm find all prime numbers below N
var f = function(n){
var primes = [2]; //1
var flag; //1
for(var i=3; i<n; i+=2){ // ( from 3 to n-1 ) / 2
flag = true; //1
var startI = 0; // 1
for(var y=primes[startI]; y<=Math.sqrt(i); y=primes[++startI]){ // ???
if(i%y === 0) // 1
flag = false; // 1
}
if(flag) // 1
primes.push(i); // 1
}
return primes; // 1
}
So far my analysis is done till the first loop, I'm not sure how to handle the second sum ( the one that is using primes.length and Math.sqrt ).
T(n) = 1 + 1 + sum( ( 1+ 1+ ??weird sum???) , from i = 3 to n -1) / 2 + 1 + 1
I understand how to analyze till the second nested loop, I suspect that is around log(N) or something like that, but I would like to know the exact number of iterations..
Questions:
How can I handle that kind of loop that is using arrays in memory to iterate ?
If not possible to get the exact number, how can I get a good approximation ?
Any help is appreciated (links to similar cases, books, etc ).
The inner loop iterates over the array of all primes below sqrt(i).
So you have to calculate the number of elements in that array. In the case of an array of primes, you have to use approximations for π(i), the number of primes below i.
You can approximate them by x/ln(x) or (much better) by li(x). More details here.
For analysis the x/ln(x) would be easier.
In total you get (assuming n = 2k+1)
T(n) = T(n-2) + O(sqrt(n)/( (1/2)⋅ln(n) )) = O( Σi = 1,...,k 2⋅sqrt(2⋅i+1)/ln(2⋅i+1) )
You get the recursive formula from the inner for loop, that iterates over the array of primes lower than sqrt(n) (approximated by sqrt(n)/( (1/2)⋅ln(n) )), and the work you have to do to come this far, represented by T(n-2).
Maybe you can simplify this more. I don't want to take the fun from you :)
Hint: Maybe you can use an integral to get an approximation of the sum. But I think there is no "nice" way to write it down.
If you forget about the 1/ln(i)-part, you can see
T(n) ∈ o(n3/2) and T(n) ∈ ω(n). Maybe you can do better.
As #vib mentioned, you can get a tighter upper bound O(n3/2/ln(n)). But notice that sqrt(n)/ln(n) is only an approximation for the number of primes lower than sqrt(n). Thus you get better bounds with better approximations. Since this approximations do not provide the exact value of π(n), we cannot say that this algorithm runs in Θ(n3/2/ln(n)).
I was studying recurrence by a slide found at (slide 7 and 8):
http://www.cs.ucf.edu/courses/cop3502h/spring2012/Lectures/Lec8_RecurrenceRelations.pdf
I just can't accept (probably I`m not seeing it right) that the recurrence equation of factorial is :
T(n) = T(n-1)+2
T(1) = 1
when considering the number of operations ("*" and "-") of the function :
int factorial(int n) {
if (n == 1)
return 1;
return n * factorial(n-1);
}
If we use n = 5 we will get 6 by the formula above while the real number of subs and molts are 8.
My teacher also told us that if analyzing only the number of "*" it would be :
T(n) = T(n-1)+1.
Again if I use n = 5, I would get 5 but if you do it on a paper you will get 4 multiplications.
I also checked on the forum, but this Question is more messed then a hell :
Recurrence Relation
Anyone could help me understand that ? thanks.
if we use n = 5 we will get 6 by the formula above while the real number of subs and molts are 8.
It seems that the slides are counting the number of operations, not just subtractions and multiplications. In particular, the return statement is counted as one operation. (The slides say, "if it’s the base case just one operation to return.")
Thus, the real number of subtractions and multiplications is 8, but the number of operations is 9. If n is 5, then, unrolling the recursion, we get 1 + 2 + 2 + 2 + 2 = 9 operations, which looks right to me.
I assume that a particular example in my book is wrong. But am I correct?
Example: 3log n + 2 is O(log n)
Justification: 3log n + 2 <= 5 log n, for n>=2.
I understand how they get the c=5 (since they take the coefficients and add them up). But I don't see how for n=2 for instance, the left function is smaller than the right one.
If I fill in 2 in n:
3 log 2 + 2 = 2.903 and 5 log 2 = 1.5051.
Only till n=10, the left function is actually smaller or equal than the right one.
Is my assumption right?
The log in this case is 2 based, not 10 based.
3log(2) + 2 = 3 + 2 = 5
5log(2) = 5
and it is true that 5 <= 5
To expand a bit on Peter's answer, the base of the logarithm is typically assumed to be base 2 when analyzing run times. It's not necessary to specify the base in the O() notation, since logarithms of different bases differ from each other only by a constant factor. (In this example, log_10(x) / log_2(x) = log(2)/log(10) = ~0.30103.) This constant factor is not relevant to the asymptotic run time.