How to calculate complexity when there are more than one recursive calls?
Such as in this problem.
F(n)
{
if (n is 1)
return;
F(n/2) //Call 1
F(n/3) //Call 2
F(n/6) //Call 3
}
You just need to solve the equation,
T(n)=T(n/2)+T(n/3)+T(n/6)+O(1)
Now as T(n/2)>T(n/3), we can instead solve for
T(n)=3T(n/2)+O(1)
Using master's theorem, T(n)=O(n^(log(base 2)3))=O(n^1.58)
Note that there might be better solution but as this is Big O notation, this is valid too
Interesting question.
I believe I can prove that the complexity of this function is O(nc) for any c > 1.
Recall the definition of big-O notation. We say a function g(n) is O(f(n)) if there exist constants k and n' such that g(n) < k*f(n) for all n > n'. (Colloquially, g(n) is bounded above by f(n) for sufficiently large n, ignoring constant factors.)
Pick any c > 1, and observe that for sufficiently large n,
1 > (1/2)c + (1/3)c + (1/6)c + 1/nc
This is easy to see because 1/2 + 1/3 + 1/6 = 1, and (1/2)c < 1/2 etc. because c > 1. And when n is big enough, 1/nc is arbitrarily small.
Multiply through by nc, to get that for sufficiently large n:
nc > (n/2)c + (n/3)c + (n/6)c + 1
Therefore, if the running time of F(m) is bounded above by mc for m=n/2, m=n/3, and m=n/6, then the running time of F(n) is bounded above by nc. Result follows by induction.
So although I was wrong that this function is O(n), it is arbitrarily close... In the sense that for any positive value epsilon, no matter how small, the function is O(n1+epsilon).
...
In general, for this type of question, I think you want to guess solutions of the form nc and then try to place a bound on c. This is essentially how the Master Theorem itself works.
Related
In definition of Big-O notation we care only about C coefficient:
f(n) ≤ Cg(n) for all n ≥ k
Why don't we care about A as well:
f(n) ≤ Cg(n) + A for all n ≥ k
There are really two cases to consider here. For starters, imagine that your function g(n) has the property that g(n) ≥ 1 for all "sufficiently large" choices of n. In that case, if you know that
f(n) ≥ cg(n) + A,
then you also know that
f(n) ≥ cg(n) + Ag(n),
so
f(n) ≥ (c + A)g(n).
In other words, if your function g is always at least one, then bounding f(n) by something of the form cg(n) + A is equivalent to bounding it with something of the form c'g(n) for some new constant c'. In that sense, adding some extra flexibility into the definition of big-O notation, at least in this case, wouldn't make a difference.
In the context of the analysis of algorithms, pretty much every function g(n) you might bound something with will be at least one, and so we can "munch up" that extra additive term by choosing a larger multiple of g.
However, big-O notation is also used in many cases to bound functions that decrease as n increases. For example, we might say that the probability that some algorithm gives back the right answer is O(1 / n), where the function 1/n drops to 0 as a function of n. In this case, we use big-O notation to talk about how fast the function drops off. If the success probability is O(1 / n2), for example, that's a better guarantee than the earlier O(1 / n) success probability, assuming n gets sufficiently large. In that case, allowing for additive terms in the definition of big-O notation would actually break things. For example, intuitively, the function 1 / n2 drops to 0 faster than the function 1 / n, and using the formal definition of big-O notation you can see this because 1 / n2 ≤ 1 / n for all n ≥ 1. However, with your modified definition of big-O notation, we could also say that 1 / n = O(1 / n2), since
1 / n ≤ 1 / n2 + 1 for all n ≥ 1,
which is true only because the additive 1 term bounds the 1/n term, not the 1/n2 we might have been initially interested in.
So the long answer to your question is "the definition you proposed above is equivalent to the regular definition of big-O if we only restrict ourselves to the case where g(n) doesn't drop to zero as a function of n, and in the case where g(n) does drop the zero as a function of n your new definition isn't particularly useful."
Big-O notation is about what happens as the data gets larger. In other words, it is a limit as n --> infinity.
As n gets large, A remains the same. So it gets smaller and smaller in comparison. On the other hand, g(n) (presumably) gets bigger and bigger, so its contribution increases more and more.
A is constant in this case, thus it will not affect much the complexity when the size of the problem grows very much.
When you have a cost of 1 million, you do not care if you add up a constant factor of 100 for example. You care for how this 1 million grows (resulting from Cg(n)); whether it becomes 2 millions for example if the size of the problem grows a bit. However, your constant will still be 100, so it doesn't really affect the overall complexity.
This is my recursive function:
function abc(n):
if n == 0
return xyz(n)
for i = 1 to n
print(xyz(n))
return abc(n/2) + abc(n/2)
and xyz() is ϴ(n^3). Will the Master theorem will be valid here? If, yes How will I write it?
The master theorem concerns recurrence relations of this form:
T(n) = a * T(n/b) + f(n)
T being the recursive procedure, a the number of subproblems into which we divide the input n, n/b the size of each subproblem and `f(n) the cost for the division of the input into subproblems and the combination of the results.
If n == 0 then n/b becomes 0, and so does a. This leaves us with:
T(0) = 0 + f(0)
Since there's no more recursion, it basically comes down to f(0). In your hypothetical case this has a complexity ϴ(n^3).
Since f(n) is the cost for the division of n into a subproblems and the combination of results, f(0) would normally have a cost of 0 or a constant. If function f(n) has a complexity of ϴ(n^3), then actually for n == 0 this still leads to a cost of 0 with regards to input size.
The master theorem provides information on the asymptotic bound for T(n), depending on the complexity of f(n), a and b. This depends on how the complexity of f(n) can be expressed using a form that employs logb(a)(log with base b of a). The log of 0 is undefined with b > 0.
What it comes down to is that it makes no sense to ask whether the master theorem holds for some specific input. Moreover, the master theorem holds anyway, it just states that depending on f(n) you can make some claims about the complexity of T or not. This depends on a and b, so without that information it is senseless to ask. If your f(n) has O(n^3) outside of the base case as well (n > 0) then you could make claims about T depending on how that 3 relates to a and b. For example, if 3 < logb(a) you'd be sure that T is ϴ(n^(logb(a)).
Suppose that the a in your algorithm is 2^n, then the master theorem could no longer be used to say anything about T's complexity.
EDIT
After your question edit, the form of your recursive procedure has become this:
T(n) = 2 * T(n/2) + f(n)
So a == 2 and b == 2 are the parameters in your case, since you divide the input into two subproblems which each get an input that's half of the input doing the recursion. The combination of the two recursive calls is constant (a simple addition abc(n/2) + abc(n/2)) and the division of the problems is trivial too, but this part in your case could simulate a ϴ(n^4) algorithm for dividing the input into subproblems:
for i = 1 to n
print(xyz(n))
Note that it's ϴ(n^4) because you stated xyz(n) is ϴ(n^3) and you repeat it n times in the loop. So your f(n) = ϴ(n^4).
The master theorem can't really state anything about this. However, if f(n) = Ω(n^4) (note the omega here), then 4 > log2(2) (the logb(a) with b = 2 and a = 2 in your case). In order to make a statement about T's complexity, another condition must now hold, the regularity condition. It states that a * f(n/b) <= k * f(n) must be true for some k < 1 and sufficiently large n.
So that gives us 2 * f(n/2) <= k * f(n). This is true for k < 1/8. This, finally, lets us state that T = ϴ(f(n)), so T = ϴ(n^4).
Meaning that final part is true if your f(n) (the loop with the xyz call) can be proven to be Ω(n^4) (again, note the omega instead of theta). Since omega is the lower bound, and your f(n) = ϴ(n^4), that should be true.
I was doing study on Asymptotic Notations Topic, i recon that its formula is so simple yet it tells nothing and there are couple of things i don't understand.
When we say
f(n) <= c.g(n) where n >= n₀
And we don't know the value of c =? and n=? at first but by doing division of f(n) or g(n) we get the value of c. (here is where confusion lies)
First Question: How do we decide which side's 'n' has to get divided in equation f(n) or g(n)?
Suppose we have to prove:
2n(square) = O(n(cube))
here f(n) = 2(n(square)) and g(n)=n(cube)
which will form as:
2(n(square)) = c . n(cube)
Now in the notes i have read they are dividing 2(n(square)) to get the value of c by doing that we get c = 1;
But if we do it dividing n(cube) [which i don't know whether we can do it or not] we get c = 2;
How do we know what value we have to divide ?
Second Problem: Where does n₀ come from what's its task ?
Well by formula we know n >= n(0) which means what ever we take n we should take the value of n(0) or should be greater what n is.
But i am confuse that where do we use n₀ ? Why it is needed ?
By just finding C and N can't we get to conclusion if
n(square) = O(n(cube)) or not.
Would any one like to address this? Many thanks in advance.
Please don't snub me if i ask anything stupid or give -1. Address it please any useful link which covers all this would be enough as well:3
I have gone through the following links before posting this question this is what i understand and here are those links:
http://openclassroom.stanford.edu/MainFolder/VideoPage.php?course=IntroToAlgorithms&video=CS161L2P8&speed=
http://faculty.cse.tamu.edu/djimenez/ut/utsa/cs3343/lecture3.html
https://sites.google.com/sites/algorithmss15
From the second url in your question:
Let's define big-Oh more formally:
O(g(n)) = { the set of all f such that there exist positive constants c and n0 satisfying 0 <= f(n) <= cg(n) for all n >= n0 }.
This means, that for f(n) = 4*n*n + 135*n*log(n) + 1e8*n the big-O is O(n*n).
Because for large enough c and n0 this is true:
4*n*n + 135*n*log(n) + 1e8*n = f(n) <= O(n*n) = c*n*n
In this particular case the [c,n0] can be for example [6, 1e8], because (this is of course not valid mathematical proof, but I hope it's "obvious" from it):
f(1e8) = 4*1e16 + 135*8*1e8 + 1e16 = 5*1e16 + 1080*1e8 <= 6*1e16 = 6*1e8*1e8 =~= O(n*n). There are of course many more possible [c,n0] for which the f(n) <= c*n*n holds true, but you need to find only one such pair to prove the f(n) has O(f(n)) of O(n*n).
As you can see, for n=1 you need quite a huge c (like 1e9), so at first look the f(n) may look much bigger than n*n, but in the asymptotic notion you don't care about the first few initial values, as long as the behaviour since some boundary is as desired. That boundary is some [c,n0]. If you can find such boundary ([6, 1e8]), then QED: "f(n) has big-O of n*n".
The n >= n₀ means that whatever you say in the lemma can be false for some first k (countable) parameters n' : n' < n₀, but since some n₀ the lemma is true for all the rest of (bigger) integers.
It says that you don't care about first few integers ("first few" can be as "little" as 1e400, or 1e400000, ...etc... from the theory point of view), and you only care about the bigger (big enough, bigger than n₀) n values.
Ultimately it means, that in the big-O notation you usually write the simplest and lowest function having the same asymptotic notion as the examined f(n).
For example for any f(n) of polynomial type like f(n) = ∑aini, i=0..k the O(f(n)) = O(nk).
So I did throw away all the lower 0..(k-1) powers of n, as they stand no chance against nk in the long run (for large n). And the ak does lose to some bigger c.
In case you are lost in that i,k,...:
f(n) = 34n4 + 23920392n2 has O(n4).
As for large enough n that n4 will "eclipse" any value created from n2. And 34n4 is only 34 times bigger than n4 => 34 is constant (relates to c) and can be omitted from big-O notation too.
In big O notation, we always say that we should ignore constant factors for most cases. That is, rather than writing,
3n^2-100n+6
we are almost always satisfied with
n^2
since that term is the fastest growing term in the equation.
But I found many algorithm courses starts comparing functions with many terms
2n^2+120n+5 = big O of n^2
then finding c and n0 for those long functions, before recommending to ignore low order terms in the end.
My question is what would I get from trying to understand and annalising these kinds of functions with many terms? Before this month I am comfortable with understanding what O(1), O(n), O(LOG(n)), O(N^3) mean. But am I missing some important concepts if I just rely on this typically used functions? What will I miss if I skipped analysing those long functions?
Let's first of all describe what we mean when we say that f(n) is in O(g(n)):
... we can say that f(n) is O(g(n)) if we can find a constant c such
that f(n) is less than c·g(n) or all n larger than n0, i.e., for all
n>n0.
In equation for: we need to find one set of constants (c, n0) that fulfils
f(n) < c · g(n), for all n > n0, (+)
Now, the result that f(n) is in O(g(n)) is sometimes presented in difference forms, e.g. as f(n) = O(g(n)) or f(n) ∈ O(g(n)), but the statement is the same. Hence, from your question, the statement 2n^2+120n+5 = big O of n^2 is just:
f(n) = 2n^2 + 120n + 5
a result after some analysis: f(n) is in O(g(n)), where
g(n) = n^2
Ok, with this out of the way, we look at the constant term in the functions we want to analyse asymptotically, and let's look at it educationally, using however, your example.
As the result of any big-O analysis is the asymptotic behaviour of a function, in all but some very unusual cases, the constant term has no effect whatsoever on this behaviour. The constant factor can, however, affect how to choose the constant pair (c, n0) used to show that f(n) is in O(g(n)) for some functions f(n) and g(n), i.e., the none-unique constant pair (c, n0) used to show that (+) holds. We can say that the constant term will have no effect of our result of the analysis, but it can affect our derivation of this result.
Lets look at your function as well as another related function
f(n) = 2n^2 + 120n + 5 (x)
h(n) = 2n^2 + 120n + 22500 (xx)
Using a similar approach as in this thread, for f(n), we can show:
linear term:
120n < n^2 for n > 120 (verify: 120n = n^2 at n = 120) (i)
constant term:
5 < n^2 for e.g. n > 3 (verify: 3^2 = 9 > 5) (ii)
This means that if we replace both 120n as well as 5 in (x) by n^2 we can state the following inequality result:
Given that n > 120, we have:
2n^2 + n^2 + n^2 = 4n^2 > {by (ii)} > 2n^2 + 120n + 5 = f(n) (iii)
From (iii), we can choose (c, n0) = (4, 120), and (iii) then shows that these constants fulfil (+) for f(n) with g(n) = n^2, and hence
result: f(n) is in O(n^2)
Now, for for h(n), we analogously have:
linear term (same as for f(n))
120n < n^2 for n > 120 (verify: 120n = n^2 at n = 120) (I)
constant term:
22500 < n^2 for e.g. n > 150 (verify: 150^2 = 22500) (II)
In this case, we replace 120n as well as 22500 in (xx) by n^2, but we need a larger less than constraint on n for these to hold, namely n > 150. Hence, we the following holds:
Given that n > 150, we have:
2n^2 + n^2 + n^2 = 4n^2 > {by (ii)} > 2n^2 + 120n + 5 = h(n) (III)
In same way as for f(n), we can, here, choose (c, n0) = (4, 150), and (III) then shows that these constants fulfil (+) for h(n), with g(n) = n^2, and hence
result: h(n) is in O(n^2)
Hence, we have the same result for both functions f(n) and h(n), but we had to use different constants (c,n0) to show these (i.e., somewhat different derivation). Note finally that:
Naturally the constants (c,n0) = (4,150) (used for h(n) analysis) are also valid to show that f(n) is in O(n^2), i.e., that (+) holds for f(n) with g(n)=n^2.
However, not the reverse: (c,n0) = (4,120) cannot be used to show that (+) holds for h(n) (with g(n)=n^2).
The core of this discussion is that:
As long as you look at sufficiently large values of n, you will be able to describe the constant terms in relations as constant < dominantTerm(n), where, in our example, we look at the relation with regard to dominant term n^2.
The asymptotic behaviour of a function will not (in all but some very unusual cases) depend on the constant terms, so we might as well skip looking at them at all. However, for a rigorous proof of the asymptotic behaviour of some function, we need to take into account also the constant terms.
Ever have intermediate steps in your work? That is what this likely is as when you are computing a big O, chances are you don't already know for sure what the highest order term is and thus you keep track of them all and then determine which complexity class makes sense in the end. There is also something to be said for understanding why the lower order terms can be ignored.
Take some graph algorithms like a minimum spanning tree or shortest path. Now, can just looking at an algorithm you know what the highest term will be? I know I wouldn't and so I'd trace through the algorithm and collect a bunch of terms.
If you want another example, consider Sorting Algorithms and whether you want to memorize all the complexities or not. Bubble Sort, Shell Sort, Merge Sort, Quick Sort, Radix Sort and Heap Sort are a few of the more common algorithms out there. You could either memorize both the algorithm and complexity or just the algorithm and derive the complexity from the pseudo code if you know how to trace them.
I'm taking Data Structures and Algorithm course and I'm stuck at this recursive equation:
T(n) = logn*T(logn) + n
obviously this can't be handled with the use of the Master Theorem, so I was wondering if anybody has any ideas for solving this recursive equation. I'm pretty sure that it should be solved with a change in the parameters, like considering n to be 2^m , but I couldn't manage to find any good fix.
The answer is Theta(n). To prove something is Theta(n), you have to show it is Omega(n) and O(n). Omega(n) in this case is obvious because T(n)>=n. To show that T(n)=O(n), first
Pick a large finite value N such that log(n)^2 < n/100 for all n>N. This is possible because log(n)^2=o(n).
Pick a constant C>100 such that T(n)<Cn for all n<=N. This is possible due to the fact that N is finite.
We will show inductively that T(n)<Cn for all n>N. Since log(n)<n, by the induction hypothesis, we have:
T(n) < n + log(n) C log(n)
= n + C log(n)^2
< n + (C/100) n
= C * (1/100 + 1/C) * n
< C/50 * n
< C*n
In fact, for this function it is even possible to show that T(n) = n + o(n) using a similar argument.
This is by no means an official proof but I think it goes like this.
The key is the + n part. Because of this, T is bounded below by o(n). (or should that be big omega? I'm rusty.) So let's assume that T(n) = O(n) and have a go at that.
Substitute into the original relation
T(n) = (log n)O(log n) + n
= O(log^2(n)) + O(n)
= O(n)
So it still holds.