Why does:
O(3n)=O(2n)=O(n)
Whereas their derivatives w.r.t n being 3, 2 and 1 respectively
Asymptotic Notation has no relation with derivatives. It is actually a measure of growth of function w.r.t. size of n. So, it tells us how a function will be changed on changing the value of n. If two functions get changed in same manner on changing n in the same way, we would call them the functions of same order. For example,Let f(n)=3n2+n+1 and g(n)=5n2+3n+1
If we double the n, both functions will roughly get 4 times of previous value. Hence they both are of order O(n2). We removed the constant coefficients(5 and 3) in Big-oh notation, because they are not contributing in telling how function is growing w.r.t. n (In every case, function will get 4 times). However we didn't remove the constant 2(power or exponent of n) because it is contributing in telling how function is growing w.r.t. n(Had we removed that constant and our function would get twice instead of 4 times hence we know it is contributing). Formally, we define Big-Oh notation as follows:
See here: Definition of Big O notation
f(n)=O(g(n)), if and only if f(n)<=c.g(n) for some c>0 and n>=n0
Now let me show you how O(3n)=O(2n)=O(n)
Informal Proof:
f(n)=2n and g(n)=3n both will grow linearly w.r.t. n. By linearly, I mean that if we double/halve n, the function output will also get doubled/halved. It doesn't matter if we change the coefficient to 2,3, or 1000, it will grow linearly w.r.t n. So, that's why O(n)=O(2n)=O(3n)Notice that it is not about removing constants, it's about whether these constants contribute in telling about how our function is growing w.r.t. n.
As a counter-example for this, Let's suppose
f(n)=2n andg(n)=22n
We can't remove the 2 in exponent because that 2 is contributing and saying that g(n) will actually change in terms of square w.r.t. how f(n) would change.So,f(n)=O(2n)while g(n)=O(4n)
Formal Proof:
Suppose n is sufficiently largeif f(n)=O(n), g(n)=O(2n) and h(n)=O(3n) then f(n)<=c1n for some c1>0g(n)<=2c2n for some c2>0, let's have c3=2c2 hence g(n)<=c3n or g(n)=O(n) or O(2n)=O(n)Similarly h(n)<=3c4n or h(n)=O(n) or O(3n)=O(n)Hence, O(3n)=O(2n)=O(n)
Final Words:The key point is just to check how a function is growing. After practicing, you'd have some idea like anp+bnp-1+...+c = O(np)And many more.Read CLRS Book. I don't remember exactly but I think Chapter-3 is dedicated for this concept.
Related
So I have have been reading the cracking the coding interview book and there is a problem where we’re we have a function that does O(n* n* n!) work. The book then says this can be expressed by O((n+2)!). It says Similarly O(n*n!)can be expressed by O((n+1)!). I looked in all rules if permutations and did not find any way to logically get there . My first step was cool I have O(n^2 +n!) now what? I don’t know what steps to take next .
You already know (I think) that n! = 1*2*3*...*n.
So n*n*n! = 1*2*3*...*n*n*n.
As n gets really big, adding 1 or 2 to a factor has a decreasingly significant effect. I'm no specialist but what matter with O() is either the power of n or, in our case, the number in the ()! expression.
Which gets us to shorten this to 1*2*3*...*n*(n+1)*(n+2)=(n+2)!.
Eventually, O(n*n*n!) can be expressed O((n+2)!).
To calculatee x! you do x*(x-1)! recursively until x-1==1 so x!==(x-1)*(x-2)*...*1 is O(n!). Therefore to do x*x! we have
(x-0)*(x-1)*...*1 which takes one extra call to our recursive function (but at the beginning, with large x value) i.e. (x+1)! iterations. Similarly, (x-0)*(x-0)*(x-1)*(x-2)*...*1==x²*x! requires (x+2)! function evaluations to compute, hence O((n+2)!) efficiency.
I thought it meant it takes a constant amount of time to run. Is that different than one step?
O(1) is a class of functions. Namely, it includes functions bounded by a constant.
We say that an algorithm has the complexity of O(1) iff the amount of steps it takes, as a function of the size of the input, is bounded by a(n arbirtary) constant. This function can be a constant, or it can grow, or behave chaotically, or undulate as a sine wave. As long as it never exceeds some predefined constant, it's O(1).
For more information, see Big O notation.
It means that even if you increase the size of whatever the algorithm is operating on, the number of calculations required to run remains the same.
More specifically it means that the number of calculations doesn't get larger than some constant no matter how big the input gets.
In contrast, O(N) means that if the size of the input is N, the number of steps required is at most a constant times N, no matter how big N gets.
So for example (in python code since that's probably easy for most to interpret):
def f(L, index): #L a list, index an integer
x = L[index]
y=2*L[index]
return x + y
then even though f has several calculations within it, the time taken to run is the same regardless of how long the list L is. However,
def g(L): #L a list
return sum(L)
This will be O(N) where N is the length of list L. Even though there is only a single calculation written, the system has to add all N entries together. So it has to do at least one step for each entry. So as N increases, the number of steps increases proportional to N.
As everyone has already tried to answer it, it simply means..
No matter how many mangoes you've got in a box, it'll always take you the same amount of time to eat 1 mango. How you plan on eating it is irrelevant, there maybe a single step or you might go through multiple steps and slice it nicely to consume it.
I'm trying to understand the algebra behind Big-O expressions. I have gone through several questions but still don't have a very clear idea how it's done.
When dealing with powers do we always omit the lower powers, for example:
O(10n^4-n^2-10) = O(10n^4)
What difference does it make when multiplication is involved? For example:
O(2n^3+10^2n) * O(n) = O(2n^3) ??
And finally, how do we deal with logs? For example:
O(n2) + O(5*log(n))
I think we try to get rid of all constants and lower powers. I'm not sure how logarithms are involved in the simplification and what difference a multiplication sign would do. Thank you.
Big-O expressions are more closely related to Calculus, specifically limits, than they are to algebraic concepts/rules. The easiest way I've found to think about expressions like the examples you've provided, is to start by plugging in a small number, and then a really large number, and observe how the result changes:
Expression: O(10n^4-n^2-10)
use n = 2: O(10(2^4) - 2^2 - 10)
O(10 * 16 - 4 - 10) = 146
use n = 100: O(10(100^4) - 100^2- 10)
O(10(100,000,000) - 10,000 - 10) = 999,989,990
What you can see from this, is that the n^4 term overpowers all other terms in the expression. Therefore, this algorithm would be denoted as having a run-time of O(n^4).
So yes, your assumptions are correct, that you should generally go with the highest power, drop constants, and drop order-1 terms.
Logarithms are effectively "undoing" exponentiation. Because of this, they will reduce the overall O-run-time of an algorithm. However, when they are added against exponential run-times, they generally get overruled by the larger order term. In the example you provided, if we again evaluate using real numbers:
Expression: O(n^2) + O(5*log(n))
use n=2: O(2^2) + O(5*log(2))
O(4) + O(3.4657) = 7.46
use n=100: O(100^2) + O(5*log(100))
O(10,000) + O(23.02) = 10,023
You will notice that although the logarithm term is increasing, it isn't a great gain compared to the increase in n's size. However, the n^2 term is still generating a massive increase compared to the increase in n's size. Because of this, the Big O of these expressions combined would still boil down to: O(n^2).
If you're interested in further reading about the mathematics side of this, you may want to check out this post: https://secweb.cs.odu.edu/~zeil/cs361/web/website/Lectures/algebra/page/algebra.html
Usually I am lazy and calculate modular inverses as follows:
def inv(a,m):
return 1 if a==1 else ((a-inv(m%a,a))*m+1)//a
print(inv(100**7000,99**7001))
But I was curious to know whether the method of passing more information back, namely the solution to Bezout's theorem (instead of just one of the pair), yields a faster or slower algorithm in practice:
def bez(a,b):
# returns [x,y,d] where a*x+b*y = d = gcd(a,b) since (b%a)*y+a*(x+(b//a)*y)=d
if a==0: return [0,1,b] if b>0 else [0,-1,-b]
r=bez(b%a,a)
return [r[1]-(b//a)*r[0],r[0],r[2]]
def inv(a,m):
r=bez(a,m)
if r[2]!=1: return None
return r[0] if r[0]>=0 else r[0]+abs(m)
print(inv(100**7000,99**7001))
I was amazed to find that the latter ran more than 50 times faster than the former! But both use nearly the same number of recursive calls and 1 integer division and 1 modulo operation per call, and the bit-length of the operands is roughly twice in the former on average (because the arguments involved are identical), so I only expect its time complexity to be roughly 4 times that of the latter, not 50 times.
So why am I observing this strange speedup?
Note that I am using Python 3, and I observed this when running online, with the following code added at the top to stop the Python interpreter complaining about exceeding maximum recursion depth or stack space:
import resource,sys
sys.setrecursionlimit(1000000)
resource.setrlimit(resource.RLIMIT_STACK,[0x10000000,resource.RLIM_INFINITY])
I figured it out finally. Suppose the initial inputs are n-bit integers. In typical cases, besides the first call to inv or inv2, the recursive calls have parameters whose sizes differ by just O(1) on average, and there are O(n) recursive calls on average, both due to some number-theoretic phenomenon. This O(1) size difference implies that the two methods actually have drastically different average time complexities. The implicit assumption in my question was that multiplication or division of two n-bit integers takes roughly O(n^2) time; this holds in the worst-case (for school-book multiplication), but not in the average case for the fast method!
For the fast method, it can be proven that when p,q > 0, the triple [x,y,d] returned by bez(p,q) satisfies abs(x) ≤ q and abs(y) ≤ p. Thus when each call bez(a,b) performs b%a and (b//a)*r[0], the modulo and division and multiplication each takes O(size(a)) time on average, since size(b)-size(a) ∈ O(1) on average and abs(r[0]) ≤ a. Total time is therefore roughly O(n^2).
For the slow method, however, when each call inv(a,m) performs ((a-inv(m%a,a))*m+1)//a, we have a-inv(m%a,a) roughly the same size as a, and so this division is of two integers where the first is roughly twice the size of the second, which would take O(size(a)^2) time! Total time is therefore roughly O(n^3).
Indeed, simulation yields very close agreement with the above analysis. For reference, using the fast method for inv(100**n,99**(n+1)) where n = 1400·k for k∈[1..6] took time/ms 17,57,107,182,281,366, while using the slow method where n = 500·k for k∈[1..6] took time/ms 22,141,426,981,1827,3101. These are very well fit by 9·k^2+9·k and 13·k^3+8·k^2 respectively, with mean fractional error ≈3.3% and ≈1.6% respectively. (We include the first two highest-order terms because they contribute a small but significant amount.)
Arrange the following functions in increasing order of growth rate
(with g(n) following f(n) in your list if and only if f(n)=O(g(n))).
a)2^log(n)
b)2^2log(n)
c)n^5/2
d)2^n^2
e)n^2 log(n)
So i think answer is in increasing order is
CEDAB
is it correct? i have confusion in option A and B.
i think option A should be at first place.. less one i mean so please help how to solve this.
This question I faced in algorithm course part 1 assignment (Coursera) .
Firstly, any positive power of n is always greater than log n, so E comes before C, not after.
Also, D comes after every other function, as either interpretation of 2^n^2 (could be 2^(n^2) or (2^n)^2 = 2^(2n); I could be wrong in ignoring BIDMAS though...) are exponentials of n itself.
Taking log to be base a, some arbitrary constant:
a)
b)
Thus, unfortunately, the actual order depends of the value of a, e.g. if the value of
is greater than 2, then A comes after E, otherwise before. Curiously the base of the log term in E is irrelevant (it still maintains its place).
The answer is aecbd
The easiest way to see why is to create a table with different values of n and compare amongst them. But some intuition:
a grows lesser than any others, specially c because of the log term in the power as opposed to the term itself
e is a with a n**2 term multiplied in, which is better than it being in an exponent
b is a double exponent, but still better than a quadratic power
d is the obvious worst because it grows exponentially with a quadratic power!