How do I prove that this algorithm is O(loglogn)
i <-- 2
while i < n
i <-- i*i
Well, I believe we should first start with n / 2^k < 1, but that will yield O(logn). Any ideas?
I want to look at this in a simple way, what happends after one iteration, after two iterations, and after k iterations, I think this way I'll be able to understand better how to compute this correctly. What do you think about this approach? I'm new to this, so excuse me.
Let us use the name A for the presented algorithm. Let us further assume that the input variable is n.
Then, strictly speaking, A is not in the runtime complexity class O(log log n). A must be in (Omega)(n), i.e. in terms of runtime complexity, it is at least linear. Why? There is i*i, a multiplication that depends on i that depends on n. A naive multiplication approach might require quadratic runtime complexity. More sophisticated approaches will reduce the exponent, but not below linear in terms of n.
For the sake of completeness, the comparison < is also a linear operation.
For the purpose of the question, we could assume that multiplication and comparison is done in constant time. Then, we can formulate the question: How often do we have to apply the constant time operations > and * until A terminates for a given n?
Simply speaking, the multiplication reduces the effort logarithmic and the iterative application leads to a further logarithmic reduce. How can we show this? Thankfully to the simple structure of A, we can transform A to an equation that we can solve directly.
A changes i to the power of 2 and does this repeatedly. Therefore, A calculates 2^(2^k). When is 2^(2^k) = n? To solve this for k, we apply the logarithm (base 2) two times, i.e., with ignoring the bases, we get k = log log n. The < can be ignored due to the O notation.
To answer the very last part of the original question, we can also look at examples for each iteration. We can note the state of i at the end of the while loop body for each iteration of the while loop:
1: i = 4 = 2^2 = 2^(2^1)
2: i = 16 = 4*4 = (2^2)*(2^2) = 2^(2^2)
3: i = 256 = 16*16 = 4*4 = (2^2)*(2^2)*(2^2)*(2^2) = 2^(2^3)
4: i = 65536 = 256*256 = 16*16*16*16 = ... = 2^(2^4)
...
k: i = ... = 2^(2^k)
Related
I know that the time complexity of a recursive function dividing its input by /2 is log n base 2,I have come across some interesting scenarios on
https://stackoverflow.com/a/42038565/8169857
Kindly help me to understand the logic behind the scenarios in the answer regarding the derivation of the formula
It's back to the recursion tree. Why for 1/2 is O(log2(n))? Because if n = 2^k, you should divide k times to reach to 1. Hence, the number of computation is k = log2(n) comparison at most. Now suppose it is (c-1)/c. Hence, if n = (c/(c-1))^k, we need log_{c/(c-1)}(n) operations to reach to 1.
Now as for any constant c > 1, limit log2(n)/log_{c/(c-1)}(n), n \to \infty is equal to a constant greater than zero, log_{c/(c-1)}(n) = \Theta(log2(n)). Indeed, you can say this for any constants a, b > 1, log_a(n) = \Theta(log_b(n)). Now, the proof is completed.
I would like some clarification regarding O(N) functions. I am using SICP.
Consider the factorial function in the book that generates a recursive process in pseudocode:
function factorial1(n) {
if (n == 1) {
return 1;
}
return n*factorial1(n-1);
}
I have no idea how to measure the number of steps. That is, I don't know how "step" is defined, so I used the statement from the book to define a step:
Thus, we can compute n ! by computing (n-1)! and multiplying the
result by n.
I thought that is what they mean by a step. For a concrete example, if we trace (factorial 5),
factorial(1) = 1 = 1 step (base case - constant time)
factorial(2) = 2*factorial(1) = 2 steps
factorial(3) = 3*factorial(2) = 3 steps
factorial(4) = 4*factorial(3) = 4 steps
factorial(5) = 5*factorial(4) = 5 steps
I think this is indeed linear (number of steps is proportional to n).
On the other hand, here is another factorial function I keep seeing which has slightly different base case.
function factorial2(n) {
if (n == 0) {
return 1;
}
return n*factorial2(n-1);
}
This is exactly the same as the first one, except another computation (step) is added:
factorial(0) = 1 = 1 step (base case - constant time)
factorial(1) = 1*factorial(0) = 2 steps
...
Now I believe this is still O(N), but am I correct if I say factorial2 is more like O(n+1) (where 1 is the base case) as opposed to factorial1 which is exactly O(N) (including the base case)?
One thing to note is that factorial1 is incorrect for n = 0, likely underflowing and ultimately causing a stack overflow in typical implementations. factorial2 is correct for n = 0.
Setting that aside, your intution is correct. factorial1 is O(n) and factorial2 is O(n + 1). However, since the effect of n dominates over constant factors (the + 1), it's typical to simplify it by saying it's O(n). The wikipedia article on Big O Notation describes this:
...the function g(x) appearing within the O(...) is typically chosen to be as simple as possible, omitting constant factors and lower order terms.
From another perspective though, it's more accurate to say that these functions execute in pseudo-polynomial time. This means that it is polynomial with respect to the numeric value of n, but exponential with respect to the number of bits required to represent the value of n. There is an excellent prior answer that describes the distinction.
What is pseudopolynomial time? How does it differ from polynomial time?
Your pseudocode is still pretty vague as to the exact details of its execution. A more explicit one could be
function factorial1(n) {
r1 = (n == 1); // one step
if r1: { return 1; } // second step ... will stop only if n==1
r2 = factorial1(n-1) // third step ... in addition to however much steps
// it takes to compute the factorial1(n-1)
r3 = n * r2; // fourth step
return r3;
}
Thus we see that computing factorial1(n) takes four more steps than computing factorial1(n-1), and computing factorial1(1) takes two steps:
T(1) = 2
T(n) = 4 + T(n-1)
This translates roughly to 4n operations overall, which is in O(n). One step more, or less, or any constant number of steps (i.e. independent of n), do not change anything.
I would argue that no you would not be correct in saying that.
If something is O(N) then it is by definition O(N+1) as well as O(2n+3) as well as O(6N + -e) or O(.67777N - e^67). We use the simplest form out of convenience for notation O(N) however we have to be aware that it would be true to say that the first function is also O(N+1) and likewise the second is as much O(n) as it wasO(n+1)`.
Ill prove it. If you spend some time with the definition of big-O it isn't too hard to prove that.
g(n)=O(f(n)), f(n) = O(k(n)) --implies-> g(n) = O(k(n))
(Dont believe me? Just google transitive property of big O notation). It is then easy to see the below implication follows from the above.
n = O(n+1), factorial1 = O(n) --implies--> factorial1 = O(n+1)
So there is absolutely no difference between saying a function is O(N) or O(N+1). You just said the same thing twice. It is an isometry, a congruency, a equivalency. Pick your fancy word for it. They are different names for the same thing.
If you look at the Θ function you can think of them as a bunch of mathematical sets full of functions where all function in that set have the same growth rate. Some common sets are:
Θ(1) # Constant
Θ(log(n)) # Logarithmic
Θ(n) # Linear
Θ(n^2) # Qudratic
Θ(n^3) # Cubic
Θ(2^n) # Exponential (Base 2)
Θ(n!) # Factorial
A function will fall into one and exactly one Θ set. If a function fell into 2 sets then by definitions all functions in both sets could be proven to fall into both sets and you really just have one set. At the end of the day Θ gives us a perfect segmentation of all possible functions into set of countably infinite unique sets.
A function being in a big-O set means that it exists in some Θ set which has a growth rate no larger than the big-O function.
And thats why I would say you were wrong, or at least misguided to say it is "more O(N+1)". O(N) is really just a way of notating "The set of all functions that have growth rate equal to or less than a linear growth". And so to say that:
a function is more O(N+1) and less `O(N)`
would be equivalent to saying
a function is more "a member of the set of all functions that have linear
growth rate or less growth rate" and less "a member of the set of all
functions that have linear or less growth rate"
Which is pretty absurd, and not a correct thing to say.
when we do the sum of n numbers using for loop for(i=1;i<=n;i++)complexity of this is O(n), but if we do this same computation using the formula of arithmetic/geometric progression series n(n-1)/2 that time if we compute the time complexity, its O(n^2). How ? please solve my doubt.
You are confused by what the numbers are representing.
Basically we are counting the # of steps when we talking about complexity.
n(n+1)/2 is the answer of Summation(1..n), that's correct, but different way take different # of steps to compute it, and we are counting the # of such steps.
Compare the following:
int ans = 0;
for(int i=1; i<=n;i++) ans += i;
// this use n steps only
int ans2 = 0;
ans2 = n*(n+1)/2;
// this use 1 step!!
int ans3 = 0;
for(int i=1, mx = n*(n+1)/2; i<=mx; i++) ans3++;
// this takes n*(n+1)/2 step
// You were thinking the formula would look like this when translated into code!
All three answers give the same value!
So, you can see only the first method & the third method (which is of course not practical at all) is affected by n, different n will cause them take different steps, while the second method which uses the formula, always take 1 step no matter what is n
Being said, if you know the formula beforehand, it is always the best you just compute the answer directly with the formula
Your second formula has O(1) complexity, that is, it runs in constant time, independent of n.
There's no contradiction. The complexity is a measure of how long the algorithm takes to run. Different algorithms can compute the same result at different speeds.
[BTW the correct formula is n*(n+1)/2.]
Edit: Perhaps your confusion has to do with an algorithm that takes n*(n+1)/2 steps, which is (n^2 + n)/2 steps. We call that O(n^2) because it grows essentially (asymptotically) as n^2 when n gets large. That is, it grows on the order of n^2, the high order term of the polynomial.
As part of a programming assignment I saw recently, students were asked to find the big O value of their function for solving a puzzle. I was bored, and decided to write the program myself. However, my solution uses a pattern I saw in the problem to skip large portions of the calculations.
Big O shows how the time increases based on a scaling n, but as n scales, once it reaches the resetting of the pattern, the time it takes resets back to low values as well. My thought was that it was O(nlogn % k) when k+1 is when it resets. Another thought is that as it has a hard limit, the value is O(1), since that is big O of any constant. Is one of those right, and if not, how should the limit be represented?
As an example of the reset, the k value is 31336.
At n=31336, it takes 31336 steps but at n=31337, it takes 1.
The code is:
def Entry(a1, q):
F = [a1]
lastnum = a1
q1 = q % 31336
rows = (q / 31336)
for i in range(1, q1):
lastnum = (lastnum * 31334) % 31337
F.append(lastnum)
F = MergeSort(F)
print lastnum * rows + F.index(lastnum) + 1
MergeSort is a standard merge sort with O(nlogn) complexity.
It's O(1) and you can derive this from big O's definition. If f(x) is the complexity of your solution, then:
with
and with any M > 470040 (it's nlogn for n = 31336) and x > 0. And this implies from the definition that:
Well, an easy way that I use to think about big-O problems is to think of n as so big it may as well be infinity. If you don't get particular about byte-level operations on very big numbers (because q % 31336 would scale up as q goes to infinity and is not actually constant), then your intuition is right about it being O(1).
Imagining q as close to infinity, you can see that q % 31336 is obviously between 0 and 31335, as you noted. This fact limits the number of array elements, which limits the sort time to be some constant amount (n * log(n) ==> 31335 * log(31335) * C, for some constant C). So it is constant time for the whole algorithm.
But, in the real world, multiplication, division, and modulus all do scale based on input size. You can look up Karatsuba algorithm if you are interested in figuring that out. I'll leave it as an exercise.
If there are a few different instances of this problem, each with its own k value, then the complexity of the method is not O(1), but instead O(k·ln k).
Here is an algorithm for finding kth smallest number in n element array using partition algorithm of Quicksort.
small(a,i,j,k)
{
if(i==j) return(a[i]);
else
{
m=partition(a,i,j);
if(m==k) return(a[m]);
else
{
if(m>k) small(a,i,m-1,k);
else small(a,m+1,j,k);
}
}
}
Where i,j are starting and ending indices of array(j-i=n(no of elements in array)) and k is kth smallest no to be found.
I want to know what is the best case,and average case of above algorithm and how in brief. I know we should not calculate termination condition in best case and also partition algorithm takes O(n). I do not want asymptotic notation but exact mathematical result if possible.
First of all, I'm assuming the array is sorted - something you didn't mention - because that code wouldn't otherwise work. And, well, this looks to me like a regular binary search.
Anyway...
The best case scenario is when either the array is one element long (you return immediately because i == j), or, for large values of n, if the middle position, m, is the same as k; in that case, no recursive calls are made and it returns immediately as well. That makes it O(1) in best case.
For the general case, consider that T(n) denotes the time taken to solve a problem of size n using your algorithm. We know that:
T(1) = c
T(n) = T(n/2) + c
Where c is a constant time operation (for example, the time to compare if i is the same as j, etc.). The general idea is that to solve a problem of size n, we consume some constant time c (to decide if m == k, if m > k, to calculate m, etc.), and then we consume the time taken to solve a problem of half the size.
Expanding the recurrence can help you derive a general formula, although it is pretty intuitive that this is O(log(n)):
T(n) = T(n/2) + c = T(n/4) + c + c = T(n/8) + c + c + c = ... = T(1) + c*log(n) = c*(log(n) + 1)
That should be the exact mathematical result. The algorithm runs in O(log(n)) time. An average case analysis is harder because you need to know the conditions in which the algorithm will be used. What is the typical size of the array? The typical size of k? What is the mos likely position for k in the array? If it's in the middle, for example, the average case may be O(1). It really depends on how you use this.