I've been trying to set up clear information on iterating through and i and then j, but i get stuck when trying to make sense of the while loop?
Can someone please give me some information on how to solve something like this please?
Disclaimer: this answer is long and overly verbose because I wanted to provide the OP with a "baby-steps" method rather than a result. I hope she can find some help from it - would it be needed.
If you get stuck when trying to derive the complexity in one go, you can try to breakdown the problem into smaller pieces easier to reason about.
Introducing notations can help in this context to structure your thought process.
Let's introduce a notation for the inner while loop. We can see the starting index j depends on n - i, so the number of operations performed by this loop will be a function of n and i. Let's represent this number of operations by G(i, n).
The outer loop depends only on n. Let's represent the number of operations by T(n).
Now, let's write down the dependency between T(n) and G(n, i), reasoning about the outer loop only - we do not care about the inner while loop for this step because we have already abstracted its complexity in the function G. So we go with:
T(n) = G(n, n - 1) + G(n, n - 2) + G(n, n - 3) + ... + G(n, 0)
= sum(k = 0, k < n, G(n, k))
Now, let's focus on the function G.
Let's introduce an additional notation and write j(t) the value of the index j at the t-th iteration performed by the while loop.
Let's call k the value of t for which the invariant of the while loop is breached, i.e. the last time the condition is evaluated.
Let's consider an arbitrary i. You could try with a couple of specific values of i (e.g. 1, 2, n) if this helps.
We can write:
j(0) = n - i
j(1) = n - i - 3
j(2) = n - i - 6
...
j(k - 1) = n - i - 3(k - 1) such that j(k-1) >= 0
j(k) = n - i - 3k such that j(k) < 0
Finding k involves solving the inequality n - 1 - 3k < 0. To make it easier, let's "ignore" the fact that k is an integer and that we need to take the integer part of the result below.
n - i - 3k < 0 <=> k = (n - i) / 3
So there are (n - i) / 3 "steps" to consider. By steps we refer here to the number of evaluation of the loop condition. The number of times the operation j <- j - 3 is performed would be the latter minus one.
So we found an expression for G(n, i):
G(n, i) = (n - i) / 3
Now let's get back to the expression of T(n) we found in (3):
T(n) = sum(k = 0, k < n, (n - k) / 3)
Since when k varies from 0 to n, n - k varies from n to 0, we can equivalently write T(n) as:
T(n) = sum(k = 0, k <= n, k / 3)
= (1/3).sum(k = 0, j <= n, k)
= (1/6).n.(n+1)
And you can therefore conclude with
T(n) = Theta(n^2)
This resolution exhibited some patterns from which you can create your own recipe to solve similar exercises:
Consider the number of operations at individual levels (one loop at a time) and introduce notations for the functions representing them;
Find relationships between these functions;
Find an algebraic expression of the most inner functions, which doesn't depend on other functions you introduced and for which the number of steps can be calculated from a loop invariant;
Using the relationships between these functions, find expressions for the functions higher in the stack.
In order to calculate all time-complexity of code, replace each loop with a summation. Moreover, consider that the second loop run (n - i)/3 since j decreases with step size of 3. So we have:
Related
I'm trying to figure out this time complexity:
for(i=0; i<=(n/2)-1; i++){
for (j=i+1; j<=(n/2)-1; j++){
--some O(1) thing--
}
}
The outer loop I understand to be on its own O(n/2.) However with the inner loop as well I can't wrap my brain around how to break down how many times O(1) executes.
If the inner one stared j=0 I could do n/2(inner) * n/2(outer) = O(n^2) time complexity right? However since j depends on i, I'm thinking some type of summation is involved from i+1 to n/2 but i can't figure out how to set it up...
Basically I need help kind of visualizing how many times it loops, and how to set the summation up. Thank you! :)
Assuming that m = n/2. You will see that in inner loop, j will iterater over range m-1, m-2, m-3,... 1. Summing all of that will be 1+2+..+m-1 = (m-1)*m/2 = O(m^2)
Premise
For simplicity, let us call m = n / 2 - 1. The outer loop runs from 0 to m. The inner loop from i + 1 to m.
Iteration counting
We need to count how often the inner statement which you labeled O(1) is executed. That is, how often the inner loop runs in total, as executed by the outer loop. So let us take a look.
The first iteration of the outer loop generates m - 1 iterations of the inner loop. The second generates m - 1, then m - 2, m - 3, m - 4, ..., 2, 1, 0.
That means that the O(1) statement is, in total, executed:
(m - 1) + (m - 2) + (m - 3) + ... + 2 + 1 + 0
That is the sum from 0 up to m - 1
sum_{i = 0}^{m - 1} i
which can be simplified to
(m^2 - m) / 2
Substitute back
Let us now substitute back m = n / 2 - 1, we get
((n / 2 - 1)^2 - (n / 2 - 1)) / 2
After simplifying, this is
n^2/8 - 3n/4 + 1
Big-O
For Big-O, we observe that it is smaller than
n^2 - 0 + n^2
= 2n^2
Which, by definition is O(n^2).
As you see, this bound is also tight. So we also receive Omega(n^2) which also concludes Theta(n^2).
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
Analyze the complexity of the following algorithms. Said T(n) the running time of the algorithm, determine a function f (n) such that T(n) = O(f(n)). Also, let's say if it also applies T(n) = Θ(f(n)). The answers must be motivated.
I never do this kind of exercise.
Could someone explain what I have to analyze and how can I do it?
j=1,k=0;
while j<=n do
for l=1 to n-j do
k=k+j;
end for
j=j*4;
end while
Thank you.
Step 1
Following on from the comments, the value of j can be written as a power of 4. Therefore the code can be re-written in the following way:
i=0,k=0; // new loop variable i
while (j=pow(4,i)) <= n do // equivalent loop condition which calculates j
for l=1 to n-j do
k=k+j;
end for
i=i+1; // equivalent to j=j*4
end while
The value of i increases as 0, 1, 2, 3, 4 ..., and the value of j as 1, 4, 16, 64, 256 ... (i.e. powers of 4).
Step 2
What is the maximum value of i, i.e. how many times does the outer loop run? Inverting the equivalent loop condition:
pow(4,i) <= n // loop condition inequality
--> i <= log4(n) // take logarithm base-4 of both sides
--> max(i) = floor(log4(n)) // round down
Now that the maximum value of i is known, it's time to re-write the code again:
i=0,k=0;
m=floor(log4(n)) // maximum value of i
while i<=m do // equivalent loop condition in terms of i only
j=pow(4,i) // value of j for each i
for l=1 to n-j do
k=k+j;
end for
i=i+1;
end while
Step 3
You have correctly deduced that the inner loop runs for n - j times for every outer loop. This can be summed over all values of j to give the total time complexity:
j≤n
T(n) = ∑ (n - j)
j
i≤m
= ∑ (n - pow(4,i)) // using the results of steps 1) and 2)
i=0
i≤m
= (m+1)*n - ∑ pow(4,i) // separate the sum into two parts
i=0
\_____/ \_________/
A B
The term A is obviously O(n log n), because m=floor(log4(n)). What about B?
Step 4
B is a geometric series, for which there is a standard formula (source – Wikipedia):
Substituting the relevant numbers "a" = 1, "n" = m+1, "r" = 4:
B = (pow(4,m+1) - 1) / (4 - 1)
= 3 * pow(4, floor(log4(n))+1) - 3
If a number is rounded down (floor), the result is always greater than the original value minus 1. Therefore m can be asymptotically written as:
m = log4(n) + O(1)
--> B = 3 * pow(4, log4(n) + O(1)) - 3
= 3 * pow(4, O(1)) * n - 3
----------------
this is O(1)
= O(n)
Step 5
A = O(n log n), B = O(n), so asymptotically A overshadows B.
The total time complexity is O(n log n).
Consider the number of times each instruction is executed depending on n (the variable input). Let's call that the cost of each instruction. Typically, some parts of the algorithm are run a significantly greater number of times more often than other parts. Also typically, this "significantly greater number" is such that it asymptotically dominates all others, meaning that as n grows larger, the cost of all other instructions become negligible. Once you understand that, you simply have to figure out the cost of the significant instruction, or at least what it is proportional to.
In your example, two instructions are potentially costly; let k=k+j; cost x, and j=j*4; cost y.
j=1,k=0; // Negligible
while j<=n do
for l=1 to n-j do
k=k+j; // Run x times
end for
j=j*4; // Run y times
end while
Being tied to only one loop, y is easier to determine. The outer loop runs for j from 1 to n, with j growing exponentially: its value follows the sequence [1, 4, 16, 64, ...] (the i-th term is 4^i, starting at 0). That simply means that y is proportional to the logarithm of n (of any base, because all logarithms are proportional). So y = O(log n).
Now for x: we know it is a multiple of y since it is tied to an inner loop. For each time the outer loop runs, this inner loop runs for l from 1 to n-j, with l growing linearly (it's a for loop). That means it simply runs n-j-1 times, or n-1 - 4^i with i being the index of the current outer loop, starting at 0.
Since y = O(log n), x is proportional to the sum of n - 1 - 4^i, for i from 0 to log n, or
(n-1 - 4^0) + (n-1 - 4^1) + (n-1 - 4^2) + ... =
((log n)-1) * (n-1) - (1-4^(log n))/(1-4) =
O(log n * n) + O(n) =
O(n log n)
And here is your answer: x = O(n log n), which dominates all other costs, so the total complexity of the algorithm is O(n log n).
You need to calculate how many times each line will execute.
j=1,k=0; // 1
while j<=n do //n+1
for l=1 to n-j do // ∑n
k=k+j; //∑n-1
end for
j=j*4; //n
end while
total complexity [add execution time of all lines]
= 1+(n+1)+ ∑ n + ∑ (n-1) + n
= 2n+2+ n^2/2 + n/2 + (n-1)^2/2 + (n-1)/2
take max term of above and skip constant factors then you will left with n^2
total runtime complexity will be o(n^2)
Looks like a homework question, but to give you a hint: The coplexity can be calculated by the amount of loops. One loop means O(n) two loops O(n^2) and three loops O(n^3).
This only goes for neste loops:
while () {
while () {
while() {
}
}
}
this is O(n^3)
But...
while () {
}
while() {
}
Still is O(n), because the loopsdo not run over each other and will stop after n iterations.
EDIT
The correct answer should be O(n*log(n)), beacuse of the inner for-loop the amount of iterations depends on the value of j. Which can be different every iteration.
My Question
What is the Big-O complexity for this code snippet?
Consider n as a power of 4.
for(int i = 1; i <= n; i = i*4)
for(int k = 1; k <= i; k++)
// constant statement
What I know so far
I tried making this code into a summation to find the complexity.
This is what I got:
I got (base 4) log(n) by computing the series 4, 4^2, 4^3 ... 4^r = n.
r = (base 4) log(n).
I'm now stuck at this summation:
Please let me know If I'm doing something wrong or there is another way to do this.
You’re on the right track here, but your innermost summation is wrong. You are right that the outer loop will iterate log_4 n times, but you’ve set up the outer sum so that i counts up as 1, 2, 3, ..., log_4 n, rather than 4^0, 4^1, 4^2, ... 4^log_4 n. As a result, that inner summation’s upper bound is incorrect. The bound should be 4^i, not i.
If you set things up this way, you’ll find that the overall sum is
4^0 + 4^1 + 4^2 + ... + 4^log_4 n
= (4^(log_4 n + 1) - 1) / (4 - 1) (using the formula for the sum of a geometric series
= (4(4^log_4 n) - 1) / 3
= (4n - 1) / 3
= Θ(n).
You may use wolframalpha to get the result for extreme accuracy.
https://www.wolframalpha.com/input/?i=sum+i,+i%3D1+to+log_4(n)
Say I have k for loops nested in the following fashion:
for a = 1 to n:
for b = 1 to n-a:
for c = 1 to n-a-b:
for d = 1 to n-a-b-c:
O(1)
for any arbitrary k, but all k of these loops "share" the limit of n iterations with each other, is the big-O complexity still O(n^k)? Or is it some order below that?
Edit: What is the Big-O of a nested loop, where number of iterations in the inner loop is determined by the current iteration of the outer loop? is indeed asking something similar but it wasn't asking (nor does the answer address) if additional levels of nesting will change anything.
Dmitry's answer explains it very well for me.
OK, let's sum them all up: using induction you can find out that numbers of loops (for large n > k) are:
1. n
2. n * (n - 1) / 2
3. n * (n - 1) * (n - 2) / 6
...
k. n * (n - 1) * ... * (n - k + 1) / k!
...
As you can see the complexity is O(n**k) as you've conjured for any k providing that n is large enough (n > k).
I am given a formula f(n) where f(n) is defined, for all non-negative integers, as:
f(0) = 1
f(1) = 1
f(2) = 2
f(2n) = f(n) + f(n + 1) + n (for n > 1)
f(2n + 1) = f(n - 1) + f(n) + 1 (for n >= 1)
My goal is to find, for any given number s, the largest n where f(n) = s. If there is no such n return None. s can be up to 10^25.
I have a brute force solution using both recursion and dynamic programming, but neither is efficient enough. What concepts might help me find an efficient solution to this problem?
I want to add a little complexity analysis and estimate the size of f(n).
If you look at one recursive call of f(n), you notice, that the input n is basically divided by 2 before calling f(n) two times more, where always one call has an even and one has an odd input.
So the call tree is basically a binary tree where always the half of the nodes on a specific depth k provides a summand approx n/2k+1. The depth of the tree is log₂(n).
So the value of f(n) is in total about Θ(n/2 ⋅ log₂(n)).
Just to notice: This holds for even and odd inputs, but for even inputs the value is about an additional summand n/2 bigger. (I use Θ-notation to not have to think to much about some constants).
Now to the complexity:
Naive brute force
To calculate f(n) you have to call f(n) Θ(2log₂(n)) = Θ(n) times.
So if you want to calculate the values of f(n) until you reach s (or notice that there is no n with f(n)=s) you have to calculate f(n) s⋅log₂(s) times, which is in total Θ(s²⋅log(s)).
Dynamic programming
If you store every result of f(n), the time to calculate a f(n) reduces to Θ(1) (but it requires much more memory). So the total time complexity would reduce to Θ(s⋅log(s)).
Notice: Since we know f(n) ≤ f(n+2) for all n, you don't have to sort the values of f(n) and do a binary search.
Using binary search
Algorithm (input is s):
Set l = 1 and r = s
Set n = (l+r)/2 and round it to the next even number
calculate val = f(n).
if val == s then return n.
if val < s then set l = n
else set r = n.
goto 2
If you found a solution, fine. If not: try it again but round in step 2 to odd numbers. If this also does not return a solution, no solution exists at all.
This will take you Θ(log(s)) for the binary search and Θ(s) for the calculation of f(n) each time, so in total you get Θ(s⋅log(s)).
As you can see, this has the same complexity as the dynamic programming solution, but you don't have to save anything.
Notice: r = s does not hold for all s as an initial upper limit. However, if s is big enough, it holds. To be save, you can change the algorithm:
check first, if f(s) < s. If not, you can set l = s and r = 2s (or 2s+1 if it has to be odd).
Can you calculate the value of f(x) which x is from 0 to MAX_SIZE only once time?
what i mean is : calculate the value by DP.
f(0) = 1
f(1) = 1
f(2) = 2
f(3) = 3
f(4) = 7
f(5) = 4
... ...
f(MAX_SIZE) = ???
If the 1st step is illegal, exit. Otherwise, sort the value from small to big.
Such as 1,1,2,3,4,7,...
Now you can find whether exists n satisfied with f(n)=s in O(log(MAX_SIZE)) time.
Unfortunately, you don't mention how fast your algorithm should be. Perhaps you need to find some really clever rewrite of your formula to make it fast enough, in this case you might want to post this question on a mathematics forum.
The running time of your formula is O(n) for f(2n + 1) and O(n log n) for f(2n), according to the Master theorem, since:
T_even(n) = 2 * T(n / 2) + n / 2
T_odd(n) = 2 * T(n / 2) + 1
So the running time for the overall formula is O(n log n).
So if n is the answer to the problem, this algorithm would run in approx. O(n^2 log n), because you have to perform the formula roughly n times.
You can make this a little bit quicker by storing previous results, but of course, this is a tradeoff with memory.
Below is such a solution in Python.
D = {}
def f(n):
if n in D:
return D[n]
if n == 0 or n == 1:
return 1
if n == 2:
return 2
m = n // 2
if n % 2 == 0:
# f(2n) = f(n) + f(n + 1) + n (for n > 1)
y = f(m) + f(m + 1) + m
else:
# f(2n + 1) = f(n - 1) + f(n) + 1 (for n >= 1)
y = f(m - 1) + f(m) + 1
D[n] = y
return y
def find(s):
n = 0
y = 0
even_sol = None
while y < s:
y = f(n)
if y == s:
even_sol = n
break
n += 2
n = 1
y = 0
odd_sol = None
while y < s:
y = f(n)
if y == s:
odd_sol = n
break
n += 2
print(s,even_sol,odd_sol)
find(9992)
This recursive in every iteration for 2n and 2n+1 is increasing values, so if in any moment you will have value bigger, than s, then you can stop your algorithm.
To make effective algorithm you have to find or nice formula, that will calculate value, or make this in small loop, that will be much, much, much more effective, than your recursion. Your recursion is generally O(2^n), where loop is O(n).
This is how loop can be looking:
int[] values = new int[1000];
values[0] = 1;
values[1] = 1;
values[2] = 2;
for (int i = 3; i < values.length /2 - 1; i++) {
values[2 * i] = values[i] + values[i + 1] + i;
values[2 * i + 1] = values[i - 1] + values[i] + 1;
}
And inside this loop add condition of possible breaking it with success of failure.