Input: Two n-bit integers x and y, where y ≥ 1.
Output: The quotient and remainder of x divided by y.
if x = 0, then return (q, r) := (0, 0);
q := 0; r := x;
while (r ≥ y) do // takes n iterations for the worse case.
{
q := q + 1;
r := r – y
}; // O(n) for each r – y, where y is n bits long.
return (q, r);
For the above algorihtm which mainly focuses on dividing two 'n' bit integers 'x' and 'y', can somone please explain and let me know the time complexity in terms of 'n'.
Can't guarantee my explanation is flawless, but here it is anyway:
Just as written in comments, the time complexity is O(n), i.e. linear. In other words, the time it takes for the computer to execute the algorithm increases linearly with the increase of x or the decrease of y.
Example: it will take 3 loops to get the result if x = 10 and y = 3, and it will take twice as many loops if we increase x by two to equal 20. That's what they call O(n).
Other time complexity types that exist are O(n^2) - quadratic time (ex: you increase x by 2 and the number of required loops increases by 4), O(n^3) - cubic (same logic), O(logn) - logarithmic time (the time is increasing less than linearly and at some point almost stops to increase at all).
And finally, there's O(1) - constant time. The above algorithm can be improved to have constant time if you type q=x/y and r=x%y instead of using the loops.
Related
Input: Two n-bit integers x and y, where x ≥ 0, y ≥ 1.
Output: The quotient and remainder of x divided by y.
if x = 0, then return (q, r) := (0, 0);
q := 0; r := x;
while (r ≥ y) do
{ q := q + 1;
r := r – y};
return (q, r);
I have obtained the Big O complexity as O(n^2) but a friends says it is O(2^n) where n is the number of bits as the input size
Please provide explanation
The number of iterations of the while-loop is exactly floor(x/y). Each iteration takes n operations, because that is the complexity of the subtraction r - y.
Hence the complexity of the algorithm is n * floor(x/y). However, we want to express the complexity as a function of n, not as a function of x and y.
Thus the question becomes: how does floor(x/y) relate to n, in the worst case?
The biggest value that can be obtained for x/y when x and y are two nonnegative n-digits numbers, and y >= 1, is obtained by taking the biggest possible value for x, and the smallest possible value for y.
The biggest possible value for x is x = 2**n - 1 (all bits of x are 1 in its binary representation);
The smallest possible value for y is y = 1.
Hence the biggest possible value for x/y is x/y = 2**n - 1.
The time-complexity of your division algorithm is O(n * 2**n), and this upper-bound is achieved when x = 2**n - 1 and y = 1.
My proposed solution:
When calculating Big O complexity we need to take n->infinity, where n is > input size, we have 3
possibilities:
x & y both become infinity when n->infinity
y becomes infinity when n->infinity
x becomes infinity when n-> infinity
We are interested only in case 3 here,
(x – i * y ) < y, i being the number of iterations
also written as x/y < i + 1
when x becomes infinity LHS(Left hand side) is infinity in this
equation which implies RHS is infinity as well
So as n-> infinity the number iterations becomes equal to n
Hence, the complexity is O(n^2)
I am currently learning about searching algorithms and I came across jump search, which has a time complexity of O(√n). Why is √n the optimal value for m (jump size) in the jump search algorithm and how does it affect the time complexity?
Let m be the jump size and n be the number of elements.
In the worst case, the maximum number of elements you have to check is the maximum number of jumps (n/m - 1) plus the number of elements between jumps (m), and the time you take is approximately proportional to the total number of elements you check.
The goal in choosing m, therefore, is to minimize: (n/m)+m-1.
The derivative by m is 1 - (n/m2), and the minimum occurs where the derivative is 0:
1 - (n/m2) = 0
(n/m2) = 1
n = m2
m = √n
Assuming the block size is k, the worst case scenario requires roughly n / k iterations for finding the block and k iterations for finding the element in the block.
To minimize n / k + k where n is a constant, we can use differentiation (see Matt's answer) or the AM-GM inequality to get:
n / k + k >= 2sqrt(n)
We can clearly see that 2sqrt(n) is the minimal number of iterations and is attainable when k = sqrt(n).
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
Analyze the complexity of the following algorithms. Said T(n) the running time of the algorithm, determine a function f (n) such that T(n) = O(f(n)). Also, let's say if it also applies T(n) = Θ(f(n)). The answers must be motivated.
I never do this kind of exercise.
Could someone explain what I have to analyze and how can I do it?
j=1,k=0;
while j<=n do
for l=1 to n-j do
k=k+j;
end for
j=j*4;
end while
Thank you.
Step 1
Following on from the comments, the value of j can be written as a power of 4. Therefore the code can be re-written in the following way:
i=0,k=0; // new loop variable i
while (j=pow(4,i)) <= n do // equivalent loop condition which calculates j
for l=1 to n-j do
k=k+j;
end for
i=i+1; // equivalent to j=j*4
end while
The value of i increases as 0, 1, 2, 3, 4 ..., and the value of j as 1, 4, 16, 64, 256 ... (i.e. powers of 4).
Step 2
What is the maximum value of i, i.e. how many times does the outer loop run? Inverting the equivalent loop condition:
pow(4,i) <= n // loop condition inequality
--> i <= log4(n) // take logarithm base-4 of both sides
--> max(i) = floor(log4(n)) // round down
Now that the maximum value of i is known, it's time to re-write the code again:
i=0,k=0;
m=floor(log4(n)) // maximum value of i
while i<=m do // equivalent loop condition in terms of i only
j=pow(4,i) // value of j for each i
for l=1 to n-j do
k=k+j;
end for
i=i+1;
end while
Step 3
You have correctly deduced that the inner loop runs for n - j times for every outer loop. This can be summed over all values of j to give the total time complexity:
j≤n
T(n) = ∑ (n - j)
j
i≤m
= ∑ (n - pow(4,i)) // using the results of steps 1) and 2)
i=0
i≤m
= (m+1)*n - ∑ pow(4,i) // separate the sum into two parts
i=0
\_____/ \_________/
A B
The term A is obviously O(n log n), because m=floor(log4(n)). What about B?
Step 4
B is a geometric series, for which there is a standard formula (source – Wikipedia):
Substituting the relevant numbers "a" = 1, "n" = m+1, "r" = 4:
B = (pow(4,m+1) - 1) / (4 - 1)
= 3 * pow(4, floor(log4(n))+1) - 3
If a number is rounded down (floor), the result is always greater than the original value minus 1. Therefore m can be asymptotically written as:
m = log4(n) + O(1)
--> B = 3 * pow(4, log4(n) + O(1)) - 3
= 3 * pow(4, O(1)) * n - 3
----------------
this is O(1)
= O(n)
Step 5
A = O(n log n), B = O(n), so asymptotically A overshadows B.
The total time complexity is O(n log n).
Consider the number of times each instruction is executed depending on n (the variable input). Let's call that the cost of each instruction. Typically, some parts of the algorithm are run a significantly greater number of times more often than other parts. Also typically, this "significantly greater number" is such that it asymptotically dominates all others, meaning that as n grows larger, the cost of all other instructions become negligible. Once you understand that, you simply have to figure out the cost of the significant instruction, or at least what it is proportional to.
In your example, two instructions are potentially costly; let k=k+j; cost x, and j=j*4; cost y.
j=1,k=0; // Negligible
while j<=n do
for l=1 to n-j do
k=k+j; // Run x times
end for
j=j*4; // Run y times
end while
Being tied to only one loop, y is easier to determine. The outer loop runs for j from 1 to n, with j growing exponentially: its value follows the sequence [1, 4, 16, 64, ...] (the i-th term is 4^i, starting at 0). That simply means that y is proportional to the logarithm of n (of any base, because all logarithms are proportional). So y = O(log n).
Now for x: we know it is a multiple of y since it is tied to an inner loop. For each time the outer loop runs, this inner loop runs for l from 1 to n-j, with l growing linearly (it's a for loop). That means it simply runs n-j-1 times, or n-1 - 4^i with i being the index of the current outer loop, starting at 0.
Since y = O(log n), x is proportional to the sum of n - 1 - 4^i, for i from 0 to log n, or
(n-1 - 4^0) + (n-1 - 4^1) + (n-1 - 4^2) + ... =
((log n)-1) * (n-1) - (1-4^(log n))/(1-4) =
O(log n * n) + O(n) =
O(n log n)
And here is your answer: x = O(n log n), which dominates all other costs, so the total complexity of the algorithm is O(n log n).
You need to calculate how many times each line will execute.
j=1,k=0; // 1
while j<=n do //n+1
for l=1 to n-j do // ∑n
k=k+j; //∑n-1
end for
j=j*4; //n
end while
total complexity [add execution time of all lines]
= 1+(n+1)+ ∑ n + ∑ (n-1) + n
= 2n+2+ n^2/2 + n/2 + (n-1)^2/2 + (n-1)/2
take max term of above and skip constant factors then you will left with n^2
total runtime complexity will be o(n^2)
Looks like a homework question, but to give you a hint: The coplexity can be calculated by the amount of loops. One loop means O(n) two loops O(n^2) and three loops O(n^3).
This only goes for neste loops:
while () {
while () {
while() {
}
}
}
this is O(n^3)
But...
while () {
}
while() {
}
Still is O(n), because the loopsdo not run over each other and will stop after n iterations.
EDIT
The correct answer should be O(n*log(n)), beacuse of the inner for-loop the amount of iterations depends on the value of j. Which can be different every iteration.
I am confused about the pseudopolynomial time in compare to polynomial time
input(n);
for (int i=0; i<n;i++){
doStuff; }
The runtime would be O(n) but writing out the number n takes x=O(log n) bits.
So, if we let x be the number of bits required to write out the input n, the runtime of this algorithm is actually O(2^x), which is not a polynomial in x.
Is this conclusion correct?
Edit: Look at simple primetest.
function isPrime(n):
for i from 2 to n - 1:
if (n mod i) = 0, return false
return true
The runtime would be O(n). But remember, the formal definition of time complexity talks about the complexity of the algorithm as a function of the number of bits of input.
Therefore, if we let x be the number of bits required to write out the input n, the runtime of this algorithm is actually O(2^x), which is not a polynomial in x.
EDIT2: i got all your points but look at Knapsack problem.
// Input:
// Values (stored in array v)
// Weights (stored in array w)
// Number of distinct items (n)
// Knapsack capacity (W)
for j from 0 to W do:
m[0, j] := 0
for i from 1 to n do:
for j from 0 to W do:
if w[i] > j then:
m[i, j] := m[i-1, j]
else m[i, j] := max(m[i-1, j], m[i-1, j-w[i]] + v[i])
if you guys are right it would mean that Knapsack problem has runtime o(n*W), therefore it has polynomial time !
Alex does 64 push-ups everyday.
and
Alex does 2^6 push-ups everyday.
If above two lines mean same to you, then O(n) and O(2^x) doesn't matter :)
O(2^x)
=> O(2^log_2(n))
=> n [as we know x^log_x(y) = y]
The formal definition of time complexity talks about the complexity of
the algorithm as a function of the number of bits of input.
Yes, you're right. But the idea of Big-O analysis is about the growth rate of algorithm with growth of input, not the precise counting of exactly how many times my loop iterates.
As for example, when n = 32, the algorithm complexity is O(2^5), but with growth of n, for example when n = 1048576, the complexity will be O(2^20). So, complexity increases with input increases.
n or 2^(log_2(n)) are all about presenting same numeric amount differently. As long as the growth rate of the algorithm is linearly proportional to the growth rate of input, the algorithm is linear - no matter whether we represent the input n as e^x or log(y).
Edit
Quoted from Wikipedia
The O(nW) complexity does not contradict the fact that the knapsack
problem is NP-complete, since W, unlike n, is not polynomial in
the length of the input to the problem. The length of the W input to
the problem is proportional to the number of bits in W, log W, not
to W itself.
You first two snippet was about to n which has obviously polynomial growth.
Since,
x = ceil(log_2(n)), 2^x becomes 2^log_2(n), which is nothing but n (using a^log_a(b) = b).
Remember to analyse the runtime of an algorithm only in the terms of your input variables, and not doing something fancy like counting the bits it would require, since (in this case, for example) the number of bits itself is a logarithm of the number!
Here n>m.
I have analyzed the worst case when n = fibonacci Nth term and m = fiboncci (N-1)th term.In this case total work will be proportinal to N or time complexity will be O(N).But I am interested in finding time complexity(theta notation) in terms of n.But I am not getting how to find relation between n and N or the upper and lower bound in terms of n.
int gcd(int n, int m) {
if (n%m ==0) return m;
if (n < m) swap(n, m);
while (m > 0) {
n = n%m;
swap(n, m);
}
return n;
}
Please help.
I would try to analyse how m changes after two iterations of the loop. One iteration might change n very little, for example (1001, 1000) -> (1000, 1). m might also change very little, for example (1999, 1000) -> (1000, 999). So analysing how either changes in a single iteration gives you very little. However, if you have two iterations then you will always have a big change.
So analyse this: If r = n/m, how does m change after two iterations of the algorithm, depending on r? By which factor does it decrease at least? (By the way, what is r for Fibonacci numbers? Does that explain the worst case? )
You might find this interesting.
It tries to explain Lamé's theorem for the number of steps of the GCD algorithm. The worst case is when the 2 numbers are consecutive Fibonacci numbers.