I am struggling with writing the formula that describes the recursive nature of the foo method.
The problem is that as far as I can tell, since every time n is divided with 2,
the binary tree formula should apply here.
This says that when in each call we divide the data we get a formula like this:
And then if we analyze for 2 so :
We get:
Which means that C(N) = log(N + 1), namely O(logN)
That all makes sense and seems to be the right choice for the foo method but it cant be because for
n = 8 I would get 3 + 1 iterations that are not n + 1 = 8 + 1 = 9 iterations
So here is your code:
void foo(int n) {
if (n == 1) System.out.println("Last line I print");
if (n > 1) {
System.out.println("I am printing one more line");
foo(n/2);
}
}
We can write a recurrence relation down for its runtime T as a function of the value of the parameter passed into it, n:
T(1) = a, a constant
T(n) = b + T(n/2), b constant, n > 1
We can write out some values of T(n) for various values of n to see if a pattern emerges:
n T(n)
---------
1 a
2 a + b
4 a + 2b
8 a + 3b
...
2^k a + kb
So for n = 2^k, T(n) = a + kb. We can solve for k in terms of n as follows:
n = 2^k <=> k = log(n)
Then we recover the expression T(n) = a + blog(n). We can verify this expression works easily:
a + blog(1) = a, as required
a + blog(n) = b + (a + blog(n/2))
= b + (a + b(log(n) - 1)
= b + a + blog(n) - b
= a + blog(n), as required
You can also use mathematical induction to do the same thing.
Related
Consider this recurrence relation: x(n) = x(n/2) + n, for n > 1 and x(1) = 0.
Now here the method of back substitution will struggle for values of n not powers of 2, so it is best known here is to use the smoothness rule to solve this types of questions, and when we use the smoothness rule, where we will solve for n = 2^k (for n = values powers of 2) we will have a solution of x(n) = 2n - 1.
However, if we use the method of backward substitution, this recurrence relation will have a solution!
x(n) = x(n/2) + n = x(n/4) + n/2 + n = x(n/8) + n/4 + n/2 + n = x(n/16) + n/8 + n/4 + n/2 + n = ....
where the pattern is
x(n) = x(n/i) + n/(i/2) + n/(i/4) + n/(i/8) + n/(i/16) + ...
which will stop when n = 1 (i.e when i = n) and in this case
x(n) = x(n/n) + n/(n/2) + n/(n/4) + n/(n/8) + n/(n/16) + ... = 1 + 2 + 4 + 8 + 16 + ... = 2^(n+1) - 1
which is two different answers!
So please I am so confused here because in the textbook (Introduction to Analysis and Design of Algorithms by Anany Levitin) it is mention that we should use here the smoothness rule, but as you can see I have solved it exactly by the method of backward substitution where the method was expected here to struggle but nothing has happened!
The transition 1 + 2 + 4 + 8 + 16 + ... = 2^(n+1) - 1 is false.
That is since the number of elements in the left series is log n so the sum is 2^(log n + 1) - 1, which is exactly 2n - 1.
The reason there are log n elements is that n/(2^i) = 1 (the last element of the series is 1) when i = log n.
I wish to determine the average processing time T(n) of the recursive algorithm:
int myTest( int n ) {
if ( n <= 0 ) {
return 0;
}
else {
int i = random( n - 1 );
return myTest( i ) + myTest( n - 1 - i );
}
}
provided that the algorithm random( int n ) spends one time unit to return
a random integer value uniformly distributed in the range [0, n] whereas
all other instructions spend a negligibly small time (e.g., T(0) = 0).
This is certainly not of the simpler form T(n) = a * T(n/b) + c so I am lost at how to write it. I can't figure out how to write it due to the fact that I am taking a random number each time from 0 to n-1 range and supplying it twice to the function and asking for the sum of those two calls.
The recurrence relations are:
T(0) = 0
T(n) = 1 + sum(T(i) + T(n-1-i) for i = 0..n-1) / n
The second can be simplified to:
T(n) = 1 + 2*sum(T(i) for i = 0..n-1) / n
Multiplying by n:
n T(n) = n + 2*sum(T(i) for i = 0..n-1)
Noting that (n-1) T(n-1) = n-1 + 2*sum(T(i) for i = 0..n-2), we get:
n T(n) = (n-1) T(n-1) + 1 + 2T(n-1)
= (n+1) T(n-1) + 1
Or:
T(n) = ((n+1)T(n-1) + 1) / n
This has the solution T(n) = n, which you can derive by telescoping the series, or by guessing the solution and then substituting it in to prove it works.
Foo(A,f,l)
**Precondition: A[f ...l] is an array of integers, f,l are two naturals ≥ 1 with f ≤ l.
if (f = l) then
return A[f]
else
m ← floor of((f+l)/2)
return min(Foo(A,f,m), Foo(A,m + 1,l))
end if
Correct me if I'm wrong, but I think this code returns the smallest integer of the array. But how do I figure out what the recurrence relation that describe the time complexity in terms of array A? Could you please guide me to the solution so I can understand? I don't even know where to begin.
The recurrence relation we can recover from the structure of the pseudocode. We can let T(n) represent the time taken by the algorithm as a function of the input size. For n = 1, the time is constant, say T(1) = a. Our question now is for larger n, how can we express T(n)?
We will be in the else clause for n > 1. We do some extra work - let's call it b - and then call the function twice, once for an input of size floor(n/2) and once for an input of size ceiling(n/2). So we can write this part of the recursion as T(n) = b + T(floor(n/2)) + T(ceiling(n/2)). We can now write out some terms.
n T(n)
1 a
2 b + a + a = b + 2a
3 b + b + 2a + a = 2b + 3a
4 b + b + 2a + b + 2a = 3b + 4a
5 b + b + 2a + 2b + 3a = 4b + 5a
... ...
k = (k-1)b + (k)a = kb - b + ka = k(a + b) - b
We find a guess that T(n) = (a + b)n - b for some constants a and b that depend upon the cost of amounts of work we might take as constant (note that computing (f + l) / 2 is not really constant in terms of n, but it will not change our analysis). We can prove this using mathematical induction:
T(1) = a = (a + b)(1) - b is right;
Assume that T(n) = (a + b)n - b for all n <= k.
Does T(k + 1) = (a + b)(k + 1) - b hold? Remember that T(k + 1) = b + T(floor((k+1)/2)) + T(ceiling((k+1)/2). Suppose k+1 is even and m = (k+1)/2. Then T(k+1) = b + 2T(m) = b + 2[(a + b)(m) - b] = b + 2(m)(a+b) - 2b = (2m)(a+b) - b = (k+1)(a+b) - b, as required. The case wherek + 1` is odd is left as an exercise.
This is linear.
You're right. It returns the smallest integer of the array.
And the complexity is
O(nlog(n)); n = size of the array
Explanation: In each call, you are breaking the array into two equal parts which calls up to f=l. It calls the function O(log(n)) times for each number in the array. So, total complexity is O(nlog(n))
I have tried determining the running time given by a recurrence relation, but my result is not correct.
Recurrence
T(n) = c + T(n-1) if n >= 1
= d if n = 0
My attempt
I constructed this recursion tree:
n
|
n-1
|
n-2
|
n-3
|
n-4
|
n-5
|
|
|
|
|
|
Till we get 1
Now at level i, the size of the sub problem should be, n-i
But at last we want a problem of size 1. Thus, at the last level, n-i=1 which gives, i=n-1.
So the depth of the tree becomes n-1 and the height becomes n-1+1= n.
Now the time required to solve this recursion = height of the tree*time required at each level which is :
n+(n-1)+(n-2)+(n-3)+(n-4)+(n-5)+ ...
==> (n+n+n+n+n+ ... )-(1+2+3+4+5+ ... )
==> n - (n(n+1)/2)
Now the time taken = n* ((n-n2)/2) which should give the order to be n2, but that is not the correct answer.
Now at level i, the size of the sub problem should be, n-i
Yes, that is correct. But you're assuming, that the runtime equals the sum of all the subproblem sizes. Just think about it, already summing the first two levels gives n + (n - 1) = 2n - 1, why would the problem size increase? Disclaimer: A bit handwavy and not an entirely accurate statement.
What the formula actually says
T(n) = c + T(n-1)
The formula says, solving it for some n takes the same time it takes to solve it for a problem size that is one less, plus an additional constant c: c + T(n - 1)
Another way to put the above statement is this: Given the problem takes some time t for a certain problem size, it will take t + c for a problem size, that is bigger by one.
We know, that at a problem size of n = 0, this takes time d. According to the second statement, for a size of one more, n = 1, it will take d + c. Applying our rule again, it thus takes d + c + c for n = 2. We conclude, that it must take d + n*c time for any n.
This is not a proof. To actually prove this, you must use induction as shown by amit.
A correct recursion tree
Your recursion tree only lists the problem size. That's pretty much useless, I'm afraid. Instead, you need to list the runtime for said problem size.
Every node in the tree corresponds to a certain problem size. What you write into that node is the additional time it takes for the problem size. I.e. you sum over all the descendants of a node plus the node itself to get the runtime for a certain problem size.
A graphical representation of such a tree would look like this
Tree Corresponding problem size
c n
|
c n - 1
|
c n - 2
|
c n - 3
.
.
.
|
c 2
|
c 1
|
d 0
Formalizing: As already mentioned, the label of a node is the additional runtime it takes to solve for that problem size, plus all its descendants. The uppermost node represents a problem size of n, bearing the label c because that's in addition to T(n-1), to which it is connected using a |.
In a formula, you would only write this relation: T(n) = c + T(n-1). Given that tree, you can see how this applies to every n>=1. You could write this down like this:
T(n) = c + T(n - 1) # This means, `c` plus the previous level
T(n - 1) = c + T(n - 2) # i.e. add the runtime of this one to the one above^
T(n - 2) = c + T(n - 3)
...
T(n - (n - 2)) = c + T(1)
T(n - (n - 1)) = c + T(0)
T(0) = d
You can now expand the terms from bottom to top:
T(n - (n - 1)) = c + T(0)
T(0) = d
T(n - (n - 2)) = c + T(1)
T(n - (n - 1)) = c + d
T(0) = d
T(n - (n - 3)) = c + T(2)
T(n - (n - 2)) = c + (c + d)
T(n - (n - 1)) = c + d
T(0) = d
T(n - (n - 4)) = c + T(3)
T(n - (n - 3)) = c + (2*c + d)
T(n - (n - 2)) = c + (c + d)
...
T(n) = c + T(n - 1)
T(n - 1) = c + ((n-2)c + d)
T(n) = c + (n-1)c + d = n*c + d
T(n - 1) = (n-1)c + d
Summing 1 to n
n+(n-1)+(n-2)+(n-3)+(n-4)+(n-5)+ ...
==> (n+n+n+n+n+ ... )-(1+2+3+4+5+ ... )
==> n - (n(n+1)/2)
From the first line to the second line, you have reduced your problem from summing 1 to n to, well, summing 1 to n-1. That's not very helpful, because you're stuck with the same problem.
I'm not sure what you did on the third line, but your transition from the first to the second is basically correct.
This would have been the correct formula:
T(n) = c + T(n-1)
= c + (c + T(n-2))
= ...
= c*i + T(n-i)
= ...
= c*n + T(0)
= c*n + d
If we assume c,d are constants - it gets you O(n)
To prove it mathematically - one can use mathematical induction
For each k < n assume T(n) = c*n + d
Base is T(0) = 0*n + d = d, which is correct for n < 1
T(n) = c + T(n-1) (*)
= c + (n-1)*c + d
= c*n + d
(*) is the induction hypothesis, and is valid since n-1 < n
The complexity would be O(n).
As you described the functions converts the problem for input n into problem for (n-1) by using a constant operation 'c'.
So moving down the recursion tree we will have in total n levels, and at each step we require some constant operation 'c'.
So there will be total c*n operations resulting into the complexity O(n).
I have the following worked out:
T(n) = T(n - 1) + n = O(n^2)
Now when I work this out I find that the bound is very loose. Have I done something wrong or is it just that way?
You need also a base case for your recurrence relation.
T(1) = c
T(n) = T(n-1) + n
To solve this, you can first guess a solution and then prove it works using induction.
T(n) = (n + 1) * n / 2 + c - 1
First the base case. When n = 1 this gives c as required.
For other n:
T(n)
= (n + 1) * n / 2 + c - 1
= ((n - 1) + 2) * n / 2 + c - 1
= ((n - 1) * n / 2) + (2 * n / 2) + c - 1
= (n * (n - 1) / 2) + c - 1) + (2 * n / 2)
= T(n - 1) + n
So the solution works.
To get the guess in the first place, notice that your recurrence relationship generates the triangular numbers when c = 1:
T(1) = 1:
*
T(2) = 3:
*
**
T(3) = 6:
*
**
***
T(4) = 10:
*
**
***
****
etc..
Intuitively a triangle is roughly half of a square, and in Big-O notation the constants can be ignored so O(n^2) is the expected result.
Think of it this way:
In each "iteration" of the recursion you do O(n) work.
Each iteration has n-1 work to do, until n = base case. (I'm assuming base case is O(n) work)
Therefore, assuming the base case is a constant independant of n, there are O(n) iterations of the recursion.
If you have n iterations of O(n) work each, O(n)*O(n) = O(n^2).
Your analysis is correct. If you'd like more info on this way of solving recursions, look into Recursion Trees. They are very intuitive compared to the other methods.
The solution is pretty easy for this one. You have to unroll the recursion:
T(n) = T(n-1) + n = T(n-2) + (n - 1) + n =
= T(n-3) + (n-2) + (n-1) + n = ... =
= T(0) + 1 + 2 + ... + (n-1) + n
You have arithmetic progression here and the sum is 1/2*n*(n-1). Technically you are missing the boundary condition here, but with any constant boundary condition you see that the recursion is O(n^2).
Looks about right, but will depend on the base case T(1). Assuming you will do n steps to get T(n) to T(0) and each time the n term is anywhere between 0 and n for an average of n/2 so n * n/2 = (n^2)/2 = O(n^2).