How can I turn the following recursive algorithm into an iterative algorithm?
count(integer: n)
for i = 1...n
return count(n-i) + count(n-i)
return 1
Essentially this algorithm computes the following:
count(n-1) + count(n-2) + ... + count(1)
This is not a tail recursion, so it is not trivial to transform it into iterative.
However, a recursion can be simulated using a stack and loop pretty easily, by pushing to the stack rather than recursing.
stack = Stack()
stack.push(n)
count = 0
while (stack.empty() == false):
current = stack.pop()
count++
for i from current-1 to 1 inclusive (and descending):
stack.push(i)
return count
Another solution is doing it with Dynamic Programming, since you don't need to calculate the same thing multiple times:
DP = new int[n+1]
DP[0] = 1
for i from 1 to n:
DP[i] = 0
for j from 0 to i-1:
DP[i] += DP[j]
return DP[n]
Note that you can even optimize it to run in O(n) rather than O(n^2), by remembering the "so far sum":
sum = 1
current = 1
for i from 1 to n:
current = sum
sum = sum + current
return current
Lastly, this actually sums to something you can easily pre-calculate: count(n) = 2^(n-1), count(0) = 1 (You can suspect it from seeing the last iterative solution we have...)
base: count(0) automatically yields 1, as the loop's body is not reached.
Hypothesis: T(k) = 2^(k-1) for all k < n
Proof:
T(n) = T(n-1) + T(n-2) + ... + T(1) + T(0) = (induction hypothesis)
= 2^(n-2) + 2^(n-3) + ... + 2^0 + 1 =
= sum { 2^i | i=0,...,n-2 } + 1 = (sum of geometric series)
= (1-2^(n-1)/(1-2)) + 1 = (2^(n-1) - 1) + 1 = 2^(n-1)
If you define your problem in the following recursive way:
count(integer : n)
if n==0 return 1
return count(n-1)+count(n-1)
Converting to an iterative algorithm is a typical application of backwards induction where you should keep all previous results:
count(integer : n):
result[0] = 1
for i = 1..n
result[i] = result[i-1] + result[i-1]
return result[n]
Ir is clear that this is more complicated than it should be because the point is to exemplify backwards induction. I could be accumulating into a single place but I wanted to provide a more general concept that could be extended to other cases. In my opinion the idea is clearer this way.
The pseudocode can be improved after the key idea is clear. In fact, there are two very simple improvements that are applicable only to this specific case:
instead of keeping all previous values, only the last one is necessary
there is no need for two identical calls as there are no side-effects expected
Going beyond, it is possible to calculate that based on the definition of the function, count(n)= 2^n
The statement return count(n-i) + count(n-i) appears to be equivalent to return 2 * count(n-i). In that case:
count(integer: n)
result = 1
for i = 1...n
result = 2 * result
return result
What am I missing here?
Related
This algorithm is from Cracking the Coding Interview, 5th Edition, found here: https://www.geekbooks.me/book/view/cracking-the-coding-interview-5th-edition
A child is running up a staircase with n steps, and can hop either 1 step, 2 steps, or
3 steps at a time. Implement a method to count how many possible ways the child
can run up the stairs.
Algorithm:
public static int countWaysDP(int n, int[] map) {
if (n < 0) {
return 0;
} else if (n == 0) {
return 1;
} else if (map[n] > -1) {
return map[n];
} else {
map[n] = countWaysDP(n - 1, map) +
countWaysDP(n - 2, map) +
countWaysDP(n - 3, map);
return map[n];
}
}
What is the time and space complexity of this algorithm? I think since memoization is used, the results are stored so values don't get calculated multiple times like in the pure recursive method. Since there are three calls to countWaysDP the time complexity is O(3n) which is an element of O(n). The space complexity would be O(n+n) one n for the size of map and one n for the recursive call stack, which is also an element of O(n). Is my reasoning correct?
Thanks
Let us execute the code:
Note the notation of recursion stack. 1.2.3. means third recursion countWaysDP(n-5) of second recursion countWaysDP(n-2) of countWaysDP(n).
Consider n=6
1. countWaysDP(6)
1.1. countWaysDP(5)
1.1.1. countWaysDP(4)
1.1.1.1. countWaysDP(3)
1.1.1.1.1. countWaysDP(2)
1.1.1.1.1.1. countWaysDP(1)
1.1.1.1.1.1.1. countWaysDP(0) //returns 1
1.1.1.1.1.1.2. countWaysDP(-1) //returns 0
1.1.1.1.1.1.3. countWaysDP(-2) //returns 0 -> map[1] = 1 <-
1.1.1.1.1.2. countWaysDP(0) //return 1
1.1.1.1.1.3. countWaysDP(-1) //return 0 -> map[2] = 2 <-
1.1.1.1.2. countWaysDP(1) //already claculated, returns map[1]
1.1.1.1.3. countWaysDP(0) //returns 1 ->map[3] = 4 <-
1.1.1.2. countWaysDP(2) //already present, returns map[2]
1.1.1.3. countWaysDP(1) //already present, returns map[1] -> map[4] = 7 <-
1.1.2. countWaysDP(3) //already present, returns map[3]
1.1.3. countWaysDP(2) //already present, returns map[2] -> map[5] = 13 <-
1.2. countWaysDP(4) //already present, returns map[4]
1.3. countWaysDP(3) //already present, returns map[3] -> map[6] = 24 <-
Now assume that involking a method is O(1) operation. The total time taken for this example would be:
6 + 3 + (2 + 2 + 2 + 2 + 2) = 19
So yes, you are correct about the TIME. Its 3n as the leftmost recursion path is taking O(n) and then all other calls are O(2n).
The recursion stack would take O(n) as the maximum stack depth is n + 3 and your map will take O(n) space. So yes again, the SPACE is O(n + n) = O(n).
Regarding your algorithm: Although you store results in your map, your algorithms does 3 recursive call each time and only afterwards inserts the solution into the map. This means you don't reuse intermediate results.
In Order to reuse you would need to start with n=1 and then iterate until you reach your desired number of steps. This way you can be sure to reuse the result for all smaller steps:
for (int i = 1; i <= n; i++) {
int current = map[i-1]; // you can make 1 step
if (i > 1)
current += map[i-2]; // you can make 2 steps
if (i > 2)
current += map[i-3]; // you can make 3 steps
map[i] = current;
}
return map[n];
Now for the complexity:
We use one map that at the end has exactly n entries. Therefore the space complexity is O(n).
We iterate once from 1 to n to make the calculation. Therefore the time complexity is also O(n).
A simple solution using memoization (O(N) time complexity and space complexity):
public static int countWaysDP(int n, int[] map) {
map[1] = 1; // when n = 1, answer is 1
map[2] = 2; // when n = 2, answer is 2, (1+1) and (2)
map[3] = 4; // (1+1+1), (1+2), (2+1), (3)
for(int i = 4;i <= n; i++)
{
map[i] = map[i-1] + map[i-2] + map[i-3];
}
return map[n];
}
Hope this helps!!!
This algorithm find all prime numbers below N
var f = function(n){
var primes = [2]; //1
var flag; //1
for(var i=3; i<n; i+=2){ // ( from 3 to n-1 ) / 2
flag = true; //1
var startI = 0; // 1
for(var y=primes[startI]; y<=Math.sqrt(i); y=primes[++startI]){ // ???
if(i%y === 0) // 1
flag = false; // 1
}
if(flag) // 1
primes.push(i); // 1
}
return primes; // 1
}
So far my analysis is done till the first loop, I'm not sure how to handle the second sum ( the one that is using primes.length and Math.sqrt ).
T(n) = 1 + 1 + sum( ( 1+ 1+ ??weird sum???) , from i = 3 to n -1) / 2 + 1 + 1
I understand how to analyze till the second nested loop, I suspect that is around log(N) or something like that, but I would like to know the exact number of iterations..
Questions:
How can I handle that kind of loop that is using arrays in memory to iterate ?
If not possible to get the exact number, how can I get a good approximation ?
Any help is appreciated (links to similar cases, books, etc ).
The inner loop iterates over the array of all primes below sqrt(i).
So you have to calculate the number of elements in that array. In the case of an array of primes, you have to use approximations for π(i), the number of primes below i.
You can approximate them by x/ln(x) or (much better) by li(x). More details here.
For analysis the x/ln(x) would be easier.
In total you get (assuming n = 2k+1)
T(n) = T(n-2) + O(sqrt(n)/( (1/2)⋅ln(n) )) = O( Σi = 1,...,k 2⋅sqrt(2⋅i+1)/ln(2⋅i+1) )
You get the recursive formula from the inner for loop, that iterates over the array of primes lower than sqrt(n) (approximated by sqrt(n)/( (1/2)⋅ln(n) )), and the work you have to do to come this far, represented by T(n-2).
Maybe you can simplify this more. I don't want to take the fun from you :)
Hint: Maybe you can use an integral to get an approximation of the sum. But I think there is no "nice" way to write it down.
If you forget about the 1/ln(i)-part, you can see
T(n) ∈ o(n3/2) and T(n) ∈ ω(n). Maybe you can do better.
As #vib mentioned, you can get a tighter upper bound O(n3/2/ln(n)). But notice that sqrt(n)/ln(n) is only an approximation for the number of primes lower than sqrt(n). Thus you get better bounds with better approximations. Since this approximations do not provide the exact value of π(n), we cannot say that this algorithm runs in Θ(n3/2/ln(n)).
I am expressing the algorithms in pseudo code. I'm just wondering if my design works just as well as the original one displayed below. The algorithm is supposed to compute the sum of n odd positive integers.
This is how the algorithm should look:
procedure sumofodds(n:positive integer)
if n = 1
return 1
else
return sumofodds(n-1) + (2n-1)
This is how i designed my algorithm:
procedure odd(n: positive integer)
if n = 1
return 1
if n % 2 > 0
return n + odd(n-1) // this means n is odd
if n % 2 = 0
return 0 + odd(n-1) // this means its even
Your algorithm is not the same as the original.
The original computes the sum of the first n odd numbers.
Your algorithm computes the sum of all the odd numbers in the range 1..n.
So for an input of n=3, the first algorithm will compute 1+3+5, while your algorithm will compute 1+3.
(If you want a quicker way, then the formula n*n computes the sum of the first n odd numbers)
One small improvement that might help is defining it with tail recursion. Tail recursion happens when the very last thing to execute is the recursive call. To make this tail recursive, use a helper method and pass the running sum as a parameter. I'm pretty sure the pseudo code below is tail recursive since, regardless of the result of the (if odd) check, the final step is the recursive call (the math happens before the recursive call).
procedure SumOdds(n)
return SumOddsHelper(n, 0)
procedure SumOddsHelper(n, sum)
if n = 1 return 1
if n is odd return SumOddsHelper(n-1, sum + n)
else return SumOddsHelper(n-1, sum)
Let me suggest that you implement your idea in Python. You may be surprised to see that the working code is very similar to pseudocode.
This is the original algorithm:
def sum_of_n_odds(n):
if n == 1:
return 1
else:
return sum_of_n_odds(n-1) + (2*n-1)
And this is the one you wrote:
def sum_of_odds_up_to_n(n):
if n == 1:
return 1
if n % 2 > 0: # this means n is odd
return n + sum_of_odds_up_to_n(n-1)
if n % 2 == 0: # this means it's even
return 0 + sum_of_odds_up_to_n(n-1)
These two algorithms compute different things. Calling sum_of_n_odds(10) yields the same result as calling sum_of_odds_up_to_n(19) or sum_of_odds_up_to_n(20). In general, sum_of_odds_up_to_n(n) is equivalent to sum_of_n_odds((n+1)//2), where // means integer division.
If you're interested in making your implementation a little more efficient, I suggest that you omit the final if condition, where n % 2 == 0. An integer is either odd or even, so if it isn't odd, it must be even.
You can get another performance gain by making the recursive call sum_of_odds_up_to(n-2) when n is odd. Currently you are wasting half of your function calls on even numbers.
With these two improvements, the code becomes:
def sum_of_odds_up_to_n(n):
if n <= 0:
return 0
if n % 2 == 0:
return sum_of_odds_up_to_n(n-1)
return n + sum_of_odds_up_to_n(n-2)
And this is the tail-recursive version:
def sum_of_odds_up_to_n(n, partial=0):
if n <= 0:
return partial
if n % 2 == 0:
return sum_of_odds_up_to_n(n-1, partial)
return sum_of_odds_up_to_n(n-2, partial+n)
You should not expect performance gains from the above because Python does not optimize for tail recursion. However, you can rewrite tail recursion as iteration, which will run faster because it doesn't spend time allocating a stack frame for each recursive call:
def sum_of_odds_up_to_n(n):
partial = 0
if n % 2 == 0:
n -= 1
while n > 0:
partial += n
n -= 2
return partial
The fastest implementation of all relies on mathematical insight. Consider the sum:
1 + 3 + 5 + ... + (n-4) + (n-2) + n
Observe that you can pair the first element with the last element, the second element with the second last element, the third element with the third last element, and so on:
(1 + n) + (3 + n-2) + (5 + n-4) + ...
It is easy to see that this is equal to:
(n + 1) + (n + 1) + (n + 1) + ...
How many terms (n + 1) are there? Since we're pairing up two terms at a time from the original sequence, there are half as many terms in the (n + 1) sequence.
You can check for yourself that the original sequence has (n + 1) / 2 terms. (Hint: see what you get if you add 1 to every term.)
The new sequence has half as many terms as that, or (n + 1) / 4. And each term in the sequence is (n + 1), so the sum of the whole sequence is:
(n + 1) * (n + 1) / 4
The resulting Python program is this:
def sum_of_odds_up_to_n(n):
if n <= 0:
return 0
if n % 2 == 0:
n -= 1
return (n+1)*(n+1)//4
nSo we were taught about recurrence relations a day ago and we were given some codes to practice with:
int pow(int base, int n){
if (n == 0)
return 1;
else if (n == 1)
return base;
else if(n%2 == 0)
return pow(base*base, n/2);
else
return base * pow(base*base, n/2);
}
The farthest I've got to getting its closed form is T(n) = T(n/2^k) + 7k.
I'm not sure how to go any further as the examples given to us were simple and does not help that much.
How do you actually solve for the recurrence relation of this code?
Let us count only the multiplies in a call to pow, denoted as M(N), assuming they dominate the cost (a nowadays strongly invalid assumption).
By inspection of the code we see that:
M(0) = 0 (no multiply for N=0)
M(1) = 0 (no multiply for N=1)
M(N), N>1, N even = M(N/2) + 1 (for even N, recursive call after one multiply)
M(N), N>1, N odd = M(N/2) + 2 (for odd N, recursive call after one multiply, followed by a second multiply).
This recurrence is a bit complicated by the fact that it handles differently the even and odd integers. We will work around this by considering sequences of even or odd numbers only.
Let us first handle the case of N being a power of 2. If we iterate the formula, we get M(N) = M(N/2) + 1 = M(N/4) + 2 = M(N/8) + 3 = M(N/16) + 4. We easily spot the pattern M(N) = M(N/2^k) + k, so that the solution M(2^n) = n follows. We can write this as M(N) = Lg(N) (base 2 logarithm).
Similarly, N = 2^n-1 will always yield odd numbers after divisions by 2. We have M(2^n-1) = M(2^(n-1)-1) + 2 = M(2^(n-2)-1) + 4... = 2(n-1). Or M(N) = 2 Lg(N+1) - 2.
The exact solution for general N can be fairly involved but we can see that Lg(N) <= M(N) <= 2 Lg(N+1) - 2. Thus M(N) is O(Log(N)).
I am trying to analysis time complexity of below function. This function is used to check if a string is made of other strings.
set<string> s; // s has been initialized and stores all the strings
bool fun(string word) {
int len = word.size();
// something else that can also return true or false with O(1) complexity
for (int i=1; i<=len; ++i) {
string prefix = word.substr(0,i);
string suffix = word.substr(i);
if (prefix in s && fun(suffix))
return true;
else
return false;
}
}
I think the time complexity is O(n) where n is the length of word (am I right?). But as the recursion is inside the loop, I don't know how to prove it.
Edit:
This code is not a correct C++ code (e.g., prefix in s). I just show the idea of this function, and want to know how to analysis its time complexity
The way to analyze this is by developing a recursion relationship based on the length of the input and the (unknown) probability that a prefix is in s. Let's assume that the probability of a prefix being in s is given by some function pr(L) of the length L of the prefix. Let the complexity (number of operations) be given by T(len).
If len == 0 (word is the empty string), then T = 1. (The function is missing a final return statement after the loop, but we're assuming that the actual code is only a sketch of the idea, not what's actually executing).
For each loop iteration, denote the loop body complexity by T(len; i). If the prefix is not in s, then the body has constant complexity (T(len; i) = 1). This event has probability 1 - pr(i).
If the prefix is in s, then the function returns true or false according to the recursive call to fun(suffix), which has complexity T(len - i). This event has probability pr(i).
So for each value of i, the loop body complexity is:
T(len; i) = 1 * (1 - pr(i)) + T(len - i) * pr(i)
Finally (and this depends on the intended logic, not the posted code), we have
T(len) = sum i=1...len(T(len; i))
For simplicity, let's treat pr(i) as a constant function with value 0.5. Then the recursive relationship for T(len) is (up to a constant factor, which is unimportant for O() calculations):
T(len) = sum i=1...len(1 + T(len - i)) = len + sum i=0...len-1(T(i))
As noted above, the boundary condition is T(0) = 1. This can be solved by standard recursive function methods. Let's look at the first few terms:
len T(len)
0 1
1 1 + 1 = 2
2 2 + 2 + 1 = 5
3 3 + (4 + 2 + 1) = 11
4 4 + (11 + 5 + 2 + 1) = 23
5 5 + (23 + 11 + 5 + 2 + 1) = 47
The pattern is clearly T(len) = 2 * T(len - 1) + 1. This corresponds to exponential complexity:
T(n) = O(2n)
Of course, this result depends on the assumption we made about pr(i). (For instance, if pr(i) = 0 for all i, then T(n) = O(1). There would also be non-exponential growth if pr(i) had a maximum prefix length—pr(i) = 0 for all i > M for some M.) The assumption that pr(i) is independent of i is probably unrealistic, but this really depends on how s is populated.
Assuming that you've fixed the bugs others have noted, then the i values are the places that the string is being split (each i is the leftmost splitpoint, and then you recurse on everything to the right of i). This means that if you were to unwind the recursion, you are looking at up to n-1 different split points, and asking if each substring is a valid word. Things are ok if the beginning of word doesn't have a lot of elements from your set, since then you can skip the recursion. But in the worst case, prefix in s is always true, and you try every possible subset of the n-1 split points. This gives 2^{n-1} different splitting sets, multiplied by the length of each such set.