Good morning, I am studying algorithms and the way to calculate complexity when doing recursive calls, but I cannot find a reference on how a level limit in recursive calls can affect the complexity calculation. For instance this code:
countFamilyMembers(int level,....,int count){
if(noOperationCondition) { // for example no need to process this item because business rules like member already counted
return count;
} else if(level >= MAX_LEVEL) { // Level validation, we want just to look up to certain level
return ++count //last level to see then no more recurrence.
} else {
for (...each memberRelatives...) { //can be a database lookup for relatives to explore
count = countFamilyMembers(++level,...,++count);
}
return count;
}
}
I think this is O(2^n) because the recursive call in the loop. However, I have two main questions:
1. What happens if the loop values is not related to the original input at all? can that be considered "n" as well?
2. The level validation is for sure cutting limiting the recursive calls, how do this affect the complexity calculation?
Thanks for the clarifications. So we'll take n as some "best metric" on the number of relatives; this is also known as the "fan-out" in some paradigms.
Thus, you'll have 1 person at level 0, n at level 1, n^2 at level 2, and so on. A rough estimate of the return value ... and the number of operations (node visits, increments, etc.) is the sum of n^level for level ranging 0 to MAX_LEVEL. The dominant term is the highest exponent, n^MAX_LEVEL.
With the given information, I believe that's your answer: O(n^^MAX_LEVEL), a.k.a. polynomial time.
Note that, if you happen to be given a value for n, even an upper bound for n, then this becomes a constant, and the complexity is O(1).
Related
I'm trying to solve question 11.1 in Elements of Programming Interviews (EPI) in Java: Search a Sorted Array for First Occurrence of K.
The problem description from the book:
Write a method that takes a sorted array and a key and returns the index of the first occurrence of that key in the array.
The solution they provide in the book is a modified binary search algorithm that runs in O(logn) time. I wrote my own algorithm also based on a modified binary search algorithm with a slight difference - it uses recursion. The problem is I don't know how to determine the time complexity of my algorithm - my best guess is that it will run in O(logn) time because each time the function is called it reduces the size of the candidate values by half. I've tested my algorithm against the 314 EPI test cases that are provided by the EPI Judge so I know it works, I just don't know the time complexity - here is the code:
public static int searchFirstOfKUtility(List<Integer> A, int k, int Lower, int Upper, Integer Index)
{
while(Lower<=Upper){
int M = Lower + (Upper-Lower)/2;
if(A.get(M)<k)
Lower = M+1;
else if(A.get(M) == k){
Index = M;
if(Lower!=Upper)
Index = searchFirstOfKUtility(A, k, Lower, M-1, Index);
return Index;
}
else
Upper=M-1;
}
return Index;
}
Here is the code that the tests cases call to exercise my function:
public static int searchFirstOfK(List<Integer> A, int k) {
Integer foundKey = -1;
return searchFirstOfKUtility(A, k, 0, A.size()-1, foundKey);
}
So, can anyone tell me what the time complexity of my algorithm would be?
Assuming that passing arguments is O(1) instead of O(n), performance is O(log(n)).
The usual theoretical approach for analyzing recursion is calling the Master Theorem. It is to say that if the performance of a recursive algorithm follows a relation:
T(n) = a T(n/b) + f(n)
then there are 3 cases. In plain English they correspond to:
Performance is dominated by all the calls at the bottom of the recursion, so is proportional to how many of those there are.
Performance is equal between each level of recursion, and so is proportional to how many levels of recursion there are, times the cost of any layer of recursion.
Performance is dominated by the work done in the very first call, and so is proportional to f(n).
You are in case 2. Each recursive call costs the same, and so performance is dominated by the fact that there are O(log(n)) levels of recursion times the cost of each level. Assuming that passing a fixed number of arguments is O(1), that will indeed be O(log(n)).
Note that this assumption is true for Java because you don't make a complete copy of the array before passing it. But it is important to be aware that it is not true in all languages. For example I recently did a bunch of work in PL/pgSQL, and there arrays are passed by value. Meaning that your algorithm would have been O(n log(n)).
Most recursive functions I have seen being asked about (e.g. Fibonacci or Hanoi) have had O(1) returns, but what would the time complexity be if it wasn't O(1) but O(n) instead?
For example, a recursive Fibonacci with O(n) base case:
class fibonacci {
static int fib(int n) {
if (n <= 1)
for (int i=0;i<n;i++) {
// something
}
return n;
return fib(n-1) + fib(n-2);
}
public static void main (String args[])
{
int n = 9;
System.out.println(fib(n));
}
}
The base case for the function that you’ve written here actually still has time complexity O(1). The reason for this is that if the base case triggers here, then n ≤ 1, so the for loop here will run at most once.
Because so many base cases trigger when the input size is small, it’s comparatively rare to get a base case whose runtime is, say, O(n) when the input to the algorithm has size n. This would mean that the base case is independent of the array size, which can happen but is somewhat unusual.
A more common occurrence - albeit one I think is still pretty uncommon - would be for a recursive function to have two different parameters to it (say, n and k), where the recursion reduces n but leaves k unmodified. For example, imagine taking the code you have here and replacing the for loop on n in the base case with a for loop on k in the base case. What happens then?
This turns out to be an interesting question. In the case of this particular problem, it means that the total work done will be given by O(k) times the number of base cases triggered, plus O(1) times the number of non-base-case recursive calls. For the Fibonacci recursion, the number of base cases triggered computing Fn is Fn+1 and there are (Fn+1 - 1) non-base-case calls, so the overall runtime would be Θ(k Fn+1 + Fn+1) = Θ(k φn). For the Towers of Hanoi, you’d similarly see a scaling effect where the overall runtime would be Θ(k 2n). But for other recursive functions the runtime might vary in different ways, depending on how those functions were structured.
Hope this helps!
public static void fun3(int i)
{
if(i<10)
{
fun3(i+1);
fun3(i+2);
System.out.println(i);
}
}
Recurrence for this code is: T(n)=T(n+1)+T(n+2)+O(9)
the problematic thing is if condition here if(i<10). Where i can grow to infinity towards the negative side (e.g -1000, or -287131287238238 etc). I need its time complexity using recurrence tree. How to calculate height of the tree??
The height of the tree is abs(10 - i) for i < 10 and 0 for i >=10 (abs indicates absolute value). At each level you have twice more branches than the level before. When you sum them up you will end up with a time complexity of O(2^abs(10-i)) or O(2^abs(i)) for i < 10 and O(1) for i>=10. The analysis is similar to fibonacci sequence for which you can find many reference like this.
Since you have no calls for i>=10, and for all i=0 .. 9 there are finite number of calls you have an upper bound for the total number of calls for all i.
Therefore the answer is O(1).
The constant factor is the maximum of calls for the values i=0...9. The code indicates that the most calls are made for i=0 (or if 0 is not allowed, i=1)
Suppose we have a recursive function which only terminates if a randomly generated parameter meets some condition:
e.g:
{
define (some-recursive-function)
x = (random in range of 1 to 100000);
if (x == 10)
{
return "this function has terminated";
}
else
{
(some-recursive-function)
}
}
I understand that for infinite loops, there would not be an complexity defined. What about some function that definitely terminates, but after an unknown amount of time?
Finding the average time complexity for this would be fine. How would one go about finding the worse case time complexity, if one exists?
Thank you in advance!
EDIT: As several have pointed out, I've completely missed the fact that there is no input to this function. Suppose instead, we have:
{define (some-recursive-function n)
x = (random in range of 1 to n);
if (x == 10)
{
return "this function has terminated";
}
else
{
(some-recursive-function)
}
}
Would this change anything?
If there is no function of n which bounds the runtime of the function from above, then there just isn't an upper bound on the runtime. There could be an lower bound on the runtime, depending on the case. We can also speak about the expected runtime, and even put bounds on the expected runtime, but that is distinct from, on the one hand, bounds on the average case and, on the other hand, bounds on the runtime itself.
As it's currently written, there are no bounds at all when n is under 10: the function just doesn't terminate in any event. For n >= 10, there is still no upper bound on any of the cases - it can take arbitrarily long to finish - but the lower bound in any case is as low as linear (you must at least read the value of n, which consists of N = ceiling(log n) bits; your method of choosing a random number no greater than n may require additional time and/or space). The case behavior here is fairly uninteresting.
If we consider the expected runtime of the function in terms of the value (not length) of the input, we observe that there is a 1/n chance that any particular invocation picks the right random number (again, for n >= 10); we recognize that the number of times we need to try to get one is given by a geometric distribution and that the expectation is 1/(1/n) = n. So, the expected recursion depth is a linear function of the value of the input, n, and therefore an exponential function of the input size, N = log n. We recover an exact expression for the expectation; the upper and lower bounds are therefore both linear as well, and this covers all cases (best, worst, average, etc.) I say recursion depth since the runtime will also have an additional factor of N = log n, or more, owing to the observation in the preceding paragraph.
You need to know that there are "simple" formulas to calculate the complexity of a recursive algorithm, using of course recurrence.
In this case we obviously need to know what is that recursive algorithm, because in the best case, it is O(1) (temporal complexity), but in the worst case, we need to add O(n) (having into account that numbers may repeat) to the complexity of the algorithm itself.
I'll put this question/answer for more facility:
Determining complexity for recursive functions (Big O notation)
In my algorithm and data structures class we were given a few recurrence relations either to solve or that we can see the complexity of an algorithm.
At first, I thought that the mere purpose of these relations is to jot down the complexity of a recursive divide-and-conquer algorithm. Then I came across a question in the MIT assignments, where one is asked to provide a recurrence relation for an iterative algorithm.
How would I actually come up with a recurrence relation myself, given some code? What are the necessary steps?
Is it actually correct that I can jot down any case i.e. worst, best, average case with such a relation?
Could possibly someone give a simple example on how a piece of code is turned into a recurrence relation?
Cheers,
Andrew
Okay, so in algorithm analysis, a recurrence relation is a function relating the amount of work needed to solve a problem of size n to that needed to solve smaller problems (this is closely related to its meaning in math).
For example, consider a Fibonacci function below:
Fib(a)
{
if(a==1 || a==0)
return 1;
return Fib(a-1) + Fib(a-2);
}
This does three operations (comparison, comparison, addition), and also calls itself recursively. So the recurrence relation is T(n) = 3 + T(n-1) + T(n-2). To solve this, you would use the iterative method: start expanding the terms until you find the pattern. For this example, you would expand T(n-1) to get T(n) = 6 + 2*T(n-2) + T(n-3). Then expand T(n-2) to get T(n) = 12 + 3*T(n-3) + 2*T(n-4). One more time, expand T(n-3) to get T(n) = 21 + 5*T(n-4) + 3*T(n-5). Notice that the coefficient of the first T term is following the Fibonacci numbers, and the constant term is the sum of them times three: looking it up, that is 3*(Fib(n+2)-1). More importantly, we notice that the sequence increases exponentially; that is, the complexity of the algorithm is O(2n).
Then consider this function for merge sort:
Merge(ary)
{
ary_start = Merge(ary[0:n/2]);
ary_end = Merge(ary[n/2:n]);
return MergeArrays(ary_start, ary_end);
}
This function calls itself on half the input twice, then merges the two halves (using O(n) work). That is, T(n) = T(n/2) + T(n/2) + O(n). To solve recurrence relations of this type, you should use the Master Theorem. By this theorem, this expands to T(n) = O(n log n).
Finally, consider this function to calculate Fibonacci:
Fib2(n)
{
two = one = 1;
for(i from 2 to n)
{
temp = two + one;
one = two;
two = temp;
}
return two;
}
This function calls itself no times, and it iterates O(n) times. Therefore, its recurrence relation is T(n) = O(n). This is the case you asked about. It is a special case of recurrence relations with no recurrence; therefore, it is very easy to solve.
To find the running time of an algorithm we need to firstly able to write an expression for the algorithm and that expression tells the running time for each step. So you need to walk through each of the steps of an algorithm to find the expression.
For example, suppose we defined a predicate, isSorted, which would take as input an array a and the size, n, of the array and would return true if and only if the array was sorted in increasing order.
bool isSorted(int *a, int n) {
if (n == 1)
return true; // a 1-element array is always sorted
for (int i = 0; i < n-1; i++) {
if (a[i] > a[i+1]) // found two adjacent elements out of order
return false;
}
return true; // everything's in sorted order
}
Clearly, the size of the input here will simply be n, the size of the array. How many steps will be performed in the worst case, for input n?
The first if statement counts as 1 step
The for loop will execute n−1 times in the worst case (assuming the internal test doesn't kick us out), for a total time of n−1 for the loop test and the increment of the index.
Inside the loop, there's another if statement which will be executed once per iteration for a total of n−1 time, at worst.
The last return will be executed once.
So, in the worst case, we'll have done 1+(n−1)+(n−1)+1
computations, for a total run time T(n)≤1+(n−1)+(n−1)+1=2n and so we have the timing function T(n)=O(n).
So in brief what we have done is-->>
1.For a parameter 'n' which gives the size of the input we assume that each simple statements that are executed once will take constant time,for simplicity assume one
2.The iterative statements like loops and inside body will take variable time depending upon the input.
Which has solution T(n)=O(n), just as with the non-recursive version, as it happens.
3.So your task is to go step by step and write down the function in terms of n to calulate the time complexity
For recursive algorithms, you do the same thing, only this time you add the time taken by each recursive call, expressed as a function of the time it takes on its input.
For example, let's rewrite, isSorted as a recursive algorithm:
bool isSorted(int *a, int n) {
if (n == 1)
return true;
if (a[n-2] > a[n-1]) // are the last two elements out of order?
return false;
else
return isSorted(a, n-1); // is the initial part of the array sorted?
}
In this case we still walk through the algorithm, counting: 1 step for the first if plus 1 step for the second if, plus the time isSorted will take on an input of size n−1, which will be T(n−1), giving a recurrence relation
T(n)≤1+1+T(n−1)=T(n−1)+O(1)
Which has solution T(n)=O(n), just as with the non-recursive version, as it happens.
Simple Enough!! Practice More to write the recurrence relation of various algorithms keeping in mind how much time each step will be executed in algorithm