T(n)=4T(n/4) + log4n is the recurrence provided and I was wondering how to write a pseudocode based on it.
This says: make four recursive calls, and have each recursive call do an amount of work equal to log 4n (rounded down).
Note that log 4n = log 4 + log n = 2 + log n.
Something like this gets pretty close::
function foo(array[1...n])
if n <= 1 then return 1
c = 1
while c < n do
c *= 4
return c + foo(arr[1...n/4]) + foo(arr[n/4+1...n/2]) + foo(arr[n/2+1...3n/4]) + foo(arr[3n/4+1...n]
The recurrence here is T(n) = 4T(n/4) + log 4n + 15, if I counted correctly, and making some assumptions about operations taking the same time to run.
We can bring that 15 down by turning the algorithm more and more silly. Store n/4 in a variable, call foo on the first 1/4 of arr four times, but only return c; this brings the 15 down to 1, I think. To get rid of that 1 operation, start c at 4 instead, killing one loop iteration (2 ops) and add one op back at the end, like return c + 1 instead of return c.
Related
I have a guess,but I am not sure.
This is the problem:
for (a = 1 ; a<n ; a= 2*a) do {
for (b=n; b > 0; b=b-a) do {
}
}
This is my first question on stackoverflow, so I hope the formatting was right.
Thank you very much.
There's a useful maxim for reasoning about big-O notation that goes like this:
When in doubt, work inside out!
More specifically, if you're trying to figure out the complexity of a loop nest, start with the innermost loop and keep replacing it with a simpler statement summarizing the amount of work done.
In your case, you have these loop:
for (a = 1 ; a<n ; a= 2*a) do {
for (b=n; b > 0; b=b-a) do {
}
}
Let's begin with the inner loop:
for (b=n; b > 0; b=b-a) do {
}
How many times will this loop run? Well, we begin with b equal to n, and on each iteration b decreases by a. That means that the number of iterations of this loop is roughly n / a, so the complexity of this loop is Θ(n / a). We can therefore replace the inner loop with something to the effect of "do Θ(n / a) work" to get this simpler structure:
for (a = 1 ; a<n ; a= 2*a) do {
do Θ(n / a) work;
}
Now, let's think about how much work this loop does. Since the amount of work done inside the loop depends on the value of a, we're not going to multiply the number of iterations by the work done per iteration, since the work done per iteration isn't a constant. Instead, we'll add up how much work is done on each iteration of the loop.
Notice that the value of a increases as 1, 2, 4, 8, 16, 32, ..., until we overshoot n. Plugging those values into the work done inside the loop gives a runtime of
Θ(n / 1 + n / 2 + n / 4 + n / 8 + n / 16 + ... )
= n Θ(1/1 + 1/2 + 1/4 + 1/8 + 1/16 + ...)
You might recognize that the sum 1/1 + 1/2 + 1/4 + 1/8 + ... happens to converge to 2. (Do you see why?) As a result, we have that the runtime of this code is
n Θ(2)
= Θ(n).
So the overall work done here is Θ(n).
The main techniques we used to determine this were the following:
Work from the inside out, replacing each loop with a summary of how much work it does.
If a loop counter increases by k on each iteration and stops at n, the loop runs for a total of Θ(n / k) iterations. (This applies equally well if we run it backwards and start at n, decreasing by k.)
The sum of the geometric series 1 + 1/2 + 1/4 + 1/8 + 1/16 + ... out to infinity is 2.
If the work done by a loop is constant per iteration, just multiply the work per iteration by the number of iterations. If the work done per iteration isn't, it's often easier to sum up the work across all loop iterations and then try to simplify the sum.
Hope this helps!
Why is the runtime closer to O(1.6^N) and not O(2^N)? I think it has something to do with the call stack, but it's still a little unclear to me.
int Fibonacci(int n)
{
if (n <= 1)
return n;
else
return Fibonacci(n - 1) + Fibonacci(n - 2);
}
For starters, remember that big-O notation always provides an upper bound. That is, if a function is O(n), it's also O(n2), O(n3), O(2n), etc. So in that sense, you are not incorrect if you say that the runtime is O(2n). You just don't have a tight bound.
To see where the tight bound of Θ(φn) comes from, it might help to look at how many recursive calls end up getting made when evaluating Fibonacci(n). Notice that
Fibonacci(0) requires one call, the call to Fibonacci(0).
Fibonacci(1) requires one call, the call to Fibonacci(1).
Fibonacci(2) requires three calls: one to Fibonacci(2), and one each to Fibonacci(0) and Fibonacci(1).
Fibonacci(3) requires five calls: one to Fibonacci(3), then the three calls generated by invoking Fibonacci(2) and the one call generated by invoking Fibonacci(1).
Fibonacci(4) requires nine calls: one to Fibonacci(4), then five from the call to Fibonacci(3) and three from the call to Fibonacci(2).
More generally, the pattern seems to be that if you're calling Fibonacci(n) for some n ≥ 2, that the number of calls made is one (for the call itself), plus the number of calls needed to evaluate Fibonacci(n-1) and Fibonacci(n-2). If we let Ln denote the number of calls made, this means that we have that
L0 = L1 = 1
Ln+2 = 1 + Ln + Ln+1.
So now the question is how fast this sequence grows. Evaluating the first few terms gives us
1, 1, 3, 5, 9, 15, 25, 41, ...
which definitely gets bigger and bigger, but it's not clear how much bigger that is.
Something you might notice here is that Ln kinda sorta ish looks like the Fibonacci numbers. That is, it's defined in terms of the sum of the two previous terms, but it has an extra +1 term. So maybe we might want to look at the difference between Ln and Fn, since that might show us how much "faster" the L series grows. You might notice that the first two values of the L series are 1, 1 and the first two values of the Fibonacci series are 0, 1, so we'll shift things over by one term to make things line up a bit more nicely:
L(n) 1 1 3 5 9 15 25 41
F(n+1) 1 1 2 3 5 8 13 21
Diff: 0 0 1 2 4 7 12 20
And wait, hold on a second. What happens if we add one to each term of the difference? That gives us
L(n) 1 1 3 5 9 15 25 41
F(n+1) 1 1 2 3 5 8 13 21
Diff+1 1 1 2 3 5 8 13 21
Whoa! Looks like Ln - Fn+1 + 1 = Fn+1. Rearranging, we see that
Ln = 2Fn+1 - 1.
Wow! So the actual number of calls made by the Fibonacci recursive function is very closely related to the actual value returned. So we could say that the runtime of the Fibonacci function is Θ(Fn+1) and we'd be correct.
But now the question is where φ comes in. There's a lovely mathematical result called Binet's formula that says that Fn = Θ(φn). There are many ways to prove this, but they all essentially boil down to the observation that
the Fibonacci numbers seem to grow exponentially quickly;
if they grow exponentially with a base of x, then Fn+2 = Fn + Fn+1 can be rewritten as x2 = x + 1; and
φ is a solution to x2 = x + 1.
From this, we can see that since the runtime of Fibonacci is Θ(Fn+1), then the runtime is also Θ(φn+1) = Θ(φn).
The number φ = (1+sqrt(5))/2 is characterized by the two following properties:
φ >= 1
φ2 = φ + 1.
Multiplying the second equation by φ{n-1} we get
φn+1 = φn + φn-1
Since f(0) = 0, f(1) = 1 and f(n+1) = f(n) + f(n-1), using 1 and 3,
it is easy to see by induction in n that f(n) <= φn
Thus f(n) is O(φn).
A similar inductive argument shows that
f(n) >= φn-3 = φ-3φn (n >= 1)
thus f(n) = Θ(φn).
I am trying to calculate the number of steps executed for the following nested loop specially for asymptotic growth. Based on the number of steps I will derive the Big O for this algorithm.
def get_multiples(list):
multiple = []
for o in list:
for i in list:
multiple.append(o*i)
return multiple
The way I have calculated is as follows (list consists of large number of elements = "n"):
Assignment statement (no. of steps = 1):
multiple = []
Nested Loops:
for o in list:
for i in list:
multiple.append(o*i)
In the outer loop the variable o is assigned n times. Each time the outer loop executes, first the variable i is assigned n times, then the variables are multiplied n times and finally the list is appended n times. Therefore the no. of steps = n*(n+n+n) = 3n2
Return statement (No. of steps = 1):
return multiple
Therefore the total no. of steps = 3n2 + 2
However the correct answer is 3n2 + n +2. Apparently the execution of the outer loop takes additional n steps which is not required for the inner loop.
Can somebody explain to me what did I miss ?
It does not make a difference to complexity since it will still be O(n2)
I think that the correct way to calculate the nested loop is as follows:
The number o is assigned n times.
the number i is assigned n2 times, o*i is calculated n2 times, the append function is called n2 times.
therefore n + n2 + n2 + n2 = 3n2 + n
add it to the rest and you get 3n2 + n + 2
def get_multiples(list):
multiple = [] // 1 STEP
for o in list: // Executed n times, so n STEPS
for i in list: // Executed n times for each value of o, so n*n STEPS
multiple.append(o*i) // 1 STEP to multiply and 1 STEP to append, repeated for each pair of (o, i), so 2*n*n STEPS
return multiple // 1 STEP
Adding the above: 1 + n + n2 + 2n2 + 1 = 3n2 + n + 2
The Fizz-Buzz function (in pseudocode) takes any positive integer n. I'm especially curious about the algebraic breakdown of the cost and time required of the if-else statement. I know its worst case running time is O(n).
Fizz-Bizz(n)
for i = 1 to n
if (n % 3 == 0)
print "fizz"
if (n % 5 == 0)
print "buzz"
if (n % 3 != 0 and n % 5 != 0)
print n
Example of breakdown of another algorithm:
The time complexity is O(n) because the if statement has no real effect on that. The complexity of the if statement is constant over a large enough dataset.
The if statement may actually do a different amount of work in iterations where you have a multiple of three or five but, the amount of extra work per loop iteration is not dependent on n. In fact, it averages out to a constant as n becomes bigger.
And, as an aside, I think that code may be wrong. At multiples of fifteen, it should print both fizz and buzz.
If you want to do it to the level in your edit (the added breakdown), you simply need to assign an arbitrary cost ci to each statement (and this cost is constant for a single execution of that statement) then figure out how many times each statement is run.
For example, the first if sequence runs n times, the print "fizz" runs for one-third of those, n/3. So you end up with something like this table:
cost times
Fizz-Buzz(n)
for i = 1 to n c1 n
if (n % 3 == 0) c2 n
print "fizz" c3 n / 3 [call this a]
else
if (n % 5 == 0) c4 n - a
print "buzz" c5 (n - a) / 5 [call this b]
else
print n c6 n - a - b
Add up all of those as per your example (substituting the n-equations for a and b) and, in the end, you'll still end up with something dependent on n, hence an O(n) algorithm. It'll look something like:
c1*n + c2*n + c3*n/3 + c4*(n-a) + c5*(n-a)/5 + c6*(n-a-b)
= c1*n + c2*n + (c3/3)*n + c4*(n-n/3) + (c5/5)*(n-n/3) + c6*(n-n/3-(n-n/3)/5)
= c1*n + c2*n + (c3/3)*n + c4*(2/3)*n + (c5/5)*(2/3)*n + c6*(n-n/3-(n-n/3)/5)
= c1*n + c2*n + (c3/3)*n + (c4*2/3)*n + (c5*2/15)*n + c6*(n*8/15)
= c1*n + c2*n + (c3/3)*n + (c4*2/3)*n + (c5*2/15)*n + (c6*8/15)*n
/ 1 2 2 8 \
= ( c1 + c2 + - c3 + - c4 + -- c5 + -- c6 ) * n
\ 3 3 15 15 /
All those values inside parentheses are in fact constants (since they're multiples of constants) so the whole thing is a constant multiplier of n.
Now, if you find a minor mistake in those equations above, I wouldn't be too surprised - I haven't done this level of math for quite a few years, and I may well have thrown in a furphy in case this is for homework and you try copying it verbatim :-)
But the only mistake you're likely to find is the value of the constant multiplier itself. It will still be a constant multiplier of some description, I'll guarantee that.
I am trying to analysis time complexity of below function. This function is used to check if a string is made of other strings.
set<string> s; // s has been initialized and stores all the strings
bool fun(string word) {
int len = word.size();
// something else that can also return true or false with O(1) complexity
for (int i=1; i<=len; ++i) {
string prefix = word.substr(0,i);
string suffix = word.substr(i);
if (prefix in s && fun(suffix))
return true;
else
return false;
}
}
I think the time complexity is O(n) where n is the length of word (am I right?). But as the recursion is inside the loop, I don't know how to prove it.
Edit:
This code is not a correct C++ code (e.g., prefix in s). I just show the idea of this function, and want to know how to analysis its time complexity
The way to analyze this is by developing a recursion relationship based on the length of the input and the (unknown) probability that a prefix is in s. Let's assume that the probability of a prefix being in s is given by some function pr(L) of the length L of the prefix. Let the complexity (number of operations) be given by T(len).
If len == 0 (word is the empty string), then T = 1. (The function is missing a final return statement after the loop, but we're assuming that the actual code is only a sketch of the idea, not what's actually executing).
For each loop iteration, denote the loop body complexity by T(len; i). If the prefix is not in s, then the body has constant complexity (T(len; i) = 1). This event has probability 1 - pr(i).
If the prefix is in s, then the function returns true or false according to the recursive call to fun(suffix), which has complexity T(len - i). This event has probability pr(i).
So for each value of i, the loop body complexity is:
T(len; i) = 1 * (1 - pr(i)) + T(len - i) * pr(i)
Finally (and this depends on the intended logic, not the posted code), we have
T(len) = sum i=1...len(T(len; i))
For simplicity, let's treat pr(i) as a constant function with value 0.5. Then the recursive relationship for T(len) is (up to a constant factor, which is unimportant for O() calculations):
T(len) = sum i=1...len(1 + T(len - i)) = len + sum i=0...len-1(T(i))
As noted above, the boundary condition is T(0) = 1. This can be solved by standard recursive function methods. Let's look at the first few terms:
len T(len)
0 1
1 1 + 1 = 2
2 2 + 2 + 1 = 5
3 3 + (4 + 2 + 1) = 11
4 4 + (11 + 5 + 2 + 1) = 23
5 5 + (23 + 11 + 5 + 2 + 1) = 47
The pattern is clearly T(len) = 2 * T(len - 1) + 1. This corresponds to exponential complexity:
T(n) = O(2n)
Of course, this result depends on the assumption we made about pr(i). (For instance, if pr(i) = 0 for all i, then T(n) = O(1). There would also be non-exponential growth if pr(i) had a maximum prefix length—pr(i) = 0 for all i > M for some M.) The assumption that pr(i) is independent of i is probably unrealistic, but this really depends on how s is populated.
Assuming that you've fixed the bugs others have noted, then the i values are the places that the string is being split (each i is the leftmost splitpoint, and then you recurse on everything to the right of i). This means that if you were to unwind the recursion, you are looking at up to n-1 different split points, and asking if each substring is a valid word. Things are ok if the beginning of word doesn't have a lot of elements from your set, since then you can skip the recursion. But in the worst case, prefix in s is always true, and you try every possible subset of the n-1 split points. This gives 2^{n-1} different splitting sets, multiplied by the length of each such set.