I do not understand the O(2^n) complexity that the recursive function for the Longest Common Subsequence algorithm has.
Usually, I can tie this notation with the number of basic operations (in this case comparisons) of the algorithm, but this time it doesn't make sense in my mind.
For example, having two strings with the same length of 5. In the worst case the recursive function computes 251 comparisons. And 2^5 is not even close to that value.
Can anyone explain the algorithmic complexity of this function?
def lcs(xstr, ystr):
global nComp
if not xstr or not ystr:
return ""
x, xs, y, ys = xstr[0], xstr[1:], ystr[0], ystr[1:]
nComp += 1
#print("comparing",x,"with",y)
if x == y:
return x + lcs(xs, ys)
else:
return max(lcs(xstr, ys), lcs(xs, ystr), key=len)
To understand it properly look at the diagram carefully and follow the recursive top-to-down approach while reading the graph.
Here, xstr = "ABCD"
ystr = "BAEC"
lcs("ABCD", "BAEC") // Here x != y
/ \
lcs("BCD", "BAEC") <-- x==y --> lcs("ABCD", "AEC") x==y
| |
| |
lcs("CD", "AEC") <-- x!=y --> lcs("BCD", "EC")
/ \ / \
/ \ / \
/ \ / \
lcs("D","AEC") lcs("CD", "EC") lcs("BCD", "C")
/ \ / \ / \
lcs("", "AEC") lcs("D","EC") lcs("CD", "C") lcs("BCD","")
| \ / \ | / |
Return lcs("", "EC") lcs("D" ,"C") lcs("D", "") lcs("CD","") Return
/ \ / \ / \ / \
Return lcs("","C") lcs("D","") lcs("","") Return lcs("D","") Return
/ \ / \ / / \
Return lcs("","") Return lcs("", "") Return
| |
Return Return
NOTE: The proper way of representation of recursive call is usually done by using tree approach, but here i used the graph approach just to compress the tree so one can easy understand the recursive call in a go. And, of course it would be easy to me to represent.
Since, in the above diagram there are some redundant pairs like lcs("CD", "EC") which is the result of deletion of "A" from the "AEC" in lcs("CD", "AEC") and of "B" from the "BCD" in lcs("BCD", "EC"). As a result, these pairs will be called more than once while execution which increases the time complexity of the program.
As you could easily see that every pair generates two outcomes for its next level until it encounters any empty string or x==y. Therefore, if the length of the strings are n, m (considering the length of the xstr is n and ystr is m and we are considering the worst case scenario). Then, we will have number outcomes at the end of the order : 2n+m. (How? think)
Since, n+m is an integer number let's say N. Therefore, the time complexity of the algorithm : O(2N), which is not efficient for lager values of N.
Therefore, we prefer Dynamic-Programming Approach over the recursive Approach. It can reduce the time complexity to: O(n.m) => O(n2) , when n == m.
Even now, if you are getting hard time to understand the logic, i would suggest you to make a tree-like (not the graph which i have shown here) representation for xstr = "ABC" and ystr = "EF". I hope you will understand it.
Any doubt, comments most welcome.
O(2^n) means the run time is proportional to (2^n) for large enough n. It doesn't mean the number is bad, high, low, or anything specific for a small n, and it doesn't give a way to calculate the absolute run-time.
To get a feel for the implication, you should consider the run-times for n = 1000, 2000, 3000, or even 1 million, 2 million, etc.
In your example, assuming that for n=5 the algorithm takes a max of 251 iteration, then the O(n) prediction is that for n=50, it would take in the range of 2^(50)/2^(5)*251 = 2^45*251 = ~8.8E15 iterations.
Related
This is an unsolved problem from my past arbitrary-precision rational numbers C++ assignment.
For calculation, I used this expression from Wikipedia (a being the initial guess, r being its remainder):
I ended up, just by guessing from experiments, with this approach:
Use an integer square root function on the numerator/denominator, use that as the guess
Iterate the continued fraction until the binary length of the denominator was at least the target precision
This worked well enough to get me through the official tests, however, from my testing, the precision was too high (sometimes almost double) – i.e. the code was inefficient – and I had no proof it worked on any input (and hence no confidence in the code).
A simplified excerpt from the code (natural/rational store arbitrary length numbers, assume all operations return fractions in their simplest form):
rational sqrt(rational input, int precision) {
rational guess(isqrt(input.numerator), isqrt(input.denominator)); // a
rational remainder = input - power(guess, 2); // r
rational result = guess;
rational expansion;
while (result.denominator.size() <= precision) {
expansion = remainder / (2 * guess + expansion);
result = guess + expansion;
// Handle rational results
if (power(root, 2) == input) {
break;
}
}
return result;
}
Can it be done better? If so, how?
Square roots can easily and very accurately be calculated by the General Continued Fractions (GCF). Being general means it can have any positive number as the numerator in contrast to the Regular or Simple Continued Fractions (RCF) where the numerators are all 1s. In order to comprehend the answer as a whole, it is best to start from the beginning.
The method used to solve the square root of any positive number n by a GFC (a + x) whereas a being the integral and x being the continued fractional part, is;
n − a^2
√n = a + x ⇒ n = a^2 + 2ax + x^2 ⇒ n − a^2 = x(2a + x) ⇒ x = _______
2a + x
Right at this moment you have a GCF since x nicely gets placed at the denominator and once you replace x with it's definition you get an indefinitely extending definition of x. Regarding a, you are free to choose it among integers which are less than the √n. So if you want to find √11 then a can be chosen among 1, 2 or 3. However it's always better to chose the biggest one in order to be able to simplify the GCF into an RCF at the next stage.
Remember that x = (n − a^2) / (2a + x) and n = 11 and a = 3. Now if we write the first two terms then we may simplify the GCF to RCF with all numerators as 1.
2 2 divide both 1
x = _____ ⇒ _________ ⇒ numerator and ⇒ _________ = x
6 + x 6 + 2 denominator by 2 3 + 1
_____ _____
6 + x 6 + x
Accordingly our RCF for √11 is;
1 ___
√11 = 3 + x ⇒ 3 + _____________ = [3;3,6]
1
3 + _________
1
6 + _____
1
3 + _....
6
Notice the coefficient notation [3; 3, 6, 3, 6, ...] which in this particular case resembles an infinite array. This is how RCF's are expressed in coefficient notation, the first item being the a and the tail after ; are the RCF coefficients of x. These two are sufficient since we already know that in RCF all numerators are fixed to 1.
Coming back to your precision question. You now have √11 = 3 + x where x is your RCF as [3;3,6,3,6,3,6...]. Normally you can try by picking a depth and reducing from right like [3,3,6,3,6,3,6...].reduceRight((p,c) => c + 1/p) as it would be done in JS. Not a precise enough result.? Then try it again from another depth. This is in fact how it is descriped in the linked wikipedia topic as bottom up. However it would be much efficient to go from left to right (top to bottom) by calculating the intermediate convergents one after the other, at a single pass. Every next intermediate convergent yields a better precision for you to test and decide weather to stop or continue. When you reach to a coefficient sufficient enough just stop there. Having said that, once you reach to the desired coefficient you may still do some fine tuning by increasing or decreasing that coefficient. Decreasing the coefficients at even indices or increasing the ones at odd indices would decrease the convergent and vice versa.
So in order to be able to do a left to right (top to bottom) analysis there is a special rule as
n2/d2 = (xn * n1 + n0)/(xn * d1 + d0)
We need to know last two interim convergents (n0/d0 and n1/d1) along with the current coefficient xn in order to be able calculate the next convergent (n2/d2).
We will start with two initial convergents as Infinity (n0/d0 = 1/0) and the a that we've chosen above (Remember √n = a + x) which is 3 so (n1/d1 = 3/1). Knowing that the 3 before the semicolon is in fact a, our first xn is the 3 right after the semicolon in our coefficients array [3;»» 3 ««,6,3,6,3,6...].
After we calculate n2/d2 and do our test, if need be, for the next step we will shift our convergents to the left so that we have the last two ready to calculate the next convergent. n0/d0 <- n1/d1 <- n2/d2
Here i present the table for the n2/d2 = (xn * n1 + n0)/(xn * d1 + d0) rule.
n0/d0 n1/d1 xn index n2/d2 decimal val.
_____ ______ __ _____ ________ ____________
1/0 3/1 3 1 odd 10/3 3.33333333..
3/1 10/3 6 2 evn 63/19 3.31578947..
10/3 63/19 3 3 odd 199/60 3.31666666..
63/19 199/60 6 4 evn 1257/379 3.31662269..
. . . . . .
. . . . . .
So as you may notice we are very quickly approaching to √11 which is 3.31662479... Note that the odd indices overshoot and evens undershoot due to cascading reciprocals. Since √11 is an irrational this will continue convergining indefinitely up until we say enough.
Remember, as mentioned earlier, once you reach to the desired coefficient you may still do some fine tuning by increasing or decreasing that coefficient (xn). Decreasing the coefficients at even indices or increasing the ones at odd indices would decrease the convergent and vice versa.
The problem here is, not all √n can simply be turned into RCF by a simple division as shown above. For a more generalized way to generate RCF from any √n you may check a more recent answer of mine.
I've been playing with wolfram language and noticed something: the same function written in different ways works very differently in terms of time.
Consider these two functions:
NthFibonacci[num_] :=
If [num == 0 || num == 1, Return[ 1],
Return[NthFibonacci[num - 1] + NthFibonacci[num - 2]]
]
Fibn[num_] := {
a = 1;
b = 1;
For[i = 0, i < num - 1, i++,
c = a + b;
a = b;
b = c;
];
Return [b];
}
NthFibonacci[30] takes around 5 seconds to evaluate.
Fibn[900 000] also takes around 5 seconds to evaluate.
So does the built-in Fibonacci[50 000 000]
I simply can't get why are there such differences in speed between the three. In theory, recursion should be more or less equivalent to a for loop. What is causing this?
It's because the recursive version you present does lots and lots of repeated calculations. Build a tree of the function calls to see what I mean. Even for an argument as small as 4, look at how many function calls are generated to get to a base case down each chain of the logic.
f(1)
/
f(2)
/ \
f(3) f(0)
/ \
/ f(1)
/
f(4)
\
\ f(1)
\ /
f(2)
\
f(0)
With your recursion, the number of function calls grows exponentially with the argument num.
By contrast, your looped version grows linearly in num. It doesn't take a very large value of n before n is a lot less work than 2n.
There are many ways to implement recursion; the Fibonacci function is a lovely example. As pjs already pointed out, the classic, double-recursive definition grows exponentially. The base is
φ = (sqrt(5)+1) / 2 = 1.618+
Your NthFibonacci implementation works this way. It's order φ^n, meaning that for large n, calling f(n+1) takes φ times as long as f(n).
The gentler approach computes each functional value only once in the stream of execution. Instead of exponential time, it takes linear time, meaning that calling f(2n) takes 2 times as long as f(n).
There are other approaches. For instance, Dynamic Programming (DP) keeps a cache of previous results. In pjs f(4) case, a DP implementation would compute f(2) only once; the second call would see that the result of the first was in cache, and return the result rather than making further calls to f(0) and f(1). This tends toward linear time.
There are also implementations that make checkpoints, such as caching f(k) and f(k)+1 for k divisible by 1000. These save time by have a starting point not too far below the desired value, giving them an upper bound of 998 iterations to find the needed value.
Ultimately, the fastest implementations use the direct computation (at least for larger numbers) and work in constant time.
φ = (1+sqrt(5)) / 2 = 1.618...
ψ = (1-sqrt(5)) / 2 = -.618...
f(n) = (φ^n - ψ^n) / sqrt(5)
The issue noted by #pjs can be addressed to a degree by having the recursive function remember prior values. (eliminating the If helps too)
Clear[NthFibonacci]
NthFibonacci[0] = 1
NthFibonacci[1] = 1
NthFibonacci[num_] :=
NthFibonacci[num] = NthFibonacci[num - 1] + NthFibonacci[num - 2]
NthFibonacci[300] // AbsoluteTiming
{0.00201479, 3.59 10^62}
Cleaning up you loop version as well (You should almost never use Return in mathematica):
Fibn[num_] := Module[{a = 1, b = 1,c},
Do[c = a + b; a = b; b = c, {num - 1}]; b]
Fibn[300] // AbsoluteTiming
{0.000522175 ,3.59 10^62}
you see the recursive form is slower, but not horribly so. (Note the recursive form hits a depth limit around 1000 as well )
I had an interview yesterday that involved a pretty basic tree data structure:
t ::= int | (t * t)
where a tTree is either an integer (leaf) or two t's which represent left and right subtrees. This implies that the tree will only have values at the leaf level.
An example tree could look like this:
t
/ \
t t
/ \ / \
1 2 3 4
The task was to write a function equal(t, t) => bool that takes two tTrees and determines whether or not they are equal, pretty simple.
I wrote pretty standard code that turned out like this (below is pseudocode):
fun equal(a, b) {
if a == b { // same memory address
return true
}
if !a || !b {
return false
}
// both leaves
if isLeaf(a) && isLeaf(b) {
return a == b
}
// both tTrees
if isTree(a) && isTree(b) {
return equal(a->leftTree, b->leftTree)
&& equal(a->rightTree, b->rightTree)
}
// otherwise
return false
}
When asked to give the time and space complexity, I quickly answered:
O(n) time
O(1) space
My interviewer claimed they could create a tree such that this equal function would run in O(2^n) exponential time. I didn't (and still don't) see how this is possible given the algorithm above. I see that the function recursively calls itself twice, but the input size is halved on each of those calls right? because you are only examining the respective subtrees in parallel.
Any thoughts or input on this would be really helpful.
As it stands, your code is O(n), and your interviewer was mistaken. The code's not O(1) in space use though: it's O(n) in the worst case (when the trees are very unbalanced) because your code is recursive (and not tail recursive).
Probably they were asking you to write a function that tests if two trees are isomorphic. That is, they were expecting you to write code that returns true when comparing these two trees:
* *
/ \ / \
1 2 2 1
Then they misread your solution, assuming you'd written the naive code that does this, which would be O(2^n).
Another possibility is that some of the pointers could be reused in both the left and right branches of the same tree, allowing a tree with 2^n nodes to be represented in O(n) space. Then if 'n' is the size of the structure in memory, rather than the number of nodes, then the interviewer's position is correct. Here's such a tree:
___ ___ ___ ___ ___
/ \ / \ / \ / \ / \
* * * * * 1
\___/ \___/ \___/ \___/ \___/
The root is on the left, and it has 32 leaf nodes (all 1).
I have function in my model that counts user's score:
def score
(MULTIPLER * Math::log10(bets.count * summary_value ** accuracy + 1)).floor
end
My point is to test that it grows logarithmically?
The point of a test isn't to prove it always works (this is the area for static typing/proofs), but to check that it is probably working. This is normally good enough. I'm guessing you are doing it for a game, and what to ensure the function doesn't "grow" too quickly.
A way we could do that is to try a number of values, and check whether they are following a general logarithmic pattern.
For example, consider a pure logarithmic function f(x) = log(x) (any base):
If f(x) = y, then f(x^n) = f(x) * n.
So, if f(x^n) == (f(x) * n), then the function is logarithmic.
Compare that to a linear function, eg f(x) == x * 2. f(x^n) = x^n * 2, ie x^(n - 1) times bigger (a lot bigger).
You may have a more complex logarithmic function, eg f(x) = log(x + 7) + 3456. The pattern still holds though, just less accurately. So what I did was:
Attempt to calculate the constant value, by using x = 1
Find the difference f(x^n) - f(x) * n.
Find the absolute difference of f((x*100)^n) - f(100x) * n
If (3)/(2) is less than 10, it is almost certainly not linear, and probably logarithmic. The 10 is just an arbitrary number. Most linear functions will be different by a factor of more than a billion. Even functions like sqrt(x) will have a bigger difference than 10.
My example code will just have the score method take a parameter, and test against that (to keep it simple + I don't have your supporting code).
require 'rspec'
require 'rspec/mocks/standalone'
def score(input)
Math.log2(input * 3 + 1000 * 3) * 3 + 100 + Math.sin(input)
end
describe "score" do
it "grows logarithmacally based on input" do
x = 50
n = 8
c = score(1)
result1 = (score(x ** n) - c) / ((score(x) -c) * n)
x *= 100
result2 = (score(x ** n) - c) / ((score(x) -c) * n)
(result2 / result1).abs.should be < 10
end
end
Though I almost forget complex math knowledge, it seems the fact can't stop me answering the question.
My suggestion as follows
describe "#score" do
it "grows logarithmically" do
round_1 = FactoryGirl.create_list(:bet, 10, value: 5).score
round_2 = FactoryGirl.create_list(:bet, 11, value: 5).score
# Then expect some math relation between round_1 and round_2,
# calculated by you manually.
end
end
If you need to treat this function like a black box, the only true solution is to get a bunch of values and see if their curve is well-approximated by a logarithmic curve, focusing on large n. If you could put reasonable bounds on it, like a log(n) < score(n) < b log(n) for some values a and b then you could just check that.
Generally speaking, the best way to see if a function grows is to plot some data on a graph. Just use some graph plotting gem and evaluate the result.
A logarithmic function will always look like this:
(source: sosmath.com)
You can then adjust how fast it grows through your parameters, and replot the graph, until you found yourself happy with the result.
We have the following input for the algorithm:
A graph G with no cycles (aka a spanning-tree) where each node has an associated weight.
I want to find an independent set S such that:
No two elements in S form an edge in G
There is no other possible subset which satisfies the above condition, for which there is a greater weight than S[0] + S[1] + ... + S[n-1] (where len(S)==n).
This is the high-level pseudocode I have so far:
MaxWeightNodes(SpanningTree S):
output = {0}
While(length(S)):
o = max(node in S)
output = output (union) o
S = S \ (o + adjacentNodes(o))
End While
Return output
Can someone tell me off the bat whether I've made any errors, or if this algorithm will give me the result I want?
The algorithm is not valid, since you'll soon face a case when excluding the adjacent nodes of an initial maximum may be the best local solution, but not the best global decision.
For example, output = []:
10
/ \
100 20
/ \ / \
80 90 10 30
output = [100]:
x
/ \
x 20
/ \ / \
x x 10 30
output = [100, 30]:
x
/ \
x x
/ \ / \
x x 10 x
output = [100, 30, 10]:
x
/ \
x x
/ \ / \
x x x x
While we know there are better solutions.
This means you're down on a greedy algorithm, without an optimal substructure.
I think the weights of the vertices make greedy solutions difficult. If all weights are equal, you can try choosing a set of levels of the tree (which obviously is easiest with a full k-ary tree, but spanning trees generally don't fall into that class). Maybe it'll be useful for greedy approximation to think about the levels as having a combined weight, since you can always choose all vertices of the same level of the tree (independent of which vertex you root it at) to go into the same independent set; there can't be an edge between two vertices of the same level. I'm not offering a solution because this seems like a difficult problem to me. The main problems seem to be the weights and the fact that you can't assume that you're dealing with full trees.
EDIT: actually, always choosing all vertices of one level seems to be a bad idea as well, as Rubens's example helps visualize; imagine the vertex on the second level on the right of his tree had a weight of 200.