I was reading Introduction to Algorithms by Cormen on dynamic programming.
The ordering of matrices is important to perform a matrix multiplication. If I multiplied a number of matrices and come up with the best result, how to keep that result optimal by adding a matrix to the series ?
If you have:
DP[i, j] = minimum cost of multiplying matrices i to through j
then DP[1, n] will be your answer.
To find DP[1, n + 1], just apply the same recurrence you used to build the table:
DP[1, n + 1] = min {DP[1, k] + DP[k + 1, n + 1] + multiplication cost}
1<=k<n+1
This will be O(n).
Related
I had such an equation on Algorithms and Data Structures subject. It looks very obvious but how exactly I can prove it?
Even on graph we can see that |sin(n!)| does not go above 1 and below 0.
Proof of |sin(n)| is O(1):
We have two functions:
f(n) = |(sin(n!)|
g(n) = 1
And we have to proof that for every n > 0 this statement is correct: f(n) <= g(n)
Let's use induction for such case:
Base - For n = 1, f(1) ~= 0.8 and g(1)=1
Induction - Suppose that f(n) <= g(n) for n = 1, 2, 3, ...k. Now let's show that for k+1 the f(k+1) <= g(k+1) is valid also.
n! range is from 1 to inifinity
sin(x) by definition can be [-1, 1], so sin(n!) has same range as well.
|sin(n!)| scopes it to be [0, 1]
Whatever k was f(k+1) still will be in range of [0, 1], therefore f(k+1) <= g(k+1), therefore our statement is correct for any k.
As it was mentioned by #Berthur in comments, we have to proof as well that |sin(n)| has a lower positive non-zero bound. We can't show that, because |sin(n)| can have arbitrary close to zero values.
I know the time complexity and space complexity of matrix chain multiplication using dynamic programming would be O(n^3) and O(n^2).
But I want to know the time as well as space complexity of the brute force approach for this problem which can be implemented with below code.
def MatrixChainOrder(p, i, j):
if i == j:
return 0
_min = sys.maxsize
for k in range(i, j):
count = (MatrixChainOrder(p, i, k)
+ MatrixChainOrder(p, k + 1, j)
+ p[i-1] * p[k] * p[j])
if count < _min:
_min = count
# Return minimum count
return _min
#arr = [1, 2, 3, 4, 3]
#n = len(arr)
# p is array name
# i=1
#j= n-1
Please elaborate...
The stack depth is linear, hence so is the space usage.
As for time, we get a recurrence
T(1) = 1
T(n) = sum_{k=1}^{n-1} (T(k) + T(n-k)) = 2 sum_{k=1}^{n-1} T(k).
We can verify that the solution is
T(1) = 1
T(n) = 2 (3^(n-1))
so the running time is Θ(3n).
I've been staring at this for a while and it's not sinking in. I think I understand at a basic level what's going on. E.g. A = {1, 2, 3, 4}
Sum = A[0] + [A[0] + A[1]] + [A[0] + A[1] + A[2]] + [A[0] + A[1] + A[2] + A[3]]
However, I'm not able to understand the steps via the explanation/notation below - or at least, it's a little fuzzy. Could someone please explain the steps/walk through what's happening.
Example 1.4 (Sums of subarrays). The problem is to compute, for each subarray a[j..j +m−1] of size m in an array a of size n, the partial sum of its elements s[j] = ∑ m−1 k=0 a[j+k]; j = 0,...,n−m. The total number of these subarrays is n−m+1.
At first glance, we need to compute n−m+1 sums, each of m items, so that the running time is proportional to m(n−m+1). If m is fixed, the time depends still linearly on n. But if m is growing with n as a fraction of n, such as m = n 2, then T(n) = cn 2n 2 +1= 0.25cn2 +0.5cn. The relative weight of the linear part, 0.5cn, decreases quickly with respect to the quadratic one as n increases.
Well, the explanation, you provided seems to be not about your understanding of the problem. I think, your Example 1.4 is really about following.
A = {1, 2, 3, 4}, m = 3.
Sum = (A[0] + A[1] + A[2]) + (A[1] + A[2] + A[3]).
Here you have n-m+1 (4-3+1=2) subsums of m(3) elements each. The described algorithm can be preformed in code like this:
function SumOfSubarrays(A, n, m) {
sum = 0;
//loop for subarrays
for (j = 0; j <= n - m; j++;) {
//loop for elements in each subarray
for (k = 0; k <= m - 1; k++) {
sum += A[j + k];
}
}
}
Time complexity of this algorithm depends linearly on n. But, as it is said in Example 1.4, if m growths as a fraction of n, then time complexity becomes quadratic.
You need totally m(n−m+1) operations: (n−m+1) for outer loop as it is a number of subarrays, m for inner loop as it is a number of elements in each subarray. If m depends on n then you have, for example:
m = 0.5 * n
m(n-m+1) = 0.5n(n-0.5n+1) = 0.5n(0.5n-1) = 0.25n^2 - 0.5n
Where quadratic part growths faster as it is quadratic.
My Question
What is the Big-O complexity for this code snippet?
Consider n as a power of 4.
for(int i = 1; i <= n; i = i*4)
for(int k = 1; k <= i; k++)
// constant statement
What I know so far
I tried making this code into a summation to find the complexity.
This is what I got:
I got (base 4) log(n) by computing the series 4, 4^2, 4^3 ... 4^r = n.
r = (base 4) log(n).
I'm now stuck at this summation:
Please let me know If I'm doing something wrong or there is another way to do this.
You’re on the right track here, but your innermost summation is wrong. You are right that the outer loop will iterate log_4 n times, but you’ve set up the outer sum so that i counts up as 1, 2, 3, ..., log_4 n, rather than 4^0, 4^1, 4^2, ... 4^log_4 n. As a result, that inner summation’s upper bound is incorrect. The bound should be 4^i, not i.
If you set things up this way, you’ll find that the overall sum is
4^0 + 4^1 + 4^2 + ... + 4^log_4 n
= (4^(log_4 n + 1) - 1) / (4 - 1) (using the formula for the sum of a geometric series
= (4(4^log_4 n) - 1) / 3
= (4n - 1) / 3
= Θ(n).
You may use wolframalpha to get the result for extreme accuracy.
https://www.wolframalpha.com/input/?i=sum+i,+i%3D1+to+log_4(n)
This is Exercise 15.5-4 of Introduction to Algorithms, 3rd edition, which is about Knuth's improvement to the DP approach to Optimal Binary Search Tree.
The DP algorithm of Optimal Binary Search Tree is:
OPTIMAL_BST(p, q, n)
let e[1..n+1, 0..n], w[1..n+1, 0..n], and root[1..n, 1..n] be new tables
for i = 1 to n+1
e[i, i - 1] = q[i - 1];
w[i, i - 1] = q[i - 1];
for l = 1 to n
for i = 1 to n - l + 1
j = i + l - 1
e[i, j] = INFINITY
w[i, j] = w[i, j - 1] + p[j] + q[j]
for r = i to j
t = e[i, r - 1] + e[r + 1, j] + w[i, j]
if t < e[i, j]
e[i, j] = t
root[i, j] = r
return e and root
The complexity is O(n3).
Knuth had observed that root[i, j - 1] <= root[i, j] <= root[i + 1, j], so Exercise 15.5-4 asks to implement an O(n2) algorithm by doing some modification to the original algorithm.
Well after some effort I have figured this out: in the innermost loop, replace the line
for r = i to j
with
for r = r[i, j - 1] to r[i + 1, j]
This has been proved by this link: Optimal binary search trees
However, I'm not sure this is really O(n2): since during each innermost loop, distance from r[i, j - 1] to r[i + 1, j] is not constant, I suspect it is still O(n3).
So my question is: can you please explain to me why the improvement to DP algorithm yields O(n2) complexity?
PS: Maybe I might have read Knuth's paper first, but really I searched the web but found no free access to the paper.
You're correct that the distance from r[i, j - 1] to r[i + 1, j] is not constant in the worst case, but it is constant on average, which suffices to imply a quadratic running time. The total number of iterations for l is
S = sum_{i = 1}^{n - l + 1} (r[i + 1, j] + 1 - r[i, j - 1]), j = i + l - 1
= sum_{i = 1}^{n - l + 1} (r[i + 1, i + l - 1] + 1 - r[i, i + l - 2])
= r[n - l + 2, n] + n - l + 1 - r[1, l - 1]
therefore the average is S / (n - l + 1), which is a constant
by simplifying the telescoping sum.
You can find the exact running time analysis with a google search or just start to write your own analysis w.r.t for loops. But just note that in all of them sum in total is calculated by telescopic sum, I mean may be one of them is big but in each iteration for first loop takes O(n), and totally takes O(n2).