Matrix Chain Multiplication has a dynamic programming solution where a recursive definition is used which works like this :
Problem : multiply i to j
Sub-problem : multiply i to k + multiply k+1 to j + multiplication cost
and this looks straight forward to memoize, due the repeating (i,j) sub-problems. But the following recursive definition which is bit different, I am facing difficulty memoizing it :
Can someone help memoizing this algo for matrix chain multiplication :
P is sequence of orders of matrices.
For eg, A(2,3)*B(3,4)*C(4,5), then P = {2,3,4,5}, i.e. order of ith matrix is P[i-1]*P[i]
also assumed P is 0-indexed.
Here I am multiplying adjacent matrices and recursing
Pseudocode :
chain_mul(P, n) {
if(n = 1) return 0
min_cost = inf
for( i = 1 to n-1) {
cost = P[i-1]*P[i]*P[i+1] + chain_mul(P-{P[i]}, n-1);
if(cost < min_cost) min_cost = cost
}
return min_cost
}
Here repeating sub-problem is structure of P, like I have shown below :
This cannot be memoized efficiently, because the argument P, iterates over all the subsets of the initial set P, so the memory required would be O(2^n).
The algoritms that can be memoized call the function specifying sections of the matrix chain, each section is characterized by two numbers, start and end index. The number of segments will be something like (n * (n + 1) / 2), and it is easy to implement a data structure to store and retrieve the results indexed by two numbers (e.g a matrix).
Related
I would like to generate a sequence of n random integers in the interval [1,n] without duplicates, i.e. a permutation of the sequence [1,2,...,n] with O(log(n)) space complexity (or a polynomial function of log(n)).
One hint is that I can assume that I have a family of l-wise uniform hash functions h : [n] -> [k] (with l<=n) such that for any y_1, y_2,..., y_l and any distinct x_1, x_2,..., x_l :
P(h(x_1) = y_1 and h(x_2) = y_2 and ... and h(x_l) = y_l) = 1/(k^l)
My first idea was to use the hash function to generate the i-th element of the sequence, i.e. x_i = h(i) , check if x_i is already used (has already been returned by the hash function for some 0<j<i) and if it's the case increment x_i by 1 and check again until x_i is a new number. My problem is I can not have a vector of booleans of size n to check if the value x_i is already used. And if I do a recursive function to get the j-th value I will need at some point O(n log2(n)) bits...
I also found here that pseudorandom generator like Linear congruential generator can be used for this kind of problem with something like x_i+1 = (a*x_i + c)%n + 1 but I am not sure to understand how to choose a for any value of n to have a period of length n. In that case the hint is not really useful except for generating the first number of the sequence thus I don't think it's the right way.
Here's a fun super simple solution with constant space; when N is a power of 2 and your definition of "random" is incredibly loose (the resulting sequence will alternate between even and odd numbers).
N = power of 2
P = prime number larger than N.
S = random starting number between 0 and N-1
For i = 1 TO N
// add our prime to the starting random number
S += P
// S Modulus N
// Bitwise And N-1 works because N is a pow of 2
T = S & (N - 1)
//T is [0, (N-1)] => we want [1, N]
PRINT (T + 1)
Next I
JS
for(let N = 64, P = 73, S = N * Math.random(), i = 1; i <= N; i++) { S += P; console.log((S & (N - 1)) + 1); }
Another answer would probably be to consider all of the numbers [1, N] as leaf nodes in a tree and your Log(N) space is the size of a the path through the tree. Your solution would be a function that permutes all N paths through the tree. The way you permute the paths in a pseudo random way would basically be a Linear Feedback Shift Register type generator that has a period grater than N.
https://www.maximintegrated.com/en/design/technical-documents/app-notes/4/4400.html
Given an array nums
Count no. of pairs (two elements) where bitwise AND is greater than K
Brute force
for i in range(0,n):
for j in range(i+1,n):
if a[i]&a[j] > k:
res += 1
Better version:
preprocess to remove all elements ≤k
and then brute force
But i was wondering, what would be the limit in complexity here?
Can we do better with a trie, hashmap approach like two-sum?
( I did not find this problem on Leetcode so I thought of asking here )
Let size_of_input_array = N. Let the input array be of B-bit numbers
Here is an easy to understand and implement solution.
Eliminate all values <= k.
The above image shows 5 10-bit numbers.
Step 1: Adjacency Graph
Store a list of set bits. In our example, 7th bit is set for numbers at index 0,1,2,3 in the input array.
Step 2: The challenge is to avoid counting the same pairs again.
To solve this challenge we take help of union-find data structure as shown in the code below.
//unordered_map<int, vector<int>> adjacency_graph;
//adjacency_graph has been filled up in step 1
vector<int> parent;
for(int i = 0; i < input_array.size(); i++)
parent.push_back(i);
int result = 0;
for(int i = 0; i < adjacency_graph.size(); i++){ // loop 1
auto v = adjacency_graph[i];
if(v.size() > 1){
int different_parents = 1;
for (int j = 1; j < v.size(); j++) { // loop 2
int x = find(parent, v[j]);
int y = find(parent, v[j - 1]);
if (x != y) {
different_parents++;
union(parent, x, y);
}
}
result += (different_parents * (different_parents - 1)) / 2;
}
}
return result;
In the above code, find and union are from union-find data structure.
Time Complexity:
Step 1:
Build Adjacency Graph: O(BN)
Step 2:
Loop 1: O(B)
Loop 2: O(N * Inverse of Ackermann’s function which is an extremely slow-growing function)
Overall Time Complexity
= O(BN)
Space Complexity
Overall space complexity = O(BN)
First, prune everything <= k. Also Sort the value list.
Going from the most significant bit to the least significant we are going to keep track of the set of numbers we are working with (initially all ,s=0, e=n).
Let p be the first position that contains a 1 in the current set at the current position.
If the bit in k is 0, then everything that would yield a 1 world definetly be good and we need to investigate the ones that get a 0. We have (end - p) * (end-p-1) /2 pairs in the current range and (end-p) * <total 1s in this position larger or equal to end> combinations with larger previously good numbers, that we can add to the solution. To continue we update end = p. We want to count 1s in all the numbers above, because we only counted them before in pairs with each other, not with the numbers this low in the set.
If the bit in k is 1, then we can't count any wins yet, but we need to eliminate everything below p, so we update start = p.
You can stop once you went through all the bits or start==end.
Details:
Since at each step we eliminate either everything that has a 0 or everything that has a 1, then everything between start and end will have the same bit-prefix. since the values are sorted we can do a binary search to find p.
For <total 1s in this position larger than p>. We already have the values sorted. So we can compute partial sums and store for every position in the sorted list the number of 1s in every bit position for all numbers above it.
Complexity:
We got bit-by-bit so L (the bit length of the numbers), we do a binary search (logN), and lookup and updates O(1), so this is O(L logN).
We have to sort O(NlogN).
We have to compute partial bit-wise sums O(L*N).
Total O(L logN + NlogN + L*N).
Since N>>L, L logN is subsummed by NlogN. Since L>>logN (probably, as in you have 32 bit numbers but you don't have 4Billion of them), then NlogN is subsummed by L*N. So complexity is O(L * N). Since we also need to keep the partial sums around the memory complexity is also O(L * N).
I'm trying to solve a very simple algorithm analysis (apparently isn't so simple to me).
The algorithm is going like this:
int findIndexOfN(int A[], int n) {
// this algorithm looks for the value n in array with size of n.
// returns the index of n in case found, otherwise returns -1.
// it is possible that n doesn't appear in the array.
// n appears at most one time.
// the probability that n doesn't appear in the array is $1/(n+1)$
// for each cell in the array, the probability that n is found in index i
// is $1/(n+1)$
int index, fIndex;
index = 0;
fIndex = -1;
while (index < n && fIndex == -1) {
if(A[index] == n) {
fIndex = index;
}
index++;
}
return fIndex;
}
Now I'm trying to calculate the average running time. I think this is Geometric series but I can't find out a way to merge between the terms probability and complexity.
For example, I know that in case the value n is found in index 1, then it would take 1 loop step to get the second index (1) and find n.
The probabilty on the other hand gives me some fractions....
Here is what I got so far:
$\sigma from i=1 to n evaluate ( (1/n) * ((n-1)/n)^i-1 )
But again, I can't find out the connection of this formula to T(n) and also I can't find a relation of BigOh, BigOmega or Theta for this function.
This algorithm is BigOh(n), BigOmega(n) and Theta(n).
To know this you don't need to compute probabilities or use the Master Theorem (as your function isn't recursive). You just need to see that the function is like a loop over n terms. Maybe it would be easier if you represented your function like this:
for (int i = 0; i < n; ++i) {
if (A[i] == n)
return i;
}
I know this seems counterintuitive, because if n is the first element of your array, indeed you only need one operation to find it. What is important here is the general case, where n is somewhere in the middle of your array.
Let's put it like this: given the probabilities you wrote, there is 50% chances that n is between the elements n/4 and 3n/4 of your array. In this case, you need between n/4 and 3n/4 tests to find your element, which evaluates to O(n) (you drop the constant when you do BogOh analysis).
If you want to know the average number of operations you will need, you can compute a series, like you wrote in the question. The actual series giving you the average number of operations is
1/(n+1) + 2/(n+1) + 3/(n+1) ... + n/(n+1)
Why? Because you need one test if n is in the first position (with probability 1/(n+1)), two tests if n is in the second position (with probability 1/(n+1)), ... i tests if n is in the ith position (with probability 1/(n+1))
This series evaluates to
n(n+1)/2 * 1/(n+1) = n/2
Assume that I have a vector, V, of positive integers. If the sum of the integers are larger than a positive integer N, I want to rescale the integers in V so that the sum is <= N. The elements in V must remain above zero. The length of V is guaranteed to be <= N.
Is there an algorithm to perform this rescaling in linear time?
This is not homework, BTW :). I need to rescale a map from symbols to symbol frequencies to use range encoding.
Some quick thinking and googling has not given a solution to the problem.
EDIT:
Ok, the question was somewhat unclear. "Rescale" means "normalize". That is, transform the integers in V, for example by multiplying them by a constant, to smaller positive integers so the criterion of sum(V) <= N is fulfilled. The better the ratios between the integers are preserved, the better the compression will be.
The problem is open-ended in that way, the method does not need to find the optimal (in, say, a least squares fit sense) way to preserve the ratios, but a "good" one. Setting the entire vector to 1, as suggested, is not acceptable (unless forced). "Good" enough would for example be finding the smallest divisor (defined below) that fulfills the sum criterion.
The following naive algorithm does not work.
Find the current sum(V), Sv
divisor := int(ceil(Sv/N))
Divide each integer in V by divisor, rounding down, but not to less than 1.
This fails on v = [1,1,1,10] with N = 5.
divisor = ceil(13 / 5) = 3.
V := [1,1,1, max(1, floor(10/3)) = 3]
Sv is now 6 > 5.
In this case, the correct normalization is [1,1,1,2]
One algorithm that would work is to do a binary search for divisor (defined above) until the smallest divisor in [1,N] fulfilling the sum criterion is found. Starting with the ceil(Sv/N) guess. This is however, not linear in number of operations, but proportional to len(V)*log(len(V)).
I am starting to think that it is impossible to do well, in linear time, in the general case. I might resort to some sort of heuristic.
Just divide all the integers by their Greatest Common Divisor. You can find the GCD efficiently with multiple applications of Euclid's Algorithm.
d = 0
for x in xs:
d = gcd(d, x)
xs = [x/d for x in xs]
The positive point is that you always have a small as possible representation this way, without throwing away any precision and without needing to choose a specific N. The downside is that if your frequencies are large coprime numbers you will have no choice but to sacrifice precision (and you didn't specify what should be done in this case).
How about this:
Find the current sum(V), Sv
divisor := int(ceil(Sv/(N - |V| + 1))
Divide each integer in V by divisor, rounding up
On v = [1,1,1,10] with N = 5:
divisor = ceil(13 / 2) = 7.
V := [1,1,1, ceil(10/7)) = 2]
I think you should just rescale the part above 1. So, subtract 1 from all values, and V.length from N. Then rescale normally, then add 1 back. You can even do slightly better if you keep running totals as you go along, instead of choosing just one factor, which will usually waste some "number space". Something like this:
public static void rescale(int[] data, int N) {
int sum = 0;
for (int d : data)
sum += d;
if (sum > N) {
int n = N - data.length;
sum -= data.length;
for (int a = 0; a < data.length; a++) {
int toScale = data[a] - 1;
int scaled = Math.round(toScale * (float) n / sum);
data[a] = scaled + 1;
n -= scaled;
sum -= toScale;
}
}
}
This is a problem of 'range normalization', but it's very easy. Suppose that S is the sum of the elements of the vector, and S>=N, then S=dN, for some d>=1. Therefore d=S/N. So just multiply every element of the vector by N/S (i.e. divide by d). The result is a vector with rescaled components which sum is exactly N. This procedure is clearly linear :)
I have a series
S = i^(m) + i^(2m) + ............... + i^(km) (mod m)
0 <= i < m, k may be very large (up to 100,000,000), m <= 300000
I want to find the sum. I cannot apply the Geometric Progression (GP) formula because then result will have denominator and then I will have to find modular inverse which may not exist (if the denominator and m are not coprime).
So I made an alternate algorithm making an assumption that these powers will make a cycle of length much smaller than k (because it is a modular equation and so I would obtain something like 2,7,9,1,2,7,9,1....) and that cycle will repeat in the above series. So instead of iterating from 0 to k, I would just find the sum of numbers in a cycle and then calculate the number of cycles in the above series and multiply them. So I first found i^m (mod m) and then multiplied this number again and again taking modulo at each step until I reached the first element again.
But when I actually coded the algorithm, for some values of i, I got cycles which were of very large size. And hence took a large amount of time before terminating and hence my assumption is incorrect.
So is there any other pattern we can find out? (Basically I don't want to iterate over k.)
So please give me an idea of an efficient algorithm to find the sum.
This is the algorithm for a similar problem I encountered
You probably know that one can calculate the power of a number in logarithmic time. You can also do so for calculating the sum of the geometric series. Since it holds that
1 + a + a^2 + ... + a^(2*n+1) = (1 + a) * (1 + (a^2) + (a^2)^2 + ... + (a^2)^n),
you can recursively calculate the geometric series on the right hand to get the result.
This way you do not need division, so you can take the remainder of the sum (and of intermediate results) modulo any number you want.
As you've noted, doing the calculation for an arbitrary modulus m is difficult because many values might not have a multiplicative inverse mod m. However, if you can solve it for a carefully selected set of alternate moduli, you can combine them to obtain a solution mod m.
Factor m into p_1, p_2, p_3 ... p_n such that each p_i is a power of a distinct prime
Since each p is a distinct prime power, they are pairwise coprime. If we can calculate the sum of the series with respect to each modulus p_i, we can use the Chinese Remainder Theorem to reassemble them into a solution mod m.
For each prime power modulus, there are two trivial special cases:
If i^m is congruent to 0 mod p_i, the sum is trivially 0.
If i^m is congruent to 1 mod p_i, then the sum is congruent to k mod p_i.
For other values, one can apply the usual formula for the sum of a geometric sequence:
S = sum(j=0 to k, (i^m)^j) = ((i^m)^(k+1) - 1) / (i^m - 1)
TODO: Prove that (i^m - 1) is coprime to p_i or find an alternate solution for when they have a nontrivial GCD. Hopefully the fact that p_i is a prime power and also a divisor of m will be of some use... If p_i is a divisor of i. the condition holds. If p_i is prime (as opposed to a prime power), then either the special case i^m = 1 applies, or (i^m - 1) has a multiplicative inverse.
If the geometric sum formula isn't usable for some p_i, you could rearrange the calculation so you only need to iterate from 1 to p_i instead of 1 to k, taking advantage of the fact that the terms repeat with a period no longer than p_i.
(Since your series doesn't contain a j=0 term, the value you want is actually S-1.)
This yields a set of congruences mod p_i, which satisfy the requirements of the CRT.
The procedure for combining them into a solution mod m is described in the above link, so I won't repeat it here.
This can be done via the method of repeated squaring, which is O(log(k)) time, or O(log(k)log(m)) time, if you consider m a variable.
In general, a[n]=1+b+b^2+... b^(n-1) mod m can be computed by noting that:
a[j+k]==b^{j}a[k]+a[j]
a[2n]==(b^n+1)a[n]
The second just being the corollary for the first.
In your case, b=i^m can be computed in O(log m) time.
The following Python code implements this:
def geometric(n,b,m):
T=1
e=b%m
total = 0
while n>0:
if n&1==1:
total = (e*total + T)%m
T = ((e+1)*T)%m
e = (e*e)%m
n = n/2
//print '{} {} {}'.format(total,T,e)
return total
This bit of magic has a mathematical reason - the operation on pairs defined as
(a,r)#(b,s)=(ab,as+r)
is associative, and the rule 1 basically means that:
(b,1)#(b,1)#... n times ... #(b,1)=(b^n,1+b+b^2+...+b^(n-1))
Repeated squaring always works when operations are associative. In this case, the # operator is O(log(m)) time, so repeated squaring takes O(log(n)log(m)).
One way to look at this is that the matrix exponentiation:
[[b,1],[0,1]]^n == [[b^n,1+b+...+b^(n-1))],[0,1]]
You can use a similar method to compute (a^n-b^n)/(a-b) modulo m because matrix exponentiation gives:
[[b,1],[0,a]]^n == [[b^n,a^(n-1)+a^(n-2)b+...+ab^(n-2)+b^(n-1)],[0,a^n]]
Based on the approach of #braindoper a complete algorithm which calculates
1 + a + a^2 + ... +a^n mod m
looks like this in Mathematica:
geometricSeriesMod[a_, n_, m_] :=
Module[ {q = a, exp = n, factor = 1, sum = 0, temp},
While[And[exp > 0, q != 0],
If[EvenQ[exp],
temp = Mod[factor*PowerMod[q, exp, m], m];
sum = Mod[sum + temp, m];
exp--];
factor = Mod[Mod[1 + q, m]*factor, m];
q = Mod[q*q, m];
exp = Floor[ exp /2];
];
Return [Mod[sum + factor, m]]
]
Parameters:
a is the "ratio" of the series. It can be any integer (including zero and negative values).
n is the highest exponent of the series. Allowed are integers >= 0.
mis the integer modulus != 0
Note: The algorithm performs a Mod operation after every arithmetic operation. This is essential, if you transcribe this algorithm to a language with a limited word length for integers.