In the question
Calculating How Many Balls in Bins Over Several Values Using Dynamic Programming
the answer discusses a dynamic programming algorithm for placing balls into bins, and I was attempting to determine the running time, as it is not addressed in the answer.
A quick summary: Given M indistinguishable balls and N distinguishable bins, the entry in the dynamic programming table Entry[i][j] represents the number of unique ways i balls can be placed into j bins.
S[i][j] = sum(x = 0 -> i, S[i-x][j-1])
It is clear that the size of the dynamic programming 2D array is O(MN). However, I am trying to determine the impact the summation has on the running time.
I know a summation of values (1....x) means we must access values from 1 to x. Would this then mean, that for each entry computation, since we must access at most from 1...M other values, the running time is in the realm of O((M^2)N)?
Would appreciate any clarification. Thanks!
You can avoid excessive time for summation if you keep column sums in additional table.
When you calculate S[i][j], also fill Sums[i,j]=Sums[i-1,j] + S[i,j] and later use this value for the cell at right side S[i,j+1]
P.S. Note that you really need to store only two rows or even one row of sum table
Related
I've been trying to wrap my head around the fact that a lot of DP questions that involve bottom up tabulation via a 2D Matrix can be simplified into a 1D Array to save on space since you only rely on the previous two rows but I don't really understand the why/how/intuition behind this.
Just wondering if anyone could offer the most dumb downed version of why this works...
Generally, we use can apply DP when the optimal solution to a given problem can be determined from the optimal solutions of its subproblems. When coming up with a solution to some algorithm, it usually helps to come up with a recursive one first. From there, if we observe that recursive subproblems are re-calculated multiple times, we can just memoize the intermediate results for fast reference to them later.
In some special cases, we don't actually need to remember the solution to all subproblems at once; we just need to know a certain subset at a time.
The space optimization described above seems to best answer to the question you're asking - how does one condense the total set of solutions as a 2D matrix into a single 1D array? Well, at a given time, we don't actually store all solutions (the 2D matrix) in any single point in time; we just store what is needed to calculate the next round of intermediate/final outputs in the algorithm.
Perhaps walking through an example application may help reinforce this description.
A nice example is the generalized stock trading problem. Basically, we have an input array prices of a stock on a given day and would like to calculate the maximum profit that can be earned if k buy-sell transactions are made, where one stock may be bought and held at any given time.
The trickiest part in my opinion is figuring out how to move from the one transaction case to the two transaction case. I'll assume we're proficient enough in dynamic programming to move immediately to the k transaction case. Notice that a nice formulation of the problem in terms of subproblems is the following:
prices = input array of prices, length is n
Define dp[k][i] = maximum profit earned by day i, after having made k transactions
dp[k][i] = max(dp[k][i - 1], (prices[i] + effectivePrice)
effectivePrice = max(dp[k - 1][i] - prices[i], effectivePrice) (compute on the fly for each i)
Now in this particular case our "naive" dp solution has a 2D matrix with k rows and n columns. The space reduction here is that, in order to calculate the result for k transactions, we only need knowledge of the case for k - 1 transactions. Therefore, it is certainly possible to solve the problem using two 1D arrays of size n.
Let oldDp = solution for the k - 1 case
Let newDp = solution for the k case (computed on the fly)
for each transaction:
for each day i:
newDp[i] = max(newDp[i - 1], (prices[i] + effectivePrice)
effectivePrice = max(oldDp[i] - prices[i], effectivePrice)
// Set up for next iteration
oldDp = newDp
newDp = blank array of size n
As we can see, we managed to save a lot of space - we went from having to use a 2D matrix with k rows and n columns to two 1D arrays of size n. An even better optimization is to just use a single 1D array; this is possible since the only indices that we examine in oldDp is the current one i when calculating effectivePrice. Because we only need to temporarily remember the old result for day i, we can just make use of a temporary variable. Thus, the optimized pseudocode (for our "naive" approach) appears as below:
Let dp = maximum profit, so that dp[i] = maximum profit after k transactions (built iteratively) on day i.
for each transaction:
for each day i:
// At this point, dp[i] is equivalent to dp[k - 1][i], yet
// for all j < i, dp[j] is equivalent to dp[k][j]!
temp = dp[i]
dp[i] = max(dp[i - 1], prices[i] + effectivePrice)
effectivePrice = max(temp - prices[i], effectivePrice)
And so, using the "naive" idea of determining the optimal solution after k transactions from k - 1 transactions, we optimize space by going from a 2D matrix of size kn to a 1D array of size n.
Imagine you have N distinct people and that you have a record of where these people are, exactly M of these records to be exact.
For example
1,50,299
1,2,3,4,5,50,287
1,50,299
So you can see that 'person 1' is at the same place with 'person 50' three times. Here M = 3 obviously since there's only 3 lines. My question is given M of these lines, and a threshold value (i.e person A and B have been at the same place more than threshold times), what do you suggest the most efficient way of returning these co-occurrences?
So far I've built an N by N table, and looped through each row, incrementing table(N,M) every time N co occurs with M in a row. Obviously this is an awful approach and takes 0(n^2) to O(n^3) depending on how you implent. Any tips would be appreciated!
There is no need to create the table. Just create a hash/dictionary/whatever your language calls it. Then in pseudocode:
answer = []
for S in sets:
for (i, j) in pairs from S:
count[(i,j)]++
if threshold == count[(i,j)]:
answer.append((i,j))
If you have M sets of size of size K the running time will be O(M*K^2).
If you want you can actually keep the list of intersecting sets in a data structure parallel to count without changing the big-O.
Furthermore the same algorithm can be readily implemented in a distributed way using a map-reduce. For the count you just have to emit a key of (i, j) and a value of 1. In the reduce you count them. Actually generating the list of sets is similar.
The known concept for your case is Market Basket analysis. In this context, there are different algorithms. For example Apriori algorithm can be using for your case in a specific case for sets of size 2.
Moreover, in these cases to finding association rules with specific supports and conditions (which for your case is the threshold value) using from LSH and min-hash too.
you could use probability to speed it up, e.g. only check each pair with 1/50 probability. That will give you a 50x speed up. Then double check any pairs that make it close enough to 1/50th of M.
To double check any pairs, you can either go through the whole list again, or you could double check more efficiently if you do some clever kind of reverse indexing as you go. e.g. encode each persons row indices into 64 bit integers, you could use binary search / merge sort type techniques to see which 64 bit integers to compare, and use bit operations to compare 64 bit integers for matches. Other things to look up could be reverse indexing, binary indexed range trees / fenwick trees.
There are n (n < 1000) groups of friends, with the size of the group being characterized by an array A[] (2 <= A[i] < 1000). Tables are present such that they can accommodate r(r>2) people at a time. What is the minimum number of tables needed for seating everyone, subject to the constraint that for every person there should be another person from his/her group sitting at his/her table.
The approach I was thinking was to break every group into sizes of twos and threes and try to solve this problem, but there are many ways of dividing a number n into groups of twos and threes and not all of them may be optimal.
Does a Mixed Integer Programming model count?
Some notes on this formulation:
I used random data to form the groups.
x(i,j) is the number of people of group i sitting at table j.
x(i,j) is a semi-integer variable, that is: it is an integer variable with values zero or between LO and UP. Not all MIP solvers offer semi-continuous and semi-integer variables but it may come handy. Here I use it to enforce that at least 2 persons from the same group need to sit at a table. If a solver does not offer these type of variables, we can formulate this construct using additional binary variables as well.
y(j) is a binary variable (0 or 1) indicating if a table is used.
the capacity equation is somewhat smart: if a table is not used (y(j)=0) its capacity is reduced to zero.
the option optcr=0 indicates we want to solve to optimality. For large, difficult problems we may want to stop say at 5%.
the order equation makes sure we start filling tables from table 1. This also reduces the symmetry of the problem and may speed up solution times.
the above model (with 200 groups and 200 potentially used tables) generates a MIP problem with 600 equations (rows) and 40k variables (columns). There are 37k integer variables. With a good MIP solver we find the proven optimal solution (with 150 tables used) in less than a minute.
Notice this is certainly not a knapsack problem (as suggested in another answer -- a knapsack problem has just one constraint) but it resembles a bin-packing problem.
It is same problem as knapsack problem which is NP complete (see https://en.wikipedia.org/wiki/Bin_packing_problem ). So finding optimal solution is pretty hard.
A heuristic that works most of the time:
Sort the groups according decreasing size.
For each group put it in the table that has least amount of space, but still can accommodate this group.
Your approach is workable. If a solution exists for a given number of tables, then a solution exists where you've split every group into some number of twos and some number of threes. First, split a three off of every group of odd size. You're left with a bunch of groups of even size. Next, split twos off of every group whose size isn't divisible by six. And forget that it's one bigger group; split it into a bunch of groups of six.
At this point, you have split all of your groups into some number of twos, some number of threes, and some number of sixes. Give each table of odd size one three, splitting sixes as necessary; now all tables have even size. All remaining sixes can now be split into twos and seated arbitrarily.
I am trying to find a dynamic approach to multiply each element in a linear sequence to the following element, and do the same with the pair of elements, etc. and find the sum of all of the products. Note that any two elements cannot be multiplied. It must be the first with the second, the third with the fourth, and so on. All I know about the linear sequence is that there are an even amount of elements.
I assume I have to store the numbers being multiplied, and their product each time, then check some other "multipliable" pair of elements to see if the product has already been calculated (perhaps they possess opposite signs compared to the current pair).
However, by my understanding of a linear sequence, the values must be increasing or decreasing by the same amount each time. But since there are an even amount of numbers, I don't believe it is possible to have two "multipliable" pairs be the same (with potentially opposite signs), due to the issue shown in the following example:
Sequence: { -2, -1, 0, 1, 2, 3 }
Pairs: -2*-1, 0*1, 2*3
Clearly, since there are an even amount of pairs, the only case in which the same multiplication may occur more than once is if the elements are increasing/decreasing by 0 each time.
I fail to see how this is a dynamic programming question, and if anyone could clarify, it would be greatly appreciated!
A quick google for define linear sequence gave
A number pattern which increases (or decreases) by the same amount each time is called a linear sequence. The amount it increases or decreases by is known as the common difference.
In your case the common difference is 1. And you are not considering any other case.
The same multiplication may occur in the following sequence
Sequence = {-3, -1, 1, 3}
Pairs = -3 * -1 , 1 * 3
with a common difference of 2.
However this is not necessarily to be solved by dynamic programming. You can just iterate over the numbers and store the multiplication of two numbers in a set(as a set contains unique numbers) and then find the sum.
Probably not what you are looking for, but I've found a closed solution for the problem.
Suppose we observe the first two numbers. Note the first number by a, the difference between the numbers d. We then count for a total of 2n numbers in the whole sequence. Then the sum you defined is:
sum = na^2 + n(2n-1)ad + (4n^2 - 3n - 1)nd^2/3
That aside, I also failed to see how this is a dynamic problem, or at least this seems to be a problem where dynamic programming approach really doesn't do much. It is not likely that the sequence will go from negative to positive at all, and even then the chance that you will see repeated entries decreases the bigger your difference between two numbers is. Furthermore, multiplication is so fast the overhead from fetching them from a data structure might be more expensive. (mul instruction is probably faster than lw).
the problem requires us to find out the number of ways of placing R coins on a N*M grid such that each row and column has at least one coin. Constraints given are N , M < 200 , R < N*M. I initially thought of backtracking, but i was made to realise that it would never finish in time . Can someone guide me to another solution? (DP , closed form formula.) any pointers would be nice. Thanks.
Answer
According to OEIS sequence A055602 one possible solution to this is:
Let a(m, n, r) = Sum_{i=0..m} (-1)^i*binomial(m, i)*binomial((m-i)*n, r)
Answer = Sum_{i=0..N} (-1)^i*binomial(N, i)*a(M, N-i, R)
You will need to evaluate N+1 different values for a.
Assuming you have precomputed binomial coefficients, each evaluation of a is O(M) so the total complexity is O(NM).
Interpretation
This formula can be derived using the inclusion-exclusion principle twice.
a(m,n,r) is the number of ways of putting r coins on a grid of size m*n such that every one of the m columns is occupied, but not all the rows are necessarily occupied.
Inclusion-Exclusion turns this into the correct answer. (The idea is that we get our first estimate from a(M,N,R). This overestimates the correct answer because not all rows are occupied so we subtract cases a(M,N-1,R) where we only occupy N-1 rows. This then underestimates so we need to correct again...)
Similarly we can compute a(m,n,r) by considering b(m,n,r) which is the number of ways of placing r coins on a grid where we don't care about rows or columns being occupied. This can be derived simply from the number of ways of choosing r places in a grid size m*n , i.e. binomial(m*n,r). We use IE to turn this into the function a(m,n,r) where we know that all columns are occupied.
If you want to allow different conditions on the number of coins on each square, then you can just change b(m,n,r) to the appropriate counting function.
This is tough, but if you begin by working out how many ways you can have at least one coin on each row and column (call them reserve coins). The answer will be the product of #1 (n! / r! (n - r)!) *, where #2 n = N*M - NUMBER_OF_RESERVE_COINS and #3 r = (R - NUMBER_OF_RESERVE_COINS) for #4 each arrangement of reserving one coin on each row/column.
#4 is where the trickier stuff takes place. For N*M where N!=M, abs(N-M) tells you how many reserve coins will be on a single rows/columns. I'm having trouble on identifying the correct way of proceeding to the next step, mainly due to lack of time (though I can return to this on the weekend), but I hope I have provided you with useful information, and if what I have said is correct that you will be able to complete the process.