I've got difficulties with understanding dynamic programming, so I decided to solve some problems. I know basic dynamic algorithms like longest common subsequence, knapsack problem, but I know them because I read them, but I can't come up with something on my own :-(
For example we have subsequence of natural numbers. Every number we can take with plus or minus. At the end we take absolute value of this sum. For every subsequence find the lowest possible result.
in1: 10 3 5 4;
out1: 2
in2: 4 11 5 5 5;
out2: 0
in3: 10 50 60 65 90 100;
out3: 5
explanation for 3rd: 5 = |10+50+60+65-90-100|
what it worse my friend told me that it is simple knapsack problem, but I can't see any knapsack here. Is dynamic programming something difficult or only I have big problems with it?
As has been pointed out by amit, this algorithm can be understood as an instance of the partition problem. For a simple implementation take a look at this Python code:
def partition(A):
n = len(A)
if n == 0:
return 0
k, s = max(A), sum(A)/2.0
table = [0 if x else 1 for x in xrange(n*k)]
for i in xrange(n):
for j in xrange(n*k-1, -1, -1):
if table[j-A[i]] > table[j]:
table[j] = 1
minVal, minIdx = float('+inf'), -1
for j in xrange(int(s)+1):
if table[j] and s-j < minVal:
minVal, minIdx = s-j, j
return int(2*minVal)
When called with one of the inputs in the question:
partition([10, 50, 60, 65, 90, 100])
It will return 5, as expected. For fully understanding the math behind the solution, please take a look at this examples and click the "Balanced Partition" link.
The knapsack in here is weight = value = number for each element.
your bound W is 1/2 * sum(elements).
The idea is - you want to maximize the amount of numbers you "pick" without passing the limit of 1/2 * sum(elements), which is exactly knapsack with value=weight.
This problem is actually the partition problem, which is a special case of the subset sum problem.
The partition problem says: "Is it possible to get a subset of the elements that sums exactly to half?"
The derivation to your problem from here is simple - if there is, take these as +, and those you didn't take as -, and you get out = 0. [the other way around works the same]. Thus, your described problem is the optimization for partition problem.
This is the same problem as in Tug Of War, without the constraint of balanced team sizes (which is not relevant):
http://acm.uva.es/p/v100/10032.html
I had solved this problem with a top-down approach. It works on the constraint that there is an upper limit to the numbers given. Do you have an upper limit or are the numbers unconstrained? If they are unconstrained I don't see how to solve this with dynamic programming.
Related
Lagrange's four-square theorem proves that any natural number can be written as the sum of four square numbers. What I need is to find any one way to write a natural number x as sum of four square numbers for all 0 <= x <= N for any given upper limit N.
What I have done so far is find two-square sum representation for all the numbers <= N for which it is possible to find one, and saved them in an array called two_square_div. Then I used a greedy approach like following:
last_two_square_sum = 0
for num in 0..N
if num can be written as sum of two square
last_two_square_sum = num
other_last_two_square_sum = num - last_two_square_sum
four_square_div[num] = (two_square_div[last_two_square_sum], two_square_div[other_last_two_square_sum]
But this approach does not work for numbers like 23, for which last_two_square_sum = 20 other_last_two_square_sum = 3. But 3 can not be written as sum of two squares so this method fails.
So could anybody provide a correct O(N) solution or any helpful hint? Thank you.
Your algorithm should make more than one attempt (if it already does, then the exit condition must be improved).
23 can be written as 3 + 20, yes; but 3 is not a decomposable of order two and can't lead to a solution.
So you go on: next you try 4 + 19, and this time it's 19 that is rejected. Next you try 5, so 23-5 is 18, and 5 is 12 + 22 while 18 is 32 + 32.
(Of course this is not O(N) at all).
It is not clear to me how you arrive at 20 and not accept previous solutions; try posting the whole of the code.
Also, try asking on Math StackExchange.
I have a problem statement which says: if you have an array of elements {x1,x2,x3,...x10}, find the combination of elements such that it just sums up above a threshold value (say the threshold value is 100).
So if there exists a combination like x2+x5+x8 = 105, x3+x5+x8=103, and x4+x5 = 101, then the algorithm should output X4, X5.
The knapsack algorithm emits a value that is near but on the lesser side of the threshold (which is 100 here). I want the opposite, that is the smallest sum of selected elements that is greater than 100.
Is there any set of algorithms or any special case of any algorithm which might solve this problem?
I'll start out by noting that you are asking for the smallest value strictly greater than some target. In general "strictly greater than" and "strictly less than" constraints are much harder than "greater than or equal to" or "less than or equal to" constraints. If you have all integer values, then you could simply translate your constraint "the sum exceeds 100" to "the sum is greater than or equal to 101". I'll assume that you've made such a transformation for the rest of the problem.
One approach would be to treat this as an integer optimization problem, in which the binary decision variable y_i for each number is whether or not we include it. Then our goal is to minimize the sum of the numbers, which can be modeled as:
min x_1*y_1 + x_2*y_2 + ... + x_n*y_n
The constraint in this case is that the sum of the numbers is at least 100:
x_1*y_1 + x_2*y_2 + ... + x_n*y_n >= 100
In general this is a hard problem (note that it is at least as hard as the subset sum problem, which is NP-complete). However modern optimization solvers may be efficient enough for your problem instances.
To test the scalability of a free solver for this problem, consider the following implementation with the lpSolve package in R (it returns the selected subset if the problem is feasible and NA otherwise):
library(lpSolve)
min.subset <- function(x, min.sum) {
mod <- lp("min", x, matrix(x, nrow=1), ">=", min.sum, all.bin=TRUE)
if (mod$status == 0) {
which(mod$solution >= 0.999)
} else {
NA
}
}
min.subset(1:10, 43.5)
# [1] 2 3 4 5 6 7 8 9
min.subset(1:10, 88)
# [1] NA
To test the scalability, I'll select n elements randomly from [1, 2, ..., 1000], setting the target sum to be half the sum of the elements. The runtimes were:
With n=100, it ran in 0.01 seconds
With n=1000, it ran in 0.1 seconds
With n=10000, it ran in 8.7 seconds
It appears you can solve this problem for more than 10k elements (with the selected distribution) without too many computational challenges. If your problem is too big for the free solver I've used here, you might consider Gurobi or cplex, two commercial solvers that are free for academic use but otherwise not free.
Suppose X is the sum of all x_i. Then equivalently, you are asking for a minimum subset of your x_i that sum up to at most X - 100 (as the complement of these x_i will be the optimum solution to your problem). So all Knapsack theory can be applied here.
In practice (really large instances), I'd suggest this form of Nemhauser-Ullman generalization which can solve instances with millions of objects.
I am having issues with understanding dynamic programming solutions to various problems, specifically the coin change problem:
"Given a value N, if we want to make change for N cents, and we have infinite supply of each of S = { S1, S2, .. , Sm} valued coins, how many ways can we make the change? The order of coins doesn’t matter.
For example, for N = 4 and S = {1,2,3}, there are four solutions: {1,1,1,1},{1,1,2},{2,2},{1,3}. So output should be 4. For N = 10 and S = {2, 5, 3, 6}, there are five solutions: {2,2,2,2,2}, {2,2,3,3}, {2,2,6}, {2,3,5} and {5,5}. So the output should be 5."
There is another variation of this problem where the solution is the minimum number of coins to satisfy the amount.
These problems appear very similar, but the solutions are very different.
Number of possible ways to make change: the optimal substructure for this is DP(m,n) = DP(m-1, n) + DP(m, n-Sm) where DP is the number of solutions for all coins up to the mth coin and amount=n.
Minimum amount of coins: the optimal substructure for this is
DP[i] = Min{ DP[i-d1], DP[i-d2],...DP[i-dn] } + 1 where i is the total amount and d1..dn represent each coin denomination.
Why is it that the first one required a 2-D array and the second a 1-D array? Why is the optimal substructure for the number of ways to make change not "DP[i] = DP[i-d1]+DP[i-d2]+...DP[i-dn]" where DP[i] is the number of ways i amount can be obtained by the coins. It sounds logical to me, but it produces an incorrect answer. Why is that second dimension for the coins needed in this problem, but not needed in the minimum amount problem?
LINKS TO PROBLEMS:
http://comproguide.blogspot.com/2013/12/minimum-coin-change-problem.html
http://www.geeksforgeeks.org/dynamic-programming-set-7-coin-change/
Thanks in advance. Every website I go to only explains how the solution works, not why other solutions do not work.
Lets first talk about the number of ways, DP(m,n) = DP(m-1, n) + DP(m, n-Sm). This in indeed correct because either you can use the mth denomination or you can avoid it. Now you say why don't we write it as DP[i] = DP[i-d1]+DP[i-d2]+...DP[i-dn]. Well this will lead to over counting , lets take an example where n=4 m=2 and S={1,3}. Now according to your solution dp[4]=dp[1]+dp[3]. ( Assuming 1 to be a base case dp[1]=1 ) .Now dp[3]=dp[2]+dp[0]. ( Again dp[0]=1 by base case ). Again applying the same dp[2]=dp[1]=1. Thus in total you get answer as 3 when its supposed to be just 2 ( (1,3) and (1,1,1,1) ). Its so because
your second method treats (1,3) and (3,1) as two different solution.Your second method can be applied to case where order matters, which is also a standard problem.
Now to your second question you say that minimum number of denominations can
be found out by DP[i] = Min{ DP[i-d1], DP[i-d2],...DP[i-dn] } + 1. Well this is correct as in finding minimum denominations, order or no order does not matter. Why this is linear / 1-D DP , well although the DP array is 1-D each state depends on at most m states unlike your first solution where array is 2-D but each state depends on at most 2 states. So in both case run time which is ( number of states * number of states each state depends on ) is the same which is O(nm). So both are correct, just your second solution saves memory. So either you can find it by 1-D array method or by 2-D by using the recurrence
dp(n,m)=min(dp(m-1,n),1+dp(m,n-Sm)). (Just use min in your first recurrence)
Hope I cleared the doubts , do post if still something is unclear.
This is a very good explanation of the coin change problem using Dynamic Programming.
The code is as follows:
public static int change(int amount, int[] coins){
int[] combinations = new int[amount + 1];
combinations[0] = 1;
for(int coin : coins){
for(int i = 1; i < combinations.length; i++){
if(i >= coin){
combinations[i] += combinations[i - coin];
//printAmount(combinations);
}
}
//System.out.println();
}
return combinations[amount];
}
This question already has answers here:
What is the fastest (known) algorithm to find the n-th Catalan number mod m?
(2 answers)
Closed 8 years ago.
in how many ways you can sum the numbers less or equal with N to be equal with n. What is the algorithm to solve that?
Example:
lets say that we have
n =10;
so there are a lot of combinations but for example we can do:
1+1+1+1+1+1+1+1+1+1 = 10
1+2+1+1+1+1+1+1+1=10
1+1+2+1+1+1+1+1+1=10
.....
1+9=10
10=10
8+2=10
and so on.
If you think is the Catalan questions, the answer is: the problem seems to be Catalan problem but is not. If you take a look to the results you will see that lets say for N=5 In Catalan algorithm you have 14 possibilities. But in right answer you have 2^4=16 possibilities if you count all, or the Fibonacci array if you keep only the unique combinations. Eg N=5 we have 8 possibilities, so the Catalan algorithm doesn't verify.
This was a question received by me in a quiz done for fun, at that time i thought that the solution is a well known formula, so i lost a lot of time trying to remember it :)
I found 2 solutions for this problem and 1 more if you are considering only the unique combinations. Eg 2+8 is the same as 8+2, you are considering only 1 of them.
So what is the algorithm to solve it?
This is an interesting problem. I do not have the solution (yet), but I think this can be done in a divide-and-conquer way. If you think of the problem space as a binary tree, you can generate it like this:
The root is the whole number n
Its children are floor(n/2) and ceil(n/2)
Example:
n=5
5
/ \
2 3
/ \ / \
1 1 1 2
/ \
1 1
If you do this recursively, you get a binary tree. If can then traverse the tree in this manner to get all the possible combinations of summing up to n:
get_combinations(root_node)
{
combinations=[]
combine(combinations, root_node.child_left, root_node.child_right)
}
combine(combinations, nodeA, nodeB)
{
new_combi = "nodeA" + "+nodeB"
combinations.add(new_combi)
if nodeA.has_children(): combinations.add( combine(combinations, nodeA.child_left, nodeA.child_right) + "+nodeB" )
if nodeB.has_children(): combinations.add( "nodeA+" + combine(combinations, nodeB.child_left, nodeB.child_right) )
return new_combi
}
This is just a draft. Of yourse you don't have to explicitly generate the tree beforehand, but you can do that along the way. Maybe I can come up with a nicer algorithm if I find the time.
EDIT:
OK, I didn't quite answer OPs question to the point, but I don't like to leave stuff unfinished, so here I present my solution as a working python program:
import math
def print_combinations(n):
for calc in combine(n):
line = ""
count = 0
for op in calc:
line += str(int(op))
count += 1
if count < len(calc):
line += "+"
print line
def combine(n):
p_comb = []
if n >= 1: p_comb.append([n])
if n >1:
comb_left = combine(math.floor(n/float(2)))
comb_right = combine(math.ceil(n/float(2)))
for l in comb_left:
for r in comb_right:
lr_merge = []
lr_merge.extend(l)
lr_merge.extend(r)
p_comb.append(lr_merge)
return p_comb
You can now generate all possible ways of summing up n with numbers <= n. For example if you want to do that for n=5 you call this: print_combinations(5)
Have fun, be aware though that you run into memory issues pretty fast (dynamic programming to the rescue!) and that you can have equivalent calculations (e.g. 1+2 and 2+1).
All the 3 solutions that I fount use Math induction:
solution 1:
if n =0 comb =1
if n =1 comb = 1
if n=2 there are 1+1, 2 comb =2 = comb(0)+comb(1)
if n=3 there are 1+1+1, 1+2, 2+1, 3 comb = 4 = comb(0)+comb(1)+comb(2)
if n=4 there are 1+1+1+1, 1+2+1,1+1+2,2+1+1,2+2,1+3,3+1,4 comb = 8 =comb(0)+comb(1)+comb(2)+comb(3)
Now we see a pattern here that says that:
at k value we have comb(k)= sum(comb(i)) where i between 0 and k-1
using math induction we can prove it for k+1 that:
comb(k+1)= sum(comb(i)) where is is between 0 and k
Solution number 2:
If we pay a little more attention to the solution 1 we can say that:
comb(0)=2^0
comb(1)=2^0
comb(2)=2^1
comb(3)=2^2
comb(4)=2^3
comb(k)=2^(k-1)
again using the math induction we can prove that
comb(k+1)=2^k
Solution number 3 (if we keep only the unique combinations) we can see that:
comb(0)=1
comb(1)=1
comb(2)= 1+1,2=2
comb(3)= 1+1+1, 1+2, 2+1, 3 we take out 1+2 because we have 2+1 and its the same comb(3)=3
comb(4) = 1+1+1+1, 1+2+1,1+1+2,2+1+1,2+2,1+3,3+1,4, here we take out the 1+2+1,,2+1+1 and 1+3 because we have them but in different order comb(4)= 5.
If we continue we can see that:
comb(5) = 8
comb(6)=13
we now can see the pattern that:
comb (k) = comb (k-1) + comb(k-2) the Fibonacci array
again using Math induction we can prove that for k+1
comb(k+1) = comb(k)+comb(k-1)
now it's easy to implement those solutions in a language using recursion for 2 of the solutions or just the non recursive method for the solution with 2^k.
And by the way this has serious connections with graph theory (how many sub-graphs you can build starting from a bigger graph - our number N, and sub-graphs being the ways to count )
Amazing isn't it?
Say S = 5 and N = 3 the solutions would look like - <0,0,5> <0,1,4> <0,2,3> <0,3,2> <5,0,0> <2,3,0> <3,2,0> <1,2,2> etc etc.
In the general case, N nested loops can be used to solve the problem. Run N nested loop, inside them check if the loop variables add upto S.
If we do not know N ahead of time, we can use a recursive solution. In each level, run a loop starting from 0 to N, and then call the function itself again. When we reach a depth of N, see if the numbers obtained add up to S.
Any other dynamic programming solution?
Try this recursive function:
f(s, n) = 1 if s = 0
= 0 if s != 0 and n = 0
= sum f(s - i, n - 1) over i in [0, s] otherwise
To use dynamic programming you can cache the value of f after evaluating it, and check if the value already exists in the cache before evaluating it.
There is a closed form formula : binomial(s + n - 1, s) or binomial(s+n-1,n-1)
Those numbers are the simplex numbers.
If you want to compute them, use the log gamma function or arbitrary precision arithmetic.
See https://math.stackexchange.com/questions/2455/geometric-proof-of-the-formula-for-simplex-numbers
I have my own formula for this. We, together with my friend Gio made an investigative report concerning this. The formula that we got is [2 raised to (n-1) - 1], where n is the number we are looking for how many addends it has.
Let's try.
If n is 1: its addends are o. There's no two or more numbers that we can add to get a sum of 1 (excluding 0). Let's try a higher number.
Let's try 4. 4 has addends: 1+1+1+1, 1+2+1, 1+1+2, 2+1+1, 1+3, 2+2, 3+1. Its total is 7.
Let's check with the formula. 2 raised to (4-1) - 1 = 2 raised to (3) - 1 = 8-1 =7.
Let's try 15. 2 raised to (15-1) - 1 = 2 raised to (14) - 1 = 16384 - 1 = 16383. Therefore, there are 16383 ways to add numbers that will equal to 15.
(Note: Addends are positive numbers only.)
(You can try other numbers, to check whether our formula is correct or not.)
This can be calculated in O(s+n) (or O(1) if you don't mind an approximation) in the following way:
Imagine we have a string with n-1 X's in it and s o's. So for your example of s=5, n=3, one example string would be
oXooXoo
Notice that the X's divide the o's into three distinct groupings: one of length 1, length 2, and length 2. This corresponds to your solution of <1,2,2>. Every possible string gives us a different solution, by counting the number of o's in a row (a 0 is possible: for example, XoooooX would correspond to <0,5,0>). So by counting the number of possible strings of this form, we get the answer to your question.
There are s+(n-1) positions to choose for s o's, so the answer is Choose(s+n-1, s).
There is a fixed formula to find the answer. If you want to find the number of ways to get N as the sum of R elements. The answer is always:
(N+R-1)!/((R-1)!*(N)!)
or in other words:
(N+R-1) C (R-1)
This actually looks a lot like a Towers of Hanoi problem, without the constraint of stacking disks only on larger disks. You have S disks that can be in any combination on N towers. So that's what got me thinking about it.
What I suspect is that there is a formula we can deduce that doesn't require the recursive programming. I'll need a bit more time though.