I wonder if the objective function of a general dynamic programming problem can always be formulated as in dynamic programming on wiki, where the objective function is a sum of items for action and state at every stage? Or that is just a specical case and what is the general formulation?
EDIT:
By "dynamic programming problem", I mean a problem that can be solved by dynamic programming technique. Such kind of problems possess the property of optimal problem and optimal structure.
But at lease for me it is sometimes not easy to identify such problems, perhaps because I have not become used to that kind of verbal description. When I came across the WIKI page for Bellman equation, I do feel mathematical formulation of the cost function will help somehow. I suspect the overall cost/gain function can always be represented as accumulation of cost/gain from all the stages? and the accumulation can be additive or multiplitive or something else?
When I posted my question, I did realize that it is more proper to discuss dynamic programming in some place more oriented to mathematical optimization. But there are quite a lot of discussion of computer algorithms in Stackoverflow.com. So I did not feel improper to ask my question here either.
That's not how I would characterize an arbitrary optimization problem (or a dynamic programming algorithm). In particular, the factor βt looks like an electrical engineering hack that programmers wouldn't usually want. More subtly, it seems like it won't always be obvious what the function F is for a given problem.
But yes, set β to 1 and any arbitrary objective function can be formulated that way. Generally the objective function may be any function of the initial state and all the actions taken; given such a function, it's easy to define a function F to plug into that formula.
Whether that's a useful thing to do or not depends on the problem, I suppose.
in computer science dynamic programming denotes the building of any algorithm in terms of recursively splitting it into subproblems when the same subproblems appear many times in this recursive expansion. A simple book example, Fibonacci numbers can be calculated using dynamic programming:
From the generic recurrence F(n) = F(n-1) + F(n-2) you could implement the following algorithm:
int fibonacci(n):
if (n < 2): return 1
else: return fibonacci(n-1) + fibonacci(n-2)
Now this is of course not efficient at all, because it creates a huge number of recursive calls, e.g.
F(8) = F(7) + F(6) = [F(6) + F(5)] + [F(5) + F(4)] = ...
So here we already see that fibonacci(5) is computed twice by the implementation. The dynamic programming paradigm is now to "memoize" or "cache" the results, like this:
integer_map store;
int memofibo(n):
if (n < 2) : return 1
else if (store.find_key(n)): return store.find_value(n)
else:
int f = memofibo(n-1) + memofibo(n-2)
store.set(n, f)
return f
This implementation ensure that the recursive step is executed at most once for every argument value of n, so it calculates the nth Fibonacci number in O(n log n) time (assuming standard O(log n)) implementation of the associative array 'store'.
So from the computer science perspective, the link you provided is the operations research / optimization problem version of the same idea (dividing problem into subproblems), but the idea has been abstracted in practice to this recursion + memoization pattern in the domain of general computer science. I hope this helps to clear some of the clouds.
Folks,
There's a new(ish) website that concentrates on operations research questions here but the low volume of traffic there may not get you a good answer very quickly.
Soapbox time:
For those who care to debate on what's appropriate for stack overflow, let us note that an algorithm is an algorithm regardless of who claims it as part of their field. The simplex method, Djikstra's method, branch and bound, lagrangian relaxation, are all algorithms or methods of solving certain types of problems. Many of these are taught and applied in both fields so the border between OR and CS is pretty blurry in this area.
For instance (and a very strong instance it is) the undergrad course in algorithms at MIT includes all of the following - Randomized Competitive Algorithm, Dynamic Programming, Greedy Algorithms, Minimum Spanning Trees, Shortest Paths, Dijkstra's Algorithm, Bellman-Ford, Linear Programming, Depth-first Search, Topological Sort, and All-pairs Shortest Paths among other topics. I'll defer to MIT in this case.
I like stack overflow because many programmers recognize an optimization problem when they encounter it, but often they just need a little help in deciding how to formulate the problem or even what the problem is called by name.
Related
Let’s use as an example the problem LeetCode 322. Coin Change
I know it is best solved by using Dynamic Programming, but I want to focus on my Brute Force solution:
class Solution:
def coinChange(self, coins: List[int], amount: int) -> int:
curr_min = float('inf')
def helper(amount):
nonlocal curr_min
if amount < 0:
return float('inf')
if amount == 0:
return 0
for coin in coins:
curr_min = min(curr_min, helper(amount-coin) + 1)
return curr_min
ans = helper(amount)
return -1 if ans == float('inf') else ans
The Recursion Tree looks like: Recursion Tree
I can say it is Divide and Conquer: We are dividing the problem into smaller sub-problems, solving individually and using those individual results to construct the result for the original problem.
I can also say it is Backtracking: we are enumerating all combinations of coin frequencies which satisfy the constraints.
I know both are implemented via Recursion, but I would like to know which paradigm my Brute Force solution belongs to: Divide and Conquer or Backtracking.
A complication in categorizing your algorithm is that there aren’t clear, well-defined boundaries between different classes of algorithms and different people might have slightly different definitions in mind.
For example, generally speaking, divide-and-conquer algorithms involve breaking the problem apart into non-overlapping subproblems. (See, for example, mergesort, quicksort, binary search, closest pair of points, etc.) In that sense, your algorithm doesn’t nicely map onto the divide-and-conquer paradigm, since the subproblems you’re considering involve some degree of overlap in the subproblems they solve. (Then again, not all divide-and-conquer algorithms have this property. See, for example, stoogesort.)
Similarly, backtracking algorithms usually, but not always, work by committing to a decision, recursively searching to see whether a solution exists given that decision, then unwinding the choice if it turns out not to lead to a solution. Your algorithm doesn’t have this property, since it explores all options and then takes the best. (When I teach intro programming, I usually classify algorithms this way. But my colleagues sometimes describe what you’re doing as backtracking!)
I would classify your algorithm as belonging to a different family of exhaustive search. The algorithm you’ve proposed essentially works by enumerating all possible ways of making change, then returning the one that uses the fewest coins. Exhaustive search algorithms are ones that work by trying all possible options and returning the best, and I think that’s the best way of classifying your strategy.
To me this doesn't fit with either paradigm.
Backtracking to me is associated with reaching a point where the candidate cannot be further developed, but here we develop it to it's end, infinity, and we don't throw it away, we use it in comparisons.
Divide and conquer I associate with a division into a relatively small number of candidate groups (the classic example is two, like binary search). To call each path in a recursion a group for the sake of Divide and Conquer would lose the latter's meaning.
The most practical answer is it doesn't matter.
Safest answer recursion. My best interpretation is that its backtracking.
I think the options here are recursion, backtracking, divide-and-conquer, and dynamic programming.
Recursion being the most general and encapsulating of backtracking, D&C, and DP. If indeed it has backtracking and D&C algorithms then recursion would be the best answer as it contains both.
In Skiena's ADM (Section 5.3.1), it says:
A typical divide-and-conquer algorithm breaks a given problem into a smaller pieces, each of which is of size n/b.
By this interpretation is doesn't meet the as we divide our solution by coins and each coin amount being a different size.
In Erickson's Algorithms (section 1.6), it says:
divide and conquer:
Divide the given instance of the problem into several independent smaller instances of exactly the same problem.
So in this case, according to the recursion tree, are not always independent (they overlap).
Which leaves backtracking. Erickson defines the 'recursive strategy' as:
A backtracking algorithm tries to construct a solution to a computational problem incrementally, one small piece at a time.
Which seems general enough to fit all DP problems under it. The provided code can be said it backtracks when a solution path fails.
Additionally, according to Wikipedia:
It is often the most convenient technique for parsing, for the knapsack problem and other combinatorial optimization problems.
Coin Change being an Unbounded Knapsack type problem, then it fits into the description of backtracking.
I reading about Dynamic Programming. I read that to be able to get good at it, needs practice and intuition but this advice seems to general to me.
The hardest part for me is to figure out a recursive formula. Sometimes the formula used in the solution does not seem that intuitive to me.
For example I read the problem following problem:
You have 2 strings S and T. Give an algorithm to find the number of
times S appears in T. It is not mandatory for all characters of S to
appear contiguous in T.
The solutions is based on the following recurrence formula, which for me is not intuitive at all:
Assume M(i, j) represents the count of how many times i characters of
S appear in j characters of T. Base cases:
i) 0 if j = 0
ii) 1 if i = 0
I was wondering is there some kind of "analysis" of the problem that helps define/find the proper recurrence formula for a solving a problem via DP?
It's described very well on the Wikipedia, here:
"In mathematics, computer science, and economics, dynamic programming is a method for solving complex problems by breaking them down into simpler sub-problems. It is applicable to problems exhibiting the properties of overlapping sub-problems and optimal substructure (described below). When applicable, the method takes far less time than naive methods."
Read also about overlapping sub-problems and optimal substructure.
Maybe this link gets useful:
https://www.geeksforgeeks.org/solve-dynamic-programming-problem/
https://www.geeksforgeeks.org/tabulation-vs-memoization/
1. How to classify a problem as a Dynamic Programming Problem?
2. Deciding the state
3. Formulating a relation among the states
4. Adding memoization or tabulation for the state
Whenever I look at solutions to computer contests, I always see the term "dynamic programming". I Googled the term and read a few articles, but none of them provide a simple example of programming VS "dynamic" programming. So how is "dynamic" programming different than "normal" programming? (simple terms please!)
Dynamic Programming uses programming more in the sense used with Linear Programming -- a mechanism of solving a problem.
One description I recently read (but can no longer recall the source -- [citation needed]) suggested that the usual approach of divide and conquer used in recursion is a top-down approach to solving problems, while dynamic programming is a bottom-up approach to solving problems.
The Wikipedia article suggests computing the Fibonocci sequence is an excellent use of dynamic programming -- you memoize results as you compute them for further use in the algorithm, to avoid re-computing similar results.
Knuth's algorithm for line-breaking paragraphs is another good example of dynamic programming: if you consider the possibility of inserting line breaks between every word (and even breaking lines inside words, at hyphenation points), it feels like the only algorithms will be exponential -- or worse. However, by keeping track of the "badness" associated with previous line breaks, Knuth's algorithm actually runs in linear time with the size of the input. (I must admit that I don't fully understand Knuth's algorithm -- only that it is supremely clever.)
By "normal" programming, I think you mean C++/Java/C# programming, right?
Dynamic programming is not "programming" in that sense. It is not about writing code, but the word "programming" there is used in the context of solving complex problems by breaking them down into simpler problems.
Dynamic programming is not really 'programming' but table lookup [and storage] - That is sacrifice a bit of space to improve time complexity [quite a bit].
I knew this is an old post but I was having the same questions so I am answering myself here.
Dynamic programming has two properties:
Optimal substructure: a globally optimal solution can be found by combining optimal solutions to local subproblems. For example, fib(x) = fib(x - 1) + fib(x – 2)
Overlapping subproblems: finding an optimal solution
involves solving the same problem multiple times. For example, compute fib(x) many times in Fibonocci sequence
By the above definition/properties, it is not clear whether certain questions like "Does an element belong to a set"? or "How to find the sum of a set" can be classified as dynamic programming? I can divide the set into subset (solve globally) and add it up (get global answer). Also, within the subset, I did the summation many times.
I found a paragraph in a book and I think it provides very helpful tips to distinguish "Dynamic Programming"(DP) and "Divide and conquer algorithm"(D&C).
In D&C subproblems, they are substantially smaller then the original problem. In contrast, DP involves solving problems that are only slightly smaller than the original problem. For example, computing Fib(19) is not a substantially smaller problem than computing Fib(20). While compute sum of ten elements is substantially smaller than sum of ten million elements.
Efficiency of D&C algorithms does not depend upon structuring the algorithm so that identical problems are solved repeatedly. In contrast, DP is efficient only when the number of distinct subproblems is significantly smaller than the total number of subproblems.
I heard the only difference between dynamic programming and back tracking is DP allows overlapping of sub problems, e.g.
fib(n) = fib(n-1) + fib (n-2)
Is it right? Are there any other differences?
Also, I would like know some common problems solved using these techniques.
There are two typical implementations of Dynamic Programming approach: bottom-to-top and top-to-bottom.
Top-to-bottom Dynamic Programming is nothing else than ordinary recursion, enhanced with memorizing the solutions for intermediate sub-problems. When a given sub-problem arises second (third, fourth...) time, it is not solved from scratch, but instead the previously memorized solution is used right away. This technique is known under the name memoization (no 'r' before 'i').
This is actually what your example with Fibonacci sequence is supposed to illustrate. Just use the recursive formula for Fibonacci sequence, but build the table of fib(i) values along the way, and you get a Top-to-bottom DP algorithm for this problem (so that, for example, if you need to calculate fib(5) second time, you get it from the table instead of calculating it again).
In Bottom-to-top Dynamic Programming the approach is also based on storing sub-solutions in memory, but they are solved in a different order (from smaller to bigger), and the resultant general structure of the algorithm is not recursive. LCS algorithm is a classic Bottom-to-top DP example.
Bottom-to-top DP algorithms are usually more efficient, but they are generally harder (and sometimes impossible) to build, since it is not always easy to predict which primitive sub-problems you are going to need to solve the whole original problem, and which path you have to take from small sub-problems to get to the final solution in the most efficient way.
Dynamic problems also requires "optimal substructure".
According to Wikipedia:
Dynamic programming is a method of
solving complex problems by breaking
them down into simpler steps. It is
applicable to problems that exhibit
the properties of 1) overlapping
subproblems which are only slightly
smaller and 2) optimal substructure.
Backtracking is a general algorithm
for finding all (or some) solutions to
some computational problem, that
incrementally builds candidates to the
solutions, and abandons each partial
candidate c ("backtracks") as soon as
it determines that c cannot possibly
be completed to a valid solution.
For a detailed discussion of "optimal substructure", please read the CLRS book.
Common problems for backtracking I can think of are:
Eight queen puzzle
Map coloring
Sudoku
DP problems:
This website at MIT has a good collection of DP problems with nice animated explanations.
A chapter from a book from a professor at Berkeley.
One more difference could be that Dynamic programming problems usually rely on the principle of optimality. The principle of optimality states that an optimal sequence of decision or choices each sub sequence must also be optimal.
Backtracking problems are usually NOT optimal on their way! They can only be applied to problems which admit the concept of partial candidate solution.
Say that we have a solution tree, whose leaves are the solutions for the original problem, and whose non-leaf nodes are the suboptimal solutions for part of the problem. We try to traverse the solution tree for the solutions.
Dynamic programming is more like BFS: we find all possible suboptimal solutions represented the non-leaf nodes, and only grow the tree by one layer under those non-leaf nodes.
Backtracking is more like DFS: we grow the tree as deep as possible and prune the tree at one node if the solutions under the node are not what we expect.
Then there is one inference derived from the aforementioned theory: Dynamic programming usually takes more space than backtracking, because BFS usually takes more space than DFS (O(N) vs O(log N)). In fact, dynamic programming requires memorizing all the suboptimal solutions in the previous step for later use, while backtracking does not require that.
DP allows for solving a large, computationally intensive problem by breaking it down into subproblems whose solution requires only knowledge of the immediate prior solution. You will get a very good idea by picking up Needleman-Wunsch and solving a sample because it is so easy to see the application.
Backtracking seems to be more complicated where the solution tree is pruned is it is known that a specific path will not yield an optimal result.
Therefore one could say that Backtracking optimizes for memory since DP assumes that all the computations are performed and then the algorithm goes back stepping through the lowest cost nodes.
IMHO, the difference is very subtle since both (DP and BCKT) are used to explore all possibilities to solve a problem.
As for today, I see two subtelties:
BCKT is a brute force solution to a problem. DP is not a brute force solution. Thus, you might say: DP explores the solution space more optimally than BCKT. In practice, when you want to solve a problem using DP strategy, it is recommended to first build a recursive solution. Well, that recursive solution could be considered also the BCKT solution.
There are hundreds of ways to explore a solution space (wellcome to the world of optimization) "more optimally" than a brute force exploration. DP is DP because in its core it is implementing a mathematical recurrence relation, i.e., current value is a combination of past values (bottom-to-top). So, we might say, that DP is DP because the problem space satisfies exploring its solution space by using a recurrence relation. If you explore the solution space based on another idea, then that won't be a DP solution. As in any problem, the problem itself may facilitate to use one optimization technique or another, based on the problem structure itself. The structure of some problems enable to use DP optimization technique. In this sense, BCKT is more general though not all problems allow BCKT too.
Example: Sudoku enables BCKT to explore its whole solution space. However, it does not allow to use DP to explore more efficiently its solution space, since there is no recurrence relation anywhere that can be derived. However, there are other optimization techniques that fit with the problem and improve brute force BCKT.
Example: Just get the minimum of a classic mathematical function. This problem does not allow BCKT to explore the state space of the problem.
Example: Any problem that can be solved using DP can also be solved using BCKT. In this sense, the recursive solution of the problem could be considered the BCKT solution.
Hope this helps a bit.
In a very simple sentence I can say: Dynamic programming is a strategy to solve optimization problem. optimization problem is about minimum or maximum result (a single result). but in, Backtracking we use brute force approach, not for optimization problem. it is for when you have multiple results and you want all or some of them.
Depth first node generation of state space tree with bounding function is called backtracking. Here the current node is dependent on the node that generated it.
Depth first node generation of state space tree with memory function is called top down dynamic programming. Here the current node is dependant on the node it generates.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 2 years ago.
Improve this question
I am trying to compare 2 algorithms. I thought I may try and write a proof for them. (My math sucks, so hence the question.)
Normally in our math lesson last year we would be given a question like <can't use symbols in here so left them out>.
Prove: (2r + 3) = n (n + 4)
Then I would do the needed 4 stages and get the answer at the end.
Where I am stuck is proving prims and Kruskals - how can I get these algorithms in to a form like the mathmatical one above, so I can proceed to prove?
Note: I am not asking people to answer it for me - just help me get it in to a form where I can have a go myself.
To prove the correctness of an algorithm, you typically have to show (a) that it terminates and (b) that its output satisfies the specification of what you're trying to do. These two proofs will be rather different from the algebraic proofs you mention in your question. The key concept you need is mathematical induction. (It's recursion for proofs.)
Let's take quicksort as an example.
To prove that quicksort always terminates, you would first show that it terminates for input of length 1. (This is trivially true.) Then show that if it terminates for input of length up to n, then it will terminate for input of length n+1. Thanks to induction, this is sufficient to prove that the algorithm terminates for all input.
To prove that quicksort is correct, you must convert the specification of comparison sorting to precise mathematical language. We want the output to be a permutation of the input such that if i ≤ j then ai ≤ aj. Proving that the output of quicksort is a permutation of the input is easy, since it starts with the input and just swaps elements. Proving the second property is a little trickier, but again you can use induction.
You don't give many details but there is a community of mathematicians (Mathematical Knowledge Management MKM) who have developed tools to support computer proofs of mathematics. See, for example:
http://imps.mcmaster.ca/
and the latest conference
http://www.orcca.on.ca/conferences/cicm09/mkm09/
Where i am stuck is proving prims and Kruskals - how can i get these algorithms in to a form like the mathmatical one above so i can proceed to prove
I don't think you can directly. Instead, prove that both generate a MST, then prove that any two MST are equal ( or equivalent, since you can have more than one MST for some graphs ). If both algorithms generate MSTs which are shown to be equivalent, then the algorithms are equivalent.
From my maths classes at Uni I (vaguely) remember proving Prims and Kruskals algorithms - and you don't attack it by writing it in a mathematical form. Instead, you take proven theories for Graphs and combine them e.g. http://en.wikipedia.org/wiki/Prim%27s_algorithm#Proof_of_correctness to build the proof.
If your looking to prove the complexity, then simply by the working of the algorithm it's O(n^2). There are some optimisations for the special case where the graph is sparse which can reduce this to O(nlogn).
Most of the times the proof depends on the problem you have in your hand. Simple argument can be suffice at times, at some other times you might need rigorous proof. I once used a corollary and proof of already proved theorem to justify my algorithm is right. But that is for a college project.
Maybe you want to try out a semi-automatic proof method. Just to go for something different ;) For example, if you have a Java specification of Prim's and Kruskal's algorithms, optimally building upon the same graph model, you can use the KeY Prover to prove the equivalence of the algorithm.
The crucial part is to formalize your proof obligation in Dynamic Logic (this is an extension of first-order logic with types and means of symbolic execution of Java programs). The formula to prove could match the following (sketchy) pattern:
\forall Graph g. \exists Tree t.
(<{KRUSKAL_CODE_HERE}>resultVar1=t) <-> (<{PRIM_CODE_HERE}>resultVar2=t)
This expresses that for all graphs, both algorithms terminate and the result is the same tree.
If you're lucky and your formula (and algorithm implementations) are right, then KeY can prove it automatically for you. If not, you might need to instantiate some quantified variables which makes it necessary to inspect the previous proof tree.
After having proven the thing with KeY, you can either be happy about having learned something or try to reconstruct a manual proof from the KeY proof - this can be a tedious task since KeY knows a lot of rules specific to Java which are not easy to comprehend. However, maybe you can do something like extracting an Herbrand disjunction from the terms that KeY used to instantiate existential quantifiers at the right-hand side of sequents in the proof.
Well, I think that KeY is an interesting tool and more people should get used to prove critical Java code using tools like that ;)