General Description of Branch and Bound
From Wikipedia's Branch and Bound:
This step is called pruning, and is usually implemented by maintaining
a global variable m (shared among all nodes of the tree) that
records the minimum upper bound seen among all subregions examined so
far. Any node whose lower bound is greater than m can be discarded.
Practical Example: Traveling Salesman Problem
A simple solution to the Traveling Salesman Problem is to keep a variable, e.g. best, that represents the shortest Hamiltonian Circuit found so far (upper bound).
Every time we consider a new step in a potential new circuit, we compute the cost of the path at the current point, e.g. cost, which is a lower bound on the cost of this path, and compare it to the best variable. If at any point cost >= best, we need not consider this path; we may prune all potential paths that begin with this subpath.
This is not difficult to implement in a procedural language such as C where we can create a variable that is in the scope of the function. For example,
int best = -1; // no best path yet
node* solveTSP() {
// best can be consulted and has a consistent
// value across all recursive calls to solveTSP()
...
}
My Actual Question
It is not clear to me how such an algorithm would be implemented purely functionally. Is there a way to simulate the global variable m mentioned in wikipedia's definition?
I know that it is simple to have a global variable in Lisp, but is this possible in a more purely-functional language such as Haskell?
You simply pass the current best value as an additional argument to recursive calls and also return the updated best value as the a part of their results. So if you have a function of type a -> b, it becomes something like a -> Int -> (b, Int). Instead of reading the global variable, you read the additional argument, and instead of writing it, you return a modified value when the function finishes.
This can be nicely expressed using the state monad. Type Int -> (b, Int) is isomorphic to State Int b, so our function of type a -> b becomes a -> State Int b.
Computation with mutable state is no more powerful than computation without it. One can be reduced to the other.
In particular, a mutable cell in a functional viewpoint becomes a sequence of successive values, usually in some arrangement where new copies shadow older ones (for example, in tail recursion.)
Related
I have a DFA (Q, Σ, δ, q0, F) with some “don't care transitions.” These transitions model symbols which are known not to appear in the input in some situations. If any such transition is taken, it doesn't matter whether the resulting string is accepted or not.
Is there an algorithm to compute an equivalent DFA with a minimal amount of states? Normal DFA minimisation algorithms cannot be used as they don't know about “don't care” transitions and there doesn't seem to be an obvious way to extend the algorithms.
I think this problem is NP-hard (more on that in a bit). This is what I'd try.
(Optional) Preprocess the input via the usual minimization algorithm with accept/reject/don't care as separate outcomes. (Since don't care is not equivalent to accept or reject, we get the Myhill–Nerode equivalence relation back, allowing a variant of the usual algorithm.)
Generate a conflict graph as follows. Start with all edges between accepting and rejecting states. Take the closure where we iteratively add edges q1—q2 such that there exists a symbol s for which there exists an edge σ(q1, s)—σ(q2, s).
Color this graph with as few colors as possible. (Or approximate.) Lots and lots of coloring algorithms out there. PartialCol is a good starting point.
Merge each color class into a single node. This potentially makes the new transition function multi-valued, but we can choose arbitrarily.
With access to an alphabet of arbitrary size, it seems easy enough to make this reduction to coloring run the other way, proving NP-hardness. The open question for me is whether having a fixed-size alphabet constrains the conflict graph in such a way as to make the resulting coloring instance easier somehow. Alas, I don't have time to research this.
I believe a slight variation of the normal Moore algorithm works. Here's a statement of the algorithm.
Let S be the set of states.
Let P be the set of all unordered pairs drawn from S.
Let M be a subset of P of "marked" pairs.
Initially set M to all pairs where one state is accepting and the other isn't.
Let T(x, c) be the transition function from state x on character c.
Do
For each pair z = <a, b> in P - M
For each character c in the alphabet
If <T(a, c), T(b, c)> is in M
Add z to M and continue with the next z
Until no new additions to M
The final set P - M is a pairwise description of an equivalence relation on states. From it you can create a minimum DFA by merging states and transitions of the original.
I believe don't care transitions can be handled by never marking (adding to M) pairs based on them. That is, we change one line:
If T(a, c) != DC and T(b, c) != DC and <T(a, c), T(b, c)> is in M
(Actually in an implementation, no real algorithm change is needed if DC is a reserved value of type state that's not a state in the original FA.)
I don't have time to think about a formal proof right now, but this makes intuitive sense to me. We skip splitting equivalence classes of states based on transitions that we know will never occur.
The thing I still need to prove to myself is whether the set P - M is still a pairwise description of an equivalence relation. I.e., can we end up with <a,b> and <b,c> but not <a,c>? If so, is there a fixup?
Background: I am currently working on an 8-puzzle implementation of the original A Star algorithm and comparing this with a slightly modified algorithm which intends to improve node expansion (using additional information, ofcourse A Star in an equally informed unidirectional search has been proven optimal). The Open List of nodes are currently ordered by their F values (G-Cost+ H-Cost).
So in implementing this in Java I have set up a comparator which orders the List by their F-Values in ascending order.
#Override
public int compare(PNode o1, PNode o2) {
return Integer.compare(o1.costF, o2.costF);
}
Question: My question is whether:
Is it permitted under A-star to implement a further Tie-breaker in A-Star using the Heuristic cost (H-Value) to break any deadlock with the F-Value among many nodes (where nodes exist with the same F-value in the Open List they will be ordered by the Heuristic value (H-Cost) in ascending order instead). The reason I am confused about this as the actual A star algorithm pseudocode only sorts the Open List by the F value.
Extending the same code above, the implementation will be:
#Override
public int compare(PNode o1, PNode o2) {
if (o1.costF == o2.costF) {
return Integer.compare(o1.costH, o2.costH);
}
return Integer.compare(o1.costF, o2.costF);
}
If this is permitted, are there any reasons why I should be wary of doing this? Logically I appreciate that ordering the Open List will be more costly. But from my experiments the difference do not seem significant enough to prevent me from using this.
Thanks so much everyone~
Yes it is allowed, pretty much for the reasons stated in mcdowella's answer.
However, I'd like to add that it is often actually a very good idea, and much more beneficial than implied in that answer. This kind of tie-breaker can result in much more significant gains in performance than only finding the goal slightly earlier. See, for example, this page, which visualizes how A* still explores a rather massive area without the tie-breaker, and only a very narrow band along the optimal path with the tie-breaker.
Intuitively, you can understand that it is so effective by thinking of the different levels of "reliability" of G costs versus H costs. G costs are very reliable, they are ground truth, they are precisely the cost you have really already had to pay to reach a node. H costs are much less reliable, they are heuristics, they can be wildly wrong. By implementing the tie-breaker you propose, you are essentially saying "whenever two nodes have the same value for F = G + H, I prefer those with greater G and smaller H over those with smaller G and greater H". This is wise because, when the more reliable G component of F dominates the less reliable H component, F itself will also be more reliable.
Another major advantage is, as described on the page I linked to, that this tie-breaker can avoid exploring large portions of lots of different paths that are all equal and all optimal.
I think this is OK for two reasons
1) You are only changing the behaviour in the case of ties, so all you are doing is selecting one possible execution path from a larger set of execution paths which are possible with the original version.
2) You preserve the property that if you retrieve the goal node from the open List, every other node in the open has G-Cost + H-Cost at least as expensive as that of the node you have just retrieved, and so must lead to a path to the goal node at least as expensive as the node you have just retrieved.
By favoring nodes with low heuristic cost in the case of ties, you are favoring any goal nodes in the case of ties, so I guess you might retrieve the goal node slightly earlier and so finish slightly earlier.
the heurstic can be changed to guarntee less points have equal F
[ this article ] 1
The quick hack to work around this problem is to either adjust the g or h values. The tie breaker needs to be deterministic with respect to
the vertex (i.e., it shouldn’t be a random number), and it needs to
make the f values differ. Since A* sorts by f value, making them
different means only one of the “equivalent” f values will be
explored.
One way to break ties is to nudge the scale of h slightly. If we scale
it downwards, then f will increase as we move towards the goal.
Unfortunately, this means that A* will prefer to expand vertices close
to the starting point instead of vertices close to the goal. We can
instead scale h upwards slightly (even by 0.1%). A* will prefer to
expand vertices close to the goal.
dx1 = current.x - goal.x
dy1 = current.y - goal.y
dx2 = start.x - goal.x
dy2 = start.y - goal.y
cross = abs(dx1*dy2 - dx2*dy1)
heuristic += cross*0.001
I'm working on an application for which I want to take the set C of all the possible k-combinations of elements in M (with ||M|| = m), and cover C with the sets of k-combinations of subsets N_i of M, with ||N_i|| = n < m ∀ N_i
So there are (m choose k) combinations to cover, and each set Q_i of n elements will contain (n choose k) combinations.
What I'd like is an algorithm that constructs the sets Qi such that q is minimized (i.e., as close to (m choose k) / (n choose k) as possible)
So, for example, if m=100, k=3, and n=10, I would want the smallest set of sets of 10 elements such that their respective sets of 3-combinations covered the set of (100 choose 3) 3-combinations of M.
I'm not sure if this will help or not, but I have written a class to handle common functions for working with the binomial coefficient, which is the type of problem that your problem falls under. It performs the following tasks:
Outputs all the K-indexes in a nice format for any N choose K to a file. The K-indexes can be substituted with more descriptive strings or letters. This method makes solving this type of problem quite trivial.
Converts the K-indexes to the proper index of an entry in the sorted binomial coefficient table. This technique is much faster than older published techniques that rely on iteration. It does this by using a mathematical property inherent in Pascal's Triangle. My paper talks about this. I believe I am the first to discover and publish this technique, but I could be wrong.
Converts the index in a sorted binomial coefficient table to the corresponding K-indexes.
Uses Mark Dominus method to calculate the binomial coefficient, which is much less likely to overflow and works with larger numbers.
The class is written in .NET C# and provides a way to manage the objects related to the problem (if any) by using a generic list. The constructor of this class takes a bool value called InitTable that when true will create a generic list to hold the objects to be managed. If this value is false, then it will not create the table. The table does not need to be created in order to perform the 4 above methods. Accessor methods are provided to access the table.
There is an associated test class which shows how to use the class and its methods. It has been extensively tested with 2 cases and there are no known bugs.
To read about this class and download the code, see Tablizing The Binomial Coeffieicent.
It should not be hard to convert this class to the language of your choice.
From your description of the problem, it looks like you should set up one loop for N (another for K if it changes as well), and then create a binomial coefficient object (BC) for that N,K combination. Call the unsigned long version of GetBinCoeff() with the BC object to get the total number of combinations. Then set up another loop to go through the total number of combinations of each BC object. Inside that loop, call the BC GetKIndexes method to get the K-Indexes for each index (i.e combination) and then do your calculation. I'm not exactly sure what you are trying to minimize. If my suggestion is not clear or helpful enough, try posting a more detailed example that clearly shows the results you are looking for.
I cross-posted this question on Math Overflow
It turns out that this is a well-trodden problem in combinatorics called the covering design problem.
There is, in general, no algorithm that guarantees a minimum, although there are algorithms that are pretty close to the minimum. You can find existing known coverings and research here
I am working on some revision at the moment and specifically going over Big-O notation. I have asked a similar question (which dealt with a different algorithm) but am still unsure if I am going the right way about it or not.
The algorithm that I am looking at is Exhaustive Search (aka Brute Force, I believe) and looks like this:
Input: G- the graph
n- the current node
p– the path so far
1) For every edge nm (from n to m) in G do
2) If m ∉ p then
3) p = p ∪ {m}
4) Exhaustive(G, m, p)
5) End If
6) End For
So far I have come to the result that this algorithm is O(n) - is this correct? I doubt that it is, and would love to know exactly how to go about working it out; what to look for, what exactly it is that I 'count' each time, etc. I understand that the number of operations taking place need to be counted, but is that all that I need to take note of/count?
EDIT: I have learned that this algorithm is, in fact, O((n-1)!) - is this correct and if so, how did this solution come about as I cannot work it out?
Usually (but not always) with graphs, the input size n is the number of nodes in the graph. It's fairly easy to prove to ourselves that the function (let alone the runtime) is called at least n times - a single path through a graph (assuming it's connected, that is, every node is reachable from every other node via some path) will take `n' calls.
To compute running time of recursive functions, an upper bound on the running time will be the number of times the recursive function is called multiplied by the runtime of the function in a single call.
To see that the worst case runtime is O((n-1)!), consider how many paths are in a fully connected graph - you can visit any node directly from any node. Another way of phrasing this is that you can visit the nodes in any order, save the starting state. This is the same as the number of permutations of (n-1) elements. I believe it's actually going to be O(n!), since we are iterating over all edges which takes O(n) for each state on the path (n*(n-1)!). EDIT: More precisely, we can say it's big-omega(N!). See comments for more details.
Sometimes, it's easier to look at what the algorithm computes than the actual code - that is, the cardinality of all the states (more specificity here, paths).
from Mexico. The truth is almost never asked or open new issues, because really the forum and not only this, if not to work instead of the network, you can find plenty of information about topic x or y, however this time I feel very defeated.
I have two years of recursion.
Define the following recursive algorithms.
a. Calculate the next n integers.
At first not referred to the master with this is that if the algorithm returns a sum, or set of numbers. Furthermore, although in principle and algorithm design for the second case is asked to resolve by its expression as a recurrence relation ... this is where I am more than lost, not how to express this as a RR. And that can be solved
b. Calculate the minimum of a set of integers
In the other case suppose that calls for the minimum of a set of integers. that's solved, but the fact and pass it to a RR fix, has left me completely flooded.
APPRECIATE ANY HELP, thanks
Answering on b)
You have a set of integers. You pick one and you know that minimal element is either that you've picked or the minimal is still in the set. Recursivly you call function unless you pick all elements from set, you assume that minimum of set that contain no elements is infinity. Then your recurrence is going back updateing the minimal value.
minimum (S) = min(any element, minimum(Rest of S))
if (S is empty) then minimum(empty) = infinity.
Not an implementation in any language cause surely depend on representation of set.
P.S why doing this recursivly?