I am reading back tracking algorithm design technique in Introduction to Design and analysis of Algorithms by Anany Levtin.
I am having tough time in understanding generic definition of back-tracking algorithm and map it to 4 queen problem.
For back-tracking algorithm design technique from a more general
perspective, most backtracking algorithms fit the following
description.
An output of a backtracking algorithm can be thought of as an n-tuple
(x1, x2, x3,...,xn) where each coordinate xi is an element of some
finite linearly ordered set Si. For example, for the n-queens
problem, each Si is the set of integers 1 through n. The tuple may
need to satisfy some additional constraints (e.g., the nonattacking
requirments in the n-queens problem).
For example for 4 queen problem each Si is set {1, 2, 3, 4}
A back-tracking algorithm generates explicitly or implicityly, a
state-space tree, its nodes represent partially constructed tuples
with the first "i" coordinates defined by the earlier actions of the
algorithm. If such a tuple (x1, x2, x3,.. xi) is not a solution, the
algorithm finds the next element in Si+1 that is consistent with the
values of (x1, x2, x3...xi) and the problems constraints and adds it
to the tuple as its (i+1)st coordinate. If such an element doesnot
exist, the algorithm backtracks to consider the next value of xi, and
so on.
My questions on above text are
What does author mean by "A back-tracking algorithm generates explicitly or implicityly, a state-space tree, its nodes represent partially constructed tuples with the first "i"
coordinates defined by the earlier actions of the algorithm. If such a tuple (x1, x2, x3,.. xi) is not a solution, the algorithm finds the next element
in Si+1 that is consistent with the values of (x1, x2, x3...xi) and the problems constraints and adds it to the tuple as its (i+1)st coordinate." ?
To be specific what does author mean by the algorithm finds next element in Si+1?
Kindly request to explain above statement with 4 queen problem.
If element does not exist the algorithm back track to consider next value of xi? Please explain this stament with 4 queen problem.
Thanks!
This explanation of backtracking is indeed hard to follow, as it uses a lot of formal notation, which is not used for a specific purpose or prove. Let me try to explain it in a less formal way with the 4-queens problem, which is an excellent example:
At the start of the backtracking process, the board is empty. This is the root of the state-space tree. This root has child nodes, one for each possibility where we could put the first queen. Rather than constructing each child node before we go to the next level (that would result in a BFS of the state-space-tree), we choose one possible position for the first queen, thereby constructing one child node. From this child node we have again multiple options to place the second queen, so we choose one, and so on.
If we arrive at the node where we have no possibilities to put a remaining queen we backtrack - we go up one level to that nodes father-node and check if there are possibilities left that we did not explore yet. If yes we explore them by creating the respective child node, if no we backtrack again, going up another level, until we either found a solution to the problem (all queens are placed) or we expanded all children of the root node and arrived at the root node during a backtrack operation - that means there is no solution.
Related
I came upon wait-for graphs and I wonder, are there any efficient algorithms for detecting if adding an edge to a directed graph results in a cycle?
The graphs in question are mutable (they can have nodes and edges added or removed). And we're not interested in actually knowing an offending cycle, just knowing there is one is enough (to prevent adding an offending edge).
Of course it'd be possible to use an algorithm for computing strongly connected components (such as Tarjan's) to check if the new graph is acyclic or not, but running it again every time an edge is added seems quite inefficient.
If I understood your question correctly, then a new edge (u,v) is only inserted if there was no path from v to u before (i.e., if (u,v) does not create a cycle). Thus, your graph is always a DAG (directed acyclic graph). Using Tarjan's Algorithm to detect strongly connected components (http://en.wikipedia.org/wiki/Tarjan%27s_strongly_connected_components_algorithm) sounds like an overkill in this case. Before inserting (u,v), all you have to check is whether there is a directed path from v to u, which can be done with a simple BFS/DFS.
So the simplest way of doing it is the following (n = |V|, m = |E|):
Inserting (u,v): Check whether there is a path from v to u (BFS/DFS). Time complexity: O(m)
Deleting edges: Simply remove them from the graph. Time complexity: O(1)
Although inserting (u,v) takes O(m) time in the worst case, it is probably pretty fast in your situation. When doing the BFS/DFS starting from v to check whether u is reachable, you only visit vertices that are reachable from v. I would guess that in your setting the graph is pretty sparse and that the number of vertices reachable by another is not that high.
However, if you want to improve the theoretical running time, here are some hints (mostly showing that this will not be very easy). Assume we aim for testing in O(1) time whether there exists a directed path from v to u. The keyword in this context is the transitive closure of a DAG (i.e., a graph that contains an edge (u, v) if and only if there is a directed path from u to v in the DAG). Unfortunately, maintaining the transitive closure in a dynamic setting seems to be not that simple. There are several papers considering this problem and all papers I found were STOC or FOCS papers, which indicates that they are very involved. The newest (and fastest) result I found is in the paper Dynamic Transitive Closure via Dynamic Matrix Inverse by Sankowski (http://dl.acm.org/citation.cfm?id=1033207).
Even if you are willing to understand one of those dynamic transitive closure algorithms (or even want to implement it), they will not give you any speed up for the following reason. These algorithms are designed for the situation, where you have a lot of connectivity queries (which then can be performed in O(1) time) and only few changes in the graph. The goal then is to make these changes cheaper than recomputing the transitive closure. However, this update is still slower that a single check for connectivity. Thus, if you need to do an update on every connectivity query, it is better to use the simple approach mentioned above.
So why do I mention this approach of maintaining the transitive closure if it does not fit your needs? Well, it shows that searching an algorithm consuming only O(1) query time does probably not lead you to a solution faster than the simple one using BFS/DFS. What you could try is to get a query time that is faster than O(m) but worse than O(1), while updates are also faster than O(m). This is a very interesting problem, but it sounds to me like a very ambitious goal (so maybe do not spend too much time on trying to achieve it..).
As Mark suggested it is possible to use data structure that stores connected nodes. It is the best to use boolean matrix |V|x|V|. Values can be initialized with Floyd–Warshall algorithm. That is done in O(|V|^3).
Let T(i) be set of vertices that have path to vertex i, and F(j) set of vertices where exists path from vertex j. First are true's in i'th row and second true's in j'th column.
Adding an edge (i,j) is simple operation. If i and j wasn't connected before, than for each a from T(i) and each b from F(j) set matrix element (a,b) to true. But operation isn't cheap. In worst case it is O(|V|^2). That is in case of directed line, and adding edge from end to start vertex makes all vertices connected to all other vertices.
Removing an edge (i,j) is not so simple, but not more expensive operation in the worst case :-) If there is a path from i to j after removing edge, than nothing changes. That is checked with Dijkstra, less than O(|V|^2). Vertices that are not connected any more are (a,b):
a in T(i) - i - T(j),
b in F(j) + j
Only T(j) is changed with removing edge (i,j), so it has to be recalculated. That is done by any kind of graph traversing (BFS, DFS), by going in opposite edge direction from vertex j. That is done in less then O(|V|^2). Since setting of matrix element is in worst case is again O(|V|^2), this operation has same worst case complexity as adding edge.
This is a problem which I recently faced in a slightly different situation (optimal ordering of interdependent compiler instructions).
While I can't improve on O(n*n) theoretical bounds, after a fair amount of experimentation and assuming heuristics for my case (for example, assuming that the initial ordering wasn't created maliciously) the following was the best compromise algorithm in terms of performance.
(In my case I had an acceptable "right side failure": after the initial nodes and arcs were added (which was guaranteed to be possible), it was acceptable for the optimiser to occasionally reject the addition of further arcs where one could actually be added. This approximation isn't necessary for this algorithm when carried to completion, but it does admit such an approximation if you wish to do so, and so limiting its runtime further).
While a graph is topologically sorted, it is guaranteed to be cycle-free. In the first phase when I had a static bulk of nodes and arcs to add, I added the nodes and then topologically sorted them.
During the second phase, adding additional arcs, there are two situations when considering an arc from A to B. If A already lies to the left of B in the sort, an arc can simply be added and no cycle can be generated, as the list is still topologically sorted.
If B is to the left of A, we consider the sub-sequence between B and A and partition it into two disjoint sequences X, Y, where X is those nodes which can reach A (and Y the others). If A is not reachable from B, ie there are no direct arcs from B into X or to A, then the sequence can be reordered XABY before adding the A to B arc, showing it is still cycle-free and maintaining the topological sort. The efficiency over the naive algorithm here is that we only need consider the subsequence between B and A as our list is topologically sorted: A is not reachable from any node to the right of A. For my situation, where localised reorderings are the most frequent and important, this an important gain.
As we don't reorder within the sequences X,A,B,Y, clearly any arcs which start or end within the same sequence are still ordered correctly, and the same in each flank, and any "fly-over" arcs from the left to the right flanks. Any arcs between the flanks and X,A,B,Y are also still ordered correctly as our reordering is restricted to this local region. So we only need to consider arcs between our four sequences. Consider each possible "problematic" arc for our final ordering XABY in turn: YB YA YX BA BX AX. Our initial order was B[XY]A, so AX and YB cannot occur. X reaches A, but Y does not, therefore YX and YA do not occur or A could be reached from the source of the arc in Y (potentially via X) a contradiction. Our criterion for acceptability was that there are no links BX or BA. So there are no problematic arcs, and we are still topologically sorted.
Our only acceptability criterion (that A is not reachable from B) is clearly sufficient to create a cycle on adding the arc A->B: B -(X)-> A -> B, so the converse is also shown.
This can be implemented reasonably efficiently if we can add a flag to each node. Consider the nodes [BXY] going right-to-left from the node immediately to the left of A. If that node has a direct arc to A then set the flag. At an arbitrary such node, we need only consider direct outgoing arcs: the nodes to its right are either after A (and so irrelevant), or else have already been flagged if reachable from A, so the flag on such an arbitrary node is set when any flagged nodes are encountered by direct link. If B is not flagged at the end of the process, the reordering is acceptable and the flagged nodes comprise X.
Though this always yields a correct ordering if carried to completion (as far as I can tell), as I mentioned in the introduction it is particularly efficient if your initial build is approximately correct (in the sense of accommodating of likely additional arcs without reordering).
There also exists an effective approximation, if your context is such that "outrageous" arcs can be rejected (those which would massively reorder) by limiting the A to B distance you are prepared to scan. If you have an initial list of the additional arcs you wish to add, they can be ordered by increasing distance in the initial ordering until you run out of some scanning "credit", and call your optimisation a day at that point.
If the graph is directed, you would only have to check the parent nodes (navigate up until you reach the root) of the node where the new edge should start. If one of the parent nodes is equal to the end of the edge, adding the edge would create a cycle.
If all previous jobs are in Topologically sorted order. Then if you add an edge that appears to brake the sort, and can not be fixed, then you have a cycle.
https://stackoverflow.com/a/261621/831850
So if we have a sorted list of nodes:
1, 2, 3, ..., x, ..., z, ...
Such that each node is waiting for nodes to its left.
Say we want to add an edge from x->z. Well that appears to brake the sort. So we can move the node at x to position z+1 which will fix the sort iif none of the nodes (x, z] have an edge to the node at x.
I want to ask a few questions about the algorithm in this paper.
https://pdfs.semanticscholar.org/9208/65aee7b072c054afa628ba413adda6f3e962.pdf
You may wonder why I don't ask the authors directly, actually I have, but unfortunately, I've got no response and I couldn't find direct emails to the authors.
You can see the core formula (8) of the algorithm.
d(n) = weight value from starting point to current node
r(n) = constraint function
ɷ = weighted value denoting the impact factor
θn = angle between vector that consists of nodes from starting point to current node1 and vector that consist of nodes from starting point to end point[2]
The steps said that to obtain the best nodes, all subsequent nodes vp (open list) will be compared to satisfy the constraint function above.
What I don't understand is:
Regarding the θn vectors, I don't really understand says, we have a graph with nodes 1,2,3,4,5,6,7,8,9,10. Suppose we start at node 1 and end at node 10, currently we have traversed node 1,2,4,5 (current node= 5). So for 1 it's supposed to be closed list (List of traversed optimal node) and the vector is v= {1,2,4,5}. Then, for [2], it's supposed to be v= {1,2,4,5,..,10}. How is that possible? For what I know, to find vector angle both vectors must have the same amount of nodes and it's weird since nodes between 5 and 10 haven't traversed yet, so it's not possible to use it. So, from my thinking, the vectors is supposed to be v1 = {1,5} and v[2] = {1,10}, but I wonder if that's even right.
About the step 3 of the algorithm, I wonder about the choosing of the best node? I can't understand the meaning of satisfying here since I can't find what condition does the compared nodes need, so it could satisfy the constraint function. Is D(n) must be the smallest or biggest among the compared list or what?
I hope anyone could answer this, and I hope this reach the author too, thanks.
I came across an interesting problem called Knapsack. You have a list of items, which all have a value and a weight. Then you have to find the combination of items that maximize the value of the objects summed, and stay within a certain limit. I saw somewhere that this is a search problem which you could use different search algorithms. Now I am trying to implement it with breadth-first.
The pseudo algorithm for BFS found on wikipedia is as follows:
Breadth-First-Search(Graph, root):
create empty set S
create empty queue Q
root.parent = NIL
Q.enqueue(root)
while Q is not empty:
current = Q.dequeue()
if current is the goal
return current
for each node n that is adjacent to current:
if n is not in S:
add n to S
n.parent = current
Q.enqueue(n)
I have really tried to understand how to apply this to the knapsack problem.
As much as I understand, it's about building a tree. I need to expand and explore each node of one level at a time. For BFS I need a FIFO queue. For each item selected, I have two choices: Either I take the item or not.
Anyway, to be specific: What I do not understand, in my context, with the pseudo code above are:
When I select an item, do I push it twice to the queue and mark one of them as used and one as not used?
How do I know if the current is the goal? I assume its something like when there are no more nodes to explore which means we are at a leaf node.. But there will be many leaf nodes, so which one do I choose and how?
What is meant with adjacent to current? If I only have a list, or an array of items (items have an ID, a weight, and a value), how do I know which is adjacent?
Say you have 4 different items. Then the graph you are searching is a hypercube like this (image by Yury Chebiryak):
The binary numbers at the nodes are all of the possible knapsacks, with a 0 in the nth place meaning item n is not in the knapsack, and a 1 meaning it is, so for example 0000 means an empty knapsack, 1001 means the knapsack containing the first item and the 4th item, and so on.
At each step you remove the current node from the queue, and if it isn't the goal, construct the adjacent nodes by finding all of the knapsacks differing from the current one by 1 item that you haven't visited already. So for example, if the current node is 1001, you would construct nodes 0001, 1101, 1011, and 1000. You then add these nodes to the queue.
The goal only has a meaning if you are looking for a "good enough" solution, rather than the best solution. To establish whether the current node is the goal you simply work out the value of the current knapsack and compare it to the goal value.
If you want the best solution, then breadth first search is not helping you because you need to explore every node in the graph. Dynamic programming or backtracking (which is a kind of Depth First Search) would allow you to reduce the search space.
If you want a "good enough" solution, then FIFO branch-and-bound or hill climbing (starting from a random knapsack) are effective ways of using breadth-first search.
Here is some context if you're unfamiliar with Knight's Travails.
Your task is to build a function knight_moves that shows the simplest possible way to get from one square to another by outputting all squares the knight will stop on along the way.
I know where to find complete solutions to this exercise, but I'm trying to mostly work through it on my own.
Where I'm stuck is how to set up a binary tree, or, specifically, what is the relationship between all the next possible moves a knight can make from its current location?
As I understand a BST, the defining property is that a tree(and any of its subtrees and leaves) store keys that are larger to the root node to the right, and smaller keys to the left of the root node.
How do I represent the value of the knight's current location and its possible(valid) moves?
It would be more valuable if the answer provided is a more of a guiding principle (philosophy?) when thinking about BST keys/values and defining parent-child relationships.
Thank you.
This is not a BST problem. Attack this as a graph search problem with backtracking. Start by learning and understanding Dijkstra's algorithm, treating the squares of the board as nodes in a graph. For instance, a1 connects to b3 and c2; all moves have a cost of 1 (which simplifies the algorithm a little).
You'll need to build your graph either before you start, connecting all possible moves; or on the fly, generating all possibilities from the current square. Most students find that doing it on the fly is easier for them.
Also, changing from algebraic chess notation to (row, col) notation helps. If the knight is currently at (row, col), then the legal next moves are
(row+-2, col +-1) and
(row+-1, col +-2)
... where x and y are both in the range 0-7 (throw out moves that take the knight past the board's edge).
Maintain a global list of squares you've already visited, and another of squares you need to visit (one move from those you've visited). You'll also need an array of least cost (number of moves) to get to each square. Initialize that to 100, and let the algorithm reduce it as it walks through the search.
Is that enough to get you moving?
Given some inputs, which consist of a left and right symbol, output chains which link the inputs.
Imagine the inputs to be dominoes which you cannot flip horizontally and need to chain them together. Creating big circular chains (ignore if you cannot physically do it with real dominos) is preferred over small circular chains which are preferred over chains where the start and end does not match.
Outputs with more circular chains (regardless of how many or chain length) are what we are looking for. For example, an output of 3 circular chains is better than 1 big chain and a leftover single domino.
Can someone point me in the right direction? What group of problems does this belong and are there existing algorithms for solving this?
Examples (outputs may be incorrect!):
in[0]=(A,B)
in[1]=(B,C)
in[2]=(C,A)
out[0]=(0,1,2)
in[0]=(A,B)
in[1]=(B,A)
in[2]=(C,D)
in[3]=(D,C)
out[0]=(0,1)
out[1]=(2,3)
in[0]=(A,B)
in[1]=(B,A)
in[2]=(C,D)
in[3]=(E,F)
out[0]=(0,1)
out[1]=(2)
out[2]=(3)
in[0]=(A,B)
in[1]=(B,A)
in[2]=(C,D)
in[3]=(D,E)
out[0]=(0,1)
out[1]=(2,3)
in[0]=(A,B)
in[1]=(B,C)
in[2]=(C,D)
out[0]=(0,1,2)
Dominoes which cannot be flipped horizontally == directed graphs.
Putting dominoes one after the other is called a "path", if it is a closed path, it's a circuit.
A circuit that includes all the vertices of a graph is a Hamiltonian circuit.
Your problem in graph theory terms is: how to split (decompose) your graph into a minimum number of subgraphs that have Hamiltonian circuits. (a.k.a. Hamiltonian graphs)
The problem as it is now is not as clearly stated as it could be - how exactly are solutions rated? What is the most important criterion? Is it the length of the longest chain? Is there a penalty for creating chains of length one?
It is often helpful in such problems to visualize the structure as a graph - say, assign a vertex (V[i]) to each tile. Then for each i, j create an edge between vertices V[i], V[j] if you can place V[i] to the left of V[j] in a chain (so if V[i] corresponds to (X, A) then V[j] corresponds to (A, Y) for some X, Y, A).
In such a graph chains are paths, cycles are closed ("circular") chains and the problem has been reduced to finding some cycle and/or path covering for a graph. This type of problems can in turn often be reduced to matching or *-flow problems (max-flow, max-cost-max-flow, min-cost-max-flow or what have you).
But before you can reduce further you have to establish the precise rules according to which one solution is determined to be "better" than another.
It is easy to check whether there exists a circular chain consisting of all dominoes. First you need to make the following directed graph G:
Nodes of G are symbols on the dominoes (A,B,C..) in your example,
For each domino (A,B) you put a directed edge from A to B.
There exists a circular chain consisting of all dominoes iff there exists a Eulerian cycle in G. To check whether there exists Eulerian cycle in G it is sufficient to check weather each node has even degree.
I'm not sure if this is really the case, but judging by your examples, your problem looks similar to the problem of decomposing a permutation into a product of disjoint cycles. Each tile (X,Y) can be seen as P(X) = Y for a permutation P. If this agrees with your assumptions, the good (or bad) news is that such decomposition is unique (up to the cycle order) and is very easy to find. Basically, you start with any tile, find a matching tile on the other hand and follow this until no matching can be found. Then you move to the next untouched point. If that's not what you are looking for, the more general graph-based solution by t.dubrownik looks like the way to go.