I saw a lot of posts about backtracking explanations on SO, but the part which still confuses me is when do I actually use the backtracking template(choose-explore-unchoose) as opposed to a normal DFS mindset while solving problems? I understand that both of them are essentially backtracking, but from the problem solving perspective, a traditional DFS approach feels a lot more intuitive when you see a problem of that sort. But, I wanted to know when your mind should go "choose-explore-unchoose".
For example: if you want to print all root-to-leaf paths, a simple DFS problem solving approach makes a lot of sense, whereas while printing permutations of a string, we take take the "choose-explore-unchoose" strategy. I am having a difficult time to categorize problems into these 2 buckets. (Categorization is the current strategy im using to make my mind think in a certain direction. If there is any other strategy to solve, anyone is welcome to share.)
Here is the template I am referring to:
function dfs(node, state):
if state is a solution:
report(state) # e.g. add state to final result list
return
for child in children:
if child is a part of a potential solution:
state.add(child) # make move
dfs(child, state)
state.remove(child) # backtrack
As you said, DFS is essentially backtracking: choose (child), explore (all his children) and unchoose (move to next child).
There might be some other different approaches for backtracking, but that it depends more on heuristics. If you look for graph algorithms (like "A-star" family algorithms) you will see a lot of backtracking algorithm-styles that differs from DFS. The difference is what heuristics they rely on, and order of search: just like with small change you can get BFS instead of DFS.
There is no "one rule" to know when to use when. Whatever is more intuitive to you is fine, as long as you know how to use all of your toolbox/
Related
I have to determine a way for a robot to get out of a maze. The thing is that the layout of the maze is unknown, and the position of the exit is unknown too. The robot also start at an unknown position in the maze.
I found 3 solutions but I have a hard time knowing which one should I use, because in the end it seems that the solutions will purely be random anyway.
I have those 3 solutions :
1) The basic "human" strategy(?), where you put your hand on a wall and go through all the maze if necessary. I also keep a variable "turn counter" to avoid situation where the robot loop.
2) Depth first search
3) Making the robot choose direction randomly
The random one seems the worse, because he could take forever to find the exit (but on the other hand he could be the fastest too...). I'm not sure about the other two though.
Also, is there a way to have some kind of heuristic? Again the lack of information makes me think that it's impossible, but maybe I'm missing something.
Last thing : When the robot find the exit, he will have to go back to his start position using A*. This means that during the first part, where he looks for the exit, he will have draw a map of the maze that he will use for the 2nd part. Maybe this can help too chose the best algorithm for the first part, but yeah I don't see why one would be better.
Could someone help me please? Thanks (Also, sorry for my english).
Problems like this are categorised as real-time search, perhaps the best known example is Learning Real-Time A*, where you combine information about what you've seen before (if you've had to backtrack or know a cheaper way to reach a state), and the actions you can take. As is the case in areas like reinforcement learning, some level of randomness helps balance exploration and exploitation.
Assuming your graph is undirected, time invariant, and the initial and exit node exist in the same component, then choosing a direction at random at each vertex is equivalent to a random walk on a graph.
Regardless of whether the graph is initially known or not, this is a very well understood field of mathematics, equivalent to an absorbing Markov chain, the time to reach the exit state in such cases has a Discrete phase-type distribution - often quite slow, but it's also worth noting that in pathological cases it's possible to design a maze where a random walk will outperform DFS.
#beaker is right in that the first two you suggested should lead the the same result. However, you may be able to improve on the search a little by keeping track of any loops you find. If the Robot finds itself in a spot it has already visited and needs to backtrack once coming to a dead end there may be no need go back so far if there is a shortcut it has found. Also use the segments that have been mapped on the way out and apply Dijkstra's algorithm or A* on it to find the most efficient way back. There may be a faster way back on an unexplored path but this would be the safest way to have a quick result.
Obviously this implementing the checks for loops to prevent unneeded back tracking will make thing more complicated to implement. Though for the return to the start using Dijkstra's algorithm should not be as complex.
If you are feeling ambitious now that found the exit you could use this information and give the robot a sense of direction though in a randomly generated maze that may not help much.
I am reviewing how to solve binary trees/binary search tree problems in Java using recursion and I feel like I'm struggling a little bit. Ex: manipulating/changing the nodes in the tree, counting certain patterns (# of even ints, height of tree), etc. I pretty much always use private helper methods.
I'd like to hear some rules of thumb to follow to make my life easier to solve these problems. Thanks in advance!
edit: be more specific.... I'm just talking about ANY kinds of problems involving using recursion to solve a binary tree/BST problem. I'm not talking about any one problem. I want to know STRATEGIES FOR SOLVING these problems when creating METHODS to solve them. Like certain things to always include or think of when solving them. I can't get more specific than that. Thanks for the votes down.
First off, always remember that you're working on a BST, not an unsorted binary tree or non-binary general tree. That means at all times you can rely on the BST invariant: every value in left subtree < this < every value in right subtree. (equality included on one of the sides in some implementations).
Example where this is relevant: in BST searching, if the value you're trying to find is less than this, there's no point looking in the right subtree for it; it's not there.
Other than that, treat it like you would any recursion problem. That means, for a given problem:
1) Determine what cases are trivial and don't require recursion. Write code that correctly identifies these cases and returns the trivial result. Some possibilities include a height-0 tree, or no tree (null root).
2) For all other cases, determine which of the following would make more sense: (both are usually possible, but one can be cleaner)
What non-recursive work you could do to then reduce the problem to a sub-problem and return that solution (recursion at end, potentially tail recursion)
What recursive work you would need to do first in order to solve this problem. (recursion at start, not tail recursion)
Having private helper methods is not necessarily a bad thing; so long as they serve a distinct and useful function you shouldn't feel bad for writing them. There are certainly some recursion problems that are much cleaner and less redundant when split into 3-4 methods rather than cramming them into one.
Take some BST problems and see if you can identify the general structure of the solution before you sit down to code it. Hope this helps! Also you may want to have a java tag on your question if you're specifically asking about BST stuff for Java.
I'm writing a program which solves a 24-puzzle (5x5 grid) using two heuristic. The first uses how many blocks the incorrect place and the second uses the Manhattan distance between the blocks current place and desired place.
I have different functions in the program which use each heuristic with an A* and a greedy search and compares the results (so 4 different parts in total).
I'm curious whether my program is wrong or whether it's a limitation of the puzzle. The puzzle is generated randomly with pieces being moved around a few times and most of the time (~70%) a solution is found with most searches, but sometimes they fail.
I can understand why greedy would fail, as it's not complete, but seeing as A* is complete this leads me to believe that there's an error in my code.
So could someone please tell me whether this is an error in my thinking or a limitation of the puzzle? Sorry if this is badly worded, I'll rephrase if necessary.
Thanks
EDIT:
So I"m fairly sure it's something I'm doing wrong. Here's a step-by-step list of how I'm doing the searches, is anything wrong here?
Create a new list for the fringe, sorted by whichever heuristic is being used
Create a set to store visited nodes
Add the initial state of the puzzle to the fringe
while the fringe isn't empty..
pop the first element from the fringe
if the node has been visited before, skip it
if node is the goal, return it
add the node to our visited set
expand the node and add all descendants back to the fringe
If you mean that sliding puzzle: This is solvable if you exchange two pieces from a working solution - so if you don't find a solution this doesn't tell anything about the correctness of your algorithm.
It's just your seed is flawed.
Edit: If you start with the solution and make (random) legal moves, then a correct algorithm would find a solution (as reversing the order is a solution).
It is not completely clear who invented it, but Sam Loyd popularized the 14-15 puzzle, during the late 19th Century, which is the 4x4 version of your 5x5.
From the Wikipedia article, a parity argument proved that half of the possible configurations are unsolvable. You are probably running into something similar when your search fails.
I'm going to assume your code is correct, and you implemented all the algorithms and heuristics correctly.
This leaves us with the "generated randomly" part of your puzzle initialization. Are you sure you are generating correct states of the puzzle? If you generate an illegal state, obviously there will be no solution.
While the steps you have listed seem a little incomplete, you have listed enough to ensure that your A* will reach a solution if there is one (albeit not optimal as long as you are just simply skipping nodes).
It sounds like either your puzzle generation is flawed or your algorithm isn't implemented correctly. To easily verify your puzzle generation, store the steps used to generate the puzzle, and run it in reverse and check if the result is a solution state before allowing the puzzle to be sent to the search routines. If you ever generate an invalid puzzle, dump the puzzle, and expected steps and see where the problem is. If the puzzle passes and the algorithm fails, you have at least narrowed down where the problem is.
If it turns out to be your algorithm, post a more detailed explanation of the steps you have actually implemented (not just how A* works, we all know that), like for instance when you run the evaluation function, and where you resort the list that acts as your queue. That will make it easier to determine a problem within your implementation.
I need an algorithm to find the best solution of a path finding problem. The problem can be stated as:
At the starting point I can proceed along multiple different paths.
At each step there are another multiple possible choices where to proceed.
There are two operations possible at each step:
A boundary condition that determine if a path is acceptable or not.
A condition that determine if the path has reached the final destination and can be selected as the best one.
At each step a number of paths can be eliminated, letting only the "good" paths to grow.
I hope this sufficiently describes my problem, and also a possible brute force solution.
My question is: is the brute force is the best/only solution to the problem, and I need some hint also about the best coding structure of the algorithm.
Take a look at A*, and use the length as boundary condition.
http://en.wikipedia.org/wiki/A%2a_search_algorithm
You are looking for some kind of state space search algorithm. Without knowing more about the particular problem, it is difficult to recommend one over another.
If your space is open-ended (infinite tree search), or nearly so (chess, for example), you want an algorithm that prunes unpromising paths, as well as selects promising ones. The alpha-beta algorithm (used by many OLD chess programs) comes immediately to mind.
The A* algorithm can give good results. The key to getting good results out of A* is choosing a good heuristic (weighting function) to evaluate the current node and the various successor nodes, to select the most promising path. Simple path length is probably not good enough.
Elaine Rich's AI textbook (oldie but goodie) spent a fair amount of time on various search algorithms. Full Disclosure: I was one of the guinea pigs for the text, during my undergraduate days at UT Austin.
did you try breadth-first search? (BFS) that is if length is a criteria for best path
you will also have to modify the algorithm to disregard "unacceptable paths"
If your problem is exactly as you describe it, you have two choices: depth-first search, and breadth first search.
Depth first search considers a possible path, pursues it all the way to the end (or as far as it is acceptable), and only then is it compared with other paths.
Breadth first search is probably more appropriate, at each junction you consider all possible next steps and use some score to rank the order in which each possible step is taken. This allows you to prioritise your search and find good solutions faster, (but to prove you have found the best solution it takes just as long as depth-first searching, and is less easy to parallelise).
However, your problem may also be suitable for Dijkstra's algorithm depending on the details of your problem. If it is, that is a much better approach!
This would also be a good starting point to develop your own algorithm that performs much better than iterative searching (if such an algorithm is actually possible, which it may not be!)
A* plus floodfill and dynamic programming. It is hard to implement, and too hard to describe in a simple post and too valuable to just give away so sorry I can't provide more but searching on flood fill and dynamic programming will put you on the path if you want to go that route.
So I've had at least two professors mention that backtracking makes an algorithm non-deterministic without giving too much explanation into why that is. I think I understand how this happens, but I have trouble putting it into words. Could somebody give me a concise explanation of the reason for this?
It's not so much the case that backtracking makes an algorithm non-deterministic.
Rather, you usually need backtracking to process a non-deterministic algorithm, since (by the definition of non-deterministic) you don't know which path to take at a particular time in your processing, but instead you must try several.
I'll just quote wikipedia:
A nondeterministic programming language is a language which can specify, at certain points in the program (called "choice points"), various alternatives for program flow. Unlike an if-then statement, the method of choice between these alternatives is not directly specified by the programmer; the program must decide at runtime between the alternatives, via some general method applied to all choice points. A programmer specifies a limited number of alternatives, but the program must later choose between them. ("Choose" is, in fact, a typical name for the nondeterministic operator.) A hierarchy of choice points may be formed, with higher-level choices leading to branches that contain lower-level choices within them.
One method of choice is embodied in backtracking systems, in which some alternatives may "fail", causing the program to backtrack and try other alternatives. If all alternatives fail at a particular choice point, then an entire branch fails, and the program will backtrack further, to an older choice point. One complication is that, because any choice is tentative and may be remade, the system must be able to restore old program states by undoing side-effects caused by partially executing a branch that eventually failed.
Out of the Nondeterministic Programming article.
Consider an algorithm for coloring a map of the world. No color can be used on adjacent countries. The algorithm arbitrarily starts at a country and colors it an arbitrary color. So it moves along, coloring countries, changing the color on each step until, "uh oh", two adjacent countries have the same color. Well, now we have to backtrack, and make a new color choice. Now we aren't making a choice as a nondeterministic algorithm would, that's not possible for our deterministic computers. Instead, we are simulating the nondeterministic algorithm with backtracking. A nondeterministic algorithm would have made the right choice for every country.
The running time of backtracking on a deterministic computer is factorial, i.e. it is in O(n!).
Where a non-deterministic computer could instantly guess correctly in each step, a deterministic computer has to try all possible combinations of choices.
Since it is impossible to build a non-deterministic computer, what your professor probably meant is the following:
A provenly hard problem in the complexity class NP (all problems that a non-deterministic computer can solve efficiently by always guessing correctly) cannot be solved more efficiently on real computers than by backtracking.
The above statement is true, if the complexity classes P (all problems that a deterministic computer can solve efficiently) and NP are not the same. This is the famous P vs. NP problem. The Clay Mathematics Institute has offered a $1 Million prize for its solution, but the problem has resisted proof for many years. However, most researchers believe that P is not equal to NP.
A simple way to sum it up would be: Most interesting problems a non-deterministic computer could solve efficiently by always guessing correctly, are so hard that a deterministic computer would probably have to try all possible combinations of choices, i.e. use backtracking.
Thought experiment:
1) Hidden from view there is some distribution of electric charges which you feel a force from and you measure the potential field they create. Tell me exactly the positions of all the charges.
2) Take some charges and arrange them. Tell me exactly the potential field they create.
Only the second question has a unique answer. This is the non-uniqueness of vector fields. This situation may be in analogy with some non-deterministic algorithms you are considering. Further consider in math limits which do not exist because they have different answers depending on which direction you approach a discontinuity from.
I wrote a maze runner that uses backtracking (of course), which I'll use as an example.
You walk through the maze. When you reach a junction, you flip a coin to decide which route to follow. If you chose a dead end, trace back to the junction and take another route. If you tried them all, return to the previous junction.
This algorithm is non-deterministic, non because of the backtracking, but because of the coin flipping.
Now change the algorithm: when you reach a junction, always try the leftmost route you haven't tried yet first. If that leads to a dead end, return to the junction and again try the leftmost route you haven't tried yet.
This algorithm is deterministic. There's no chance involved, it's predictable: you'll always follow the same route in the same maze.
If you allow backtracking you allow infinite looping in your program which makes it non-deterministic since the actual path taken may always include one more loop.
Non-Deterministic Turing Machines (NDTMs) could take multiple branches in a single step. DTMs on the other hand follow a trial-and-error process.
You can think of DTMs as regular computers. In contrast, quantum computers are alike to NDTMs and can solve non-deterministic problems much easier (e.g. see their application in breaking cryptography). So backtracking would actually be a linear process for them.
I like the maze analogy. Lets think of the maze, for simplicity, as a binary tree, in which there is only one path out.
Now you want to try a depth first search to find the correct way out of the maze.
A non deterministic computer would, at every branching point, duplicate/clone itself and run each further calculations in parallel. It is like as if the person in the maze would duplicate/clone himself (like in the movie Prestige) at each branching point and send one copy of himself into the left subbranch of the tree and the other copy of himself into the right subbranch of the tree.
The computers/persons who end up at a dead end they die (terminate without answer).
Only one computer will survive (terminate with an answer), the one who gets out of the maze.
The difference between backtracking and non-determinism is the following.
In the case of backtracking there is only one computer alive at any given moment, he does the traditional maze solving trick, simply marking his path with a chalk and when he gets to a dead end he just simply backtracks to a branching point whose sub branches he did not yet explore completely, just like in a depth first search.
IN CONTRAST :
A non deteministic computer can clone himself at every branching point and check for the way out by running paralell searches in the sub branches.
So the backtracking algorithm simulates/emulates the cloning ability of the non-deterministic computer on a sequential/non-parallel/deterministic computer.