Course Schedule leetcode: https://leetcode.com/problems/course-schedule/
This problem involves detecting a cycle, and if there is one then you cannot complete all course.
I've heard that DFS is most recommended for detecting a cycle, yet Kahn's Algorithm is recommended for the course schedule problem, which is a BFS solution.
So.. which is it? Is DFS better for detecting cycles or is BFS?
Both have a time complexity of O(V+E) and both have a space complexity of O(V+E). So in those terms there is no winner.
One uses a queue, the other a stack. One uses a degree per node, the other a visited mark per node.
One difference is that DFS can use an implicit stack using recursion, which may make the code a bit more compact. But then again, you're then limited by the available call stack space, so using recursion may not be a viable solution for large inputs, and using an explicit stack may in the end be the option to take for a DFS-based solution.
So all in all, they come out equal. Choose either.
In real work, it's always a bad idea to use O(N) stack space, so practical DFS-based solutions would use an explicit stack.
The explicit stack implementation is a little complicated, because the stack isn't just a stack of nodes -- you also need to store the current position in the child list for each open node -- and the traversal logic gets a little complicated.
The DFS solution is not awful, but if you want to write a good solid solution, then Khan's algorithm will end up simpler and faster in the worst case. It will also use less memory, because the list of pending nodes is just a list of nodes. (It doesn't matter if you use that list like a stack or queue. In most cases using it like a stack is faster/easier)
So if you're going to explicitly check a DAG to see if it has cycles, usually Khan's algorithm is best. The DFS technique is useful if you're already doing a DFS for some other reason, and you want to detect cycles along the way.
Related
Why should one choose recursion over iteration, when a solution has the same time complexity for both cases but better space complexity for iterative?
Here's a particular example of a case where there are extra considerations. Tree search algorithms can be defined recursively (because each subtree of a tree is a tree) or iteratively (with a stack). However, while a recursive search can work perfectly for finding the first leaf with a certain property or searching over all leaves, it does not lend itself to producing a well-behaved iterator: an object or function state that returns a leaf, and later when called again returns the next leaf, etc. In an iterative design the search stack can be stored as a static member of the object or function, but in a recursive design the call stack is lost whenever the function returns and is difficult or expensive to recreate.
Iteration is more difficult to understand in some algorithms. An algorithm that can naturally be expressed recursively may not be as easy to understand if expressed iteratively. It can also be difficult to convert a recursive algorithm into an iterative algorithm, and verifying that the algorithms are equivalent can also be difficult.
Recursion allows you to allocate additional automatic objects at each function call. The iterative alternative is to repeatedly dynamically allocate or resize memory blocks. On many platforms automatic allocation is much faster, to the point that its speed bonus outweighs the speed penalty and storage cost of recursive calls. (But some platforms don't support allocation of large amounts of automatic data, as mentioned above; it's a trade-off.)
recursion is very beneficial when the iterative solutions requires that you simulate recursion with a stack. Recursion acknowledges that the compiler already manages a stack to accomplish precisely what you need. When you start managing your own, not only are you likely re-introducing the function call overhead you intended to avoid; but you're re-inventing a wheel (with plenty of room for bugs) that already exists in a pretty bug-free form.
Some Benefits for Recursion
Code is Perfect Elegant (compared to loops)
very useful in backtracking data structures like LinkedList, Binary Search Trees as the recursion works by calling itself in addition stack made especially for this recursive calls and each call chained by its previous one
In Graph Theory, we know that an Vertex adjacencies can be represented using Adjacency list data structure. On contrary, adjacency set is not widely mentioned anywhere in the Graph Theory. Why is that so?
Here's the pros, i can think of.
As Set property, the graph could provide a guarantee in term of duplicated edge and many other properties of Set. Moreover all set operation from the Set Theory became available which is more intuitive to work with the analysis. Such as:
vertex_set_A | vertex_setB is the union operation.
vertex_set_A & vertex_set_B, is the intersection operation.
*Opinion, Set is more understandable as it has correlation in the mathematical proofing. It is also provide a good abstraction of how the low level code dealing with array and stuff.
In term of Performance, one could use HashSet implementation which would provide a constant time operation. Or TreeSet when the graph need to dynamically changed frequently on log time operation.
List data structure also maintain the ordering property of the element which serve no purpose in most graph. In fact, list iterates in ordered fashion which should not happened in the first place. Indexed Ordered should not matter and Set could provide that. The only time when ordering mattered is when the graph is weighted, and thus the ordering based on the weight, in which TreeSet operate mostly in log time operation.
So, I am unsure why most graph algorithms only mentioning adjacency list. Is it because of the technology barrier where Set is harder to implement, whereas List is easier?
This is indeed a very good question, the fact that there isn't a comprehensive answer yet just iterates the question one more time, 'why not' indeed.
The reasons in the comments to me seem more like make-do excuses in that this is just what has existed throughout history, still not enough to warrant an actual reason.
List is merely a general label and you could(and should) use sets if that's better suited to your task.
Some people would argue the set doesn't provide you a guaranteed O(1) lookup time - it's amortized and that the worst case is still O(n) even if very unlikely, others would reason faster iteration via lists, and then there's the argument about implementation feasibility.
While they're technically not wrong, I don't really buy into any of those being the primary reasons. I feel the real reason is that they're used 'just because convention'. The general label was 'list' and it was taken literally often enough to lose its genericness.
Certainly go ahead and use a set if your application lends itself to it.
My textbook used them too.
(Oh and an example to drive that home when I say 'application lending itself to it'; If you need to find indegrees very often in your application, the set will give you an O(V) runtime where V denotes number of vertices, adjacency list will give you O(E) runtime where E denotes number of edges. For dense graphs, O(E) tends to become O(V^2) assuming no parallel edges are allowed. Thus an adjacency set would give you a better runtime performance here.)
I just read this short post about mental models for Recursive Memoization vs Dynamic Programming, written by professor Krishnamurthi. In it, Krishnamurthi represents memoization's top-down structure as a recursion tree, and DP's bottom-up structure as a DAG where the source vertices are the first – likely smallest – subproblems solved, and the sink vertex is the final computation (essentially the graph is the same as the aforementioned recursive tree, but with all the edges flipped). Fair enough, that makes perfect sense.
Anyways, towards the end he gives a mental exercise to the reader:
Memoization is an optimization of a top-down, depth-first computation
for an answer. DP is an optimization of a bottom-up, breadth-first
computation for an answer.
We should naturally ask, what about
top-down, breadth-first
bottom-up, depth-first
Where do they fit into
the space of techniques for avoiding recomputation by trading off
space for time?
Do we already have names for them? If so, what?, or
Have we been missing one or two important tricks?, or
Is there a reason we don't have names for these?
However, he stops there, without giving his thoughts on these questions.
I'm lost, but here goes:
My interpretation is that a top-down, breadth-first computation would require a separate process for each function call. A bottom-up, depth-first approach would somehow piece together the final solution, as each trace reaches the "sink vertex". The solution would eventually "add up" to the right answer once all calls are made.
How off am I? Does anyone know the answer to his three questions?
Let's analyse what the edges in the two graphs mean. An edge from subproblem a to b represents a relation where a solution of b is used in the computation of a and must be solved before it. (The other way round in the other case.)
Does topological sort come to mind?
One way to do a topological sort is to perform a Depth First Search and on your way out of every node, process it. This is essentially what Recursive memoization does. You go down Depth First from every subproblem until you encounter one that you haven't solved (or a node you haven't visited) and you solve it.
Dynamic Programming, or bottom up - breadth first problem solving approach involves solving smaller problems and constructing solutions to larger ones from them. This is the other approach to doing a topological sort, where you visit the node with a in-degree of 0, process it, and then remove it. In DP, the smallest problems are solved first because they have a lower in-degree. (Smaller is subjective to the problem at hand.)
The problem here is the generation of a sequence in which the set of subproblems must be solved. Both top-down breadth-first and bottom-up depth-first can't do that.
Top-down Breadth-first will still end up doing something very similar to the depth-first counter part even if the process is separated into threads. There is an order in which the problems must be solved.
A bottom-up depth-first approach MIGHT be able to partially solve problems but the end result would still be similar to the breadth first counter part. The subproblems will be solved in a similar order.
Given that these approaches have almost no improvements over the other approaches, do not translate well with analogies and are tedious to implement, they aren't well established.
#AndyG's comment is pretty much on the point here. I also like #shebang's answer, but here's one that directly answers these questions in this context, not through reduction to another problem.
It's just not clear what a top-down, breadth-first solution would look like. But even if you somehow paused the computation to not do any sub-computations (one could imagine various continuation-based schemes that might enable this), there would be no point to doing so, because there would be sharing of sub-problems.
Likewise, it's unclear that a bottom-up, depth-first solution could solve the problem at all. If you proceed bottom-up but charge all the way up some spine of the computation, but the other sub-problems' solutions aren't already ready and lying in wait, then you'd be computing garbage.
Therefore, top-down, breadth-first offers no benefit, while bottom-up, depth-first doesn't even offer a solution.
Incidentally, a more up-to-date version of the above blog post is now a section in my text (this is the 2014 edition; expect updates.
I am reading through this article, where it is mentioned that we can get rid of one side of the tree by performing rotations in a specific manner, and then traversing the tree down in one direction and deleting the elements.
Although I understand what they are trying to do, I do not understand why?
What advantages might this type of deletion provide as opposed to a simple postorder deletion?
One advantage I can think of is saving on the memory used by recursion, but I think that is an insignificant overhead as compared to traversing the tree twice, once for rotating, and then for deleting. Am I missing something here?
The article seems to be staying that the point of this method is to avoid recursion (and its consumption of stack space): "Hmm...what if we rearranged nodes so that they didn't have any left subtrees? Then we could just descend to the right, without need to keep track of anything on a stack."
In general, I prefer to avoid recursion when I cannot be sure that its depth will be reasonable, because you will run out of stack space long before you run out of any other sort of memory - in some cases because the system is designed to limit recursion to catch errors causing infinite recursion. However, I think this is less important here, where you have already admitted that other routines in the same package need recursion. Also, the depth of recursion depends on the depth of the tree, and for a balanced tree this will be roughly the logarithm of the number of nodes it it, and so should never be too deep.
I am enrolled in Stanford's ai-class.com and have just learned in my first week of lecture about a* algorithm and how it's better used then other search algo.
I also show one of my class mate implement it on 4x4 sliding block puzzle which he has published at: http://george.mitsuoka.org/StanfordAI/slidingBlocks/
While i very much appreciate and thank George to implement A* and publishing the result for our amusement.
I (and he also) were wondering if there is any way to make the process more optimized or if there is a better heuristic A*, like better heuristic function than the max of "number of blocks out of place" or "sum of distances to goals" that would speed things up?
and Also if there is a better algo then A* for such problems, i would like to know about them as well.
Thanks for the help and in case of discrepancies, before down grading my profile please allow me a chance to upgrade my approach or even if req to delete the question, as am still learning the ways of stackoverflow.
It depends on your heuristic function. for example, if you have a perfect heuristic [h*], then a greedy algorithm(*), will yield better result then A*, and will still be optimal [since your heuristic is perfect!]. It will develop only the nodes needed for the solution. Unfortunately, it is seldom the case that you have a perfect heuristic.
(*)greedy algorithm: always develop the node with the lowest h value.
However, if your heuristic is very bad: h=0, then A* is actually a BFS! And A* in this case will develop O(B^d) nodes, where B is the branch factor and d is the number of steps required for solving.
In this case, since you have a single target function, a bi-directional search (*) will be more efficient, since it needs to develop only O(2*B^(d/2))=O(B^(d/2)) nodes, which is much less then what A* will develop.
bi directional search: (*)run BFS from the target and from the start nodes, each iteration is one step from each side, the algorithm ends when there is a common vertex in both fronts.
For the average case, if you have a heuristic which is not perfect, but not completely terrbile, A* will probably perform better then both solutions.
Possible optimization for average case: You also can run bi-directional search with A*: from the start side, you can run A* with your heuristic, and a regular BFS from the target side. Will it get a solution faster? no idea, you should probably benchmark the two possibilities and find which is better. However, the solution found with this algorithm will also be optimal, like BFS and A*.
The performance of A* is based on the quality of the expected cost heuristic, as you learned in the videos. Getting your expected cost heuristic to match as closely as possible to the actual cost from that state will reduce the total number of states that need to be expanded. There are also a number of variations that perform better under certain circumstances, like for instance when faced with hardware restrictions in large state space searching.