So hopefully this is a simple question, but I can't seem to find the answer.
The time complexity of DFS is allegedly O(|V|+|E|). Now I'm having issues seeing why it depends on the number of edges. The usual explanation I've seen goes as follows:
Say we implement a DFS using an explicit stack (for simplicity). Say we have a graph where each node is connected to all the rest. We start at some node, visit it and then push all it's neighbors onto the stack. Now we pop the next node and put all of it's neighbors onto the stack. We repeat until we visit all the nodes.
Let's pretend that the node that finds itself on top of the stack is not visited yet in each iteration (best case scenario for this graph). In this case we visited all the nodes in |V| moves, but for each of them we pushed |V|-1 nodes on the stack which means that all the edges are pushed on the stack and the complexity is O(|E|)
A few notes. I'm arguing that the complexity is LESS than that so this proof that only looks at the best scenario for a worst case graph is fine. I'm also assuming that |E| is always larger than |V|. In fact, I'm assuming it's O(|V|^2). This means that O(|V|+|E|) and O(|E|) mean the same thing to me.
Ok, now here's my deal. What if we don't use an explicit stack?
The explosion here is due to the fact that we keep stacking up useless nodes that will never be processed. What if we instead just recurse? The advantage is that we can check if we're done before each recursive call.
Since there's no explicit stack and I'm still only visiting nodes I haven't seen before, I don't see how I can exceed the complexity of O(|V|).
The explosion here is due to the fact that we keep stacking up useless nodes that will never be processed. What if we instead just recurse? The advantage is that we can check if we're done before each recursive call.
That check still contributes to the run time. For each node you visit, you need to see which of its neighbors still need to be visited, which means checking each adjacent edge.
Related
People always talk about how if there are infinite nodes downwards, then DFS will get stuck traversing this infinitely long branch and never reaching the answer in another branch.
Isn't this applicable to BFS as well? For example if the root node has an infinite amount of neighbours, wouldn't the program just spend an infinite amount of time trying to add each one into a queue?
In some cases, yes.
However, in order to have an infinite graph you basically need an implicit graph, https://en.wikipedia.org/wiki/Implicit_graph and many of them have bounded degree which avoids that problem.
Additionally, another advantage with BFS over DFS is that a path with fewer vertices often is "better" in some way - and by having a cost for the vertices that can be formulated using algorithms like Djikstra's that in some cases can be extended even to unbounded degrees.
Yes you are right, in the second case BFS will not have any real progress. For this theoretical infinite scenarios, let's discuss all the three possible cases:
If the graph had infinite nodes downwards and finite neighbors, then
we should use BFS (you already explained the reason)
But if the graph has infinite neighbors and finite nodes downwards,
then we should use DFS as in this case while doing DFS search for
each neighbor we would be able to search it's complete
path in finite time and then move on to the next neighbor. Here, BFS wouldn't have gotten any real progress while searching.
If graph had both infinite neighbors and infinite nodes downwards, then DFS and BFS will seize to differ as we are dealing with infinity on both ends.
I was trying to implement Karger's min cut algorithm in the same way it is explained here but I don't like the fact that at each step of the while loop we can pick an edge with it's two endpoints already in a supernode. More specifically, this part
// If two corners belong to same subset,
// then no point considering this edge
if (subset1 == subset2)
continue;
Is there a quick fix for avoiding this problem?
It might help to back up and think about why there’s a union-find structure here at all and why it’s worth improving on the continue statement.
Conceptually, each contraction performed changes the graph in the following way:
The nodes contracted get replaced with a single node.
The edges incident to either node get replaced with an edge to the new joint node.
The edges running between the two earlier nodes get removed.
The question, then, is how to actually do this to the graph. The code you’ve found does this lazily. It doesn’t actually change the graph when the contraction is done. Instead, it uses the union-find structure to show which nodes are now equivalent to one another. When it samples a random edge, it then has to check whether that edge is one of the ones that would have been deleted in step (3). If so, it skips it and moves on. This has the effect that early contractions are really fast (the likelihood of picking two edges that are part of contracted nodes is very low when few edges are contracted), but later contractions might be a lot slower (once edges have started being contracted, lots of edges may have been deleted).
Here’s a simple modification you can use to speed this step up. Whenever you pick an edge to contract and find that its endpoints are already connected, discard that edge, and remove it from the list of edges so that it never gets picked again. You can do this by swapping that edge to the end of the list of edges, then removing the last element of the list. This has the effect that every edge processed will never be seen again, so across all iterations of the algorithm every edge will be processed at most once. That gives a runtime of one randomized contraction phase as O(m + nα(n)), where m is the number of edges and n is the number of nodes. The factor of α(n) comes from the use of the union-find structure.
If you truly want to remove all semblance of that continue statement, an alternative approach would be to directly simulate the contraction. After each contraction, iterate over all m edges and adjust each one by seeing whether it needs to remain unchanged, point to the new contracted node, or be removed altogether. This will take time O(m) per contraction for a net cost of O(mn) for the overall min cut calculation.
There are ways to speed things up beyond this. Karger’s original paper suggests generating a random permutation of the edges and using binary search over that array with a clever use of BFS or DFS to find the cut produced in time O(m), which is slightly faster than the O(m + nα(n)) approach for large graphs. The basic idea is the following:
Probe the middle element of the list of edges.
Run a BFS on the graph formed by only using those edges and see if there are exactly two connected components.
If so, great! Those two CCs are the ones you want.
If there is only one CC, discard the back half of the array of edges and try again.
If there is more than one CC, contract each CC into a single node and update a global table indicating which CC each node belongs to. Then discard the first half of the array and try again.
The cost of each BFS is O(m), where m is the number of edges in the graph, and this gives the recurrence T(m) = T(m/2) + O(m) because at each stage we’re throwing away half of the edges. That solves to O(m) total time, though as you can see, it’s a much trickier way of coding this algorithm up!
To summarize:
With a very small modification to the provided code, you can keep the continue statement in but still have a very fast implementation of the randomized contraction algorithm.
To eliminate that continue without sacrificing the runtime of the algorithm, you need to do some major surgery and change approaches to something only marginally asymptotically faster than keeping the continue in.
Hope this helps!
So I recently implemented a non-recursive version of DFS. Turns out that I can mark the nodes "visited" as soon as they are pushed on the stack or when they are popped out. The problem which I was working on specifically stated to mark it "visited" when pushed on stack. Are both versions some kind of DFS. Or is it like one is DFS and the other is not. Any suggestions are welcomed.
What I think is that if I do the second way, it will mimic the recursive dfs. But why does the other one work?
A recursive dfs (please ignore this)
dfsRec(node)
{
visitedArray[node]=1;
for all neighbours of node
if they aren't visited
dfsRec(neighbour);
}
dfs(startNode)
{
visitedArray;
dfsRec(startNode);
}
The problem with the second way (i.e. marking the node visited when they are popped out) is that your code will loop forever whenever your graph has a cycle. Once DFS reaches that cycle, it would continue going in circles, because the nodes would not be marked visited until they are popped of the stack, so any node reachable through a cycle would be pushed again and again, until you run out of memory.
Note that the issue is not too different from the recursive implementation of DFS: recursive implementation will cause stack overflow instead of running out of memory, but the reason for it would be the same.
I have read about DFS and BFS many times but I have this doubt lingering my mind since long. In a lot of articles it is mentioned that DFS can get stuck in infinite loops.
As far as I know, this limitation can easily be removed by keeping track of the visited nodes. In fact, in all the books that I have read, this little check is a part of DFS.
So why are 'infinite loops' mentioned as a disadvantage of DFS? Is it just because the original DFS algorithm did not have this check for visited nodes? Please explain.
(1) In graph search algorithms [used frequently on AI], DFS's main advantage is space efficiency. It is its main advantage on BFS. However, if you keep track of visited nodes, you lose this advantage, since you need to store all visited nodes in memory. Don't forget the size of visited nodes increases drastically over time, and for very large/infinite graphs - might not fit in memory.
(2) Sometimes DFS can be in an infinite branch [in infinite graphs]. An infinite branch is a branch that does not end [always has "more sons"], and also does not get you to your target node, so for DFS, you might keep expanding this branch inifinitely, and 'miss' the good branch, that leads to the target node.
Bonus:
You can overcome this flaw in DFS, while maintaining relatively small memory size by using a combination of DFS and BFS: Iterative Deepening DFS
a conventional DFS algorithm does track down nodes. A local search algorithm does not track down states and behaves with amnesia. So I think the loop mainly refers to the one an infinite branch(a branch with infinite possible states). In that case, DFS simply goes down and become too focused on one branch.
If you do not check for cycles, then DFS can get stuck in one and never find its target whereas BFS will always expand out to all nodes at the next depth and therefore will eventually find its target, even if cycles exist.
Put simply:
If your graph can have cycles and you're using DFS, then you must account for cycles. On the other hand, BFS provides the option to ignore cycles at the expense of efficiency, which is often acceptable when searching a small number of nodes.
According to Norvig in AIMA (Artificial Intelligence: A modern approach), the Depth-first algorithm is not complete (will not always produce a solution) because there are cases when the subtree being descended will be infinite.
On the other hand, the Breadth-first approach is said to be complete if the branching factor is not infinite. But isn't that somewhat the same "thing" as in the case of the subtree being infinite in DFS?
Can't the DFS be said to be complete if the tree's depth is finite? How is then that the BFS is complete and the DFS is not, since the completeness of the BFS relies on the branching factor being finite!
A tree can be infinite without having an infinite branching factor. As an example, consider the state tree for Rubik's Cube. Given a configuration of the cube, there is a finite number of moves (18, I believe, since a move consists of picking one of the 9 "planes" and rotating it in one of the two possible directions). However, the tree is infinitely deep, since it is perfectly possible to e.g. only rotate the same plane alternatingly back and forth (never making any progress). In order to prevent a DFS from doing this, one normally caches all the visited states (effectively pruning the state tree) - as you probably know. However, if the state space is too large (or actually infinite), this won't help.
I have not studied AI extensively, but I assume that the rationale for saying that BFS is complete while DFS is not (completeness is, after all, just a term that is defined somewhere) is that infinitely deep trees occur more frequently than trees with infinite branching factors (since having an infinite branching factor means that you have an infinite number of choices, which I believe is not common - games and simulations are usually discrete). Even for finite trees, BFS will normally perform better because DFS is very likely to start out on a wrong path, exploring a large portion of the tree before reaching the goal. Still, as you point out, in a finite tree, DFS will eventually find the solution if it exists.
DFS can not stuck in cycles (if we have a list of opened and closed states). The algorithm is not complete since it does not find a solution in an infinite space, even though the solution is in depth d which is much lower than infinity.
Imagine a strangely defined state space where each node has same number of successors as following number in Fibonacci sequence. So, it's recursively defined and therefore infinite. We're looking for node 2 (marked green in the graph). If DFS starts with the right branch of tree, it will take infinite number of steps to verify that our node is not there. Therefore it's not complete (it won't finish in reasonable time). BFS would find the solution in 3rd iteration.
Rubik's cube state space is finite, it is huge, but finite (human stuck in cycles but DFS won't repeat the same move twice). DFS would find very inefficient way how to solve it, sometimes this kind of solution is infeasible. Usually we consider maximum depth infinite, but our resources (memory) are always finite.
The properties of depth-first search depend strongly on whether the graph-search or
tree-search version is used. The graph-search version, which avoids repeated states and redundant
paths, is complete in finite state spaces because it will eventually expand every node.
The tree-search version, on the other hand, is not complete—for example, in Figure 3.6 the
algorithm will follow the Arad–Sibiu–Arad–Sibiu loop forever
Source: AI: a modern approach