Wikipedia says A* runs in O(|E|) where |E| is the number of edges in the graph. But my friend says A* is just a general case of Dijkstra's algorithm, and Dijkstra's algorithm runs in O(|E| + |V| log |V|). So I am confused about why A* runs faster than Dijkstra's algorithm.
I think the time complexity of A* listed on Wikipedia is incorrect (or at least, it's misleading). That time complexity only seems to count the number of states expanded in the search, rather than the time required to determine which states to explore.
To be efficient, A* search needs to store a priority queue containing what nodes in the fringe need to be explored and it has to be able to call decrease-key on those priorities. The runtime for this is, in the worst-case, O(n log n + m) if implemented using a good priority queue. Therefore, in the worst case, you'd expect A* to degrade to Dijkstra's algorithm. Given a good heuristic, A* will not expand out all the nodes and edges that Dijkstra's algorithm would, which is the main reason why A* is faster.
Of course, the time complexity of A* search needs to factor in the cost of computing the heuristic. Some complex heuristics might not be computable in time O(1), in which case the runtime for A* could actually be worse than Dijkstra's algorithm.
Hope this helps!
Essentially A* is faster because it can use heuristics to make more educated guesses about which route is the best to case, something which Dijkstra's algorithm does not do.
Related
I was testing the two algorithms and Bellman-Ford performed better on sparse graphs and looking at the big-O analysis of both, O(VE) for Bellman-Ford and O(E + V lg V) for Dijkstra's. I believe this is correct. I did some researching that said
Dijkstra's is always faster and Bellman-Ford is only used when negative weight cycles are present.
Is that really the case?
TRUE.
Wikipedia: However, Dijkstra's algorithm greedily selects the minimum-weight node
that has not yet been processed, and performs this relaxation process
on all of its outgoing edges; in contrast, the Bellman–Ford algorithm
simply relaxes all the edges, and does this |V | − 1 times, where |V |
is the number of vertices in the graph. In each of these repetitions,
the number of vertices with correctly calculated distances grows, from
which it follows that eventually all vertices will have their correct
distances. This method allows the Bellman–Ford algorithm to be applied
to a wider class of inputs than Dijkstra.
Bellman-Ford performs the check on all the vertices, Dijkstra only on the one with the best distance calculated so far. Again already noted, this improves the complexity of the Dijkstra approach, however it requires to compare all the vertices to find out the minimum distance value. Being this not necessary in the Bellman-Ford, it is easier to implement in a distributed environment. That's why it is used in Distance Vector routing protocols (e.g., RIP and IGRP), where mostly local information is used. To use Dijkstra in routing protocols, instead, it is necessary first to distribute the entire topology, and this is what happens in Link State protocols, such as OSPF and ISIS.
Asymptotically, for any graph where E ≥ V, the runtime of Dijkstra’s algorithm (O(E + V log V)) is smaller than that of Bellman-Ford (O(EV)). This means that, at least in theory, Dijkstra’s algorithm will outperform Bellman-Ford for large graphs of any density (assuming there’s at least as many edges as nodes). That’s borne out in practice, with Dijkstra’s algorithm usually running much faster.
Bellman-Ford is used, as you’ve mentioned, in cases where there are negative edges, which Dijkstra can’t handle. But there are other cases where Bellman-Ford is useful, too. For example, in network routing, where each network router needs to find shortest paths and there isn’t a central computer coordinating everything, the network routers can run a distributed version of Bellman-Ford to find shortest paths between computers due to how the computation only requires local updates to node distances. Dijkstra’s algorithm doesn’t work in this case.
There are also ways to improve the performance of Bellman-Ford in practice for many types of graphs. The Shortest Paths Faster Algorithm (SPFA) is a relatively simple optimization of Bellman-Ford that, while still retaining Bellman-Ford’s worst case runtime, is empirically faster in practice. Dijkstra’s algorithm, IIRC, is usually still faster than SPFA, but this does close the gap in some circumstances.
As you’ve mentioned
My Algorithms class is talking about Prim's Algorithm as a method of finding Minimum Spanning Trees of weighted graphs. Our professor asked us to try to think of an example of a graph that Prim's Algorithm takes N^2 time to solve (N = number of Vertices). No one in the class could think of one off the top of their head, so I'm asking you. I'm pretty sure Prim's Algorithm = O(N^2), so this would be the worst-case scenario for the algorithm.
What's a good example of a graph that takes N^2 time for Prim's Algorithm to solve?
If I understand your question correctly, the example is trivial.
If the graph is complete, there're O(N^2) edges, so just reading the graph is O(N^2).
How could A* algorithm be efficiently modified to provide e.g. 2nd or 8th shortest path, not first?
If at all possible, I suggest that you try and make your program look like a shortest path problem to which Dijkstra applies, and then use one of the answers you have already been pointed at to find the kth shortest path in this situation, such as Eppstein's algorithm and Yen's algorithm for k shortest paths.
But there is another approach. There is a general technique for finding the Kth best answer to combinatorial problems by adding extra constraints which allow you to split the solution tree. It is known as Lawler-Murty and is described for example in section 2 of http://www.vldb.org/pvldb/vol4/p1028-golenberg.pdf
You should check out the algorithm K*. It was published in a paper titled: "K*: A heuristic search algorithm for finding the k shortest paths". The paper was published in the research journal Artificial Intelligence in 2011 (one of the most prestigious in the field), so, as far as I know it is kind of state-of-the-art for what you are looking for.
If you used the algorithm with a consistent heuristic has an asymptotic worst-case complexity of O(m + n log(n) + k) in runtime and space (where n is the number of vertices and m the number of edges in the graph).
Is there a linear О(n+m) time algorithm for finding just the value r of the minimum spanning tree of a given graph G(V,E)? We do not want to find that MST, just the sum of its edges.
I have searched for solution of the problem, but Kruskal's and Prim's algorithms are with higher complexity because of the comparison structures they use(UnionFind(Kruskal) and PQ(Prim)). Also they find the MST, which is not desired and maybe there is faster way to find only r.
If your edges are integer weighted, there is a linear algorithm from Ferdman and Willard in the following publication:
http://www.sciencedirect.com/science/article/pii/S0022000005800649
There is also a randomize linear time algorithm from Karger, Klein and Tarjan using a comparaison model:
http://dl.acm.org/citation.cfm?doid=201019.201022
I belive that in the comparaison model Chazelle's algorithm using soft heap is the fastest deterministic one, but it's not a linear one (you have a inverse Akermann overhead).
No. There's no linear solution.
You can optimize Kruskal with disjoin-set optimizations and radix/counting sort so that the complexity is O(E alpha(V)) where alpha is a very slow growing inverse Akermann function. For most datasets this will be almost indistinguishable from linear. At this point you can probably gain more at run-time by optimizing the code rather than the algorithm.
After a lot of Googling, I've found that most sources say that the Dijkstra algorithm is "more efficient" than the Bellman-Ford algorithm. But under what circumstances is the Bellman-Ford algorithm better than the Dijkstra algorithm?
I know "better" is a broad statement, so specifically I mean in terms of speed and also space if that applies. Surely there is some situation in which the Bellman-Ford approach is better than the Dijkstra approach.
Bellman-Ford algorithm is a single-source shortest path algorithm, so when you have negative edge weight then it can detect negative cycles in a graph.
The only difference between the two is that Bellman-Ford is also capable of handling negative weights whereas Dijkstra Algorithm can only handle positives.
From wiki
However, Dijkstra's algorithm greedily selects the minimum-weight node
that has not yet been processed, and performs this relaxation process
on all of its outgoing edges; in contrast, the Bellman–Ford algorithm
simply relaxes all the edges, and does this |V | − 1 times, where |V |
is the number of vertices in the graph. In each of these repetitions,
the number of vertices with correctly calculated distances grows, from
which it follows that eventually all vertices will have their correct
distances. This method allows the Bellman–Ford algorithm to be applied
to a wider class of inputs than Dijkstra.
Dijkstra is however generally considered better in the absence of negative weight edges, as a typical binary heap priority queue implementation has O((|E|+|V|)log|V|) time complexity [A Fibonacci heap priority queue gives O(|V|log|V| + |E|)], while the Bellman-Ford algorithm has O(|V||E|) complexity
As already stated in the chosen answer, Bellman-Ford performs the check on all the vertices, Dijkstra only on the one with the best distance calculated so far. Again already noted, this improves the complexity of the Dijkstra approach, however it requires to compare all the vertices to find out the minimum distance value. Being this not necessary in the Bellman-Ford, it is easier to implement in a distributed environment. That's why it is used in Distance Vector routing protocols (e.g., RIP and IGRP), where mostly local information is used. To use Dijkstra in routing protocols, instead, it is necessary first to distribute the entire topology, and this is what happens in Link State protocols, such as OSPF and ISIS.
There are 4 major difference among them I know:-
1. bellman time complexity is O(VE) and Dijkstra Algo has O(ElogV)in case of maxheap is used.
Bellman does relaxation for n-1 times and Dijkstra Algo only 1 time.
Bellman can handle negative weights but Dijkstra Algo can't.
Bellman visit a vertex more then once but Dijkstra Algo only once.
The only difference is that Dijkstra's algorithm cannot handle negative edge weights which Bellman-ford handles.And bellman-ford also tells us whether the graph contains negative cycle.
If graph doesn't contain negative edges then Dijkstra's is always better.
An efficient alternative for Bellman-ford is Directed Acyclic Graph (DAG) which uses topological sorting.
http://www.geeksforgeeks.org/shortest-path-for-directed-acyclic-graphs/
Dijkstra Algo
Dijkstra algo is not capable to differentiate between Negative edge weight cycle is present in graph or not
1. Positive edge weight:- Dijkstra always PASS if all edge weight in a graph is positive
2. Negative edge wt. and No -ve edge wt. cycle:- Dijkstra always PASS even if we have some edges weight as Negative but NO cycle/loop in graph having negative edge weight.
[i.e No Negative edge weight cycle is present]
3. Negative edge wt. and -ve edge wt. cycle:- Dijkstra may PASS/FAIL even if we have some edges weight as negative along with cycle/loop in graph having negative edge weight.
In a normal introduction to algorithm class, you will learn that the only difference between Dijkstra and Bell-man Ford, is that the latter works for negative edges at the cost of more computation time. The discussion on time complexity is already give in the accepted answer.
However I want to emphasize and add a bit more to #Halberdier's answer that in a distributed system, Bellman-Ford is implemented EVEN WHEN ALL EDGES ARE Positive. This is because in a Bellman-Ford algorithm, the entity S does not need to know every weight of every edge in the graph to compute the shortest distance to T - it only needs to know the shortest distance for all neighbors of S to T, plus the weight of S to all its neighbors.
A typical application of such algorithm is in Computer Networking, where you need to find the shortest route between two routers. Dijkstra is implemented in a centralized manner called link state Routing, while
Bellman-Ford allows each router to update themselves asynchronously, called distance-vector routing.
I believe no one explains better than Jim Kurose, the author of <Computer Network, a top-down approach>. See his youtube videos below.
Link State routing:
https://www.youtube.com/watch?v=bdh2kfgxVuw&list=TLPQMTIwNjIwMjLtHllygYsxMg&index=3
Distance Vector routing:
https://www.youtube.com/watch?v=jJU2AVX6gpU&list=TLPQMTIwNjIwMjLtHllygYsxMg&index=4
Bellman Ford’s Algorithm
Dijkstra’s Algorithm
Bellman Ford’s Algorithm works when there is a negative weight edge, it also detects the negative weight cycle.
Dijkstra’s Algorithm may or may not work when there is a negative weight edge. But will definitely not work when there is a negative weight cycle.
The result contains the vertices which contain the information about the other vertices they are connected to.
The result contains the vertices containing whole information about the network, not only the vertices they are connected to.
It can easily be implemented in a distributed way.
It can not be implemented easily in a distributed way.
It is more time-consuming than Dijkstra’s algorithm. Its time complexity is O(VE).
It is less time-consuming. The time complexity is O(E logV).
Dynamic Programming approach is taken to implement the algorithm.
Greedy approach is taken to implement the algorithm.
Bellman Ford’s Algorithm has more overheads than Dijkstra’s Algorithm.
Dijkstra’s Algorithm has less overheads than Bellman Ford’s Algorithm.
Bellman Ford’s Algorithm has less scalability than Dijkstra’s Algorithm.
Dijkstra’s Algorithm has more scalability than Bellman Ford’s Algorithm.
Source 1
I do not agree completely, difference is in implementation and complexity, Dijsktra's algorithm is faster (O(n^2)) but difficult to implement, while Bellman Ford complexity is O(n^3) but is easier to implement.