I was reading Introduction To Algorithms 3rd Edition. There are 3 methods given to solve the problem. My inquiry is about two of them.
The one with no name
The algorithm starts by topologically sorting the dag (see Section 22.4) to impose a linear ordering on the vertices. If the dag contains a path from vertex u to vertex v, then u precedes v in the topological sort. We make just one pass over the vertices in the topologically sorted order. As we process each vertex, we relax each edge that leaves the vertex.
Dijkstra's Algorithm
This is quite well known
As far as the book shows, time complexity of without name one is O(V+E) but of Dijstra's is O(ElogV). We cannot use Dijkstra's on negative weight but we can use the other. What are the advantages of using Dijkstra's Algorithm except it can be used in cyclic ones?
Because the first algorithm you give only works on acyclic graphs, whereas Dijkstra runs on graph with non-negative weight.
The limitations are not the same.
In real-world, many applications can be modelled as graphs with non-negative weights, that's why Dijkstra is so used. Plus, it is very simple to implement. The complexity of Dijkstra is higher because it relies on priority queue, but this does not mean it takes necessarily more time to execute. (nlog(n) time is not that bad, because log(n) is a relatively small number: log(10^80) = 266)
However, this stand for sparse graphs (low density of edges). For dense graphs, other algorithms may be more efficient.
Related
I am building a dependency based scheduler where my tasks/nodes form a directed acyclic graph. I have the following constraints and am trying to decide on the most appropriate algorithm:
New tasks can be added (with or without dependencies) at any time
Some tasks can run in parallel
Two algorithms are mentioned repeatedly with regards to topological sorting; depth first search and Kahn's algorithm.
What are the pro's and con's of these two algorithms?
Is one algorithm objectively better for my scenario?
Is there an alternate that better fits my scenario?
I have one further question about vocabulary. Given dependencies such as:
c->b
b->a
e->d
Is this considered to be a single directed acyclic graph, 2 acyclic graphs (since e and d are not dependent on the other tasks) or an acyclic graph with sub acyclic graphs?
I think you misunderstood these algorithms.
Depth First Search:
So the Depth First Search algorithm (I looked it up on wikipedia. There it is actually called an algorithm. I would rather call it a strategy) is an algorithm for traversing through a graph. So to visit all nodes. It will not return you the topological order of your graph!!
Kahn's algorithm: Kahn's algorithm on the other hand, is the right algorithm for your problem. It will return you the topoligical order if there is one. The algorithm has an asymptotic running time of O(m+n) where m is the amount of edges and n is the amount of vertices in your graph. I do not think, that you will find a better algorithm for that problem.
Vocabulary: In your example, you have one DAG (directed acyclic graph) with two weakly connected components.
EDIT:
After your mention of an algorithm based on Depth First Search:
For the asymptotic running time, it does not matter which algorithm you are using. Both are in O(m+n). Also concerning the storage, it actually should not matter.
Is this a right algorithm for finding minimal spanning tree.
Divide Graph into 2 equally connected parts. Find its minimal spanning trees. Connect them using the smallest edge that connects them. I am trying to get counterexample of this algorithm, but can't.
Consider a four-node graph, connected in a square, with the left edge having cost 10 and all other edges having cost 1. If you divide the graph into left and right for your recursive step, you will end up with a spanning tree of cost 12, instead of cost 3.
MST is not well-adapted to "divide-and-conquer" algorithms. The closest thing is probably the Reverse-Delete algorithm; whenever you fail to remove an edge (because it would disconnect the graph), you can think of the remaining steps as executing recursively on the two sides of that edge.
You have described a divide and conquer algorithm which will not work when determining an MST. Sneftel provided a good counter-example and recursively dividing the graph into two connected parts would be extremely costly.
Instead, a good approach to finding an MST would be to use a greedy algorithm such as Prim's algorithm. We know a greedy algorithm will work because this problem exhibits optimal substructure. For this algorithm, you will want to represent your graph as an adjacency list. First, start at an arbitrary node and add it to a visited list. Add all edges from this node into a min-heap. Include the cheapest edge in your MST and add the connecting node to your visited list. From that node add all edges to your min-heap and then select the cheapest edge to a node that has yet to be visited. Continue doing so until all nodes are visited. Once that is done you have your MST.
You can use other data structures to store the graph and the visited edges, but the ones I have outlined above will maximize the runtime. If we analyze the run-time with these data structures we can see that the runtime is O(E log V) which is the time to update the cost of the elements and maintaining your heap after an edge has been removed. More specifically O(log V) to fix the heap and that is done E number of times.
I also found this quick 2-minute video that outlines Prim's algorithm with an example: Prim's Algorithm in 2 Minutes
I hope this information is helpful!
I have a graph, I want to walk through the graph (not necessary through all vertices), always taking path with greates weight. I cannot go through same vertex twice, I stop if there are no more moves I can make. What is the complexity? I assume it's "n" (where n is number of vertices) but I'm not sure.
If you can't go through the same vertice twice, your upper bound for edge traversals is n.
It's easy to think of examples where that would be a tight bound (a single chain of vertices connected for example).
However, keep in mind that complexity is for a given algorithm, not a general task, you haven't described your algorithm or how your graph is organized, so this question doesn't have any meaning.
If for example the graph is a clique, perhaps picking the highest weight edge for each traversal would itself take n computation steps (if the edges are kept in an unsorted list kept in each vertice), making the naive algorithm O(n^2) in this case. Other representations may have different complexity, but require different algorithms.
EDIT
If you're asking about finding the path with greatest overall weight (which may require you in some traversals to pick an edge that doesn't have the highest weight), than the problem is NP-hard. If it had a polynomial algorithm, then you could take an unweighted graph and find the longest path (a known NP hard problem as jimifiki pointed), and solve it with that algorithm.
From Longest Path Problem
This problem is called the Longest Path Problem, and is NP-complete.
After a lot of Googling, I've found that most sources say that the Dijkstra algorithm is "more efficient" than the Bellman-Ford algorithm. But under what circumstances is the Bellman-Ford algorithm better than the Dijkstra algorithm?
I know "better" is a broad statement, so specifically I mean in terms of speed and also space if that applies. Surely there is some situation in which the Bellman-Ford approach is better than the Dijkstra approach.
Bellman-Ford algorithm is a single-source shortest path algorithm, so when you have negative edge weight then it can detect negative cycles in a graph.
The only difference between the two is that Bellman-Ford is also capable of handling negative weights whereas Dijkstra Algorithm can only handle positives.
From wiki
However, Dijkstra's algorithm greedily selects the minimum-weight node
that has not yet been processed, and performs this relaxation process
on all of its outgoing edges; in contrast, the Bellman–Ford algorithm
simply relaxes all the edges, and does this |V | − 1 times, where |V |
is the number of vertices in the graph. In each of these repetitions,
the number of vertices with correctly calculated distances grows, from
which it follows that eventually all vertices will have their correct
distances. This method allows the Bellman–Ford algorithm to be applied
to a wider class of inputs than Dijkstra.
Dijkstra is however generally considered better in the absence of negative weight edges, as a typical binary heap priority queue implementation has O((|E|+|V|)log|V|) time complexity [A Fibonacci heap priority queue gives O(|V|log|V| + |E|)], while the Bellman-Ford algorithm has O(|V||E|) complexity
As already stated in the chosen answer, Bellman-Ford performs the check on all the vertices, Dijkstra only on the one with the best distance calculated so far. Again already noted, this improves the complexity of the Dijkstra approach, however it requires to compare all the vertices to find out the minimum distance value. Being this not necessary in the Bellman-Ford, it is easier to implement in a distributed environment. That's why it is used in Distance Vector routing protocols (e.g., RIP and IGRP), where mostly local information is used. To use Dijkstra in routing protocols, instead, it is necessary first to distribute the entire topology, and this is what happens in Link State protocols, such as OSPF and ISIS.
There are 4 major difference among them I know:-
1. bellman time complexity is O(VE) and Dijkstra Algo has O(ElogV)in case of maxheap is used.
Bellman does relaxation for n-1 times and Dijkstra Algo only 1 time.
Bellman can handle negative weights but Dijkstra Algo can't.
Bellman visit a vertex more then once but Dijkstra Algo only once.
The only difference is that Dijkstra's algorithm cannot handle negative edge weights which Bellman-ford handles.And bellman-ford also tells us whether the graph contains negative cycle.
If graph doesn't contain negative edges then Dijkstra's is always better.
An efficient alternative for Bellman-ford is Directed Acyclic Graph (DAG) which uses topological sorting.
http://www.geeksforgeeks.org/shortest-path-for-directed-acyclic-graphs/
Dijkstra Algo
Dijkstra algo is not capable to differentiate between Negative edge weight cycle is present in graph or not
1. Positive edge weight:- Dijkstra always PASS if all edge weight in a graph is positive
2. Negative edge wt. and No -ve edge wt. cycle:- Dijkstra always PASS even if we have some edges weight as Negative but NO cycle/loop in graph having negative edge weight.
[i.e No Negative edge weight cycle is present]
3. Negative edge wt. and -ve edge wt. cycle:- Dijkstra may PASS/FAIL even if we have some edges weight as negative along with cycle/loop in graph having negative edge weight.
In a normal introduction to algorithm class, you will learn that the only difference between Dijkstra and Bell-man Ford, is that the latter works for negative edges at the cost of more computation time. The discussion on time complexity is already give in the accepted answer.
However I want to emphasize and add a bit more to #Halberdier's answer that in a distributed system, Bellman-Ford is implemented EVEN WHEN ALL EDGES ARE Positive. This is because in a Bellman-Ford algorithm, the entity S does not need to know every weight of every edge in the graph to compute the shortest distance to T - it only needs to know the shortest distance for all neighbors of S to T, plus the weight of S to all its neighbors.
A typical application of such algorithm is in Computer Networking, where you need to find the shortest route between two routers. Dijkstra is implemented in a centralized manner called link state Routing, while
Bellman-Ford allows each router to update themselves asynchronously, called distance-vector routing.
I believe no one explains better than Jim Kurose, the author of <Computer Network, a top-down approach>. See his youtube videos below.
Link State routing:
https://www.youtube.com/watch?v=bdh2kfgxVuw&list=TLPQMTIwNjIwMjLtHllygYsxMg&index=3
Distance Vector routing:
https://www.youtube.com/watch?v=jJU2AVX6gpU&list=TLPQMTIwNjIwMjLtHllygYsxMg&index=4
Bellman Ford’s Algorithm
Dijkstra’s Algorithm
Bellman Ford’s Algorithm works when there is a negative weight edge, it also detects the negative weight cycle.
Dijkstra’s Algorithm may or may not work when there is a negative weight edge. But will definitely not work when there is a negative weight cycle.
The result contains the vertices which contain the information about the other vertices they are connected to.
The result contains the vertices containing whole information about the network, not only the vertices they are connected to.
It can easily be implemented in a distributed way.
It can not be implemented easily in a distributed way.
It is more time-consuming than Dijkstra’s algorithm. Its time complexity is O(VE).
It is less time-consuming. The time complexity is O(E logV).
Dynamic Programming approach is taken to implement the algorithm.
Greedy approach is taken to implement the algorithm.
Bellman Ford’s Algorithm has more overheads than Dijkstra’s Algorithm.
Dijkstra’s Algorithm has less overheads than Bellman Ford’s Algorithm.
Bellman Ford’s Algorithm has less scalability than Dijkstra’s Algorithm.
Dijkstra’s Algorithm has more scalability than Bellman Ford’s Algorithm.
Source 1
I do not agree completely, difference is in implementation and complexity, Dijsktra's algorithm is faster (O(n^2)) but difficult to implement, while Bellman Ford complexity is O(n^3) but is easier to implement.
What is the Best order of traversing edges within Bellman-Ford algorithm,
in order to achieve a minimum number of iterations that are necessary?
If the graph is acyclic the best way is traverse the vertices in topological order. This means only one iteration of the algorithm is necessary.
For cyclic graphs without negative edges you can use a generic shortest path algorithm as described in this pdf (sssp.pdf) with a Fifo queue, this can visit fewer vertices than the standard Bellman-ford algorithm which loops over vertices and then edges. In practice the Fifo queue approach is often faster than using a priority queue (Dijkstra) as mentioned in this answer ( Are there faster algorithms than Dijkstra? ). However, a dis-advantage of this approach is the unlike the standard Bellman-ford algorithm this algorithm will not terminate if the graph has negative cycles.
We can't generalize the best order of traversing so that no. of iterations are minimum as best order of traversing will vary from graph to graph but yes you will get the Single Source Shortest Paths in at most |V|-1 iterations. No graph would require more than |V|-1 iterations and if distance further decreases after |V|-1 iterations then it means that graph contains a negative edge cycle.