AFAIK, heuristic algorithm is problem-dependent and meta-heuristic are problem-independent, according to this answer.1
But greedy algorithm can apply to many problems, such as minimum spanning tree and shortest path problem. My question is that why is it problem-dependent, not problem-independent?
There are a lot of greedy algorithms for different problems, greedy algorithm is not the one particular algorithm, it's a class of algorithms that use the same approach to the problem. Dijkstra's algorithm, Prim's algorithm, Kruskal's algorithm, etc. are completely different, but they are all greedy.
In Dijkstra's algorithm you take an untouched node with minimal distance to it.
In Prim's algorithm you take an edge, that connects tree node with not-tree node, with minimal weight.
In Kruskal's algorithm you take an edge, that connects two different trees, with minimal weight.
And there are many greedy algorithms that don't even work with graphs.
All these heuristics are different and problem-specific, because these algorithms solve completely different problems.
Related
I want to detect cycles in an undirected graph such that I can find the minimum spanning tree (in particular I want to use Kruskal algorithm). Since I want to parallelize the code, I was wondering which algorithm is the best, Depth-first search of union-find algorithm?
Thanks for any suggestions.
Of all three MST algorithms only Boruvka's MST algorithm is easily parallelizable whereas the kruskal and prims are sequential greedy algorithms hence there is minimum scope for parallel implementation of them.
Note: It is a research topic to achieve efficient parallel boruvka might find some papers
After a lot of Googling, I've found that most sources say that the Dijkstra algorithm is "more efficient" than the Bellman-Ford algorithm. But under what circumstances is the Bellman-Ford algorithm better than the Dijkstra algorithm?
I know "better" is a broad statement, so specifically I mean in terms of speed and also space if that applies. Surely there is some situation in which the Bellman-Ford approach is better than the Dijkstra approach.
Bellman-Ford algorithm is a single-source shortest path algorithm, so when you have negative edge weight then it can detect negative cycles in a graph.
The only difference between the two is that Bellman-Ford is also capable of handling negative weights whereas Dijkstra Algorithm can only handle positives.
From wiki
However, Dijkstra's algorithm greedily selects the minimum-weight node
that has not yet been processed, and performs this relaxation process
on all of its outgoing edges; in contrast, the Bellman–Ford algorithm
simply relaxes all the edges, and does this |V | − 1 times, where |V |
is the number of vertices in the graph. In each of these repetitions,
the number of vertices with correctly calculated distances grows, from
which it follows that eventually all vertices will have their correct
distances. This method allows the Bellman–Ford algorithm to be applied
to a wider class of inputs than Dijkstra.
Dijkstra is however generally considered better in the absence of negative weight edges, as a typical binary heap priority queue implementation has O((|E|+|V|)log|V|) time complexity [A Fibonacci heap priority queue gives O(|V|log|V| + |E|)], while the Bellman-Ford algorithm has O(|V||E|) complexity
As already stated in the chosen answer, Bellman-Ford performs the check on all the vertices, Dijkstra only on the one with the best distance calculated so far. Again already noted, this improves the complexity of the Dijkstra approach, however it requires to compare all the vertices to find out the minimum distance value. Being this not necessary in the Bellman-Ford, it is easier to implement in a distributed environment. That's why it is used in Distance Vector routing protocols (e.g., RIP and IGRP), where mostly local information is used. To use Dijkstra in routing protocols, instead, it is necessary first to distribute the entire topology, and this is what happens in Link State protocols, such as OSPF and ISIS.
There are 4 major difference among them I know:-
1. bellman time complexity is O(VE) and Dijkstra Algo has O(ElogV)in case of maxheap is used.
Bellman does relaxation for n-1 times and Dijkstra Algo only 1 time.
Bellman can handle negative weights but Dijkstra Algo can't.
Bellman visit a vertex more then once but Dijkstra Algo only once.
The only difference is that Dijkstra's algorithm cannot handle negative edge weights which Bellman-ford handles.And bellman-ford also tells us whether the graph contains negative cycle.
If graph doesn't contain negative edges then Dijkstra's is always better.
An efficient alternative for Bellman-ford is Directed Acyclic Graph (DAG) which uses topological sorting.
http://www.geeksforgeeks.org/shortest-path-for-directed-acyclic-graphs/
Dijkstra Algo
Dijkstra algo is not capable to differentiate between Negative edge weight cycle is present in graph or not
1. Positive edge weight:- Dijkstra always PASS if all edge weight in a graph is positive
2. Negative edge wt. and No -ve edge wt. cycle:- Dijkstra always PASS even if we have some edges weight as Negative but NO cycle/loop in graph having negative edge weight.
[i.e No Negative edge weight cycle is present]
3. Negative edge wt. and -ve edge wt. cycle:- Dijkstra may PASS/FAIL even if we have some edges weight as negative along with cycle/loop in graph having negative edge weight.
In a normal introduction to algorithm class, you will learn that the only difference between Dijkstra and Bell-man Ford, is that the latter works for negative edges at the cost of more computation time. The discussion on time complexity is already give in the accepted answer.
However I want to emphasize and add a bit more to #Halberdier's answer that in a distributed system, Bellman-Ford is implemented EVEN WHEN ALL EDGES ARE Positive. This is because in a Bellman-Ford algorithm, the entity S does not need to know every weight of every edge in the graph to compute the shortest distance to T - it only needs to know the shortest distance for all neighbors of S to T, plus the weight of S to all its neighbors.
A typical application of such algorithm is in Computer Networking, where you need to find the shortest route between two routers. Dijkstra is implemented in a centralized manner called link state Routing, while
Bellman-Ford allows each router to update themselves asynchronously, called distance-vector routing.
I believe no one explains better than Jim Kurose, the author of <Computer Network, a top-down approach>. See his youtube videos below.
Link State routing:
https://www.youtube.com/watch?v=bdh2kfgxVuw&list=TLPQMTIwNjIwMjLtHllygYsxMg&index=3
Distance Vector routing:
https://www.youtube.com/watch?v=jJU2AVX6gpU&list=TLPQMTIwNjIwMjLtHllygYsxMg&index=4
Bellman Ford’s Algorithm
Dijkstra’s Algorithm
Bellman Ford’s Algorithm works when there is a negative weight edge, it also detects the negative weight cycle.
Dijkstra’s Algorithm may or may not work when there is a negative weight edge. But will definitely not work when there is a negative weight cycle.
The result contains the vertices which contain the information about the other vertices they are connected to.
The result contains the vertices containing whole information about the network, not only the vertices they are connected to.
It can easily be implemented in a distributed way.
It can not be implemented easily in a distributed way.
It is more time-consuming than Dijkstra’s algorithm. Its time complexity is O(VE).
It is less time-consuming. The time complexity is O(E logV).
Dynamic Programming approach is taken to implement the algorithm.
Greedy approach is taken to implement the algorithm.
Bellman Ford’s Algorithm has more overheads than Dijkstra’s Algorithm.
Dijkstra’s Algorithm has less overheads than Bellman Ford’s Algorithm.
Bellman Ford’s Algorithm has less scalability than Dijkstra’s Algorithm.
Dijkstra’s Algorithm has more scalability than Bellman Ford’s Algorithm.
Source 1
I do not agree completely, difference is in implementation and complexity, Dijsktra's algorithm is faster (O(n^2)) but difficult to implement, while Bellman Ford complexity is O(n^3) but is easier to implement.
I have a set of points and a distance function applicable to each pair of points. I would like to connect ALL the points together, with the minimum total distance. Do you know about an existing algorithm I could use for that ?
Each point can be linked to several points, so this is not the usual "salesman itinerary" problem :)
Thanks !
What you want is a Minimum spanning tree.
The two most common algorithms to generate one are:
Prim's algorithm
Kruskal's algorithm
As others have said, the minimum spanning tree (MST) will allow you to form a minimum distance sub-graph that connects all of your points.
You will first need to form a graph for your data set though. To efficiently form an undirected graph you could compute the Delaunay triangulation of your point set. The conversion from the triangulation to the graph is then fairly literal - any edge in the triangulation is also an edge in the graph, weighted by the length of the triangulation edge.
There are efficient algorithms for both the MST (Prim's/Kruskal's O(E*log(V))) and Delaunay triangulation (Divide and Conquer O(V*log(V))) phases, so efficient overall approaches are possible.
Hope this helps.
The algorithm you are looking for is called minimum spanning tree. It's useful to find the minimum cost for a water, telephone or electricity grid. There is Prim's algorithm or Kruskal algorithm. IMO Prim's algorithm is a bit easier to understand.
here http://en.wikipedia.org/wiki/Minimum_spanning_tree you can find more information about the minimum spanning tree, so you can adapt it to solve your problem.
Take a look at the Dijkstra's algorithm:
Dijkstra's algorithm, conceived by Dutch computer scientist Edsger Dijkstra in 1956 and published in 1959, is a graph search algorithm that solves the single-source shortest path problem for a graph with nonnegative edge path costs, producing a shortest path tree. This algorithm is often used in routing and as a subroutine in other graph algorithms.
http://en.wikipedia.org/wiki/Dijkstra's_algorithm
I was trying to figure out why the heuristic used in A* tree search needs to be admissible if A* has to be optimal. By tree search I mean that there is no explored set maintained by the algorithm.
While doing this, I ran into the question: Does A* work for negative edge weights?
The A* algorithm is basically Dijkstra’s algorithm with heuristics. And Dijkstra’s algorithm does not work with negative edge weights. So A* won’t work with negative edge weights neither.
If you’re looking for an algorithm that works with negative edge weights, take a look at the Bellman-Ford algorithm (but it doesn’t use heuristics).
This excellent article regarding Dijkstra's may prove helpful and provides a nice example about negative edges...
http://www.ics.uci.edu/~eppstein/161/960208.html
What I'm looking for is a comprehensive list of graph traversal algorithms, with brief descriptions of their purpose, as a jump off point for researching them. So far I'm aware of:
Dijkstra's - single-source shortest path
Kruskal's - finds a minimum spanning tree
What are some other well-known ones? Please provide a brief description of each algorithm to each of your answers.
the well knowns are :
Depth-first search http://en.wikipedia.org/wiki/Depth-first_search
Breadth-first search http://en.wikipedia.org/wiki/Breadth-first_search
Prim's algorithm http://en.wikipedia.org/wiki/Prim's_algorithm
Kruskal's algorithm http://en.wikipedia.org/wiki/Kruskal's_algorithm
Bellman–Ford algorithm http://en.wikipedia.org/wiki/Bellman%E2%80%93Ford_algorithm
Floyd–Warshall algorithm http://en.wikipedia.org/wiki/Floyd%E2%80%93Warshall_algorithm
Reverse-delete algorithm http://en.wikipedia.org/wiki/Reverse-Delete_algorithm
Dijkstra's_algorithm http://en.wikipedia.org/wiki/Dijkstra's_algorithm
network flow
Ford–Fulkerson algorithm http://en.wikipedia.org/wiki/Ford%E2%80%93Fulkerson_algorithm
Maximum Flow http://en.wikipedia.org/wiki/Maximum_flow_problem
A few off of the top of my head:
Depth-first and Breadth-first traversals, really just two different ways of touching all of the nodes.
Floyd-Warshall algorithm finds the shortest paths between any pair of points, in (big-theta)(v^3) time.
Prim's algorithm is an alternative to Kruskal's for MST.
There are also algorithms for finding fully connected components, which are groups of nodes where you can get from any member in the component to any other member. This only matters for "directed graphs", where you can traverse an edge only one direction.
Personally, I think the coolest extension of graph theory (not exactly related to your question, but if you're interested in learning more about graphs in general its certainly worth your while) is the concepts of "flow networks": http://en.wikipedia.org/wiki/Flow_network . It is a way of computing about, say, how much electricity can one distribute over houses with a variety of power needs and requirements, and a variety of power stations.
Graph algorithms
http://en.wikipedia.org/wiki/List_of_algorithms#Graph_algorithms
All algorithms in one place
http://en.wikipedia.org/wiki/List_of_algorithms
Dictionary of algorithms and data structures:
Dictionary: http://xlinux.nist.gov/dads/
By area: http://xlinux.nist.gov/dads/termsArea.html
By terms type: http://xlinux.nist.gov/dads/termsType.html
List of all implementations in diff. languages: http://xlinux.nist.gov/dads/termsImpl.html