In this post it is described Dijkstras as a greedy algorithm, while here and here it is shown to have connections with dynamic programming algorithms.
Which one is it then?
It's greedy because you always mark the closest vertex. It's dynamic because distances are updated using previously calculated values.
I would say it's definitely closer to dynamic programming than to a greedy algorithm. To find the shortest distance from A to B, it does not decide which way to go step by step. Instead, it finds all places that one can go from A, and marks the distance to the nearest place. Marking that place, however, does not mean you'll go there. It only means that distance can no longer be made shorter assuming all edges of the graph are positive. The algorithm itself does not have a good sense of direction as to which way will get you to place B faster. The optimal decisions are not made greedily, but are made by exhausting all possible routes that can make a distance shorter. Therefore, it's a dynamic programming algorithm, the only variation being that the stages are not known in advance, but are dynamically determined during the course of the algorithm. You can call it a "dynamic" dynamic programming algorithm, if you like, to tell it apart from other dynamic programming algorithms with predetermined stages of decision making to go through.
Compared with the Kruskal minimal spanning tree algorithm, the difference is clear. In Kruskal's algorithm, you always choose the shortest edge that does not lead to a cycle, and then the next shortest edge, and so on. The optimal strategies are chosen step by step and only one choice gets played out in the algorithm. The other possibilities are not checked or compared algorithmically, even though mathematically a theorem guarantees that they will not be optimal. So to me Kruskal is greedy but Dijkstra is dynamic programming.
Related
I would like to know the difference between Boruvkas algorithm and Kruskals algorithm.
What they have in common:
both find the minimum spanning tree (MST) in a undirected graph
both add the shortest edge to the existing tree until the MST is found
both look at the graph in it`s entirety, unlike e.g. Prims algorithm, which adds one node after another to the MST
Both algorithmns are greedy
The only difference seems to be, that Boruvkas perspective is each individual node (from where it looks for the cheapest edge), instead of looking at the entire graph (like Kruskal does).
It therefore seems to be, that Boruvka should be relatively easy to do in parallel (unlike Kruskal). Is that true?
In case of Kruskal's algorithm, first of all we want to sort all edges from the cheapest to the most expensive ones. Then in each step we remove min-weight edge and if it doesn't create a cycle in our graph (which initially consits of |V|-1 separate vertices), then we add it to MST. Otherwise we just remove it.
Boruvka's algorithm looks for nearest neighbour of each component (initially vertex). It keeps selecting cheapest edge from each component and adds it to our MST. When we have only one connected component, it's done.
Finding cheapest outgoing edge from each node/component can be done easily in parallel. Then we can just merge new, obtained components and repeat finding phase till we find MST. That's why this algorithm is a good example for parallelism (in case of finding MST).
Regarding parallel processing using Kruskal's algorithm, we need to keep and check edges in strict order, that's why it's hard to achieve explicit parallelism. It's rather sequential and we can't do much about this (even if we still may consider e.g. parallel sorting). Although there were few approaches to implement this method in parallel way, those papers can be found easily to check their results.
Your description is accurate, but one detail can be clarified: Boruvka's algorithm's perspective is each connected component rather than each individual node.
Your intuition about parallelization is also right -- this paper has more details. Excerpt from the abstract:
In this paper we design and implement four parallel MST algorithms (three variations of Boruvka plus our new approach) for arbitrary sparse graphs that for the first time give speedup when compared with the best sequential algorithm.
The important difference between Boruvka's algorithm and Kruskal's or Prim's is that with Boruvka's you don't need to presort the edges or maintain a priority queue.
Boruvka's still incurs the extra log N factor in the cost, but it does it by requiring O(log N) passes over the edges.
You can parallelize Boruvka's algorithm, but you can also parallelize sorting, so I don't know if Boruvka's has any real advantages over Kruskal's in practice.
I'm developing a path finding program. It is said theoretically that A* is better than Dijkstra. In fact, the latter is a special case of the former. However, when testing in the real world, I begin to doubt that is A* really better?
I used data of New York City, from 9th DIMACS Implementation Challenge - Shortest Paths, in which each node's latitude and longitude is given.
When applying A*, I need to calculate the spherical distance between two points, using Haversine Formula, which involves sin, cos, arcsin, square root. All of those are very very time-consuming.
The result is,
Using Dijkstra: 39.953 ms, expanded 256540 nodes.
Using A*, 108.475 ms, expanded 255135 nodes.
Noticing that in A*, we expanded less 1405 nodes. However, the time to compute a heuristic is much more than that saved.
To my understanding, the reason is that in a very large real graph, the weight of the heuristic will be very small, and the effect of it can be ignored, while the computing time is dominating.
I think you're more or less missing the point of A*. It is intended to be a very performant algorithm, partially by intentionally doing more work but with cheap heuristics, and you're kind of tearing that to bits when burdening it with a heavy extremely accurate prediction heuristic.
For node selection in A* you should just use an approximation of distance. Simply using (latdiff^2)+(lngdiff^2) as the approximated distance should make your algorithm much more performant than Dijkstra, and not give much worse results in any real world scenario. Actually the results should even be exactly the same if you do calculate the travelled distance on a selected node properly with the Haversine. Just use a cheap algorithm for selecting potential next traversals.
A* can be reduced to Dijkstra by setting some trivial parameters. There are three possible ways in which it does not improve on Dijkstra:
The heuristic used is incorrect: it is not a so-called admissible heuristic. A* should use a heuristic which does not overestimate the distance to the goal as part of its cost function.
The heuristic is too expensive to calculate.
The real-world graph structure is special.
In the case of the latter you should try to build on existing research, e.g. "Highway Dimension, Shortest Paths, and Provably Efficient Algorithms" by Abraham et al.
Like everything else in the universe, there's a trade-off. You can take dijkstra's algorithm to precisely calculate the heuristic, but that would defeat the purpose wouldn't it?
A* is a great algorithm in that it makes you lean towards the goal having a general idea of which direction to expand first. That said, you should keep the heuristic as simple as possible because all you need is a general direction.
In fact, more precise geometric calculations that are not based on the actual data do not necessarily give you a better direction. As long as they are not based on the data, all those heuristics give you just a direction which are all equally (in)correct.
In general A* is more performant than Dijkstra's but it really depends the heuristic function you use in A*. You'll want an h(n) that's optimistic and finds the lowest cost path, h(n) should be less than the true cost. If h(n) >= cost, then you'll end up in a situation like the one you've described.
Could you post your heuristic function?
Also bear in mind that the performance of these algorithms highly depends on the nature of the graph, which is closely related to the accuracy of the heuristic.
If you compare Dijkstra vs A* when navigating out of a labyrinth where each passage corresponds to a single graph edge, there is probably very little difference in performance. On the other hand, if the graph has many edges between "far-away" nodes, the difference can be quite dramatic. Think of a robot (or an AI-controlled computer game character) navigating in almost open terrain with a few obstacles.
My point is that, even though the New York dataset you used is definitely a good example of "real-world" graph, it is not representative of all real-world path finding problems.
I am trying to find a optimal solution for the following problem
The numbers denoted inside each node are represented as (x,y).
The adjacent nodes to a node always have a y value that is (current nodes y value +1).
There is a cost of 1 for a change in the x value when we go from one node to its adjacent
There is no cost for going from node to its adjacent, if there is no change in the value of x.
No 2 nodes with the same y value are considered adjacent.
The optimal solution is the one with the lowest cost, I'm thinking of using A* path finding algorithm for finding an optimal solution.
My question, Is A* a good choice for the this kind of problems, or should i look at any other algorithm, and also i was thinking of using recursive method to calculate the Heuristic cost, but i get the feeling that it is not a good idea.
This is the example of how I'm thinking the heuristic function will be like this
The heuristic weight of a node = Min(heuristic weight of it's child nodes)
The same goes for the child nodes too.
But as far as my knowledge goes, heuristic is meant to be an approximate, so I think I'm going in the wrong direction as far as the heuristic function is concerned
A* guarantees to find the lowest cost path in a graph with nonnegative edge path costs, provided that you use an appropriate heuristic. What makes a heuristic function appropriate?
First, it must be admissible, i. e. it should, for any node, produce either an underestimate or a correct estimate for the cost of the cheapest path from that node to any of goal nodes. This means the heuristic should never overestimate the cost to get from the node to the goal.
Note that if your heuristic computes the estimate cost of 0 for every node, then A* just turns into breadth-first exhaustive search. So h(n)=0 is still an admissible heuristic, only the worst possible one. So, of all admissible heuristics, the tighter one estimates the cost to the goal, the better it is.
Second, it must be cheap to compute. It should be certainly O(1), and should preferably look at the current node alone. Recursively evaluating the cost as you propose will make your search significantly slower, not faster!
The question of A* applicability is thus whether you can come up with a reasonably good heuristic. From your problem description, it is not clear whether you can easily come up with one.
Depending on the problem domain, A* may be very useful if requirements are relaxed. If heuristic becomes inadmissible, then you lose the guarantee of finding the best path. Depending on the degree of overestimation of the distance, hovewer, the solution might still be good enough (for problem specific definition of "good enough"). The advantage is that sometimes you can compute that "good enough" path much faster. In some cases, probabilistic estimate of heuristics works good (it can have additional constraints on it to stay in the admissible range).
So, in general, you have breadth-first search for tractable problems, next faster you have A* for tractable problems with admissible heuristic. If your problem is intractable for breadth-first exhaustive search and does not admit a heuristic, then your only option is to settle for a "good enough" suboptimal solution. Again, A* may still work with inadmissible heuristic here, or you should look at beam search varieties. The difference is that beam searches have a limit on the number of ways the graph is currently being explored, while A* limits them indirectly by choosing some subset of less costly ones. There are practical cases not solvable by A* even with relaxed admissbility, when difference in cost among different search path is slight. Beam search with its hard limit on the number of paths works more efficiently in such problems.
Seems like an overkill to use A* when something like Dijkstra's would work. Dijkstra's will only work on non-negative cost transitions which seems true in this example. If you can have negative cost transitions then Bellman-Ford should also work.
I was reading about dijkstra algorithm and A* star algorithm. I know that the difference is the heuristic used. But what is a heuristic and how this influence the algorithms? Heuristic is just an way to measure the distance? But dijkstra considers the distance too? Sorry, but my question is about heuristic and what it means and why to use them... (I had read about it, but don't understant)
Other question: When should be used each one?
Thank you
In this context, a heuristic is a way of providing the algorithm with some form of extra evaluative information, so that the algorithm can find a 'good enough' solution, without exhaustively searching every possible solution.
Dijkstra's Algorithm does not use a heuristic. It expands outwards from the start node, and examines every node in the graph in order to find the shortest path. While this is accurate, it can be computationally expensive.
By comparison, the A* algorithm uses a distance + cost heuristic, to guide the algorithm in its choice of the next node to explore. This means that the algorithm finds a possible search solution without examining every node on the graph. It is therefore much cheaper to run, but at a loss of complete accuracy. It works because the result is usually close enough to the optimal solution, and is found cheaper than an exhaustive search of the entire graph.
As to when you should use each, it really depends on the application. However, using the A* algorithm requires an admissible heuristic, so this may not be applicable in situations where such information is unavailable to the algorithm.
A heuristic basically means an idea or an intuition! Any strategy that you use to solve a hard problem is a heuristic! In some fields (like combinatorial problems) it refers to the strategy that can help you solve a NP hard problem suboptimally within a polynomial time.
Dijkstra solves the single-source routing problem, i.e. it gives you the cost of going froma single point to any other point in the space.
A* solves a single-source single-target problem. It gives you a minimum distance path from a given point to another given point.
A* is usually faster than Dijkstra provided you give it an admissible heuristic, this is, an estimate of the distance to the goal, that never overestimates said distance. Contrary to a previous answer given here, if the heuristic used is admissible, A* is complete and will give you the optimal answer.
Suppose I have 10 points. I know the distance between each point.
I need to find the shortest possible route passing through all points.
I have tried a couple of algorithms (Dijkstra, Floyd Warshall,...) and they all give me the shortest path between start and end, but they don't make a route with all points on it.
Permutations work fine, but they are too resource-expensive.
What algorithms can you advise me to look into for this problem? Or is there a documented way to do this with the above-mentioned algorithms?
Have a look at travelling salesman problem.
You may want to look into some of the heuristic solutions. They may not be able to give you 100% exact results, but often they can come up with good enough solutions (2 to 3 % away from optimal solutions) in a reasonable amount of time.
This is obviously Travelling Salesman problem. Specifically for N=10, you can either try the O(N!) naive algorithm, or using Dynamic Programming, you can reduce this to O(n^2 2^n), by trading space.
Beyond that, since this is an NP-hard problem, you can only hope for an approximation or heuristic, given the usual caveats.
As others have mentioned, this is an instance of the TSP. I think Concord, developed at Georgia Tech is the current state-of-the-art solver. It can handle upwards of 10,000 points within a few seconds. It also has an API that's easy to work with.
I think this is what you're looking for, actually:
Floyd Warshall
In computer science, the Floyd–Warshall algorithm (sometimes known as
the WFI Algorithm[clarification needed], Roy–Floyd algorithm or just
Floyd's algorithm) is a graph analysis algorithm for finding shortest
paths in a weighted graph (with positive or negative edge weights). A
single execution of the algorithm will find the lengths (summed
weights) of the shortest paths between all pairs of vertices though it
does not return details of the paths themselves
In the "Path reconstruction" subsection it explains the data structure you'll need to store the "paths" (actually you just store the next node to go to and then trivially reconstruct whichever path is required as needed).