I had a directed weighted graph G, where weight is duration of transition.
I wrote all paths search algorithm between two vertices using DFS with modification: the search continues until total weight of path (the sum of the weights of its parts) will be less some value.
My code works in small graph, but in big graph (|V|=1800, |E|=50870) it freezes.
def find_paths(self, start, end, weight_limit=10):
res = []
def dfs(start, end, path=[], weight=0):
path = path + [start]
if len(path) > 1:
weight += self.weights[(path[-2], start)]
if weight > weight_limit:
return []
if start == end:
res.append((path, weight))
return [path]
if start not in self.adjacent:
return []
paths = []
for node in self.adjacent[start]:
if node not in path:
paths.extend(dfs(node, end, path, weight))
return paths
dfs(start, end)
return res
Your code seems correct (especially since it works on small graphs).
The problem is that there can be lots of paths between the nodes. For a fully connected graph the number of paths is on the order of N! which is a lot. Since you need all of them your program is going to be slow (especially if you run out of ram and need to swap things to disk).
If you limit the maximum total weight like you did in your code assuming all weights are one you still run in O(weight), which I assume you set to a large value since the graph is large.
You need to figure out if you really need all those paths.
If you are looking for the shortest path use Dijkstra or something for finding the shortest path.
Related
This post has is the result that constantly appears for this problem but doesn't provide an optimal solution.
Currently I am trying to return all shortest paths starting atfrom and ending at target using BFS but I am running into a bottleneck with either my algorithm or the data structures I use.
pseudocode:
// The graph is an adjacency list of type unordered_map<string, unordered_set<string>>
// deque with pair of (visited unordered_set, vector with current path)
deque q = [({from}, [from]);
while q:
pair = q.dequeue()
visited = pair.first
path = pair.second
foreach adjacent_node to path[-1] in the graph:
if (adjacent_node == target):
res.append(path + [adjacent_node])
else if adjacent_node not in visited:
newPath = path + [adjacent_node]
visited.add(adjacent_node)
q.push((visited, newPath))
Currently the bottleneck seems to be with the queue's pair of items. I'm unsure how to solve the problem without storing a visited set with every path, or without copying a new path into the queue.
Firstly you should know that number of shortest paths can be huge and returning them all is not practical. Consider a graph with 2k+1 layers numbered from 1 to 2k+1, in which each layer is fully connected with the next layer, and odd layers has only one point while even layers has q points. Although this graph only has k(q+1)+1 nodes and kq edges, there are in total q^k different shortest paths which can be inefficient for normal computers to handle. However if you're sure that the number of shortest paths is relatively small I can introduce the following algorithm.
The basic idea is to store a list back for each node, meaning the shortest distance between from and x equals to the shortest distance between from and v plus one if and only if v in back[x]. back[x] can be computed during the process. Then you can perform a depth-first search to print all the shortest path. Pseudo code (BTW I noticed that your code is not correct):
queue q = [ from ]
visited = set<node>
back = map<node, list<node>>
while q.not_empty():
now = q.front()
if (now == target):
continue
foreach adjacent_node to now in the graph:
if (adjacent_node in visited):
back[adjacent_node].push(now)
else:
visited.add(adjacent_node)
back[adjacent_node] = [ now ]
q.push(adjacent_node)
# Now collect all shortest paths
ret = []
current = []
def collect(x):
current.push(x)
if (x == from):
ret.push(current.reversed())
return
foreach v in back[x]:
collect(v)
current.pop()
Sorry for my poor English. Feel free to point out my typos and mistakes.
How can I get the shortest path using DFS. I've seen this question asked several times, but the replies are generally use BFS or a different algorithm. What about in the context of a robot traversing a maze? BFS isn't possible since it jumps from node to node and a robot would require backtracking.
Right now I am trying to solve the problem using:
def dfs(self, v):
v.visited = True
for adj in v.adj:
if adj.visited is False:
# set the parent
adj.successor = v
# explore the other nodes
self.dfs(adj)
However, this does not necessarily return the shortest path. Is there another way to approach this problem? I've seen some suggestions to use depth first iterative deepening, but I can't find many examples implementing this algorithm.
If anyone is interested in how I solved this here is my solution:
def dfs(self, v):
if v.successor is None: # this is the root node
v.cost = 0
v.visited = True
for adj in v.adj:
if adj.visited is False:
# set the parent
adj.successor = v
adj.cost = v.cost + 1
self.dfs(adj)
# if the cost is less switch the successor
# and set the new cost
elif adj.cost > v.cost + 1:
adj.successor = v
adj.cost = v.cost + 1
self.dfs(adj)
This is essentially a version of DFS that keeps track of the cost for each vertex. If the cost of the vertex is more than a successor + 1 step then it sets the new successor. This allows for finding the shortest path without using BFS.
I am trying to achieve 2nd, preferably nth shortest path using the A* pathfinding algorithm. I have implemented the shortest path already:
while(open.length > 0) {
max = worldSize;
min = -1;
for(i in open) {
if(open[i].f < max) {
max = open[i].f;
min = i;
}
}
node = open.splice(min, 1)[0];
if(node.value === nodeEnd.value) {
path = closed[closed.push(node)-1];
do {
result.push({x: path.x, y:path.y});
} while(path = path.parent);
open = closed = astar = [];
result.reverse();
} else {
neighbors = findNeighbors(node.x, node.y);
for(i = 0; i < neighbors.length; ++i) {
path = newNode(node, neighbors[i]);
if(!astar[path.value]) {
path.g = node.g + manhattanDistance(neighbors[i], node);
path.f = path.g + manhattanDistance(neighbors[i], nodeEnd);
open.push(path);
astar[path.value] = true;
}
}
closed.push(node);
}
}
What can I do? I have zero experience in this and don't even understand the algorithm to its fullest (still researching at the moment). Thank you.
So this problem is in general NP hard. Since you only need the second shortest path, you can do it tractably. Basically, given the shortest path, you generate a collection of graphs by taking the original graph and removing one edge from the shortest path. So if you have a shortest path of length N, on a graph G(E,N), you end up with N graphs of G(E-1,V). Now you run A* on each of these graphs, and the shortest one is your second shortest path, as is it the shortest path which is different from the original shortest path by at least one edge.
This also shows why it is NP hard in practice. If I want the third shortest path, I have to to the following procedure only removing one edge from each of the two shortest paths, and the number of such pairs grows exponentially. N->N^2->N^3 etc
if(node.value === nodeEnd.value) is search termination condition. It means that the algorithm found some path from start to end. The essense of A* and, specifically, properties of heuristic function (admissability and consistency) guarantees that first time you arrive at termination condition gives you a shortest path.
Moreover, admissable and consistent heuristics also guarantees that all possible paths from start to end always checked from shortest to longest
So in order to get Nth closest path you only need to allow search algorithm to continue N-1 times, e.g.
hit_count = 0
while(open.length > 0) {
// same as before
if(node.value === nodeEnd.value) {
path = closed[closed.push(node)-1];
hit_count += 1;
if (hit_count == N - 1) {
do {
result.push({x: path.x, y:path.y});
} while(path = path.parent);
open = closed = astar = [];
result.reverse();
}
}
else {
// same as before
}
}
Please note that such an approach will yield paths of the same length as new paths, i.e. if you have two paths of exaclty same length one of them would be reported as shortest and other as second shortest, depending on details of implementation.
If you want to consider all paths with the same length "identical", so "second-shortest" actually longer than shortest, just replace hit_count += 1 to
// Don't forget to initialize last_found_path_length outside the loop to zero
if (path.length() != last_found_path_length) {
last_found_path_length = path.length();
hit_count += 1
}
Plase note that you haven't specified what language it is (feels like Javascript) so examples here might contain syntactic errors or refer to missing methods. But I hope the approach is clear.
An approximate solution is to run the A* algorithm multiple times with the following caveat:
After you find the current shortest path
mark the node that came right before the end node
next time you run the algorithm, don't allow that node to be used e.g. set it's path.f value to infinity or something huge. or just don't add it to the open list when it is a neighbor of your current node.
the newly found path will be the next shortest
A couple of notes:
This is approximate
It will not work after all the nodes connected to the end have been cycled through.
In terrain with a complicated set of paths and obstacles you may cut off the possible routes
if the next shortest path could be acheived by making a decision earlier in the path finding algorithm you won't capture it. to capture that you would have to cycle through the whole path in the manner described disallowing one node at a time and taking shortest path of all the possible computed paths - this will become a mess and get out of hand quickly as the number of nodes increase.
hope that helps.
I have a graph and I want to find all shortest paths between two nodes. I've found a shortest path between two nodes by BFS. However, it just gives me one of the shortest paths if there exists one more than.
How could I get all of them using BFS?
I've implement my code from well-known BFS pseudocode.
Also, I have a adjacency list vector which holds adjacency vertices for all nodes.
You can easily do it by maintaining a list or vector of parents for each node.
If two or more nodes ( say X, Y, Z) at the same distance from the starting node , leads to another node M , make all X , Y and Z as the parents of M.
You just have to add a check to see while adding a parent to the node whether that parent is in the same level as the previous parents.
By level , I mean the distance from the starting point.
This way you can get all the shortest paths by tracing back the parent vectors.
Below is my C++ implementation.
I hope you know how to print the paths by starting from the destination ,tracing the parents and reach the starting point.
EDIT : Pseudo Code
bfs (start , end)
enqueue(start)
visited[start] = 1
while queue is NOT empty
currentNode = queue.front()
dequeue()
if(currentNode == end)
break
for each node adjacent to currentNode
if node is unvisited
visited[node] = visited[curr] + 1
enqueue(node)
parent[node].add(currentNode)
else if(currentNode is in same level as node's parents)
parent[node].add(currentNode)
return
If the graph is large, finding all paths from start to end and then selecting the shortest ones can be very inefficient. Here is a better algorithm:
Using BFS, label each node with its distance from the start node. Stop when you get to the end node.
def bfs_label(start, end):
depth = {start: 0}
nodes = [start]
while nodes:
next_nodes = []
for node in nodes:
if node == end:
return depth
for neighbor in neighbors(node):
if neighbor not in depth:
depth[neighbor] = depth[node] + 1
fringe.append(neighbor)
Using DFS, find all paths from the start node to the end node such that the depth strictly increases for each step of the path.
def shortest_paths(node, end, depth, path=None):
if path is None:
path = []
path.append(node)
if node == end:
yield tuple(path)
else:
for neighbor in neighbors(node):
if neighbor in depth and depth[neighbor] == depth[node]+1:
for sp in shortest_paths(neighbor, end, depth, path):
yield sp
path.pop()
A simpler way is to find all paths from source to destination using dfs. Now find the shortest paths among these paths. Here is a sudo code:
dfs(p,len)
if(visited[p])
return
if(p== destination)
paths.append(len)
return
visited[p]=1
for each w adjacent to p
dfs(w,len+1)
visited[p]=0
You can find the path by maintaining an array for paths. I will leave that to you as an assignment
We can use a simple BFS algorithm for finding all the shortest paths. We can maintain the path along with the current node. I have provided the link to the python code for the same below.
https://gist.github.com/mridul111998/c24fbdb46492b57f7f17decd8802eac2
I have a problem where I need to search for all unique paths in an undirected graph of degree <=4. The graph is basically a grid, and all connections are between direct neighbors only (4-way).
A path cannot visit the same vertex
more than once.
A path can visit any
number of vertices to make a path.
A path contains at least 2 vertices.
How do I go about this problem?
Here's the pseudocode I just came up with:
Start at any node.
Get all of its paths
See where they lead, if it's a node that has not been visited then visit it.
Call the same function recursively for the nodes from the previous paths.
Keep a counter for the number of paths.
This would be this code in Java (untested):
public int getPaths (Node n, Set<Node> nodesVisited) {
int pathCount = 0;
for (Path p : n.getPaths()) {
Node otherSide = p.getOtherNode(n); // Where this function basically takes a node and gets the other node in the path
if (!(nodesVisited.contains(otherSide))) {
nodesVisited.add(otherSide);
pathCount += 1 + getPaths(otherSide, new Set<Nodes>(nodesVisited));
}
}
return pathCount;
}
This should find the paths from one starting node. You can start it on each node but you'd get some duplicates. To weed them out you'd also need to return the paths though.