I am working on an assignment which takes some a graph, adds an extra vertex to the graph, applies Bellman Ford with the new vertex as the source, then uses applies Dijkstra's all pairs to the graph.
The algorithms being used have run-times/space requirements of:
Adding extra vertex
-- Running Time: V
-- Space: V
Bellman Ford single source shortest path algorithm
-- Running time: EV
-- Space: V
Dijkstra's all pairs shortest path algorithm
-- Running time: EV log V
-- Space: V
I am having difficulty understanding if I am calculating the big O of the total process. Each program is run separately, and the output is piped from program to program. My though is the total algorithm would have a Big-O run time of:
O( V + EV + EV log V ) which would simplify to O( EV log V )
The space requirement would be calculated in a similar fashion. I'm I thinking of this correctly? Thanks!
Exactly, a "rule of thumb" is that, in a sequence of code blocks, the overall complexity is dominated by the block with greatest complexity (asymptotically)
Matematically, when V tends to very large numbers, it's smaller than EV, which is smaller than EVlogV. So, for large V, the complexity of your algorithm is approximated well by EVlogV
Related
I read here that for Undirected graph the space complexity is O(V + E) when represented as a adjacency list where V and E are number of vertex and edges respectively.
My analysis is, for a completely connected graph each entry of the list will contain |V|-1 nodes then we have a total of |V| vertices hence, the space complexity seems to be O(|V|*|V-1|) which seems O(|V|^2) what I am missing here?
You analysis is correct for a completely connected graph. However, note that for a completely connected graph the number of edges E is O(V^2) itself, so the notation O(V+E) for the space complexity is still correct too.
However, the real advantage of adjacency lists is that they allow to save space for the graphs that are not really densely connected. If the number of edges is much smaller than V^2, then adjacency lists will take O(V+E), and not O(V^2) space.
Note that when you talk about O-notation, you usually have three types of variables (or, well, input data in general). First is the variables dependence on which you are studying; second are those variables that are considered constant; and third are kind of "free" variables, which you usually assume to take the worst-case values. For example, if you talk about sorting an array of N integers, you usually want to study the dependence of sorting time on N, so N is of the first kind. You usually consider the size of integers to be constant (that is, you assume that comparison is done in O(1), etc.), and you usually consider the particular array elements to be "free", that is, you study that runtime for the worst possible combination of particular array elements. However, you might want to study the same algorithm from a different point of view, and it will lead to a different expression of complexity.
For graph algorithms, you can, of course, consider the number of vertices V to be of first kind, and the number of edges to be the third kind, and study the space complexity for given V and for the worst-case number of edges. Then you indeed get O(V^2). But it is also often useful to treat both V and E as variables of the first type, thus getting the complexity expression as O(V+E).
Size of array is |V| (|V| is the number of nodes). These |V| lists each have the degree which is denoted by deg(v). We add up all those, and apply the Handshaking Lemma.
∑deg(v)=2|E| .
So, you have |V| references (to |V| lists) plus the number of nodes in the lists, which never exceeds 2|E| . Therefore, the worst-case space (storage) complexity of an adjacency list is O(|V|+2|E|)= O(|V|+|E|).
Hope this helps
according to u r logic, total space = O(v^2-v)
as total no. of connections(E) = v^2 - v,
space = O(E).
even I used to think the same,
But if you have total no. of connections(E) much lesser than no. of vertices(V),
lets say out of 10 people (V=10),
only 2 knows each other, therefore (E=2)
then according to your logic
space = O(E) = O(2)
but we in actual we have to allocate much larger space
which is
space = O(V+E) = O(V+2) = O(V)
that's why,
space = O(V+E), this will work if V > E or E > V
We have a directed graph G=(V,E) ,at which each edge (u, v) in E has a relative value r(u, v) in R and 0<=r(u, v) <= 1, that represents the reliability , at a communication channel, from the vertex u to the vertex v.
Consider as r(u, v) the probability that the chanel from u to v will not fail the transfer and that the probabilities are independent.
I want to write an efficient algorithm that finds the most reliable path between two given nodes.
I have tried the following:
DIJKSTRA(G,r,s,t)
1. INITIALIZE-SINGLE-SOURCE(G,s)
2. S=Ø
3. Q=G.V
4. while Q != Ø
5. u<-EXTRACT-MAX(Q)
6. if (u=t) return d[t]
7. S<-S U {u}
8. for each vertex v in G.Adj[u]
9. RELAX(u,v,r)
INITIAL-SINGLE-SOURCE(G,s)
1. for each vertex v in V.G
2. d[v]=-inf
3. pi[v]=NIL
4. d[s]=1
RELAX(u,v,r)
1. if d[v]<d[u]*r(u,v)
2 d[v]<-d[u]*r(u,v)
3. pi[v]<-u
and I wanted to find the complexity of the algorithm.
The time complexity of INITIALIZE-SINGLE-SOURCE(G,s) is O(|V|).
The time complexity of the line 4 is O(1).
The time complexity of the line 5 is O(|V|).
The time complexity of the line 7 is O(log(|V|)).
The time complexity of the line 8 is O(1).
Which is the time complexityof the command S<-S U {u} ?
The line 10 is executed in total O(Σ_{v \in V} deg(v))=O(E) times and the time complexity of RELAX is O(1).
So the time complexity of the algorithm is equal to the time complexity of the lines (3-9)+O(E).
Which is the time complexity of the union?
So the time complexity of the algorithm is equal to the time
complexity of the lines (3-9)+O(E). Which is the time complexity of
the union?
No, it is not the complexity of the union, union can be done pretty efficiently if you are using hash table for example. Moreover, since you use S only for the union, it seems to be redundant.
The complexity of the algorithm also depends heavily on your EXTRACT-MAX(Q) function (usually it is logarithmic in the size of the Q, so logV per iteration), and on RELAX(u,v,r) (which is also usually logarithmic in the size of Q, since you need to update entries in your priority queue).
As expected, this brings us to the same complexity of original Dijkstra's algorithm, which is O(E+VlogV) or O(ElogV), depending on implementation of your priority queue.
I think that the solution should be based on the classic Dijkstra algorithm (complexity of which is well-known), as you suggested, however in your solution you define the "shortest path" problem incorrectly.
Note that the probability of A and B is p(A) * p(B) (if they're independent). Hence, you should find a path, whose multiplication of edges is maximized. Whereas Dijkstra algorithm finds the path whose sum of edges is minimized.
To overcome this issue you should define the weight of your edges as:
R*(u, v) = -log ( R(u, v) )
By introducing the logarithm, you convert multiplicative problem to additive.
The Kruskal's algorithm is the following:
MST-KRUSKAL(G,w)
1. A={}
2. for each vertex v∈ G.V
3. MAKE-SET(v)
4. sort the edges of G.E into nondecreasing order by weight w
5. for each edge (u,v) ∈ G.E, taken in nondecreasing order by weight w
6. if FIND-SET(u)!=FIND-SET(v)
7. A=A U {(u,v)}
8. Union(u,v)
9. return A
According to my textbook:
Initializing the set A in line 1 takes O(1) time, and the time to sort
the edges in line 4 is O(E lgE). The for loop of lines 5-8 performs
O(E) FIND-SET and UNION operations on the disjoint-set forest. Along
with the |V| MAKE-SET operations, these take a total of O((V+E)α(V))
time, where α is a very slowly growing function. Because we assume
that G is connected, we have |E| <= |V|-1, and so the disjoint-set
operations take O(E α(V)) time. Moreover, since α(V)=O(lgV)=O(lgE),
the total running time of Kruskal's algorithm is O(E lgE). Observing
that |E|<|V|^2, we have lg |E|=O(lgV), and so we can restate the
running time of Kruskal's algorithm as O(E lgV).
Could you explain me why we deduce that the time to sort the edges in line 4 is O(E lgE)?
Also how do we get that the total time complexity is O((V+E)α(V)) ?
In addition, suppose that all edge weights in a graph are integers from 1 to |V|. How fast can you make Kruskal's algorithm run? What if the edges weights are integers in the range from 1 to W for some constant W?
How does the time complexity depend on the weight of the edges?
EDIT:
In addition, suppose that all edge weights in a graph are integers
from 1 to |V|. How fast can you make Kruskal's algorithm run?
I have thought the following:
In order the Kruskal's algorithm to run faster, we can sort the edges applying Counting Sort.
The line 1 requires O(1) time.
The lines 2-3 require O(v) time.
The line 4 requires O(|V|+|E|) time.
The lines 5-8 require O(|E|α(|V|)) time.
The line 9 requires O(1) time.
So if we use Counting Sort in order to solve the edges, the time complexity of Kruskal will be
Could you tell me if my idea is right?
Also:
What if the edges weights are integers in the range from 1 to W for
some constant W?
We will again use Counting Sort. The algorithm will be the same. We find the time complexity as follows:
The line 1 requires O(1) time.
The lines 2-3 require O(|V|) time.
The line 4 requires O(W+|E|)=O(W)+O(|E|)=O(1)+O(|E|)=O(|E|) time.
The lines 5-8 require O(|E|α(|V|)) time.
The line 9 requires O(1) time.
So the time complexity will be:
Could you explain me why we deduce that the time to sort the edges in line 4 is O(E*lgE)?
To sort a set of N items we use O(Nlg(N)) algorithm, which is quick sort, merge sort or heap sort. To sort E edges we therefore need O(Elg(E)) time. This however is not necessary in some cases, as we could use sorting algorithm with better complexity (read further).
Also how do we get that the total time complexity is O((V+E)α(V))?
I don't think total complexity is O((V+E)α(V)). That would be complexity of the 5-8 loop. O((V+E)α(V)) complexity comes from V MAKE-SET operations and E Union operations. To find out why we multiply that with α(V) you will need to read in depth analysis of disjoint set data structure in some algorithmic book.
How fast can you make Kruskal's algorithm run?
For first part, line 4, we have O(E*lg(E)) complexity and for second part, line 5-8, we have O((E+V)α(V)) complexity. This two summed up yield O(Elg(E)) complexity. If we use O(N*lg(N)) sort this can't be improved.
What if the edges weights are integers in the range from 1 to W for
some constant W?
If that is the case, than we could use counting sort for first part. Giving line 4 complexity of O(E+W) = O(E). In that case algorithm would have O((E+V)*α(V)) total complexity. Note that however O(E + W) in reality includes a constant that could be rather large and might be impractical for large W.
How does the time complexity depend on the weight of the edges?
As said, if weight of the edges is small enough we can use counting sort and speed up the algorithm.
EDIT:
In addition, suppose that all edge weights in a graph are integers
from 1 to |V|. How fast can you make Kruskal's algorithm run? I have
thought the following:
In order the Kruskal's algorithm to run faster, we can sort the edges
applying Counting Sort.
The line 1 requires O(1) time. The lines 2-3 require O(vα(|V|)) time.
The line 4 requires O(|V|+|E|) time. The lines 5-8 require
O(|E|α(|V|)) time. The line 9 requires O(1) time.
Your idea is correct, however you can make bounds smaller.
The lines 2-3 requires O(|V|) rather than O(|V|α(|V|)). We however simplified it to O(|V|α(|V|)) in previous calculations to make calculations easier.
With this you get the time of:
O(1) + O(|V|) + O(|V| + |E|) + O(|E|α(|V|)) + O(1) = O(|V| + |E|) + O(|E|α(|V|))
You can simplify this to either O((|V| + |E|) * α(|V|) or to O(|V| + |E|*α(|V|).
So while you were correct, since O((|V| + |E|) * α(|V|) < O((|V| + |E|) * lg(|E|)
Calculations for the |W| are analogous.
I was reading Dijkstras Algorithm in Chap. 24 and got confused with meaning of sparse graph. They say "If the graph is sufficiently sparse—in particular,E= o(V^2/lg V)-we can
improve the algorithm by implementing the min-priority queue with a binary minheap."
My questions
From where they have derived the expression E= o(V^2/lg V)for sparse graph?
Can't we use min-priority queue in case of dense graph. What will be the affect of it on Dijkstra's time complexity?
Reference-CLRS Page-662 3rd Ed.
Please Read:
Substitute that expression for E into the total running time, O((V + E)lg V), and you'll see that if E=o(V^2/lg V) the total will be o(V^2), which is an improvement over the O(V^2) running time of not using a minheap.
Once again, substitute. Let's assume a complete graph, E = V^2. Then, the running time becomes O((V + V^2)lg V) = O(V^2 lg V), which is worse than O(V^2).
I am comparing two algorithms, Prim's and Kruskal's.
I understand the basic concept of time complexity and when the two work best (sparse/dense graphs)
I found this on the Internet, but I am struggling to convert it to English.
dense graph: Prim = O(N2)
Kruskal = O(N2*log(N))
sparse graph: Prim = O(N2)
Kruskal = O(N log(N))
It's a bit of a long shot, but could anyone explain what is going on here?
Prim is O(N^2), where N is the number of vertices.
Kruskal is O(E log E), where E is the number of edges. The "E log E" comes from a good algorithm sorting the edges. You can then process it in linear E time.
In a dense graph, E ~ N^2. So Kruskal would be O( N^2 log N^2 ), which is simply O( N^2 log N ).
OK, here goes. O(N2) (2 = squared) means that the speed of the algorithm for large N varies as the square of N - so twice the size of graph will result in four times the time to compute.
The Kruskal rows are merely simplified, and assume that E = c * N2. c here is presumably a constant, that we can assume to be significantly smaller than N as N gets large. You need to know the following laws of logarithms: log(ab) = log a + log b and log(a^n) = n * log a. These two combined with the fact that log c << log N (is much less than and can be ignored) should let you understand the simplifications there.
Now, as for the original expressions and where they were derived from, you'd need to check the page you got these from. But I'm assuming that if you're looking at Prim's and Kruskal's then you will be able to understand the derivation, or at least that if you can't my explaining it to you is not actually going to help you in the long run...
Kruskal is sensitive to the number of edges (E) in a graph, not the number of nodes.
Prim however is only affected by the number of nodes (N), evaluting to O(N^2).
This means that in dense graphs where the number of edges approaches N^2 (all nodes connected) it's complexity factor of O(E*log(E)) is roughly equivalent to O(N^2*log(N)).
The c is a constant to account for the 'almost' and is irrelevant in O notation. Also log(N^2) is of the same order of magnitude as log(N) as logarithm outweighs the power of 2 by a substantial margin ( log(N^2) => 2*log(N) which in O notation is O(log(N)) ).
In a sparse graph E is closer to N giving you O(N*log(N)).
The thought is that in a dense graph, the number of edges is O(N^2) while in sparse graphs, the number of edges is O(N). So they're taking the O(E \lg E) and expanding it with this approximation of E in order to compare it directly to the running time of Prim's O(N^2).
Basically, it's showing that Kruskal's is better for sparse graphs and Prim's is better for dense graphs.
The two algorithms have big-O defined for different inputs (nodes and edges). So they are converting one to the other to compare them.
N is the number nodes in the graph E is the number of edges.
for a dense graph there are O(N^2) Edges
for a sparse graph there are O(N) Edges.
constants are of course irrelavent for big-O hence the c drops out
First: n is the number of vertices.
Prim is O(n^2) that part is easy enough.
Kruskal is O(Elog(E)) where E is the number of edges. in a dense graph, there are as many as N choose 2 edges, which is roughly n^2 (actually it's n(n-1)/2, but who's counting?) so, it's roughly n^2 log (n^2) which is 2n^2 log n which is O(n^2logn) which is bigger than O(n^2)
In a sparse graph, there are as few as n edges, so we have n log n which is less than O(n^2).