I have a certain problem understanding the complexity of the Djisktra algorithm and hope someone can correct me.
For my example I took a complete graph with n vertices.
You pick a starting vertex, lets say a1, mark it, and then compute all n-1 weights on the edges. O(n)
You pick the smallest one. Let's say vertex a2 and mark it. O(n)
After that you compute n-2 new weights on the edges and look for the next yet unmarked vertex to add your set of marked vertices.
And so on...
The algorithm runs til you could mark all vertices. Complexity: n-1 + n-2 + ... + n - (n - 1) = Binom(n,2) which is in O(n^2), not O(n*ln(n)) what I want.
I read about many many times people use a heap for optimization, however I still don't see how to avoid Binom(n,2) computations.
I have to be wrong at some point, but do not see where.
If you have a complete graph, then of course you can't do any better than O(n^2) -- because, that's the size of your input.
If you don't have a complete graph, and are storing your edges in an adjacency list, then you can do better. You still need to look at all your edges, but with a priority queue you can manage O(e + n log n) where e is the number of edges in your adjacency list.
Related
In the running time of Kruskal's algorithm, the time for sorting edges is mlogm when we use quick sort. But as descried in 'Algorithm Design', the max number of edges is smaller than the square of number of nodes, so sorting the edge costs at most mlogn time. I can't understand this statement as even in a complete graph, shouldn't the max number of edges be n(n-1)/2?
You're right in that a complete graph has n(n-1)/2 edges. Since n(n-1)/2 = n^2/2 - n/2 < n^2, the statement you quote holds true.
If your source of confusion is "why did the author say less than n2 instead of n(n-1)/2", then it's likely because n2 is a simpler expression and a sufficient one to prove that paragraph is trying to say, namely that the cost of sorting is at most m log n.
If you prove that something holds whenever the number of edges is <= n2, then it will also hold in the real world, since the number of edges is <=n/(n-1)/2 <= n2.
I know that there are a ton of questions out there about the time complexity of BFS which is : O(V+E)
However I still struggle to understand why is the time complexity O(V+E) and not O(V*E)
I know that O(V+E) stands for O(max[V,E]) and my only guess is that it has something to do with the density of the graph and not with the algorithm itself unlike say Merge Sort where it's time complexity is always O(n*logn).
Examples I've thought of are :
A Directed Graph with |E| = |V|-1 and yeah the time complexity will be O(V)
A Directed Graph with |E| = |V|*|V-1| and the complexity would in fact be O(|E|) = O(|V|*|V|) as each vertex has an outgoing edge to every other vertex besides itself
Am I in the right direction? Any insight would be really helpful.
Your "examples of thought" illustrate that the complexity is not O(V*E), but O(E). True, E can be a large number in comparison with V, but it doesn't matter when you say the complexity is O(E).
When the graph is connected, then you can always say it is O(E). The reason to include V in the time complexity, is to cover for the graphs that have many more vertices than edges (and thus are disconnected): the BFS algorithm will not only have to visit all edges, but also all vertices, including those that have no edges, just to detect that they don't have edges. And so we must say O(V+E).
The complexity comes off easily if you walk through the algorithm. Let Q be the FIFO queue where initially it contains the source node. BFS basically does the following
while Q not empty
pop u from Q
for each adjacency v of u
if v is not marked
mark v
push v into Q
Since each node is added once and removed once then the while loop is done O(V) times. Also each time we pop u we perform |adj[u]| operations where |adj[u]| is the number of
adjacencies of u.
Therefore the total complexity is Sum (1+|adj[u]|) over all V which is O(V+E) since the sum of adjacencies is O(E) (2E for undirected graph and E for a directed one)
Consider a situation when you have a tree, maybe even with cycles, you start search from the root and your target is the last leaf of your tree. In this case you will traverse all the edges before you get into your destination.
E.g.
0 - 1
1 - 2
0 - 2
0 - 3
In this scenario you will check 4 edges before you actually find a node #3.
It depends on how the adjacency list is implemented. A properly implemented adjacency list is a list/array of vertices with a list of related edges attached to each vertex entry.
The key is that the edge entries point directly to their corresponding vertex array/list entry, they never have to search through the vertex array/list for a matching entry, they can just look it up directly. This insures that the total number of edge accesses is 2E and the total number of vertex accesses is V+2E. This makes the total time O(E+V).
In improperly implemented adjacency lists, the vertex array/list is not directly indexed, so to go from an edge entry to a vertex entry you have to search through the vertex list which is O(V), which means that the total time is O(E*V).
Lets say I have implemented dijkstras using a PriorityQueue so that adding and removing from the unvisited nodes takes O(log n).
The PQ will contain at most E nodes so to empty it we get O(E). While PQ is not empty we take the best node and remove it, visit if not visited and go through all of its neighbors (potentially adding them to the PQ).
What I do not understand: How can going through all neighbors (at worst V) for at worst E items not have the time-complexity of O(E*V). I have seen so many explanation that we are supposed to just look at the operations and observe how many times they will execute and draw our conclusions from this. I do not see how we can disregard the fact that we are looping through V neighbors, an empty for-loop of n items is still O(n)?
For me the final complexity seems to be O(V + E*V log E) instead of O(V + V log E). I mean there are a lot of variances but the main point is I am missing something trivial :P
First point of terminology that you seem to have confused. E is not the number of items, it is the number of edges between vertices. V is the number of vertices, which (depending on context) is likely to be the number of items.
Next, "this vertex is a neighbor of that vertex" means that there is an edge between them. Each edge contributes 2 neighbor relationships. (One in each direction.) Therefore 2 E is the number of neighbor relationships that can exist, total.
Your intuition that every one of V nodes can have up to V-1 neighbors for a total of V2-V neighbor relationships is correct - but you can tell how close you are to that worst case from the number of edges.
Therefore we wind up with the following potential work:
for each of E edges:
for each vertex on that edge:
O(1) work to check whether it was processed yet
(processing the vertex if needed accounted for separately)
for each of V vertices:
O(log(V)) to add to the priority queue
O(log(V)) to remove from the priority queue
(processing its edges accounted for separately
The first chunk is O(E). The second chunk is O(V log(V)). The total is O(E + V log(V)).
Hopefully this explanation clarifies why the complexity is what it is.
Some definitions first: an "accumulated graph" is a graph where edges are added to it later on. The topic is orientation problem (making an undirected graph directed), when our goal is to make the maximum out-degree the lowest possible.
Now, they gave this algorithm: "When the edge (u,v) is added, we will direct it from the vertex with the lower outdegree to the higher one. if equal, choose randomly."
For example, if outdegree(u) = 3 and outdegree(v) = 4, and we just added (u,v), the algorithm will direct it from u-->v. If their outdegree was equal, both u,v and v,u were possible.
The question is to prove an upper bound of O(log n) for the maximum outdegree possible. The second question is to form a graph where this algorithm will lead to Omega(log n) maximum outdegree.
So far, all I can point out is that the question is wrong.
First of all, my friend suggested that they actually meant O(log m) [m = # of edges], since that for n vertices in an undirected graph, we have at most n(n-1)/2 edges, which is eventually O(n^2). If we assume that in a complete graph, every node's outdegree is log(n), we get total of n*log(n) outdegrees, which is significally smaller than n^2 (and doesn't make sense because all outdegrees should sum to the # of edges.).
I came up with this algorithm that proves that because we choose randomly in cases both are equal, the highest possible outdegree is of O(n):
Direct n-1 vertices to the last one (possible in the worst-case scenario where all orientations are in the same direction). We now have n-1 vertices with an outdegree of 1, and 1 with 0. Then repeat recusively to accomplish n+(n-1)+(n-2)+...+1+0 outdegrees.
I might be understanding the problem wrong but that's what I got so far.
EDIT: The instructor edited the question and said the graph in this question is a forest (which means you can have at most V-1 edges).
I believe I managed to prove the first question:
Imagine this: Since the only way (worst case scenario) to have an outdegree of 2 is to connect two nodes with an outdegree of 1, we have to have 4 nodes where A is connected to B and C is connected to D in order to add an edge from A to C and make one of them with an outdegree of 2. Because this is the smallest tree possible to create an outdegree of 2, the only way to create an outdegree of 3 is to build two identical trees and connect them together again. If we repeat this process until we reach n vertices, we will notice we eventually get to 1+2+4+8+...+log n as our outdegree.
Right now I keep thinking about the second question.
First of all, because m = O(n^2), we have O(log m) = O(log n^2) = O(2 * log(n)) = O(log n). In other words, the problem doesn't contradict with the reasoning suggested by your friend.
But you are right that the problem (as you've stated) is not correct -- the worst-case outdegree is O(n), using the process you described. (But the highest outdegree is n-1, not n as you've written.)
Perhaps the problem actually asks you to bound the maximum expected outdegree?
I strongly suspect that the intended problem was to bound the competitive ratio between this online algorithm and the offline optimum (e.g., if the offline optimum has maximum out degree 1, then the online solution has maximum out degree O(log n); if the offline optimum has maximum out degree √n, then the online solution has maximum out degree O((√n) log n)). The lower bound is easy enough (see the lower bound for union-by-rank union-find for inspiration), and I conjecture that the upper bound is attainable as well.
I am looking at the algorithm of Breadth-first search, that is the following:
BFS(G,s)
for each u ∈ V\ {s}
color(u)=white
d(u)=oo
π(u)=NIL
color(s)=GRAY
d(s)=0
π(s)=NIL
Q=∅
ENQUEUE(Q,s)
while (Q!=∅)
u=DEQUEUE(Q)
for each v ∈ Adj(u)
if (color(v)=white)
color(v)=GRAY
d(v)=d(u)+1
π(v)=u
ENQUEUE(Q,v)
color(v)=BLACK
I thought that it is like that:
The time complexity of the first for loop is O(V).
The time complexity of the while-loop is O(V), while the time complexity of the for loop that is executed inside the while-loop is O(E).
Then the time complexity of the algorithm would be O(VE+E)=O(VE).
But, according to my textbook it is O(V+E).
So, have I calculated something wrong?
Well, since it's a worst case your analysis is correct but not tight. The while loop runs as long as there are vertices in the queue. Only vertices whose color is WHITE are enqueued in which case their color becomes GRAY so they will never be enqueued again. This tells you that the queue can get as large as V.
In each iteration you iterate over the adjacency list of a vertex so the overall running time is the sum of the lengths of the adjacency lists + V. The sum is O(E). The running time is O(V+E).
It might be useful to remember that in an undirected graph, the following holds: sum of degrees of all vertices = 2 * E. To see this, note that every edge (x,y) will be counted twice: once in the degree of x and once in the degree of y.
No, the algorithm really is O(VE), but it's also O(V + E). Your bound is loose because each edge is considered at most twice when scanning the adjacency lists.