I read here that for Undirected graph the space complexity is O(V + E) when represented as a adjacency list where V and E are number of vertex and edges respectively.
My analysis is, for a completely connected graph each entry of the list will contain |V|-1 nodes then we have a total of |V| vertices hence, the space complexity seems to be O(|V|*|V-1|) which seems O(|V|^2) what I am missing here?
You analysis is correct for a completely connected graph. However, note that for a completely connected graph the number of edges E is O(V^2) itself, so the notation O(V+E) for the space complexity is still correct too.
However, the real advantage of adjacency lists is that they allow to save space for the graphs that are not really densely connected. If the number of edges is much smaller than V^2, then adjacency lists will take O(V+E), and not O(V^2) space.
Note that when you talk about O-notation, you usually have three types of variables (or, well, input data in general). First is the variables dependence on which you are studying; second are those variables that are considered constant; and third are kind of "free" variables, which you usually assume to take the worst-case values. For example, if you talk about sorting an array of N integers, you usually want to study the dependence of sorting time on N, so N is of the first kind. You usually consider the size of integers to be constant (that is, you assume that comparison is done in O(1), etc.), and you usually consider the particular array elements to be "free", that is, you study that runtime for the worst possible combination of particular array elements. However, you might want to study the same algorithm from a different point of view, and it will lead to a different expression of complexity.
For graph algorithms, you can, of course, consider the number of vertices V to be of first kind, and the number of edges to be the third kind, and study the space complexity for given V and for the worst-case number of edges. Then you indeed get O(V^2). But it is also often useful to treat both V and E as variables of the first type, thus getting the complexity expression as O(V+E).
Size of array is |V| (|V| is the number of nodes). These |V| lists each have the degree which is denoted by deg(v). We add up all those, and apply the Handshaking Lemma.
∑deg(v)=2|E| .
So, you have |V| references (to |V| lists) plus the number of nodes in the lists, which never exceeds 2|E| . Therefore, the worst-case space (storage) complexity of an adjacency list is O(|V|+2|E|)= O(|V|+|E|).
Hope this helps
according to u r logic, total space = O(v^2-v)
as total no. of connections(E) = v^2 - v,
space = O(E).
even I used to think the same,
But if you have total no. of connections(E) much lesser than no. of vertices(V),
lets say out of 10 people (V=10),
only 2 knows each other, therefore (E=2)
then according to your logic
space = O(E) = O(2)
but we in actual we have to allocate much larger space
which is
space = O(V+E) = O(V+2) = O(V)
that's why,
space = O(V+E), this will work if V > E or E > V
Related
I just want to have confirmation of the time complexity of the algorithm below.
EDIT: in what follows, we do not assume that the optimal data structures are being used. However, feel free to propose solutions that use such structures (clearly mentioning what data structures are used and what beneficial impact they have on complexity).
Notation: in what follows n=|V|, m=|E|, and avg_neigh denotes the average number of neighbors of any given node in the graph.
Assumption: the graph is unweighted, undirected, and we have its adjacency list representation loaded in memory (a list of lists containing the neighbors of each vertex).
Here is what we have so far:
Line 1: computing the degrees is O(n), as it simply involves getting the length of each sublist in the adjacency list representation, i.e., performing n O(1) operations.
Line 3: finding the lowest value requires checking all values, which is O(n). Since this is nested in the while loop which visits all nodes once, it becomes O(n^2).
Lines 6-7: removing vertex v is O(avg_neigh^2) as we know the neighbors of v from the adjacency list representation and removing v from each of the neighbors' sublists is O(avg_neigh). Lines 6-7 are nested in the while loop, so it becomes O(n * avg_neigh^2).
Line 9: it is O(1), because it simply involves getting the length of one list. It is nested in the for loop and while loop so it becomes O(n * avg_neigh).
Summary: the total complexity is O(n) + O(n^2) + O(n * avg_neigh^2) + O(n * avg_neigh) = O(n^2).
Note 1: if the length of each sublist is not available (e.g., because the adjacency list cannot be loaded in memory), computing the degrees in line 1 is O(n * avg_neigh), as each list features avg_neigh elements on average and there are n sublists. And line 9, the total complexity becomes O(n * avg_neigh^2).
Note 2: if the graph is weighted, we can store the edge weights in the adjacency list representation. However, getting the degrees in line 1 requires summing over each sublist and is now O(n * avg_neigh) if the adjacency list is loaded in RAM and O(n * avg_neigh^2) else. Similarly, line 9 becomes O(n * avg_neigh^2) or O(n * avg_neigh^3).
There is an algorithm that (1) is recognizable as an implementation of Algorithm 1 and (2) runs in time O(|E| + |V|).
First, let's consider the essence of Algorithm 1. Until the graph is empty, do the following: record the priority of the node with the lowest priority as the core number of that node, and delete that node. The priority of a node is defined dynamically as the maximum over (1) its degree in the residual graph and (2) the core numbers of its deleted neighbors. Observe that the priority of a node never increases, since its degree never increases, and a higher-priority neighbor would not be deleted first. Also, priorities always decrease by exactly one in each outer loop iteration.
The data structure support sufficient to achieve O(|E| + |V|) time splits nicely into
A graph that, starting from the initial graph (initialization time O(|E| + |V|)), supports
Deleting a node (O(degree + 1), summing to O(|E| + |V|) over all nodes),
Getting the residual degree of a node (O(1)).
A priority queue that, starting from the initial degrees (initialization time O(|V|)), supports
Popping the minimum priority element (O(1) amortized),
Decreasing the priority of an element by one (O(1)), never less than zero.
A suitable graph implementation uses doubly linked adjacency lists. Each undirected edge has two corresponding list nodes, which are linked to each other in addition to the previous/next nodes originating from the same vertex. Degrees can be stored in a hash table. It turns out we only need the residual degrees to implement this algorithm, so don't bother storing the residual graph.
Since the priorities are numbers between 0 and |V| - 1, a bucket queue works very nicely. This implementation consists of a |V|-element vector of doubly linked lists, where the list at index p contains the elements of priority p. We also track a lower bound on the minimum priority of an element in the queue. To find the minimum priority element, we examine the lists in order starting from this bound. In the worst case, this costs O(|V|), but if we credit the algorithm whenever it increases the bound and debit it whenever the bound decreases, we obtain an amortization scheme that offsets the variable cost of this scanning.
If choosing a data structure that can support efficient lookup of min value and efficient deletions (such as self balancing binary tree or min heap), this can be done in O(|E|log|V|).
Creating the data structure in line 1 is done in O(|V|) or
O(|V|log|V|).
The loop goes exactly once over each node v in V.
For each such node, it needs to peek the min value, and adjust the DS so you can peek at the next min efficiently next iteration. O(log|V|) is required for that (otherwise, you could do heap sort in o(nlogn) - note little o notation here).
Getting neighbors, and removing items from V and E could be done in O(|neighbors|) using efficient set implementation, such as a hash table. Since each edge and node is chosen exactly once for this step, it sums over all iterations in O(|V| + |E|).
The for loop over neighbors, again, sums to O(|E|), thanks to the fact that each edge is counted exactly once here.
In this loop, however, you might need to update p, and such an update costs O(log|V|), so this sums to O(|E|log|V|)
This sums in O(|V|log|V| + |E|log|V|). Assuming the graph is not very sparse (|E| is in Omega(|V|) - this is O(|E|log|V|).
I could see in many books that the worst case memory requirement for graphs is O(V). But, If I have not mistaken, graphs are usually represented as adjacency matrix and not by the creation of nodes ( as in linked list / trees ). So, for a graph containing 5 vertices, I need 5x5 matrix which is O(V^2). They why do they say it as O(V)?
Am I missing something somewhere? Sorry if the question is too naive.
The three main ways of representing a graph are:
Adjacency matrix - Θ(|V|²) space.
Adjacency list - Θ(|V| + |E|) space.
Collection of node objects/structs with pointers to one another - This is basically just another way of representing an adjacency list. Θ(|V| + |E|). (Remember that pointers require memory too.)
Since we're talking worst case, all of these reduce to Θ(|V|²) since that's the maximum number of edges in a graph.
I'm guessing you misread the book. They probably weren't talking about the space required to store the graph structure itself, but rather the amount of extra space required for some graph algorithm.
If what you say is true, it's possible they are referring to other ways to represent a graph, other than using an adjacency matrix and are possibly making an edge-density assumption. One way is, for each vertex, just store a list of pointers / references to its neighbors (called an adjacency list). This would be O(|V| + |E|). If we assume |E| ~ |V|, which is an assumption we do sometimes see, then we have O(|V|) space. But note that in the worst-case, |E| ~ |V|^2 and so even this approach to representing a graph is O(|V|^2) in the worst case.
Look, it's quite simple; there's no escaping the fact that in the worst case |E| ~ |V|^2. There can not possibly be, in general, a representation of E that in the worst case is not O(|V|^2).
But, it'd be nice to have an exact quote to work with. This is important. We don't want to find ourselves tearing apart your misunderstanding of a correct statement.
Check if 2 tree nodes are related (i.e. ancestor-descendant)
solve it in O(1) time, with O(N) space (N = # of nodes)
pre-processing is allowed
That's it. I'll be going to my solution (approach) below. Please stop if you want to think yourself first.
For a pre-processing I decided to do a pre-order (recursively go through the root first, then children) and give a label to each node.
Let me explain the labels in details. Each label will consist of comma-separated natural numbers like "1,2,1,4,5" - the length of this sequence equals to (the depth of the node + 1). E.g. the label of the root is "1", root's children will have labels "1,1", "1,2", "1,3" etc.. Next-level nodes will have labels like "1,1,1", "1,1,2", ..., "1,2,1", "1,2,2", ...
Assume that "the order number" of a node is the "1-based index of this node" in the children list of its parent.
Common rule: node's label consists of its parent label followed by comma and "the order number" of the node.
Thus, to answer if two nodes are related (i.e. ancestor-descendant) in O(1), I'll be checking if the label of one of them is "a prefix" of the other's label. Though I'm not sure if such labels can be considered to occupy O(N) space.
Any critics with fixes or an alternative approach is expected.
You can do it in O(n) preprocessing time, and O(n) space, with O(1) query time, if you store the preorder number and postorder number for each vertex and use this fact:
For two given nodes x and y of a tree T, x is an ancestor of y if and
only if x occurs before y in the preorder traversal of T and after y
in the post-order traversal.
(From this page: http://www.cs.arizona.edu/xiss/numbering.htm)
What you did in the worst case is Theta(d) where d is the depth of the higher node, and so is not O(1). Space is also not O(n).
if you consider a tree where a node in the tree has n/2 children (say), the running time of setting the labels will be as high as O(n*n). So this labeling scheme wont work ....
There are linear time lowest common ancestor algorithms(at least off-line). For instance have a look here. You can also have a look at tarjan's offline LCA algorithm. Please note that these articles require that you know the pairs for which you will be performing the LCA in advance. I think there are also online linear time precomputation time algorithms but they are very complex. For instance there is a linear precomputation time algorithm for the range minimum query problem. As far as I remember this solution passed through the LCA problem twice . The problem with the algorithm is that it had such a large constant that it require enormous input to be actually faster then the O(n*log(n)) algorithm.
There is much simpler approach that requires O(n*log(n)) additional memory and again answers in constant time.
Hope this helps.
I was talking with a student the other day about the common complexity classes of algorithms, like O(n), O(nk), O(n lg n), O(2n), O(n!), etc. I was trying to come up with an example of a problem for which solutions whose best known runtime is super-exponential, such as O(22n), but still decidable (e.g. not the halting problem!) The only example I know of is satisfiability of Presburger arithmetic, which I don't think any intro CS students would really understand or be able to relate to.
My question is whether there is a well-known problem whose best known solution has runtime that is superexponential; at least ω(n!) or ω(nn). I would really hope that there is some "reasonable" problem meeting this description, but I'm not aware of any.
Maximum Parsimony is the problem of finding an evolutionary tree connecting n DNA sequences (representing species) that requires the fewest single-nucleotide mutations. The n given sequences are constrained to appear at the leaves; the tree topology and the sequences at internal nodes are what we get to choose.
In more CS terms: We are given a bunch of length-k strings that must appear at the leaves of some tree, and we have to choose a tree, plus a length-k string for each internal node in the tree, so as to minimise the sum of Hamming distances across all edges.
When a fixed tree is also given, the optimal assignment of sequences to internal nodes can be determined very efficiently using the Fitch algorithm. But in the usual case, a tree is not given (i.e. we are asked to find the optimal tree), and this makes the problem NP-hard, meaning that every tree must in principle be tried. Even though an evolutionary tree has a root (representing the hypothetical ancestor), we only need to consider distinct unrooted trees, since the minimum number of mutations required is not affected by the position of the root. For n species there are 3 * 5 * 7 * ... * (2n-5) leaf-labelled unrooted binary trees. (There is just one such tree with 3 species, which has a single internal vertex and 3 edges; the 4th species can be inserted at any of the 3 edges to produce a distinct 5-edge tree; the 5th species can be inserted at any of these 5 edges, and so on -- this process generates all trees exactly once.) This is sometimes written (2n-5)!!, with !! meaning "double factorial".
In practice, branch and bound is used, and on most real datasets this manages to avoid evaluating most trees. But highly "non-treelike" random data requires all, or almost all (2n-5)!! trees to be examined -- since in this case many trees have nearly equal minimum mutation counts.
Showing all permutation of string of length n is n!, finding Hamiltonian cycle is n!, minimum graph coloring, ....
Edit: even faster Ackerman functions. In fact they seems without bound function.
A(x,y) = y+1 (if x = 0)
A(x,y) = A(x-1,1) (if y=0)
A(x,y) = A(x-1, A(x,y-1)) otherwise.
from wiki:
A(4,3) = 2^2^65536,...
Do algorithms to compute real numbers to a certain precision count? The formula for the area of the Mandelbrot set converges extremely slowly; 10118 terms for two digits, 101181 terms for three.
This is not a practical everyday problem, but it's a way to construct relatively straightforward problems of increasing complexity.
The Kolmogorov complexity K(x) is the size of the smallest program that outputs the string $x$ on a pre-determined universal computer U. It's easy to show that most strings cannot be compressed at all (since there are more strings of length n than programs of length n).
If we give U a maximum running time (say some polynomial function P), we get a time-bounded Kolmogorov complexity. The same counting argument holds: there are some strings that are incompressible under this time bounded Kolmogorov complexity. Let's call the first such string (of some length n) xP
Since the time-bounded Kolmogorov complexity is computable, we can test all strings, and find xP
Finding xP can't be done in polynomial time, or we could use this algorithm to compress it, so finding it must be a super-polynomial problem. We do know we can find it in exp(P) time, though. (Jumping over some technical details here)
So now we have a time-bound E = exp(P). We can repeat the procedure to find xE, and so on.
This approach gives us a decidable super-F problem for every time-constructible function F: find the first string of length n (some large constant) that is incompressible under time-bound F.
So I'm curious to know what the running time for the algorithm is on on priority queue implemented by a sorted list/array. I know for an unsorted list/array it is O((n^2+m)) where n is the number of vertices and m the number of edges. Thus that equates to O(n^2) time. But would it be faster if i used an sorted list/array...What would the running time be? I know extractmin would be constant time.
Well, Let's review what we need for dijkstra's algorithm(for future reference, usually vertices and edges are used as V and E, for example O(VlogE)):
Merging together all the sorted adjacency lists: O(E)
Extract Minimum : O(1)
Decrease Key : O(V)
Dijkstra uses O(V) extract minimum operations, and O(E) decrease key operations, therefore:
O(1)*O(V) = O(V)
O(E)*O(V) = O(EV) = O(V^2)
Taking the most asymptotically significant portion:
Eventual asymptotic runtime is O(V^2).
Can this be made better? Yes. Look into binary heaps, and better implementations of priority queues.
Edit: I actually made a mistake, now that I look at it again. E cannot be any higher than V^2, or in other words E = O(V^2).
Therefore, in the worst case scenario, the algorithm that we concluded runs in O(EV) is actually O(V^2 * V) == O(V^3)
I use SortedList
http://blog.devarchive.net/2013/03/fast-dijkstras-algorithm-inside-ms-sql.html
It is faster about 20-50 times than sorting List once per iteration