Is the time complexity of the following code O(NV^2)?
for i from 1 to N:
for j from 1 to V:
for k from 1 to A[i]://max(A) = V
z = z + k
yeah,whenever we talk about O-notation, we always think about the upper-bound(OR the worst case).
So,the complexity for this code will be equal to
O(N*V*maximum_value_of_A)
=O(N*V*V) // since,maximum value of A=V,so third loop can maximally iterate from 1 to V---V times
=O(N*V^2).
For sure it is O(NV^2) as it means the code is never slower than that. Because max(A) = V, you can say the worst case would be when at every index of A there is V. If so, then the complexity can be limited to O(NV*V).
You can calculate very roughly that the complexity of the for k loop can be O(avg(A)). This allows us to say that the whole function is Omega(NV*avg(A)), where avg(A) <= V.
Theta notation (meaning asympthotical complexity) would can be stated like Theta(NV*O(V)), O(V) representing complexity of a function which will never grow faster than V, but is not constant.
Related
I have come up with the following algorithm for a task. primarily the for-loop in line 23-24 that is making me unsure.
function TOP-DOWN-N(C, n, p)
let N[1...n, 1...C] be a new array initialized to -1 for all indicies
return N(C, n)
function N(C, n)
if N[C,n] >= 0 then
return N[C,n]
if C < 0 or n < 1 then
return 0
elif C = 0 then
return 1
elif C > 0 and i > 0 then
r = 0
for i = 0 to n do
r += N(C-p[n-i], n-(i+1))
N[C,n] = r
return N
Let's ignore the fact that this algorithm is implemented recursively. In general, if a dynamic programming algorithm is building an array of N results, and computing each result requires using the values of k other results from that array, then its time complexity is in Ω(Nk), where Ω indicates a lower bound. This should be clear: it takes Ω(k) time to use k values to compute a result, and you have to do that N times.
From the other side, if the computation doesn't do anything asymptotically more time-consuming than reading k values from the array, then O(Nk) is also an upper bound, so the time complexity is Θ(Nk).
So, by that logic we should expect that the time complexity of your algorithm is Θ(n2C), because it builds an array of size nC, computing each result uses Θ(n) other results from that array, and that computation is not dominated by something else.
However, your algorithm has an advantage over an iterative implementation because it doesn't necessarily compute every result in the array. For example, if the number 1 isn't in the array p then your algorithm won't compute N(C-1, n') for any n'; and if the numbers in p are all greater than or equal to C, then the loop is only executed once and the running time is dominated by having to initialize the array of size nC.
It follows that Θ(n2C) is the worst-case time complexity, and the best-case time complexity is Θ(nC).
for(i=1;i<=n;i=pow(2,i)) { print i }
What will be the time complexity of this?
Approximate kth term for value of i will be pow(2,(pow(2,pow(2,pow(2, pow(2,pow(2,...... k times)))))))
How can the above value, let's say kth value of i < n be solved for k.
What you have is similar to tetration(2,n) but its not it as you got wrong ending condition.
The complexity greatly depends on the domain and implementation. From your sample code I infer real domain and integers.
This function grows really fast so after 5 iterations you need bigints where even +,-,*,/,<<,>> are not O(1). Implementation of pow and print have also a great impact.
In case of small n<tetration(2,4) you can assume the complexity is O(1) as there is no asymptotic to speak of for such small n.
Beware pow is floating point in most languages and powering 2 by i can be translated into simple bit shift so let assume this:
for (i=1;i<=n;i=1<<i) print(i);
We could use previous state of i to compute 1<<i like this:
i0=i; i<<=(i-i0);
but there is no speedup on such big numbers.
Now the complexity of decadic print(i) is one of the following:
O( log(i)) // power of 10 datawords (like 1000000000 for 32 bit)
O((log(i))^2) // power of 2 datawords naive print implementation
O( log(i).log(log(i))) // power of 2 datawords subdivision or FFT based print implementation
The complexity of bit shift 1<<i and comparison i<=n is:
O(log(i)) // power of 2 datawords
So choosing the best implementation for print in power of 2 datawords lead to iteration:
O( log(i).log(log(i) + log(i) + log(i) ) -> O(log(i).log(log(i)))
At first look one would think we would need to know the number of iterations k from n:
n = tetration(2,k)
k = slog2(n)
or Knuth's notation which is directly related to Ackermann function:
n = 2↑↑k
k = 2↓↓n
but the number of iterations is so small in comparison to inner complexity of the stuff inside loop and next iterations grows so fast that the previous iteration is negligible fraction of the next one so we can ignore them all and only consider the last therm/iteration...
After all these assumptions I got final complexity:
O(log(n).log(log(n)))
I was trying to estimate the worst case scenario for an algorithm that looks like this (estimated complexity in comments are mine in which V is the number of Vertices and E is the number of Edges in a graph):
while(nodes.size()!=0) { // O(V) in which nodes is a LinkedList
Vertex u = Collections.min(nodes); // O(V)
nodes.remove(u); // O(1)
for(Map.Entry<Vertex, Integer> adjacency : u.adj.entrySet()) { // O(E)
// Some O(1) Statement
if(nodes.contains(v)) { // O(V)
// Some O(1) Statement
}
}
}
My question is very straightforward:
After every round in the while loop, the nodes LinkedList will go smaller and smaller.
Eventually, both Collections.min() and nodes.contains() operations will take less time every round.
My understanding is Big O Notation always considers the worst, thus the above complexities should be correct.
Otherwise, would you please explain the plot of how to figure out the correct Complexity in the above scenario?
nodes.contains has worst-case time complexity in Θ(V), the for-loop runs a number of times in Θ(E) and so has worst-case time complexity in Θ(V*E), Collections.min has worst-case time complexity in Θ(V), so the body of the while loop has worst-case time complexity in Θ(V+V*E), but V+V*E is itself Θ(V*E) (see later), so the body of the while loop has worst-case time complexity in Θ(V*E). The while loop executes V times. So the worst-case time to execute the algorithm is in Θ(V^2*E).
The simplification there -- replacing Θ(V+V*E) with Θ(V*E) -- is an acceptable one because we are looking at the general case of V>1. That is, V*E will always be a bigger number than V, so we can absorb V into a boundedly constant factor. It would also be correct to say the worst-cast time is in Θ(V^2+E*V^2), but one wouldn't use that since the simplified form is more useful.
Incidentally, as a matter of intuition, you can generally ignore the effect of containers being "used up" during an algorithm, such as with insertion sort having fewer and fewer items to look through, or this algorithm having fewer and fewer nodes to scan. Those effects turn into constant factors and disappear. It's only when you're eliminating an interesting number of elements each time, such as with the quickselect algorithm or binary search, that that sort of thing starts to affect the asymptotic runtime.
You can take the largest possible values at each step but this can give a final value that is too much of an overestimate. To make sure the value is accurate, you can leave taking the upper-bound until the end, but often it ends up being the same anyway.
When the value of V changes, then introduce another variable v which is the value for one specific iteration. Then the complexity of each iteration is v+(E*v). The total complexity is then the sum of each iteration:
sum(v = 1...V) v+(E*v)
= 1+1E + 2+2E + 3+3E + ... + V+VE - Expand the sum
= (1 + 2 + 3 + ... + V) + (1 + 2 + 3 + ... + V)E - Regroup terms
= (V^2 + V)/2 + (V^2 + V)E/2 - Sum of arithmetic series
= (V^2 + V + EV^2 + EV)/2 - Addition of fractions
= O(EV^2) - EV^2 greater than all other terms
Yes, these look correct. And putting them together you will get time O(V*(V+E)). (Correction, O((1+E)*V^2) - I had missed the O(V) inside of an O(E) inner loop.)
However there is an important correction to your understanding. Big O notation not always worst case. The notation is a way of estimating the growth of mathematical functions. Whether those functions are worst case, or average, or what they measure is entirely up to the problem at hand. For example quicksort can be implemented in O(n^2) worst case running time, with O(n log(n)) average running time, using O(log(n)) extra memory on average and O(n) extra memory in the worst case.
In a program, I’m using two data structures
1: An array of pointers of size k, each pointer points to a link lists(hence, total ‘k’ lists) . Total number of nodes in all the lists = M…..(something like hashing with separate chaining, k is fixed, M can vary)
2: Another array of integers of size M (where M=number of nodes above)
Question is: What is the overall space complexity of the program? Is it something like below?
First part: O(k+M) or just O(M)….both are correct I guess!
Second part: O(2M) or just O(M)…again both correct?
Overall O(k+M) + O(2M) ==> O(max(k+M, 2M)
Or just O(M)?
Please help.
O(K+M) is O(M) if the M is always greater than K. So, the final result is O(M).
First part: O(k+M) is not correct its just O(M)
Second part: O(2M) is not correct because we don't use constants in order so correct is O(M)
Overall O(M) + O(M) ==> O(M).
Both are correct in the two cases. But since O(k+M) = O(M), supposing k constant, everybody will use the simplest notation, which is O(M).
For the second part, a single array is O(M).
For the overall, it would be O(k+M+M) = O(max(k+M,2M)) = O(M) (we can "forget" multiplicative and additive constants in the big-O notation - except if your are in constant time).
As a reminder g(x) = O(f(x)) iff there exist x0 and c such that x>x0 implies g(x) >= c.f(x)
Here is an algorithm for finding kth smallest number in n element array using partition algorithm of Quicksort.
small(a,i,j,k)
{
if(i==j) return(a[i]);
else
{
m=partition(a,i,j);
if(m==k) return(a[m]);
else
{
if(m>k) small(a,i,m-1,k);
else small(a,m+1,j,k);
}
}
}
Where i,j are starting and ending indices of array(j-i=n(no of elements in array)) and k is kth smallest no to be found.
I want to know what is the best case,and average case of above algorithm and how in brief. I know we should not calculate termination condition in best case and also partition algorithm takes O(n). I do not want asymptotic notation but exact mathematical result if possible.
First of all, I'm assuming the array is sorted - something you didn't mention - because that code wouldn't otherwise work. And, well, this looks to me like a regular binary search.
Anyway...
The best case scenario is when either the array is one element long (you return immediately because i == j), or, for large values of n, if the middle position, m, is the same as k; in that case, no recursive calls are made and it returns immediately as well. That makes it O(1) in best case.
For the general case, consider that T(n) denotes the time taken to solve a problem of size n using your algorithm. We know that:
T(1) = c
T(n) = T(n/2) + c
Where c is a constant time operation (for example, the time to compare if i is the same as j, etc.). The general idea is that to solve a problem of size n, we consume some constant time c (to decide if m == k, if m > k, to calculate m, etc.), and then we consume the time taken to solve a problem of half the size.
Expanding the recurrence can help you derive a general formula, although it is pretty intuitive that this is O(log(n)):
T(n) = T(n/2) + c = T(n/4) + c + c = T(n/8) + c + c + c = ... = T(1) + c*log(n) = c*(log(n) + 1)
That should be the exact mathematical result. The algorithm runs in O(log(n)) time. An average case analysis is harder because you need to know the conditions in which the algorithm will be used. What is the typical size of the array? The typical size of k? What is the mos likely position for k in the array? If it's in the middle, for example, the average case may be O(1). It really depends on how you use this.