I have been doing a program in which we are popping the elements of the queue until the queue becomes empty.The thing is that we know that time complexity of pop operation is O(1) and while it is running in a loop.So we can say we have a loop that is running O(1) operations for the time till the queue is not empty.
Suppose q has some number of elements say 1,2,3,4,5,6
while(!q->empty())
{
cout<<q->front()<<" ";
q->pop();
}
And I have read on geeksforgeeks that if loop is running for a constant number of times and is running O(1) operations in it..then that loop is considered to have time complexity of O(1).
This was said like that:
A loop or recursion that runs a constant number of times is also considered as O(1). For example the following loop is O(1).
// Here c is a constant
for (int i = 1; i <= c; i++) {
// some O(1) expressions
}
So the while loop for queue also has O(1) or O(n) or I am mistaken . Plz help me to clear my ambiguity.
The link for geeksforgeeks article is:
https://www.geeksforgeeks.org/analysis-of-algorithms-set-4-analysis-of-loops/
You can look at it this way: if the number of steps in your algorithm does not depend on the size of your data n - it has a constant complexity.
If you have to do "something" once for each of your n - you have a linear complexity, or O(n). That's the case with your queue example.
Related
I was trying to estimate the worst case scenario for an algorithm that looks like this (estimated complexity in comments are mine in which V is the number of Vertices and E is the number of Edges in a graph):
while(nodes.size()!=0) { // O(V) in which nodes is a LinkedList
Vertex u = Collections.min(nodes); // O(V)
nodes.remove(u); // O(1)
for(Map.Entry<Vertex, Integer> adjacency : u.adj.entrySet()) { // O(E)
// Some O(1) Statement
if(nodes.contains(v)) { // O(V)
// Some O(1) Statement
}
}
}
My question is very straightforward:
After every round in the while loop, the nodes LinkedList will go smaller and smaller.
Eventually, both Collections.min() and nodes.contains() operations will take less time every round.
My understanding is Big O Notation always considers the worst, thus the above complexities should be correct.
Otherwise, would you please explain the plot of how to figure out the correct Complexity in the above scenario?
nodes.contains has worst-case time complexity in Θ(V), the for-loop runs a number of times in Θ(E) and so has worst-case time complexity in Θ(V*E), Collections.min has worst-case time complexity in Θ(V), so the body of the while loop has worst-case time complexity in Θ(V+V*E), but V+V*E is itself Θ(V*E) (see later), so the body of the while loop has worst-case time complexity in Θ(V*E). The while loop executes V times. So the worst-case time to execute the algorithm is in Θ(V^2*E).
The simplification there -- replacing Θ(V+V*E) with Θ(V*E) -- is an acceptable one because we are looking at the general case of V>1. That is, V*E will always be a bigger number than V, so we can absorb V into a boundedly constant factor. It would also be correct to say the worst-cast time is in Θ(V^2+E*V^2), but one wouldn't use that since the simplified form is more useful.
Incidentally, as a matter of intuition, you can generally ignore the effect of containers being "used up" during an algorithm, such as with insertion sort having fewer and fewer items to look through, or this algorithm having fewer and fewer nodes to scan. Those effects turn into constant factors and disappear. It's only when you're eliminating an interesting number of elements each time, such as with the quickselect algorithm or binary search, that that sort of thing starts to affect the asymptotic runtime.
You can take the largest possible values at each step but this can give a final value that is too much of an overestimate. To make sure the value is accurate, you can leave taking the upper-bound until the end, but often it ends up being the same anyway.
When the value of V changes, then introduce another variable v which is the value for one specific iteration. Then the complexity of each iteration is v+(E*v). The total complexity is then the sum of each iteration:
sum(v = 1...V) v+(E*v)
= 1+1E + 2+2E + 3+3E + ... + V+VE - Expand the sum
= (1 + 2 + 3 + ... + V) + (1 + 2 + 3 + ... + V)E - Regroup terms
= (V^2 + V)/2 + (V^2 + V)E/2 - Sum of arithmetic series
= (V^2 + V + EV^2 + EV)/2 - Addition of fractions
= O(EV^2) - EV^2 greater than all other terms
Yes, these look correct. And putting them together you will get time O(V*(V+E)). (Correction, O((1+E)*V^2) - I had missed the O(V) inside of an O(E) inner loop.)
However there is an important correction to your understanding. Big O notation not always worst case. The notation is a way of estimating the growth of mathematical functions. Whether those functions are worst case, or average, or what they measure is entirely up to the problem at hand. For example quicksort can be implemented in O(n^2) worst case running time, with O(n log(n)) average running time, using O(log(n)) extra memory on average and O(n) extra memory in the worst case.
I have am trying to learn Big O and am confused on an algorithm I just came across. The algorithm is:
void pairs(int[] array){
for (int i=0; i < array.length; i++){
for (int j=i+1; j<array.length; j++){
System.out.println(array[i]+","+array[j]);
}
}
}
I think the first for loop is O(n) and I know the second for loop is O(1/2*n(n+1)). The answer to the problem was that the run time for the function is O(n^2). I simplified O(1/2*n(n+1)) to O(1/2*(n^2+n)). So I'm confused because I thought that you needed to multiply the two run time terms since the for loop is nested, which would give O(n) * O(1/2*(n^2+n)). I simplified this to O(1/2n^3 + 1/2n^2). From what I understand of Big O, you only keep the largest term so this would reduce to O(n^3). Can anyone help me out with where I went wrong? Not sure how the answer is O(n^2) instead of O(n^3).
When you say the inner loop is O(1/2*n(n+1)), you are actually describing the big-O complexity of both loops.
To say that the outer loop has complexity O(N) basically means its body runs N times. But for your calculation of the inner loop's complexity, you already took account of all iterations of the outer loop, because you added up the number of times the inner loop runs over all iterations of the outer loop. If you multiply again by N, you would be saying that the outer loop itself is re-run another N times.
Put another way, what your analysis shows is that the inner loop body (the System.out.println call) runs 1/2*n(n+1) times overall. That means the overall complexity of the two-loop combination is O(1/2*n(n+1)) = O(n^2). The overall complexity of the two-loop combination describes how many times the innermost code is run.
your mistake is counting the second loop as O(1/2n^2)...
first, you can clearly see it is capped to N-1 (when j = 0)
first loop is clearly N
Second loop is MAX of N-1...
threrefore, O(N^2)...
if we examine it little more,
second loop will run N-1 when i=0,
then N-2 for i=1,
and ONE single time for i=n-1,
this is 1/2n(n-1) = 1/2n^2 - 1/2n = O(n^2)
Notice this includes all iteration of the outer loop too!
I recently learned about formal Big-O analysis of algorithms; however, I don't see why these 2 algorithms, which do virtually the same thing, would have drastically different running times. The algorithms both print numbers 0 up to n. I will write them in pseudocode:
Algorithm 1:
def countUp(int n){
for(int i = 0; i <= n; i++){
print(n);
}
}
Algorithm 2:
def countUp2(int n){
for(int i = 0; i < 10; i++){
for(int j = 0; j < 10; j++){
... (continued so that this can print out all values 0 - Integer.MAX_VALUE)
for(int z = 0; z < 10; z++){
print("" + i + j + ... + k);
if(("" + i + j + k).stringToInt() == n){
quit();
}
}
}
}
}
So, the first algorithm runs in O(n) time, whereas the second algorithm (depending on the programming language) runs in something close to O(n^10). Is there anything with the code that causes this to happen, or is it simply the absurdity of my example that "breaks" the math?
In countUp, the loop hits all numbers in the range [0,n] once, thus resulting in a runtime of O(n).
In countUp2, you do somewhat the exact same thing, a bunch of times. The bounds on all your loops is 10.
Say you have 3 loop running with a bound of 10. So, outer loop does 10, inner does 10x10, innermost does 10x10x10. So, worst case your innermost loop will run 1000 times, which is essentially constant time. So, for n loops with bounds [0, 10), your runtime is 10^n which, again, can be called constant time, O(1), since it is not dependent on n for worst case analysis.
Assuming you can write enough loops and that the size of n is not a factor, then you would need a loop for every single digit of n. Number of digits in n is int(math.floor(math.log10(n))) + 1; lets call this dig. So, a more strict upper bound on the number of iterations would be 10^dig (which can be kinda reduced to O(n); proof is left to the reader as an exercise).
When analyzing the runtime of an algorithm, one key thing to look for is the loops. In algorithm 1, you have code that executes n times, making the runtime O(n). In algorithm 2, you have nested loops that each run 10 times, so you have a runtime of O(10^3). This is because your code runs the innermost loop 10 times for each run of the middle loop, which in turn runs 10 times for each run of the outermost loop. So the code runs 10x10x10 times. (This is purely an upper bound however, because your if-statement may end the algorithm before the looping is complete, depending on the value of n).
To count up to n in countUp2, then you need the same number of loops as the number of digits in n: so log(n) loops. Each loop can run 10 times, so the total number of iterations is 10^log(n) which is O(n).
The first runs in O(n log n) time, since print(n) outputs O(log n) digits.
The second program assumes an upper limit for n, so is trivially O(1). When we do complexity analysis, we assume a more abstract version of the programming language where (usually) integers are unbounded but arithmetic operations still perform in O(1). In your example you're mixing up the actual programming language (which has bounded integers) with this more abstract model (which doesn't). If you rewrite the program[*] so that is has a dynamically adjustable number of loops depending on n (so if your number n has k digits, then there's k+1 nested loops), then it does one iteration of the innermost code for each number from 0 up to the next power of 10 after n. The inner loop does O(log n) work[**] as it constructs the string, so overall this program too is O(n log n).
[*] you can't use for loops and variables to do this; you'd have to use recursion or something similar, and an array instead of the variables i, j, k, ..., z.
[**] that's assuming your programming language optimizes the addition of k length-1 strings so that it runs in O(k) time. The obvious string concatenation implementation would be O(k^2) time, meaning your second program would run in O(n(log n)^2) time.
Here's the code I've implemented in a nutshell. The for loop should have a complexity of O(n). I just can't figure out the time complexity of the inner while loop.
int x,n; // Inputted by the user.
for (int i=0; i<n; i++)
{
int done=0;
if (Condition)
{
while (done < x)
{
done++; // Based on a lot of operations
}
}
}
I can post the whole code if you want. Thanks in advance :)
Here, the complexity is measured by studying the number of times the program will run the operations of the inner loop.
Each time Condition is triggered, the inner loop runs x times. Thus the inner loop complexity is O(x).
This loop can run at most n times. This provides you an overall worst-case complexity of O(x.n).
Having additional knowledge about Condition can get you a more precise analysis. You may be able to compute average complexity for example.
As an example : let Condition be !(i & (i-1)). This is true if and only if i is either 0 or a power of 2. In this case, your loop would get run exactly E(ln2(n)) + 2 times (E(.) being the integer part function). In the end, the overall complexity knowing this becomes O(x.ln(n))
The question is rather simple, but I just can't find a good enough answer. On the most upvoted SO question regarding the big-O notation, it says that:
For example, sorting algorithms are typically compared based on comparison operations (comparing two nodes to determine their relative ordering).
Now let's consider the simple bubble sort algorithm:
for (int i = arr.length - 1; i > 0; i--) {
for (int j = 0; j < i; j++) {
if (arr[j] > arr[j+1]) {
switchPlaces(...)
}
}
}
I know that worst case is O(n²) and best case is O(n), but what is n exactly? If we attempt to sort an already sorted algorithm (best case), we would end up doing nothing, so why is it still O(n)? We are looping through 2 for-loops still, so if anything it should be O(n²). n can't be the number of comparison operations, because we still compare all the elements, right?
When analyzing the Big-O performance of sorting algorithms, n typically represents the number of elements that you're sorting.
So, for example, if you're sorting n items with Bubble Sort, the runtime performance in the worst case will be on the order of O(n2) operations. This is why Bubble Sort is considered to be an extremely poor sorting algorithm, because it doesn't scale well with increasing numbers of elements to sort. As the number of elements to sort increases linearly, the worst case runtime increases quadratically.
Here is an example graph demonstrating how various algorithms scale in terms of worst-case runtime as the problem size N increases. The dark-blue line represents an algorithm that scales linearly, while the magenta/purple line represents a quadratic algorithm.
Notice that for sufficiently large N, the quadratic algorithm eventually takes longer than the linear algorithm to solve the problem.
Graph taken from http://science.slc.edu/~jmarshall/courses/2002/spring/cs50/BigO/.
See Also
The formal definition of Big-O.
I think two things are getting confused here, n and the function of n that is being bounded by the Big-O analysis.
By convention, for any algorithm complexity analysis, n is the size of the input if nothing different is specified. For any given algorithm, there are several interesting functions of the input size for which one might calculate asymptotic bounds such as Big-O.
The commonest such function for a sorting algorithm is the worst case number of comparisons. If someone says a sorting algorithm is O(n^2), without specifying anything else, I would assume they mean the worst case comparison count is O(n^2), where n is the input size.
Another interesting function is the amount of work space, of space in addition to the array being sorted. Bubble sort's work space is O(1), constant space, because it only uses a few variables regardless of the array size.
Bubble sort can be coded to do only n-1 array element comparisons in the best case, by finishing after any pass that does no exchanges. See this pseudo code implementation, which uses swapped to remember whether there were any exchanges. If the array is already sorted the first pass does no exchanges, so the sort finishes after one pass.
n is usually the size of the input. For array, that would be the number of elements.
To see the different cases, you would need to change the algorithm:
for (int i = arr.length - 1; i > 0 ; i--) {
boolean swapped = false;
for (int j = 0; j<i; j++) {
if (arr[j] > arr[j+1]) {
switchPlaces(...);
swapped = true;
}
}
if(!swapped) {
break;
}
}
Your algorithm's best/worst cases are both O(n^2), but with the possibility of returning early, the best-case is now O(n).
n is array length. You want to find T(n) algorithm complexity.
It is much expensive to access memory then check condition if. So, you define T(n) to be number of access memory.
In the given algorithm BC and WC use O(n^2) accesses to memory because you check the if-condition O(n^2) times.
Make the complexity better: Hold a flag and if you don't do any swaps in the main-loop, it means your array is sorted and you can put a break.
Now, in BC the array is sorted and you access all elements once so O(n).
And in WC still O(n^2).