If an algorithm worst case running time is 6n^4 + 2, and its best case running time is 67+ 6n^3. What is the most appropriate asymptotic notation.
I'm trying learn about Big O notation.
is it Θ(n^2) ?
Essentially, Big-Oh time complexity analysis is defined for best case, worst case or average number of operations algorithm performs. "is it Θ(n^2) ?" So, you should specify which case are you looking for? Or do you mean to say is it Θ(n^2) for all cases? (which is obviously not correct)
Having said that, we know that algorithm performs 6n^4 + 2 operations in worst case. So it has Θ(n^4) worst case complexity. I've used theta here because I know exactly how many operations are going to be performed. In the best case, it performs 67+ 6n^3 operations. So it has Θ(n^3) time complexity for the best case.
How about average time complexity? Well, I can't know as long as I am not provided with the probability distribution of the inputs. It's maybe the case that best-case-like scenario rarely occurs and average time complexity is Θ(n^4), or vice versa. So we cannot directly infer the average time complexity from the worst/best case time complexities as long as we are not provided with input probability distribution, the algorithm itself or the recurrence relation. (Well, if best and worst time complexities are the same, then of course we can conclude that average time complexity is equal to them)
If algorithm is provided, we can calculate average time complexity making some very basic assumptions on the input (like equally likely distribution etc.). For example in linear search, best case is O(1). Worst case is O(n). Assuming equally likely distribution, you can conclude that average time complexity is O(n) using expectation formula. [sum of (probability of input i) * (number of operations for that input)]
Lastly, your average time complexity CANNOT be Θ(n^2) because your best and worst time complexities are worst than quadratic. It doesn't make sense to wait this algorithm perform n^2 operations in average, while it performs n^3 operations in best case.
Time complexity for best case <= time complexity for average <= time complexity for worst case
Related
I found many examples for the worst case and best case complexity, but average-case complexity was the same as worst-case complexity in most cases. Are there examples where average-case complexity can be different from worst-case complexity? If there are, please some cases in both recursive and iterative cases.
Algorithms based on partitioning using a pivot will typically have poor worst-case performance, because of the possibility of choosing an inefficient partition (i.e. most elements end up in the same side):
Quicksort has time complexity O(n log n) in the average case, but O(n2) in the worst case.
Quickselect has time complexity O(n) in the average case, but O(n2) in the worst case.
Note that both quicksort and quickselect can be implemented with or without recursion.
Algorithms based on hashes also typically have poor worst-case performance, because of the possibility of hash collision. For example, basic operations on a hash table are O(1) time in the average case, but O(n) time in the worst case.
More generally, algorithms designed for efficient performance on roughly-uniform data are likely to have worse performance on very non-uniform data, for example interpolation search degrades from an average O(log log n) on uniform data to O(n) in the worst case.
Yes ,
Average case complexity tends to be in between best and worst case in absolute terms.
but, the Complexities are dependent on input size and distribution
so they are expressed in the function of n which makes them either
equal to best case or worst case most of the times
So , No for the specific answer you are looking for.
Moreover Average case complexity has various constraints associated with it for e.g
* A probability distribution on the inputs has to be specified.
* Simple assumption: all equally likely inputs of size n.
* Sometimes this assumption may not hold for the real inputs.
* The analysis might be a difficult math challenge.
* Even if the average-case complexity is good, the worst-case
one may be very bad.
so , it's less likely to be used
This question has appeared in my algorithms class. Here's my thought:
I think the answer is no, an algorithm with worst-case time complexity of O(n) is not always faster than an algorithm with worst-case time complexity of O(n^2).
For example, suppose we have total-time functions S(n) = 99999999n and T(n) = n^2. Then clearly S(n) = O(n) and T(n) = O(n^2), but T(n) is faster than S(n) for all n < 99999999.
Is this reasoning valid? I'm slightly skeptical that, while this is a counterexample, it might be a counterexample to the wrong idea.
Thanks so much!
Big-O notation says nothing about the speed of an algorithm for any given input; it describes how the time increases with the number of elements. If your algorithm executes in constant time, but that time is 100 billion years, then it's certainly slower than many linear, quadratic and even exponential algorithms for large ranges of inputs.
But that's probably not really what the question is asking. The question is asking whether an algorithm A1 with worst-case complexity O(N) is always faster than an algorithm A2 with worst-case complexity O(N^2); and by faster it probably refers to the complexity itself. In which case you only need a counter-example, e.g.:
A1 has normal complexity O(log n) but worst-case complexity O(n^2).
A2 has normal complexity O(n) and worst-case complexity O(n).
In this example, A1 is normally faster (i.e. scales better) than A2 even though it has a greater worst-case complexity.
Since the question says Always it means it is enough to find only one counter example to prove that the answer is No.
Example for O(n^2) and O(n logn) but the same is true for O(n^2) and O(n)
One simple example can be a bubble sort where you keep comparing pairs until the array is sorted. Bubble sort is O(n^2).
If you use bubble sort on a sorted array, it will be faster than using other algorithms of time complexity O(nlogn).
You're talking about worst-case complexity here, and for some algorithms the worst case never happen in a practical application.
Saying that an algorithm runs faster than another means it run faster for all input data for all sizes of input. So the answer to your question is obviously no because the worst-case time complexity is not an accurate measure of the running time, it measures the order of growth of the number of operations in a worst case.
In practice, the running time depends of the implementation, and is not only about this number of operations. For example, one has to care about memory allocated, cache-efficiency, space/temporal locality. And obviously, one of the most important thing is the input data.
If you want examples of when the an algorithm runs faster than another while having a higher worst-case complexity, look at all the sorting algorithms and their running time depending of the input.
You are correct in every sense, that you provide a counter example to the statement. If it is for exam, then period, it should grant you full mark.
Yet for a better understanding about big-O notation and complexity stuff, I will share my own reasoning below. I also suggest you to always think the following graph when you are confused, especially the O(n) and O(n^2) line:
Big-O notation
My own reasoning when I first learnt computational complexity is that,
Big-O notation is saying for sufficient large size input, "sufficient" depends on the exact formula (Using the graph, n = 20 when compared O(n) & O(n^2) line), a higher order one will always be slower than lower order one
That means, for small input, there is no guarantee a higher order complexity algorithm will run slower than lower order one.
But Big-O notation tells you an information: When the input size keeping increasing, keep increasing....until a "sufficient" size, after that point, a higher order complexity algorithm will be always slower. And such a "sufficient" size is guaranteed to exist*.
Worst-time complexity
While Big-O notation provides a upper bound of the running time of an algorithm, depends on the structure of the input and the implementation of the algorithm, it may generally have a best complexity, average complexity and worst complexity.
The famous example is sorting algorithm: QuickSort vs MergeSort!
QuickSort, with a worst case of O(n^2)
MergeSort, with a worst case of O(n lg n)
However, Quick Sort is basically always faster than Merge Sort!
So, if your question is about Worst Case Complexity, quick sort & merge sort maybe the best counter example I can think of (Because both of them are common and famous)
Therefore, combine two parts, no matter from the point of view of input size, input structure, algorithm implementation, the answer to your question is NO.
This is an argument for justifying that the running time of an algorithm can't be considered Θ(f(n)) but should be O(f(n)) instead.
E.g. this question about binary search: Is binary search theta log (n) or big O log(n)
MartinStettner's response is even more confusing.
Consider *-case performances:
Best-case performance: Θ(1)
Average-case performance: Θ(log n)
Worst-case performance: Θ(log n)
He then quotes Cormen, Leiserson, Rivest: "Introduction to Algorithms":
What we mean when we say "the running time is O(n^2)" is that the worst-case running time (which is a function of n) is O(n^2) ...
Doesn't this suggest the terms running time and worst-case running time are synonymous?
Also, if running time refers to a function with natural input f(n), then there has to be Θ class which contains it, e.g. Θ(f(n)), right? This indicates that you are obligated to use O notation only when the running time is not known very precisely (i.e. only an upper bound is known).
When you write O(f(n)) that means that the running time of your algorithm is bounded above by a function c*f(n) where c is a constant. That also means that that your algorithm can complete in much less number of steps than c*f(n). We often use the Big-O notation because we want to include the possibility that the algorithm completes faster than we indicate. On the other hand, Theta(f(n)) means that the algorithm always completes in c*f(n) steps. Binary search is O(log(n)) because usually it will complete in log(n) steps, but could complete in one step if you get lucky (best-case performance).
I get always confused, if I read about running times.
For me a running time is the time an implementation of an algorithm needs to be executed on a computer. This can be differ in many ways and so is a complicated thing.
So I think complexity of an algorithm is a better word.
Now, the complexity is (in most cases) a worst-case-complexity. If you know an upper bound for the worst case, you also know that it can only get better in other cases.
So, if you know, that there exist some (maybe trivial) cases, where your algorithm does only a few (constant number) steps and stops, you don't have to care about an lower bound and so you (normaly) use an upper bound in big-O or little-o notation.
If you do your calculations carfully, you can also use the Θ notation.
But notice: all complexities only hold on the cases they are attached to. This means: If you make assumtions like "Input is a best case", this effects your calculation and also the resulting complexity. In the case of binary search you posted the complexity for three different assumtions.
You can generalize it by saying: "The complexity of Binary Search is in O(log n)", since Θ(log n) means "Ω(log n) and O(log n)" and O(1) is a subset of O(log n).
To summerize:
- If you know the very precise function for the complexity, you can give the complexity in Θ-notation
- If you want to get an overall upper bound, you have to use O-notation, until the lower bound over all input cases does not differ from the upper bound.
- In most cases you have to use the O-notation, since the algorithms are too complex to get a close upper and lower bound.
I found a couple questions on geeksforgeeks.org that i can't seem to understand(#1 and #3). I was hoping someone could clarify the answers for me:
clarify whether true/valid or false
1.Time Complexity of QuickSort is Θ(n^2)
I answered true but it is false, why? If quicksort has a time complexity of O(n^2) and we know that Θ(g(n))={f(n) where c1*g(n) <= f(n) <=c2*g(n) and n >= n0} then doesn't that prove that it is true since c2*g(n) being the upper bound can equal f(n)?
2.Time Complexity of QuickSort is O(n^2) - true
3.Time complexity of all computer algorithms can be written as Ω(1)
This is true but i have no understanding of why this is true. A search algorithm can have a lower bound of Ω(1) assuming we find what we were looking for on the first element but how does this hold true for ALL computer algorithms such as insertion sort algorithm where the worst case is O(n^2)?
link:
http://www.geeksforgeeks.org/analysis-of-algorithms-set-3asymptotic-notations/
Time Complexity of QuickSort is Θ(n^2)----This means for every value of n, time taken by the algorithm to produce the output is equal to a function which is f(n)=n^2.but we know this is not true for quick sort because we know for some input, running time of quick sort may be equal to a function which is g(n)=nlogn. so we need to specify if it is worst,best or average case.It is correct to say "Worst case time complexity of quicksort is Θ(n^2)".
"Time Complexity of QuickSort is O(n^2)"---this means for each input value of n,running time of the algorithm is at most a function which is f(n)=n^2.This implies there exist some input, for which the algorithm has a running time which may be less than f(n)=n^2.we know best case time complexity of quicksort is g(n)=nlogn and g(n)< f(n).As this statement covers all the cases so the statement is true.
Similarly it is correct to say "Time complexity of quicksort is Ω(nlogn)".because this means running time of the algorithm is at least nlogn, and n^2>nlogn.
"Time complexity of all computer algorithms can be written as Ω(1)"---here 1 represent constant time function.the above statement implies: to execute any computer algorithms we need a minimum constant time.which is correct for all computer algorithms.
The worst case scenario for QuickSort is O(n^2). But you expect it to run in O(n log n) time. Hence the running time of the algorithm varies per case and you cannot use the theta symbol to give the general running time of the algorithm.
And of course the lowerbound on the running time of any algorithm is constant time (Ω(1)). It doesn't have to reach this lower bound though but the algorithm should be run, and should have at least one operation.
how can we tell whether the obtaining time complexity for an algorithm is in best case or worst case or an average case?
example if we get time complexity as
t(n) = 2n^2 + 3n -1, how to estimate the best or worst or average cases?
first note : t(n) = 2n^2 + 3n -1 will always be a big O(n^2) in worst, best and average case.
In some cases the complexity depends on the input of your algorithm, in these cases usually people calculate the complexity in worst case.
But when you think worst case is not relevant and too restrictive, you do an average case analysis or an amortized analysis. For example if an algorithm works in O(n) for (1-1/n)% of his inputs and O(n^2) for (1/n)%, you don't want to say it's O(n^2), and give the average complexity that will be more like O(n). But the worst case can still happen.
Look at this post for more detail on average case analysis and amortized analysis.
Difference between average case and amortized analysis
and the wikipedia's articles :
average case complexity
amortized analysis
You can only tell that by carefully examining the algorithm.
If you know that exactly t(n)=2n^2+3n-1, then t(n)=O(n^2) and that's the best, the worst and consequently the average time complexity.
That simplifies to O(2n^2) or just merely O(n^2). You remove all the other elements because it simplifies.
This is known as Big O notation. This is simply the worst case time. THe others all are irrelevant for the most part.
>>Bast Case:
If you are finding the Best case Complexity then you check image and following statement.
T(n)= c1n+c2(n-1)+c4(n-1)+c5(n-1)+c8(n-1)
(c1+c2+c5+c8)n-(c2+c4+c5+c6)
The Maximum Power of n is 1, so we are say tha Best Case Complexity is Insertion Sort O(n).
>>Worst Case:
If you are finding the Worst case Complexity then you check image and following statement.
T(n)= c1n+c2(n-1)+c4(n-1)+c5(n(n+1)/2)+c6(n(n-1)/2)+c7(n(n-1)/2)+c8(n-1)
(c5/2 + C6/2 +C7/2)n^2 +(c1+c2+c4+ c5/2 -C6/2 -C7/2+ c8)
The Maximum Power of n is 2, so we are say tha Best Case Complexity is Insertion Sort O(n^2).