Hey just a quick question,
I've just started looking into algorithm analysis and I'm attempting to learn Big-Oh notation.
The algorithm I'm looking at contains a quicksort (of complexity O(nlog(n))) to sort a dataset, and then the algorithm that operates upon the set itself has a worst case run-time of n/10 and complexity O(n).
I believe that the overall complexity of the algorithm would just be O(n), because it's of the highest order, so it makes the complexity of the quicksort redundant. However, could someone confirm this or tell me if I'm doing something wrong?
Wrong.
Quicksort has worst case complexity O(n^2). But even if you have an O(nlogn) sort algorithm, this is still more than O(n).
Related
In the below 4 options, wasn't 3., i.e, O(n²) supposed to be the worst time complexity? Because it takes more time to run and log(n) takes lesser time to run? But unfortunately the answer given is option 4.
What's wrong with my logic?
O(log(n!))
O(n)
O(n²)
O(log(log(n)))
To answer the direct question, this is the order of the functions from best to worst, sorted by their worst-case time-complexities (what we would call big-O notation):
O(log(log(n))
O(n)
O(log(n!))
O(n^2)
This is pretty easy to see in a graphing of the functions:
(Note that O(log(log(n))) is nearly a constant function, an iterated logarithm. Thus it does not stray far from the x-axis).
Big-O notation describes the worst-case time complexity for any function, but the question likely originally asked for the best worst-case time complexity.
The worst case for time taken by linear search is when the item is at the end of the list/array, or doesn't exist. In this case, the algorithm will need to perform n comparisons, to see if each element is the required value, assuming n is the length of the array/list.
From what I've understood of big-O notation, it makes sense to say that the time complexity of this algorithm is O(n), as it COULD happen that the worst case occurs, and big-O is used when we want to make a conservative estimate of the "worst case".
From a lot posts and answers on Stack Overflow, it seems this thinking is flawed, with claims made such as Big-O notation has nothing to do with the worst case analysis.
Please help me to understand the distinction in a way that doesn't just add to my confusion, as the answers here: Why big-Oh is not always a worst case analysis of an algorithm? do.
I'm not seeing how big-O has NOTHING to do with worst case analysis. From my current hilltop, it looks like big-O expresses how the worst case grows as the input size grows, which seems very much "to do" with worst-case analysis.
Statements such as this, from https://medium.com/omarelgabrys-blog/the-big-scary-o-notation-ce9352d827ce :
As an example, worst case analysis gives the maximum number of operations assuming that the input is in the worst possible state, while the big o notation express the max number of operations done in the worst case.
don't help much, as I cannot see what distinction is being referred to.
Any added clarity much appreciated.
The big-O notation is indeed independent of the worst-case analysis. It applies to any function you want.
In the case of a linear seach,
the worst-case complexity is O(n) (in fact even Θ(n)),
the average-case complexity is O(n) (in fact even Θ(n)),
the best-case complexity is O(1) (in fact even Θ(1)).
So big-O and worst-case are different concepts, though a big-O bound for the running time of an algorithm must hold for the worst-case.
This is the case:
If an algorithm to find a solution for a problem is in O(f(n)), means that the worst-case scenario for finding the solution for the problem by the algorithm is in O(f(n)). In other words, if the worst-case scenario can be found in g(n) steps by the algorithm, then g(n) is in O(f(n)).
For example, for the search algorithm, as you have mentioned, we know that the worst-case scenario can be found in O(n). Now, although the algorithm is in O(n), we can say the algorithm is in O(n^2) as well. As you see, here is the distinction between Big-Oh complexity and the worst-case scenario.
In sum, the worst-case scenario complexity of an algorithm is a subset of the Big-Oh complexity of the algorithm.
I am learning about time complexity recently. As I trying to find the Big O of quicksort, I saw on the Internet saying quicksort has an O(n log n) for Best-case performance and O(n^2) for Worst-case performance. From what I have understood, Big O notation describes the upper bound limit of an algorithm, and it states its worst-case performance. Why is there saying "quicksort has an O(n log n) for Best-case performance"? I thought Big O is used to describe the worst-case performance.
This happens because the worst-case performance rarely occurs in QuickSort - you have to be really unlucky to partition your input so that your pivot is always your smallest or largest element in that input.
Since the average-case performance, which is equal to the best-case, happens most of the time, people choose to mention that instead or both. I think that many programming languages with built-in sorting functions have implemented QuickSort for their function (but not for every type of input). That shows how strong this algorithm is.
You may also find this article useful: Analysis of quicksort
This question has appeared in my algorithms class. Here's my thought:
I think the answer is no, an algorithm with worst-case time complexity of O(n) is not always faster than an algorithm with worst-case time complexity of O(n^2).
For example, suppose we have total-time functions S(n) = 99999999n and T(n) = n^2. Then clearly S(n) = O(n) and T(n) = O(n^2), but T(n) is faster than S(n) for all n < 99999999.
Is this reasoning valid? I'm slightly skeptical that, while this is a counterexample, it might be a counterexample to the wrong idea.
Thanks so much!
Big-O notation says nothing about the speed of an algorithm for any given input; it describes how the time increases with the number of elements. If your algorithm executes in constant time, but that time is 100 billion years, then it's certainly slower than many linear, quadratic and even exponential algorithms for large ranges of inputs.
But that's probably not really what the question is asking. The question is asking whether an algorithm A1 with worst-case complexity O(N) is always faster than an algorithm A2 with worst-case complexity O(N^2); and by faster it probably refers to the complexity itself. In which case you only need a counter-example, e.g.:
A1 has normal complexity O(log n) but worst-case complexity O(n^2).
A2 has normal complexity O(n) and worst-case complexity O(n).
In this example, A1 is normally faster (i.e. scales better) than A2 even though it has a greater worst-case complexity.
Since the question says Always it means it is enough to find only one counter example to prove that the answer is No.
Example for O(n^2) and O(n logn) but the same is true for O(n^2) and O(n)
One simple example can be a bubble sort where you keep comparing pairs until the array is sorted. Bubble sort is O(n^2).
If you use bubble sort on a sorted array, it will be faster than using other algorithms of time complexity O(nlogn).
You're talking about worst-case complexity here, and for some algorithms the worst case never happen in a practical application.
Saying that an algorithm runs faster than another means it run faster for all input data for all sizes of input. So the answer to your question is obviously no because the worst-case time complexity is not an accurate measure of the running time, it measures the order of growth of the number of operations in a worst case.
In practice, the running time depends of the implementation, and is not only about this number of operations. For example, one has to care about memory allocated, cache-efficiency, space/temporal locality. And obviously, one of the most important thing is the input data.
If you want examples of when the an algorithm runs faster than another while having a higher worst-case complexity, look at all the sorting algorithms and their running time depending of the input.
You are correct in every sense, that you provide a counter example to the statement. If it is for exam, then period, it should grant you full mark.
Yet for a better understanding about big-O notation and complexity stuff, I will share my own reasoning below. I also suggest you to always think the following graph when you are confused, especially the O(n) and O(n^2) line:
Big-O notation
My own reasoning when I first learnt computational complexity is that,
Big-O notation is saying for sufficient large size input, "sufficient" depends on the exact formula (Using the graph, n = 20 when compared O(n) & O(n^2) line), a higher order one will always be slower than lower order one
That means, for small input, there is no guarantee a higher order complexity algorithm will run slower than lower order one.
But Big-O notation tells you an information: When the input size keeping increasing, keep increasing....until a "sufficient" size, after that point, a higher order complexity algorithm will be always slower. And such a "sufficient" size is guaranteed to exist*.
Worst-time complexity
While Big-O notation provides a upper bound of the running time of an algorithm, depends on the structure of the input and the implementation of the algorithm, it may generally have a best complexity, average complexity and worst complexity.
The famous example is sorting algorithm: QuickSort vs MergeSort!
QuickSort, with a worst case of O(n^2)
MergeSort, with a worst case of O(n lg n)
However, Quick Sort is basically always faster than Merge Sort!
So, if your question is about Worst Case Complexity, quick sort & merge sort maybe the best counter example I can think of (Because both of them are common and famous)
Therefore, combine two parts, no matter from the point of view of input size, input structure, algorithm implementation, the answer to your question is NO.
In the book is written:
The worst-case running time of heapsort is (nlgn). This is clear since
sorting has a lower bound of (nlgn)
But can someone help me and show me explicitly that the lower-bound of this function is equal to Omega(nlgn)?
It sounds like the book is drawing on the fact that heapsort is a comparison-based sorting algorithm in that statement. So automatically, this paradigm of sorting algorithms have already been shown to have a lower-bound of Omega(nlgn):
http://en.wikipedia.org/wiki/Comparison_sort#Number_of_comparisons_required_to_sort_a_list