I have two running time functions here.
TA(n)= n^2+n+8
TB(n)= n+4
I suppose I need to use big O notation here. The first one is O(n^2) and the second one is O(n). Tnen I need to compare these two algorithms, however, I do not know how I can find the value of n, when two algorithms are the same fast, and when TA is faster then TB and when TB is faster than TA.
The correct answer is when n=2, two algorithms are the same fast, but i do not know how to get n. Can anyone help?
Related
In merge sort we divide in two parts while solving . Why we didn't divided it in 3 or more parts ? Similarly in many divide and conquer problems I have seen one tend to divide in two parts . Why not 3 or more parts ? What impact would it have on solution / complexity ?
Dividing the problem in 3 or more parts is usually inconvenient and has no effect on the algorithmic complexity. It will usually impact real-world performance negatively because more complicated code tends to be less efficient in practice than simple streamlined code. But this is not always the case.
There are a few practical D&C algorithms that divide the problem in more than two parts because it's beneficial to do so. The dual-pivot quicksort is one example. It tends to be somewhat faster (by a constant factor) than the regular quicksort.
You can. But it would not matter that much. Dividing yields a complexity of log (let's talk about it very generally).
To be precise you get log_2 but it is generalized to log because constant factors don't matter in complexity analysis (Big-O-Notation, Wikipedia). And you can always transform a log_a into a log_b by only using a constant factor:
log_a(x) = log_b(x) * (1 / log_b(a))
On a more specific level you get a constant factor of 2 somewhere when splitting into two halves. If you now split into 4 you'll substitute that 2 but that's actually just a constant factor change, in any case.
Actually splitting into more than two parts is often done in practice when using parallel or distributed programming.
But besides of that divide and conquer is simply a technique which makes problems easier to understand for humans. You transform a harder problem into small easier problems.
Most times the only benefit of divide and conquer will be that it is more easy to understand.
Because implementation complexity versus time complexity tradeoffs.
Lets understand it through an example and how "divide" works. Lets say we have n elements, on which we have to perform some algorithm. Lets see the difference in linear and log complexities.
Say, for example, n~10^6, then any linear function on the data-set would take about 10^6 * t units of time, where t is time spent on processing one data point. For a divide and conquer approach, this reduces to log(10^6) * t units of time. This varies on the value of base of log, which in divide and conquer algorithms, are just the number of "parts" we divide our dataset into.
Lets look at the numbers for log for different bases.
base log 10^6 in base
2 20
3 12.6
4 10
etc.
So, we can see that dividing in more parts doesn't actually decrease time by that much, because for like 10^6 elements in data set, you make 20 iterations or 12 or 10, it doesn't make a difference, but compare it to 10^6 (O(n)-complex), there you see it.
Now, lets see what happens if we divide into 3 parts and work with it. Its implementation is a lot more complex than working with 2 parts. We are not able to achieve any significant time drops but implementation will need for us to keep a counter at all times to know which part we are working with. While in binary distributions, we can use boolean states easily to maintain this counter as it is intuitive.
Also, as others have explained in their answers, base of log doesn't change the theoretical big-O factor for time complexity.
You can and it sometimes implemented. However the worst case complexity of the algorithm will still be the same with regards to big-O notation.
Obviously divide and conquor algorithms are recursive by nature and so you will still split down until the problem is in a trivial state (a base case - eg sorting two numbers). You may arrive at this case sooner but do more work to get there, they will be the same in worst case performance.
Please see the following as this question has been asked before online and people have provided better and more complete answers than I could:
https://softwareengineering.stackexchange.com/questions/197107/divide-and-conquer-algorithms-why-not-split-in-more-parts-than-two
Hope this helps!
Also from the above link which I found particulaly intesting:
"On the other hand, some tree-ish data structure do use a high
branching factor (much bigger than 3, often 32 or more), though
usually for other reasons. It improves utilization of the memory
hierarchy: data structures stored in RAM make better use of the cache,
data structures stored on disk require fewer reads HDD->RAM."
I've been trying to compare the runtimes of selection sort, quick sort and merge sort and so far, what I got is this table:
n S(n) S(n))/n^2 Q(n) (Q(n))/(nlog2 n) M(n) M(n))/(nlog2n)
1000 1.4 1.40000E-06 0.104 1.04357E-05 0.151 1.51518E-05
2000 5.41 1.35250E-06 0.223 1.01680E-05 0.327 1.49100E-05
4000 21.23 1.32688E-06 0.486 1.01540E-05 0.704 1.47086E-05
8000 84.15 1.31484E-06 1.036 9.98783E-06 1.496 1.44226E-05
16000 336.47 1.31434E-06 2.179 9.75151E-06 3.177 1.42178E-05
32000 1322.99 1.29198E-06 4.673 9.75767E-06 6.709 1.40090E-05
n is the size of the array being sorted.
I know that S(n) is the total runtime of the sorting program. I just don't get why I'm being asked to divide this total, by its order (i.e S(n) is O(n^2)). And what these values represent or what information can I infer from these results (3rd, 5th, & 7th column)?
If anyone can briefly explain this to me, that would be greatly appreciated.
Knowing that something is O(n^2) is great, but it's not the whole story. As you probably know, big-O notation disregards pesky things like constant factors and lower-order terms. So it's a good guideline for how well something will scale, but doesn't tell you a lot about how well it will perform on real data. Your professor is trying to show you the constant factors involved. As you can see, SS has a constant factor that is about 1/10th of the other two. So for smaller inputs, (probably up to about 50 elements(since 50^2/10 ~= 50log2(50))) that might outweigh the O(nlogn) vs O(n^2) difference and make SS the better choice.
Edit: This table doesn't conclusively show that the algorithms are actually in the expected complexity classes. But the results you got do match what you would expect to see from O(n^2) and O(nlog2n) algorithms. So if you did the math to calculate the complexities of the algorithms, you could run a test like this one as a sanity check, but if there happened to be a log(log(log(n))) factor in one of them, you would never notice it unless you ran it on a mind-bogglingly large data set.
(even worse, you can make an exponential function or high-degree polynomial approximate a low-degree polynomial for a while and then skyrocket, and without showing mathematically that your algorithm has a certain complexity you can't really be sure you won't get bitten by something like that.)
These results should help by showing you that the n^2 and nlog2n estimates actually correspond to real runtimes, that you (probably) implemented the algorithms correctly since they're scaling as you would expect them to, and that some of the naive algorithms may not scale well, but they have good constant factors and that makes them well-suited for smaller problems.
Your Quicksort is probably implemented with singleton arrays as the base case, but you could possibly speed it up a bit by making arrays with fewer than 50 elements the base case and running insertion sort on those.
I am currently working on some algorithms homework and I have a few questions I would like clarified so that I can make sure that the work I am doing is correct.
One of the questions asks us to compare ~20 functions by the big-Oh notation and then group together functions that are big-theta of one another. To order them, I graphed each one from 0 to 100 and compared the graphs to find which was better than the others. Is this a correct method of comparing? If there is an easier method, what can I do? How am I able to tell if one function is big-theta of another function? For example, a small part of the list that I have so far is this:
1/n
2^100
log(log(n))
n^.5 , 3n^.5
These two are grouped, yet I am not exactly sure how it is found that one is big-theta of the other.. it was my class mate that suggested it to me
2^(log(n)), 5n
Any and all help is appreciated.. I am struggling to wrap my head around Big O, Theta and the likes.
This notation has to do with time complexity theory and how problem size (ie. size of 'solution' space) is a function of the size of the input parameters.
Your question is more of a mathematics question and would be more suited for the Mathematics Exchange. Having said that, this Wikipedia article is a good starting point for you.
Generally problems are divided into (from simplest to most complex):
Constant time solvable - O(1)
Log time solvable - O(log(n))
Polynomial time solvable - O(n^2)
Super-polynomial time solvable (more than polynomial)
Exponential time solvable - O(2^poly(n))
If you'd like to determine how they are ranked, pick a set N = {1...n} and plug each element of this set into each function and plot how quickly they individually increase in size.
I have an issue with how to determine if 1 function is faster or slower than another function. If the professor uses an example of O(1) and O(n), I know O(1) is faster but I really only know that from memorizing the simple functions running time order... But if more complex examples are given, I don't understand how to find the faster function.
For example, let's say I want to compare n^logn and n^(logn)^2 and n^(sqrt(n)). How can I compare these functions and be able to tell which has a fastest and slowest running time (big-O notation)? Is there a step by step process that I can follow each time so that I can use when comparing functions running time?
Here's my thought about the above example. I know n^2 is faster than n^3. So I want to compare the n^____ of each function. So if I plug in n=1000000 in each, logn will have the smallest value, logn^2 will have the second, and logn^sqrt(n) will have the biggest. Does this mean that the smallest value (n^logn) will be the fastest and the biggest value (n^sqrt(n)) will be the slowest?
1. n^logn (fastest)
2. n^logn^2
3. n^sqrt(n) (slowest)
Usually Big O is written as a function of N (except in case of constant, O(1)).
So it is simple a matter of plugging any N (3 or 4 values, or preferably enough values to see the curve) into both functions you are comparing and compute. Graph them if you can.
But you shouldn't need to do that, you should have a basic understanding for the classes of functions for Big O. If you can't calculate it, you should still know that O(log N) is larger than O(1), etc. O notation is about worst case. So usually the comparisons are easy if you are familiar with the most common functions.
Does this mean that the smallest value (n^logn) will be the fastest
and the biggest value (n^sqrt(n)) will be the slowest? 1. n^logn
(fastest) 2. n^logn^2 3. n^sqrt(n) (slowest)
For the purpose of your comparison, yes. O notation is used to compare worst case, complexity, or class of algorithm, so you just assume worst case on all candidates in the comparison. You can't tell from O notation what the best, typical or average performance will be.
Comparing O notations is basically the matter of comparing the curves. I recommend you to draw the curve - that will be helpful to your understanding.
If you use python, I'd like to recommend to try mathplotlib.pyplot. It's very convenient.
This is a very simple question but I'm struggling too much to understand the concept completely.
I'm trying to understand the difference between the following statements:
There exists an algorithm which sorts an array of n numbers in O(n) in the best case.
Every algorithm sorts an array of n numbers in O(n) in the best case.
There exists an algorithm which sorts an array of n numbers in Omega(n) in the best case.
Every algorithm sorts an array of n numbers in Omega(n) in the best case.
I will first explain what is driving me crazy. I'm not sure regarding 1 and 3 - but I know that for one of them the answer is correct just by specifying one case and for the other one the answer is correct by examining all the possible inputs. Therefore I know one of them must be true just by specifying that the array is already sorted but I can't tell which.
My teacher always told me to think about it like we are examining who's the heighest guy in the class and again by one of these options(1,3) it's enough to say that he is and there is no reason to examine all the class.
I do know that if we were to examine the worst case then none of these statements could be true because the best sorting algorithm without any assumptions or additional memory is Omega(nlogn).
IMPORTANT NOTE: I'm not looking for a solution (an algorithm which is able to do the matching sort) - only trying to understand the concept a little better.
Thank you!
For 1+3 ask yourself - do you know an algorithm that can sort an array at best case in Theta(n) - if the answer is true, then both 1+3 are true - since Theta(n) is O(n) [intersection] Omega(n), and thus if you do have such an algorithm (that runs in Theta(n) best case) - both 1+3 are correct.
Hint: optimized bubble sort.
For 2: ask yourself - does EVERY algorithm sorts an array of numbers in O(n) best case? Do you know an algorithm that have a worst case and best case identical time complexity? What happens to the mentioned bubble sort if you take all optimizations off?
For 4: ask yourself - do you need to read all elements in order to ensure the array is sorted? If you do - Omega(n) is a definite lower bound, you cannot go better then it.
Good Luck!
The difference, obviously, is in terms "O" and "Omega". One says "rising not faster than", second says "rising not slower than".
Make sure that you understand the difference between those terms, and you'll see the difference in the sentences.
1 and 3 both state completely different things, just as 2 and 4 are.
Look at those (those are NOT the same!):
1~ there exists an algorithm that for 10 items doesn't take more than 30 in the best case.
3~ there exists an algorithm that for 10 items doesn't take less than 30 in the best case.
2~ every algorithm that for 10 items takes not more than 30 in the best case.
4~ every algorithm that for 10 items takes not less than 30 in the best case.
Do you sense the difference now? With O/Omega the difference is similar, but the subject of investigation differs. The examples above say about different performance in some point/case, while O/Omega notation tell you about the performance, related to the size of data, but only if the data "is large enough", be it three items or milions, and it drops constant factors:
function 1000000*n is O(n)
function 0.00000*n*n is O(n^2)
For small amounts data, second one is obviously very very better than first. But as the quantity of data rises, soon the first starts to be much better!
Rewriting the above examples into "more proper" terms, that are more similar to your original sentences:
1~ there exists an algorithm that, for more than N items, doesn't take more than X*N in the best case.
3~ there exists an algorithm that, for more than N items, doesn't take less than X*n in the best case.
2~ every algorithm that, for more than N items, takes not more than X*N in the best case.
4~ every algorithm that, for more than N items, takes not less than X*N in the best case.
I hope that this helps you with "seeing"/"feeling" the difference!