I couldn't understand this question, could someone who knows about this issue show me the solution?
Priest Francis, responsible for ringing the bells, made a device that rings the bells automatically. At every exact hour, the device rings at least 1 of the n bells. Specifically, the i-th bell rings every 1 hour. For example, suppose that n = 4 and that Father Francisco turned on his device just after midnight. The ringing scheme of the bell is shown below. What is the amortized number of bells ringing per hour as a function of n?
To get amortized complexity, you find the complexity in terms of the average cost over a sequence of operations, rather than taking the complexity naively as the worst case for a single operation times the number of operations.
Applied to this case, without amortization you could take the complexity as the worst case for a single hour (all n bells could ring), times the number of hours (you didn't specify, so I'll assume it's a variable, h). In this way, we would get O(nh) total bells being run with O(n) bells being rung in any given hour. However, all n bells ring so infrequently (specifically, only every n! hours), that maybe with amortization we can do better.
We then aim to get the amortized number of bells rung per hour as the total number of bells rung over all h hours, T(n,h), divided by h. To derive this total sum, note that Bell1 will ring h times, Bell2 will ring h/2 times, Bell3 h/3 times, and so on. T(n,h) is then given by the sum:
T(n,h) is h times the harmonic series up to n, so we can represent it as hH(n), where H(n) denotes the nth harmonic number. This mean the amortized complexity for a single hour, as the total number of bells rung divided by the number of hours, is H(n).
It is known that H(n) is approximated by ln(n) as n goes to infinity (essentially because H(n) is approximately the integral of 1/x dx = ln(n)), so the amortized complexity, in terms of the average number of bells being rung per hour, is O(ln(n)) = O(logn).
Related
Consider this example of binary search tree.
n =10 ;and if base = 2 then
log n = log2(10) = 3.321928.
I am assuming it means 3.321 steps(accesses) will be required at max to search for an element. Also I assume that BST is balanced binary tree.
Now to access node with value 25. I have to go following nodes:
50
40
30
25
So I had to access 4 nodes. And 3.321 is nearly equal to 4.
Is this understanding is right or erroneous?
I'd call your understanding not quite correct.
The big-O notation does not say anything about an exact amount of steps done. A notation O(log n) means that something is approximately proportional to log n, but not neccesarily equal.
If you say that the number of steps to search for a value in a BST is O(log n), this means that it is approximately C*log n for some constant C not depending on n, but it says nothing about the value of C. So for n=10 this never says that the number of steps is 4 or whatever. It can be 1, or it can be 1000000, depending on what C is.
What does this notation say is that if you consider two examples with different and big enough sizes, say n1 and n2, then ratio of the number of steps in these two examples will be approximately log(n1)/log(n2).
So if for n=10 it took you, say, 4 steps, then for n=100 it should take approximately two times more, that is, 8 steps, because log(100)/log(10)=2, and for n=10000 it should take 16 steps.
And if for n=10 it took you 1000000 steps, then for n=100 it should take 2000000, and for n=10000 — 4000000.
This is all for "large enough" n — for small ns the number of steps can deviate from this proportionality. For most practical algorithms the "large enough" usually starts from 5-10, if not even 1, but from a strict point of view the big-O notation does not set any requirement on when the proportionality should start.
Also in fact O(log n) notation does not require that number of steps growths proportionally to log n, but requires that the number of steps growths no faster than proportionally to log n, that is the ratio of the numbers of steps should not be log(n1)/log(n2), but <=log(n1)/log(n2).
Note also another situation that can make the background for O-notation more clear. Consider not the number of steps, but the time spent for search in a BST. You clearly can not predict this time because it depends on the machine you are running on, on a particular implementation of the algorithm, after all on the units you use for time (seconds or nanoseconds, etc.). So the time can be 0.0001 or 100000 or whatever. However, all these effects (speed of your machine, etc) routhly changes all the measurement results by some constant factor. Therefore you can say that the time is O(log n), just in different cases the C constant will be different.
Your thinking is not correct totally. The steps/accesses which are considered are for comparisons. But, O(log n) is just a mere parameter to measure asymptotic complexity, and not the exact steps calculation. As exactly answered by Petr, you should go through the points mentioned in his answer.
Also, BST is binary search tree,also sometimes called ordered or sorted binary trees.
Exact running time/comparisons can't be derived from Asymptotic complexity measurement. For that, you'll have to return to the exact derivation of searching an element in a BST.
Assume that we have a “balanced” tree with n nodes. If the maximum number of comparisons to find an entry is (k+1), where k is the height, we have
2^(k+1) - 1 = n
from which we obtain
k = log2(n+1) – 1 = O(log2n).
As you can see, there are other constant factors which are removed while measuring asymptotic complexity in worst case analysis. So, the complexity of the comparisons gets reduced to O(log2n).
Next, demonstration of how element is searched in a BST based on how comparison is done :-
1. Selecting 50,if root element //compare and move below for left-child or right-child
2. Movement downwards from 50 to 40, the leftmost child
3. Movement downwards from 40 to 30, the leftmost child
4. Movement downwards from 30 to 25, found and hence no movement further.
// Had there been more elements to traverse deeper, it would have been counted the 5th step.
Hence, it searched the item 25 after 3 iterative down-traversal. So, there is 4 comparisons and 3 downward-traversals(because height is 3).
Usually you say something like this :-
Given a balanced binary search tree with n elements, you need O(log n) operations to search.
or
Search in a balanced binary search tree of n elements is in O(log n).
I like the second phrase more, because it emphasizes that O is a function returning a set of functions given x (short: O(x)). x: N → N is a function. The input of x is the size of the input of a function and the output of x can be interpreted as the number of operations you need.
An function g is in O(x) when g is lower than x multiplied by an arbitrary non-negative constant from some starting point n_0 for all following n.
In computer science, g is often set equal with an algorithm which is wrong. It might be the number of operations of an algorithm, given the input size. Note that this is something different.
More formally:
So, regarding your question: You have to define what n is (conceptually, not as a number). In your example, it is eventually the number of nodes or the number of nodes on the longest path to a leaf.
Usually, when you use Big-O notation, you are not interested for a "average" case (and especially not for some given case) but you want to say something about the worst case.
I think I understand the concept. Let me explain how I understand it in terms of its application to doubling array vectors.
The rate of copying items to an array remains constant. While the growth of the array is exponential, the rate at which the array needs to double in size is logarithmic. Because of this, the decreased occurrence of doubling the array size 'sort of' cancels out the resources required to double the array and copy it's elements, as this only happens O(n Log N) times throughout the life of the array. Thus, the O(n^2) for the rate of growth combined with O(n Log N) for the frequency at which the array grows resolves to somewhere around O(n).
Is this correct? If not, which parts are wrong? I'm having trouble wrapping my head around this. I'm pretty sure that the Big Oh notation I gave is incorrect.
Thanks
Let's say you double the array when it reaches 2 elements, 4, 8, 16, 32, ..., 2^k.
This means O(log n) doubling operations for an array of size n. It's tempting to say this makes the thing O(n log n), but that's not the case.
The number of operations performed for all of these doubling ops is bounded above by:
1 + 2 + 4 + ... + 2^k = (2^k - 1) (sum of geometric series)
Notice however that k = number of doubling ops = O(log n), so we have the number of operations needed for doubling bounded above by 2^(log n) = n
So, the whole thing remains O(n): while you do a lot of operations when doubling the array, considering those "a lot of operations" in relation to the entire array size and how many times you have performed them, they are no longer "a lot", just O(n).
Amortized basically means you keep the big picture in mind: sure, I might have to work a lot today, but that means I get to stay home for the rest of the week. Does that mean I will have worked a lot this week? No, I will have actually worked very little.
I think your explanation is not entirely accurate and overly complicated. There are no sort ofs, and there are no O(n^2) rates of growth. The math just adds up to O(n) as you've seen.
My suggestion in general is that you ignore the word amortized and just do the math. I've seen it cause a lot of confusion for no good reason. Sure, it might be the formal thing that's involved in such an analysis, but most of the time it will only spark confusion about what happens. Just ask yourself: "ok, how many operations will this algorithm perform?". More often than not, you won't need anything fancy to answer that question.
Not discussing how you calculated "amortised" time, but what it's good for: You may have an algorithm that you use repeatedly to get from state 1 to state 2 to state 3 ... and so on. The cost of going from one state to another may vary wildly, and the maximum cost to go from one state to another may be very high. However, sometimes you can prove that the total cost of going through states 1 to state n is much less than n times the maximum. So the "amortised" cost is the average cost of going from state 1 to state n.
Sometimes the maximum cost may be important. For example in a music playing application, the time to output the next music sample must always be low, or the sound will stutter, which is not acceptable. However, in other cases, what matters is the total time to go through all the states and that is fine as long as the average is fine.
Note that amortised time is not the same as average time. An algorithm may require different time depending on the data that it is given. If some input X takes much longer than average, then it could be that you use the algorithm a million times to handle the same input X and the total time is huge.
I'm currently learning Big O Notations, but I'm kinda confused with time/iterations calculation according to different complexities.
I made up this problem:
An algorithm that goes through all possible solutions takes 10^(-7) seconds with each test.
If the number of solutions are the following functions: logn, n, nlog. n^2, what's the maximum n I can calculate in, for example, less than 1 second?
What I though about (for case logn) was 10^-7 times logn must take less than 1 second:
10^(-7) * logn < 1 <=> n = 10^(1/10^-7)
Incredibly big (If it's not wrong, oh damn --'). But what about n^2 ?
10^(-7) * n^2 < 1 <=> n = square_root(1/10^-7)
But how can the number of solutions in n^2 case be less than the number in n, if the complexity is bigger? This is confusing me...?
"This is bad on so many levels."
First off, O(f(n)) is not the same as f(n).
Complexity is typically used to represent the time it takes to solve a problem, as a function of the size of the input.
Of course if you can solve more problems in the same amount of time using X vs Y, X will solve more problems in a fixed amount of time than Y. But since you're not using the term complexity in the right way, it is no surprise you get a (seemingly) paradoxical answer.
You just upper bounded your algorithm to a real world time of 10^(-7) seconds, which implies that your algorithm will guarantee to finish, for all complexities, in 10^(-7) seconds.
Let's refrain from discussing if this is possible in reality. But since you just defined your algorithm to go through all possible solutions in 10^(-7), it means no matter what n is, it will finish in that time. So your n is positive infinity.
Besides, I don't think you should use big O to denote the number of solutions.
Good news first there is nothing "wrong" with the result of your calculation.
Naturally if the complexity of an algirithm is higher (like O(n^2) complexity is higher than O(log n)), the size of the problem you are still able to handle in "acceptable" time will be smaller, in total accordance to your calculation.
Nevertheless the example seems a bit twisted and I have to admit, I didn't completely understand the purpose, you invented the 10^-7 factor for and thus suspect, you didn't get the concept of the O notation right.
Essentially the main idea of the O notation is, that you don't care about any linear factors (like the 10^7 factor you invented in your calculation) in comparing two algorithms, but only about how fast computation time grows with the size of the problem, as constant (and thus not growing) factors sooner or later will become irrelevant compared to the growth in computation time due to the problem size.
Going by example:
With the O(n^2) algorithm A, taking the time t=2*n^2 milliseconds and the O(log n) algorithm B, taking t=200*log(n) millliseconds on a specific machine for given problem size nr things will look as follows:
For a very small problem, say n=10, Algorithm A might be faster than Algorithm B:
2*10^2 = 2*100 = 200 ms
400*log10 = 400*1 = 400 ms
But with growing problem size, say n=100, Algorithm B will sooner or later overtake Algorithm A in speed:
2*100^2 = 2*10,000 = 20,000 ms
400*log100 = 400*2 = 800 ms
While for even bigger problem sizes, say n=1,000,000, waiting for A to complete might take a lot of patience.
2*1,000,000^2 = 2*10^12 ms = 2*10^9 s = 33333333 min = 555555 h = 23148 days = 63 years
Algorithm B might still operate in acceptable time.
400*log1,000,000 = 400*6 = 2,400 ms = 2.4 s
As the constant factor plays an ever smaller role with growing problem size, for bigger and bigger problems it becomes more and more irrelevant and is therefore (together with terms of lower order, that follow the same rule) left out in the O notation.
Thus the "right" way to look at complexities given in O notation is not trying to look at fixed values for fixed n or even reinvent constant factors and additional terms of lower order already abstracted away, but to look how fast computation time grows with problem size.
So, again going by example the "right" way to look at complexity of O(n^2) and O(log10) would be to compare their growth.
If the problem size grows by a factor of 10 Algorithm A's computation time will rise by a factor of 100, thus taking 100 times as long as before, as:
(n*10)^2 = n^2 * 10^2 = n^2 * 100
While Algorithm B's computation time will just grow by a constant ammount, as:
log(n*10) = log(n) + log(10) = log(n) + 1
Do the higher order of growth functions take the longest time? So, that O(n2) takes less time than (2n)?
Are the highest order of growth functions take the longest to actually run when N is a large number?
The idea of Big O Notation is to express the worst case scenario of algorithm complexity. For instance, a function using a loop may be described as O(n) even if it contains several O(1) statements, since it may have to run the entire loop over n items.
n is the number of inputs to the algorithm you are measuring. Big-O in terms of n tells you how that algorithm will perform as the number of inputs gets increasingly large. This may mean that it will take more time to execute, or that something will take more space to store. If you notice your algorithm has a high Big-O (like O(2^n) or O(n!)), you should consider an alternate implementation that scales better (assuming n will ever get large -- if n is always small it doesn't matter). The key take-away here is that Big-O is useful for showing you which of two algorithms will scale better, or possibly just the input size for which one algorithm would become a serious bottleneck on performance.
Here is an example image comparing several polynomials which might give you an idea of their growth rates in terms of Big-O. The growth time is as n approaches infinity, which in graphical terms is how sharply the function curves upwards along the y-axis as x grows large.
In case it is not clear, the x-axis here is your n, and the y-axis the time taken. You can see from this how much more quickly, for instance, something O(n^2) would eat up time (or space, or whatever) than something O(n). If you graph more of them and zoom out, you will see the incredible difference in, say, O(2^n) and O(n^3).
Attempting a Concrete Example
Using your example of comparing two string arrays of size 20, let's say we do this (pseudocode since this is language agnostic):
for each needle in string_array_1:
for each haystack in string_array_2:
if needle == haystack:
print 'Found!'
break
This is O(n^2). In the worst case scenario, it has to run completely through the second loop (in case no match is found), which happens on every iteration of the first loop. For two arrays of size 20, this is 400 total iterations. If each array was incrased by just one string to size 21, the total number of iterations in the worst case grows to 441! Obviously, this could get out of hand quickly. What if we had arrays with 1000 or more members? Note that it's not really correct to think of n as being 20 here, because the arrays could be of different sizes. n is an abstraction to help you see how bad things could get under more and more load. Even if string_array_1 was size 20 and string_array_2 was size 10 (or 30, or 5!), this is still O(n^2).
O-time is only relevant when compared against itself, but 2^n will grow faster than n^2.
Compare as n grows:
N n^2 2^n
1 1 2
2 4 4
3 9 8
4 16 16
5 25 32
6 36 64
...
10 100 1024
20 400 1048576
Relevant Link: Wolfram Alpha
You can think in reverse: let an algorithm be such that T(n) = t; how many more elements can I process in time 2.t ?
O(n^2) -> 41% elements more ((n + 0.41 n)^2 = 2.n^2)
O(2^n) -> a single element more (2^(n+1) = 2.2^n)
Or in time 1000000.t ?
O(n^2) -> 1000 times more
O(2^n) -> 20 elements more
I keep seeing the search time for linked lists listed as O(N) but if you have 100 elements in a list aren't you on average only comparing against 50 of them before you've found a match?
So is O(N/2) being rounded to O(N) or am I just wrong in thinking it's N/2 on average for a linked list lookup?
Thanks!
The thing is, the order is really only talking about how the time increases as n increases.
So O(N) means that you have linear growth. If you double N then the time taken also doubles. N/2 and N both have the same growth behaviour so in terms of Order they are identical.
Functions like log(N), and N^2 on the other hand have non-linear growth, N^2 for example means that if you double N the time taken increases 4 times.
It is all about ratios. If something on average takes 1 minute for 1 item will it on average take 2 minutes or 4 minutes for 2 items? O(N) will be 2 minutes, O(N^2) will take 4 minutes. If the original took 1 second then O(N) will take 2 seconds, O(N^2) 4 seconds.
The algorithm that takes 1 minute and the algorithm that takes 1 second are both O(N)!
The other answers make the main points required for the answer, but there's a detail I'd like to add: If it's not explicitly stated, complexity statements typically discuss the worst case behaviour, rather than the average, simply because analysing the worst case behaviour is frequently a lot easier than the average behaviour. So if I say, without any additional qualifications, search in a linked list is O(N), then I'm admittedly being sloppy, but I would be talking about the worst case of search in a linked list taking a number of steps linear in N.
The O means something. The average number of elements that need to be traversed, to find something in a linked list is N/2. N/2 = O(N).
Note that saying search in linked list on average takes n/2 operations is wrong as operation is not defined. I could argue that for each node you need to read it's value, compare it to what you are searching for and then read the pointer to next node, and thus that the algorithm performs 3N/2 operations on average. Using O notation allows us to avoid such insignificant details.