Perhaps a dumb question. In a balanced binary tree where n is the total number of nodes, I understand why the height is equal to log(n). What I don't understand is what people mean when they refer to the height as being O(log(n)). I've only seen Big O used in the context of algorithms, where if an algorithm runs in O(n) and if the input doubles then the running time doubles. But height isn't an algorithm. How does this apply to the height of a tree? What does it mean for the height to be O(log(n))?
This is because a complete binary tree of n nodes does not have height log(n).
Consider a complete binary tree of height k. Such a tree has 2k leaf nodes. How many nodes does it have in total? If you look at each level, you will find that it has 1 + 2 + 4 + 8 + ... + 2k nodes, or 20 + 21 + 22 + 23 + ... 2k.
After some math, you will find that this series equals 2k+1 - 1.
So, if your tree has n nodes, what is its height? If you solve the equation n = 2k+1 - 1 with respect to k, you obtain k = log2(n+1) - 1.
This expression is slightly less nice than log2(n), and it is certainly not the same number. However, by the properties of big-O notation,
log2(n+1) - 1 = O(log(n)).
In the source you are reading, emphasis is given on that the height grows as fast as log(n). They belong to the same complexity class. This information can be useful when designing algorithms, since you know that doubling the input will increase the tree height only by a constant. This property gives tree structures immense power. So even if the expression for the exact height of the tree is more complicated (and if your binary tree is not complete, it will look more complicated still), it is of logarithmic complexity with respect to n.
To add to Berthur's excellent answer, Big-Oh notation is not specific to analysis of algorithms; it applies to any functions. In analysis f algorithms we care about the function T(n) which gives the (typically worst-case) runtime for input size n, and we want to know an upper bound (Big-Oh) on that function's rate of growth. Here, there is a function that gives the true height of a tree with whatever property, and we want an upper bound on that function's rate of growth. We could find upper bounds on arbitrary functions devoid of any context at all like f(n) = n!^(1/2^n) or whatever.
I think they mean it takes O(log(n)) to traverse the tree
Related
If someone wants to generates a complete binary tree. This tree has h levels where h can be any positive integer and as an input to the algorithm. What complexity will it lie in and why?
A complete binary tree is tree where all levels are full of nodes except the last level, we can define the time complexity in terms of upper bound.
If we know the height of the tree is h, then the maximum number of possible nodes in the tree are 2h - 1.
Therefore, time complexity = O(2h - 1).
To sell your algorithm in the market, you need tight upper bounds to prove that your algorithm is better than the others'.
A slightly tight upper bound for this problem can be defined after knowing exactly how many nodes are there in the tree. Let's say there are N.
Then, the time complexity = O(N).
One way to find the lower bound of a comparison based algorithm is to use the decision tree. U have two questions regarding this method :
1) We know that the height of the tree is path that connects the root node to the farthest leaf node ( longest path) which is equal to the number of comparisons made from the root node to the leaf node. Therefore , when we draw the tree for a comparison based algorithm we simply need to find the worst case time which corresponds to the largest path and therefore corresponds to the height of the tree. Now for any tree the height<=log2(number of leaf nodes) which is identical to Cworst(n) <= log2(n) and now we have a lower bound for the Cworst(n) and therefore the lower bound of the problem = log2(n). Is my understanding right ?
2) What is the meaning of having an inequality for Cworst(n) for a specific comparison problem? Does this means that for a specific comparison problem we can draw many trees and every time for the path of the worst case scenario the height will have a value that satisfies the equality ? This means that for a specific problem we can draw many different trees ?
A decision tree illustrates the possible executions of an algorithm on a specific category of inputs. In the case of comparison-based sorting, a category would consist of all input lists of a certain size, so there's one tree for n = 1, one for n = 2, one for n = 3, and so on. Each tree is agnostic to the exact input values, but traces the possible directions the computation might go in depending on how the input values compare to each other.
One application of a decision tree is to reason about upper and lower bounds on the runtime complexity. As you mentioned, the height of a tree represents the upper bound on the runtime for that tree's input size, and if you can find a general relation between the input size and the tree height for all the decision trees of an algorithm, you have found an expression for the upper bound on the runtime. For example, if you analyze Bubble Sort, you will find that the decision tree for input size n has a height that's roughly n * (n + 1) / 2, so now you know that its runtime is bounded by O(n^2). And since we have an upper bound on the general runtime, that also becomes an upper bound on the worst-case runtime.
When looking at a specific algorithm, we're usually interested in how fast it could possibly be (best case), how fast it usually is (average case), and how slow it could possibly be (worst case). And we often express the best case using a lower bound (Omega), because a lower bound for the best case is also a lower bound for the general runtime; similarly, we often express the worst case using an upper bound (O), because an upper bound for the worst case is also an upper bound for the general runtime. But we don't have to - O, Omega, and Theta are only mathematical tools that say something about a function, not caring that the function in our case describes a runtime complexity. So let's do something unusual: let's look at all the possible algorithms for a problem, and use decision trees to try to figure something out about all their worst-case complexities. Then, the interesting question isn't what the upper bound is, because it's easy to make extremely slow sorting algorithms. Instead, we're interested in the lower bound: what's the best worst case? Which algorithm makes the best guarantee about how slow it will be in the worst case?
Any sorting algorithm must be able to handle any order of its input elements. Each leaf node represents one particular final permutation (rearrangement) of the input elements, and with input size n, there are n! permutations. So a decision tree for input size n has at least n! leaf nodes. Any algorithm that wants to have a good worst case needs to have a balanced tree where all the leaf nodes are on the deepest or second-deepest level. And a balanced tree with n! leaf nodes must have a height of at Omega(n lg n). Now we know something very interesting: for any comparison-based sorting algorithm, the best possible height (which represents the worst-case runtime) is at least n lg n! In other words, it is impossible to create a comparison-based sorting algorithm that always is faster than n lg n.
(Note: height <= log2(leaf nodes) is only the case for balanced trees. A tree's eight might be as much as the number of nodes minus one.)
I have two questions:
What is the difference between nearly balanced BST and nearly Complete Binary tree. Even if the definition of the former is clear then we can differenciate, but not able to get a relevant article.
Today, in my class I was taught about the condition to be balanced as:
max( size(root.left) , size(root.right) ) <= 3*n/4 ------------ (eqn 1).
Hence, H(n) = height for the tree of n nodes following the above property < = 1+H(3*n/4).
Continuing the recursive steps we get the bound for logn.
My question is that, is this a specific type of BST ? For example in case of AVL trees, as I remember the condtion is that the difference in the heights of left and right childs being atmost 1, or is this a more general result and the equation 1 as stated earlier can be reduced to prove the result for AVL Trees as well ? i.e. any Balanced BST will result in difference of heights of siblings being atmost 1 ?
In case its different than AVL, how do we manage the Insertion and Delete Operations in this new kind of tree ?
EDIT : Also if you can explain why 3*n/4 only ?
My Thought: It is because we can then surely say that H(n) <= 1+H(3*n/4), since if we take something like 3n/5 less than 3n/4 then H(3n/5) wont be necessarily less that H(2n/5) as the raio of 3n/5 and 2n/5 is less than 2 and as we know a factor of 2 for number of nodes increases the height by 1.
So we wont surely write H(n) <= 1 + H(3n/5), it may be H(2n/5) in place of H(3n/5) as well, am I right ?
A nearly complete BST is a BST where all levels are filled, except the last one. Definitions are kind of messed up here (some call this property perfect). Please refer to wikipedia for this.
Being balanced is a less strict criterion, i.e. all (nearly) complete BSTs are balanced, but not all balanced BSTs are complete. In that Wikipedia article is a definition for that too. Im my world a BST is balanced, if it leads to O(log n) operation cost.
For example, one can say, a BST is balanced, if each subtree has at most epsilon * n nodes, where epsilon < 1 (for example epsilon = 3/4 or even epsilon = 0.999 -- which are practically not balanced at all).
The reason for that is that the height of such a BST is roughly log_{1/epsilon} n = log_2 n / (- log_2 epsilon) = O(log n), but 1 / (- log_2 0.99) = 99.5 is a huge constant. You can try to prove that with the usual ration of epsilon = 1/2, where both subtrees have roughly the same size.
I don't know of a common BST, which uses this 3/4. Common BSTs are for example Red-Black-Trees, Splay-Trees or - if you are on a hard disk - a whole family of B-Trees. For your example, you can probably implement the operations by augumenting each node with two integers representing the number of nodes in the left and the right subtree respectively. When inserting or deleting someting, you update the numbers as you walk from the root to the leaf (or up) and if the condition is validated, you do a rotation.
I came to know the height of Random-BST/Red-Black trees and some other trees are O(log n).
I wonder, how this can be. Lets say I have a tree like this
The height of the tree is essentially the depth of the tree, which is in this case will be 4 (leaving the parent depth). But how could people say that the height can be represented by O(log n) notion?
I'm very to algorithms, and this point is confusing me a lot. Where I'm missing the point?
In algorithm complexity the variable n typically refers to the total number of items in a collection or involved in some calculation. In this case, n is the total number of nodes in the tree. So, in the picture you posted n=31. If the height of the tree is O(log n) that means that the height of the tree is proportional to the log of n. Since this is a binary tree, you'd use log base 2.
⌊log₂(31)⌋ = 4
Therefore, the height of the tree should be about 4—which is exactly the case in your example.
As I explained in a comment, a binary tree can have multiple cases:
In the degenerate case, a binary tree is simply a chain, and its height is O(n).
In the best case (for most search algorithms), a complete binary tree has the property that for any node, the height of the subtrees are the same. In this case the length will be the floor of log(n) (base 2, or base k, for k branches). You can prove this by induction on the size of the tree (structural induction in the constructors)
In the general case you will have a mix of these, a tree constructed where any node has subtress with possibly different height.
Say we have an initially empty BST where I perform n arbitrary inserts, how would I find the average height of this BST? The expression/pseudocode for this would be (if I'm not mistaken):
H(T) = 1 + max(H(T.left), H(T.right))
My guess at a recurrence relation for this would be T(n) = 1 + 2*T(n/2), but I'm not sure if this is correct.
Now here's my dilemma, if my recurrence relation is correct, how do I calculate the average complexity for my average height algorithm?
In general average case analysis is more complicated and you can't really use the same big-O techniques you would use in a normal worst-case proof. While your definition of height is correct, translating it to the recurrence will probably be more complicated then that. First off, you probably meant T(n) = 1 + T(n/2) (this would give a O(log n) height while your version gives O(n)) and then, nothing guarantees that values are evenly split 50-50 between right and left.
If you search a bit you will see that there is plenty of material out there on the average height of BSTs. For example, one of the results I got said that the expected height of a BST tends to 4.3 * (log n) as n grows but goes through lots of complicated math to get there.
T(n/2)+c
where c is some constant
and we divide our array in two parts but we use only single part to search.if out ans is the larger then the middle value of then we only search in (mid+1.....j)
and if its smaller then the middle value then we only search in(i.....mid)
so, at time we only work with the single sub-array