In Data Structures and Algorithm Analysis in C++ (4th Edition) by Mark Allen Weiss, on page 162 at figure 4.50, the book describes how splaying the left most child of tree with only left children would ultimately look like.
Where I am confused is how the book gets from step 2 to step 3. Here are the steps highlighted on page 162 at figure 4.50:
Whereas my thirds step looks like this:
7
/
6
/
1
\
3
/ \
2 4
\
5
And my fourth and final step like this:
1
\
4
/ \
3 6
/ / \
2 5 7
I am confused as to how the book is balancing the tree. For me, when 1 surpassed 4 the bottom of the tree looked like this:
...
/
1
\
2
\
3
\
4
My logic was that the root where the balance would occur would be 2. Then you would do a left rotation and the bottom of the tree would look like this:
...
/
1
\
3
/ \
2 4
Am I doing something wrong or am I just doing it in a different but equally valid to the book way? I am also confused because the book's final tree is imbalanced from the root of 6 whereas my root of 4 doesn't have an imbalance.
This is mostly the "Zig-zig" case till upwards, so each time you do a right rotation on the node's grandparent followed by a right rotation on the parent.
Take example:
5
/
4
/
3
/
2
/
1
If you want to splay 1, you rotate right around 3 then 2, resulting in:
5
/
4
/
1
\
2
\
3
Since it is Zig-Zig case again, we rotate around 5 then 4, resulting:
1
\
4
/ \
2 5
\
3
So you keep doing this until you have 1 as root. There are 3 cases, Zig-Zig, Zig and Zig-Zag. Here is a tool that visualizes the whole concept it will be helpful I believe.
Here are the combinations of different BSTs with same element, the arrangement looks different depending on the sequence of addition of nodes to the tree structure:
1 1 1 1 1
\ \ \ \ \
2 2 3 4 4
\ \ / \ / /
3 4 2 4 3 2
\ / / \
4 3 2 3
2 2 3 3 4 4 4
/ \ / \ / \ / \ / / /
1 3 1 4 2 4 1 4 3 2 3
\ / / \ / / \ /
4 3 1 2 2 1 3 1
/ \
1 2
4 4
/ /
1 1
\ \
2 3
\ /
3 2
Their in-order traversal will be same, so how do we distinguish them? Especially when there are more than one sequence of adding the nodes and they all generate the same structure, example 2,1,4,5,2,4,1,3,2,4,3,1.
You mentioned that all your examples have the same in-order traversal (1234), meaning inorder is not enough information to derive the tree structure. However, both post-order and pre-order sequences are sufficient for deriving the tree structure of a binary search tree.
For example, given the pre-order 2-1-3-4, the only binary search tree that satisfies this pre-order is your row 2 col 1 example. Compare that to pre-order 2-1-4-3 which would be your row 2 col 2 example.
For all of your examples, here are their pre-order notations
[1234] [1243] [1324] [1432] [1423]
[2134] [2143] [3214] [3124] [4321] [4213] [4312]
[4123] [4132]
As you can see, they are all distinct. You can repeat the same process for their post-order traversal, and you should get all distinct results.
Does a binary search beat an exponential search in any way, except in space complexity?
Both these algorithms search for a value in an ordered list of elements, but they address different issues. Exponential search is explicitly designed for unbounded lists whereas binary search deals with bounded lists.
The idea behind exponential search is very simple: Search for a bound, and then perform a binary search.
Example
Let's take an example. A = [1, 3, 7, 8, 10, 11, 12, 15, 19, 21, 22, 23, 29, 31, 37]. This list can be seen as a binary tree (although there is no need to build the tree):
15
____/ \____
/ \
__8__ _23__
/ \ / \
3 11 21 31
/ \ / \ / \ / \
1 7 10 12 19 22 29 37
Binary search
A binary search for e = 27 (for example) will undergo the following steps
b0) Let T, R be the tree and its root respectively
15 (R)
____/ \____
/ \
__8__ _23__
/ \ / \
3 11 21 31
/ \ / \ / \ / \
1 7 10 12 19 22 29 37
b1) Compare e to R: e > 15. Let T, R be T right subtree and its root respectively
15
____/ \____
/ \
__8__ _23_(R)
/ \ / \
3 11 21 31
/ \ / \ / \ / \
1 7 10 12 19 22 29 37
b2) Compare e to R: e > 23. Let T, R be T right subtree and its root respectively
15
____/ \____
/ \
__8__ _23__
/ \ / \
3 11 21 31 (R)
/ \ / \ / \ / \
1 7 10 12 19 22 29 37
b3) Compare e to R: e < 31. Let T, R be T left subtree and its root respectively
15
____/ \____
/ \
__8__ _23__
/ \ / \
3 11 21 31___
/ \ / \ / \ / \
1 7 10 12 19 22 29 (R) 37
b4) Compare e to R: e <> 29: the element is not in the list, since T has no subtree.
Exponential search
An exponential search for e = 27 (for example) will undergo the following steps
Let T, R be the leftmost subtree (ie the leaf 1) and its root (1) respectively
15
____/ \____
/ \
__8__ _23__
/ \ / \
3 11 21 31
/ \ / \ / \ / \
(R) 1 7 10 12 19 22 29 37
e1) Compare e to R: e > 1. Let R be the parent of R and T be the tree having R as root
15
____/ \____
/ \
__8__ _23__
/ \ / \
(R) 3 11 21 31 (R)
/ \ / \ / \ / \
1 7 10 12 19 22 29 37
e2) Compare e to R: e > 3. Let R be the parent of R and T be the tree having R as root:
15
____/ \____
/ \
(R)_8__ _23__
/ \ / \
3 11 21 31 (R)
/ \ / \ / \ / \
1 7 10 12 19 22 29 37
e3) Compare e to R: e > 8. Let R be the parent of R and T be the tree having R as root:
(R) 15
____/ \____
/ \
__8__ _23__
/ \ / \
3 11 21 31 (R)
/ \ / \ / \ / \
1 7 10 12 19 22 29 37
e4) Compare e to R: e > 15. R has no parent. Let T be the right subtree of T and R be its root:
15
____/ \____
/ \
__8__ _23_(R)
/ \ / \
3 11 21 31
/ \ / \ / \ / \
1 7 10 12 19 22 29 37
e5..7) See steps b2..4)
Time complexity
For the sake of demonstration, let N = 2^n be the size of A and let indices start from 1. If N is not a power of two, the results are almost the same.
Let 0 <= i <= n be the minimum so that A[2^(i-1)] < e <= A[2^i] (let A[2^-1] = -inf). Note that this kind of interval may not be unique if you have duplicate values, hence the "minimum".
Exponential search
You need i + 1 iterations to find i. (In the example, you are jumping from child to parent repeatedly until you find a parent greater than e or there is no more parent)
Then you use a binary search on the selected interval. The size of this interval is 2^i - 2^(i-1) = 2^(i-1).
The cost of a binary search in an array of size 2^k is variable: you might find the value in the first iteration, or after k iterations (There are sophisticated analysis depending on the distribution of the elements, but basically, it's between 1 and k iterations and you can't know it in advance)
Let j_i, 1 <= j_i <= i - 1 be the number of iterations needed for the binary search in our case (The size of this interval is 2^(i-1)).
Binary search
Let i be the minimum so that A[2^(i-1)] < e <= A[2^i]. Because of the assumption that N = 2^n, the binary search will meet this interval:
We start with the root A[2^(n-1)]. If e > A[2^(n-1)], i = n because R = A[2^(n-1)] < e < A[2^n]. Else, we have e <= A[2^(n-1)]. If e > A[2^(n-2)], then i = n-1, else we continue until we find i.
You need n - i + 1 steps to find i using a binary search:
if i = n, you know it at the first iteration (e > R) else, you select the left subtree
if i = n-1, you need two iterations
and so on: if i = 0, you'll need n iterations.
Then you'll need j_i iterations as shown above to complete the search.
Comparison
As you see, the j_i iterations are common to both algorithms. The question is: Is i + 1 < n - i + 1? i.e. Is i < n - i or 2i < n? If yes, the exponential search will be faster than the binary search. If no, the binary search will be faster than the exponential search (or equally fast)
Let's get some distance: 2i < n is equivalent to (2^i)^2 < 2^n or 2^i < sqrt(2^n). While 2^i < sqrt(N), the exponential search is faster. As soon as 2^i > sqrt(N), the binary search is faster. Remember that the index of e is lower or equal than 2^i because e <= A[2^i].
In simple words, if you have N elements and if e is in the firstsqrt(N) elements, then exponential search will be faster, else binary search will be faster.
It depends on the distribution, but N - sqrt(N) > sqrt(N) if N > 4, and thus the binary search is likely to be faster than the exponential search unless you know that the element will be among the first ones or the list is ridiculously short.
If 2^n < N < 2^(n+1)
I won't go into details, but this does not change the general conclusion.
If the value is beyond the last power of two, the cost of exponential to find the bound is already n+2, more than the binary search (less than or equal to 2^(n+1)). Then you have a binary search to perform, maybe in a small interval, but binary search is already the winner.
Else you add the value A[N] to the list until you have 2^(n+1) value. This won't change anything for exponential search, and this will slow down the binary search. But this slow binary search remains faster if e is not in the firstsqrt(2^(n+1)) values.
Space complexity
That's an interesting question which I don't talk about, size of the pointer and things like that. If you are performing an exponential search and consuming elements as they arrive (imagine timestamps), you don't need to store the whole list at once. You just have to store one element (the first), then one element (the second), then two elements (the third and the fourth), then four elements, ... then 2^(i-1) elements. If i is small, then you won't need to store a large list as in a regular binary search.
Implementation
Implementation is really not a problem here. See the Wikipedia pages for information: Binary search algorithm and Exponential search.
Applications and how to choose among the two
Use the exponential search only when the sequence is unbounded or when you know the value is likely to be among the first ones. Unbounded: I like the example of timestamps: they are strictly growing. You can imagine a server with stored timestamps. You can ask for n timestamps and you are looking for a specific timestamp. Ask 1, then 2, then 4, then 8,... timestamps and perform the binary search when one timestamps exceeds the value you are looking for.
In other cases, use the binary search.
Remark: the idea behind the first part of the exponential search has some applications:
Guess an integer number when the upper limit is unbounded: Try 1, 2, 4, 8, 16,... and narrow the guess when you exceed the number (this is exponential search);
Find a bridge to cross a river by a foggy day: Make 100 steps left. If you didn't find the bridge, return to the initial point and make 200 steps right. If you still didn't find the bridge, return to the initial point and make 400 steps left. Repeat until you find the bridge (or swim);
Comput a congestion window in the TCP slow start: Double the quantity of data sent until there is a congestion. The TCP congestion algorithms are in general more careful and perform something similar to a linear search in the second part of the algorithm, because exceeding tries have a cost here.
1
/ \
2 7
/ \ / \
3 4 5 6
1
/ / \ \
3 4 5 6
I'm looking for an algorithm to reduce a tree by cutting out nodes at a specific level and save all the leaves. This example is very simple, so the function my have to be recursive - imagine cutting a tree at level 3 and eliminating nodes at levels 4,5,6,7 - but preserving the leaves by inserting them all at level 4.
I wanted to draw a balanced binary search tree for numbers from 1 to 20.
_______10_______
/ \
___5___ 15
/ \ / \
3 8 13 18
/ \ / \ / \ / \
2 4 7 9 12 14 17 19
/ / / /
1 6 11 16
Is the above tree correct and balanced?
In answer to your original question as to whether or not you need to first calculate the height, no, you don't need to. You just have to understand that a balanced tree is one where the height difference between the tallest and shortest node is zero or one, and the simplest way to achieve this is to ensure that you always pick the midpoint of the possible list, when populating the top node in a sub-tree.
Your sample tree is balanced since all leaf nodes are either at the bottom or next-to-bottom level, hence the difference in heights between any two leaf nodes is at most one.
To create a balanced tree from the numbers 1 through 20 inclusive, you can just make the root entry 10 or 11 (the midpoint being 10.5 for those numbers), so that there's an equal quantity of numbers in either sub-tree.
Then just do that recursively for each sub-tree. On the lower side of 10, 5 is the midpoint:
10
/ \
5 11-thru-19 sub-tree
/ \
1-thru-4 6-thru-9
sub-tree sub-tree
Just expand on that and you'll end up with something like:
_______10_______
/ \
___5___ 15
/ \ / \
2 7 13 17
/ \ / \ / / \
1 3 6 8 11 16 18 <- depth of highest leaf node
\ \ \ \
4 9 12 19 <- depth of lowest leaf node
^
|
Difference is 1
The midpoint can be found at the number where the difference between quantities above and below that numbers is one or zero. For the whole list of numbers 1 through 20 inclusive, there are nine less than 10 and ten greater than 10 (or, if you chose 11 as the midpoint, the quantities are ten and nine).
The difference between your sample and mine is probably to do with the fact that I preferred to pick the midpoint by rounding down where there was a choice (meaning my right sub-trees tend to be "heavier"). Because your left sub-trees are heavier, you appear to have rounded up.
After choosing 10 as the initial midpoint, there's no leeway on the left sub-tree, you have to choose 5 since it has four above and below it. Any other midpoint would result in a difference of at least two between the two halves (for example, choosing 4 as the midpoint would have the two halves of size three and five). This can still give you a balanced sub-tree depending on the data but it's "safer" to choose the midpoint.