In my algorithms and data structures course weβre given a proof for the upper bounds of the minimum height of a tree, a general tree, not a binary tree.
In the course, the degree of a tree is based upon the largest amount of child nodes found in a parent node thatβs defined as d. Secondly, the total number of nodes in the tree is defined as n.
The big O for the minimum height of the tree is h <= ceiling(log_d(n)).
I tried to find out what it means by using an example: d = 4, n = 21
The result is h <= 3. What Iβm now trying to understand is what the meaning of this result is. My biggest difficulty is just understanding why there is a big O used for a minimum.
The minimum height for a tree with π nodes and a maximum degree of π is when you pack those π nodes with as many as possible in the upper levels of the tree, because you'll want to use as few levels as possible.
Practically, when you have π nodes, you'd use the first as the root, the second batch of π nodes as direct children of the root, and so filling up levels of the tree as needed. This is called a complete tree. There is no way to reduce the height in such a tree by moving nodes around. All levels -- except maybe the bottom one -- will already be filled at full capacity.
So for π=21 and π=4 you would build a complete tree as follows:
__________________1___________________
/ / \ \
___2__ ___3___ ___4___ ___5___
/ /\ \ / / \ \ / / \ \ / / \ \
6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21
Note how this is a perfect tree: all levels are filled to maximum capacity. Adding even one more node will necessarily increase the tree's height.
It seems you are using a definition of height that is slightly different from the standard definition: the above pictured tree has height 2, not 3. Height is defined (standard) as the length of the longest path from root to leaf, expressed as the number of edges on that path.
Now to derive the formula for the minimum height we can observe how a level can contain π times more nodes than the previous level (since all nodes on that level can have at most π children).
So we have this maximum capacity of the levels:
Β Β Β 1, π, πΒ², πΒ³, ... πβ
The sum of those terms is a geometric series, and is therefore:
Β Β Β (πβ+1β1) / (πβ1)
Just to verify, for β=2 and π=4 (the example), we then have:
Β Β Β (42+1β1) / (4β1) = (64β1) / 3 = 21
For a given height β, the number of nodes in a complete tree is thus bounded like this:
Β Β Β (πββ1) / (πβ1) < π β€ (πβ+1β1) / (πβ1)
Let's move things around to isolate β:
Β Β Β πβ < π(πβ1) + 1 β€ πβ+1
If we subtract 1 from all terms, we can change the comparators:
Β Β Β πβ β€ π(πβ1) < πβ+1
Taking the logarithm with base π, we get:
Β Β Β β β€ logπ(π(πβ1)) < β+1
And so we can now derive the minimum β by rounding downwards:
Β Β Β β = β logπ(π(πβ1)) β
Verification
Let's try again with the example π=21 and π=4:
Β Β Β β = β log4(21(4β1)) β
Β Β Β β = β log4(63) β = β 2.9886... β = 2
Big O
In Big O notation, the base of the logarithm is not significant, and things like rounding up or down are not relevant either.
So the big O for the minimum height given π and π is:
Β Β Β O(β logπ(π(πβ1)) β) = O(log(ππ))
Related
So I want to figure out the runtime of the following algorithm (in pseudocode):
Algorithm(N):
int i = n;
int j;
new Array sum[(n + 1) / 2];
while i > 1 do
j = i;
while j < n do
for k = 0; k < n; k = k + 2 do
sum[k / 2] = sum[(k / 2) - 1] + k;
end
j = j * 2;
end
i = i / 2;
end
return sum
The outer loop will make βlog2πβ iterations.
The middle loop will make 0 iterations the first time it runs, then 1, then 2, then 3, ... until βlog2πβ. This is a triangular number, and so the middle loop will have a total number of executions that is equal to:
Β Β Β βlog2πβ(βlog2πβ + 1) / 2
The inner loop will always make the same number of iterations for each iteration of the middle loop, as it is independent of the value of π or π: it is π / 2. So in total it will have the following number of iterations:
Β Β Β πβlog2πβ(βlog2πβ + 1) / 4
There are two ways time complexity is calculated with respect to arithmetic calculations:
In a first approach we assume that all involved arithmetic calculations are performed in constant time (like addition, decrement, division by 2, and multiplication by 2), and so we can derive the following asymptotic order:
Β Β Β O(πβlog2πβ(βlog2πβ + 1) / 4) = O(πlogΒ²π)
A more purist approach does not assume that arithmetic calculations are performed in constant time, taking into account that integers are not stored in limited words (like 64-bit), but can be arbitrary long. This means that the above-mentioned arithmetic calculations have a O(logπ) time complexity. So then we must add that factor, as these are used in the body of the inner loop:
Β Β Β O(πlogΒ³π)
The assignment should specify which of the two approaches is to be used.
How to formulate Mathematical equation for 'time complexity' of Searching an integer x in Linked List Using Recursion. Recursive Equation should obviously be in terms of n (where n=total number of elements in linked list). Below is the Java implementation of Recursive Search in Linked list.
public static int search(Node head, int x) {
if(head==null)
return -1;
else if(head.data==x)
return 1;
else {
int res=search(head.next,x);
if(res==-1)
return -1;
else
return res+1;
}
}
I know that time complexity is obviously O(n) as it traverses every element to search in worst case but I want to derive the same from recursive equation.
Define ππ, π as the time complexity for performing the search, when the first node that has the targetted data is at position π (1-based) in a list that has π nodes. To get a useful value for π when there is no match, let's define π as the number of nodes in the longest prefix of the list that does not have the matching node, ... plus 1. So when there is no match, we will have π = π + 1.
Then:
Β Β Β π1, 0 represents the case of an empty list where there is (obviously) no such node.
Β Β Β π1, π represents the case where the head node has the targetted data
Then we have these equations:
Β Β Β π1, 0 = O(1)
Β Β Β π1, π = O(1) when π > 0
Β Β Β ππ, π = O(1) + ππβ1, πβ1 when π > 1 and π > 0
The last equation reflects that the recursive call is made with head.next: that call deals with a list that is one node shorter, and so both the involved size (π) and matching position (π) are one less.
Which one of the base cases kicks in first depends on how π and π relate. If π = π + 1, then the first base case will kick in. In all other cases, the second base case will kick in.
The recurrence relation will thus resolve to:
Β Β Β ππ, π = O(π) when π <= π
Β Β Β ππ, π = O(1 + π) when π = π + 1
So in both cases:
Β Β Β ππ, π = O(π)
...which in the worst case is O(1 + π) = O(π).
The time can be proportional to sum of depth of all nodes.
T(n): the minimal depth sum of all possible BSTs with n elements.
So, T(n) = n-1 + min(T(i)+T(n-1-i)), i from 0 to n-1.
I think that the lower bound should be Ξ(n lg n), but I can not prove it.
The lower bound time complexity for building a binary tree from an input stream would be achieved when we can add nodes as high up in the tree as possible (having the least depth). This way the tree remains optimally balanced throughout the process. For instance, we could have this input stream:
4 2 6 1 3 5 7
...which really provides the values for the future tree in level order (breadth-first order).
The time complexity for the individual insertions is linear to the number of nodes to traverse, i.e. the height of the tree at the moment of insertion. This height is O(logπ), where π is the size of the tree at the moment of insertion.
In general, we have this equality for the combined work of all insertions:
Β Β Β Tπ = Ξ( βπ=1..π logπ )
Which by the logarithm product rule is:
Β Β Β Tπ = Ξ(log(π!))
Which by Stirling's approximation means that:
Β Β Β Tπ = Ξ(πlogπ)
// sorts the items from L[i] through L[j]
void threewaysort(int[] L, int i, int j) {
if (L[i] > L[j]) swap (i, j);
if (j - i + 1 > 2) {
t = (j - i + 1) / 3;
threewaysort(L, i, j - t);
threewaysort(L, i + t, j);
threewaysort(L, i, j - t);
}
}
The above code is sorting a list L from smallest to largest. To prove this algorithm is correct, I think we can use induction? The hint is, the call threewaysort(L, 0, L.length-1) actually has the side effect of making L sorted.
But I'm currently stuck in the steps of an induction proof.
You can indeed use induction. Let's use the notation Li,j to denote the subarray with the items from L[i] through L[j].
The base case
There are two base cases for this induction proof:
j - i + 1 = 1
This means there is only one element in Li,j, and by consequence it is already sorted. Neither if condition is true, and so nothing happens: Li,j is sorted after calling threewaysort(L, i, j).
j - i + 1 = 2
There are two elements in Li,j. If not yet sorted, the first if condition is true, and the call to swap will effectively sort Li,j. The second if condition is false. So Li,j is sorted after calling threewaysort(L, i, j)
The induction step
We arrive at the cases where j - i + 1 > 2
There are now at least 3 elements in Li,j. The proof by induction let's us assume that the threewaysort works correctly for smaller subarrays.
We ignore for the moment that a swap might be performed and focus on the body of the second if only, which will be executed:
t is guaranteed to be greater than zero.
Three recursive calls are made: on subarrays Li,j-t, Li+t,j, and again Li,j-t.
Let's define:
Β Β Β A = Li,i+t-1
Β Β Β B = Li+t,j-t
Β Β Β C = Lj-t+1,j
These are non-overlapping, adjacent ranges of Li,j. The sizes of A and C are both t. B has size of at least t (could be t, t+1 or t+2).
Let's also define the plus notation to represent the union of two subarrays. So then Li,j = A+B+C, and the recursive calls actually sort A+B, B+C and then A+B again.
As t is strictly positive, A+B and B+C are smaller subarrays than A+B+C, and so we may assume these recursive calls successfully sort the corresponding subarrays (induction premise).
Let's see what happens with the t greatest values in A+B+C. Those that are not in C will end up in B after the first recursive call (recall that the size of B is at least t). So then we are sure the t greatest values are all in B+C. After the second recursive call we can thus be sure that all those t greatest values are only to be found in C.
A similar thing happens with the t smallest values in A+B+C. After the first recursive call none of them can be in B any longer, but that is not really helpful. After the second recursive call none of them can be in C anymore. After the third recursive call none of them can be in B either, and so they are all in A.
Summarising we get that:
A is sorted (because after the last recursive call A+B was sorted)
B is sorted (same reason)
C is sorted (because after the second recursive call B+C was sorted, and C was not touched during the third call)
A has the smallest values from A+B+C
C has the greatest values from A+B+C
B has the remaining values from A+B+C
This means that A+B+C is sorted.
This completes the proof by induction.
Fewer swaps
The proof demonstrates also that the swap is optional for when the size of the array is different from 2. So the code would even be correct like this:
void threewaysort(int[] L, int i, int j) {
if (j - i + 1 > 2) {
t = (j - i + 1) / 3;
threewaysort(L, i, j - t);
threewaysort(L, i + t, j);
threewaysort(L, i, j - t);
} else if (L[i] > L[j]) {
swap(L, i, j);
}
}
However, performing the swaps as depicted in the original code will on average lead to fewer swaps (but more comparisons).
Time Complexity
First we note that apart from the recursive calls all other statements execute in constant time.
Secondly, the recursive calls are made on an array that has a size that is approximately one third smaller.
So with n = j - i + 1, the recurrence relation is:
Β Β Β f(n) = 3Β·f((2/3)n)
Β Β Β f(2) = f(1) = 1
If we expand the recurrence, we get:
Β Β Β f(n) = 32Β·f((2/3)2n) = ... = 3kΒ·f((2/3)kn)
When k is chosen such that (2/3)kn = 2 (or 1) then we know that f((2/3)kn) = 1, and that factor can be omitted from the expression:
Β Β Β f(n) = 3k
Now we must resolve k in terms of n:
Β Β Β (2/3)kn = 2
Β Β Β k = log3/2(n/2)
Β Β Β k = log3(n/2) / log3(3/2)
Β Β Β k = 2.7 log3(n/2)
So, now we have:
Β Β Β f(n) = 3k
Β Β Β f(n) = (3log3(n/2))2.7
Β Β Β f(n) = (n/2)2.7
Which sets the time complexity approximately to:
Β Β Β f(n) = O(n2.7)
... a quite inefficient sorting algorithm; less efficient than Bubble Sort.
I realise that the question is not clear, but I'm not really sure what this problem itself is all about. I have an "algorithm and analysis" mid exam coming up.
Problem 4 (5 points)
Consider the following equation system:
Β Β Β Β max Β Β x1 + 4x2 + 3x3
Β Β Β Β x1 + 4x2 + 3x3 Β Β β€ Β Β 4
I think that I can solve this using dynamic programming, but I'm not sure... It looks a lot like a knapsack problem, but I'm not sure what I would consider as the value, and what as the weight...
As the expression to maximise is also the expression that is constrained to an upper limit, you obviously need to solve this equation:
x1 + 4x2 + 3x3 = 4
As you have more than one freedom (3 variables, 1 equation, so 2 dimensions of freedom), there are an infinite number of solutions.
You can pick any value for x2 and x3, and then the corresponding x1 is:
x1 = 4 β 4x2 β 3x3
One obvious solution would be
x_1 = 0
x_2 = 1
x_3 = 0
which can be found by inspection; another one would be as follows.
x_1 = 1
x_2 = 0
x_3 = 1