We're supposed to "consider the following algorithm, which operates on an array A[1 . . n] of integers."
for i in [1 . . n] do
A[i] ← 0
for i in [1 . . n] do
j ← i
while j ≤ n do
A[j] ← A[j] + 1
j ← j + i
The assignment asks us to demonstrate that this algorithm runs in O(n log n).
The first loop is quite clearly going to add n to the run time, which would simply be dropped.
The second nested loops will run faster than a pure O(n^2) algorithm, since the while loop doesn't always run n times. When i = 1 it goes n times, i = 2 it will run n-1 times, all the way up to i = n where it will run once.
But, using the same method as Gauss summing the integers between 1 and 100, we can see that the while loop will run an average of (n+1)/2 times. Multiply by n for the for loop, and we get to (n^2 + n)/2, which can be simplified down to O(n^2), not O(n log n)
How does this result in an O(n log n) running time?
Considering the following:
for i in [1 . . n] do
j ← i
while j ≤ n do
A[j] ← A[j] + 1
j ← j + i
Each time, j is incremented with +i, not +1. As such, the first while loop while be iterated n times, then n/2, then n/3... up to n/n.
Ignoring integer rounding, we can write this in the following format:
n(1 + 1/2 + 1/3 + ... + 1/n).
It seems that we are dealing with an harmonic serie, with an additional multiplication by n. Some more details here and here. This would be O(n log n).
Related
I have an array of n random integers
I choose a random integer and partition by the chosen random integer (all integers smaller than the chosen integer will be on the left side, all bigger integers will be on the right side)
What will be the size of my left and right side in the average case, if we assume no duplicates in the array?
I can easily see, that there is 1/n chance that the array is split in half, if we are lucky. Additionally, there is 1/n chance, that the array is split so that the left side is of length 1/2-1 and the right side is of length 1/2+1 and so on.
Could we derive from this observation the "average" case?
You can probably find a better explanation (and certainly the proper citations) in a textbook on randomized algorithms, but here's the gist of average-case QuickSort, in two different ways.
First way
Let C(n) be the expected number of comparisons required on average for a random permutation of 1...n. Since the expectation of the sum of the number of comparisons required for the two recursive calls equals the sum of the expectations, we can write a recurrence that averages over the n possible divisions:
C(0) = 0
1 n−1
C(n) = n−1 + ― sum (C(i) + C(n−1−i))
n i=0
Rather than pull the exact solution out of a hat (or peek at the second way), I'll show you how I'd get an asymptotic bound.
First, I'd guess the asymptotic bound. Obviously I'm familiar with QuickSort and my reasoning here is fabricated, but since the best case is O(n log n) by the Master Theorem, that's a reasonable place to start.
Second, I'd guess an actual bound: 100 n log (n + 1). I use a big constant because why not? It doesn't matter for asymptotic notation and can only make my job easier. I use log (n + 1) instead of log n because log n is undefined for n = 0, and 0 log (0 + 1) = 0 covers the base case.
Third, let's try to verify the inductive step. Assuming that C(i) ≤ 100 i log (i + 1) for all i ∈ {0, ..., n−1},
1 n−1
C(n) = n−1 + ― sum (C(i) + C(n−1−i)) [by definition]
n i=0
2 n−1
= n−1 + ― sum C(i) [by symmetry]
n i=0
2 n−1
≤ n−1 + ― sum 100 i log(i + 1) [by the inductive hypothesis]
n i=0
n
2 /
≤ n−1 + ― | 100 x log(x + 1) dx [upper Darboux sum]
n /
0
2
= n−1 + ― (50 (n² − 1) log (n + 1) − 25 (n − 2) n)
n
[WolframAlpha FTW, I forgot how to integrate]
= n−1 + 100 (n − 1/n) log (n + 1) − 50 (n − 2)
= 100 (n − 1/n) log (n + 1) − 49 n + 100.
Well that's irritating. It's almost what we want but that + 100 messes up the program a little bit. We can extend the base cases to n = 1 and n = 2 by inspection and then assume that n ≥ 3 to finish the bound:
C(n) = 100 (n − 1/n) log (n + 1) − 49 n + 100
≤ 100 n log (n + 1) − 49 n + 100
≤ 100 n log (n + 1). [since n ≥ 3 implies 49 n ≥ 100]
Once again, no one would publish such a messy derivation. I wanted to show how one could work it out formally without knowing the answer ahead of time.
Second way
How else can we derive how many comparisons QuickSort does in expectation? Another possibility is to exploit the linearity of expectation by summing over each pair of elements the probability that those elements are compared. What is that probability? We observe that a pair {i, j} is compared if and only if, at the leaf-most invocation where i and j exist in the array, either i or j is chosen as the pivot. This happens with probability 2/(j+1 − i), since the pivot must be i, j, or one of the j − (i+1) elements that compare between them. Therefore,
n n 2
C(n) = sum sum ―――――――
i=1 j=i+1 j+1 − i
n n+1−i 2
= sum sum ―
i=1 d=2 d
n
= sum 2 (H(n+1−i) − 1) [where H is the harmonic numbers]
i=1
n
= 2 sum H(i) − n
i=1
= 2 (n + 1) (H(n+1) − 1) − n. [WolframAlpha FTW again]
Since H(n) is Θ(log n), this is Θ(n log n), as expected.
The algorithm below has runtime O(n) according to our professor, however I am confused as to why it is not
O(n log(n)), because the outer loop can run up to log(n) times and the inner loop can run up to n times.
Algoritme Loop5(n)
i = 1
while i ≤ n
j = 1
while j ≤ i
j = j + 1
i = i∗2
Your professor was right, the running time is O(n).
In the i-th iteration of the outer while-loop, when we have i=2^k for k=0,1,...,log n, the inner while-loop makes O(i) iterations. (When I say log n I mean the base-2 logarithm log_2 n.)
The running time is O(1+2+2^2+2^3+...+2^k) for k=floor(log n). This sums to O(2^{k+1}) which is O(2^{log n}). (This follows from the formula for the partial sum of geometric series.)
Because 2^{log n} = n, the total running time is O(n).
For the interested, here's a proof that the powers of two really sum to what I claim they sum to. (This is a very special case of a more general result.)
Claim. For any natural k, we have 1+2+2^2+...+2^k = 2^{k+1}-1.
Proof. Note that (2-1)*(1+2+2^2+...+2^k) = (2 - 1) + (2^2 - 2) + ... + (2^{k+1} - 2^k) where all 2^i for 0<i<k+1 cancel out, except for i=0 and i=k+1, and we are left with 2^{k+1}-1. QED.
My attempt for the Big-O of each of these two algorithms..
1) Algorithm threeD(matrix, n)
// a 3D matrix of size n x n x n
layer ← 0
while (layer < n)
row ← 0
while (row < layer)
col ← 0
while (col < row)
print matrix[layer][row][col]
col ← col + 1
done
row ← row + 1
done
layer ← layer * 2
done
O((n^2)log(n)) because the two outer loops are each O(N) and the innermost one seems to be O(log n)
2) Algorithm Magic(n)
//Integer, n > 0
i ← 0
while (i < n)
j ← 0
while (j < power(2,i))
j ← j + 1
done
i ← i + 1
done
O(N) for outer loop, O(2^n) for inner? = O(n(2^n))?
1. Algorithm
First of all: This algorithm never terminates due to layer is initiated with zero. layer is only multipyed by 2 so it will never get bigger than zero, specially not bigger than n.
To get this work, you have to start with layer > 0.
So lets start with layer = 1.
The time-complexity can be written as T(n) = T(n/2) + n^2.
You can get this by looking that way: At the end the layer is setted at most to n. Then the inner loops do n^2 steps. Bevor that, the layer was only half that big. So you have to do the n^2 steps on the last rould of the outer loop and all stepps of the round bevor wirtten as T(n/2).
The masters theorem gets you Theta(n^2).
2. Algorithm
You can just count the steps:
2^0 + 2^1 + 2^2 + ... + 2^(n-1) = sum_(i=0)^(n-1)2^i = 2^n-1
To get this simplification just take a look at binary numbers: The sum of steps corresponds to a binary number containing only 1's (like 1111 1111). This number equals 2^n-1.
So the time complexity is Theta(2^n)
Notice: Both your Big-O's are not wrong, there are bette boundings.
The following algorithm returns the previous larger element of an array. It is from page 11 of these notes.
// Input: An array of numeric values a[1..n]
// Returns: An array p[1..n] where p[i] contains the index of the previous
// larger element to a[i], or 0 if no such element exists.
previousLarger(a[1..n])
for (i = 1 to n)
j = i-1;
while (j > 0 and a[j] <= a[i]) j--;
p[i] = j;
return p
My homework question is: Given input sequence {a1,...,an} is a random permutation of the set {1,...,n}, what is the expected running time?
I think this requires some sort of probabilistic analysis, but I need some hints since I have only done worst-case analysis in the past. I'm trying to find a formula for the cost of the j-loop for a given i (1 + the number of times we do operation j--), then sum that formula up from 1 to n.
What does "expected" mean? I don't really know how to interpret this.
Building on #Heuster's answer:
1) You know that the answer is between O(n) and O(n^2). This is just to check the final result.
2) The expected number of steps for element i would indeed be:
sum_{k=1}^i 1 / (k+1)
= O(log i)
3) You have to sum all those number over i. This gives you:
sum_{i=1}^n O(log i)
= O(n log n)
What I did is not rigorous at all but you can prove derive it. O(n log n) is between O(n) and O(n^2) so it seems a good candidate :)
For and arbitrary index i, what is the chance that a[i-1] > a[i] (in other words, the inner while loop will take one step)? That one is easy: all elements in a are different, so P(a[i-1] > a[i]) = P(a[i] > a[i-1]) = 1/2.
Now, look at the case that the inner while loop would need to take two steps. That is, a[i-2] > a[i] > a[i-1]. This is exactly one of the 6 permutations of 3 elements, so the chance is 1 / 3! = 1 / 6.
Let's generalize this and assume that the inner while loop would need to take k steps. We consider the sublist a[i-k], a[i-k+1], ..., a[i]. We know that a[i-k] is the maximum element of this sublist and a[i] the second largest (otherwise, the inner loop would have stopped sooner). The order of the elements in between is irrelevant. The chance that we take k steps is thus 1 / (k + 1) * 1 / k = 1 / (k * (k + 1)). Note that this indeed degeneralizes to 1/2 for k = 1 and 1/6 for k = 2.
The chance that no element before a[i] is larger is simply 1 / i (a[i] is the maximum element in that sublist). In that case, the inner loop would need i steps.
The expected number of steps for element i would be (sum of probability times value):
Sum[{k, 1, i} k * 1 / ((k * (k + 1))] + i / i
= Sum[{k, 1, i} 1 / (k + 1)] + 1
= H_{i+1}
where H_{i} is the ith harmonic number, which is the discrete variant of log i. That is, the number of steps for element i is Θ(i).
What is remaining now is sum over all i to find the expected running time. With the exact value (H_{i+1}) this doesn't lead to a nice expression, see Wolfram Alpha.
The standard way to proceed, however, is to continue with the approximated log i. Clearly, log 0 + log 1 + ... + log n-1 is less than n log n. Now, consider the last half of the sum:
log n/2 + log n/2+1 + ... + log n-1 > n/2 log n/2
= n/2 (log n - log 2)
= Ω(n log n)
Therefore, the expected running time is Θ(n log n).
I have analyzed running time of following alogirthm i analyzed theta but can its running time could be Big O?
Cost Time
1. for i ←1 to n c1 n
2. do for j ← i to n c2 n
3. do k ← k+ j c3 n-1
T(n) = c1n +c2n+c3(n-1)
= C1n+C2n+C3(n-1)
= n(C1+C2)+n-1
= n+n-1
Or T(n) = Ө(n)
So running time is Ө(n)
Your loop will continue as follows (well-known arithmetic progression formula):
-which also can be estimated as since big-O gives majority estimation.
1. for i ←1 to n c1 n
2. do for j ← i to n c2 n
3. do k ← k+ j c3 1
T(n) = n * n * 1 = O(n^2) #Giulio Franco
It's a nested loop that does a constant time operation in there.
do k ← k+ j is constant because it's a fixed length of time for the operation to take place no matter what inputs you put. k + j
loop(n)
loop(n)
constant time(1)
When it's a loop inside a loop you multiply. n*n*1
loop(n)
loop(n)
These loops aren't nested.
this would be n + n
O(n+n) which reduces to O(n)