I need an algorithm which can help me divide N-elements array into pairs, elements in each pair must have minimum difference
So, I assume that we may throw away at most 1 element (in case when N is odd). Let a1 ≤ a2 ≤ … ≤ aN be our set of numbers sorted in non-decreasing order. Let f(k) be the minimum possible sum of differences inside pairs formed by the first k numbers. (So minimum is taken over all partitions of the first k numbers into pairs.) As it is mentioned in comments for even N we just need to take elements one by one. Then
f(0) = f(1) = 0,
f(2·k) = f(2·k − 2) + (a2·k − a2·k − 1) for k ≥ 1 and
f(2·k + 1) = min { f(2·k), f(2·k − 1) + (a2·k + 1 − a2·k) } for k ≥ 1.
The last formula means that we can either throw away (2·k + 1)th element or throw away one element among the first (2·k − 1). On the remaining 2·k elements we apply known solution for even N.
This is a way to find numerical answer for the problem in O(N) time after sorting numbers, i. e. in general it takes O(N log N) time. After that if you need partition of all numbers into pairs just use backward step of dynamic programming. For even N just make pairs trivially. For odd N throw away aN if f(N) = f(N − 1) or take pair (aN, aN − 1) and decrease N by 2 otherwise.
Related
I have an array of n random integers
I choose a random integer and partition by the chosen random integer (all integers smaller than the chosen integer will be on the left side, all bigger integers will be on the right side)
What will be the size of my left and right side in the average case, if we assume no duplicates in the array?
I can easily see, that there is 1/n chance that the array is split in half, if we are lucky. Additionally, there is 1/n chance, that the array is split so that the left side is of length 1/2-1 and the right side is of length 1/2+1 and so on.
Could we derive from this observation the "average" case?
You can probably find a better explanation (and certainly the proper citations) in a textbook on randomized algorithms, but here's the gist of average-case QuickSort, in two different ways.
First way
Let C(n) be the expected number of comparisons required on average for a random permutation of 1...n. Since the expectation of the sum of the number of comparisons required for the two recursive calls equals the sum of the expectations, we can write a recurrence that averages over the n possible divisions:
C(0) = 0
1 n−1
C(n) = n−1 + ― sum (C(i) + C(n−1−i))
n i=0
Rather than pull the exact solution out of a hat (or peek at the second way), I'll show you how I'd get an asymptotic bound.
First, I'd guess the asymptotic bound. Obviously I'm familiar with QuickSort and my reasoning here is fabricated, but since the best case is O(n log n) by the Master Theorem, that's a reasonable place to start.
Second, I'd guess an actual bound: 100 n log (n + 1). I use a big constant because why not? It doesn't matter for asymptotic notation and can only make my job easier. I use log (n + 1) instead of log n because log n is undefined for n = 0, and 0 log (0 + 1) = 0 covers the base case.
Third, let's try to verify the inductive step. Assuming that C(i) ≤ 100 i log (i + 1) for all i ∈ {0, ..., n−1},
1 n−1
C(n) = n−1 + ― sum (C(i) + C(n−1−i)) [by definition]
n i=0
2 n−1
= n−1 + ― sum C(i) [by symmetry]
n i=0
2 n−1
≤ n−1 + ― sum 100 i log(i + 1) [by the inductive hypothesis]
n i=0
n
2 /
≤ n−1 + ― | 100 x log(x + 1) dx [upper Darboux sum]
n /
0
2
= n−1 + ― (50 (n² − 1) log (n + 1) − 25 (n − 2) n)
n
[WolframAlpha FTW, I forgot how to integrate]
= n−1 + 100 (n − 1/n) log (n + 1) − 50 (n − 2)
= 100 (n − 1/n) log (n + 1) − 49 n + 100.
Well that's irritating. It's almost what we want but that + 100 messes up the program a little bit. We can extend the base cases to n = 1 and n = 2 by inspection and then assume that n ≥ 3 to finish the bound:
C(n) = 100 (n − 1/n) log (n + 1) − 49 n + 100
≤ 100 n log (n + 1) − 49 n + 100
≤ 100 n log (n + 1). [since n ≥ 3 implies 49 n ≥ 100]
Once again, no one would publish such a messy derivation. I wanted to show how one could work it out formally without knowing the answer ahead of time.
Second way
How else can we derive how many comparisons QuickSort does in expectation? Another possibility is to exploit the linearity of expectation by summing over each pair of elements the probability that those elements are compared. What is that probability? We observe that a pair {i, j} is compared if and only if, at the leaf-most invocation where i and j exist in the array, either i or j is chosen as the pivot. This happens with probability 2/(j+1 − i), since the pivot must be i, j, or one of the j − (i+1) elements that compare between them. Therefore,
n n 2
C(n) = sum sum ―――――――
i=1 j=i+1 j+1 − i
n n+1−i 2
= sum sum ―
i=1 d=2 d
n
= sum 2 (H(n+1−i) − 1) [where H is the harmonic numbers]
i=1
n
= 2 sum H(i) − n
i=1
= 2 (n + 1) (H(n+1) − 1) − n. [WolframAlpha FTW again]
Since H(n) is Θ(log n), this is Θ(n log n), as expected.
Suppose an array = {2,5,7,8,10}. You need to find the length of Longest Increasing Sub-sequence such that a element is not less than the sum of all elements before it.
In this case the answer can be {2,5,7}, {2,5,8} or {2,8,10}. So Length = 3
This is easily solvable in O(n^2). As LIS Length can be found in O(n log n). As the problem is asking only the length, so, I think this problem is also solvable in O(n log n). But how can I do that?
Actually you don't need a DP Solution at all.
First sort the numbers in non-decreasing order. And loop from left to right. keep track of the current sum.
If next number is not-less than the sum add it to LIS. else proceed to next number.
It can be proven that the greedy solution is the optimal solution. Prove it yourself ;)
There's an O(N^2) dynamic programming solution which goes like:
Let f(i, j) be the smallest sum that a "correct" subsequence ending in one of the first i elements and consisting of exactly j elements can have.
The base case is f(0, 0) = 0 (empty prefix, no elements)
The transition are f(i, j) -> f(i + 1, j) (not adding a new element) and
f(i, j) -> f(i + 1, j + 1) if a[i] > f(i, j) (adding the i-th element to the end of the subsequence if we can).
The correctness of this solution is self-evident.
A cool fact: let A be a "correct" subsequence of k elements. Than the last element of A is not less than max(1, 2^(k-2))(proof: it's the case for k = 1 and k = 2. Now we can use induction and the fact that 1 + sum i = 0 .. k of 2^k = 2^(k+1))
Thus, j ranges over 0..log MAX_A + C in the dynamic programming solution described above, so it works in O(N * log MAX_A).
O(N * log MAX_A) is not O(N log N), but this solution can be good for practical purposes.
Many algorithms have loops in them that look like this:
for a from 1 to n
for b from 1 to a
for c from 1 to b
for d from 1 to c
for e from 1 to d
...
// Do O(1) work
In other words, the loop nest is k layers deep, the outer layer loops from 1 to n, and each inner layer loops up from 1 to the index above it. This shows up, for example, in code to iterate over all k-tuples of positions inside an array.
Assuming that k is fixed, is the runtime of this code always Θ(nk)? For the special case where n = 1, the work is Θ(n) because it's just a standard loop over an array, and for the case where n = 2 the work is Θ(n2) because the work done by the inner loop is given by
0 + 1 + 2 + ... + n-1 = n(n-1)/2 = Θ(n2)
Does this pattern continue when k gets large? Or is it just a coincidence?
Yes, the time complexity will be Θ(nk). One way to measure the complexity of this code is to look at what values it generates. One particularly useful observation is that these loops will iterate over all possible k-element subsets of the array {1, 2, 3, ..., n} and will spend O(1) time producing each one of them. Therefore, we can say that the runtime is given by the number of such subsets. Given an n-element set, the number of k-element subsets is n choose k, which is equal to
n! / k!(n - k)!
This is given by
n (n-1)(n-2) ... (n - k + 1) / k!
This value is certainly no greater than this one:
n · n · n · ... · n / k! (with k copies of n)
= nk / k!
This expression is O(nk), since the 1/k! term is a fixed constant.
Similarly, when n - k + 1 ≥ n / 2, this expression is greater than or equal to
(n / 2) · (n / 2) · ... · (n / 2) / k! (with k copies of n/2)
= nk / k! 2k
This is Ω(nk), since 1 / k! 2k is a fixed constant.
Since the runtime is O(nk) and Ω(nk), the runtime is Θ(nk).
Hope this helps!
You may consume the following equation:
Where c is the number of constant time operations inside the innermost loop, n is the number of elements, and r is the number of nested loops.
The following algorithm returns the previous larger element of an array. It is from page 11 of these notes.
// Input: An array of numeric values a[1..n]
// Returns: An array p[1..n] where p[i] contains the index of the previous
// larger element to a[i], or 0 if no such element exists.
previousLarger(a[1..n])
for (i = 1 to n)
j = i-1;
while (j > 0 and a[j] <= a[i]) j--;
p[i] = j;
return p
My homework question is: Given input sequence {a1,...,an} is a random permutation of the set {1,...,n}, what is the expected running time?
I think this requires some sort of probabilistic analysis, but I need some hints since I have only done worst-case analysis in the past. I'm trying to find a formula for the cost of the j-loop for a given i (1 + the number of times we do operation j--), then sum that formula up from 1 to n.
What does "expected" mean? I don't really know how to interpret this.
Building on #Heuster's answer:
1) You know that the answer is between O(n) and O(n^2). This is just to check the final result.
2) The expected number of steps for element i would indeed be:
sum_{k=1}^i 1 / (k+1)
= O(log i)
3) You have to sum all those number over i. This gives you:
sum_{i=1}^n O(log i)
= O(n log n)
What I did is not rigorous at all but you can prove derive it. O(n log n) is between O(n) and O(n^2) so it seems a good candidate :)
For and arbitrary index i, what is the chance that a[i-1] > a[i] (in other words, the inner while loop will take one step)? That one is easy: all elements in a are different, so P(a[i-1] > a[i]) = P(a[i] > a[i-1]) = 1/2.
Now, look at the case that the inner while loop would need to take two steps. That is, a[i-2] > a[i] > a[i-1]. This is exactly one of the 6 permutations of 3 elements, so the chance is 1 / 3! = 1 / 6.
Let's generalize this and assume that the inner while loop would need to take k steps. We consider the sublist a[i-k], a[i-k+1], ..., a[i]. We know that a[i-k] is the maximum element of this sublist and a[i] the second largest (otherwise, the inner loop would have stopped sooner). The order of the elements in between is irrelevant. The chance that we take k steps is thus 1 / (k + 1) * 1 / k = 1 / (k * (k + 1)). Note that this indeed degeneralizes to 1/2 for k = 1 and 1/6 for k = 2.
The chance that no element before a[i] is larger is simply 1 / i (a[i] is the maximum element in that sublist). In that case, the inner loop would need i steps.
The expected number of steps for element i would be (sum of probability times value):
Sum[{k, 1, i} k * 1 / ((k * (k + 1))] + i / i
= Sum[{k, 1, i} 1 / (k + 1)] + 1
= H_{i+1}
where H_{i} is the ith harmonic number, which is the discrete variant of log i. That is, the number of steps for element i is Θ(i).
What is remaining now is sum over all i to find the expected running time. With the exact value (H_{i+1}) this doesn't lead to a nice expression, see Wolfram Alpha.
The standard way to proceed, however, is to continue with the approximated log i. Clearly, log 0 + log 1 + ... + log n-1 is less than n log n. Now, consider the last half of the sum:
log n/2 + log n/2+1 + ... + log n-1 > n/2 log n/2
= n/2 (log n - log 2)
= Ω(n log n)
Therefore, the expected running time is Θ(n log n).
I was studying the merge-sort subject that I ran into this concept that the number of comparisons in merge-sort (in the worst-case, and according to Wikipedia) equals (n ⌈lg n⌉ - 2⌈lg n⌉ + 1); in fact it's between (n lg n - n + 1) and (n lg n + n + O(lg n)). The problem is that I cannot figure out what these complexities try to say. I know O(nlogn) is the complexity of merge-sort but the number of comparisons?
Why to count comparisons
There are basically two operations to any sorting algorithm: comparing data and moving data. In many cases, comparing will be more expensive than moving. Think about long strings in a reference-based typing system: moving data will simply exchange pointers, but comparing might require iterating over a large common part of the strings before the first difference is found. So in this sense, comparison might well be the operation to focus on.
Why an exact count
The numbers appear to be more detailed: instead of simply giving some Landau symbol (big-Oh notation) for the complexity, you get an actual number. Once you have decided what a basic operation is, like a comparison in this case, this approach of actually counting operations becomes feasible. This is particularly important when comparing the constants hidden by the Landau symbol, or when examining the non-asymptotic case of small inputs.
Why this exact count formula
Note that throughout this discussion, lg denotes the logarithm with base 2. When you merge-sort n elements, you have ⌈lg n⌉ levels of merges. Assume you place ⌈lg n⌉ coins on each element to be sorted, and a merge costs one coin. This will certainly be enough to pay for all the merges, as each element will be included in ⌈lg n⌉ merges, and each merge won't take more comparisons than the number of elements involved. So this is the n⌈lg n⌉ from your formula.
As a merge of two arrays of length m and n takes only m + n − 1 comparisons, you still have coins left at the end, one from each merge. Let us for the moment assume that all our array lengths are powers of two, i.e. that you always have m = n. Then the total number of merges is n − 1 (sum of powers of two). Using the fact that n is a power of two, this can also be written as 2⌈lg n⌉ − 1, and subtracting that number of returned coins from the number of all coins yields n⌈lg n⌉ − 2⌈lg n⌉ + 1 as required.
If n is 1 less than a power of two, then there are ⌈lg n⌉ merges where one element less is involved. This includes a merge of two one-element lists which used to take one coin and which now disappears altogether. So the total cost reduces by ⌈lg n⌉, which is exactly the number of coins you'd have placed on the last element if n were a power of two. So you have to place fewer coins up front, but you get back the same number of coins. This is the reason why the formula has 2⌈lg n⌉ instead of n: the value remains the same unless you drop to a smaller power of two. The same argument holds if the difference between n and the next power of two is greater than 1.
On the whole, this results in the formula given in Wikipedia:
n ⌈lg n⌉ − 2⌈lg n⌉ + 1
Note: I'm pretty happy with the above proof. For those who like my formulation, feel free to distribute it, but don't forget to attribute it to me as the license requires.
Why this lower bound
To proove the lower bound formula, let's write ⌈lg n⌉ = lg n + d with 0 ≤ d < 1. Now the formula above can be written as
n (lg n + d) − 2lg n + d + 1 =
n lg n + nd − n2d + 1 =
n lg n − n(2d − d) + 1 ≥
n lg n − n + 1
where the inequality holds because 2d − d ≤ 1 for 0 ≤ d < 1
Why this upper bound
I must confess, I'm rather confused why anyone would name n lg n + n + O(lg n) as an upper bound. Even if you wanted to avoid the floor function, the computation above suggests something like n lg n − 0.9n + 1 as a much tighter upper bound for the exact formula. 2d − d has its minimum (ln(ln(2)) + 1)/ln(2) ≈ 0.914 for d = −ln(ln(2))/ln(2) ≈ 0.529.
I can only guess that the quoted formula occurs in some publication, either as a rather loose bound for this algorithm, or as the exact number of comparisons for some other algorithm which is compared against this one.
(Two different counts)
This issue has been resolved by the comment below; one formula was originally quoted incorrectly.
equals (n lg n - n + 1); in fact it's between (n lg n - n + 1) and (n lg n + n + O(lg n))
If the first part is true, the second is trivially true as well, but explicitely stating the upper bound seems kind of pointless. I haven't looked at the details myself, but these two statements appear strange when taken together like this. Either the first one really is true, in which case I'd omit the second one as it is only confusing, or the second one is true, in which case the first one is wrong and should be omitted.