I am currently learning Loop Invariants and is wondering whether I have generated them correctly here. The algorithm pseudocode is:
**EvenSumming(A)**
outcome=0
for i=1 to n
if A[i] is even
outcome=outcome+A[i]
return outcome
So far my LI and proving:
Loop Invariant (LI): At every iteration, the variable outcome is always even and at most can only be composed of i-1 elements originally in A[i....n].
Initialization: Let i = 1, thus outcome starts at 0, which makes it an even number and is composed of 0 elements from A[1...n]
Maintenance: By math, we have that adding two even numbers always results in an even number, and since every iteration in the array increments i by 1 then trivially we know that there are i-1 elements that have been iterated through before i. Thus, at iteration i, outcome will be an even value composed of at most i-1 elements from A[i...n]
Termination: At termination, let i > n and thus we can say that i = n+1. Then by LI we have that outcome is still even value composed from at most n elements originally existing in A[n]
Related
i am learning correctness and struggling to find the appropriate loop invariant and prove its correctness.
I think the loop invariant should be the t=sum of the positive value but i don't know how to write it correctly or is there any other loop invariant?
SumPos(A[0..n - 1])
// Returns the sum of the positive numbers in an array A of length n. We
// assume that the sum of the numbers in an array of length zero is
zero.
t = 0
i = 0
while i != n do
if A[i] > 0
t = t + A[i]
i = i + 1
return t
Before getting formal, it is sometimes helpful to think in terms of "what stays the same" and "what changes" as regards the loop.* For the loop written, we have these variables of interest:
A - the array of numbers to sum
n - the integral number of elements in A.
t - as written, I assume intended to be the eventual sum-of-positives
i - the index variable; sometimes called the variant
So what changes each iteration? The array A does not change. The number of elements n in the array does not change. As written, the sum t might change. The index variable i will change.
As pertains to the loop, then, folks generally say that i is the variant. It increments every iteration and a comparison of it against n is what exits the loop.
The invariant of interest to me is that t will always represent the calculated-so-far sum-of-positives. For example, on the first iteration:
Before the iteration, i == 0 and t is also correctly 0
After the iteration, i == 1 and t will be correct with respect to the first element.
However, as written, the return statement precludes any processing beyond the first element of the array. Moving from theory to practice, how then might you fix the implementation?
* For the pedantic, the qualifier is important because strictly speaking an "invariant" is something that does not vary -- does not change, or always holds true -- for every iteration of the loop. So? Lots of statements are invariant with respect to the loop! For example, my mother's name is invariant for the loop!
I am covering the counting sort algorithm, and I understand how it works, but I would like to know if there is a specific way to prove that counting sort is a stable algorithm. I have an idea on how to inductively prove this, but I am unsure of how to do it.
for i = 1 to k
C[i] = 0
for j = 1 to n
C[A[j]] = C[A[i]] + 1
for i = 2 to k
C[i] = C[i] + C[i-1]
for j=n to 1
B[C[A[j]]] = A[j]
C[A[j]]--
I assume that the proof would start with a base case where an array has only one element
base case of n = 1, unsorted array A[1] = a1 , sorted array B[1] = a1
Inductive step: ???
I'm not sure how to handle this type of inductive proof.
To write a proof by mathematical induction that counting sort is a stable sort, you must do three things:
Choose a base case and show that your claim is true in the base case.
Assume that the claim is true for all problem instances of size up to and including some size k. This is strong induction but we might as well do it because it's one less thing to worry about.
Demonstrate how problem instances of the next higher size must preserve the truth of the claim.
Our base cases might as well be empty arrays and arrays of size one. Any sorting algorithm is trivially stable on these.
The induction hypothesis is similarly easy to state: counting sort is stable on all arrays of no more than k elements.
The inductive step is the tricky part. Consider an array of size k+1. This array has a largest, last element (of the elements that are largest by the sorting criteria, there is one that appears last in the array). Consider the array of size k obtained by removing this element from the array of size k+1 and shifting elements appearing to the right of it left to fill the gap. By the induction hypothesis, counting sort is stable on this array of size k. To prove that counting sort is stable on the array of size k+1 elements, we must therefore show only that counting sort cannot place the largest, last element before any elements of the same size. To see this is true, notice that when the output array B is being constructed, j assumes the values n, n-1, …, 1 in descending order, so of all the elements with the same value as our largest, last element, it will reach our element first. Because of this, it is guaranteed to place this element further to the right in B than it will place any other element with the same value since C[A[j]] is decremented, never incremented, in the loop constructing B.
The title is slightly misleading: What I am trying to is a little bit more complicated, but I don't know how to describe it in a title.
I have two sequences (vectors, arrays, lists) p and q of numbers. I sum all of the numbers in p except the first element, then perform a test. If the fails, I replace the first element of p with the first element of q, and start over with the second element: Sum all elements of p except the second one (including the new element copied from q). If the test fails the second time, replace the second element in p with the second element in q, and do it again with the third element. The test always succeeds on some element. Pseudocode:
i = 0
if test(sum_except p i) then exit loop
else p[i] := q[i], and run again with i+1
This means that if I have 1000 elements, I sum 999 of them, and then the next time through I sum 998 of those plus a new element, and so on. As the process goes on, I am also summing more and more elements from q rather than from the original p, but I will have summed most of them, previously, as well. So at each step, I'm summing a bunch of numbers that I had already added together. This seems inefficient, but I have not figured out a more sensible way to write this.
In case it matters, what the test does is check whether the sum of the values in p except the element at i, plus the value of q at i, is greater or equal to than 1 (or is less than or equal to 1--there are two variants). i.e.:
(sum_except p i) + q[i] > 1
This all happens inside an inner loop, and the sequence length will probably be between 500 and 5000, although larger numbers are possible.
(My code actually is in OCaml, fwiw.)
Calculate the sum of all numbers at the start (henceforth referred to as sum). sum_except p i is then equivalent to sum - p[i].
When you update an item in the list, update the sum as well so that you do not have to recalculate it by looping through each element in p every time.
// Remove old value
sum := sum - p[i]
p[i] := q[i]
// Add new value
sum := sum + p[i]
Since you will be modifying at most one value in the list at a time, you can simply update the sum manually to reflect the change instead of summing all elements in the list again, which makes the loop body run in constant time.
CLRS says that
We must show three things about a loop invariant:
Initialization: It is true prior to the first iteration of the loop.
Maintenance: If it is true before an iteration of the loop, it remains true before the next iteration.
Termination: When the loop terminates, the invariant gives us a useful property that helps show that the algorithm is correct.
My question is can I edit the steps and make them these instead:
Initialization: It is true after the first iteration of the loop.
Maintenance: If it is true after an iteration of the loop, it remains true after the next iteration.
Termination: When the loop terminates, the invariant gives us a useful property that helps show that the algorithm is correct.
Explanation: Basically we use Principle Of Mathematical Induction to prove correctness. Using "Initialization" we prove that the property holds after the 1st iteration. Using "Maintenance" we can show that the property holds for all iterations since "Initialization" and "Maintenance" together create a chain. And hence the property holds after the last iteration too.
Example:
RANDOMIZE-IN-PLACE(A)
1 n ← length[A]
2 for i ← to n
3 do swap A[i] ↔ A[RANDOM(i, n)]
For this algorithm, there is already a proof given in the textbook using the standard procedure (I didn't include it here since it is too long).
My suggestion:
Loop Invariant: Just after the ith iteration of the for loop of lines 2-3, for each possible (i)-permutation, the subarray A[1 .. i] contains this (i)-permutation with probability (n - i)!/n!.
Initialization: After 1st iteration A[1] contains this permutation with probability (n-1)!/n!=1/n which is true.
Maintenance: Let it be true after (i-1)th iteration. After (i)th iteration, A[1...i] contains this permutation with probability [(n-i+1)!/n!]*[1/(n-i+1)]=(n-i)!/n! which is what we want.
Termination: At termination, i = n, and we have that the subarray A[1 .. n] is a given n-permutation with probability (n - n)!/n! = 1/n!. Thus, RANDOMIZE-IN-PLACE produces a uniform random permutation.
Is my explanation logically sound?
Any help would be greatly appreciated. Thanks.
Apart from the fact, that you have to make an additional step, which covers the loop never running at all, you can of course also prove that the invariant is true after the first iteration instead of proving that it is true before the first iteration.
However I doubt that this will often make much sense
First, as already stated you have already a special case (what happens if the loop isn't executed at all) which probably results in a proof, which is similar to the Initialization, that you wanted to skip in the first place
Secondly, the proof that the invariant is true after the first iteration, is very likely to be very similar to the Maintanance proof, so you are likely to write two very similar proofs, just to skip the Initialization, which is likely to be quite an easy proof.
Analogy
Although this is not directly related, this question sounds a lot like trying to optimize the following (pseudo)code:
int product(int[] values)
product = 1
for i = 0 to values.size - 1
product *= values[i]
return product
To this:
int product(int[] values)
product = values[0] // to make the code quicker, start with the first item already
for i = 1 to values.size - 1 // start the loop at the second item
product *= values[i]
return product
just, that you now have to include the additional check, whether the values array is empty (which I did not in the above code)
Algorithm(a-array, n-length):
for(i=2;i<=n;i++)
if(a[1]<a[i]) Swap(a,1,i);
for(i=n-1;i>=2;i--)
if(a[n]<a[i]) Swap(a,n,i);
I'm interested in determining how many times Swap is called in the code above in the worst case, so I have some questions.
What's the worst case there?
If I had only the first for loop, it could be said that the worst case for this algorithm is that the array a is already sorted in ascending order, and Swap would be called n-1 times.
If I had only the second loop, the worst case would also be that a is already sorted, but this time, the order would be descending. That means that if we consider the first worst case, the Swap wouldn't be called in the second loop, and vice versa, i.e. it can't be called in both loops in each iteration.
What should I do now? How to combine those two worst cases that are opposite to each other?
Worst case means that I want to have as many Swap calls as possible. : )
P.S. I see that the complexity is O(n), but I need to estimate as precisely as possible how many times is the Swap executed.
EDIT 1: Swap(a,i,j) swaps the elements a[i] and a[j].
Let s and r be the positions of the largest and next to largest elements in the original array. At the end of the first loop:-
the largest will come to the first position.
If r < s then the position of the next to largest will now be r. if r > s it will still be r.
At the end of second loop the next to largest element will be at the end
For the first loop the worst case for fixed s is when all elements upto s are in ascending order. The number of swaps is s.
For the second loop the worst case occurs if the next to largest is closer to the beginning of the array. this occurs when r < s and all elements after the largest were in descending order in the original array(they will be untouched even after the first loop). The number of swaps is n-s-1
Total = n-1 in the worst case independent of r and s.
eg A = [1 2 5 7 3 4] Here upto max elemnt 7 it is ascending and after that descending
number of swaps = 5
The worst case for the first loop is that every ai is smaller than aj with 1 ≤ i < j ≤ n. In that case, every aj is swapped with a1 so that at the end a1 is the largest number. This swapping can only happen at most n-1 times, e.g.:
[1,2,3,4,5] ⟶ [5,1,2,3,4]
Similarly, the worst case for the second loop is that every ai is greater than aj with 2 ≤ i < j ≤ n. In that case, every ai is swapped with an so that at the end an is the largest number of the sub-array a2,…,an. This swapping can only happen at most n-2 times, e.g.:
[x,4,3,2,1] ⟶ [x,3,2,1,4]
Now the tricky part is to combine both conditions as the conditions for a Swap call in both loops are mutually exclusive: For any pair ai, aj with 1 ≤ i < j ≤ n and ai < aj, the first loop will call Swap. But for any of such pairs, the second loop won’t call Swap as it expects the opposite: ai > aj.
So the maximum number of Swap calls is n-1.