Bellman-Ford algorithm proof of correctness - algorithm

I'm trying to learn about Bellman-Ford algorithm but I'm stucked with the proof of the correctness.
I have used Wikipedia, but I simply can't understand the proof. I did not find anything on Youtube that's helpfull.
Hope anyone of you can explain it briefly. This page "Bellman-ford correctness can we do better" does not answer my question.
Thank you.

Let's see the problem from the perspective of dynamic programming of a graph with no negative cycle.
We can visualize the memoization table of the dynamic programming as follows:
The columns represent nodes and the rows represent update steps(node 0 is the source node), and the arrows directing from one box in a step to another in the next step are the min updates(step 0 is the initialization).
We choose one path from all shortest paths and illustrate why it is correct. Let's choose the 0 -> 3 -> 2 -> 4 -> 5. It is the shortest path from 0 to 5, we can choose any other one otherwise. We can prove the correctness by reduction. The initial is the source 0, and obviously, the distance between 0 and itself should be 0, the shortest. And we assume 0 -> 3 -> 2 is the shortest path between 0 and 2, and we are going to prove that 0 -> 3 -> 2 -> 4 is the shortest path between 0 and 4 after the third iteration.
First, we prove that after the third iteration the node 4 must be fixed/tightened. If node 4 is not fixed it means that there is at least one path other than 0 -> 3 -> 2 -> 4 that can reach 4 and that path should be shorter than 0 -> 3 -> 2 -> 4, which contradicts our assumption that 0 -> 3 -> 2 -> 4 -> 5 is the shortest path between 0 and 5. Then after the third iteration, 2 and 4 should be connected.
Second, we prove that that relaxation should be the shortest. It cannot be greater and smaller because it is the only shortest path.
Let's see a graph with a negative cycle.
And here is its memoization table:
Let's prove that at |V|'s iteration, here |V| is the number of vertices 6, the update should not be stopped.
We assume that the update stopped(and there is a negative cycle). Let's see the cycle 3 -> 2 -> 4 -> 5 -> 3.
dist(2) <= dist(3) + w(3, 2)
dist(4) <= dist(2) + w(2, 4)
dist(5) <= dist(4) + w(4, 5)
dist(3) <= dist(5) + w(5, 3)
And we can obtain the following inequlity from the above four inequalities by summing up the left-hand side and the right-hand side:
dist(2) + dist(4) + dist(5) + dist(3) <= dist(3) + dist(2) + dist(4) + dist(5) + w(3, 2) + w(2, 4) + w(4, 5) + w(5, 3)
We subtract the distances from both sides and obtain that:
w(3, 2) + w(2, 4) + w(4, 5) + w(5, 3) >= 0, which contradicts our claim that 3 -> 2 -> 4 -> 5 -> 3 is a negative cycle.
So we are certain that at |V|'s step and after that step the update would never stop.
My code is here on Gist.
Reference:
dynamic programming - bellman-ford algorithm
Lecture 14: Bellman-Ford Algorithm

Related

Dijkstra's Algorithm with Negative Weights Query

In this scenario, the aim is to find a path with the smallest total weight. There are 5 sections with each section having different nodes. The nodes are only connected to nodes in adjacent sections and a path must consist of only a single node from each section.
For example, let:
section 1 has nodes [1, 2, 3].
section 2 has nodes [4, 5].
section 3 has nodes [6].
section 4 has nodes [7, 8, 9, 10, 11].
section 5 has nodes [12, 13, 14].
A valid path through the sections is [1, 4, 6, 7 , 12] and also [1, 5, 6, 11, 14] etc...
All nodes have negative weights but negative cycles are impossible (due to the one node per section policy). Therefore, does the process of adding a constant to each node resolve the issue of negative weights? If it can fix the issue, are there any papers which show this? I know there are other algorithms to resolve negative weights but I'm interestted in Dijkstra's algorithm. Thanks.
No, you can't do this. Let's have a look at the counter example. Suppose we have a graph with A, B, C nodes and egdes:
A - B -2 (negative)
A - C 6
B - C 7
we are looking for the shortest path from A to C. In the original graph we have
A - B - C => -2 + 7 = 5 (the shortest path, 5 < 6)
A - C => 6
The best choice is A - B - C. Now, let's get rid of negative edges by adding 2. We'll have now
A - B 0
A - C 8
B - C 9
A - B - C => 0 + 9 = 9
A - C => 8 (the shortest path, 8 < 9)
Please note, that now the shortest path is A - C. Alas! While adding constant value to each edge we ruin the problem itself; and it doesn't matter now which algorithm we use.
Edit: Counter example with all edges (arc to prevent negative loops) being negative:
A -> B -6
B -> C -1
A -> C -5
Before adding 6 we have
A -> B -> C = -6 - 1 = -7 (the shortest path)
A -> C = -5
After adding 6 we get
A -> B 0
B -> C 5
A -> C 1
A -> B -> C = 0 + 5 = 5
A -> C = 1 (the shortest path)

Algorithm - Finding the most rewarding path in a given graph

Question: You are given the following inputs:
3
0 0 1
3 1 1
6 0 9
The first line is the number of points on the graph.
The rest of the lines contain the points on the graph, and their reward. For example:
0 0 1 would mean at point (0,0) [which is the starting point] you are given a reward of 1.
3 1 1 would mean at point (3,1) you are given a reward of 1.
6 0 9 would mean at point (6, 0) you are given a reward of 9.
Going from point a, to point b costs 1.
Therefore if you go from (0,0) -> (3,1) -> (6,0) your reward is 11-2 (cost of traversing 2 nodes) * sqrt(10).
Goal: Determine the maximum amount of rewards you can make (the total amount of reward you collect - the cost) based on the provided inputs.
How would I go about solving this? It seems like dynamic programming is the way to go, but I am not sure where to start.

Minimum Hamiltonian path length using brute force approach

Assumption At least one Hamiltonian path exists in the graph. I am trying to find minimum path length among all Hamiltonian paths.
My approach
Let us say we have three nodes.
Possible paths are
1 -> 2 -> 3
1 -> 3 -> 2
2 -> 1 -> 3
2 -> 3 -> 1
3 -> 1 -> 2
3 -> 2 -> 1
Find path length of all tracks and take minimum among them. Time complexity of this approach will be O(N*(N!)) where N = #nodes
I am getting the wrong answer with this approach. Is the above approach correct? Please help.

Number of steps taken to split a number

I cannot get my head around this:
Say I got a number 9. I want to know the minimum steps needed to split it so that no number is greater than 3.
I always thought that the most efficient way is to halve it every loop.
So, 9 -> 4,5 -> 2,2,5 -> 2,2,2,3 so 3 steps in total. However, I just realised a smarter way: 9 -> 3,6 -> 3,3,3 which is 2 steps only...
After some research, the number of steps is in fact (n-1)/target, where target=3 in my example.
Can someone please explain this behaviour to me?
If we want to cut a stick of length L into pieces of size no greater than S, we need ceiling(L/S) pieces. Each time we make a new cut, we increase the number of pieces by 1. It doesn't matter what order we make the cuts in, only where. For example, if we want to break a stick of length 10 into pieces of size 2 or less:
-------------------
0 1 2 3 4 5 6 7 8 9 10
we should cut it in the following places:
---|---|---|---|---
0 1 2 3 4 5 6 7 8 9 10
and any order of cuts is fine, as long as these are the cuts that are made. On the other hand, if we start by breaking it in half:
---------|---------
0 1 2 3 4 5 6 7 8 9 10
we have made a cut that isn't part of the optimal solution, and we have wasted our time.
I really like #user2357112's explanation of why cutting in half is not the right first step, but I also like algebra, and you can prove that ceil(n / target) - 1 is optimal using induction.
Let's prove first that you can always do it in ceil(n / target) - 1 steps.
If n <= target, obviously no step are required, so the formula works. Suppose n > target. Split n into target and n - target (1 step). By induction, n - target can be split in ceil((n - target)/target) - 1 steps. Therefore the total number of steps is
1 + ceil((n - target) / target) - 1
= 1 + ceil(n / target) - target/target - 1
= ceil(n / target) - 1.
Now let's prove that you can't do it in fewer than ceil(n / target) - 1 steps. This is obvious if n <= target. Suppose n > target and the first step is n -> a + b. By induction, a requires at least ceil(a / target) - 1 steps and b requires at least ceil(b / target) - 1 steps. The minimum number of steps required is therefore at least
1 + ceil(a / target) - 1 + ceil(b / target) - 1
>= ceil((a + b) / target) - 1 using ceil(x) + ceil(y) >= ceil(x + y)
= ceil(n / target) - 1 using a + b = n
Every n can be thought of as a priority queue of \lfloor n/target \rfloor target elements placed first on the queue and one element whose value is n%target. Every time you remove an element from the queue, you place it back on the queue. Remove all but the last element: you have clearly removed \lfloor (n-1)/target \rfloor elements. If the last element is less than or equal to the target, we are done. If it is greater than the target, we have a contradiction. So, after \lfloor (n-1)/target \rfloor steps we have a queue consisting only of elements less than or equal to target.

Hanoi configuration at a certain move

I am interested in finding how many disks are on each peg at a given move in the towers of Hanoi puzzle. For example, given n = 3 disks we have this sequence of configurations for optimally solving the puzzle:
0 1 2
0. 3 0 0
1. 2 0 1 (move 0 -> 2)
2. 1 1 1 (move 0 -> 1)
3. 1 2 0 (move 2 -> 1)
4. 0 2 1 (move 0 -> 2)
5. 1 1 1 (move 1 -> 0)
6. 1 0 2 (move 1 -> 2)
7. 0 0 3 (move 0 -> 2)
So given move number 5, I want to return 1 1 1, given move number 6, I want 1 0 2 etc.
This can easily be done by using the classical algorithm and stopping it after a certain number of moves, but I want something more efficient. The wikipedia page I linked to above gives an algorithm under the Binary solutions section. I think this is wrong however. I also do not understand how they calculate n.
If you follow their example and convert the disk positions it returns to what I want, it gives 4 0 4 for n = 8 disks and move number 216. Using the classical algorithm however, I get 4 2 2.
There is also an efficient algorithm implemented in C here that also gives 4 2 2 as the answer, but it lacks documentation and I don't have access to the paper it's based on.
The algorithm in the previous link seems correct, but can anyone explain how exactly it works?
A few related questions that I'm also interested in:
Is the wikipedia algorithm really wrong, or am I missing something? And how do they calculate n?
I only want to know how many disks are on each peg at a certain move, not on what peg each disk is on, which is what the literature seems to be more concerned about. Is there a simpler way to solve my problem?
1) If your algo says Wikipedia is broken I'd guess you are right...
2) As for calculating the number of disks in each peg, is it pretty straightfoward to do a recursive algorithm for it:
(Untested, unelegant and possibly full of +-1 errors code follows:)
function hanoi(n, nsteps, begin, middle, end, nb, nm, ne)
// n = number of disks to mive from begin to end
// nsteps = number of steps to move
// begin, middle, end = index of the pegs
// nb, nm, ne = number of disks currently in each of the pegs
if(nsteps = 0) return (begin, middle, end, nb, nm, ne)
//else:
//hanoi goes like
// a) h(n-1, begin, end, middle) | 2^(n-1) steps
// b) move 1 from begin -> end | 1 step
// c) h(n-1, middle, begin, end) | 2^(n-1) steps
// Since we know how the pile will look like after a), b) and c)
// we can skip those steps if nsteps is large...
if(nsteps <= 2^(n-1)){
return hanoi(n-1, nsteps, begin, end, middle, nb, ne, nm):
}
nb -= n;
nm += (n-1);
ne += 1;
nsteps -= (2^(n-1) + 1);
//we are now between b) and c)
return hanoi((n-1), nsteps, middle, begin, end, nm, nb, ne);
function(h, n, nsteps)
return hanoi(n, nsteps, 1, 2, 3, n, 0, 0)
If you want effieciency, it should try to convert this to an iterative form (shouldn't be hard - you don't need to mantain a stack anyways) and find a way to better represent the state of the program, instead of using 6+ variables willy nilly.
You can make use of the fact that the position at powers of two is easily known. For a tower of size T, we have:
Time Heights
2^T-1 | { 0, 0, T }
2^(T-1) | { 0, T-1, 1 }
2^(T-1)-1 | { 1, T-1, 0 }
2^(T-2) | { 1, 1, T-2 }
2^(T-2)-1 | { 2, 0, T-2 }
2^(T-2) | { 2, T-3, 1 }
2^(T-2)-1 | { 3, T-3, 0 }
...
0 | { T, 0, 0 }
It is easy to find out in between which those levels your move k is; simply look at log2(k).
Next, notice that between 2^(a-1) and 2^a-1, there are T-a disks which stay in the same place (the heaviest disks). All the other blocks will move however, since at this stage the algorithm is moving the subtower of size a. Hence use an iterative approach.
It might be a bit tricky to get the book-keeping right, but here you have the ingredients to find the heights for any k, with time complexity O(log2(T)).
Cheers
If you look at the first few moves of the puzzle, you'll see an important pattern. Each move (i - j) below means on turn i, move disc j. Discs are 0-indexed, where 0 is the smallest disc.
1 - 0
2 - 1
3 - 0
4 - 2
5 - 0
6 - 1
7 - 0
8 - 3
9 - 0
10 - 1
11 - 0
12 - 2
13 - 0
14 - 1
15 - 0
Disc 0 is moved every 2 turns, starting on turn 1. Disc 1 is moved every 4 turns, starting on turn 2 ... Disc i is moved every 2^(i+1) turns, starting on turn 2^i.
So, in constant time we can determine how many times a given disc has moved, given m:
moves = (m + 2^i) / (2^(i+1)) [integer division]
The next thing to note is that each disc moves in a cyclic pattern. Namely, the odd-numbered discs move to the left each time they move (2, 3, 1, 2, 3, 1...) and the even-numbered discs move to the right (1, 3, 2, 1, 3, 2...)
So once you know how many times a disc has moved, you can easily determine which peg it ends on by taking mod 3 (and doing a little bit of figuring).

Resources