How to check if this algorithm may not terminate? - algorithm

Question

Let x denote a vector of p values (i.e. a data point in p dimensional space).
I have two sets: set A of n elements A = {xi, .., xn} and a set B of m elements B = {xj, .., xm}, where |A| > 1 and |B| > 1. Given an integer k > 0, let dist(x, k, A) a function which returns the mean Euclidean distance from x to its k nearest points in A; and dist(x, k, B) the mean Euclidean distance from x to its k nearest points in B.
I have the following algorithm:
Repeat
{
A' = { x in A, such that dist(x, k, A) > dist(x, k, B) }
B' = { x in B, such that dist(x, k, A) < dist(x, k, B) }
A = { x in A such that x not in A' } U B'
B = { x in B such that x not in B' } U A'
}
Until CONDITION == True
Termination: CONDITION is True when no more elements move from A to B or from B to A (that is A' and B' becomes empty), or when |A| or |B| becomes less than or equals to 1.
1) Is it possible to prove that this algorithm terminates ?
2) And if so, is it also possible to have an upper bound for the number of iterations required to terminate ?
Note: the k nearest points to x in a set S, means: the k points (others than x) in S, having the smallest Euclidean distance to x.

Answer 1

It looks like this algorithm can loop forever, oscillating between two or more states. I determined this experimentally using the following Python program:
def mean(seq):
if len(seq) == 0:
raise IndexError("didn't expect empty sequence for mean")
return sum(seq) / float(len(seq))
def dist(a,b):
return abs(a-b)
def mean_dist(x, k, a):
neighbors = {p for p in a if p != x}
neighbors = sorted(neighbors, key=lambda p: dist(p,x))
return mean([dist(x, p) for p in neighbors[:k]])
def frob(a,b,k, verbose = False):
def show(msg):
if verbose:
print msg
seen_pairs = set()
iterations = 0
while True:
iterations += 1
show("Iteration #{}".format(iterations))
a_star = {x for x in a if mean_dist(x, k, a) > mean_dist(x,k,b)}
b_star = {x for x in b if mean_dist(x, k, a) < mean_dist(x,k,b)}
a_temp = {x for x in a if x not in a_star} | b_star
b_temp = {x for x in b if x not in b_star} | a_star
show("\tA`: {}".format(list(a_star)))
show("\tB`: {}".format(list(b_star)))
show("\tA becomes {}".format(list(a_temp)))
show("\tB becomes {}".format(list(b_temp)))
if a_temp == a and b_temp == b:
return a, b
key = (tuple(sorted(a_temp)), tuple(sorted(b_temp)))
if key in seen_pairs:
raise Exception("Infinite loop for values {} and {}".format(list(a_temp),list(b_temp)))
seen_pairs.add(key)
a = a_temp
b = b_temp
import random
#creates a set of random integers, with the given number of elements.
def randSet(size):
a = set()
while len(a) < size:
a.add(random.randint(0, 10))
return a
size = 2
k = 1
#p equals one because I don't feel like doing vector math today
while True:
a = randSet(size)
b = randSet(size)
try:
frob(a,b, k)
except IndexError as e:
continue
except Exception as e:
print "infinite loop detected for initial inputs {} and {}".format(list(a), list(b))
#run the algorithm again, but showing our work this time
try:
frob(a,b,k, True)
except:
pass
break
Result:
infinite loop detected for initial inputs [10, 4] and [1, 5]
Iteration #1
A`: [10, 4]
B`: [1, 5]
A becomes [1, 5]
B becomes [10, 4]
Iteration #2
A`: [1, 5]
B`: [10, 4]
A becomes [10, 4]
B becomes [1, 5]
Iteration #3
A`: [10, 4]
B`: [1, 5]
A becomes [1, 5]
B becomes [10, 4]
In this case, the loop never terminates because A and B continually switch entirely. While experimenting with larger set sizes, I found a case where only some elements switch:
infinite loop detected for initial inputs [8, 1, 0] and [9, 4, 5]
Iteration #1
A`: [8]
B`: [9]
A becomes [0, 1, 9]
B becomes [8, 4, 5]
Iteration #2
A`: [9]
B`: [8]
A becomes [0, 1, 8]
B becomes [9, 4, 5]
Iteration #3
A`: [8]
B`: [9]
A becomes [0, 1, 9]
B becomes [8, 4, 5]
Here, elements 8 and 9 move back and forth while the other elements stay in place.