Given a set of n items, I'd like to generate all the ways we can select from this set (with replacement) k times such that order matters and each element is used as least once. So k>=n to have any valid arrangements. If k=n, this is just a permutation. So k>n is kind of like an extension of a permutation, but I do not know what it is called.
It's of course easy to get an algorithm, albeit a horribly slow one: just iterate through all possible selections and toss the ones that don't have each element at least once. So to make something efficient will require tricks similar to iterating through permutations, or break it into sub-problems where we can use existing permutation algorithms directly.
I tried to break this into a permutation and combination problem by doing something like the following using python to play with some ideas.
import itertools
def func(inputSet,k):
n = len(inputSet)
assert(k>n)
# first, guarantee we have each element once
for p in itertools.permutations(inputSet):
# now select (k-n) locations to insert other elements
for c in itertools.combinations_with_replacement(range(n+1),k-n):
insertions = [c[i]+i for i in range(len(c))]
out = list(p)
for index in insertions:
out.insert(index,0)
# now select values to put in those locations
for vals in itertools.product(inputSet,repeat=len(insertions)):
for i in xrange(len(insertions)):
out[c[i]] = vals[i]
yield tuple(out)
But different insertions can yield the same result, so this first stab is likely not starting down the correct path. I could add conditionals to check for these cases and filter out some results, but an algorithm for a combinatoric iteration problem that resorts to filtering is likely not the most efficient algorithm.
Does this "permutation extension" have a name?
What is an efficient algorithm for iterating through the arrangements?
For what it's called, this is the same as a permutation, just framed a little differently. Consider the set of elements P, you're essentially asking to generate all of the permutations in the set P unioned with (k-n) elements of P, which can be found with itertools.combinations_with_replacement.
To generate the actual permutations, you can then either use list(set(itertools.permutations)) or more_itertools.distinct_permutations: https://more-itertools.readthedocs.io/en/latest/api.html#more_itertools.distinct_permutations
Putting this into actual code
>>> x = [1,2,3]
>>> k = 5
>>> results = set()
>>> for y in itertools.combinations_with_replacement(x, k - len(x)):
... for z in itertools.permutations(x + list(y)):
... results.add(z)
...
>>> results
set([(1, 1, 1, 2, 3), (1, 3, 3, 1, 2), (1, 2, 3, 3, 2), (3, 3, 2, 1, 3), (1, 3, 3, 2, 2), (1, 1, 2, 2, 3), (3, 1, 2, 3, 2), (1, 1, 3, 2, 3), (1, 3, 1, 2, 2), (1, 2, 2, 1, 3), (3, 2, 1, 2, 2), (3, 1, 2, 1, 2), (3, 2, 1, 3, 1), (3, 1, 1, 3, 2), (2, 3, 3, 2, 1), (1, 2, 1, 3, 3), (3, 1, 2, 3, 3), (2, 3, 2, 1, 1), (2, 3, 2, 2, 1), (1, 2, 2, 3, 3), (2, 1, 3, 2, 3), (2, 2, 2, 3, 1), (1, 3, 2, 2, 3), (2, 3, 3, 1, 1), (1, 2, 1, 1, 3), (3, 2, 2, 1, 1), (2, 1, 2, 2, 3), (2, 2, 1, 3, 1), (1, 3, 3, 2, 3), (2, 3, 3, 1, 3), (3, 2, 3, 2, 1), (3, 2, 3, 1, 1), (2, 1, 1, 2, 3), (3, 3, 1, 1, 2), (3, 2, 2, 2, 1), (2, 2, 3, 2, 1), (1, 3, 1, 2, 3), (2, 2, 3, 1, 2), (3, 1, 2, 1, 3), (2, 1, 1, 3, 3), (3, 3, 2, 2, 1), (1, 1, 2, 3, 1), (1, 2, 2, 3, 2), (3, 2, 1, 1, 2), (2, 1, 3, 2, 2), (1, 3, 2, 2, 2), (3, 1, 1, 1, 2), (3, 3, 2, 1, 1), (2, 3, 3, 1, 2), (3, 2, 1, 3, 2), (1, 2, 1, 3, 1), (2, 3, 1, 2, 3), (1, 2, 3, 2, 1), (3, 1, 3, 1, 2), (3, 3, 1, 2, 2), (1, 2, 3, 1, 1), (2, 2, 1, 1, 3), (2, 1, 1, 3, 2), (1, 1, 2, 3, 2), (3, 2, 1, 1, 3), (2, 1, 3, 2, 1), (2, 1, 1, 3, 1), (1, 3, 2, 2, 1), (1, 3, 2, 1, 1), (3, 2, 2, 1, 3), (2, 2, 3, 3, 1), (3, 1, 1, 2, 1), (2, 2, 1, 3, 3), (1, 3, 3, 2, 1), (3, 2, 3, 1, 2), (3, 1, 2, 3, 1), (2, 2, 2, 1, 3), (1, 1, 3, 1, 2), (1, 1, 2, 1, 3), (2, 1, 3, 3, 2), (3, 3, 1, 2, 3), (1, 3, 2, 3, 2), (3, 3, 2, 3, 1), (3, 1, 3, 2, 2), (2, 1, 3, 1, 1), (1, 1, 2, 3, 3), (2, 1, 3, 1, 2), (1, 3, 2, 1, 2), (1, 2, 1, 2, 3), (3, 2, 2, 1, 2), (3, 1, 1, 2, 2), (2, 2, 1, 3, 2), (2, 3, 2, 3, 1), (1, 1, 1, 3, 2), (2, 3, 1, 2, 1), (1, 2, 3, 1, 3), (2, 3, 1, 1, 1), (1, 2, 3, 2, 3), (2, 1, 2, 3, 3), (3, 2, 1, 2, 3), (1, 2, 2, 2, 3), (3, 2, 1, 2, 1), (2, 1, 1, 1, 3), (1, 3, 2, 3, 3), (1, 1, 3, 3, 2), (3, 2, 2, 3, 1), (3, 1, 3, 2, 3), (2, 1, 2, 1, 3), (1, 3, 3, 3, 2), (3, 2, 3, 3, 1), (2, 2, 3, 1, 3), (3, 2, 1, 1, 1), (2, 1, 3, 1, 3), (1, 2, 1, 3, 2), (3, 3, 1, 3, 2), (1, 3, 2, 1, 3), (2, 3, 1, 3, 2), (3, 1, 1, 2, 3), (2, 3, 3, 3, 1), (1, 1, 3, 2, 1), (2, 3, 1, 1, 2), (1, 2, 3, 2, 2), (2, 1, 2, 3, 2), (1, 3, 1, 1, 2), (3, 1, 3, 3, 2), (3, 1, 2, 2, 2), (3, 3, 1, 2, 1), (3, 3, 3, 1, 2), (3, 2, 3, 1, 3), (1, 3, 1, 3, 2), (2, 3, 2, 1, 3), (2, 1, 3, 3, 3), (1, 2, 2, 3, 1), (2, 3, 1, 3, 3), (1, 2, 3, 3, 1), (2, 3, 1, 2, 2), (3, 3, 2, 1, 2), (2, 3, 1, 3, 1), (3, 2, 1, 3, 3), (1, 1, 3, 2, 2), (2, 3, 1, 1, 3), (2, 1, 2, 3, 1), (1, 2, 3, 3, 3), (1, 3, 1, 2, 1), (3, 1, 2, 2, 3), (3, 1, 2, 2, 1), (2, 1, 3, 3, 1), (3, 1, 2, 1, 1), (1, 3, 2, 3, 1), (1, 2, 3, 1, 2), (3, 1, 3, 2, 1), (2, 2, 1, 2, 3), (2, 3, 2, 1, 2), (3, 3, 3, 2, 1), (2, 2, 3, 1, 1)])
Be aware that this explodes pretty quickly combinatorially, however because both itertools.combinations_with_replacement and itertools.permutations return generators, you can also yield the results. You could also write it yourself recursively, however I personally think that's a lot less satisfying.
I believe it's also sufficient to use distinct_permutations here and you'll end up with a list of wholly distinct results, as each iteration of the outer loop results in a different frequency signature for the elements.
Here's an additional solution, which also iterates in lexigraphical order.
import itertools
def _recurse_extended_perm(remaining,inputList,requireList):
if remaining==len(requireList):
for p in itertools.permutations(requireList):
yield p
elif remaining==1:
for x in inputList:
yield [x]
else:
for x in inputList:
req = list(requireList)
if x in req:
req.remove(x)
for seq in _recurse_extended_perm(remaining-1,inputList,req):
out = [x]
out.extend(seq)
yield out
def extended_permutation(seq,k):
inputList = list(seq)
inputList.sort()
assert(k>len(inputList))
for seq in _recurse_extended_perm(k,inputList,inputList):
yield tuple(seq)
We can generate the list recursively. Here's a module-less implementation that also gives us a little flexibility:
def multisets_with_at_least_m(inputList, k, m):
def f(idx, n, multiset):
if idx == len(inputList):
return [multiset]
if (len(inputList) - idx) * m > n:
return []
minNum = n if idx == len(inputList) - 1 else m
maxNum = n - (len(inputList) - idx) * m + m
results = []
for i in xrange(minNum, maxNum + 1):
multisetCopy = multiset.copy()
multisetCopy[inputList[idx]] = i
results = results + f(idx + 1, n - i, multisetCopy)
return results
return f(0, k, {})
def distinct_permutations_from_multiset(multiset):
def f(multiset, permutation):
if multiset == {}:
return [permutation]
results = []
for k in multiset:
multisetCopy = multiset.copy()
if multiset[k] == 1:
del multisetCopy[k]
else:
multisetCopy[k] = multisetCopy[k] - 1
results = results + f(multisetCopy, permutation + [k])
return results
return f(multiset, [])
Output:
print [distinct_permutations_from_multiset(x) for x in multisets_with_at_least_m([1,2,3], 5, 1)]
"""
[[[1, 2, 3, 3, 3], [1, 3, 2, 3, 3], [1, 3, 3, 2, 3], [1, 3, 3, 3, 2]
, [2, 1, 3, 3, 3], [2, 3, 1, 3, 3], [2, 3, 3, 1, 3], [2, 3, 3, 3, 1]
, [3, 1, 2, 3, 3], [3, 1, 3, 2, 3], [3, 1, 3, 3, 2], [3, 2, 1, 3, 3]
, [3, 2, 3, 1, 3], [3, 2, 3, 3, 1], [3, 3, 1, 2, 3], [3, 3, 1, 3, 2]
, [3, 3, 2, 1, 3], [3, 3, 2, 3, 1], [3, 3, 3, 1, 2], [3, 3, 3, 2, 1]]
, [[1, 2, 2, 3, 3], [1, 2, 3, 2, 3], [1, 2, 3, 3, 2], [1, 3, 2, 2, 3]
, [1, 3, 2, 3, 2], [1, 3, 3, 2, 2], [2, 1, 2, 3, 3], [2, 1, 3, 2, 3]
, [2, 1, 3, 3, 2], [2, 2, 1, 3, 3], [2, 2, 3, 1, 3], [2, 2, 3, 3, 1]
, [2, 3, 1, 2, 3], [2, 3, 1, 3, 2], [2, 3, 2, 1, 3], [2, 3, 2, 3, 1]
, [2, 3, 3, 1, 2], [2, 3, 3, 2, 1], [3, 1, 2, 2, 3], [3, 1, 2, 3, 2]
, [3, 1, 3, 2, 2], [3, 2, 1, 2, 3], [3, 2, 1, 3, 2], [3, 2, 2, 1, 3]
, [3, 2, 2, 3, 1], [3, 2, 3, 1, 2], [3, 2, 3, 2, 1], [3, 3, 1, 2, 2]
, [3, 3, 2, 1, 2], [3, 3, 2, 2, 1]]
, [[1, 2, 2, 2, 3], [1, 2, 2, 3, 2], [1, 2, 3, 2, 2], [1, 3, 2, 2, 2]
, [2, 1, 2, 2, 3], [2, 1, 2, 3, 2], [2, 1, 3, 2, 2], [2, 2, 1, 2, 3]
, [2, 2, 1, 3, 2], [2, 2, 2, 1, 3], [2, 2, 2, 3, 1], [2, 2, 3, 1, 2]
, [2, 2, 3, 2, 1], [2, 3, 1, 2, 2], [2, 3, 2, 1, 2], [2, 3, 2, 2, 1]
, [3, 1, 2, 2, 2], [3, 2, 1, 2, 2], [3, 2, 2, 1, 2], [3, 2, 2, 2, 1]]
, [[1, 1, 2, 3, 3], [1, 1, 3, 2, 3], [1, 1, 3, 3, 2], [1, 2, 1, 3, 3]
, [1, 2, 3, 1, 3], [1, 2, 3, 3, 1], [1, 3, 1, 2, 3], [1, 3, 1, 3, 2]
, [1, 3, 2, 1, 3], [1, 3, 2, 3, 1], [1, 3, 3, 1, 2], [1, 3, 3, 2, 1]
, [2, 1, 1, 3, 3], [2, 1, 3, 1, 3], [2, 1, 3, 3, 1], [2, 3, 1, 1, 3]
, [2, 3, 1, 3, 1], [2, 3, 3, 1, 1], [3, 1, 1, 2, 3], [3, 1, 1, 3, 2]
, [3, 1, 2, 1, 3], [3, 1, 2, 3, 1], [3, 1, 3, 1, 2], [3, 1, 3, 2, 1]
, [3, 2, 1, 1, 3], [3, 2, 1, 3, 1], [3, 2, 3, 1, 1], [3, 3, 1, 1, 2]
, [3, 3, 1, 2, 1], [3, 3, 2, 1, 1]]
, [[1, 1, 2, 2, 3], [1, 1, 2, 3, 2], [1, 1, 3, 2, 2], [1, 2, 1, 2, 3]
, [1, 2, 1, 3, 2], [1, 2, 2, 1, 3], [1, 2, 2, 3, 1], [1, 2, 3, 1, 2]
, [1, 2, 3, 2, 1], [1, 3, 1, 2, 2], [1, 3, 2, 1, 2], [1, 3, 2, 2, 1]
, [2, 1, 1, 2, 3], [2, 1, 1, 3, 2], [2, 1, 2, 1, 3], [2, 1, 2, 3, 1]
, [2, 1, 3, 1, 2], [2, 1, 3, 2, 1], [2, 2, 1, 1, 3], [2, 2, 1, 3, 1]
, [2, 2, 3, 1, 1], [2, 3, 1, 1, 2], [2, 3, 1, 2, 1], [2, 3, 2, 1, 1]
, [3, 1, 1, 2, 2], [3, 1, 2, 1, 2], [3, 1, 2, 2, 1], [3, 2, 1, 1, 2]
, [3, 2, 1, 2, 1], [3, 2, 2, 1, 1]]
, [[1, 1, 1, 2, 3], [1, 1, 1, 3, 2], [1, 1, 2, 1, 3], [1, 1, 2, 3, 1]
, [1, 1, 3, 1, 2], [1, 1, 3, 2, 1], [1, 2, 1, 1, 3], [1, 2, 1, 3, 1]
, [1, 2, 3, 1, 1], [1, 3, 1, 1, 2], [1, 3, 1, 2, 1], [1, 3, 2, 1, 1]
, [2, 1, 1, 1, 3], [2, 1, 1, 3, 1], [2, 1, 3, 1, 1], [2, 3, 1, 1, 1]
, [3, 1, 1, 1, 2], [3, 1, 1, 2, 1], [3, 1, 2, 1, 1], [3, 2, 1, 1, 1]]]
"""