How can I write this combinatorics algorithm more efficiently? - algorithm

Question

A group contains a set of entities and each entity has a value.
Each entity can be a part of more than one group.
Problem: Find largest N groups where each entity appears no more than once in the result. An entity can be excluded from a group if necessary.
Example:
Entities with values:
A = 2
B = 2
C = 2
D = 3
E = 3
Groups
1: (A,B,C) total value: 2+2+2 = 6
2: (B,D) total value: 2 + 3 = 5
3: (C,E) total value: 2 + 3 = 5
4: (D) total value: 3
5: (E) total value: 3
**Answers**:
Largest 1 group is obviously (A,B,C) with total value 6
Largest 2 groups are (B,D), (C,E) with total value 10
Largest 3 groups are either {(A,B,C),(D),(E)}, {(A,B),(C,E),(D)} or {(A,C), (B,D), (E)} with total value 12
The input data to the algorithm should be:
A set of entities with values
Groups containing one or more of the entities
The amount of groups in the result
If there are multiple answers then finding one of them is sufficient.
I included the example to try to make the problem clear, the amount of entities in practise should be less than about 50, and amount of groups should be less than the amount of entities. The amount of N groups to find will be between 1 and 10.
I am currently solving this problem by generating all possible combinations of N groups, excluding the results that contains duplicate entities and then picking the combination with largest total value. This is of course extremely inefficient but i cant get my head around how to obtain a general result in a more efficient way.
My question is if it's possible to solve this in a more efficient way, and if so, how? Any hints or answers are greatly appreciated.
edit
To be clear, in my solution i generate "fake" groups where duplicate entities are excluded from "real" groups. In the example entities (B, C, D, E) are duplicates (exist in more than one group. Then for group 1 (A,B,C) i add the fake groups (A,B),(A,C),(A) to the list of groups that I generate combinations for.

Answer 1

This problem can be formulated as a linear integer program. Although the integer programming is not super efficient in terms of complexity, it works very quick with this number of variables.
Here is how we turn this problem into an integer program.
Let v be a vector of size K representing the entity values.
Let G be a K x M binary matrix that defines the groups: G(i,j)=1 means that the entity i belongs to the group j and G(i,j)=0 otherwise.
Let x be a binary vector of size M, which represents the choice of groups: x[j]=1 indicates we pick the group j.
Let y be a binary vector of size K, which represents the inclusion of entities: y[i]=1 means that the entity i is included in the outcome.
Our goal is to choose x and y so as to maximize sum(v*y) under the following conditions:
G x >= y ... all included entities must belong to at least one of chosen groups
sum(x) = N ... we choose exactly N groups.
Below is an implementation in R. It uses lpSolve library, an interface to lpsolve.
library(lpSolve)
solver <- function(values, groups, N)
{
n_group <- ncol(groups)
n_entity <- length(values)
object <- c(rep(0, n_group), values)
lhs1 <- cbind(groups, -diag(n_entity))
rhs1 <- rep(0, n_entity)
dir1 <- rep(">=", n_entity)
lhs2 <- matrix(c(rep(1, n_group), rep(0, n_entity)), nrow=1)
rhs2 <- N
dir2 <- "="
lhs <- rbind(lhs1, lhs2)
rhs <- c(rhs1, rhs2)
direc <- c(dir1, dir2)
lp("max", object, lhs, direc, rhs, all.bin=TRUE)
}
values <- c(A=2, B=2, C=2, D=3, E=3)
groups <- matrix(c(1,1,1,0,0,
0,1,0,1,0,
0,0,1,0,1,
0,0,0,1,0,
0,0,0,0,1),
nrow=5, ncol=5)
rownames(groups) <- c("A", "B", "C", "D", "E")
ans <- solver(values, groups, 1)
print(ans)
names(values)[tail(ans$solution, length(values))==1]
# Success: the objective function is 6
# [1] "A" "B" "C"
ans <- solver(values, groups, 2)
print(ans)
names(values)[tail(ans$solution, length(values))==1]
# Success: the objective function is 10
# [1] "B" "C" "D" "E"
ans <- solver(values, groups, 3)
print(ans)
names(values)[tail(ans$solution, length(values))==1]
# Success: the objective function is 12
# [1] "A" "B" "C" "D" "E"
Below is to see how this can work with large problem. It finishes in one second.
# how does it scale?
n_entity <- 50
n_group <- 50
N <- 10
entity_names <- paste("X", 1:n_entity, sep="")
values <- sample(1:10, n_entity, replace=TRUE)
names(values) <- entity_names
groups <- matrix(sample(c(0,1), n_entity*n_group,
replace=TRUE, prob=c(0.99, 0.01)),
nrow=n_entity, ncol=n_group)
rownames(groups) <- entity_names
ans <- solver(values, groups, N)
print(ans)
names(values)[tail(ans$solution, length(values))==1]

Answer 2

if the entity values are always positive, I think you can get a solution without generating all combinations:
sort the groups by their largest element, 2nd largest element, nth largest element. in this case you would have 3 copies since the largest group has 3 elements.
for each copy, make one pass from the largest to the smallest adding the group to the solution only if it doesn't contain an element you've already added. this yields 3 results, take the largest. there shouldn't be a larger possible solution unless weights could be negative.
here's an implementation in C#
var entities = new Dictionary<char, int>() { { 'A', 2 }, { 'B', 2 }, { 'C', 2 }, { 'D', 3 }, { 'E', 3 } };
var groups = new List<string>() { "ABC", "BD", "CE", "D", "E" };
var solutions = new List<Tuple<List<string>, int>>();
for(int i = 0; i < groups.Max(x => x.Length); i++)
{
var solution = new List<string>();
foreach (var group in groups.OrderByDescending(x => x.Length > i ? entities[x[i]] : -1))
if (!group.ToCharArray().Any(c => solution.Any(g => g.Contains(c))))
solution.Add(group);
solutions.Add(new Tuple<List<string>, int>(solution, solution.Sum(g => g.ToCharArray().Sum(c => entities[c]))));
}
solutions.Dump();
solutions.OrderByDescending(x => x.Item2).First().Dump();
output: