Currently I'm working on a 1 day tournament scheduling application. Since, each year the number of participating teams is different, I want to automate the scheduling.
Teams are split in 2 groups. each group plays a single round robin tournament.
I managed to generate all the games to play, but I'm struggling with the planning.
additionally, the teams need to compete in 3 different sport disciplines, with each a dedicated field. (e.g. football field, volleybal field)
Given:
- games to play
- fields per sport + available timeslots per field (slots of +-15 minutes)
assumptions:
- timeslots are not limited
- 1 field per sport available
- schedule doesn't need to be balanced in 1st iteration
problems:
- the quality of my schedule is not that good. in fact, not all timeslots are fully filled, even if there is a solution. the 'density' of my schedule also depends on the order of games processed.
code snippet:
//algo
while (_games.Any())
{
gameToPlan = _games.Dequeue();
var occupiedHomeTeam = GetTimeslotsOccupiedByTeam(gameToPlan.HomeTeam);
var occupiedAwayTeam = GetTimeslotsOccupiedByTeam(gameToPlan.AwayTeam);
var occupiedTeams = occupiedHomeTeam.Union(occupiedAwayTeam);
var availableFields = fields.Where(f => f.AllowedSports.Contains(gameToPlan.Sport))
.Where(f => f.Timeslots.Any(t => t.Game == null &&
!t.Occupied &&
!occupiedTeams.Any(oc => oc.Start == t.Start &&
oc.End == t.End)));
if (!availableFields.Any())
{
_games.Enqueue(gameToPlan);
continue;
}
var field = availableFields.First();
var timeSlots = field.Timeslots.Where(t => t.Game == null &&
!t.Occupied &&
!occupiedTeams.Any(oc => oc.Start == t.Start &&
oc.End == t.End))
.OrderBy(t => t.Start);
if (!timeSlots.Any())
{
_games.Enqueue(gameToPlan);
continue;
}
var ts = timeSlots.First();
ts.Occupied = true;
ts.Game = gameToPlan;
gameToPlan.Timeslot = ts;
gameToPlan.TimeslotId = ts.Id;
_uow.Save();
}
Can anyone give me an overview of approach, available algorithms,...?
thanks in advance
Regarding your problem, this is clearly a discrete optimization problem. For tournament/timetable problems, you should think about using constraint programming solvers. You need to be familiar with linear/integer programming to do so. For example you can use Choco solver which is in Java. Fun fact is that the last question on their forum is related to tournament scheduling.
Related
In our Xamarin Forms app this code:
return database.GetAllWithChildren<Review>(x => x.ProductId == prodId, true).OrderByDescending(x => x.ReviewId).FirstOrDefault();
is having performance issues and it takes more time the more reviews exist for that product.
With a product with 7 reviews it took about 17 seconds which is unacceptable.
How could I optimize the performance?
After all I don't need all the reviews info, just the latest one.
It seems that this retrieves all 7 reviews and then sorts them descending and then gets the first in the list.
Is there a way to get only that one with the max ID?
It seems using the GetAllWithChildren is the problem. This method recursively loads the info of all the reviews, not just the one. You are then performing the ordering and selection after all the data returned from the DB.
The solution would be to first filter and only then return the Review:
var review = database.Table<Review>().Where(x => x.Product == prodId ).
OrderByDescending( x => x.ReviewId ).FirstOrDefault();
database.GetChildren( review, true );
return element;
database.GetAllWithChildren<Review> returns a List<Review> of fully constructed reviews. This means that the more reviews a product has, the more reviews get discarded by FirstOrDefault() right after being constructed.
Try getting the max id first, then filter reviews on it:
// This assumes that ID is int. Change to another nullable type matching ReviewId
int? maxId = database.Reviews.Where(r => r.ProductId == prodId).Max((int?)r.ReviewId);
return database
.GetAllWithChildren<Review>(x => x.ProductId == prodId && x.ReviewId == maxId, true)
.SingleOrDefault();
I'm implementing a social chess game. Every user can create a new game, and they'll wait until the system will find an opponent for them.
When user creates a game, they specify constraints: color they'd like to play, and opponent's minimal chess rating.
Opponents can either match or not match. For example, the following two opponents will match:
// User 1 with rating 1700 // User 2 with rating 1800
// creates this game // creates this game
game: { game: {
color: 'white', minRating: 1650
minRating: 1600 }
} // User did not specify a preferred color,
// meaning they do not care which color to play
So, if User 1 is the first user in the system, and created their game, they'll wait. Once User 2 creates their game, they should be matched immediately with User 1.
On the other side, the following two opponents won't match, because they both want to play white. In this case, both should wait until someone else creates a game with color: 'black' (or color not specified), and minRating that would match the requirements.
// User 1 with rating 1700 // User 2 with rating 1800
// creates this game // creates this game
game: { game: {
color: 'white', color: 'white'
minRating: 1600 minRating: 1650
} }
My concerns related to scenarios where thousands of users creates new games at the same time. How do I make sure that I match opponents without creating deadlocks? i.e. how do I prevent scenarios when User 1, User 2, and User 3 are trying to find an opponent at the same time, and their matching algorithms return User 99. How do I recover from this scenario, assigning User 99 to only one of them?
How would you use the power of Firebase to implement such a matching system?
The obvious choice for a starting point would be the color, since this is an exclusive requirement. The others seem more like weighted results, so those could simply increment or decrement the weight.
Utilize priorities for min/max ranges, and keep each in a separate "index". Then grab the matches for each and create a union. Consider this structure:
/matches
/matches/colors/white/$user_id
/matches/ranking/$user_id (with a priority equal to ranking)
/matches/timezones/$user_id (with a priority of the GMT relationship)
Now to query, I would simply grab the matches in each category and rank them by the number of matches. I can start with colors, because this presumably isn't an optional or relative rating:
var rootRef = new Firebase('.../matches');
var VALUE = {
"rank": 10, "timezone": 5, "color": 0
}
var matches = []; // a list of ids sorted by weight
var weights = {}; // an index of ids to weights
var colorRef = rootRef.child('colors/black');
colorRef.on('child_added', addMatch);
colorRef.child('colors/black').on('child_removed', removeMatch);
var rankRef = rootRef.child('ranking').startAt(minRank).endAt(maxRank);
rankRef.on('child_added', addWeight.bind(null, VALUE['rank']));
rankRef.on('child_removed', removeWeight.bind(null, VALUE['rank']));
var tzRef = ref.child('timezone').startAt(minTz).endAt(maxTz);
tzRef.on('child_added', addWeight.bind(null, VALUE['timezone']));
tzRef.on('child_removed', removeWeight.bind(null, VALUE['timezone']));
function addMatch(snap) {
var key = snap.name();
weights[key] = VALUE['color'];
matches.push(key);
matches.sort(sortcmp);
}
function removeMatch(snap) {
var key = snap.name();
var i = matches.indexOf(key);
if( i > -1 ) { matches.splice(i, 1); }
delete weights[key];
}
function addWeight(amt, snap) {
var key = snap.name();
if( weights.hasOwnProperty(key) ) {
weights[key] += amt;
matches.sort(sortcmp);
}
}
function removeWeight(amt, snap) {
var key = snap.name();
if( weights.hasOwnProperty(key) ) {
weights[key] -= amt;
matches.sort(sortcmp);
}
}
function sortcmp(a,b) {
var x = weights[a];
var y = weights[b];
if( x === y ) { return 0; }
return x > y? 1 : -1;
}
Okay, now I've given what everyone asks for in this use case--how to create a rudimentary where clause. However, the appropriate answer here is that searches should be performed by a search engine. This is no simple where condition. This is a weighted search for the best matches, because fields like color are not optional or simply the best match, while others--ranking maybe--are the closest match in either direction, while some simply affect the quality of the match.
Check out flashlight for a simple ElasticSearch integration. With this approach, you should be able to take advantage of ES's great weighting tools, dynamic sorting, and everything else you need to conduct a proper matching algorithm.
Regarding deadlocks. I would not put too much focus here until you have hundreds of transactions per second (i.e. hundreds of thousands of users competing for matches). Split out the path where we will write to accept a join and do a transaction to ensure only one person succeeds in obtaining it. Keep it separate from the read data so that the lock on that path won't slow down processing. Keep the transaction to a minimal size (a single field if possible).
It is a challenging task in NoSQL environment especially if you want to match multiple fields
in your case, I would setup a simple index by color and within the color I would store the reference to the game with priority set to minRating.
That way you can query the games by the prefered colour with the priority of minRating.
indexes: {
color:{
white:{
REF_WITH_PRIORITY_TO_RATING: true
},
black:{
REF_WITH_PRIORITY_TO_RATING: true
}
}
}
if you want to get info whenever the match opens the game:
ref = new(Firebase)('URL');
query =ref.child('color_index/white/').startAt(minPriority);
query.on('child_added',function(snapshot){
//here is your new game matching the filter
});
This, however, it would get more complex if you introduce multiple fields for filtering the games for example dropRate, timeZone, 'gamesPlayed' etc... In this case, you can nest the indexes deeper:
indexes: {
GMT0: {
color:{
white:{
REF_WITH_PRIORITY_TO_RATING: true
},
black:{
REF_WITH_PRIORITY_TO_RATING: true
},
}
GMT1: {
// etc
}
}
I have the following LINQ query using EF5 and generic repository, unit of work patterns to a SQL Server 2008 db
var countriesArr = GetIdsFromDelimStr(countries);
var competitionsArr = GetIdsFromDelimStr(competitions);
var filterTeamName = string.Empty;
if (teamName != null)
{
filterTeamName = teamName.ToUpper();
}
using (var unitOfWork = new FootballUnitOfWork(ConnFooty))
{
// give us our selection of teams
var teams =
(from team in
unitOfWork.TeamRepository.Find()
where ((string.IsNullOrEmpty(filterTeamName) || team.Name.ToUpper().Contains(filterTeamName)) &&
(countriesArr.Contains(team.Venue.Country.Id) || countriesArr.Count() == 0))
select new
{
tId = team.Id
}).Distinct();
// give us our selection of contests
var conts = (
from cont in
unitOfWork.ContestRepository.Find(
c =>
((c.ContestType == ContestType.League && competitionsArr.Count() == 0) ||
(competitionsArr.Contains(c.Competition.Id) && competitionsArr.Count() == 0)))
select new
{
contId = cont.Id
}
).Distinct();
// get selection of home teams based on contest
var homecomps = (from fixt in unitOfWork.FixtureDetailsRepository.Find()
where
teams.Any(t => t.tId == fixt.HomeTeam.Id) &&
conts.Any(c => c.contId == fixt.Contest.Id)
select new
{
teamId = fixt.HomeTeam.Id,
teamName = fixt.HomeTeam.Name,
countryId = fixt.HomeTeam.Venue.Country.Id != null ? fixt.HomeTeam.Venue.Country.Id : 0,
countryName = fixt.HomeTeam.Venue.Country.Id != null ? fixt.HomeTeam.Venue.Country.Name : string.Empty,
compId = fixt.Contest.Competition.Id,
compDesc = fixt.Contest.Competition.Description
}).Distinct();
// get selection of away teams based on contest
var awaycomps = (from fixt in unitOfWork.FixtureDetailsRepository.Find()
where
teams.Any(t => t.tId == fixt.AwayTeam.Id) &&
conts.Any(c => c.contId == fixt.Contest.Id)
select new
{
teamId = fixt.AwayTeam.Id,
teamName = fixt.AwayTeam.Name,
countryId = fixt.AwayTeam.Venue.Country.Id != null ? fixt.AwayTeam.Venue.Country.Id : 0,
countryName = fixt.AwayTeam.Venue.Country.Id != null ? fixt.AwayTeam.Venue.Country.Name : string.Empty,
compId = fixt.Contest.Competition.Id,
compDesc = fixt.Contest.Competition.Description
}).Distinct();
// ensure that we return the max competition based on id for home teams
var homemax = (from t in homecomps
group t by t.teamId
into grp
let maxcomp = grp.Max(g => g.compId)
from g in grp
where g.compId == maxcomp
select g).Distinct();
// ensure that we return the max competition based on id for away teams
var awaymax = (from t in awaycomps
group t by t.teamId
into grp
let maxcomp = grp.Max(g => g.compId)
from g in grp
where g.compId == maxcomp
select g).Distinct();
var filteredteams = homemax.Union(awaymax).OrderBy(t => t.teamName).AsQueryable();
As you can see we want to return the following format which is passed across to a WebAPI so we cast the results to types we can relate to in the UI.
Essentially what we are trying to do is get the home and away teams from a fixture, these fixtures have a contest which relates to a competition. We then get the highest competition id from the grouping and then this is returned with that team. The country is related to the team based on the venue id, when I was originally doing this i had problems figuring out how to do OR joins in linq which is why i split it down to getting home teams and away team and then grouping them based on competition then unioning them together.
An idea of current table size is fixtures has 7840 rows, teams has 8581 rows, contests has 337 rows and competitions has 96 rows. The table that is likely to increase rapidly is the fixture table as this is related to football.
The output we want to end up with is
Team Id, Team Name, Country Id, Country Name, Competition Id, Competition Name
Using no filtering this query takes on average around 5 secs, just wondering if anybody has any ideas/pointers on how to make it quicker.
thanks in advance Mark
I can't judge whether it will speed up things, but your homemax and awaymax queries could be
var homemax = from t in homecomps
group t by t.teamId into grp
select grp.OrderByDescending(x => x.compId).FirstOrDefault();
var awaymax = from t in awaycomps
group t by t.teamId into grp
select grp.OrderByDescending(x => x.compId).FirstOrDefault();
Further, as you are composing one very large query it may perform better when you cut it up in a few smaller queries that fetch intermediary results. Sometimes a few more roundtrips to the database perform better than one very large query for which the database engine can't find a good execution plan.
Another thing is all these Distinct()s. Do you always need them? I think you can do without because you are always fetching data from one table without joining a child collection. Removing them may save a bunch.
Yet another optimization could be to remove the ToUpper. The comparison is done by the database engine in SQL and chances are that the database has a case-insensitive collation. If so, the comparison is never case sensitive even if you'd want it to be! Constructs like Name.ToUpper cancel the use of any index on Name (it is not sargable).
I have a requirement to produce an "flattened" version of data for export, in which the customer wants the related table rows to appear as columns. We're using Entity Framework 4 & .NET 4.0
The customer has the ability to create custom questions for their events. There may be zero to infinite questions. Users attending the event may answer none, some or all of those questions.
What I need to produce is like this:
User 1 User 1 info field A User 1 info field B User 1 Answer to question A User 1 Answer to Question B
User 2 User 2 info field A User 2 info field B User 2 Answer to question A User 2 Answer to Question B
Creating the user info portion of the result set is not problem, but I'm stumped on how to take my related questions and answers, i.e.:
from urf in tblUserRegistrationFields.Include("tblRegistrationFields")
where urf.UserID == eachUserID && urf.tblRegistrationField.EventID == thisEventID
select new
{
FieldLabel = urf.tblRegistrationField.FieldLabel, //this is the question name
value = urf.value //this is the user's answer
}
And add those to my user select statement as individual columns, i.e.:
var users = from u in context.tblUsers
<snip>
select new
{
UserID = u.UserID,
<snip>,
CustomQuestionA = AnswerA,
CustomQuestionB = AnswerB,
... etc.
};
Ultimately I need to create a grid which I can then export to their desired format and I'm open to non-LINQ solutions (although the purchase of 3rd party controls is not an option.) I suspect there is a way to leverage LINQ's grouping features, but I haven't been able to apply it to this scenario.
I found a way to do dynamic pivots in SQL, but couldn't get it to work with two related tables, only with one. Dynamic SQL pivots here:
http://www.simple-talk.com/blogs/2007/09/14/pivots-with-dynamic-columns-in-sql-server-2005/
Ultimately, I fell back and created a dataset manually. It's certainly not an optimized solution, but it gives the customer what they want. I wouldn't recommend this approach for a high volume of data.
I create the dataset and datatable fixed columns, then loop through the list of questions:
//get the list of questions
var context = new EventRegistrationEntities();
var customQuestions = from rf in context.tblRegistrationFields
where rf.EventID == vEventID
select new
{
RegistrationFieldID = rf.RegistrationFieldID,
FieldLabel = rf.FieldLabel
};
//add a question column for each question
List<string> extracolumns = new List<string>();
foreach (var q in customQuestions)
{
dt.Columns.Add(q.FieldLabel, typeof(string));
//store the question names for later user
extracolumns.Add(q.RegistrationFieldID.ToString() + "-" + q.FieldLabel.ToString());
}
Next I then loop through the list of users and insert the fixed data, then within that loop (ugh) I add the users' answers:
foreach (var c in extracolumns) //these are the custom *questions*
{
int regID = Convert.ToInt32(c.Substring(0, c.IndexOf("-")));
string question = c.Substring(c.IndexOf("-") + 1); //we need to pass the question text (aka column header) in because if the user hasn't answered the question we have no access to it from here
//get the question answer
var userAnswer = (from urf in context.tblUserRegistrationFields.Include("tblRegistrationFields")
where urf.UserID == u.UserID && urf.RegistrationFieldID == regID
select new
{
Question = urf.tblRegistrationField.FieldLabel,
Answer = urf.value
}).FirstOrDefault();
//add the answer to the data row
if (userAnswer != null)
{
dr[question] = userAnswer.Answer.ToString();
}
else
{
dr[question] = ""; //if the user has not answered the question, insert a blank
}
}
What does it mean and why (if at all) is it important?
It means you can add additional "operators" to a query. It's important because you can do it extremely efficiently.
For example, let's say you have a method that returns a list (enumerable) of employees:
var employees = GetEmployees();
and another method that uses that one to return all managers:
IEnumerable<Employee> GetManagers()
{
return GetEmployees().Where(e => e.IsManager);
}
You can call that function to get managers that are approaching retirement and send them an email like this:
foreach (var manager in GetManagers().Where(m => m.Age >= 65) )
{
SendPreRetirementMessage(manager);
}
Pop quiz: How many times will that iterate over your employees list? The answer is exactly once; the entire operation is still just O(n)!
Also, I don't need to have separate methods for this. I could compose a query with these steps all in one place:
var retiringManagers = GetEmployees();
retiringManagers = retiringManagers.Where(e => e.IsManager);
retiringManagers = retiringManagers.Where(m => m.Age >= 65);
foreach (var manager in retiringMangers)
{
SendPreRetirementMessage();
}
One cool thing about this is that I can change is at run time, such that I can include or not include one part of the composition inside an if block, such that the decision to use a specific filter is made at run time, and everything still comes out nice and pretty.
I think it means that you can daisy chain your queries, like this
var peterJacksonsTotalBoxOffice
= movies.Where(movie => movie.Director == "Peter Jackson")
.Sum(movie => movie.BoxOffice);