Having to call .ToList() in entity framework 4.1 before .Skip() and .Take() on large table - asp.net-mvc-3

Question

I'm trying to do something a little clever with my app. I have a table, Adverts - which contains info on cars: model, mileage etc. The table is related to a few other tables via foreign keys e.g. model name is retrieved through a foreign key linking to a "VehicleModels" table etc.
Within the app's "Entities" dir (classes which map to tables in the database) I have one for the Adverts table, Advert.cs. This has a couple of properties which EF has been told to ignore (in fluent api) as they don't map to actual fields in the Adverts table.
The idea behind these fields is to store the calculated distance from a postcode (zip code) the user enters in a search form which filters through the Adverts table if they only want to see cars available within a certain radius. e.g.:
IQueryable<Advert> FilteredAdverts = repository.Adverts
.Where(am => mfr == "" || am.Manufacturer == mfr) &&
(am => model == etc etc...)
Later on, to calculate the distance the code resembles:
if (userPostcode != null) {
foreach (var ap in FilteredAdverts.ToList()) {
distmiles = //calculate distance in miles
distkm = //calculate distance in km
ap.DistanceMiles = Convert.ToInt32(distmiles);
ap.DistanceKm = Convert.ToInt32(distkm);
}
}
The problem I'm having is that in order to assign values to these two fields, I'm having to use .ToList() which is pulling all rows from the table. Works ok if there are only a few rows, but when there are ~1,000 it takes approx. 2.2 seconds, when I increased it to about 12,000 rows it took 32 seconds for the page to load when no filters were applied i.e. all active adverts returned.
The reason I'm pulling all adverts before calling .Skip and .Take to display them is that the filters available in the search form are based on possible options of all current adverts that are active i.e. have time remaining, rather than just selecting a list of manufacturers from the manufacturers table (where a user could choose a manufacturer for which there are no search results). e.g.
VehicleManufacturers = (from vm in FilteredAdverts.Select(x => x.VehicleManufacturer).Distinct().OrderBy(x => x)
select new SearchOptionsModel
{
Value = vm,
Text = vm,
Count = FilteredAdvertsVM.Where(x => x.VehicleManufacturer == vm).Count(),
})
.... filters for model, mileage etc
To get an idea of what I'm trying to achieve - take a look at the search form on the Autotrader website.
Once all the filters are applied, just before the model is passed to the view, .Skip and .Take are applied, but of course by this time all rows have been pulled.
My question is, how do I go about redoing this? Is there a better method to make use of these non-mapped properties in my Advert entity class? I'm working on my home PC - C2D # 3.4GHz, 2GB ram - would the slow queries run ok on a propert web host ?

Answer 1

You cannot use server-side paging on a client side function. That's the short answer. Assuming I understand your need correctly (to filter a list based on proximity to a given zip code), a solution I've used in the past is storing each 'Advert' record with a lat/long for that record's zip code. This data is persisted.
Then, when it comes time to query, calculate a bounding box (lat1, lng1, lat2, lng2) based on X distance from the center (user provided zip code) and filter the query results based on records whose lat/lng fits within this box. You can then apply client side calculations to further and more accurately filter the list, but using this method, you can establish a base filter to minimize the number of records pulled.
Edit: You can order the results of the query based on the absolute distance from the center point in terms of abs(latU-latR) and abs(lngU-lngR) where latU/lngU is the lat/lng of the user provided zip code and latR/lngR is the lat/lng of the record in the db.