Managing large data using caching in C# - performance

Question

Suppose, I've a table Tab1 which contain a huge amount of data, let's say 500000 rows (half a million rows). Now, I want to cache that data every half an hour from Tab1 to HttpContext.Current.Cache["BookData"]. Now, the problem that I am facing is that whenever my c# code tries to fetch that data, it takes long as the data is huge. So, I want my c# code to show that older cached data unless the new data gets fetched from database and when new data is fetched completely, it would remove the older cached data and set the new cached data.
What I've tried is working nice and fine whenever the data is small :
[HttpGet]
public IEnumerable<OG_Books> GetBooks()
{
List<OG_Make> BookList = null;
if (HttpContext.Current.Cache["BookData"] != null)
{
DataSet ds = new DataSet();
ds = HttpContext.Current.Cache["BookData"] as DataSet;
BookList = EnumerableExtension.ToList<OG_Books>(ds.Tables[0]);
}
else
{
SqlConnection con = new SqlConnection(ConnectionString);
var command = new SqlCommand("Select_OG_Book", con);
command.CommandType = CommandType.StoredProcedure;
SqlDataAdapter da = new SqlDataAdapter(command);
DataSet ds = new DataSet();
da.Fill(ds);
HttpContext.Current.Cache.Insert("BookData", ds, null, Cache.NoAbsoluteExpiration, new TimeSpan(0, 30, 0));
BookList = EnumerableExtension.ToList<OG_Books>(ds.Tables[0]);
}
return BookList;
}
The above code caches the a new data every half an hour, which is working nice and fine. Now whenever, that data is huge, my code get stuck for a long time on line :
da.Fill(ds);
So, my question is that, how do I return the older cached data while the new data is being fetched from database and when the new data is fetched completely, it would remove the older cached data and cache the new one, how is it possible?

Answer 1

You can use a grace period for your object in cache.
You can implement it with 2 objects in cache with different TTL.
The first one for your "BookData" (your DataSet) with a TTL of 1 hour for example (instead of 30 minutes).
The second one (can be an object) is to control the duration of BookData in cache before refreshing the data. You can use "BookDataInCache" with a TTL of 30 minutes for example.
When you insert "BookData", you have to insert "BookDataInCache" at the same time.
When you get "BookData", you have to check if "BookDataInCache" is still in cache.
If "BookDataInCache" is in cache you can return "BookData",
else you can return "BookData" and make an async request to get and insert the new "BookData" with "BookDataInCache".