Implement HTTP Cache (ETag) in ASP.NET Core Web API - asp.net-web-api

Question

I am working on ASP.NET Core (ASP.NET 5) Web API application and have to implement HTTP Caching with the help of Entity Tags. Earlier I used CacheCow for the same but it seems it does not support ASP.NET Core as of now. I also didn't find any other relevant library or framework support details for the same.
I can write custom code for the same but before that I want to see if anything is already available. Kindly share if something is already available and what is the better way to implement that.

Answer 1

After a while trying to make it work with middleware I figured out that MVC action filters are actually better suited for this functionality.
public class ETagFilter : Attribute, IActionFilter
{
private readonly int[] _statusCodes;
public ETagFilter(params int[] statusCodes)
{
_statusCodes = statusCodes;
if (statusCodes.Length == 0) _statusCodes = new[] { 200 };
}
public void OnActionExecuting(ActionExecutingContext context)
{
}
public void OnActionExecuted(ActionExecutedContext context)
{
if (context.HttpContext.Request.Method == "GET")
{
if (_statusCodes.Contains(context.HttpContext.Response.StatusCode))
{
//I just serialize the result to JSON, could do something less costly
var content = JsonConvert.SerializeObject(context.Result);
var etag = ETagGenerator.GetETag(context.HttpContext.Request.Path.ToString(), Encoding.UTF8.GetBytes(content));
if (context.HttpContext.Request.Headers.Keys.Contains("If-None-Match") && context.HttpContext.Request.Headers["If-None-Match"].ToString() == etag)
{
context.Result = new StatusCodeResult(304);
}
context.HttpContext.Response.Headers.Add("ETag", new[] { etag });
}
}
}
}
// Helper class that generates the etag from a key (route) and content (response)
public static class ETagGenerator
{
public static string GetETag(string key, byte[] contentBytes)
{
var keyBytes = Encoding.UTF8.GetBytes(key);
var combinedBytes = Combine(keyBytes, contentBytes);
return GenerateETag(combinedBytes);
}
private static string GenerateETag(byte[] data)
{
using (var md5 = MD5.Create())
{
var hash = md5.ComputeHash(data);
string hex = BitConverter.ToString(hash);
return hex.Replace("-", "");
}
}
private static byte[] Combine(byte[] a, byte[] b)
{
byte[] c = new byte[a.Length + b.Length];
Buffer.BlockCopy(a, 0, c, 0, a.Length);
Buffer.BlockCopy(b, 0, c, a.Length, b.Length);
return c;
}
}
And then use it on the actions or controllers you want as an attribute:
[HttpGet("data")]
[ETagFilter(200)]
public async Task<IActionResult> GetDataFromApi()
{
}
The important distinction between Middleware and Filters is that your middleware can run before and after MVC middlware and can only work with HttpContext. Also once MVC starts sending the response back to the client it's too late to make any changes to it.
Filters on the other hand are a part of MVC middleware. They have access to the MVC context, with which in this case it's simpler to implement this functionality. More on Filters and their pipeline in MVC.

Answer 2

Building on Eric's answer, I would use an interface that could be implemented on an entity to support entity tagging. In the filter you would only add the ETag if the action is returning a entity with this interface.
This allows you to be more selective about what entities get tagged and allows you have each entity control how its tag is generated. This would be much more efficient than serializing everything and creating a hash. It also eliminates the need to check the status code. It could be safely and easily added as a global filter since you are "opting-in" to the functionality by implementing the interface on your model class.
public interface IGenerateETag
{
string GenerateETag();
}
[AttributeUsage(AttributeTargets.Class | AttributeTargets.Method, AllowMultiple = false)]
public class ETagFilterAttribute : Attribute, IActionFilter
{
public void OnActionExecuting(ActionExecutingContext context)
{
}
public void OnActionExecuted(ActionExecutedContext context)
{
var request = context.HttpContext.Request;
var response = context.HttpContext.Response;
if (request.Method == "GET" &&
context.Result is ObjectResult obj &&
obj.Value is IGenerateETag entity)
{
string etag = entity.GenerateETag();
// Value should be in quotes according to the spec
if (!etag.EndsWith("\""))
etag = "\"" + etag +"\"";
string ifNoneMatch = request.Headers["If-None-Match"];
if (ifNoneMatch == etag)
{
context.Result = new StatusCodeResult(304);
}
context.HttpContext.Response.Headers.Add("ETag", etag);
}
}
}

Answer 3

I am using a middleware that works fine for me.
It adds HttpCache headers to responses (Cache-Control, Expires, ETag, Last-Modified), and implements cache expiration & validation models.
You can find it on nuget.org as a package called Marvin.Cache.Headers.
You could find more information from its Github home page:
https://github.com/KevinDockx/HttpCacheHeaders

Answer 4

Here's a more extensive version for MVC Views (tested with asp.net core 1.1):
using System;
using System.IO;
using System.Security.Cryptography;
using System.Text;
using System.Threading.Tasks;
using Microsoft.AspNetCore.Http;
using Microsoft.AspNetCore.Http.Extensions;
using Microsoft.Net.Http.Headers;
namespace WebApplication9.Middleware
{
// This code is mostly here to generate the ETag from the response body and set 304 as required,
// but it also adds the default maxage (for client) and s-maxage (for a caching proxy like Varnish) to the cache-control in the response
//
// note that controller actions can override this middleware behaviour as needed with [ResponseCache] attribute
//
// (There is actually a Microsoft Middleware for response caching - called "ResponseCachingMiddleware",
// but it looks like you still have to generate the ETag yourself, which makes the MS Middleware kinda pointless in its current 1.1.0 form)
//
public class ResponseCacheMiddleware
{
private readonly RequestDelegate _next;
// todo load these from appsettings
const bool ResponseCachingEnabled = true;
const int ActionMaxAgeDefault = 600; // client cache time
const int ActionSharedMaxAgeDefault = 259200; // caching proxy cache time
const string ErrorPath = "/Home/Error";
public ResponseCacheMiddleware(RequestDelegate next)
{
_next = next;
}
// THIS MUST BE FAST - CALLED ON EVERY REQUEST
public async Task Invoke(HttpContext context)
{
var req = context.Request;
var resp = context.Response;
var is304 = false;
string eTag = null;
if (IsErrorPath(req))
{
await _next.Invoke(context);
return;
}
resp.OnStarting(state =>
{
// add headers *before* the response has started
AddStandardHeaders(((HttpContext)state).Response);
return Task.CompletedTask;
}, context);
// ignore non-gets/200s (maybe allow head method?)
if (!ResponseCachingEnabled || req.Method != HttpMethods.Get || resp.StatusCode != StatusCodes.Status200OK)
{
await _next.Invoke(context);
return;
}
resp.OnStarting(state => {
// add headers *before* the response has started
var ctx = (HttpContext)state;
AddCacheControlAndETagHeaders(ctx, eTag, is304); // intentional modified closure - values set later on
return Task.CompletedTask;
}, context);
using (var buffer = new MemoryStream())
{
// populate a stream with the current response data
var stream = resp.Body;
// setup response.body to point at our buffer
resp.Body = buffer;
try
{
// call controller/middleware actions etc. to populate the response body
await _next.Invoke(context);
}
catch
{
// controller/ or other middleware threw an exception, copy back and rethrow
buffer.CopyTo(stream);
resp.Body = stream; // looks weird, but required to keep the stream writable in edge cases like exceptions in other middleware
throw;
}
using (var bufferReader = new StreamReader(buffer))
{
// reset the buffer and read the entire body to generate the eTag
buffer.Seek(0, SeekOrigin.Begin);
var body = bufferReader.ReadToEnd();
eTag = GenerateETag(req, body);
if (req.Headers[HeaderNames.IfNoneMatch] == eTag)
{
is304 = true; // we don't set the headers here, so set flag
}
else if ( // we're not the only code in the stack that can set a status code, so check if we should output anything
resp.StatusCode != StatusCodes.Status204NoContent &&
resp.StatusCode != StatusCodes.Status205ResetContent &&
resp.StatusCode != StatusCodes.Status304NotModified)
{
// reset buffer and copy back to response body
buffer.Seek(0, SeekOrigin.Begin);
buffer.CopyTo(stream);
resp.Body = stream; // looks weird, but required to keep the stream writable in edge cases like exceptions in other middleware
}
}
}
}
private static void AddStandardHeaders(HttpResponse resp)
{
resp.Headers.Add("X-App", "MyAppName");
resp.Headers.Add("X-MachineName", Environment.MachineName);
}
private static string GenerateETag(HttpRequest req, string body)
{
// TODO: consider supporting VaryBy header in key? (not required atm in this app)
var combinedKey = req.GetDisplayUrl() + body;
var combinedBytes = Encoding.UTF8.GetBytes(combinedKey);
using (var md5 = MD5.Create())
{
var hash = md5.ComputeHash(combinedBytes);
var hex = BitConverter.ToString(hash);
return hex.Replace("-", "");
}
}
private static void AddCacheControlAndETagHeaders(HttpContext ctx, string eTag, bool is304)
{
var req = ctx.Request;
var resp = ctx.Response;
// use defaults for 404s etc.
if (IsErrorPath(req))
{
return;
}
if (is304)
{
// this will blank response body as well as setting the status header
resp.StatusCode = StatusCodes.Status304NotModified;
}
// check cache-control not already set - so that controller actions can override caching
// behaviour with [ResponseCache] attribute
// (also see StaticFileOptions)
var cc = resp.GetTypedHeaders().CacheControl ?? new CacheControlHeaderValue();
if (cc.NoCache || cc.NoStore)
return;
// sidenote - https://tools.ietf.org/html/rfc7232#section-4.1
// the server generating a 304 response MUST generate any of the following header
// fields that WOULD have been sent in a 200(OK) response to the same
// request: Cache-Control, Content-Location, Date, ETag, Expires, and Vary.
// so we must set cache-control headers for 200s OR 304s
cc.MaxAge = cc.MaxAge ?? TimeSpan.FromSeconds(ActionMaxAgeDefault); // for client
cc.SharedMaxAge = cc.SharedMaxAge ?? TimeSpan.FromSeconds(ActionSharedMaxAgeDefault); // for caching proxy e.g. varnish/nginx
resp.GetTypedHeaders().CacheControl = cc; // assign back to pick up changes
resp.Headers.Add(HeaderNames.ETag, eTag);
}
private static bool IsErrorPath(HttpRequest request)
{
return request.Path.StartsWithSegments(ErrorPath);
}
}
}

Answer 5

As an addendum to Erik Božič's answer I found that the HttpContext object was not reporting back the StatusCode correctly when inheriting from ActionFilterAttribute, and applied controller-wide. HttpContext.Response.StatusCode was always 200, indicating it was probably not set by this point in the pipeline. I was instead able to grab the StatusCode from ActionExecutedContext context.Result.StatusCode.

Answer 6

I found an alternative solution which is "closer" to the web api controller method - so you can decide per method which ETag to set...
See my response here: How to use ETag in Web API using action filter along with HttpResponseMessage

Answer 7

We can write simple extension method on ControllerBase class
using Microsoft.AspNetCore.Mvc;
namespace WebApiUtils.Caching
{
public static class ExtensionMethods
{
public static IActionResult OkOr304<T>(
this ControllerBase controller,
T resultObject,
Func<T, string> etagBuilder
)
{
var etag = etagBuilder(resultObject);
if (
// Add additional headers if needed
controller.Request.Headers.Keys.Contains("If-None-Match")
&& controller.Request.Headers["If-None-Match"].ToString() == etag
)
{
return controller.StatusCode(304);
}
controller.Response.Headers.Add("ETag", new[] { etag });
return controller.Ok(resultObject);
}
public static IActionResult OkOr304<T>(this ControllerBase controller, T resultObject)
{
return controller.OkOr304(
resultObject,
x =>
{
// Implement default ETag strategy
return "";
}
);
}
}
}
Then we can use it inside controller with
return this.OkOr304(resultObject, etagBuilder);
or
return this.OkOr304(resultObject);
this works very well if result objects have some version indicator e.g.
return this.OkOr304(resultObject, x => x.VersionNumber.ToString());