A/B testing. Routing Clients in a gateway API - microservices

Question

I am working on a new project that will be based on microservices. It's an internal app and only about 10 microservices. We will be using a gateway API for authentication and possibly some microservice aggregation. (Probably Netflix zuul with Spring Boot)
What I'm not clear on is how we do the routing for A/B testing and Canary testing. Lets assume I have 100 clients and we want to A/B test a new version of a microservice. The client app needs no changes, it's just internal changes to the function that the microservice provides.
I understand we would stand up a new microservice which is (say) v2. What I'm puzzled on is how do I direct (say) clients 1-10 to the new version. We need to be able to configure this centrally and not change anything on the client.
We know their mac addresses (as well as other identifying attributes) and can insert any kind of header we want to identify their messages.
So how would I direct these to v2 of the API for the A/B test or Canary deployment?

Answer 1

If describe the high level, generic approach, you may do something like this:
Your clients need to have some parameters which will allow to uniquely identify them. Looks like you already have this.
Implement additional API service (let's call it Experiment API). This service should have at least one endpoint that receives client identifying attributes and says whether the client is involved in A/B testing or not.
On each incoming request, the Gateway API need to use that Experiment API endpoint to decide which microservice version (v1 or v2) uses for redirect/call.
To avoid calling Experiment API each time you may introduce some caching layer in the Gateway API. As another option, you may use some custom cookie (that contains whether client under "experiment"), do call to Experiment API only if that cookie is not specified and return the cookie to client with the response.

Answer 2

I have published a prototype on Github that shows how you could achieve the routing using a Zuul Gateway. This prototype just shows how you can route traffic based on a cookie to different instances of the same application. You can do the routing based on any other criteria.
You should also have a look at Spring Cloud Gateway as an alternative to Zuul. Seems to be very promising.
https://github.com/adiesner/spring-boot-sample-ci-gateway
A more simple setup would be to just add nginx in front of your service and use the split_clients method.
http {
# ...
# application version 1a
upstream version_1a {
server 10.0.0.100:3001;
server 10.0.0.101:3001;
}
# application version 1b
upstream version_1b {
server 10.0.0.104:6002;
server 10.0.0.105:6002;
}
split_clients "${arg_token}" $appversion {
95% version_1a;
* version_1b;
}
server {
# ...
listen 80;
location / {
proxy_set_header Host $host;
proxy_pass http://$appversion;
}
}
}
https://www.nginx.com/blog/performing-a-b-testing-nginx-plus/

Answer 3

To expound a bit on #Set's answer. You'll need to introduce some instrumentation code into your gateway API to make the decision about what downstream endpoint to call. If, and only if, the only component of your distributed backend that is concerned with this is the gateway API, the above solution is over-engineered: you can get by with just a library. But it's likely that you will soon discover that one or more of your other services needs to know about the experiment, in which case you DO need a standalone service.
Generally speaking, building a robust experimentation framework is a hard task though. You will quickly run into unexpected problems, e.g. experience stability (how to guarantee the same experience to return visitors) or how to change the allocation proportion (or perhaps completely turn off the new code) without the need to restart the host application. You ought to investigate the open source frameworks out there, or even commercial server side instrumentation. (We have one at Variant).