How to scale horizontally a spring-boot oauth2 server with JDBC implementation - spring-boot

Question

I have a spring boot oauth2 server that uses a JDBC implementation. It is configured as an authorization server with #EnableAuthorizationServer.
I'd like to scale that application horyzontally but it doesn't seem to work properly.
I can connect only if I have one instance (pods) of the server.
I use autorisation_code_client grant from another client service to get the token.
So first the client service redirect the user to the oauth2 server form, then once the user is authenticated he is supposed to be redirect to the client-service with a code attached to the url, finally the client use that code to request the oauth2 server again and obtain the token.
Here the user is not redirected at all if I have several instance of the oauth2-server. With one instance it works well.
When I check the log of the two instances in real time, I can see that the authentication works on one of them. I don't have any specific error the user is just not redirected.
Is there a way to configure the oauth2-server to be stateless or other way to fix that issue ?
Here is my configuration, the AuthorizationServerConfigurerAdapter implementation.
#Configuration
public class AuthorizationServerConfiguration extends AuthorizationServerConfigurerAdapter {
#Bean
#ConfigurationProperties(prefix = "spring.datasource")
public DataSource oauthDataSource() {
return DataSourceBuilder.create().build();
}
#Autowired
#Qualifier("authenticationManagerBean")
private AuthenticationManager authenticationManager;
#Bean
public JdbcClientDetailsService clientDetailsSrv() {
return new JdbcClientDetailsService(oauthDataSource());
}
#Bean
public TokenStore tokenStore() {
return new JdbcTokenStore(oauthDataSource());
}
#Bean
public ApprovalStore approvalStore() {
return new JdbcApprovalStore(oauthDataSource());
}
#Bean
public AuthorizationCodeServices authorizationCodeServices() {
return new JdbcAuthorizationCodeServices(oauthDataSource());
}
#Bean
public TokenEnhancer tokenEnhancer() {
return new CustomTokenEnhancer();
}
#Bean
#Primary
public AuthorizationServerTokenServices tokenServices() {
DefaultTokenServices tokenServices = new DefaultTokenServices();
tokenServices.setTokenStore(tokenStore());
tokenServices.setTokenEnhancer(tokenEnhancer());
return tokenServices;
}
#Override
public void configure(ClientDetailsServiceConfigurer clients) throws Exception {
clients.withClientDetails(clientDetailsSrv());
}
#Override
public void configure(AuthorizationServerSecurityConfigurer oauthServer) {
oauthServer
.tokenKeyAccess("permitAll()")
.checkTokenAccess("isAuthenticated()")
.allowFormAuthenticationForClients();
}
#Override
public void configure(AuthorizationServerEndpointsConfigurer endpoints) {
endpoints
.authenticationManager(authenticationManager)
.approvalStore(approvalStore())
//.approvalStoreDisabled()
.authorizationCodeServices(authorizationCodeServices())
.tokenStore(tokenStore())
.tokenEnhancer(tokenEnhancer());
}
}
The main class
#SpringBootApplication
#EnableResourceServer
#EnableAuthorizationServer
#EnableConfigurationProperties
#EnableFeignClients("com.oauth2.proxies")
public class AuthorizationServerApplication {
public static void main(String[] args) {
SpringApplication.run(AuthorizationServerApplication.class, args);
}
}
The Web Security Configuration
#Configuration
#Order(1)
public class WebSecurityConfiguration extends WebSecurityConfigurerAdapter {
#Bean
#Override
public UserDetailsService userDetailsServiceBean() throws Exception {
return new JdbcUserDetails();
}
#Override
#Bean
public AuthenticationManager authenticationManagerBean() throws Exception {
return super.authenticationManagerBean();
}
#Override
protected void configure(HttpSecurity http) throws Exception { // #formatter:off
http.requestMatchers()
.antMatchers("/",
"/login",
"/login.do",
"/registration",
"/registration/confirm/**",
"/registration/resendToken",
"/password/forgot",
"/password/change",
"/password/change/**",
"/oauth/authorize**")
.and()
.authorizeRequests()//autorise les requetes
.antMatchers(
"/",
"/login",
"/login.do",
"/registration",
"/registration/confirm/**",
"/registration/resendToken",
"/password/forgot",
"/password/change",
"/password/change/**")
.permitAll()
.and()
.requiresChannel()
.anyRequest()
.requiresSecure()
.and()
.authorizeRequests()
.anyRequest()
.authenticated()
.and()
.formLogin()
.loginPage("/login")
.loginProcessingUrl("/login.do")
.usernameParameter("username")
.passwordParameter("password")
.and()
.userDetailsService(userDetailsServiceBean());
} // #formatter:on
#Override
protected void configure(AuthenticationManagerBuilder auth) throws Exception {
auth.userDetailsService(userDetailsServiceBean()).passwordEncoder(passwordEncoder());
}
}
Client side the WebSecurityConfigurerAdapter
#EnableOAuth2Sso
#Configuration
public class UiSecurityConfig extends WebSecurityConfigurerAdapter {
#Override
public void configure(HttpSecurity http) throws Exception {
http.antMatcher("/**")
.authorizeRequests()
.antMatchers(
"/",
"/index.html",
"/login**",
"/logout**",
//resources
"/assets/**",
"/static/**",
"/*.ico",
"/*.js",
"/*.json").permitAll()
.anyRequest()
.authenticated()
.and()
.csrf().csrfTokenRepository(csrfTokenRepository())
.and()
.addFilterAfter(csrfHeaderFilter(), SessionManagementFilter.class);
}
}
the oauth2 configuration properties
oauth2-server is the service name (load balancer) on kubernetes and also the server path that is why it appears twice.
security:
oauth2:
client:
clientId: **********
clientSecret: *******
accessTokenUri: https://oauth2-server/oauth2-server/oauth/token
userAuthorizationUri: https://oauth2.mydomain.com/oauth2-server/oauth/authorize
resource:
userInfoUri: https://oauth2-server/oauth2-server/me
Here an important detail, the value of userAuthorizationUri is the address to access the oauth2-server from the outside of the k8s cluster. The client-service send back that address into the response with a 302 http code if the user is not connected and tries to access to the /login path of the client-service. then the user is redirected to the /login path of the oauth2-server.
https://oauth2.mydomain.com target an Nginx Ingress controller that handle the redirection to the load balancer service.

Answer 1

Here is a solution to this problem. It's not a Spring issue at all but a bad configuration of the Nginx Ingress controller.
The authentication process is done in several stages :
1 - the user clic on a login button that target the /login path of the client-server
2 - the client-server, if the user is not authenticated yet, send a response to the
browser with a 302 http code to redirect the user to the oauth2-server, the value of
the redirection is composed with the value of the
security.oauth2.client.userAuthorizationUri property
and the redirection url that will be used by the browser to allow the client-server to get the Token once the user is authenticated.
That url look like this :
h*tps://oauth2.mydomain.com/oauth2-server/oauth/authorize?client_id=autorisation_code_client&redirect_uri=h*tps://www.mydomain.com/login&response_type=code&state=bSWtGx
3 - the user is redirected to the previous url
4 - the oauth2-server send a 302 http code to the browser with the login url of the
oauth2-server, h*tps://oauth2.mydomain.com/oauth2-server/login
5 - the user submit his credentials and the token is created if they are correct.
6 - the user is redirected to the same address as at the step two, and the oauth-server
add informations to the redirect_uri value
7 - the user is redirected to the client-server. The redirection part of the response look like this :
location: h*tps://www.mydomain.com/login?code=gnpZ0r&state=bSWtGx
8 - the client-server contact the oauth2-server and obtain the token from the code and the state that authenticates it. It doesn't matter if the instance of the oauth2
server is different than the one used by the user to authenticate himself. Here the
client-server use the value of security.oauth2.client.accessTokenUri to get the
token, this is the internal load balancing service address that targets the oauth2 server
pods, so it doesn't pass through any Ingress controller.
So at the steps 3 to 6 the user must communicate with the same instance of the oauth2-server throught the Ingress controller in front of the load balancer service.
Its is possible by configuring the Nginx Ingress controller with a few annotations :
"annotations": {
...
"nginx.ingress.kubernetes.io/affinity": "cookie",
"nginx.ingress.kubernetes.io/session-cookie-expires": "172800",
"nginx.ingress.kubernetes.io/session-cookie-max-age": "172800",
"nginx.ingress.kubernetes.io/session-cookie-name": "route"
}
That way we ensure that the user will be redirected to the same pods/instance of the oauth2-server during the authentication process as long he's identified with the same cookie.
The affinity session mecanism is a great way to scale the authentication server and also the client-server. Once the user is authenticated he will always use the same instance of the client and keep his session informations.
Thanks to Christian Altamirano Ayala for his help.

Answer 2

By default an in-memory TokenStore is used.
The default InMemoryTokenStore is perfectly fine for a single server
If you want multiple pods, you probably should go for JdbcTokenStore
The JdbcTokenStore is the JDBC version of the same thing, which stores token data in a relational database. Use the JDBC version if you can share a database between servers, either scaled up instances of the same server if there is only one, or the Authorization and Resources Servers if there are multiple components. To use the JdbcTokenStore you need "spring-jdbc" on the classpath.
Source Spring Security: OAuth 2 Developers Guide