I am using IAM credentials for my Spring DataSource to connect to Database. IAM credentials expire in 15 minutes. So the spring.datasource.hikari.maxLifeTime is set to 14 minutes.
However, the service runs properly for only 15 minutes. After that I see "org.postgresql.util.PSQLException: FATAL: PAM authentication failed for user ... error.
I suspect this issue is due to the Hikari connections running with outdated credentials.
However, I don't understand why the new Hikari connections, which are created after existing connection timeouts, are not loading the new IAM credentials.
Here is my Datasource:
#Configuration
public class DatabaseConfiguration {
#Value("${database.url:null}")
private String dataBaseURL;
#Value("${database.username:null}")
private String username;
#Bean
#Profile("postgres")
public DataSource getDataSource() {
DataSourceBuilder dataSourceBuilder = DataSourceBuilder.create();
dataSourceBuilder.url(dataBaseURL);
dataSourceBuilder.username(username);
// extracting IAM credentials by calling IAM Token generator.
dataSourceBuilder.password(RdsIamHikariDataSource.getPassword(dataBaseURL, username));
return dataSourceBuilder.build();
}
}
Doing a google on the classname leads to this blog post. Assuming you are attempting to follow that (and haven't got it to work). Do the following.
Add the following to your application.properties.
spring.datasource.url=<your-url>
spring.datasource.username=<your-iam-role>
spring.datasource.type=package.of.your.RdsIamHikariDataSource
In your java config remove your DatabaseConfiguration, Spring Boot auto configuration will now take care of creating the proper datasource.
Related
I have an Spring boot application that runs several services and uses oracle database. The database is maintained properly, indexes also added up, and when executing SQL statements directly on SQL Developer, it's getting executed in milliseconds.
In the spring boot, I use this to execute the statement:
Session session = sessionFactory.getCurrentSession();
session.createQuery("from Table where id = :id and status = 0").setParameter("id", id);
Here is the config properties for the database:
spring.jpa.properties.hibernate.enable_lazy_load_no_trans=true
hibernate.hbm2ddl.auto=none
Here is the way of datasource initialization in the datasource config:
#Bean
public DataSource dataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource(url, name, pw);
dataSource.setDriverClassName(...);
return dataSource;
}
Recently, it takes so much time to acquire the database connection, it can go up to 10 seconds jut for acquiring connection. I don't think there is any problem in the query. As from the database side, it's also ok. The resources of server which running this service and the database server are also fine, as well as the network. The servers also have auto-scale feature to create new instance when the memory getting low. I just can't figure out what should I do to improve the acquisition time. Could you please help?
I want to set the password for the datasource programatically and still use auto configuration of Spring GCP and Spring Data.
Background
I want to obtain my password as encrypted blob and decrypt it via Google KMS. I read that spring gcp auto configuration mutates the DataSourceProperties bean to apply gcp secific configurations (https://docs.spring.io/spring-cloud-gcp/docs/1.0.0.RELEASE/reference/html/_spring_jdbc.html#_literal_datasource_literal_creation_flow).
Now I would like to mutate the DataSourceProperties Bean exposed by GcpCloudSqlAutoConfiguration once more.
Exposing the DataSourceProperties as follows does not work because GcpCloudSqlAutoConfiguration then doesn't know which Bean to take: the one configured by spring boot auto configuration or mine. But mine should be applied in the very last step of configuration.
#Bean
public DataSourceProperties dataSourceProperties(DataSourceProperties dataSourceProperties) {
dataSourceProperties.setPassword(getDecryptedSecret());
return dataSourceProperties;
}
private String getDecryptedSecret() {
// get encrypted password and decrypt it
return "myDecryptedPassword"
}
You should be able to provide your own bootstrap property source that contains a value for spring.datasource.password.
See: https://cloud.spring.io/spring-cloud-commons/multi/multi__spring_cloud_context_application_context_services.html#customizing-bootstrap-property-sources
Full answer is on project GitHub: https://github.com/spring-cloud/spring-cloud-gcp/issues/2330
Cloud KMS offers encrypting and decrypting data with a symmetric key[1] and/or an asymmetric key[2].
You can see DataSourceProperties[3] for more of the supported options for Spring Boot.
[1]-https://cloud.google.com/kms/docs/encrypt-decrypt
[2]-https://cloud.google.com/kms/docs/encrypt-decrypt-rsa
[3]-https://github.com/spring-projects/spring-boot/blob/v2.1.8.RELEASE/spring-boot-project/spring-boot-autoconfigure/src/main/java/org/springframework/boot/autoconfigure/jdbc/DataSourceProperties.java
Does Spring Boot 2 + Sping Security OAuth2 still support the #AuthorizationServer annotation? From reading the release notes some things haven't been ported over:
Oauth2 Support
Here is the relevant section of my build.grade:
Auth Server
// security
compile "org.springframework.boot:spring-boot-starter-security:${springBootVersion}"
// oauth
// https://mvnrepository.com/artifact/org.springframework.security.oauth/spring-security-oauth2
compile "org.springframework.security.oauth:spring-security-oauth2:2.2.1.RELEASE"
Client Server
// support for Oauth2 user token services not yet migrated into Spring Boot 2.0
compile "org.springframework.security.oauth.boot:spring-security-oauth2-autoconfigure:2.0.1.BUILD-SNAPSHOT"
And right now my Authorization Server Oauth2 endpoints just return a 401 when i try to pass a client-id and client-secret in as Basic Authentication to /oauth/token. Passing in a username and password gives a different code path. So it looks like the OAuth filters aren't quite lined up.
I also found this as well: Spring Boot 2 OAuth2 starter changes.
Has there been a configuration update or do I need a different set of gradle dependencies to restore the Authorization Server to it's previous state?
Thanks!
UPDATE
I wanted to close the loop on this question. In addition to encrypting the client-secrets. The RedisTokenStore issue has also been resolved as of Spring OAuth 2.3.2: Spring OAuth 2.3.2
Spring Security 5 uses a modernized password storage, see OAuth2 Autoconfig:
If you use your own authorization server configuration to configure the list of valid clients through an instance of ClientDetailsServiceConfigurer as shown below, take note that the passwords you configure here are subject to the modernized password storage that came with Spring Security 5.
To solve your problem, see Spring Security Reference:
Troubleshooting
The following error occurs when one of the passwords that are stored has no id as described in the section called “Password Storage Format”.
java.lang.IllegalArgumentException: There is no PasswordEncoder mapped for the id "null"
at org.springframework.security.crypto.password.DelegatingPasswordEncoder$UnmappedIdPasswordEncoder.matches(DelegatingPasswordEncoder.java:233)
at org.springframework.security.crypto.password.DelegatingPasswordEncoder.matches(DelegatingPasswordEncoder.java:196)
The easiest way to resolve the error is to switch to explicitly provide the PasswordEncoder that you passwords are encoded with. The easiest way to resolve it is to figure out how your passwords are currently being stored and explicitly provide the correct PasswordEncoder. If you are migrating from Spring Security 4.2.x you can revert to the previous behavior by exposing a NoOpPasswordEncoder bean. For example, if you are using Java Configuration, you can create a configuration that looks like:
Reverting to NoOpPasswordEncoder is not considered to be secure. You should instead migrate to using DelegatingPasswordEncoder to support secure password encoding.
#Bean
public static NoOpPasswordEncoder passwordEncoder() {
return NoOpPasswordEncoder.getInstance();
}
if you are using XML configuration, you can expose a PasswordEncoder with the id passwordEncoder:
<b:bean id="passwordEncoder"
class="org.springframework.security.crypto.NoOpPasswordEncoder" factory-method="getInstance"/>
Alternatively, you can prefix all of your passwords with the correct id and continue to use DelegatingPasswordEncoder. For example, if you are using BCrypt, you would migrate your password from something like:
$2a$10$dXJ3SW6G7P50lGmMkkmwe.20cQQubK3.HZWzG3YB1tlRy.fqvM/BG
to
{bcrypt}$2a$10$dXJ3SW6G7P50lGmMkkmwe.20cQQubK3.HZWzG3YB1tlRy.fqvM/BG
OAuth2 AuthorizationServer uses basic authentication.
So, you also need to encode your client secret with delegatedPasswordEncoder in AuthorizationServerConfig to completely solve "There is no PasswordEncoder mapped for the id "null" " exception.
Yao Liu's answer solved my problem.
1) created a bean to auto wire PasswordEncoder;
#Bean
public PasswordEncoder passwordEncoder() {
String idForEncode = "bcrypt";
Map<String, PasswordEncoder> encoderMap = new HashMap<>();
encoderMap.put(idForEncode, new BCryptPasswordEncoder());
return new DelegatingPasswordEncoder(idForEncode, encoderMap);
}
2) Auto wired passwordEncoder in AuthorizationServerConfig class;
#Autowired
private PasswordEncoder passwordEncoder;
3) encoded CLIENT_SECRET with passwordEncoder.
#Override
public void configure(ClientDetailsServiceConfigurer configurer) throws Exception {
configurer
.inMemory()
.withClient(CLIENT_ID)
.secret(passwordEncoder.encode(CLIENT_SECRET))
.authorizedGrantTypes(GRANT_TYPE_FOR_LOGIN, GRANT_TYPE_FOR_REFRESH)
.scopes(SCOPE_READ, SCOPE_WRITE)
.accessTokenValiditySeconds(TOKEN_VALIDITY_SECONDS)
.refreshTokenValiditySeconds(TOKEN_VALIDITY_SECONDS)
.resourceIds(RESOURCES_IDS);
}
That's it.
As #false_memories stated above, with Spring Boot 2 you need to encode your secret. In my project, it looks like:
public void configure(ClientDetailsServiceConfigurer clients) throws Exception {
String secretEncoded = passwordEncoder().encode("secret");
clients.inMemory().withClient("some-web-app").secret(secretEncoded).accessTokenValiditySeconds(expiration)
.scopes("read", "write").authorizedGrantTypes("password", "refresh_token").resourceIds("resource");
}
I wanted to close the loop on this question. In addition to encrypting the client-secrets. The RedisTokenStore issue has also been resolved as of Spring OAuth 2.3.2: Spring OAuth 2.3.2
I have a spring boot application that is deployed to Azure. This application uses JPA to connect to a SQL Server database. Everything seems to work nicely, until after ~30 minutes of inactivity. My guess is the database is closing the connection and the spring boot (tomcat) container isn't properly handling this situation.
I'm injecting the datasouce properties via:
#Bean
#Primary
#ConfigurationProperties(prefix = "spring.datasource")
public DataSource getDataSource() {
return DataSourceBuilder.create().build();
}
My application.properties are:
spring.datasource.url=jdbc:sqlserver://xxxxxx:1433;database=xxxxxx
spring.datasource.username=xxxxxxx
spring.datasource.password=xxxxxxxx
spring.datasource.driver-class-name=com.microsoft.sqlserver.jdbc.SQLServerDriver
spring.datasource.tomcat.test-on-borrow=true
spring.datasource.tomcat.validation-query=SELECT 1
spring.datasource.tomcat.validation-interval=0
spring.datasource.tomcat.removeAbandoned=true
spring.datasource.tomcat.removeAbandonedTimeout=120
spring.datasource.tomcat.timeBetweenEvictionRunsMillis=34000
spring.datasource.tomcat.minEvictableIdleTimeMillis=55000
spring.datasource.tomcat.initialSize=4
spring.datasource.tomcat.maxActive=16
However after a period of inactivity i'm getting SQLExceptions:
com.microsoft.sqlserver.jdbc.SQLServerException: The connection is closed.
I'm trying to determine the cause of this. I'm guesses connection validation isn't working correctly, but as I understand this, the connection should be validate by tomcat before the application is allowed to use it. Any help would be greatly appreciated.
I'm aware that H2 has a boolean property/setting called DATABASE_TO_UPPER, which you can set at least in the connection URL, as in: ;DATABASE_TO_UPPER=false
I’d like to set this to false, but in my Spring Boot app, I don’t explicitly have a H2 connection URL anywhere. Implicitly there sure is a connection URL though, as I can see in the logs:
o.s.j.d.e.EmbeddedDatabaseFactory: Shutting down embedded database:
url='jdbc:h2:mem:2fb4805b-f927-49b3-a786-2a2cac440f44;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=false'
So the question is, what's the easiest way to tell H2 to disable DATABASE_TO_UPPER in this scenario? Can I do it in code when creating the H2 datasource with EmbeddedDatabaseBuilder (see below)? Or in application properties maybe?
This is how the H2 database is explicitly initialised in code:
#Configuration
#EnableTransactionManagement
public class DataSourceConfig {
#Bean
public DataSource devDataSource() {
return new EmbeddedDatabaseBuilder()
.generateUniqueName(true)
.setType(EmbeddedDatabaseType.H2)
.setScriptEncoding("UTF-8")
.ignoreFailedDrops(true)
.addScripts("db/init.sql", "db/schema.sql", "db/test_data.sql")
.build();
}
}
Also, I'm telling JPA/Hibernate not to auto-generate embedded database (without this there was an issue that two in-memory databases were launched):
spring.jpa.generate-ddl=false
spring.jpa.hibernate.ddl-auto=none
You can't w\ the generateUniqueName, but if you call setName("testdb;DATABASE_TO_UPPER=false") you can add parameters. I doubt this is officially supported, but it worked for me.
The spring code that generates the connection url is like this:
String.format("jdbc:h2:mem:%s;DB_CLOSE_DELAY=-1;DB_CLOSE_ON_EXIT=false", databaseName)
You may want abandon using explicit creation via EmbeddedDatabaseBuilder. Spring Boot creates H2 instance automatically based on configuration. So I would try this in application.properties:
spring.datasource.url=jdbc:h2:file:~/testdb;DATABASE_TO_UPPER=false