I'm facing a problem with Spring: I'm migrating from Spring Security ver. 3.2.7.RELEASE to 4.0.2.RELEASE. Everything was working fine in older version, however a problem occured when it came to loading DataSource.
Let me describe the architecture:
Application is secured with both SAML and LDAP mechanisms (SAML configuration is pretty similar to config given here: https://github.com/vdenotaris/spring-boot-security-saml-sample/blob/master/src/main/java/com/vdenotaris/spring/boot/security/saml/web/config/WebSecurityConfig.java).
They both need to connect to database in order to get some required data. We use MyBatis with Spring Mybatis to get needed data. That's, where the problem begins.
My DAO configuration class looks like this:
#Configuration
#EnableConfigurationProperties
#MapperScan(basePackages = { "pl.myapp" })
public class DaoConfiguration {
#Bean
#Primary
#ConfigurationProperties(prefix = "spring.datasource")
public DataSource dataSource() {
return DataSourceBuilder.create().build();
}
#Bean
#Primary
public JdbcTemplate jdbcTemplate() {
return new JdbcTemplate(dataSource());
}
#Bean
#Primary
public SqlSessionFactoryBean sqlSessionFactoryBean() {
SqlSessionFactoryBean sqlSessionFactoryBean = new SqlSessionFactoryBean();
sqlSessionFactoryBean.setDataSource(dataSource());
// some stuff happens here
return sqlSessionFactoryBean;
}
#Bean
#Primary
public DataSourceTransactionManager transactionManager() {
return new DataSourceTransactionManager(dataSource());
}
#Bean
#ConfigurationProperties(prefix = "liquibase.datasource")
#ConditionalOnProperty(name="liquibase.enabled")
public DataSource liquibaseDataSource() {
DataSource liquiDataSource = DataSourceBuilder.create().build();
return liquiDataSource;
}
}
In previous version it worked like a charm, but now it has a problem loading mappers, resulting in Bean creation exception on FactoryBean type check: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'someMapper' defined in file [<filename>]: Invocation of init method failed; nested exception is java.lang.IllegalArgumentException: Property 'sqlSessionFactory' or 'sqlSessionTemplate' are required
over and over again (it's not my problem, it's a known Spring/MyBatis bug).
I did some debugging and discovered something interesting: it looks like DaoConfiguration is not treated like a configuration here! I mean: if I add
#Bean
public SqlSessionFactory sqlSessionFactory() throws Exception {
return sqlSessionFactoryBean().getObject();
}
to this config, "normal" call of #Bean annotated method should result in calling proper interceptor, here it lacks this funcionality.
My prediction is that: this config class has not been properly wrapped yet and Spring Security already needs beans produced by it.
Is there any solution to properly load this configuration before Spring Security is initialized? Or am I just wrong and missing something (maybe not so) obvious?
Related
My system has got 3 data sources, all exposed as beans named datasourceA, datasourceB, datasourceC. I am trying to set the data source of spring batch to datasourceB but I am getting quite some issues.
My spring batch class
#Configuration
#EnableBatchProcessing
public class JobBatchConfiguration extends DefaultBatchConfigurer {
#Override
public void setDataSource(#Qualifier("dataSourceB") DataSource dataSource) {
super.setDataSource(dataSource);
}
#Bean
public BatchDataSourceInitializer batchDatabaseInitializer(#Qualifier("dataSourceB") DataSource dataSource, ResourceLoader resourceLoader){
BatchProperties batchProperties = new BatchProperties();
batchProperties.setInitializeSchema(DataSourceInitializationMode.ALWAYS);
BatchDataSourceInitializer batchDatabaseInitializer = new BatchDataSourceInitializer(dataSource, resourceLoader, batchProperties);
return batchDatabaseInitializer;
}
}
With this setup I am getting this error upon startup
Field dataSource in org.springframework.batch.core.configuration.annotation.AbstractBatchConfiguration required a single bean, but 3 were found:
Consider marking one of the beans as #Primary, updating the consumer to accept multiple beans, or using #Qualifier to identify the bean that should be consumed
I cannot set any of my datasources to #Primary since my spring batch writer reads and writes using all 3 datasources. I am using JPA repository and spring data.
Any solution? I thought overriding setDataSource should be enough
You should place #Qualifier both on your bean definition and in place when the bean is being autowired.
// data sources config
#Bean
#Qualifier("dataSourceB")
public DataSource dataSourceB() { ... }
// batch processing config
public void setDataSource(#Qualifier("dataSourceB") DataSource dataSource) { ... }
Also, there is no need to use #Qualifier because Spring uses a bean name as a fallback qualifier. Cite from https://docs.spring.io/spring-framework/docs/current/reference/html/core.html#beans-autowired-annotation-qualifiers:
For a fallback match, the bean name is considered a default qualifier value.
Thus, you can rename your variable to match the bean name you want to autowire.
// data sources config
#Bean
public DataSource dataSourceB() { ... }
// batch processing config
public void setDataSource(DataSource dataSourceB) { ... }
I'm trying to write an application that accesses data from two sources. I'm using Spring Boot 2.3.2. I've looked at several sources for info about how to configure the app: the Spring documentation talks about setting up multiple datasources, but does not explain how to link up JPA repositories. This Baeldung article goes a lot further, but I'm looking to take advantage of autoconfiguration in Spring.
So far, I've created a separate package, added a config class (along with model and repositories), and included this package in scanBasePackages so that it's picked up. Since I'll have more than one datasource, I've added this to my #SpringBootApplication:
#Bean
#Primary
public DataSourceProperties dataSourceProperties() {
return new DataSourceProperties();
}
This successfully loads up my Spring app using the standard spring config values. The two databases are on different servers, but should share characteristics (other than url and credentials).
So, my auxiliary configuration file looks like this
#Configuration
#EnableAutoConfiguration
#EnableTransactionManagement
#EnableJpaRepositories(
entityManagerFactoryRef = "orgEntityManagerFactory",
transactionManagerRef = "orgTransactionManager",
basePackages = {
"pacage2.repositories"
}
)
public class DataSourceConfiguration {
// added because of this answer: https://stackoverflow.com/a/51305724/167889
#Bean
public EntityManagerFactoryBuilder entityManagerFactoryBuilder() {
return new EntityManagerFactoryBuilder(new HibernateJpaVendorAdapter(), new HashMap<>(), null);
}
#Bean
#ConfigurationProperties(prefix = "external.datasource")
public DataSourceProperties orgDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
public HikariDataSource orgDataSource(#Qualifier("orgDataSourceProperties") DataSourceProperties properties) {
return properties.initializeDataSourceBuilder().type(HikariDataSource.class)
.build();
}
#Bean
public LocalContainerEntityManagerFactoryBean orgEntityManagerFactory(EntityManagerFactoryBuilder builder,
#Qualifier("orgDataSource") DataSource dataSource) {
return builder
.dataSource(dataSource)
.packages("package2.model")
.build();
}
#Bean
public PlatformTransactionManager orgTransactionManager(
#Qualifier("orgEntityManagerFactory") EntityManagerFactory entityManagerFactory
) {
return new JpaTransactionManager(entityManagerFactory);
}
}
Now, the error I'm getting right now is Access to DialectResolutionInfo cannot be null when 'hibernate.dialect' not set. However, I have that value in my config and it's applied by the Spring auto config. I believe it needs to be set in the EntityManagerFactoryBuilder and by creating my own, the autoconfig isn't getting applied.
How can I have my cake and eat it too? I'd like to leverage as much of the robust autoconfiguration that Spring provides to setup datasources and wire them to the appropriate repositories. Effectively, all that I want to change is the url and credentials, and I can separate the entities and repositories into a completely separate package for easy scanning.
I have a SpringBoot app. that has to access to different datasources to export data from 1 to another (1 local datasource and another remote datasource)
This is how my persistenceConfig looks like
public class PersistenceConfig {
#Bean
public JdbcTemplate localJdbcTemplate() {
return new JdbcTemplate(localDataSource());
}
#Bean
public JdbcTemplate remoteJdbcTemplate() {
return new JdbcTemplate(remoteDataSource());
}
#Bean
public DataSource localDataSource(){
HikariConfig config = new HikariConfig();
config.setMaximumPoolSize(getLocalDbPoolSize());
config.setMinimumIdle(5);
config.setDriverClassName(getLocalDbDriverClassName());
config.setJdbcUrl(getLocalDbJdbcUrl());
config.addDataSourceProperty("user", getLocalDbUser());
config.addDataSourceProperty("password", getLocalDbPwd());
return new HikariDataSource(config);
}
#Bean
public DataSource remoteDataSource(){
HikariConfig config = new HikariConfig();
config.setMaximumPoolSize(getRemoteDbPoolSize());
config.setMinimumIdle(5);
config.setDriverClassName(getRemoteDbDriverClassName());
config.setJdbcUrl(getRemoteDbJdbcUrl());
config.addDataSourceProperty("user", getRemoteDbUser());
config.addDataSourceProperty("password", getRemoteDbPwd());
return new HikariDataSource(config);
}
}
But when I init my app I got this error:
Caused by: org.springframework.beans.factory.NoUniqueBeanDefinitionException: No qualifying bean of type 'javax.sql.DataSource' available: expected single matching bean but found 2: localDataSource,remoteDataSource
I also tried to user qualified beans, as follows:
#Bean(name = "localJdbcTemplate")
public JdbcTemplate localJdbcTemplate() {
return new JdbcTemplate(localDataSource());
}
#Bean(name = "remoteJdbcTemplate")
public JdbcTemplate remoteJdbcTemplate() {
return new JdbcTemplate(remoteDataSource());
}
#Bean(name = "localDataSource")
public DataSource localDataSource(){
HikariConfig config = new HikariConfig();
config.setMaximumPoolSize(getLocalDbPoolSize());
config.setMinimumIdle(5);
config.setDriverClassName(getLocalDbDriverClassName());
config.setJdbcUrl(getLocalDbJdbcUrl());
config.addDataSourceProperty("user", getLocalDbUser());
config.addDataSourceProperty("password", getLocalDbPwd());
return new HikariDataSource(config);
}
#Bean(name = "remoteDataSource")
public DataSource remoteDataSource(){
HikariConfig config = new HikariConfig();
config.setMaximumPoolSize(getRemoteDbPoolSize());
config.setMinimumIdle(5);
config.setDriverClassName(getRemoteDbDriverClassName());
config.setJdbcUrl(getRemoteDbJdbcUrl());
config.addDataSourceProperty("user", getRemoteDbUser());
config.addDataSourceProperty("password", getRemoteDbPwd());
return new HikariDataSource(config);
}
but then I got this other error:
A component required a bean of type 'org.springframework.transaction.PlatformTransactionManager' that could not be found.
- Bean method 'transactionManager' not loaded because #ConditionalOnSingleCandidate (types: javax.sql.DataSource; SearchStrategy: all) did not find a primary bean from beans 'remoteDataSource', 'localDataSource'
I also tried
#SpringBootApplication(exclude = {
DataSourceAutoConfiguration.class,
DataSourceTransactionManagerAutoConfiguration.class})
#EnableAutoConfiguration(exclude = {
DataSourceAutoConfiguration.class,
DataSourceTransactionManagerAutoConfiguration.class})
but then I got
A component required a bean of type 'org.springframework.transaction.PlatformTransactionManager' that could not be found.
You can use bean names to qualify them:
#Bean(name = "localDataSource")
public DataSource localDataSource() {
...
}
#Bean(name = "remoteDataSource")
public DataSource remoteDataSource() {
...
}
Please note: You have to do the same for your JdbcTemplate beans - just give them a name and it will work.
See the Spring JavaDoc for more Information: Bean
#Bean(name = "localJdbcTemplate")
public JdbcTemplate localJdbcTemplate() {
return new JdbcTemplate(localDataSource());
}
When you use your JdbcTemplate beans within your export service implementation via autowiring (#Autowired), you need to use #Qualifier to qualify them:
#Autowired
#Qualifier("localJdbcTemplate")
private JdbcTemplate jdbcTemplate;
#Autowired
#Qualifier("remoteJdbcTemplate")
private JdbcTemplate jdbcTemplate;
Bean gets its name from the method name, providing name attribute just makes it explicit (keeping the name the same as the method name). Overall suggestion about #Bean(name="...") and #Qualifier didn't fix the error for me.
I set up sample project with two embedded databases and got the same error as the aothor. Spring suggestion was to annotate one of the DataSource beans as #Primary and, in fact, this fixes the error. Usually it happens, when some other application parts want to see only one or one primary DataSource, if several present.
What seems to be a better solution is to disable not needed autoconfiguration beans keeping rest of the code as it is:
#SpringBootApplication(exclude = {
DataSourceAutoConfiguration.class,
DataSourceTransactionManagerAutoConfiguration.class})
or:
#EnableAutoConfiguration(exclude = {
DataSourceAutoConfiguration.class,
DataSourceTransactionManagerAutoConfiguration.class})
depending on which annotations are in use.
If author doesn't use any JPA provider and operates directly with JdbcTemplate it may be a suitable solution.
Confident this hasn't been asked but reading through the Spring docs and testing utilities I found this annotation and thought I'd start using it. Reading through the fine print I read:
Regular #Component beans will not be loaded into the ApplicationContext.
That sounded good and I even liked the idea of using H2 except from what I found the entity I wanted to use had catalog and schema modifiers to it and the default H2 I couldn't figure out how to support that. I made an H2 datasource for the test branch and use that and override the replace. I wound up with
#RunWith(SpringRunner.class)
#ContextConfiguration(classes=ABCH2Congfiguration.class)
#DataJpaTest
#AutoConfigureTestDatabase(replace= AutoConfigureTestDatabase.Replace.NONE)
public class StatusRepositoryTest {
}
However my tests fails fro Caused by: org.springframework.beans.factory.NoSuchBeanDefinitionException: No qualifying bean of type.
which leads to:
Error creating bean with name 'customerServiceImpl': Unsatisfied dependency.
However the customerServiceImpl is this bean:
#Component
public class CustomerServiceImpl implements CustomerService {
}
That says #Component. The fine print for DataJpaTest says it doesn't load #Components. Why is it doing that and thus failing the test?
As Kyle and Eugene asked below here's the rest:
package com.xxx.abc.triage;
#Component
public interface CustomerService {
}
Configuration
#ComponentScan("com.xxx.abc")
#EnableJpaRepositories("com.xxx.abc")
//#Profile("h2")
public class ABMH2Congfiguration {
#Primary
#Bean(name = "h2source")
public DataSource dataSource() {
EmbeddedDatabase build = new EmbeddedDatabaseBuilder().setType(EmbeddedDatabaseType.H2).setName("ABC").addScript("init.sql").build();
return build;
}
#Bean
public JpaVendorAdapter jpaVendorAdapter() {
HibernateJpaVendorAdapter bean = new HibernateJpaVendorAdapter();
bean.setDatabase(Database.H2);
bean.setShowSql(true);
bean.setGenerateDdl(true);
return bean;
}
#Bean
public LocalContainerEntityManagerFactoryBean entityManagerFactory(
DataSource dataSource, JpaVendorAdapter jpaVendorAdapter) {
LocalContainerEntityManagerFactoryBean bean = new LocalContainerEntityManagerFactoryBean();
bean.setDataSource(dataSource);
bean.setJpaVendorAdapter(jpaVendorAdapter);
bean.setPackagesToScan("com.xxx.abc");
return bean;
}
#Bean
public JpaTransactionManager transactionManager(EntityManagerFactory emf) {
return new JpaTransactionManager(emf);
}
}
And just to clarify the question, why is #Component being loaded into the context within a #DataJpaTest?
#ComponentScan automatically inject all found #Component and #Service into context. You could override it by separate #Bean:
#Bean
CustomerService customerService{
return null;
}
Or remove #Component annotation from CustomerService and CustomerServiceImpl, but you should add #Bean at your production #Configuration
#DataJpaTest does not load #Component, #Service... by default, only #Repository and internal things needed to configure Spring data JPA.
In your test, you can load any #Configuration you need, and in your case, you load #ABMH2Congfiguration which performs a #ComponentScan that's why Spring try to load your CustomerService.
You should only scanning the #Repository in this configuration class, and scan others #Component, #Service... in another #Configuration like DomainConfiguration. It's always a good practice to separate different types of configurations.
My goal is to have a have integration tests that ensures that there isn't too many database queries happening during lookups. (This helps us catch n+1 queries due to incorrect JPA configuration)
I know that the database connection is correct because there is no configuration problems during the test run whenever MyDataSourceWrapperConfiguration is not included in the test. However, once it is added, the circular dependency happens. (see error below) I believe #Primary is necessary in order for the JPA/JDBC code to use the correct DataSource instance.
MyDataSourceWrapper is a custom class that tracks the number of queries that have happened for a given transaction, but it delegates the real database work to the DataSource passed in via constructor.
Error:
The dependencies of some of the beans in the application context form a cycle:
org.springframework.boot.autoconfigure.orm.jpa.HibernateJpaAutoConfiguration
┌─────┐
| databaseQueryCounterProxyDataSource defined in me.testsupport.database.MyDataSourceWrapperConfiguration
↑ ↓
| dataSource defined in org.springframework.boot.autoconfigure.jdbc.DataSourceConfiguration$Tomcat
↑ ↓
| dataSourceInitializer
└─────┘
My Configuration:
#Configuration
public class MyDataSourceWrapperConfiguration {
#Primary
#Bean
DataSource databaseQueryCounterProxyDataSource(final DataSource delegate) {
return MyDataSourceWrapper(delegate);
}
}
My Test:
#ActiveProfiles({ "it" })
#RunWith(SpringJUnit4ClassRunner.class)
#SpringApplicationConfiguration({ DatabaseConnectionConfiguration.class, DatabaseQueryCounterConfiguration.class })
#EnableAutoConfiguration
public class EngApplicationRepositoryIT {
#Rule
public MyDatabaseQueryCounter databaseQueryCounter = new MyDatabaseQueryCounter ();
#Rule
public ErrorCollector errorCollector = new ErrorCollector();
#Autowired
MyRepository repository;
#Test
public void test() {
this.repository.loadData();
this.errorCollector.checkThat(this.databaseQueryCounter.getSelectCounts(), is(lessThan(10)));
}
}
UPDATE: This original question was for springboot 1.5. The accepted answer reflects that, however, the answer from #rajadilipkolli works for springboot 2.x
In your case you will get 2 DataSource instances which is probably not what you want. Instead use BeanPostProcessor which is the component actually designed for this. See also the Spring Reference Guide.
Create and register a BeanPostProcessor which does the wrapping.
public class DataSourceWrapper implements BeanPostProcessor {
public Object postProcessBeforeInitialization(Object bean, String beanName) {
if (bean instanceof DataSource) {
return new MyDataSourceWrapper((DataSource)bean);
}
return bean;
}
public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException {
return bean;
}
}
Then just register that as a #Bean instead of your MyDataSourceWrapper.
Tip: Instead of rolling your own wrapping DataSource you might be interested in datasource-proxy combined with datasource-assert which has counter etc. support already (saves you maintaining your own components).
Starting from spring boot 2.0.0.M3 using BeanPostProcessor wont work.
As a work around create your own bean like below
#Bean
public DataSource customDataSource(DataSourceProperties properties) {
log.info("Inside Proxy Creation");
final HikariDataSource dataSource = (HikariDataSource) properties
.initializeDataSourceBuilder().type(HikariDataSource.class).build();
if (properties.getName() != null) {
dataSource.setPoolName(properties.getName());
}
return ProxyDataSourceBuilder.create(dataSource).countQuery().name("MyDS")
.logSlowQueryToSysOut(1, TimeUnit.MINUTES).build();
}
Another way is to use datasource-proxy version of datasource-decorator starter
Following solution works for me using Spring Boot 2.0.6.
It uses explicit binding instead of annotation #ConfigurationProperties(prefix = "spring.datasource.hikari").
#Configuration
public class DataSourceConfig {
private final Environment env;
#Autowired
public DataSourceConfig(Environment env) {
this.env = env;
}
#Primary
#Bean
public MyDataSourceWrapper primaryDataSource(DataSourceProperties properties) {
DataSource dataSource = properties.initializeDataSourceBuilder().build();
Binder binder = Binder.get(env);
binder.bind("spring.datasource.hikari", Bindable.ofInstance(dataSource).withExistingValue(dataSource));
return new MyDataSourceWrapper(dataSource);
}
}
You can actually still use BeanPostProcessor in Spring Boot 2, but it needs to return the correct type (the actual type of the declared Bean). To do this you need to create a proxy of the correct type which redirects DataSource methods to your interceptor and all the other methods to the original bean.
For example code see the Spring Boot issue and discussion at https://github.com/spring-projects/spring-boot/issues/12592.