#Primary datasource not picked up in testing - spring

I want to test only the persistence layer using #DataJpaTest, and I have two datasource configurations, one in src/main and the other is src/test, and I am using #primary on test datasource to get picked up only, but main datasource get picked up also.
src/main configuration
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(basePackages = "com.****.repository",
entityManagerFactoryRef = "platformEntityManagerFactory",
transactionManagerRef = "platformTransactionManager"
)
#Import(CommonPersistenceConfig.class)
public class PlatformPersistenceConfig {
#Value("classpath:application.yml")
private Resource resource;
#Bean
#Qualifier("platformTransactionManager")
public PlatformTransactionManager platformTransactionManager(#Qualifier("platformEntityManagerFactory") EntityManagerFactory entityManagerFactory) {
// code...
}
#Bean
public LocalContainerEntityManagerFactoryBean platformEntityManagerFactory(DataSource dataSource,
HibernateProperties hibernateProperties) {
// code...
}
#Bean
public DataSource getDataSource() throws IOException, NoSuchPaddingException, NoSuchAlgorithmException, InvalidKeyException, BadPaddingException, IllegalBlockSizeException {
// code...
}
}
src/test configuration
#Configuration
#Import({CommonPersistenceConfig.class, DbPropertyConfig.class})
public class DbTestSetupConfig {
#Autowired
private TestDatasourceProperties dbProperties;
#Primary
#Bean(destroyMethod = "close")
public DataSource getDataSource() throws Exception {
// code...
}
#PostConstruct
public void dbSetup() throws Exception {
// code...
}
}
my test
#DataJpaTest
#AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE)
public class StripeCardRepositoryTest {
#Autowired
StripeCardRepository stripeCardRepository;
#Test
public void test() {
}
}

The point of #AutoConfigureTestDatabase is to replace an existing datasource defined in src/main/.. with a datasource defined in src/test. But to do so you must inform it what strategy it must use. You have chosen AutoConfigureTestDatabase.Replace.NONE therefore it does not replace the main datasource with the one that you need.
Try to switch to AutoConfigureTestDatabase.Replace.ANY and it will correctly replace it with the one defined in test.
Also probably you show this in the documentation
In the case of multiple DataSource beans, only the #Primary
DataSource is considered.
This applies in the case that you have multiple datasources in project defined. It means that it would replace only the primary datasource with the strategy defined. It does not mean that it would pick only the primary and ignore the others.

Related

Why are the data sources interfering in Spring Batch when using a RepositoryItemReader?

I am trying to migrate some data between a Postgres database and MongoDB using Spring Batch. I have a very simple ItemReader, ItemProcessor, and ItemWriter configured, and it everything works as intended. However, if I switch to a RepositoryItemReader, I'm getting the following error:
java.lang.IllegalStateException: Already value [org.springframework.jdbc.datasource.ConnectionHolder#684430c1] for key [HikariDataSource (HikariPool-1)] bound to thread
If I understand correctly, there is something wrong with the EntityManager or TransactionManager, but I cannot figure out what, and why it's working with a simple ItemReader that doesn't work with a repository, but it uses the same data source.
I would be very grateful for any help.
Here is my source db configuration:
package com.example.batch.primary;
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(
entityManagerFactoryRef = "primaryEntityManagerFactory",
transactionManagerRef = "primaryTransactionManager",
basePackages = {"com.example.batch.primary"}
)
public class PrimaryDBConfig {
#Bean(name = "primaryDataSource")
#Primary
public DataSource primaryDatasource(){
DataSourceBuilder dataSourceBuilder = DataSourceBuilder.create()
.driverClassName("org.postgresql.Driver")
.url("jdbc:postgresql://localhost:5432/postgres")
.username("test")
.password("test");
return dataSourceBuilder.build();
}
#Bean(name = "primaryEntityManagerFactory")
#Primary
public LocalContainerEntityManagerFactoryBean primaryEntityManagerFactory(EntityManagerFactoryBuilder builder,
#Qualifier("primaryDataSource")
DataSource primaryDataSource){
return builder.dataSource(primaryDataSource)
.packages("com.example.batch.primary")
.build();
}
#Bean(name = "primaryTransactionManager")
public PlatformTransactionManager primaryTransactionManager(
#Qualifier("primaryEntityManagerFactory") EntityManagerFactory primaryEntityManagerFactory)
{
return new JpaTransactionManager(primaryEntityManagerFactory);
}
}
Here is the configuration of MongoDB:
package com.example.batch.secondary;
#EnableMongoRepositories(basePackages = "com.example.batch.secondary")
#Configuration
public class MongoDBConfig {
#Bean
public MongoClient mongo() {
ConnectionString connectionString = new ConnectionString("mongodb+srv://mongoadmin:blablabla.mongodb.net/?retryWrites=true&w=majority");
MongoClientSettings mongoClientSettings = MongoClientSettings.builder()
.applyConnectionString(connectionString)
.build();
return MongoClients.create(mongoClientSettings);
}
#Bean
public MongoTemplate mongoTemplate() throws Exception {
return new MongoTemplate(mongo(), "test");
}
}
Here is the RepositoryItemReader:
package com.example.batch.stepcomponents;
#Component
public class RepositoryReader extends RepositoryItemReader<Partner> {
public RepositoryReader(#Autowired PartnerRepository partnerRepository){
setRepository(partnerRepository);
setPageSize(1);
setSort(Map.of("id", Sort.Direction.ASC));
setMethodName("findAll");
}
}
Batch Config:
#Configuration
#EnableBatchProcessing
public class BatchConfig {
#Autowired
public JobBuilderFactory jobBuilderFactory;
#Autowired
public StepBuilderFactory stepBuilderFactory;
#Autowired
RepositoryReader repositoryReader;
#Autowired
CustomWriter customWriter;
#Autowired
CustomProcessor customProcessor;
#Bean
public Job createJob() {
return jobBuilderFactory.get("MyJob")
.incrementer(new RunIdIncrementer())
.flow(createStep())
.end()
.build();
}
#Bean
public Step createStep() {
return stepBuilderFactory.get("MyStep")
.<Partner, Student> chunk(1)
.reader(repositoryReader)
.processor(customProcessor)
.writer(customWriter)
.build();
}
}
So I tried taking out the EntityManagerFactory and the TransactionManager, and now it works. I guess they are already initialized automatically when starting up the server..
Yes, by default, if you provide a DataSource bean, Spring Batch will use a DataSourceTransactionManager, not the JPA one as you expect. This is explained in the Javadoc of EnableBatchProcessing:
The transaction manager provided by this annotation will be of type:
* ResourcelessTransactionManager if no DataSource is provided within the context
* DataSourceTransactionManager if a DataSource is provided within the context
In order to use the JPA transaction manager, you need to configure a custom a BatchConfigurer and override getTransactionManager, something like:
#Bean
public BatchConfigurer batchConfigurer(DataSource dataSource, EntityManagerFactory entityManagerFactory) {
return new DefaultBatchConfigurer(dataSource) {
#Override
public PlatformTransactionManager getTransactionManager() {
return new JpaTransactionManager(entityManagerFactory);
}
};
}
Note this will not be required anymore starting from v5, see:
Revisit the configuration of infrastructure beans with #EnableBatchProcessing
Spring Batch 5.0.0-M6 and 4.3.7 are out!
You can also set the JPA transaction manager on your step:
#Bean
public Step createStep(JpaTransactionManager jpaTransactionManager) {
return stepBuilderFactory.get("MyStep")
.<Partner, Student> chunk(1)
.reader(repositoryReader)
.processor(customProcessor)
.writer(customWriter)
.transactionManager(jpaTransactionManager)
.build();
}
Adding 'spring-data-jpa' as a dependency will automatically configure aJpaTransactionManager if no other TransactionManager is defined

SpringBoot JPA TransactionRequiredException when saveAndFlush is called

I'm writing a SpringBoot REST server. I have some problems with the data access layer. In my saveReport() service method, if I call save() method of JPA repository, I can't see any records in the db. If I use saveAndFlush() method of JPA repository, I get the following exception:
Caused by: javax.persistence.TransactionRequiredException: no transaction is in progress
Db config class:
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(
entityManagerFactoryRef = "qaEntityManagerFactory",
basePackages = { "qa.repository" }
)
public class QADbConfig {
#Bean(name = "qaDataSource")
#ConfigurationProperties(prefix = "spring.qa-datasource")
public HikariDataSource dataSource() {
return DataSourceBuilder.create().type(HikariDataSource.class).build();
}
#Bean(name = "qaEntityManagerFactory")
public LocalContainerEntityManagerFactoryBean
entityManagerFactory(
EntityManagerFactoryBuilder builder,
#Qualifier("qaDataSource") DataSource dataSource
) {
return builder
.dataSource(dataSource)
.packages("qa.model")
.persistenceUnit("qa")
.build();
}
#Bean(name = "qaTransactionManager")
public PlatformTransactionManager transactionManager(
#Qualifier("qaEntityManagerFactory") EntityManagerFactory
entityManagerFactory
) {
return new JpaTransactionManager(entityManagerFactory);
}
}
In my service layer, I added #Transactional
#Service
public class ReportService implements IReportService {
static final ObjectMapper objectMapper = new ObjectMapper();
#Autowired
ReportRepository reportJpaRepository;
#Transactional(propagation=Propagation.REQUIRED)
#Override
public Report saveReport(Report report) {
reportJpaRepository.save(report); // I get no transaction exception if I use saveAndFlush method here
return report;
}
Also in the repository class, I added #Transactional
#Transactional
#Repository
public interface ReportJpaRepository extends JpaRepository<Report, Integer> {
List<Report> findByPrivateReport(Boolean privateReport);
}
Do you see what is wrong with my code?
Thanks

problem with writing to 2 databases from Spring boot

I was trying this code from github:https://github.com/kodinor/spring-data-many-dbs
It's an example of how to use multiple db's from a Spring-boot application. it worked fine but I added the Spring-boot-starter-web dependency and now I'm getting an error:
>Method requestMappingHandlerMapping in >org.springframework.boot.autoconfigure.web.servlet.WebMvcAutoConfiguration$En>ableWebMvcConfiguration required a single bean, but 2 were found:
> - productDSEmFactory: defined by method 'productDSEmFactory' in class >path resource [com/kodinor/configuration/ProductDBConfiguration.class]
> - userDSEmFactory: defined by method 'userDSEmFactory' in class path >resource [com/kodinor/configuration/UserDBConfiguration.class]
>
>
>Action:
>
>Consider marking one of the beans as #Primary, updating the consumer to >accept multiple beans, or using #Qualifier to identify the bean that should >be consumed<br>
I have two config files:
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(basePackageClasses = UserRepository.class, entityManagerFactoryRef = "userDSEmFactory", transactionManagerRef = "userDSTransactionManager")
public class UserDBConfiguration {
#Primary
#Bean
#ConfigurationProperties("spring.datasource1")
public DataSourceProperties userDSProperties() {
return new DataSourceProperties();
}
#Primary
#Bean
public DataSource userDS(#Qualifier("userDSProperties") DataSourceProperties userDSProperties) {
return userDSProperties.initializeDataSourceBuilder().build();
}
#Bean
public LocalContainerEntityManagerFactoryBean userDSEmFactory(#Qualifier("userDS") DataSource userDS, EntityManagerFactoryBuilder builder) {
return builder.dataSource(userDS).packages(User.class).build();
}
#Primary
#Bean
public PlatformTransactionManager userDSTransactionManager(EntityManagerFactory userDSEmFactory) {
return new JpaTransactionManager(userDSEmFactory);
}
}
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(basePackageClasses = ProductRepository.class, entityManagerFactoryRef = "productDSEmFactory", transactionManagerRef = "productDSTransactionManager")
public class ProductDBConfiguration {
#Bean
#ConfigurationProperties("spring.datasource2")
public DataSourceProperties productDSProperties() {
return new DataSourceProperties();
}
#Bean
public DataSource productDS(#Qualifier("productDSProperties") DataSourceProperties productDSProperties) {
return productDSProperties.initializeDataSourceBuilder().build();
}
#Bean
public LocalContainerEntityManagerFactoryBean productDSEmFactory(#Qualifier("productDS") DataSource productDS, EntityManagerFactoryBuilder builder) {
return builder.dataSource(productDS).packages(Product.class).build();
}
#Bean
public PlatformTransactionManager productDSTransactionManager(EntityManagerFactory productDSEmFactory) {
return new JpaTransactionManager(productDSEmFactory);
}
}
Application.properties:
spring.jpa.hibernate.ddl-auto=create
spring.jpa.generate-ddl=true
spring.datasource1.url=jdbc:mysql://localhost:3306/userdb?autoReconnect=true&useSSL=false
spring.datasource1.username=user
spring.datasource1.password=pass
spring.datasource2.url=jdbc:mysql://localhost:3306/productdb?autoReconnect=true&useSSL=false
spring.datasource2.username=user
spring.datasource2.password=pass
And a simple main app that adds some init data:
#SpringBootApplication
public class SpringDataManyDbsApplication {
public static void main(String[] args) {
SpringApplication.run(SpringDataManyDbsApplication.class, args);
}
#Autowired
private UserRepository userRepository;
#Autowired
private ProductRepository productRepository;
#PostConstruct
void init () {
User user = new User("john", "doe");
userRepository.save(user);
Product product = new Product("chair", 5);
productRepository.save(product);
}
}
I tried to add the #Primary annotation to userDSEmFactory. The error goes away but the product data isn't saved anymore. Any ideas how to save this problem? I don't have a lot of experience with Spring-boot and I've read dozens of articles but many seem to do things in a different way. Thanks so much for helping me out!
update
I've added the #Primary annotation like this:
#Primary
#Bean
public LocalContainerEntityManagerFactoryBean userDSEmFactory(#Qualifier("userDS") DataSource userDS, EntityManagerFactoryBuilder builder) {
return builder.dataSource(userDS).packages(User.class).build();
}
No there are no more errors, but only the user is being saved in the primary db and nothing is being saved to the product (second) database.
So if anybody has suggestions what could be the cause of that..
You are missing #Primary annotation on your userDSEmFactory
#Primary
#Bean
public LocalContainerEntityManagerFactoryBean userDSEmFactory(#Qualifier("userDS") DataSource userDS, EntityManagerFactoryBuilder builder) {
return builder.dataSource(userDS).packages(User.class).build();
}
And also it is goot to use once #EnableTransactionManagement at your main class.

Spring jdbc configuration

I have been trying to implement a web service using spring. This webservice will provide data access to a mySQL database using JDBC. I am trying to not use any xml configuration files, so I have come across a problem trying to connect to the database.
I am following the tutorial: http://spring.io/guides/tutorials/rest/ but I changed a few things along the way.
Now that I am trying to implement the connection with the database I get an error when trying to execute the tomcat instance, and I guess the problem is within the configurations.
Here follows some of my code:
Datasource configuration:
#Configuration
#Profile("mySQL")
#PropertySource("classpath:/services.properties")
public class MySQLDataSourceConfiguration implements DataSourceConfiguration{
#Inject
private Environment environment;
#Bean
public DataSource dataSource() throws Exception {
BasicDataSource dataSource = new BasicDataSource();
dataSource.setPassword(environment.getProperty("dataSource.password"));
dataSource.setUrl(environment.getProperty("dataSource.url"));
dataSource.setUsername(environment.getProperty("dataSource.user"));
dataSource.setDriverClassName(environment.getPropertyAsClass("dataSource.driverClass", Driver.class).getName());
return dataSource;
}
}
the file service.properties is where I keep my configurations for the database, so when I desire to change the database I will just have to change 4 fields.
The JDBCConfiguration class for the setup of the JDBCtemplate
#Configuration
#EnableTransactionManagement
#PropertySource("classpath:/services.properties")
#Import( { MySQLDataSourceConfiguration.class })
public class JdbcConfiguration {
#Autowired
private DataSourceConfiguration dataSourceConfiguration;
#Inject
private Environment environment;
#Bean
public JdbcTemplate setupJdbcTemplate() throws Exception {
return new JdbcTemplate(dataSourceConfiguration.dataSource());
}
#Bean
public PlatformTransactionManager transactionManager(DataSource dataSource) throws Exception {
return new DataSourceTransactionManager(dataSource);
}
}
Then there is the Repository, that recieves the template.
#Transactional
#Repository
#Qualifier("jdbcRepository")
public class JdbcIndividualRepository implements IndividualsRepository{
private static final Logger LOG = LoggerFactory.getLogger(JdbcIndividualRepository.class);
#Autowired
private JdbcTemplate jdbcTemplate;
#Autowired
public JdbcIndividualRepository(DataSource jdbcDataSource) {
LOG.info("JDBCRepo arg constructor");
this.jdbcTemplate = new JdbcTemplate(jdbcDataSource);
}
#Override
public Individual save(Individual save) {
String sql = "INSERT INTO Individual(idIndividual, Name) VALUES(?,?)";
this.jdbcTemplate.update(sql, save.getId(), save.getName());
return save;
}
#Override
public void delete(String key) {
String sql = "DELETE FROM Individual WHERE idIndividual=?";
jdbcTemplate.update(sql, key);
}
#Override
public Individual findById(String key) {
String sql = "SELECT i.* FROM Individual i WHERE i.idIndividual=?";
return this.jdbcTemplate.queryForObject(sql, new IndividualRowMapper(), key);
}
#Override
public List<Individual> findAll() {
String sql = "SELECT * FROM Individual";
return new LinkedList<Individual>(this.jdbcTemplate.query(sql, new IndividualRowMapper()));
}
}
Then I register the jdbc configuration in the initializer class when creating the root context of the application as follows:
private WebApplicationContext createRootContext(ServletContext servletContext) {
AnnotationConfigWebApplicationContext rootContext = new AnnotationConfigWebApplicationContext();
rootContext.register(CoreConfig.class, SecurityConfig.class, JdbcConfiguration.class);
rootContext.refresh();
servletContext.addListener(new ContextLoaderListener(rootContext));
servletContext.setInitParameter("defaultHtmlEscape", "true");
return rootContext;
}
However, the Tomcat server wont run because it can't autowire the class MySQLDataSourceConfiguration.
Anyone knows what the problem might be? I can give more details on the code, but the question is already really large.
Appreciate any kind of help!
Cheers
EDIT
Solved changing the JdbcConfiguration class to:
#Configuration
#EnableTransactionManagement
#PropertySource("classpath:/services.properties")
#Import( { MySQLDataSourceConfiguration.class })
public class JdbcConfiguration {
#Autowired
private DataSource dataSource;
#Inject
private Environment environment;
#Bean
public JdbcTemplate setupJdbcTemplate() throws Exception {
return new JdbcTemplate(dataSource);
}
#Bean
public PlatformTransactionManager transactionManager(DataSource dataSource) throws Exception {
return new DataSourceTransactionManager(dataSource);
}
#Bean
public IndividualsRepository createRepo(){
return new JdbcIndividualRepository(dataSource);
}
}
Remove
#Autowired
private DataSourceConfiguration dataSourceConfiguration;
Because that's not how it's supposed to be used. Instead add to the same class the following:
#Autowired DataSource dataSource;
and use it like this: new JdbcTemplate(dataSource);
Also, try adding #ComponentScan to JdbcConfiguration class. From what I see in your code the class JdbcIndividualRepository is not picked up by anything.
In your class JdbcConfiguration, you are trying to autowire DataSourceConfiguration. I'm not really sure if that's possible - typically you should try to autwire the DataSource, not the DataSourceConfiguration.
#Import( { MySQLDataSourceConfiguration.class })
public class JdbcConfiguration {
#Autowired
private DataSource dataSource;
#Bean
public JdbcTemplate setupJdbcTemplate() throws Exception {
return new JdbcTemplate(dataSource);
}
Also if you have several DataSources and you're using Spring profiles to separate them, it's easier to provide all the DataSource beans in one file and annotate each bean with a different profile:
#Configuration
public class DataSourceConfig {
#Bean
#Profile("Test")
public DataSource devDataSource() {
.... configure data source
}
#Bean
#Profile("Prod")
public DataSource prodDataSource() {
... configure data source
}

Multiple transaction managers annotation configuration

I have two transaction managers configured in annotation-based configuration class:
#Configuration
#EnableTransactionManagement
public class DBConfig implements TransactionManagementConfigurer {
//...
#Override
public PlatformTransactionManager annotationDrivenTransactionManager() {
return defTransactionManager();
}
#Bean
#Qualifier("defSessionFactory")
public LocalSessionFactoryBean defSessionFactory() {
LocalSessionFactoryBean sfb = new LocalSessionFactoryBean();
sfb.setDataSource(defDataSource());
Properties props = new Properties();
//...
sfb.setHibernateProperties(props);
sfb.setPackagesToScan("my.package");
return sfb;
}
#Bean
#Qualifier("defTransactionManager")
public PlatformTransactionManager defTransactionManager() {
return new HibernateTransactionManager(defSessionFactory().getObject());
}
#Bean
#Qualifier("secondSessionFactory")
public LocalSessionFactoryBean secondSessionFactory() {
LocalSessionFactoryBean sfb = new LocalSessionFactoryBean();
sfb.setDataSource(secondDataSource());
Properties props = new Properties();
//...
sfb.setHibernateProperties(props);
sfb.setPackagesToScan("my.package.subpackage");
return sfb;
}
#Bean
#Qualifier("secondTM")
public PlatformTransactionManager secondTransactionManager() {
return new HibernateTransactionManager(secondSessionFactory().getObject());
}
}
My intention is use annotation transactions with two transaction managers.
Methonds annotated like this
#Transactional
public void method() {}
should be handled by defTransactionManager
And methods annotated like this
#Transactional("secondTM")
public void anotherMethod() {}
by secondTransactionManager
defTransactionManager works fine but when it comes to anotherMethod() I get:
org.hibernate.HibernateException: No Session found for current thread
When I use programmatic transaction management for anotherMethod (autowire secondSessionFactory, use TransactionTemplate) everything works fine.
In case of #EnableTranscationManagement Spring will use by-type lookup, you can provide your own lookup method to a single transaction manager, but it will not work for two tx managers
If you want to check how Spring determines the transaction to execute, you can try to debug the TransactionAspectSupport class. The key methods are setTransactionManagerBeanName and determineTransactionManager.
Just in case anyone runs into this problem, I found a solution:
#Configuration
#EnableTransactionManagement
#DependsOn("myTxManager")
#ImportResource("classpath:applicationContext.xml")
public class AppConfig implements TransactionManagementConfigurer {
#Autowired
private PlatformTransactionManager myTxManager;
...
#Override
public PlatformTransactionManager annotationDrivenTransactionManager() {
return this.myTxManager;
}
In this way, you can use a specific txManager defined in an xml configuration.
In case you want to define the txManager used on service-level, you shall remove the #EnableTransactionManagement annotation from the #Configuration class and specify the txManager in the #Transactional annotations, e.g.
#Service
#Transactional(value="myTxManager", readOnly = true)
public class MyServiceImpl implements MyService { ... }

Resources