How to use two data source for same entity in spring boot application running on tomcat - spring

Question

I have one micro-service named order-service in spring boot.
:: Spring Boot :: (v2.0.5.RELEASE)
in that micro-service, in service layer I have 2 service class 1. OrderManagerService and OrderService. From OrderService I call Repository Interface IOrderRequestRepository which extends PagingAndSortingRepository interface. so The call is basically OrderManagerService -> OrderService -> IOrderRequestRepository.
Right now I am using one data source for the micro-service. Now my requirement is I have to use 2 data source because in OrderManagerService there is one method getCustomerVisitsForMonth(Long customerID) which frequently used and we don't want to give load on prod DB server. Instead we will use prod-replica server for this one particular method.
I have googled a lot of blogs and tried to implement but ending up getting different exception.
below are exceptions:
could not execute query; SQL [SELECT * FROM ....(this is the actual query)]; nested exception is org.hibernate.exception.SQLGrammarException: could not execute query",
Could not open JPA EntityManager for transaction; nested exception is org.hibernate.TransactionException: JDBC begin transaction failed:
Below are my sample code:
application-env.properties
spring.datasource.url=jdbc:mysql://someurl:3306/orders?zeroDateTimeBehavior=convertToNull&useSSL=false
spring.datasource.username=<username>
spring.datasource.password=<password>
spring.datasource.driverClassName=com.mysql.jdbc.Driver
#replica datasource details
spring.replica.datasource.url=jdbc:mysql://someurl:3306/order?zeroDateTimeBehavior=convertToNull&useSSL=false
spring.replica.datasource.username=<username>
spring.replica.datasource.password=<password>
spring.replica.datasource.driverClassName=com.mysql.jdbc.Driver
spring.datasource.tomcat.initial-size=15
spring.datasource.tomcat.max-wait=20000
spring.datasource.tomcat.max-active=5
spring.datasource.tomcat.max-idle=4
spring.datasource.tomcat.min-idle=2
spring.datasource.tomcat.default-auto-commit=true
#replica data source details
spring.replica.datasource.tomcat.initial-size=15
spring.replica.datasource.tomcat.max-wait=20000
spring.replica.datasource.tomcat.max-active=5
spring.replica.datasource.tomcat.max-idle=4
spring.replica.datasource.tomcat.min-idle=2
spring.replica.datasource.tomcat.default-auto-commit=true
Main class
package in.demo.order.web.app;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.autoconfigure.domain.EntityScan;
import org.springframework.boot.autoconfigure.security.servlet.SecurityAutoConfiguration;
import org.springframework.boot.builder.SpringApplicationBuilder;
import org.springframework.boot.web.servlet.support.SpringBootServletInitializer;
import org.springframework.data.jpa.repository.config.EnableJpaRepositories;
import org.springframework.transaction.annotation.EnableTransactionManagement;
#SpringBootApplication(scanBasePackages = "in.demo.order",
exclude = { SecurityAutoConfiguration.class })
public class OrderServiceApplication extends SpringBootServletInitializer {
public static void main(String[] args) {
SpringApplication.run(OrderServiceApplication.class, args);
}
#Override
protected SpringApplicationBuilder configure(SpringApplicationBuilder builder) {
return builder.sources(OrderServiceApplication.class);
}
}
I have created 2 different packages for Repository classes and for Entity classes, however my Entity and Repository are same.
packages
in.demo.order.dao
in.demo.order.repodao
Configuration classes
MainDataSourceConfiguration
package in.demo.order.web.config;
import in.demo.order.replicadao.IOrderRequestRepositoryForReplica;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.autoconfigure.jdbc.DataSourceProperties;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.boot.orm.jpa.EntityManagerFactoryBuilder;
import org.springframework.context.annotation.*;
import org.springframework.core.env.Environment;
import org.springframework.data.jpa.repository.config.EnableJpaRepositories;
import org.springframework.orm.jpa.JpaTransactionManager;
import org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean;
import org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter;
import org.springframework.transaction.PlatformTransactionManager;
import org.springframework.transaction.annotation.EnableTransactionManagement;
import javax.persistence.EntityManagerFactory;
import javax.sql.DataSource;
import java.util.HashMap;
import java.util.Map;
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(
basePackages = "in.demo.order.dao",
entityManagerFactoryRef = "mainEntityManagerFactory",
transactionManagerRef = "mainTransactionManager"
)
public class MainDataSourceConfiguration {
#Autowired
private Environment environment;
#Primary
#Bean(name = "mainDataSourceProperties")
#ConfigurationProperties("spring.datasource")
public DataSourceProperties mainDataSourceProperties() {
return new DataSourceProperties();
}
#Primary
#Bean(name = "mainDataSource")
#ConfigurationProperties("spring.datasource.configuration")
public DataSource mainDataSource(
#Qualifier("mainDataSourceProperties") DataSourceProperties mainDataSourceProperties) {
return mainDataSourceProperties
.initializeDataSourceBuilder()
.type(org.apache.tomcat.jdbc.pool.DataSource.class)
.build();
}
#Primary
#Bean(name = "mainEntityManagerFactory")
public LocalContainerEntityManagerFactoryBean mainEntityManagerFactory(
EntityManagerFactoryBuilder mainEntityManagerFactoryBuilder,
#Qualifier("mainDataSource") DataSource mainDataSource) {
Map<String, String> mainJpaProperties = new HashMap<>();
mainJpaProperties.put("hibernate.dialect",
environment.getProperty("spring.jpa.properties.hibernate.dialect"));
mainJpaProperties.put("hibernate.hbm2ddl.auto",
environment.getProperty("spring.jpa.hibernate.ddl-auto"));
LocalContainerEntityManagerFactoryBean localContainerEntityManagerFactoryBean = mainEntityManagerFactoryBuilder
.dataSource(mainDataSource)
.packages("in.demo.order.dao.model")
.persistenceUnit("mainDataSource")
.properties(mainJpaProperties)
.build();
localContainerEntityManagerFactoryBean.setJpaVendorAdapter(new HibernateJpaVendorAdapter());
return localContainerEntityManagerFactoryBean;
}
#Primary
#Bean(name = "mainTransactionManager")
public PlatformTransactionManager mainTransactionManager(
#Qualifier("mainEntityManagerFactory") EntityManagerFactory mainEntityManagerFactory) {
return new JpaTransactionManager(mainEntityManagerFactory);
}
}
ReplicaDataSourceConfiguration
package in.demo.order.web.config;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.autoconfigure.jdbc.DataSourceProperties;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.boot.orm.jpa.EntityManagerFactoryBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.env.Environment;
import org.springframework.data.jpa.repository.config.EnableJpaRepositories;
import org.springframework.orm.jpa.JpaTransactionManager;
import org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean;
import org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter;
import org.springframework.transaction.PlatformTransactionManager;
import org.springframework.transaction.annotation.EnableTransactionManagement;
import javax.persistence.EntityManagerFactory;
import javax.sql.DataSource;
import java.util.HashMap;
import java.util.Map;
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(
basePackages = "in.demo.order.replicadao",
entityManagerFactoryRef = "replicaEntityManagerFactory",
transactionManagerRef = "replicaTransactionManager"
)
public class ReplicaDataSourceConfiguration {
#Autowired
private Environment environment;
#Bean(name = "replicaDataSourceProperties")
#ConfigurationProperties("spring.replica.datasource")
public DataSourceProperties replicaDataSourceProperties() {
return new DataSourceProperties();
}
#Bean(name = "replicaDataSource")
#ConfigurationProperties("spring.replica.datasource.configuration")
public DataSource replicaDataSource(
#Qualifier("replicaDataSourceProperties") DataSourceProperties replicaDataSourceProperties) {
return replicaDataSourceProperties
.initializeDataSourceBuilder()
.type(org.apache.tomcat.jdbc.pool.DataSource.class)
.build();
}
#Bean(name = "replicaEntityManagerFactory")
public LocalContainerEntityManagerFactoryBean replicaEntityManagerFactory(
EntityManagerFactoryBuilder replicaEntityManagerFactoryBuilder,
#Qualifier("replicaDataSource") DataSource replicaDataSource) {
Map<String, String> replicaJpaProperties = new HashMap<>();
replicaJpaProperties.put("hibernate.dialect",
environment.getProperty("spring.jpa.properties.hibernate.dialect"));
replicaJpaProperties.put("hibernate.hbm2ddl.auto",
environment.getProperty("spring.jpa.hibernate.ddl-auto"));
LocalContainerEntityManagerFactoryBean localContainerEntityManagerFactoryBean = replicaEntityManagerFactoryBuilder
.dataSource(replicaDataSource)
.packages("in.demo.order.replicadao.model")
.persistenceUnit("replicaDataSource")
.properties(replicaJpaProperties)
.build();
localContainerEntityManagerFactoryBean.setJpaVendorAdapter(new HibernateJpaVendorAdapter());
return localContainerEntityManagerFactoryBean;
}
#Bean(name = "replicaTransactionManager")
public PlatformTransactionManager replicaTransactionManager(
#Qualifier("replicaEntityManagerFactory") EntityManagerFactory replicaEntityManagerFactory) {
return new JpaTransactionManager(replicaEntityManagerFactory);
}
}
OrderManagerService
package in.demo.order.service;
import javax.transaction.Transactional;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
#Service
#Transactional
public class OrderManagerService{
#Autowired
private OrderService orderService;
public CustomerVisitsResponse getCustomerVisitsForMonthForReplicaDataSource(Long customerId) {
return orderService.getTotalCustomerVisitsForMonth(customerId);
}
public CustomerVisitsResponse getCustomerVisitsForMonthForMainDataSource(Long customerId) {
return orderService.getTotalCustomerVisitsForMonthForMainDataSource(customerId);
}
}
OrderService
package in.demo.order.service;
import java.text.DateFormat;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.util.ArrayList;
import java.util.Date;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
import in.demo.order.replicadao.IOrderRequestRepositoryForReplica;
import in.demo.order.dao.IOrderRequestRepository;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;
import org.springframework.transaction.annotation.Transactional;
#Service
#Transactional(rollbackFor = Exception.class)
public class OrderService {
#Autowired
private IOrderRequestRepositoryForMain orderRequestRepositoryForMain;
#Autowired
private IOrderRequestRepositoryForReplica orderRequestRepositoryForReplica;
public CustomerVisitsResponse getTotalCustomerVisitsForMonthForReplicaDataSource(Long customerId) {
List<Integer> statusIdList= new ArrayList<>();
CustomerVisitsResponse response= new CustomerVisitsResponse();
statusIdList.add(OrderServiceConstants.SOME_CONSTANT_VALUE);
List<Integer> sellerIdList= new ArrayList<>();
sellerIdList.add(OrderServiceConstants.SOME_CONSTANT_VALUE);
SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM");
Date date = new Date();
String month= dateFormat.format(date)+"%";
try {
List<in.demo.order.replicadao.model.OrderEntity> orders =
orderRequestRepositoryForReplica.getOrderBycustomerIdAndBdMonth(customerId,statusIdList,month,sellerIdList);
if(orders!=null && !orders.isEmpty())
{
double total=0d;
for(in.demo.order.replicadao.model.OrderEntity order:orders)
{
total+=order.getPayableAmount();
}
response.setTotalSpent(total);
response.setTotalVisits((long)(orders.size()));
}
else {
response.setTotalSpent(0d);
response.setTotalVisits(0l);
}
} catch (Exception e) {
logger.info("Error occured while fetching getTotalCustomerVisitsForMonth customerid: " + customerId);
response.setTotalSpent(0d);
response.setTotalVisits(0l);
return response;
}
return response;
}
public CustomerVisitsResponse getTotalCustomerVisitsForMonthForMainDataSource(Long customerId) {
List<Integer> statusIdList= new ArrayList<>();
CustomerVisitsResponse response= new CustomerVisitsResponse();
statusIdList.add(OrderServiceConstants.SOME_CONSTANT_VALUE);
List<Integer> sellerIdList= new ArrayList<>();
sellerIdList.add(OrderServiceConstants.SOME_CONSTANT_VALUE);
SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM");
Date date = new Date();
String month= dateFormat.format(date)+"%";
try {
List<in.demo.order.dao.model.OrderEntity> orders =
orderRequestRepositoryForMain.getOrderBycustomerIdAndBdMonth(customerId,statusIdList,month,sellerIdList);
if(orders!=null && !orders.isEmpty())
{
double total=0d;
for(in.demo.order.dao.model.OrderEntity order:orders)
{
total+=order.getPayableAmount();
}
response.setTotalSpent(total);
response.setTotalVisits((long)(orders.size()));
}
else {
response.setTotalSpent(0d);
response.setTotalVisits(0l);
}
} catch (Exception e) {
logger.info("Error occured while fetching getTotalCustomerVisitsForMonth customerid: " + customerId);
response.setTotalSpent(0d);
response.setTotalVisits(0l);
return response;
}
return response;
}
}
Repository classes
package in.demo.order.replicadao;
IOrderRequestRepositoryForReplica
package in.demo.order.replicadao;
import in.demo.order.replicadao.model.OrderEntity;
import org.springframework.data.jpa.repository.Query;
import org.springframework.data.repository.PagingAndSortingRepository;
import org.springframework.stereotype.Repository;
import java.util.List;
#Repository
public interface IOrderRequestRepositoryForReplica extends PagingAndSortingRepository<OrderEntity, Long> {
#Query(nativeQuery = true,
value = "SELECT * FROM orders ...<some query>")
public List<OrderEntity> getOrderBycustomerIdAndBdMonth(
Long customerId, List<Integer> statusIdList, String month, List<Integer> sellerIdList);
}
package in.demo.order.dao;
IOrderRequestRepositoryForMain
package in.demo.order.dao;
import java.util.List;
import org.springframework.data.jpa.repository.Query;
import org.springframework.data.repository.PagingAndSortingRepository;
import org.springframework.stereotype.Repository;
import in.demo.order.dao.model.OrderEntity;
#Repository
public interface IOrderRequestRepositoryForMain extends PagingAndSortingRepository<OrderEntity, Long> {
#Query(nativeQuery = true,
value = "SELECT * FROM orders ...<some query>")
public List<OrderEntity> getOrderBycustomerIdAndBdMonth(Long customerId,
List<Integer> statusIdList, String month, List<Integer> sellerIdList);
}
Same Entity in different package with different serialVersionUID
packages are:
package in.demo.order.replicadao.model;
package in.demo.order.dao.model;
OrderEntity
package in.demo.order.replicadao.model;
import java.io.Serializable;
import javax.persistence.Column;
import javax.persistence.Entity;
import javax.persistence.GeneratedValue;
import javax.persistence.GenerationType;
import javax.persistence.Id;
import javax.persistence.Table;
#Entity
#Table(name = "orders")
public class OrderEntity implements Serializable {
/**
*
*/
private static final long serialVersionUID = 4667868888563693990L;
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#Column(unique = true)
private String orderId;
public Long getId() {
return id;
}
public void setId(Long id) {
this.id = id;
}
public String getOrderId() {
return orderId;
}
public void setOrderId(String orderId) {
this.orderId = orderId;
}
}
Controller class
OrderController
package in.demo.order.controller;
import java.util.List;
import javax.sql.DataSource;
import in.demo.order.service.OrderManagerService;
#RestController
#RequestMapping("/api/order")
public class OrderController {
#Autowired
private OrderManagerService orderManagerService;
#Autowired
private DataSource mainDataSource;
#Autowired
#Qualifier("replicaDataSource")
private DataSource replicaDataSource;
#GetMapping("/replica")
public CustomerVisitsResponse getCustomerVisitResponseForReplica() {
Long customerId = 123L;
return orderManagerService.getCustomerVisitsForMonthForReplicaDataSource(customerId);
}
#GetMapping("/main")
public CustomerVisitsResponse getCustomerVisitResponseForMain() {
Long customerId = 123L;
return orderManagerService.getCustomerVisitsForMonthForMainDataSource(customerId);
}
#GetMapping("/maindatasource")
public String getMainDataSource() {
return mainDataSource.toString();
}
#GetMapping("/replicadatasource")
public String getReplicaDataSource() {
return replicaDataSource.toString();
}
}
I have checked manually in both Database for the query I am running. Data is present. But when I am running from postman to test
for replica, I am getting
Method threw 'org.springframework.dao.InvalidDataAccessResourceUsageException' exception. caused by org.hibernate.exception.SQLGrammarException: could not execute query
for main, I am getting
Transaction was marked for rollback only; cannot commit; nested exception is org.hibernate.TransactionException: Transaction was marked for rollback only; cannot commit
When I am using one datasource, the query is working fine.

Answer 1

The issue was with Entity class column name mapping issue, previously we were not mentioning #Column(name="column_name") , once I added the annotation, it started to work. But still I am confused how it is working fine when using single Data source