I am trying to write a DB sync utility with Spring JPA, where in I have a Source DB and Target DB, having same schema. How do I reuse the Entity objects, Repositories? Currently I am running into issue with a repository bean not being able to be instantiated as it is already associated with first (Source) datasource.
I believe , #Profile cannot be used here , as the Source and Target beans have to co-exist.
Appreciate your thoughts.
It is possible to create several data sources with Spring.
#Configuration
public class JpaConfig {
#Bean(name = "source")
public DataSource h2DataSource() {
DataSourceBuilder dataSourceBuilder = DataSourceBuilder.create();
dataSourceBuilder.driverClassName("org.h2.Driver");
dataSourceBuilder.url("jdbc:h2:file:C:/temp/test");
dataSourceBuilder.username("sa");
dataSourceBuilder.password("");
return dataSourceBuilder.build();
}
#Bean(name = "target")
#Primary
public DataSource mySqlDataSource() {
DataSourceBuilder dataSourceBuilder = DataSourceBuilder.create();
dataSourceBuilder.url("jdbc:mysql://localhost/testdb");
dataSourceBuilder.username("dbuser");
dataSourceBuilder.password("dbpass");
return dataSourceBuilder.build();
}
}
Then, you can get the datasources from the Dependency Injection. You can name them source and target so you can keep track of them.
#Qualifier("source")
DataSource source;
#Qualifier("target")
DataSource target;
Here's the solution to your issue with some theoretical background to make understanding easier. Comes with a working example on Github.
I suggest you clone the repo, get it to run and work and then copy the necessary pieces of the code.
https://reflectoring.io/flyway-spring-boot-multitenancy/
Related
I need to build a connection to db2 database, but I can not use application.properties at this moment, so how can I create this connection?
You can implement a datasource bean in the app itself. For example:
#Configuration
public class MyClass {
#Bean
public DataSource getDataSource() {
DataSourceBuilder<?> dataSourceBuilder = DataSourceBuilder.create();
dataSourceBuilder.driverClassName(JDBC_DRIVER_CLASS_NAME);
dataSourceBuilder.url(myprops.getJDBCUrl());
dataSourceBuilder.username(myprops.getJDBCUsername());
dataSourceBuilder.password(myprops.getJDBCPassword());
return dataSourceBuilder.build();
}
}
Then, make sure any autowired references to the data source are "lazy", so the app has time to load the properties (maybe from another file) and to instantiate the data source, as in:
#Lazy
#Autowired
private DataSource dataSource;
I'm trying to write an application that accesses data from two sources. I'm using Spring Boot 2.3.2. I've looked at several sources for info about how to configure the app: the Spring documentation talks about setting up multiple datasources, but does not explain how to link up JPA repositories. This Baeldung article goes a lot further, but I'm looking to take advantage of autoconfiguration in Spring.
So far, I've created a separate package, added a config class (along with model and repositories), and included this package in scanBasePackages so that it's picked up. Since I'll have more than one datasource, I've added this to my #SpringBootApplication:
#Bean
#Primary
public DataSourceProperties dataSourceProperties() {
return new DataSourceProperties();
}
This successfully loads up my Spring app using the standard spring config values. The two databases are on different servers, but should share characteristics (other than url and credentials).
So, my auxiliary configuration file looks like this
#Configuration
#EnableAutoConfiguration
#EnableTransactionManagement
#EnableJpaRepositories(
entityManagerFactoryRef = "orgEntityManagerFactory",
transactionManagerRef = "orgTransactionManager",
basePackages = {
"pacage2.repositories"
}
)
public class DataSourceConfiguration {
// added because of this answer: https://stackoverflow.com/a/51305724/167889
#Bean
public EntityManagerFactoryBuilder entityManagerFactoryBuilder() {
return new EntityManagerFactoryBuilder(new HibernateJpaVendorAdapter(), new HashMap<>(), null);
}
#Bean
#ConfigurationProperties(prefix = "external.datasource")
public DataSourceProperties orgDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
public HikariDataSource orgDataSource(#Qualifier("orgDataSourceProperties") DataSourceProperties properties) {
return properties.initializeDataSourceBuilder().type(HikariDataSource.class)
.build();
}
#Bean
public LocalContainerEntityManagerFactoryBean orgEntityManagerFactory(EntityManagerFactoryBuilder builder,
#Qualifier("orgDataSource") DataSource dataSource) {
return builder
.dataSource(dataSource)
.packages("package2.model")
.build();
}
#Bean
public PlatformTransactionManager orgTransactionManager(
#Qualifier("orgEntityManagerFactory") EntityManagerFactory entityManagerFactory
) {
return new JpaTransactionManager(entityManagerFactory);
}
}
Now, the error I'm getting right now is Access to DialectResolutionInfo cannot be null when 'hibernate.dialect' not set. However, I have that value in my config and it's applied by the Spring auto config. I believe it needs to be set in the EntityManagerFactoryBuilder and by creating my own, the autoconfig isn't getting applied.
How can I have my cake and eat it too? I'd like to leverage as much of the robust autoconfiguration that Spring provides to setup datasources and wire them to the appropriate repositories. Effectively, all that I want to change is the url and credentials, and I can separate the entities and repositories into a completely separate package for easy scanning.
I would like to create a stateless bean in Spring boot which enable user to connect to a specific database, so I started with this code, but i'm still new with spring boot.
#Bean
#Primary
public DataSource helloDataSource() {
return DataSourceBuilder.create()
.username("myUsername")
.password("myPassword")
.driverClassName("myDBDriver")
.build();
}
So what is the best way to follow to make this code work and connect to any database ( Also remote Database )
There are several ways that this could be done. One of my preferred way is to provide configuration of data sources via properties file. Here is a sample property file for postgresql:
pg.datasource.url=jdbc:postgresql://db-server-bar:5432/app-db
pg.datasource.username=root
pg.datasource.password=toor
pg.datasource.driver-class-name=org.postgresql.Driver
Now you can create configuration class for each datasource:
public class BarDbConfig {
#Bean(name = "pgDataSource")
#ConfigurationProperties(prefix = "pg.datasource")
public DataSource dataSource() {
return DataSourceBuilder.create().build();
}
#Bean(name = "barEntityManagerFactory")
public LocalContainerEntityManagerFactoryBean
barEntityManagerFactory(
EntityManagerFactoryBuilder builder,
#Qualifier("pgDataSource") DataSource dataSource
) {
return
builder
.dataSource(dataSource)
.packages("com.app.domain")
.persistenceUnit("pg")
.build();
}
}
For a detailed tutorial, you can refer to this tutorial.
There can also be situations where you would like to switch databases at runtime. In which case, you can use something called AbstractRoutingDataSource. A detailed tutorial on how to use this feature can be found at Spring's official blog site.
I am using Spring Boot 1.4.1 which uses Hibernate 5.0.11. Initially I configured a data source using application.properties like this:
spring.datasource.uncle.url=jdbc:jtds:sqlserver://hostname:port/db
spring.datasource.uncle.username=user
spring.datasource.uncle.password=password
spring.datasource.uncle.dialect=org.hibernate.dialect.SQLServer2012Dialect
spring.datasource.uncle.driverClassName=net.sourceforge.jtds.jdbc.Driver
I configured it with "uncle" because that will be the name of one of multiple data sources that I'll configure. I configured this data source like this, according to the Spring docs:
#Bean
#Primary
#ConfigurationProperties(prefix = "spring.datasource.uncle")
public DataSource uncleDataSource() {
return DataSourceBuilder.create().build();
}
At this point everything worked fine.
I created an #Entity class without any #Column annotations and let Hibernate figure out the column names, for example if I have a Java property named idBank, Hibernate will automatically assume the column name is id_bank. This is used when generating ddl, running SQL statements, etc. I want to utilize this feature because I'm going to have a lot of entity classes and don't want to have to create and maintain all of the #Column annotations. At this point, this worked fine.
I then added another data source like this:
spring.datasource.aunt.url=jdbc:sybase:Tds:host2:port/db2
spring.datasource.aunt.username=user2
spring.datasource.aunt.password=password2
spring.datasource.aunt.dialect=org.hibernate.dialect.SybaseDialect
spring.datasource.aunt.driverClassName=com.sybase.jdbc4.jdbc.SybDriver
... and also this, following the Spring docs for setting up multiple data sources. Apparently once you define a 2nd data source, it can't configure the default beans and you have to define your own EntityManager and TransactionManager. So in addition to the data source configured above, I added these configurations:
#Bean
#Primary
PlatformTransactionManager uncleTransactionManager(#Qualifier("uncleEntityManagerFactory") final EntityManagerFactory factory) {
return new JpaTransactionManager(factory);
}
#Bean
#Primary
LocalContainerEntityManagerFactoryBean uncleEntityManagerFactory(
EntityManagerFactoryBuilder builder) {
return builder
.dataSource(uncleDataSource())
.packages(Uncle.class)
.persistenceUnit("uncle")
.build();
}
#Bean
#ConfigurationProperties(prefix = "spring.datasource.aunt")
public DataSource auntDataSource() {
return DataSourceBuilder.create().build();
}
#Bean
PlatformTransactionManager auntTransactionManager(#Qualifier("auntEntityManagerFactory") final EntityManagerFactory factory) {
return new JpaTransactionManager(factory);
}
#Bean
LocalContainerEntityManagerFactoryBean auntEntityManagerFactory(
EntityManagerFactoryBuilder builder) {
return builder
.dataSource(auntDataSource())
.packages(Aunt.class)
.persistenceUnit("aunt")
.build();
}
This works in terms of connecting to the database and trying to fetch data.
HOWEVER (and here's the problem, thanks for reading this far). After these configurations I have lost the implied naming strategy that translates Java column names to snake case names, so now if I have a Java property idBank it incorrectly uses column name idBank instead of id_bank. I would really like to get that functionality back.
There is a JPA property for this spring.jpa.hibernate.naming-strategy, and there are various naming strategy classes in Spring and Hibernate such as org.springframework.boot.orm.jpa.hibernate.SpringNamingStrategy. So I tried setting it like this:
spring.jpa.hibernate.naming_strategy=org.springframework.boot.orm.jpa.hibernate.SpringNamingStrategy
But it did not work. I tried some variations such as:
spring.datasource.uncle.naming_strategy=org.springframework.boot.orm.jpa.hibernate.SpringNamingStrategy
and
spring.datasource.uncle.hibernate.naming_strategy=org.springframework.boot.orm.jpa.hibernate.SpringNamingStrategy
but this did not have any effect.
Then I read that in Hibernate 5, the naming strategy was broken up into two parts, "physical" and "implicit" and there are different settings for each. So I tried this, with a few variations:
spring.jpa.hibernate.naming.physical-strategy=org.springframework.boot.orm.jpa.hibernate.SpringPhysicalNamingStrategy
and
spring.jpa.hibernate.naming.implicit-strategy=org.springframework.boot.orm.jpa.hibernate.SpringImplicitNamingStrategy
and
spring.datasource.uncle.naming.physical-strategy=org.springframework.boot.orm.jpa.hibernate.SpringPhysicalNamingStrategy
and
spring.datasource.uncle.hibernate.naming.implicit-strategy=org.springframework.boot.orm.jpa.hibernate.SpringImplicitNamingStrategy
But none of these worked.
It seems like there should be a way for me to set this configuration in the beans directly, such as on the SessionFactory, but I could not find that API. The documentation around this seems to have some gaps.
I'd really like to avoid setting up a persistence.xml also, which I have not needed up to this point.
So here is where I'm stuck and I'm hoping someone can help out. Really what I would like is a way to debug these property settings, I turned on trace logging in both org.springframework and org.hibernate but there was nothing useful there. I tried stepping through the code when these beans were configured but couldn't find the place where this happens. If anyone has that info and could share it I'd be really grateful.
I had the same problem and fixed it with the following code (adapted to the code in the question - for a single entity manager):
protected Map<String, Object> jpaProperties() {
Map<String, Object> props = new HashMap<>();
props.put("hibernate.physical_naming_strategy", SpringPhysicalNamingStrategy.class.getName());
props.put("hibernate.implicit_naming_strategy", SpringImplicitNamingStrategy.class.getName());
return props;
}
#Primary
#Bean(name = "defaultEntityManager")
public LocalContainerEntityManagerFactoryBean defaultEntityManagerFactory(
EntityManagerFactoryBuilder builder) {
return builder
.dataSource(auntDataSource())
.packages(Aunt.class)
.persistenceUnit("aunt")
.properties(jpaProperties())
.build();
}
The same as #ewert answer can be gained using properties:
# this works
spring.jpa.properties.hibernate.implicit_naming_strategy=org.springframework.boot.orm.jpa.hibernate.SpringImplicitNamingStrategy
spring.jpa.properties.hibernate.physical_naming_strategy=org.springframework.boot.orm.jpa.hibernate.SpringPhysicalNamingStrategy
# but that doesn't work
spring.jpa.hibernate.naming.physical-strategy=org.springframework.boot.orm.jpa.hibernate.SpringPhysicalNamingStrategy
spring.jpa.hibernate.naming.implicit-strategy=org.springframework.boot.orm.jpa.hibernate.SpringImplicitNamingStrategy
I think I can explain why the default behaviour disappears, as per your latest question.
As of Spring Boot 2.4.2 the deafult configuration kicks in in this method of JpaBaseConfiguration:
#Bean
#Primary
#ConditionalOnMissingBean({ LocalContainerEntityManagerFactoryBean.class, EntityManagerFactory.class })
public LocalContainerEntityManagerFactoryBean entityManagerFactory(EntityManagerFactoryBuilder factoryBuilder) {
Map<String, Object> vendorProperties = getVendorProperties();
customizeVendorProperties(vendorProperties);
return factoryBuilder.dataSource(this.dataSource).packages(getPackagesToScan()).properties(vendorProperties)
.mappingResources(getMappingResources()).jta(isJta()).build();
}
it happens within the customizeVendorProperties method call.
By creating your own LocalContainerEntityManagerFactoryBean bean (two of them actually) this customization is not performed anymore.
If you are using SessionFactory you should use next lines to set naming strategies.
sessionFactory.setImplicitNamingStrategy(SpringImplicitNamingStrategy.INSTANCE);
sessionFactory.setPhysicalNamingStrategy(new SpringPhysicalNamingStrategy());
The only way I get this running properly with Spring-Boot 2+ was setting the following manually:
#Bean(name = "myEmf")
public LocalContainerEntityManagerFactoryBean sapEntityManagerFactory(
EntityManagerFactoryBuilder builder, #Qualifier("myDataSource") DataSource dataSource) {
return builder
.dataSource(dataSource)
.packages("my.custom.package")
.persistenceUnit("myPu")
.properties(getProperties())
.build();
}
public Map<String, String> getProperties() {
val props = new HashMap<String, String>();
if (isTest()) {
props.put("hibernate.hbm2ddl.auto", "create");
} else {
props.put("hibernate.dialect", "org.hibernate.dialect.PostgreSQL95Dialect");
}
return props;
}
Two different H2 instances to be created in-memory. To make sure this happens, I initialized both instances with same shema and different data. So that when I query using DAO different set of data picked from different DataSource. However this is not happening. What am I doing wrong? How to name the instance of H2 correct?
#Bean(name = "DS1")
#Primary
public EmbeddedDatabase dataSource1() {
return new EmbeddedDatabaseBuilder().
setType(EmbeddedDatabaseType.H2).
setName("DB1").
addScript("schema.sql").
addScript("data-1.sql").
build();
}
#Bean(name = "DS2")
public EmbeddedDatabase dataSource2() {
return new EmbeddedDatabaseBuilder().
setType(EmbeddedDatabaseType.H2).
setName("DB2").
addScript("schema.sql").
addScript("data-2.sql").
build();
}
You have created two DataSources and have marked one as #Primary -- this is the one which will be used when your EntityManagerFactories and Repositories are autoconfigured. That's why both DAO's are accessing the same database.
In order to get around this, you need to declare two separate EntityManagerFactories, as described in the Spring Boot documentation:
http://docs.spring.io/spring-boot/docs/current-SNAPSHOT/reference/htmlsingle/#howto-use-two-entity-managers
After that, you'll need to declare two separate repositories and tell each repository which of the EntityManagerFactory to use. To do this, in your #EnableJpaRepositories annotation you'll have to specify the correct EntityMangerFactory. This article describes very nicely how to do that:
http://scattercode.co.uk/2013/11/18/spring-data-multiple-databases/
It would be nice if Spring Boot supported autoconfiguration with two DataSources, but I don't think it's going to happen soon:
https://github.com/spring-projects/spring-boot/issues/808
UPDATE
The author of the above article has published an updated approach:
https://scattercode.co.uk/2016/01/05/multiple-databases-with-spring-boot-and-spring-data-jpa/
The issue was not with multiple instances of H2; but with DataSource injection.
I solved it by passing the Qualifier in method argument.
#Autowired
#Bean(name = "jdbcTemplate")
public JdbcTemplate getJdbcTemplate(#Qualifier("myDataSource") DataSource dataSource) {
return new JdbcTemplate(dataSource);
}