I need to be able to deal with a dynamic list of Datasources.
To do that i wrote a #Bean method that returns a map of datasources.
#Bean
#ConfigurationProperties(prefix = "partitioning.shards")
public Map<String, DataSource> datasources() {
Map<String, DataSource> datasources = new HashMap<>();
...
return datasources;
}
i a second step i need to process these datasources in a different class. unfortunately i have another factory-method that also creates Datasources to a different database.
is it possible to only get the Datasources injected that my own method created? i tried to annotate my method with a qualifier but that doesn't seem to work.
i was wrong on multiple fronts:
returning a map of datasources does not add the datasources to my context. it just looked like that because of something else my code did. after adding my beans with seperate #Bean methods the Qualifier annotation worked aggain without a problem.
Related
I am learning about springboot and trying to connect to a DB2 database. I got that working just fine.
Below are my working DB2 properties:
spring.datasource.url=jdbc:db2://server:port/database:currentSchema=schema-name;
spring.datasource.username=user1
spring.datasource.password=password1
But I renamed them to start with "db2" instead "spring" like:
db2.datasource.url=jdbc:db2://server:port/database:currentSchema=schema-name;
db2.datasource.username=user1
db2.datasource.password=password1
My app still runs, hHowever, when I do that, my controllers no longer return results as they did before the rename.
The reason I ask this is that if I add 2nd data source in the future, I could distinguish easily properties by their data sources if I name them like this.
UPDATE:
Thanks to #Kosta Tenasis answer below and this article (https://www.javadevjournal.com/spring-boot/multiple-data-sources-with-spring-boot/), I was able to resolve and figure this out.
Then going back to my specific question, once you have the configuration for data source in place, you can then modify application.properties to have:
db2.datasource.url=...
instead of having:
spring.datasource.url=...
NOTE1: if you are using Springboot 2.0, they changed to use Hikari and Hikari does not have url property but instead uses jdbc-url, so just change above to:
db2.datasource.jdbc-url=...
NOTE2: In your datasource that you had to create when adding multiple datasources to your project, you will have annotation #ConfigurationProperties. This annotation needs to point to your updated application.properties for datasource (the db2.datasource.url).
By default Spring looks for spring.datasource.** for the properties of the DataSource to connect to.
So you might be getting wrong results because you are not connecting to the database. If you want to configure a DataSource with different,from default, properties you can do like so
#Configuration
public class DataSourceConfig {
#Bean
#ConfigurationProperties(prefix="db2.datasource")
public DataSource dataSource() {
return DataSourceBuilder.create()
.build();
}
And let's say a day comes along and you want a second DataSource you can modify the previous class to something like:
#Configuration
public class DataSourceConfig {
#Bean
#ConfigurationProperties(prefix="db2.datasource")
public DataSource d2Datasource() {
return DataSourceBuilder.create()
.build();
}
#Bean
#ConfigurationProperties(prefix="db3.datasource")
public DataSource db3Datasource() { //pun intented
return DataSourceBuilder.create()
.build();
}
}
and after that in each Class that you want a DataSource you can specify which of the beans you like:
public class DB3DependedClass{
private final DataSource dataSource;
public DB3DependedClass(#Qualifier("db3Datasource") DataSource dataSource){
this.dataSource = dataSource;
}
}
So by default spring will look for
spring.datasource.url (or spring.datasource.jdbc-url)
spring.datasource.username
spring.datasource.password
If you specify another DataSource of your own, those values are not needed.
So in the above example where we specified let's say db3.datasource spring will look for
db3.datasource.url
db3.datasource.username
db3.datasource.password
Important thing here is that the spring IS NOT inferred meaning the complete path is indeed: db3.datasource.url
and NOT
spring.db3.datasource.url
Finally to wrap this up you do have the flexibility to make it start with spring if you want so by declaring a prefix like spring.any.path.ilike.datasouce and of course under that the related values. Spring will pick up either path as long as you specify it.
NOTE: This answer is written solely in the text box provided here and was not tested in an IDE for compilation errors. The logic still holds though
I am using Spring Cloud Task + Batch in a project.
I plan to use different datasources for business data and Spring audit data on the task. So I configured something like:
#Bean
public TaskConfigurer taskConfigurer() {
return new DefaultTaskConfigurer(this.singletonNotExposedSpringDatasource());
}
#Bean
public BatchConfigurer batchConfigurer() {
return new DefaultBatchConfigurer(this.singletonNotExposedSpringDatasource());
}
whereas main datasource is autoconfigured through JpaBaseConfiguration.
The problem comes when SimpleBatchConfiguration+DefaultBatchConfigurer expose a PlatformTransactionManager bean, since JpaBaseConfiguration has a #ConditionalOnMissingBean on PlatformTransactionManager. Therefore Batch's PlatformTransactionManager, binded to the spring.datasource takes place.
So far, this seems to be caused because this bug
So I tried to emulate what JpaBaseConfiguration does, defining my own PlatformTransactionManager over my biz datasource/entityManager.
#Primary
#Bean
public PlatformTransactionManager appTransactionManager(final LocalContainerEntityManagerFactoryBean appEntityManager) {
JpaTransactionManager transactionManager = new JpaTransactionManager();
transactionManager.setEntityManagerFactory(appEntityManager.getObject());
this.appTransactionManager = transactionManager;
return transactionManager;
}
Note I have to define it with a name other than transactionManager, otherwise Spring finds 2 beans and complains (unregardless of #Primary!)
But now it comes the funny part. When running the tests, everything runs smooth, tests finish and DDLs are properly created for both business and Batch/Task's databases, database reads work flawlessly, but business data is not persisted in my testing database, so final assertThats fail when counting. If I #Autowire in my test PlatformTransactionManager or ÈntityManager, everything indicates they are the proper ones. But if I debug within entityRepository.save, and execute org.springframework.transaction.interceptor.TransactionAspectSupport.currentTransactionStatus(), it seems the DatasourceTransactionManager from Batch's configuration is overriding, so my custom exposed PlatformTransactionManager is not being used.
So I guess it is not a problem of my PlatformManager being the primary, but that something is configuring my JPA layer TransactionInterceptor to use the non primary but transactionManager named bean of Batch.
I also tried with making my #Configuration implement TransactionManagementConfigurer and override PlatformTransactionManager annotationDrivenTransactionManager() but still no luck
Thus, I guess what I am asking is whether there is a way to configure the primary TransactionManager for the JPA Layer.
The problem comes when SimpleBatchConfiguration+DefaultBatchConfigurer expose a PlatformTransactionManager bean,
As you mentioned, this is indeed what was reported in BATCH-2788. The solution we are exploring is to expose the transaction manager bean only if Spring Batch creates it.
In the meantime you can set the property spring.main.allow-bean-definition-overriding=true to allow bean definition overriding and set the transaction manager you want Spring Batch to use with BatchConfigurer#getTransactionManager. In your case, it would be something like:
#Bean
public BatchConfigurer batchConfigurer() {
return new DefaultBatchConfigurer(this.singletonNotExposedSpringDatasource()) {
#Override
public PlatformTransactionManager getTransactionManager() {
return new MyTransactionManager();
}
};
}
Hope this helps.
We have a multitenant application with multiple datasources and want to configure the data pool properties (maxActive, minIdle, etc.) individually for each.
Currently I'm constructing a org.apache.tomcat.jdbc.pool.DataSource and setting a few properties manually such as username and password with dataSource.setUserName() and dataSource.setPassword(). I'd like to set the rest of the properties by loading the configuration from a string, for example minIdle=20;initialSize=15.
There are two methods on DataSource which appear like they would accomplish this, but don't seem to be doing what I expect them to. I tried dataSource.setConnectionProperties("..") with some properties as well as populating a Properties object and passing it to dataSource.setDbProperties(), though neither seemed to have an effect when I viewed the pool attributes through JMX. I was only able to change these properties through the specific setters such as dataSource.setInitialSize().
The only way I can think of to set each of the properties from a string of them without the above attempts working is to iterate through each of the properties and have if-else or switch-case logic to determine which of the dataSource setters to call to set the value.
So is there a way to dynamically set these properties from a string without calling each individual setter?
When I set the username either of the setConnectionProperties or setDbProperties, it did change, but I think this may be specific for things like username and password as the other properties I tried to set didn't have an effect.
edit: To clarify, data source properties will be loaded from the database and a new datasource may be added on the fly, so using application properties won't work.
I'm assuming that you are working on spring-boot project. In that case,
Create properties in the application.properties file of spring-boot
spring.datasource.username=XXX
spring.datasource.password=XXX
spring.datasource.max-active=XXX
spring.datasource.min-idle=XXX
and create a configuration file to create datasource as shown below
#Configuration
public class DataSourceConfiguration {
#Value("${spring.datasource.url}")
private String url;
#Value("${spring.datasource.driverClassName}")
private String driverClass;
#Value("${spring.datasource.username}")
private String username;
#Value("${spring.datasource.password}")
private String password;
#Value("${spring.datasource.min-idle}")
private Long minIdle;
#Bean
#Primary
public DataSource dataSource() {
DataSource dataSource = new DataSource();
dataSource.setJdbcUrl(url);
dataSource.setDriverClassName(driverClass);
dataSource.setUsername(username);
dataSource.setPassword(password);
dataSource.setMinimumIdle(minIdle);
return dataSource;
}
}
Like this, you can create multiple properties with different names and refer it while creating different Beans of datasource.
If you want, you can place the application.properties file outside of your project and access it using #PropertySource annotation in your main class.
I'd use the Spring Boot ConfigurationProperties support. From the manual:
74.2 Configure Two DataSources
Creating more than one data source works the same as creating the first one. You might want to mark one of them as #Primary if you are using the default auto-configuration for JDBC or JPA (then that one will be picked up by any #Autowired injections).
#Bean
#Primary
#ConfigurationProperties(prefix="datasource.primary")
public DataSource primaryDataSource() {
return DataSourceBuilder.create().build();
}
#Bean
#ConfigurationProperties(prefix="datasource.secondary")
public DataSource secondaryDataSource() {
return DataSourceBuilder.create().build();
}
If you're concerned about having loads of similar properties in a properties file, the Spring config server (which allows them to be set up and versioned in GIT) might help.
I am using Spring Boot 1.4.1 which uses Hibernate 5.0.11. Initially I configured a data source using application.properties like this:
spring.datasource.uncle.url=jdbc:jtds:sqlserver://hostname:port/db
spring.datasource.uncle.username=user
spring.datasource.uncle.password=password
spring.datasource.uncle.dialect=org.hibernate.dialect.SQLServer2012Dialect
spring.datasource.uncle.driverClassName=net.sourceforge.jtds.jdbc.Driver
I configured it with "uncle" because that will be the name of one of multiple data sources that I'll configure. I configured this data source like this, according to the Spring docs:
#Bean
#Primary
#ConfigurationProperties(prefix = "spring.datasource.uncle")
public DataSource uncleDataSource() {
return DataSourceBuilder.create().build();
}
At this point everything worked fine.
I created an #Entity class without any #Column annotations and let Hibernate figure out the column names, for example if I have a Java property named idBank, Hibernate will automatically assume the column name is id_bank. This is used when generating ddl, running SQL statements, etc. I want to utilize this feature because I'm going to have a lot of entity classes and don't want to have to create and maintain all of the #Column annotations. At this point, this worked fine.
I then added another data source like this:
spring.datasource.aunt.url=jdbc:sybase:Tds:host2:port/db2
spring.datasource.aunt.username=user2
spring.datasource.aunt.password=password2
spring.datasource.aunt.dialect=org.hibernate.dialect.SybaseDialect
spring.datasource.aunt.driverClassName=com.sybase.jdbc4.jdbc.SybDriver
... and also this, following the Spring docs for setting up multiple data sources. Apparently once you define a 2nd data source, it can't configure the default beans and you have to define your own EntityManager and TransactionManager. So in addition to the data source configured above, I added these configurations:
#Bean
#Primary
PlatformTransactionManager uncleTransactionManager(#Qualifier("uncleEntityManagerFactory") final EntityManagerFactory factory) {
return new JpaTransactionManager(factory);
}
#Bean
#Primary
LocalContainerEntityManagerFactoryBean uncleEntityManagerFactory(
EntityManagerFactoryBuilder builder) {
return builder
.dataSource(uncleDataSource())
.packages(Uncle.class)
.persistenceUnit("uncle")
.build();
}
#Bean
#ConfigurationProperties(prefix = "spring.datasource.aunt")
public DataSource auntDataSource() {
return DataSourceBuilder.create().build();
}
#Bean
PlatformTransactionManager auntTransactionManager(#Qualifier("auntEntityManagerFactory") final EntityManagerFactory factory) {
return new JpaTransactionManager(factory);
}
#Bean
LocalContainerEntityManagerFactoryBean auntEntityManagerFactory(
EntityManagerFactoryBuilder builder) {
return builder
.dataSource(auntDataSource())
.packages(Aunt.class)
.persistenceUnit("aunt")
.build();
}
This works in terms of connecting to the database and trying to fetch data.
HOWEVER (and here's the problem, thanks for reading this far). After these configurations I have lost the implied naming strategy that translates Java column names to snake case names, so now if I have a Java property idBank it incorrectly uses column name idBank instead of id_bank. I would really like to get that functionality back.
There is a JPA property for this spring.jpa.hibernate.naming-strategy, and there are various naming strategy classes in Spring and Hibernate such as org.springframework.boot.orm.jpa.hibernate.SpringNamingStrategy. So I tried setting it like this:
spring.jpa.hibernate.naming_strategy=org.springframework.boot.orm.jpa.hibernate.SpringNamingStrategy
But it did not work. I tried some variations such as:
spring.datasource.uncle.naming_strategy=org.springframework.boot.orm.jpa.hibernate.SpringNamingStrategy
and
spring.datasource.uncle.hibernate.naming_strategy=org.springframework.boot.orm.jpa.hibernate.SpringNamingStrategy
but this did not have any effect.
Then I read that in Hibernate 5, the naming strategy was broken up into two parts, "physical" and "implicit" and there are different settings for each. So I tried this, with a few variations:
spring.jpa.hibernate.naming.physical-strategy=org.springframework.boot.orm.jpa.hibernate.SpringPhysicalNamingStrategy
and
spring.jpa.hibernate.naming.implicit-strategy=org.springframework.boot.orm.jpa.hibernate.SpringImplicitNamingStrategy
and
spring.datasource.uncle.naming.physical-strategy=org.springframework.boot.orm.jpa.hibernate.SpringPhysicalNamingStrategy
and
spring.datasource.uncle.hibernate.naming.implicit-strategy=org.springframework.boot.orm.jpa.hibernate.SpringImplicitNamingStrategy
But none of these worked.
It seems like there should be a way for me to set this configuration in the beans directly, such as on the SessionFactory, but I could not find that API. The documentation around this seems to have some gaps.
I'd really like to avoid setting up a persistence.xml also, which I have not needed up to this point.
So here is where I'm stuck and I'm hoping someone can help out. Really what I would like is a way to debug these property settings, I turned on trace logging in both org.springframework and org.hibernate but there was nothing useful there. I tried stepping through the code when these beans were configured but couldn't find the place where this happens. If anyone has that info and could share it I'd be really grateful.
I had the same problem and fixed it with the following code (adapted to the code in the question - for a single entity manager):
protected Map<String, Object> jpaProperties() {
Map<String, Object> props = new HashMap<>();
props.put("hibernate.physical_naming_strategy", SpringPhysicalNamingStrategy.class.getName());
props.put("hibernate.implicit_naming_strategy", SpringImplicitNamingStrategy.class.getName());
return props;
}
#Primary
#Bean(name = "defaultEntityManager")
public LocalContainerEntityManagerFactoryBean defaultEntityManagerFactory(
EntityManagerFactoryBuilder builder) {
return builder
.dataSource(auntDataSource())
.packages(Aunt.class)
.persistenceUnit("aunt")
.properties(jpaProperties())
.build();
}
The same as #ewert answer can be gained using properties:
# this works
spring.jpa.properties.hibernate.implicit_naming_strategy=org.springframework.boot.orm.jpa.hibernate.SpringImplicitNamingStrategy
spring.jpa.properties.hibernate.physical_naming_strategy=org.springframework.boot.orm.jpa.hibernate.SpringPhysicalNamingStrategy
# but that doesn't work
spring.jpa.hibernate.naming.physical-strategy=org.springframework.boot.orm.jpa.hibernate.SpringPhysicalNamingStrategy
spring.jpa.hibernate.naming.implicit-strategy=org.springframework.boot.orm.jpa.hibernate.SpringImplicitNamingStrategy
I think I can explain why the default behaviour disappears, as per your latest question.
As of Spring Boot 2.4.2 the deafult configuration kicks in in this method of JpaBaseConfiguration:
#Bean
#Primary
#ConditionalOnMissingBean({ LocalContainerEntityManagerFactoryBean.class, EntityManagerFactory.class })
public LocalContainerEntityManagerFactoryBean entityManagerFactory(EntityManagerFactoryBuilder factoryBuilder) {
Map<String, Object> vendorProperties = getVendorProperties();
customizeVendorProperties(vendorProperties);
return factoryBuilder.dataSource(this.dataSource).packages(getPackagesToScan()).properties(vendorProperties)
.mappingResources(getMappingResources()).jta(isJta()).build();
}
it happens within the customizeVendorProperties method call.
By creating your own LocalContainerEntityManagerFactoryBean bean (two of them actually) this customization is not performed anymore.
If you are using SessionFactory you should use next lines to set naming strategies.
sessionFactory.setImplicitNamingStrategy(SpringImplicitNamingStrategy.INSTANCE);
sessionFactory.setPhysicalNamingStrategy(new SpringPhysicalNamingStrategy());
The only way I get this running properly with Spring-Boot 2+ was setting the following manually:
#Bean(name = "myEmf")
public LocalContainerEntityManagerFactoryBean sapEntityManagerFactory(
EntityManagerFactoryBuilder builder, #Qualifier("myDataSource") DataSource dataSource) {
return builder
.dataSource(dataSource)
.packages("my.custom.package")
.persistenceUnit("myPu")
.properties(getProperties())
.build();
}
public Map<String, String> getProperties() {
val props = new HashMap<String, String>();
if (isTest()) {
props.put("hibernate.hbm2ddl.auto", "create");
} else {
props.put("hibernate.dialect", "org.hibernate.dialect.PostgreSQL95Dialect");
}
return props;
}
Two different H2 instances to be created in-memory. To make sure this happens, I initialized both instances with same shema and different data. So that when I query using DAO different set of data picked from different DataSource. However this is not happening. What am I doing wrong? How to name the instance of H2 correct?
#Bean(name = "DS1")
#Primary
public EmbeddedDatabase dataSource1() {
return new EmbeddedDatabaseBuilder().
setType(EmbeddedDatabaseType.H2).
setName("DB1").
addScript("schema.sql").
addScript("data-1.sql").
build();
}
#Bean(name = "DS2")
public EmbeddedDatabase dataSource2() {
return new EmbeddedDatabaseBuilder().
setType(EmbeddedDatabaseType.H2).
setName("DB2").
addScript("schema.sql").
addScript("data-2.sql").
build();
}
You have created two DataSources and have marked one as #Primary -- this is the one which will be used when your EntityManagerFactories and Repositories are autoconfigured. That's why both DAO's are accessing the same database.
In order to get around this, you need to declare two separate EntityManagerFactories, as described in the Spring Boot documentation:
http://docs.spring.io/spring-boot/docs/current-SNAPSHOT/reference/htmlsingle/#howto-use-two-entity-managers
After that, you'll need to declare two separate repositories and tell each repository which of the EntityManagerFactory to use. To do this, in your #EnableJpaRepositories annotation you'll have to specify the correct EntityMangerFactory. This article describes very nicely how to do that:
http://scattercode.co.uk/2013/11/18/spring-data-multiple-databases/
It would be nice if Spring Boot supported autoconfiguration with two DataSources, but I don't think it's going to happen soon:
https://github.com/spring-projects/spring-boot/issues/808
UPDATE
The author of the above article has published an updated approach:
https://scattercode.co.uk/2016/01/05/multiple-databases-with-spring-boot-and-spring-data-jpa/
The issue was not with multiple instances of H2; but with DataSource injection.
I solved it by passing the Qualifier in method argument.
#Autowired
#Bean(name = "jdbcTemplate")
public JdbcTemplate getJdbcTemplate(#Qualifier("myDataSource") DataSource dataSource) {
return new JdbcTemplate(dataSource);
}