I'm using Dynamic DataSource in my Spring Boot application.
The problem is I need to generate tables from my entities. There is a way with
spring.jpa.hibernate.ddl-auto=update
but it doesn't work for me since I need to connect to Database in run-time.
What I need to know is can I call some method to do same things as Spring does on application startup with mentioned option.
Okey after some research I found the answer. All you need is to ask sessionFactoryBuilder to generate update scripts for your database and execute than with JdbcTemplate.
LocalSessionFactoryBuilder sessionFactory = new LocalSessionFactoryBuilder(dataSource);
sessionFactory.scanPackages("su");
JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource);
try{
List<SchemaUpdateScript> scripts = sessionFactory.generateSchemaUpdateScriptList(new PostgreSQL9Dialect(),
new DatabaseMetadata(dataSource.getConnection(), new PostgreSQL9Dialect(), sessionFactory));
log.info("Schema update scripts["+scripts.size()+"]");
for (SchemaUpdateScript script:scripts ) {
log.info(script.getScript());
jdbcTemplate.execute(script.getScript());
}
}catch (Exception e){
log.error("error updating schema",e);
}
Related
I am new to Spring boot development.
What I did?
I tested the sample MySQL integration with Spring boot and tested it works fine.
What I am trying?
I am trying to use Hibernate APIs with my existing project. So I create a util class for creating Hibernate sessionFactory.
My code:
public static void initSessionFactory() {
System.out.print("initSessionFactory");
if (sessionFactory == null) {
try {
Configuration configuration = new Configuration();
//***** I don't want configuration here. ******
// Hibernate settings equivalent to hibernate.cfg.xml's properties
//Properties settings = new Properties();
// settings.put(Environment.DRIVER, "com.mysql.cj.jdbc.Driver");
// settings.put(Environment.URL, "jdbc:mysql://localhost:3306/hibernate_db?useSSL=false");
// settings.put(Environment.USER, "root");
// settings.put(Environment.PASS, "root");
// settings.put(Environment.DIALECT, "org.hibernate.dialect.MySQL5Dialect");
// settings.put(Environment.CURRENT_SESSION_CONTEXT_CLASS, "thread");
// settings.put(Environment.HBM2DDL_AUTO, "create-drop");
// settings.put(Environment.SHOW_SQL, "true");
// configuration.setProperties(settings);
//configuration.addAnnotatedClass(Student.class);
ServiceRegistry serviceRegistry = new StandardServiceRegistryBuilder()
.applySettings(configuration.getProperties()).build();
sessionFactory = configuration.buildSessionFactory(serviceRegistry);
session = sessionFactory.openSession();
} catch (Exception e) {
e.printStackTrace();
}
}
}
My application.properties:(It works fine for without Hibernate)
spring.jpa.hibernate.ddl-auto=update
spring.datasource.url=jdbc:mysql://localhost:3306/tcc
spring.datasource.username=root
spring.datasource.password=password
spring.datasource.driverClassName=com.mysql.cj.jdbc.Driver
spring.jpa.database-platform=org.hibernate.dialect.MySQL8Dialect
spring.jpa.show-sql=true
My problem:
I already configured the jdbc environment & other configuration in my application.properties. It works fine. So I don't want to repeat the same configuration in my Java code. So I commented this configuration.
But without Java configuration it throws "The application must supply JDBC connections" error.
My Question:
How to set configuration from application.properties for Hibernate?
When you set the properties below in your app.properties, the Spring Boot will already make your DB connection ready when the application start.
spring.jpa.hibernate.ddl-auto=update
spring.datasource.url=jdbc:mysql://${MYSQL_HOST:localhost}:3306/db_example
spring.datasource.username=springuser
spring.datasource.password=ThePassword
spring.datasource.driver-class-name =com.mysql.jdbc.Driver
You do not need to create SessionFactory , ServiceRegistery etc. even if you do not want to manage DB connection manually.
This link also explain the steps in more detail. You will see that there is no any custom bean, factory or registerer to establish DB connection. Please see it.
https://spring.io/guides/gs/accessing-data-mysql/
Implemented Spring's AbstractRoutingDatasource by dynamically determining the actual DataSource based on the current context.
Refered this article : https://www.baeldung.com/spring-abstract-routing-data-source.
Here on spring boot application start up . Created a map of contexts to datasource objects to configure our AbstractRoutingDataSource. All these client context details are fetched from a database table.
#Bean
#DependsOn("dataSource")
#Primary
public DataSource routeDataSource() {
RoutingDataSource routeDataSource = new RoutingDataSource();
DataSource defaultDataSource = (DataSource) applicationContext.getBean("dataSource");
List<EstCredentials> credentials = LocalDataSourcesDetailsLoader.getAllCredentails(defaultDataSource); // fetching from database table
localDataSourceRegistrationBean.registerDataSourceBeans(estCredentials);
routeDataSource.setDefaultTargetDataSource(defaultDataSource);
Map<Object, Object> targetDataSources = new HashMap<>();
for (Credentials credential : credentials) {
targetDataSources.put(credential.getEstCode().toString(),
(DataSource) applicationContext.getBean(credential.getEstCode().toString()));
}
routeDataSource.setTargetDataSources(targetDataSources);
return routeDataSource;
}
The problem is if i add a new client details, I cannot get that in routeDataSource. Obvious reason is that these values are set on start up.
How can I achieve to add new client context and I had to re intialize the routeDataSource object.
Planning to write a service to get all the client context newly added and reset the routeDataSource object, no need to restart the server each time any changes in the client details.
A simple solution to this situation is adding #RefreshScope to the bean definition:
#Bean
#Primary
#RefreshScope
public DataSource routeDataSource() {
RoutingDataSource routeDataSource = new RoutingDataSource();
DataSource defaultDataSource = (DataSource) applicationContext.getBean("dataSource");
List<EstCredentials> credentials = LocalDataSourcesDetailsLoader.getAllCredentails(defaultDataSource); // fetching from database table
localDataSourceRegistrationBean.registerDataSourceBeans(estCredentials);
routeDataSource.setDefaultTargetDataSource(defaultDataSource);
Map<Object, Object> targetDataSources = new HashMap<>();
for (Credentials credential : credentials) {
targetDataSources.put(credential.getEstCode().toString(),
(DataSource) applicationContext.getBean(credential.getEstCode().toString()));
}
routeDataSource.setTargetDataSources(targetDataSources);
return routeDataSource;
}
Add Spring Boot Actuator as a dependency:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
Then trigger the refresh endpoint POST to /actuator/refresh to update the DataSource (actually every refresh scoped bean).
So this will depend on how much you know about the datasources to be added, but you could set this up as a multi-tenant project. Another example of creating new datasources:
#Autowired private Map <String, Datasource> mars2DataSources;
public void addDataSourceAtRuntime() {
DataSourceBuilder dataSourcebuilder = DataSourcebuilder.create(
MultiTenantJPAConfiguration.class.getclassloader())
.driverclassName("org.postgresql.Driver")
.username("postgres")
.password("postgres")
.url("Jdbc: postgresql://localhost:5412/somedb");
mars2DataSources("tenantX", datasourcebuilder.build())
}
Given that you are using Oracle, you could also use its database change notification features.
Think of it as a listener in the JDBC driver that gets notified whenever something changes in your database table. So upon receiving a change, you could reinitialize/add datasources.
You can find a tutorial of how to do this here: https://docs.oracle.com/cd/E11882_01/java.112/e16548/dbchgnf.htm#JJDBC28820
Though, depending on your organization database notifications need some extra firewall settings for the communication to work.
Advantage: You do not need to manually call the REST Endpoint if something changes, (though Marcos Barberios answer is perfectly valid!)
I will mention my problem.
I need two databases.
DB1 for my application tables
DB2 for saving only the audited tables jv_...
To solve the problem I did the following
`#Bean
public ConnectionProvider jpaConnectionProvider() {
OtherConnectionProvider other = new OtherConnectionProvider();
try {
other.setConnection(dataSource().getConnection());
} catch (SQLException e) {
e.printStackTrace();
}
return scp;
}`
OtherConnectionProvider is a implementation of org.javers.repository.sql.ConnectionProvider.
dataSource() is the normal javax.sql.Datasource.
After using this, spring ignores the database properties mentioned in application.properties and creates the schema and javers related tables in this new schema since I have the following in my application.properties.
spring.jpa.hibernate.ddl-auto=create
Setting dedicated database for Javers audit data is easy for MongoDB (see https://javers.org/documentation/spring-boot-integration/#starter-repository-configuration), but there is no out-of-the box solution for SQL. The main problem is coordinating transactions in two independent SQL databases.
See How to configure transaction management for working with 2 different db in Spring?
Thanks for the reply. I fixed it the following way. See https://www.baeldung.com/spring-data-jpa-multiple-databases. But in the url, it has mentioned about two database configuration. One of the configuration which is Primary should be picked up from application.properties. Second database configuration can be picked up from spring configuration as mentioned in the URL https://javers.org/documentation/spring-integration/#jpa-entity-manager-integration. The solution is tricky enough since the standard properties of spring.datasource are not applicable here. Moreover, addition of commit properties using javers will help. This will act as tenant information.
Following is the typical code where CustomJpaHibernateConnectionProvider is the implementation of org.javers.repository.sql.ConnectionProvider
#Bean
public ConnectionProvider jpaConnectionProvider() {
CustomJpaHibernateConnectionProvider scp = new
CustomJpaHibernateConnectionProvider();
try {
scp.setConnection(dataSource().getConnection());
} catch (SQLException e) {
e.printStackTrace();
}
return scp;
}
and the datasource would like this.
#Bean
#ConfigurationProperties(prefix="spring.javers-datasource")
public DataSource dataSource(){
return DataSourceBuilder.create().build();
}
The data sources properties are not standard Spring boot properties.
spring.datasource.jdbcUrl = jdbc:postgresql://localhost/test
spring.datasource.username = postgres
spring.datasource.driverClassName=org.postgresql.Driver
We are implementing a multitenant application (database per tenant) and would like to include dynamic provisioning of new tenants without restarting the server. This is Grails 3.2.9 / GORM 6.
Among other things this involves creating a dataSource at runtime, without it being configured in application.yml at the application startup.
According to the documentation (11.2.5. Adding Tenants at Runtime) there exists ConnectionSources API for adding tenants at runtime, but the ConnectionSource created this way doesn't seem to be properly registered with Spring (beans for the dataSource, session and transaction manager) and Grails complain about missing beans when we try to use the new datasource.
We expect that when we use the ConnectionSources API to create a connection source for a new database, Grails should initialise it with all the tables according to the GORM Domains in our application, execute Bootstrap.groovy, etc., just like it does for the sources statically configured in application.yml This is not happening either though.
So my question is whether the ConnectionSources API is intended for a different purpose than we are trying to use it for, or it is just not finished/tested yet.
I meant to come back to you. I did manage to figure out a solution. Now this is for schema per customer, not database per customer, but I suspect it would be easy to adapt. I first create the schema using a straight Groovy Sql object as follows:
void createAccountSchema(String tenantId) {
Sql sql = null
try {
sql = new Sql(dataSource as DataSource)
sql.withTransaction {
sql.execute("create schema ${tenantId}" as String)
}
} catch (Exception e) {
log.error("Unable to create schema for tenant $tenantId", e)
throw e
} finally {
sql?.close()
}
}
Then I run the same code as the Liquibase plugin uses, with some simple defaults, as follows:
void updateAccountSchema(String tenantId) {
def applicationContext = Holders.applicationContext
// Now try create the tables for the schema
try {
GrailsLiquibase gl = new GrailsLiquibase(applicationContext)
gl.dataSource = applicationContext.getBean("dataSource", DataSource)
gl.dropFirst = false
gl.changeLog = 'changelog-m.groovy'
gl.contexts = []
gl.labels = []
gl.defaultSchema = tenantId
gl.databaseChangeLogTableName = defaultChangelogTableName
gl.databaseChangeLogLockTableName = defaultChangelogLockTableName
gl.afterPropertiesSet() // this runs the update command
} catch (Exception e) {
log.error("Exception trying to create new account schema tables for $tenantId", e)
throw e
}
}
Finally, I tell Hibernate about the new schema as follows:
try {
hibernateDatastore.addTenantForSchema(tenantId)
} catch (Exception e) {
log.error("Exception adding tenant schema for ${tenantId}", e)
throw e
}
Anywhere you see me referring to 'hibernateDatastore' or 'dataSource' I have those injected by Grails as follows:
def hibernateDatastore
def dataSource
protected String defaultChangelogTableName = "databasechangelog"
protected String defaultChangelogLockTableName = "databasechangeloglock"
Hope this helps.
Currently I'm setting autocommit to false in spring through adding a property to a datasource bean id like below :
<property name="defaultAutoCommit" value="false" />
But i need to add it specifically in a single java method before executing my procedure.
I used the below code snippet.
getJdbcTemplate().getDataSource().getConnection().setAutoCommit(false);
But the above line was not setting autocommit to false?
Am i missing anything ?
or any alternative to set autocommit in a specific java method by spring
Thanks
The problem is that you are setting autocommit on a Connection, but JdbcTemplate doesn't remember that Connection; instead, it gets a new Connection for each operation, and that might or might not be the same Connection instance, depending on your DataSource implementation. Since defaultAutoCommit is not a property on DataSource, you have two options:
Assuming your concrete datasource has a setter for defaultAutoCommit (for example, org.apache.commons.dbcp.BasicDataSource), cast the DataSource to your concrete implementation. Of course this means that you can no longer change your DataSource in your Spring configuration, which defeats the purpose of dependency injection.
((BasicDataSource)getJdbcTemplate().getDataSource()).setDefaultAutoCommit(false);
Set the DataSource to a wrapper implementation that sets AutoCommit to false each time you fetch a connection.
final DataSource ds = getJdbcTemplate().getDataSource();
getJdbcTemplate().setDataSource(new DataSource(){
// You'll need to implement all the methods, simply delegating to ds
#Override
public Connection getConnection() throws SQLException {
Connection c = ds.getConnection();
c.setAutoCommit(false);
return c;
}
});
You need to get the current connection. e.g.
Connection conn = DataSourceUtils.getConnection(jdbcTemplate.getDataSource());
try {
conn.setAutoCommit(false);
/**
* Your Code
*/
conn.commit();
} catch (SQLException e) {
conn.rollback();
e.printStackTrace();
}
I'm posting this because I was looking for it everywhere: I used configuration property in Spring boot to achieve setting the default autocommit mode with:
spring.datasource.hikari.auto-commit: false
Spring Boot 2.4.x Doc for Hikari
You will have to do for each statement that the jdbcTemplate executes. Because for each jdbcTemplate.execute() etc it gets a new connection from the Datasource's connection pool. So you will have to set it for the connection that the connection the jdbcTemplate uses for that query. So you will have to do something like
jdbcTemplate.execute("<your sql query", new PreparedStatementCallback<Integer>(){
#Override
public Integer doInPreparedStatement(PreparedStatement stmt) throws SQLException, DataAccessException
{
Connection cxn = stmt.getConnection();
// set autocommit for that cxn object to false
cxn.setAutoCommit(false);
// set parameters etc in the stmt
....
....
cxn.commit();
// restore autocommit to true for that cxn object. because if the same object is obtained from the CxnPool later, autocommit will be false
cxn.setAutoCommit(true);
return 0;
}
});
Hope this helps
after 5 years still a valid question, i resolved my issue in this way :
set a connection with connection.setAutoCommit(false);
create a jbc template with that connection;
do your work and commit.
Connection connection = dataSource.getConnection();
connection.setAutoCommit(false);
JdbcTemplate jdbcTemplate =
new JdbcTemplate(newSingleConnectionDataSource(connection, true));
// ignore case in mapping result
jdbcTemplate.setResultsMapCaseInsensitive(true);
// do your stuff
connection.commit();
I just came across this and thought the solution would help someone even if it's too late.
As Yosef said, the connection that you get by calling getJdbcTemplate().getDataSource().getConnection() method may or may not be the one used for the communication with database for your operation.
Instead, if your requirement is to just test your script, not to commit the data, you can have a Apache Commons DBCP datasource with auto commit set to fault. The bean definition is given below:
/**
* A datasource with auto commit set to false.
*/
#Bean
public DataSource dbcpDataSource() throws Exception {
BasicDataSource ds = new BasicDataSource();
ds.setUrl(url);
ds.setUsername(username);
ds.setPassword(password);
ds.setDefaultAutoCommit(false);
ds.setEnableAutoCommitOnReturn(false);
return ds;
}
// Create either JdbcTemplate or NamedParameterJdbcTemplate as per your needs
#Bean
public NamedParameterJdbcTemplate dbcpNamedParameterJdbcTemplate() throws Exception {
return new NamedParameterJdbcTemplate(dbcpDataSource());
}
And use this datasource for any such operations.
If you wish to commit your transactions, I suggest you to have one more bean of the datasource with auto commit set to true which is the default behavior.
Hope it helps someone!
I needed it to do some unit testing
In fact Spring already provides the SingleConnectionDataSource implementation with the setAutoCommit method
// import org.springframework.jdbc.datasource.SingleConnectionDataSource;
SingleConnectionDataSource dataSource = new SingleConnectionDataSource();
dataSourceRX71.setAutoCommit(false);
dataSourceRX71.setDriverClassName("xxx");
dataSourceRX71.setUrl("xxx");
dataSourceRX71.setUsername("xxx");
dataSourceRX71.setPassword("xxx");
In some case you could just add #Transactional in the method, e.g. After some batch insert, execute commit at last.