I got one requirement where each company has separate database with the same schema, in future new company might be added to same schema structure.I have to connect their company schema based on user login. So that I have to connect separate database each time when user login.I wan to use spring data JPA to take care of connection.Only I will configure new schema credential in properties file.Is there any way we can add dynamic database schema which uses same model classes.All should happen in run time only.
Sounds like you are asking about multi-tenant support. You can create a LocalContainerEntityManagerFactoryBean something like this:
#Bean
public LocalContainerEntityManagerFactoryBean entityManagerFactory(
DataSource dataSource,
MultiTenantConnectionProvider multiTenantConnectionProvider,
CurrentTenantIdentifierResolver tenantIdentifierResolver) {
LocalContainerEntityManagerFactoryBean emfBean = new LocalContainerEntityManagerFactoryBean();
emfBean.setDataSource(dataSource);
emfBean.setPackagesToScan(Application.class.getPackage().getName());
emfBean.setJpaVendorAdapter(jpaVendorAdapter());
Map<String, Object> jpaProperties = new HashMap<>();
jpaProperties.put("hibernate.dialect", "org.hibernate.dialect.PostgreSQLDialect");
// Improved naming strategy deprecated as of hibernate 5.0
// jpaProperties.put("hibernate.ejb.naming_strategy", "org.hibernate.cfg.ImprovedNamingStrategy");
jpaProperties.put("hibernate.implicit_naming_strategy", "org.hibernate.boot.model.naming.ImplicitNamingStrategyLegacyHbmImpl");
jpaProperties.put("hibernate.physical_naming_strategy", "com.example.config.HibernateLegacyImprovedNamingStrategy");
jpaProperties.put("hibernate.show_sql", "true");
jpaProperties.put("hibernate.format_sql", "true");
jpaProperties.put(org.hibernate.cfg.Environment.MULTI_TENANT, MultiTenancyStrategy.SCHEMA);
jpaProperties.put(org.hibernate.cfg.Environment.MULTI_TENANT_CONNECTION_PROVIDER, multiTenantConnectionProvider);
jpaProperties.put(org.hibernate.cfg.Environment.MULTI_TENANT_IDENTIFIER_RESOLVER, tenantIdentifierResolver);
emfBean.setJpaPropertyMap(jpaProperties);
return emfBean;
}
And provide implementations of MultiTenantConnectionProvider and CurrentTenantIdentifierResolver
As an example, for schema tenancy, the CurrentTenantIdentifierResolver could look up the current user from session to determine what schema to use.
The MultiTenantConnectionProvider could alter the connection with a command like SELECT set_schema_to(?) before returning the connection for use (and reset it on return).
#Override
public Connection getConnection(String tenantIdentifier) throws SQLException {
final Connection connection = getAnyConnection();
setSchemaTo(connection, tenantIdentifier);
return connection;
}
You'll need to read up on multi-tenancy and see what type you want along with which implementations you are using.
Related
I have a Rest service and when it gets it has to do some insertion and updation to almost 25 database. So when I tried like the below code, it was working in my localhost but when I deploy to my staging server I was getting FATAL: too many connections for role "user123"
List<String> databaseUrls = null;
databaseUrls.forEach( databaseUrl -> {
DataSource dataSource = DataSourceBuilder.create()
.driverClassName("org.postgresql.Driver")
.url(databaseUrl)
.username("user123")
.password("some-password")
.build();
JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource);
jdbcTemplate.update("Some...Update...Query");
});
As per my understanding DataSource need not to be closed because it is never opened.
Note:
A DataSource implementation need not be closed, because it is never
“opened”. A DataSource is not a resource, is not connected to the
database, so it is not holding networking connections nor resources on
the database server. A DataSource is simply information needed when
making a connection to the database, with the database server's
network name or address, the user name, user password, and various
options you want specified when a connection is eventually made.
Can someone tell why I am getting this issue
The problem is in DataSourceBuilder, it actually creates of the connection pools which spawns some number of connections and keeps them running:
private static final String[] DATA_SOURCE_TYPE_NAMES = new String[] {
"org.apache.tomcat.jdbc.pool.DataSource",
"com.zaxxer.hikari.HikariDataSource",
"org.apache.commons.dbcp.BasicDataSource" };
Javadoc says:
/**
* Convenience class for building a {#link DataSource} with common implementations and
* properties. If Tomcat, HikariCP or Commons DBCP are on the classpath one of them will
* be selected (in that order with Tomcat first). In the interest of a uniform interface,
* and so that there can be a fallback to an embedded database if one can be detected on
* the classpath, only a small set of common configuration properties are supported. To
* inject additional properties into the result you can downcast it, or use
* <code>#ConfigurationProperties</code>.
*/
Try to use e.g. SingleConnectionDataSource, then your problem will gone:
List<String> databaseUrls = null;
Class.forName("org.postgresql.Driver");
databaseUrls.forEach( databaseUrl -> {
SingleConnectionDataSource dataSource;
try {
dataSource = new SingleConnectionDataSource(
databaseUrl, "user123", "some-password", true /*suppressClose*/);
JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource);
jdbcTemplate.update("Some...Update...Query");
} catch (Exception e) {
log.error("Failed to run queries for {}", databaseUrl, e);
} finally {
// release resources
if (dataSource != null) {
dataSource.destroy();
}
}
});
First thing it is very bad architecture decision to have single application managing 50 database. Anyway instead of creating DataSource in for loop, you should make use of Factory Design pattern to create DataSource for each DB. You should add some connection pooling mechanism to your system . HijariCP and TomcatPool are most widely used. Analyse logs of failure thread for any further issues.
Implemented Spring's AbstractRoutingDatasource by dynamically determining the actual DataSource based on the current context.
Refered this article : https://www.baeldung.com/spring-abstract-routing-data-source.
Here on spring boot application start up . Created a map of contexts to datasource objects to configure our AbstractRoutingDataSource. All these client context details are fetched from a database table.
#Bean
#DependsOn("dataSource")
#Primary
public DataSource routeDataSource() {
RoutingDataSource routeDataSource = new RoutingDataSource();
DataSource defaultDataSource = (DataSource) applicationContext.getBean("dataSource");
List<EstCredentials> credentials = LocalDataSourcesDetailsLoader.getAllCredentails(defaultDataSource); // fetching from database table
localDataSourceRegistrationBean.registerDataSourceBeans(estCredentials);
routeDataSource.setDefaultTargetDataSource(defaultDataSource);
Map<Object, Object> targetDataSources = new HashMap<>();
for (Credentials credential : credentials) {
targetDataSources.put(credential.getEstCode().toString(),
(DataSource) applicationContext.getBean(credential.getEstCode().toString()));
}
routeDataSource.setTargetDataSources(targetDataSources);
return routeDataSource;
}
The problem is if i add a new client details, I cannot get that in routeDataSource. Obvious reason is that these values are set on start up.
How can I achieve to add new client context and I had to re intialize the routeDataSource object.
Planning to write a service to get all the client context newly added and reset the routeDataSource object, no need to restart the server each time any changes in the client details.
A simple solution to this situation is adding #RefreshScope to the bean definition:
#Bean
#Primary
#RefreshScope
public DataSource routeDataSource() {
RoutingDataSource routeDataSource = new RoutingDataSource();
DataSource defaultDataSource = (DataSource) applicationContext.getBean("dataSource");
List<EstCredentials> credentials = LocalDataSourcesDetailsLoader.getAllCredentails(defaultDataSource); // fetching from database table
localDataSourceRegistrationBean.registerDataSourceBeans(estCredentials);
routeDataSource.setDefaultTargetDataSource(defaultDataSource);
Map<Object, Object> targetDataSources = new HashMap<>();
for (Credentials credential : credentials) {
targetDataSources.put(credential.getEstCode().toString(),
(DataSource) applicationContext.getBean(credential.getEstCode().toString()));
}
routeDataSource.setTargetDataSources(targetDataSources);
return routeDataSource;
}
Add Spring Boot Actuator as a dependency:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
Then trigger the refresh endpoint POST to /actuator/refresh to update the DataSource (actually every refresh scoped bean).
So this will depend on how much you know about the datasources to be added, but you could set this up as a multi-tenant project. Another example of creating new datasources:
#Autowired private Map <String, Datasource> mars2DataSources;
public void addDataSourceAtRuntime() {
DataSourceBuilder dataSourcebuilder = DataSourcebuilder.create(
MultiTenantJPAConfiguration.class.getclassloader())
.driverclassName("org.postgresql.Driver")
.username("postgres")
.password("postgres")
.url("Jdbc: postgresql://localhost:5412/somedb");
mars2DataSources("tenantX", datasourcebuilder.build())
}
Given that you are using Oracle, you could also use its database change notification features.
Think of it as a listener in the JDBC driver that gets notified whenever something changes in your database table. So upon receiving a change, you could reinitialize/add datasources.
You can find a tutorial of how to do this here: https://docs.oracle.com/cd/E11882_01/java.112/e16548/dbchgnf.htm#JJDBC28820
Though, depending on your organization database notifications need some extra firewall settings for the communication to work.
Advantage: You do not need to manually call the REST Endpoint if something changes, (though Marcos Barberios answer is perfectly valid!)
I am currently developing a project using the following libs/frameworks:
hibernate: 5.2.11.Final
spring: 5.0.2.RELEASE
postgresql: 9.4.1212
mysql: 5.1.31
spring-data-jpa: 2.0.2.RELEASE
The project should be available in both Postgres and mysql. I am also using a profiler program called "Prefix" to track possible performance issues.
I realized that all my queries on postgresql were taking longer than the ones made on mysql, so I decided to look on profiler what was the issue.
Whenever I tried to establish a new connection to the database on mysql (DriverManager.getConnection), it took something between 5~10ms, and the same connection to postgresql is taking about 200ms+.
Do you guys know what could be making driver connections on postgresql take longer than mysql?
Here is how the db is configured:
public DataSource dataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
...
dataSource.setDriverClassName(org.postgresql.Driver);
...
}
private Properties hibernateProperties() {
Properties properties = new Properties();
properties.put("spring.jpa.hibernate.ddl-auto", props.getHibernateJPAAutoConfig());
if (props.getDatabaseName().equals(ProjectConstants.DATABASE_POSTGRE)) {
properties.put("spring.jpa.properties.hibernate.dialect", "org.hibernate.dialect.PostgreSQL9Dialect");
properties.put("spring.jpa.properties.hibernate.jdbc.use_get_generated_keys", true);
properties.put("spring.jpa.properties.hibernate.temp.use_jdbc_metadata_defaults", false); //started using those properties because I read that it could make postgresql connections faster; however it did not work.
}
return properties;
}
public EntityManagerFactory entityManagerFactory() {
LocalContainerEntityManagerFactoryBean factory = new LocalContainerEntityManagerFactoryBean();
factory.setDataSource(dataSource());
factory.setJpaProperties(hibernateProperties());
...
}
My Spring Boot 1.3.1 based application relies on an Oracle 11.2 database and I want to tune the fetching of SELECT statement results.
JdbcTemplate offers public void setFetchSize(int fetchSize) to tune the fetch size, which for Oracle is preset to 10 by the driver:
Set the fetch size for this JdbcTemplate. This is important for
processing large result sets: Setting this higher than the default
value will increase processing speed at the cost of memory
consumption; setting this lower can avoid transferring row data that
will never be read by the application. Default is -1, indicating to
use the JDBC driver's default (i.e. to not pass a specific fetch size
setting on the driver).
The Oracle JDBC driver (I use ojdbc7.jar because it is downwards compatible) offers a defaultRowPrefetch parameter to increase the fetch size for the complete database connection.
According to the docs this parameter could be set this way:
java.util.Properties info = new java.util.Properties();
info.put ("user", "scott");
info.put ("password","tiger");
info.put ("defaultRowPrefetch","15");
getConnection ("jdbc:oracle:oci8:#",info);
But my application is configured using application.yml:
datasource:
url: jdbc:oracle:thin:#xyz:1521:abc
username: ${name}
password: ${password}
driver-class-name: oracle.jdbc.driver.OracleDriver
...
And even if I wanted to change that configuration to use spring.datasource.url=jdbc:... instead there is no way to set the fetch size globally according to this post.
Is there a more "Spring Boot style" approach or do I need to configure each template manually ?
A BeanPostProcessor will process all the beans in the ApplicationContext and that way you can add additional configuration or replace it totally if you would like.
You could create a BeanPostProcessor that would add the properties to the configured DataSource. The sample below assumes the use of commons-dbcp 1 or 2 if you use a different DataSource modify accordingly.
public class DataSourceConfiguringBeanPostProcessor implements BeanPostProcessor {
private final Map<String,String> properties = new HashMap<>;
public Object postProcessBeforeInitialization(Object bean, String beanName) throws BeansException {
if (bean instance BasicDataSource ) {
for (Map.Entry<String, String> prop : properties.entrySet()) {
((BasicDataSource) bean).addConnectionProperty(prop.getKey(), prop.getValue());
}
}
return bean;
}
public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException {
return bean;
}
public void setProperties(Map<String, String> properties) {
this.properties.putAll(properties);
}
}
Now you can add this to your configuration and it will add the properties to DataSource beans.
#Bean
public BeanPostProcessor dataSourcePostProcessor() {
DataSourceConfiguringBeanPostProcessor processor = new DataSourceConfiguringBeanPostProcessor();
Map<String, String> properties = new HashMap<>();
properties.put("defaultRowPrefetch", "15");
properties.put("defaultBatchValue", "25");
processor.setProperties(properties);
return processor;
}
That should do the trick for configuring the datasource.
I had read several articles on obtaining Connection using Spring DataSource.
But in our Company Setup, connection object is obtained through already configured environment. Following the sample code:
String pool = PropertyFileReader.getPropertyValue("database.properties", "development.connectionPool");
Connection connection = RequestUtils.getConnection(pool);
Hence, After reading this tutorial
I am confused on using JDBCTemplate using connection object from above code.
I believe JdbcTemplate is not designed to work against a Connection as what you expected. As a workaround, if you are fine to create a separate JdbcTemplate for each connection you created, you may wrap your connection in a thin wrapper of DataSource, and feed it to JdbcTemplate.
I think it should work but I haven't tried it anyway...
class SingleConnectionDataSource implements DataSource {
private Connection connection;
public SingleConnectionDataSource(Connection connection) {
this.connection = connection;
}
public Connection getConnection() {
return this.connection;
}
public Connection getConnection(String username, String password) {
return this.connection;
}
}
// at the place you want to use JdbcTemplate
Connection conn = blablabla; // your own way to get it
JdbcTemplate jdbcTemplate = new JdbcTemplate(new SingleConnectionDataSource(conn));
Actually, Spring already provided SingleConnectionDataSource implementation (have seen in version 4.1.7).
It is even allows you to supress connection closing by template.