I'm trying to support a multi-tenant by schema in my spring boot (1.4) application. I have the following in my config:
hibernate:
format_sql: true
default_schema: corrto
multiTenancy: SCHEMA
tenant_identifier_resolver: com.config.HeaderTenantIdentifierResolver
multi_tenant_connection_provider: com.config.SchemaPerTenantConnectionProvider
My MultiTenantConnectionProvider implementation is as follows:
public class SchemaPerTenantConnectionProvider implements MultiTenantConnectionProvider {
#Autowired
private DataSource dataSource;
#Override
public Connection getAnyConnection() throws SQLException {
return this.dataSource.getConnection();
}
#Override
public void releaseAnyConnection(Connection connection) throws SQLException {
connection.close();
}
#Override
public Connection getConnection(String tenantIdentifier) throws SQLException {
final Connection connection = this.getAnyConnection();
// need to do stuff here
return connection;
}
#Override
public void releaseConnection(String tenantIdentifier, Connection connection) throws SQLException {
}
#Override
public boolean supportsAggressiveRelease() {
return true;
}
#Override
public boolean isUnwrappableAs(Class unwrapType) {
return false;
}
#Override
public <T> T unwrap(Class<T> unwrapType) {
return null;
}
}
It is failing because dataSource is null. I'm assuming it hasn't been created yet but I'm having a hard time finding solutions via Google.
I met the same problem.It seems that in the yml ,HeaderTenantIdentifierResolver and SchemaPerTenantConnectionProvider is managed by hibernate.See here.
Related
I wrote integration test using Mockito, but it works when connection to database was set. Actually test just check possibility access some endpoints and not related to the data access layer. So I don't need database for it yet.
Reason of failing test when database is down - HikariDatasource check connection to the database when spring instantiates context. Mocking doesn't return Connection and it lead to the fail of application. Solution that i have found is use hsql in memory database, but for me it looks like work around. Probably exists other solution providing some fake data?
Not sure that this is elegant solution, but I need to force work tests like this
mockMvc.perform(
post("/some").contentType(MediaType.APPLICATION_JSON_UTF8)
.content(objectMapper.writeValueAsString(someDto))
.header(HttpHeaders.AUTHORIZATION, AUTH_HEADER)
.accept(MediaType.APPLICATION_JSON_UTF8)
).andExpect(status().is(201));
After debugging and searching I have found solution that allowed start container without database in memory.
#TestConfiguration
#ComponentScan(basePackages = "com.test")
#ActiveProfiles("test")
public class TestConfig {
//Other Beans
#Bean
public DataSource getDatasource() {
return new MockDataSource();
}
}
class MockDataSource implements DataSource {
#Override
public Connection getConnection() throws SQLException {
return createMockConnection();
}
#Override
public Connection getConnection(String username, String password) throws SQLException {
return getConnection();
}
#Override
public PrintWriter getLogWriter() throws SQLException {
return null;
}
#Override
public void setLogWriter(PrintWriter out) throws SQLException {
}
#Override
public void setLoginTimeout(int seconds) throws SQLException {
}
#Override
public int getLoginTimeout() throws SQLException {
return 0;
}
public Logger getParentLogger() throws SQLFeatureNotSupportedException {
return null;
}
#Override
public <T> T unwrap(Class<T> iface) throws SQLException {
return null;
}
#Override
public boolean isWrapperFor(Class<?> iface) throws SQLException {
return false;
}
public static Connection createMockConnection() throws SQLException {
// Setup mock connection
final Connection mockConnection = mock(Connection.class);
// Autocommit is always true by default
when(mockConnection.getAutoCommit()).thenReturn(true);
// Handle Connection.createStatement()
Statement statement = mock(Statement.class);
when(mockConnection.createStatement()).thenReturn(statement);
when(mockConnection.createStatement(anyInt(), anyInt())).thenReturn(statement);
when(mockConnection.createStatement(anyInt(), anyInt(), anyInt())).thenReturn(statement);
when(mockConnection.isValid(anyInt())).thenReturn(true);
// Handle Connection.prepareStatement()
PreparedStatement mockPreparedStatement = mock(PreparedStatement.class);
when(mockConnection.prepareStatement(anyString())).thenReturn(mockPreparedStatement);
when(mockConnection.prepareStatement(anyString(), anyInt())).thenReturn(mockPreparedStatement);
when(mockConnection.prepareStatement(anyString(), (int[]) any())).thenReturn(mockPreparedStatement);
when(mockConnection.prepareStatement(anyString(), (String[]) any())).thenReturn(mockPreparedStatement);
when(mockConnection.prepareStatement(anyString(), anyInt(), anyInt())).thenReturn(mockPreparedStatement);
when(mockConnection.prepareStatement(anyString(), anyInt(), anyInt(), anyInt())).thenReturn(mockPreparedStatement);
doAnswer((Answer<Void>) invocation -> null).doNothing().when(mockPreparedStatement).setInt(anyInt(), anyInt());
ResultSet mockResultSet = mock(ResultSet.class);
when(mockPreparedStatement.executeQuery()).thenReturn(mockResultSet);
when(mockResultSet.getString(anyInt())).thenReturn("aString");
when(mockResultSet.next()).thenReturn(true);
// Handle Connection.prepareCall()
CallableStatement mockCallableStatement = mock(CallableStatement.class);
when(mockConnection.prepareCall(anyString())).thenReturn(mockCallableStatement);
when(mockConnection.prepareCall(anyString(), anyInt(), anyInt())).thenReturn(mockCallableStatement);
when(mockConnection.prepareCall(anyString(), anyInt(), anyInt(), anyInt())).thenReturn(mockCallableStatement);
ResultSet mockResultSetTypeInfo = mock(ResultSet.class);
DatabaseMetaData mockDataBaseMetadata = mock(DatabaseMetaData.class);
when(mockDataBaseMetadata.getDatabaseProductName()).thenReturn("PostgreSQL");
when(mockDataBaseMetadata.getDatabaseMajorVersion()).thenReturn(8);
when(mockDataBaseMetadata.getDatabaseMinorVersion()).thenReturn(2);
when(mockDataBaseMetadata.getConnection()).thenReturn(mockConnection);
when(mockDataBaseMetadata.getTypeInfo()).thenReturn(mockResultSetTypeInfo);
when(mockConnection.getMetaData()).thenReturn(mockDataBaseMetadata);
// Handle Connection.close()
doAnswer((Answer<Void>) invocation -> null).doThrow(new SQLException("Connection is already closed")).when(mockConnection).close();
// Handle Connection.commit()
doAnswer((Answer<Void>) invocation -> null).doThrow(new SQLException("Transaction already committed")).when(mockConnection).commit();
// Handle Connection.rollback()
doAnswer((Answer<Void>) invocation -> null).doThrow(new SQLException("Transaction already rolled back")).when(mockConnection).rollback();
return mockConnection;
}
}
Mocking DataSource allows start container and provide post call to the controller using MockMvc.
I want to connect to two Database schemas which are on same database server , for that I want to use only one set of models and JPA repositories , with two data soources connecting to two different schemas .But as of now i am not able to find a way where we can reuse the existing models and repositories , for now I have created two sets of models and repositories having different schemas .Is there a way we can reuse the models and Repositories ?
But as of now i am not able to find a way where we can reuse the existing models and repositories , for now I have created two sets of models and repositories having different schemas
NOTE:
i am able to connect to two schemas via two data sources hence multi-tenancy support is not needed the only thing is while connecting to two separate schemas i am creating models(enitities) and JPA repositories twices even though tables are identical in both the schemas , is there are a to remove code duplicacy
looks like you need multi-tenancy support. Tenant per schema
You need TenantInterceptor which resolves tenantId e.g. from session or JWT token. And MultiTenantConnectionProvider which returns desired provider.
Some code from Multi-Tenancy Implementation for Spring Boot + Hibernate Projects
#Component
public class TenantInterceptor extends HandlerInterceptorAdapter {
#Autowired
private JwtTokenUtil jwtTokenUtil;
#Value("${jwt.header}")
private String tokenHeader;
#Override
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler)
throws Exception {
String authToken = request.getHeader(this.tokenHeader);
String tenantId = jwtTokenUtil.getTenantIdFromToken(authToken);
TenantContext.setCurrentTenant(tenantId);
return true;
}
#Override
public void postHandle(
HttpServletRequest request, HttpServletResponse response, Object handler, ModelAndView modelAndView)
throws Exception {
TenantContext.clear();
}
}
and
#Component
public class MultiTenantConnectionProviderImpl implements MultiTenantConnectionProvider {
#Autowired
private DataSource dataSource;
#Override
public Connection getAnyConnection() throws SQLException {
return dataSource.getConnection();
}
#Override
public void releaseAnyConnection(Connection connection) throws SQLException {
connection.close();
}
#Override
public Connection getConnection(String tenantIdentifie) throws SQLException {
String tenantIdentifier = TenantContext.getCurrentTenant();
final Connection connection = getAnyConnection();
try {
if (tenantIdentifier != null) {
connection.createStatement().execute("USE " + tenantIdentifier);
} else {
connection.createStatement().execute("USE " + DEFAULT_TENANT_ID);
}
}
catch ( SQLException e ) {
throw new HibernateException(
"Problem setting schema to " + tenantIdentifier,
e
);
}
return connection;
}
#Override
public void releaseConnection(String tenantIdentifier, Connection connection) throws SQLException {
try {
connection.createStatement().execute( "USE " + DEFAULT_TENANT_ID );
}
catch ( SQLException e ) {
throw new HibernateException(
"Problem setting schema to " + tenantIdentifier,
e
);
}
connection.close();
}
#SuppressWarnings("rawtypes")
#Override
public boolean isUnwrappableAs(Class unwrapType) {
return false;
}
#Override
public <T> T unwrap(Class<T> unwrapType) {
return null;
}
#Override
public boolean supportsAggressiveRelease() {
return true;
}
}
I am working on an application with Spring-starter-data-jpa and multi datasources, that need to lazy initialize datasouce connection, but i dosen't manage to work with it.
For exemple if i put a wrong password on a datasource configuration, the application startup fails.
I try to put a #Lazy annotation on the Datasource #Configuration class but the application still crash on startup.
I try to implement a LazyConnectionDataSourceProxy instead the Datasource without success.
I can't imagine that it is not possible.
Is somewone have an idea to how lazy load datasources configuration on the first jpa repository call ?
I tryed to do this with a kind of adapter/singleton mix pattern like this :
#Component
public class TestErrorDatasource implements DataSource {
private DataSource datasource;
#Autowired Environment env;
#Override
public PrintWriter getLogWriter() throws SQLException {
return getDatasource().getLogWriter();
}
#Override
public int getLoginTimeout() throws SQLException {
return getDatasource().getLoginTimeout();
}
#Override
public Logger getParentLogger() throws SQLFeatureNotSupportedException {
return getDatasource().getParentLogger();
}
#Override
public void setLogWriter(PrintWriter arg0) throws SQLException {
getDatasource().setLogWriter(arg0);
}
#Override
public void setLoginTimeout(int arg0) throws SQLException {
getDatasource().setLoginTimeout(arg0);
}
#Override
public boolean isWrapperFor(Class<?> arg0) throws SQLException {
return getDatasource().isWrapperFor(arg0);
}
#Override
public <T> T unwrap(Class<T> arg0) throws SQLException {
return getDatasource().unwrap(arg0);
}
#Override
public Connection getConnection() throws SQLException {
return getDatasource().getConnection();
}
#Override
public Connection getConnection(String arg0, String arg1) throws SQLException {
return getDatasource().getConnection(arg0, arg1);
}
private DataSource getDatasource() {
if(datasource == null) {
this.datasource = DataSourceBuilder
.create()
.driverClassName(env.getProperty("testError.datasource.driverClassName"))
.url(env.getProperty("testError.datasource.url"))
.username(env.getProperty("testError.datasource.username"))
.password(env.getProperty("testError.datasource.password"))
.build();
}
return datasource;
}
}
But it dosen't work.
I put a break point on the getConnection() methode and i raised tha it's called on startup by
org.hibernate.engine.jdbc.connections.internal.DatasourceConnectionProviderImpl
class
Is there any configuration to avoid hibernate check the getConnection() methode on starup ?
Thanks in advance
A datasource is just an interface with a 'getConnection()' method on it.
So create your own implementation that delegates to a singleton that is created only when getConnection is first called.
In other words, whilst your implementation can be a Spring bean, its singleton dependency isn't, and isn't injected, but set the first time you access getConnection.
I am developing a data analytics app with Apache Flink and Neo4J (Community Edition).
In this application, the Flink sink must save/update relations in Neo4J.
Which is the best way for Neo4J session management, and why?
First implementation:
public class MySink extends RichSinkFunction<Link> {
private DbConfiguration dbconfig;
private Driver driver;
#Override
public void open(Configuration parameters) throws Exception {
this.driver = Neo4JManager.open(this.dbconfig);
}
#Override
public void close() throws Exception {
this.driver.close();
}
#Override
public void invoke(Link link) throws Exception {
Session session = this.driver.session();
Neo4JManager.saveLink(session, link);
session.close();
}
}
Second implementation:
public class MySink extends RichSinkFunction<Link> {
private DbConfiguration dbconfig;
private Driver driver;
private Session session;
#Override
public void open(Configuration parameters) throws Exception {
this.driver = Neo4JManager.open(this.dbconfig);
this.session = driver.session();
}
#Override
public void close() throws Exception {
this.session.close();
this.driver.close();
}
#Override
public void invoke(Link link) throws Exception {
Neo4JManager.saveLink(this.session, link);
}
}
In both implementations, the following functions have been used:
public class Neo4JManager {
public static Driver open(DbConfiguration dbconf) {
AuthToken auth = AuthTokens.basic(dbconf.getUsername(), dbconf.getPassword());
Config config = Config.build().withEncryptionLevel(Config.EncryptionLevel.NONE ).toConfig();
return GraphDatabase.driver(dbconf.getHostname(), auth, config);
}
public static void saveLink(Session session, Link link) {
Value params = parameters("x", link.x, "y", link.y);
session.run('CREATE (Person {id:{x}}-[FOLLOWS]->(Person {id:{y}}))'
}
}
Thank you.
In my application, I need to use pure JDBC together with Guice. However, Guice doesn't provide any built-in support to manage transactions. guice-persist only provides support based on JPA, which I cannot use.
so I tried to implement a simple solution to manage transactions with Guice and JDBC. here is the first version:
use TransactionHolder to store the transaction per thread.
public class JdbcTransactionHolder {
private static ThreadLocal<JdbcTransaction> currentTransaction = new ThreadLocal<JdbcTransaction>();
public static void setCurrentTransaction(JdbcTransaction transaction) {
currentTransaction.set(transaction);
}
public static JdbcTransaction getCurrentTransaction() {
return currentTransaction.get();
}
public static void removeCurrentTransaction() {
currentTransaction.remove();
}
}
implements a transaction manager for JDBC, for now only begin(), getTransaction(), commit() and rollback() method:
public class JdbcTransactionManager implements TransactionManager {
#Inject
private DataSource dataSource;
#Override
public void begin() throws NotSupportedException, SystemException {
logger.debug("Start the transaction");
try {
JdbcTransaction tran = JdbcTransactionHolder.getCurrentTransaction();
Connection conn = null;
if(tran == null) {
conn = dataSource.getConnection();
}
else {
conn = tran.getConnection();
}
// We have to put the connection in the holder so that we can get later
// from the holder and use it in the same thread
logger.debug("Save the transaction for thread: {}.", Thread.currentThread());
JdbcTransactionHolder.setCurrentTransaction(new JdbcTransaction(conn));
} catch (Exception e) {
throw new RuntimeException(e);
}
}
#Override
public void commit() throws RollbackException, HeuristicMixedException,
HeuristicRollbackException, SecurityException,
IllegalStateException, SystemException {
logger.debug("Commit the transaction");
try {
logger.debug("Get the connection for thread: {}.", Thread.currentThread());
Transaction transaction = JdbcTransactionHolder.getCurrentTransaction();
transaction.commit();
}
catch(Exception e) {
throw new RuntimeException(e);
}
finally {
JdbcTransactionHolder.removeCurrentTransaction();
}
}
#Override
public Transaction getTransaction() throws SystemException {
logger.debug("Get transaction.");
final JdbcTransaction tran = JdbcTransactionHolder.getCurrentTransaction();
if(tran == null) {
throw new DBException("No transaction is availble. TransactionManager.begin() is probably not yet called.");
}
return tran;
}
#Override
public void rollback() throws IllegalStateException, SecurityException,
SystemException {
logger.debug("Rollback the transaction");
try {
logger.debug("Get the transaction for thread: {}.", Thread.currentThread());
Transaction conn = JdbcTransactionHolder.getCurrentTransaction();
conn.commit();
}
catch(Exception e) {
throw new RuntimeException(e);
}
finally {
JdbcTransactionHolder.removeCurrentTransaction();
}
}
}
implement a wrapper for DataSource which can get the current connection from the transaction holder if a transaction has been started:
public class JdbcDataSource implements DataSource {
private final static org.slf4j.Logger logger = LoggerFactory.getLogger(JdbcDataSource.class);
private DataSource dataSource;
public JdbcDataSource(DataSource dataSource) {
this.dataSource = dataSource;
}
#Override
public PrintWriter getLogWriter() throws SQLException {
return dataSource.getLogWriter();
}
#Override
public int getLoginTimeout() throws SQLException {
return dataSource.getLoginTimeout();
}
#Override
public Logger getParentLogger() throws SQLFeatureNotSupportedException {
return dataSource.getParentLogger();
}
#Override
public void setLogWriter(PrintWriter out) throws SQLException {
this.dataSource.setLogWriter(out);
}
#Override
public void setLoginTimeout(int seconds) throws SQLException {
this.dataSource.setLoginTimeout(seconds);
}
#Override
public boolean isWrapperFor(Class<?> arg0) throws SQLException {
return this.isWrapperFor(arg0);
}
#Override
public <T> T unwrap(Class<T> iface) throws SQLException {
return this.unwrap(iface);
}
#Override
public Connection getConnection() throws SQLException {
JdbcTransaction transaction = JdbcTransactionHolder.getCurrentTransaction();
if(transaction != null) {
// we get the connection from the transaction
logger.debug("Transaction exists for the thread: {}.", Thread.currentThread());
return transaction.getConnection();
}
Connection conn = this.dataSource.getConnection();
conn.setAutoCommit(false);
return conn;
}
#Override
public Connection getConnection(String username, String password)
throws SQLException {
JdbcTransaction transaction = JdbcTransactionHolder.getCurrentTransaction();
if(transaction != null) {
// we get the connection from the transaction
logger.debug("Transaction exists for the thread: {}.", Thread.currentThread());
return transaction.getConnection();
}
return this.dataSource.getConnection(username, password);
}
}
then create a DataSourceProvider so that we can inject DataSource to any POJO using guice:
public class DataSourceProvider implements Provider {
private static final Logger logger = LoggerFactory.getLogger(DataSourceProvider.class);
private DataSource dataSource;
public DataSourceProvider() {
JdbcConfig config = getConfig();
ComboPooledDataSource pooledDataSource = new ComboPooledDataSource();
try {
pooledDataSource.setDriverClass(config.getDriver());
} catch (Exception e) {
throw new RuntimeException(e);
}
pooledDataSource.setJdbcUrl(config.getUrl());
pooledDataSource.setUser(config.getUsername());
pooledDataSource.setPassword(config.getPassword() );
pooledDataSource.setMinPoolSize(config.getMinPoolSize());
pooledDataSource.setAcquireIncrement(5);
pooledDataSource.setMaxPoolSize(config.getMaxPoolSize());
pooledDataSource.setMaxStatements(config.getMaxStatementSize());
pooledDataSource.setAutoCommitOnClose(false);
this.dataSource = new JdbcDataSource(pooledDataSource);
}
private JdbcConfig getConfig() {
JdbcConfig config = new JdbcConfig();
Properties prop = new Properties();
try {
//load a properties file from class path, inside static method
prop.load(JdbcConfig.class.getResourceAsStream("/database.properties"));
//get the property value and print it out
config.setDriver(prop.getProperty("driver"));
config.setUrl(prop.getProperty("url"));
config.setUsername(prop.getProperty("username"));
config.setPassword(prop.getProperty("password"));
String maxPoolSize = prop.getProperty("maxPoolSize");
if(maxPoolSize != null) {
config.setMaxPoolSize(Integer.parseInt(maxPoolSize));
}
String maxStatementSize = prop.getProperty("maxStatementSize");
if(maxStatementSize != null) {
config.setMaxStatementSize(Integer.parseInt(maxStatementSize));
}
String minPoolSize = prop.getProperty("minPoolSize");
if(minPoolSize != null) {
config.setMinPoolSize(Integer.parseInt(minPoolSize));
}
}
catch (Exception ex) {
logger.error("Failed to load the config file!", ex);
throw new DBException("Cannot read the config file: database.properties. Please make sure the file is present in classpath.", ex);
}
return config;
}
#Override
public DataSource get() {
return dataSource;
}
and then implement TransactionalMethodInterceptor to manage the transaction for the method with Transactional annotation:
public class TransactionalMethodInterceptor implements MethodInterceptor {
private final static Logger logger = LoggerFactory.getLogger(TransactionalMethodInterceptor.class);
#Inject
private JdbcTransactionManager transactionManager;
#Override
public Object invoke(MethodInvocation method) throws Throwable {
try {
// Start the transaction
transactionManager.begin();
logger.debug("Start to invoke the method: " + method);
Object result = method.proceed();
logger.debug("Finish invoking the method: " + method);
transactionManager.commit();
return result;
} catch (Exception e) {
logger.error("Failed to commit transaction!", e);
try {
transactionManager.rollback();
}
catch(Exception ex) {
logger.warn("Cannot roll back transaction!", ex);
}
throw e;
}
}
}
Finally, the code to put all together so that Guice can inject the instances:
bind(DataSource.class).toProvider(DataSourceProvider.class).in(Scopes.SINGLETON);
bind(TransactionManager.class).to(JdbcTransactionManager.class);
TransactionalMethodInterceptor transactionalMethodInterceptor = new TransactionalMethodInterceptor();
requestInjection(transactionalMethodInterceptor);
bindInterceptor(Matchers.any(), Matchers.annotatedWith(Transactional.class), transactionalMethodInterceptor);
bind(TestDao.class).to(JdbcTestDao.class);
bind(TestService.class).to(TestServiceImpl.class);
I use c3p0 for the datasource pool. so, it works just fine in my test.
I find another related question: Guice, JDBC and managing database connections
but so far I haven't find any similar approach, except something in SpringFramework. but even the implementation in Spring seems quite complex.
I would like to ask if anyone has any suggestion for this solution.
thanks.