Define hive-jdbc JNDI data source on WebSphere - jdbc

I am trying to setup a JNDI Data Source on WebSphere 8.5.5.11, using hive-jdbc.jar.
When using the console in WebSphere, and using the form to create a new JDBC provider, there is a field for the implementation class name. WebSphere requires that the class implements javax.sql.XADataSource or javax.sql.ConnectionPoolDataSource. However, the hive-jdbc driver implements non of those, it implements only java.sql.DataSource.
For this reason, it doesn't work, WebSphere reports an error when trying to save the form.
Any idea what can I do about this?

You can write a trivial implementation of javax.sql.ConnectionPoolDataSource that delegates to the javax.sql.DataSource implementation. Here is an example,
package example.datasource;
import java.sql.*;
import javax.sql.*;
public class HiveConnectionPoolDataSource extends org.apache.hive.jdbc.HiveDataSource implements ConnectionPoolDataSource {
public PooledConnection getPooledConnection() throws SQLException {
return new HivePooledConnection(null, null);
}
public PooledConnection getPooledConnection(String user, String password) throws SQLException {
return new HivePooledConnection(user, password);
}
public boolean isWrapperFor(Class<?> iface) throws SQLException {
return ConnectionPoolDataSource.class.equals(iface) || super.isWrapperFor(iface);
}
public <T> T unwrap(Class<T> iface) throws SQLException {
return ConnectionPoolDataSource.class.equals(iface) ? (T) this : super.unwrap(iface);
}
class HivePooledConnection implements PooledConnection {
private Connection con;
private final String user;
private final String password;
HivePooledConnection(String user, String password) {
this.user = user;
this.password = password;
}
public void addConnectionEventListener(ConnectionEventListener listener) {}
public void addStatementEventListener(StatementEventListener listener) {}
public void close() throws SQLException {
if (con != null) {
con.close();
con = null;
}
}
public Connection getConnection() throws SQLException {
if (con == null || con.isClosed()) {
con = user == null
? HiveConnectionPoolDataSource.this.getConnection()
: HiveConnectionPoolDataSource.this.getConnection(user, password);
return con;
} else
throw new IllegalStateException();
}
public void removeConnectionEventListener(ConnectionEventListener listener) {}
public void removeStatementEventListener(StatementEventListener listener) {}
}
}
Package your compiled class in a JAR alongside the JDBC driver JAR(s), and configure your custom JDBC provider in WebSphere Application Server to point at this JAR as though it were a part of the JDBC driver. Specify the implementation class name as example.datasource.HiveConnectionPoolDataSource or whatever package/name you chose for your own implementation. You should then be able to use the JDBC driver.
Also adding a link to the WebSphere Application Server request for enhancements page if anyone wants to request that support for javax.sql.DataSource be added.

Related

How to mock bean HikariDataSource correctly?

I wrote integration test using Mockito, but it works when connection to database was set. Actually test just check possibility access some endpoints and not related to the data access layer. So I don't need database for it yet.
Reason of failing test when database is down - HikariDatasource check connection to the database when spring instantiates context. Mocking doesn't return Connection and it lead to the fail of application. Solution that i have found is use hsql in memory database, but for me it looks like work around. Probably exists other solution providing some fake data?
Not sure that this is elegant solution, but I need to force work tests like this
mockMvc.perform(
post("/some").contentType(MediaType.APPLICATION_JSON_UTF8)
.content(objectMapper.writeValueAsString(someDto))
.header(HttpHeaders.AUTHORIZATION, AUTH_HEADER)
.accept(MediaType.APPLICATION_JSON_UTF8)
).andExpect(status().is(201));
After debugging and searching I have found solution that allowed start container without database in memory.
#TestConfiguration
#ComponentScan(basePackages = "com.test")
#ActiveProfiles("test")
public class TestConfig {
//Other Beans
#Bean
public DataSource getDatasource() {
return new MockDataSource();
}
}
class MockDataSource implements DataSource {
#Override
public Connection getConnection() throws SQLException {
return createMockConnection();
}
#Override
public Connection getConnection(String username, String password) throws SQLException {
return getConnection();
}
#Override
public PrintWriter getLogWriter() throws SQLException {
return null;
}
#Override
public void setLogWriter(PrintWriter out) throws SQLException {
}
#Override
public void setLoginTimeout(int seconds) throws SQLException {
}
#Override
public int getLoginTimeout() throws SQLException {
return 0;
}
public Logger getParentLogger() throws SQLFeatureNotSupportedException {
return null;
}
#Override
public <T> T unwrap(Class<T> iface) throws SQLException {
return null;
}
#Override
public boolean isWrapperFor(Class<?> iface) throws SQLException {
return false;
}
public static Connection createMockConnection() throws SQLException {
// Setup mock connection
final Connection mockConnection = mock(Connection.class);
// Autocommit is always true by default
when(mockConnection.getAutoCommit()).thenReturn(true);
// Handle Connection.createStatement()
Statement statement = mock(Statement.class);
when(mockConnection.createStatement()).thenReturn(statement);
when(mockConnection.createStatement(anyInt(), anyInt())).thenReturn(statement);
when(mockConnection.createStatement(anyInt(), anyInt(), anyInt())).thenReturn(statement);
when(mockConnection.isValid(anyInt())).thenReturn(true);
// Handle Connection.prepareStatement()
PreparedStatement mockPreparedStatement = mock(PreparedStatement.class);
when(mockConnection.prepareStatement(anyString())).thenReturn(mockPreparedStatement);
when(mockConnection.prepareStatement(anyString(), anyInt())).thenReturn(mockPreparedStatement);
when(mockConnection.prepareStatement(anyString(), (int[]) any())).thenReturn(mockPreparedStatement);
when(mockConnection.prepareStatement(anyString(), (String[]) any())).thenReturn(mockPreparedStatement);
when(mockConnection.prepareStatement(anyString(), anyInt(), anyInt())).thenReturn(mockPreparedStatement);
when(mockConnection.prepareStatement(anyString(), anyInt(), anyInt(), anyInt())).thenReturn(mockPreparedStatement);
doAnswer((Answer<Void>) invocation -> null).doNothing().when(mockPreparedStatement).setInt(anyInt(), anyInt());
ResultSet mockResultSet = mock(ResultSet.class);
when(mockPreparedStatement.executeQuery()).thenReturn(mockResultSet);
when(mockResultSet.getString(anyInt())).thenReturn("aString");
when(mockResultSet.next()).thenReturn(true);
// Handle Connection.prepareCall()
CallableStatement mockCallableStatement = mock(CallableStatement.class);
when(mockConnection.prepareCall(anyString())).thenReturn(mockCallableStatement);
when(mockConnection.prepareCall(anyString(), anyInt(), anyInt())).thenReturn(mockCallableStatement);
when(mockConnection.prepareCall(anyString(), anyInt(), anyInt(), anyInt())).thenReturn(mockCallableStatement);
ResultSet mockResultSetTypeInfo = mock(ResultSet.class);
DatabaseMetaData mockDataBaseMetadata = mock(DatabaseMetaData.class);
when(mockDataBaseMetadata.getDatabaseProductName()).thenReturn("PostgreSQL");
when(mockDataBaseMetadata.getDatabaseMajorVersion()).thenReturn(8);
when(mockDataBaseMetadata.getDatabaseMinorVersion()).thenReturn(2);
when(mockDataBaseMetadata.getConnection()).thenReturn(mockConnection);
when(mockDataBaseMetadata.getTypeInfo()).thenReturn(mockResultSetTypeInfo);
when(mockConnection.getMetaData()).thenReturn(mockDataBaseMetadata);
// Handle Connection.close()
doAnswer((Answer<Void>) invocation -> null).doThrow(new SQLException("Connection is already closed")).when(mockConnection).close();
// Handle Connection.commit()
doAnswer((Answer<Void>) invocation -> null).doThrow(new SQLException("Transaction already committed")).when(mockConnection).commit();
// Handle Connection.rollback()
doAnswer((Answer<Void>) invocation -> null).doThrow(new SQLException("Transaction already rolled back")).when(mockConnection).rollback();
return mockConnection;
}
}
Mocking DataSource allows start container and provide post call to the controller using MockMvc.

can we connect with two DataBase Schemas with spring data jpa using same repository , model

I want to connect to two Database schemas which are on same database server , for that I want to use only one set of models and JPA repositories , with two data soources connecting to two different schemas .But as of now i am not able to find a way where we can reuse the existing models and repositories , for now I have created two sets of models and repositories having different schemas .Is there a way we can reuse the models and Repositories ?
But as of now i am not able to find a way where we can reuse the existing models and repositories , for now I have created two sets of models and repositories having different schemas
NOTE:
i am able to connect to two schemas via two data sources hence multi-tenancy support is not needed the only thing is while connecting to two separate schemas i am creating models(enitities) and JPA repositories twices even though tables are identical in both the schemas , is there are a to remove code duplicacy
looks like you need multi-tenancy support. Tenant per schema
You need TenantInterceptor which resolves tenantId e.g. from session or JWT token. And MultiTenantConnectionProvider which returns desired provider.
Some code from Multi-Tenancy Implementation for Spring Boot + Hibernate Projects
#Component
public class TenantInterceptor extends HandlerInterceptorAdapter {
#Autowired
private JwtTokenUtil jwtTokenUtil;
#Value("${jwt.header}")
private String tokenHeader;
#Override
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler)
throws Exception {
String authToken = request.getHeader(this.tokenHeader);
String tenantId = jwtTokenUtil.getTenantIdFromToken(authToken);
TenantContext.setCurrentTenant(tenantId);
return true;
}
#Override
public void postHandle(
HttpServletRequest request, HttpServletResponse response, Object handler, ModelAndView modelAndView)
throws Exception {
TenantContext.clear();
}
}
and
#Component
public class MultiTenantConnectionProviderImpl implements MultiTenantConnectionProvider {
#Autowired
private DataSource dataSource;
#Override
public Connection getAnyConnection() throws SQLException {
return dataSource.getConnection();
}
#Override
public void releaseAnyConnection(Connection connection) throws SQLException {
connection.close();
}
#Override
public Connection getConnection(String tenantIdentifie) throws SQLException {
String tenantIdentifier = TenantContext.getCurrentTenant();
final Connection connection = getAnyConnection();
try {
if (tenantIdentifier != null) {
connection.createStatement().execute("USE " + tenantIdentifier);
} else {
connection.createStatement().execute("USE " + DEFAULT_TENANT_ID);
}
}
catch ( SQLException e ) {
throw new HibernateException(
"Problem setting schema to " + tenantIdentifier,
e
);
}
return connection;
}
#Override
public void releaseConnection(String tenantIdentifier, Connection connection) throws SQLException {
try {
connection.createStatement().execute( "USE " + DEFAULT_TENANT_ID );
}
catch ( SQLException e ) {
throw new HibernateException(
"Problem setting schema to " + tenantIdentifier,
e
);
}
connection.close();
}
#SuppressWarnings("rawtypes")
#Override
public boolean isUnwrappableAs(Class unwrapType) {
return false;
}
#Override
public <T> T unwrap(Class<T> unwrapType) {
return null;
}
#Override
public boolean supportsAggressiveRelease() {
return true;
}
}

Spring Boot + Hibernate + Oracle schema multitenancy

I'm trying to get a schema-based multitenancy solution working, similar to this example but with Oracle instead of Postgres.
For example, I have three schemas: FOO, BAR, and BAZ. BAR and BAZ each have a table called MESSAGES. FOO has been granted SELECT on both BAR.MESSAGES and BAZ.MESSAGES. So if I connect as FOO and then execute
SELECT * FROM BAR.MESSAGES;
then I get a result as expected. But if I leave out the schema name (e.g. SELECT * FROM MESSAGES), then I get ORA-00942: table or view does not exist (the connection is using the wrong schema).
Here's my Dao / repository:
#Repository
public interface MessageDao extends CrudRepository<Foo, Long> {
}
The controller:
#GetMapping("/findAll")
public List<Message> findAll() {
TenantContext.setCurrentTenant("BAR");
var result = messageDao.findAll();
return result;
}
The Config:
#Configuration
public class MessageConfig {
#Bean
public JpaVendorAdapter jpaVendorAdapter() {
return new HibernateJpaVendorAdapter();
}
#Bean
public LocalContainerEntityManagerFactoryBean entityManagerFactory(DataSource dataSource,
MultiTenantConnectionProvider multiTenantConnectionProvider,
CurrentTenantIdentifierResolver tenantIdentifierResolver) {
LocalContainerEntityManagerFactoryBean em = new LocalContainerEntityManagerFactoryBean();
em.setDataSource(dataSource);
em.setPackagesToScan(Message.class.getPackageName());
em.setJpaVendorAdapter(this.jpaVendorAdapter());
Map<String, Object> jpaProperties = new HashMap<>();
jpaProperties.put(Environment.MULTI_TENANT, MultiTenancyStrategy.SCHEMA);
jpaProperties.put(Environment.MULTI_TENANT_CONNECTION_PROVIDER, multiTenantConnectionProvider);
jpaProperties.put(Environment.MULTI_TENANT_IDENTIFIER_RESOLVER, tenantIdentifierResolver);
jpaProperties.put(Environment.FORMAT_SQL, true);
em.setJpaPropertyMap(jpaProperties);
return em;
}
The MultitenantConnectionProvider:
#Component
public class MultiTenantConnectionProviderImpl implements MultiTenantConnectionProvider {
#Autowired
private DataSource dataSource;
#Override
public Connection getAnyConnection() throws SQLException {
return dataSource.getConnection();
}
#Override
public void releaseAnyConnection(Connection connection) throws SQLException {
connection.close();
}
#Override
public Connection getConnection(String currentTenantIdentifier) throws SQLException {
String tenantIdentifier = TenantContext.getCurrentTenant();
final Connection connection = getAnyConnection();
try (Statement statement = connection.createStatement()) {
statement.execute("ALTER SESSION SET CURRENT_SCHEMA = BAR");
} catch (SQLException e) {
throw new HibernateException("Problem setting schema to " + tenantIdentifier, e);
}
return connection;
}
#Override
public void releaseConnection(String tenantIdentifier, Connection connection) throws SQLException {
try (Statement statement = connection.createStatement()) {
statement.execute("ALTER SESSION SET CURRENT_SCHEMA = FOO");
} catch (SQLException e) {
throw new HibernateException("Problem setting schema to " + tenantIdentifier, e);
}
connection.close();
}
#SuppressWarnings("rawtypes")
#Override
public boolean isUnwrappableAs(Class unwrapType) {
return false;
}
#Override
public <T> T unwrap(Class<T> unwrapType) {
return null;
}
#Override
public boolean supportsAggressiveRelease() {
return true;
}
}
And the TenantIdentifierResolver (though not really relevant because I'm hard-coding the tenants right now in the ConnectionProviderImpl above):
#Component
public class TenantIdentifierResolver implements CurrentTenantIdentifierResolver {
#Override
public String resolveCurrentTenantIdentifier() {
String tenantId = TenantContext.getCurrentTenant();
if (tenantId != null) {
return tenantId;
}
return "BAR";
}
#Override
public boolean validateExistingCurrentSessions() {
return true;
}
}
Any ideas as to why the underlying Connection isn't switching schemas as expected?
UPDATE 1
Maybe it's something to do with the underlying Oracle connection. There is a property on OracleConnection named CONNECTION_PROPERTY_CREATE_DESCRIPTOR_USE_CURRENT_SCHEMA_FOR_SCHEMA_NAME. The documentation says:
The user also has an option to append the CURRENT_USER value to the
ADT name to obtain fully qualified name by setting this property to
true. Note that it takes a network round trip to fetch the
CURRENT_SCHEMA value.
But the problem remains even if I set this to true (-Doracle.jdbc.createDescriptorUseCurrentSchemaForSchemaName=true). This may be because the "username" on the Connection is still "FOO", even after altering the sesssion to set the schema to "BAR" (currentSchema on the Connection is "BAR"). But that would mean that the OracleConnection documentation is incorrect, wouldn't it?
UPDATE 2
I did not include the fact that we are using Spring Data JPA here as well. Maybe that has something to do with the problem?
I have found that if I include the schema name hard-coded in the entity then it works (e.g. #Table(schema="BAR")), but having a hard-coded value there is not an acceptable solution.
It might also work if we rewrite the queries as native #Query and then include {h-schema} in the SQL, but in Hibernate this is the default schema, not the 'current' (dynamic) schema, so it's not quite right either.
It turns out that setting the current tenant on the first line of the Controller like that (TenantContext.setCurrentTenant("BAR")) is "too late" (Spring has already created a transaction?). I changed the implementation to use a servlet filter to set the tenant id from a header to a request attribute, and then fetch that attribute in the TenantIdentifierResolver, instead of using the TenantContext. Now it works as it should, without any of the stuff I mentioned in the updates.

Change Multitenancy sessions manually

I need to create a multitenanacy application with ability to switch between schemas inside my java-code (not based on a user request).
I've read articles:
https://fizzylogic.nl/2016/01/24/make-your-spring-boot-application-multi-tenant-aware-in-2-steps/
http://www.greggbolinger.com/tenant-per-schema-with-spring-boot/
Solution works fine, when the schema is passed in Rest-request.
However I need to implement the following logic:
public void compare(String originalSchema, String secondSchema){
TenantContext.setCurrentTenant(originalSchema);
List<MyObject> originalData = myRepository.findData();
TenantContext.setCurrentTenant(secondSchema);
List<MyObject> migratedData = myRepository.findData();
}
The point is, that connection is not switched, when I manually set up TenenantContext. MultiTenantConnectionProviderImpl.getConnection is invoked only on the first call to my repository.
#Component
public class MultiTenantConnectionProviderImpl implements MultiTenantConnectionProvider {
#Override
public Connection getConnection(String tenantIdentifier) throws SQLException {
final Connection connection = getAnyConnection();
try {
connection.createStatement().execute( "ALTER SESSION SET CURRENT_SCHEMA = " + tenantIdentifier );
}
catch ( SQLException e ) {
throw new HibernateException(
"Could not alter JDBC connection to specified schema [" + tenantIdentifier + "]",e);
}
return connection;
}
}
Is it possible to force switching sessions?
Found a hard-coded solution.
#Service
public class DatabaseSessionManager {
#PersistenceUnit
private EntityManagerFactory entityManagerFactory;
public void bindSession() {
if (!TransactionSynchronizationManager.hasResource(entityManagerFactory)) {
EntityManager entityManager = entityManagerFactory.createEntityManager();
TransactionSynchronizationManager.bindResource(entityManagerFactory, new EntityManagerHolder(entityManager));
}
}
public void unbindSession() {
EntityManagerHolder emHolder = (EntityManagerHolder) TransactionSynchronizationManager
.unbindResource(entityManagerFactory);
EntityManagerFactoryUtils.closeEntityManager(emHolder.getEntityManager());
}
}
Each block, loading data in a new tenantContext should execute the following:
databaseSessionManager.unbindSession();
TenantContext.setCurrentTenant(schema);
databaseSessionManager.bindSession();
//execute selects
Well, you need it
public interface Service {
List<MyObject> myObjects();
}
#Service
#Transactional(propagation = Propagation.REQUIRES_NEW)
public class ServiceImpl implements Service {
#Autowired
private MyRepository myRepository;
#Override
public List<MyObject> myObjects() {
return myRepository.findData();
}
}
#Service
public class AnotherService() {
#Autowired
private Service service;
public void compare(String originalSchema, String secondSchema){
TenantContext.setCurrentTenant(originalSchema);
List<MyObject> originalData = service.myObjects();
TenantContext.setCurrentTenant(secondSchema);
List<MyObject> migratedData = service.myObjects();
}
}
Try using
spring.jpa.open-in-view=false
in your application.properties file.
More info on this
What is this spring.jpa.open-in-view=true property in Spring Boot?
Hope this helps..

Session management in a Neo4J Flink sink

I am developing a data analytics app with Apache Flink and Neo4J (Community Edition).
In this application, the Flink sink must save/update relations in Neo4J.
Which is the best way for Neo4J session management, and why?
First implementation:
public class MySink extends RichSinkFunction<Link> {
private DbConfiguration dbconfig;
private Driver driver;
#Override
public void open(Configuration parameters) throws Exception {
this.driver = Neo4JManager.open(this.dbconfig);
}
#Override
public void close() throws Exception {
this.driver.close();
}
#Override
public void invoke(Link link) throws Exception {
Session session = this.driver.session();
Neo4JManager.saveLink(session, link);
session.close();
}
}
Second implementation:
public class MySink extends RichSinkFunction<Link> {
private DbConfiguration dbconfig;
private Driver driver;
private Session session;
#Override
public void open(Configuration parameters) throws Exception {
this.driver = Neo4JManager.open(this.dbconfig);
this.session = driver.session();
}
#Override
public void close() throws Exception {
this.session.close();
this.driver.close();
}
#Override
public void invoke(Link link) throws Exception {
Neo4JManager.saveLink(this.session, link);
}
}
In both implementations, the following functions have been used:
public class Neo4JManager {
public static Driver open(DbConfiguration dbconf) {
AuthToken auth = AuthTokens.basic(dbconf.getUsername(), dbconf.getPassword());
Config config = Config.build().withEncryptionLevel(Config.EncryptionLevel.NONE ).toConfig();
return GraphDatabase.driver(dbconf.getHostname(), auth, config);
}
public static void saveLink(Session session, Link link) {
Value params = parameters("x", link.x, "y", link.y);
session.run('CREATE (Person {id:{x}}-[FOLLOWS]->(Person {id:{y}}))'
}
}
Thank you.

Resources