Access to postgresql server from Spring application - spring

I have a Spring application (Hibernate c3p0) that connects well to an Oracle DB server.
I want to do the same thing with a PostgreSQL DB server, but it sends me the error java.net.UnknownHostException.
In applicationContext, my dataSource is :
<bean id="appDS" class="com.mchange.v2.c3p0.ComboPooledDataSource">
<property name="driverClass" value="org.postgresql.Driver" />
<property name="jdbcUrl" value="jdbc:postgresql://THE_SERVER:5432/schema" />
<property name="user" value="appc0" />
<property name="password" value="THE_PWD" />
</bean>
I tried a connection from another very simple java application (from the same server):
public static void main(String[] args) {
try {
System.out.println("TEST Driver Manager START");
Class.forName("org.postgresql.Driver");
String url = "jdbc:postgresql://THE_SERVER:5432/schema";
String user = "appc0";
String passwd = "THE_PWD";
Connection conn = DriverManager.getConnection(url, user, passwd);
System.out.println("Schema : " + conn.getSchema());
System.out.println("ClientInfo : " + conn.getClientInfo());
System.out.println("TEST Driver Manager END");
System.out.println("TEST C3P0 START");
ComboPooledDataSource cpds = new ComboPooledDataSource();
cpds.setDriverClass( "org.postgresql.Driver" );
cpds.setJdbcUrl( "jdbc:postgresql://THE_SERVER:5432/schema" );
cpds.setUser("appc0");
cpds.setPassword("THE_PWD");
System.out.println("User :"+ cpds.getUser());
System.out.println("TEST C3P0 END");
} catch(Exception e) {
e.printStackTrace();
}
And no mistake in this case, I connect well. This does not seem to be a network problem.
Does anyone know where the problem may come from?
Versions are:
postgresql-42.2.0.jre7.jar
c3p0-0.9.1.2.jar

Related

ActiveMQ Broker inside WSL or docker throws invalid username/password; logon denied

I wrote a simple program to connect to a DB using a spring bean from inside Docker or WSl2. It connects fine.
The moment I add a broker tag to the XML it fails with error.
NOTE : the 'broker' tag, even when added as empty, commenting everything out, causes the same issue.
Below is my program and the bean XML
public class TestDB {
public static void main(String[] args) {
Connection connection1 = null, connection2 = null;
try (ConfigurableApplicationContext mContext = new ClassPathXmlApplicationContext("test.xml");) {
String jdbcClassName="oracle.jdbc.OracleDriver";
String url="jdbc:oracle:thin:#xyz.company.com:1521:abc";
String user="TEST_DB";
String password="test";
BasicDataSource ds = new BasicDataSource();
ds.setDriverClassName(jdbcClassName);
ds.setUrl(url);
ds.setUsername(user);
ds.setPassword(password);
connection1 = ds.getConnection();
BasicDataSource ds2 = (BasicDataSource) mContext.getBean("application-ds");
connection2 = ds2.getConnection();
}
catch (SQLException e) {
e.printStackTrace();
} finally {
if (connection1!=null) {
System.out.println("Connected successfully.");
try {
connection1.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
if (connection2!=null) {
System.out.println("Connected via bean.");
try {
connection2.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
}
}
}
Here is the test.xml:
<beans .....>
<broker xmlns="http://activemq.apache.org/schema/core" persistent="true" brokerName="ABC_MESSAGING_SERVER" advisorySupport="false" useJmx="false">
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry queue="my_account" />
</policyEntries>
</policyMap>
</destinationPolicy>
<persistenceAdapter>
<jdbcPersistenceAdapter dataSource="#application-ds"/>
</persistenceAdapter>
</broker>
<bean id="application-ds" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="oracle.jdbc.OracleDriver" />
<property name="url" value="jdbc:oracle:thin:#xyz.company.com:1521:abc" />
<property name="username" value="TEST_DB" />
<property name="password" value="test" />
<property name="maxActive" value="200" />
<property name="poolPreparedStatements" value="true" />
</bean>
</beans>
The output, with the broker part commented is:
and with broker :
org.apache.commons.dbcp.SQLNestedException: Cannot create PoolableConnectionFactory (ORA-01017: invalid username/password; logon denied)
at org.apache.commons.dbcp.BasicDataSource.createPoolableConnectionFactory(BasicDataSource.java:1549)
at org.apache.commons.dbcp.BasicDataSource.createDataSource(BasicDataSource.java:1388)
at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044)
at TestDB.main(TestDB.java:29)
Caused by: java.sql.SQLException: ORA-01017: invalid username/password; logon denied
at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:509)
at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:456)
at oracle.jdbc.driver.T4CTTIoer11.processError(T4CTTIoer11.java:451)
at oracle.jdbc.driver.T4CTTIfun.processError(T4CTTIfun.java:1040)
at oracle.jdbc.driver.T4CTTIoauthenticate.processError(T4CTTIoauthenticate.java:552)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:550)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:268)
at oracle.jdbc.driver.T4CTTIoauthenticate.doOAUTH(T4CTTIoauthenticate.java:501)
at oracle.jdbc.driver.T4CTTIoauthenticate.doOAUTH(T4CTTIoauthenticate.java:1292)
at oracle.jdbc.driver.T4CTTIoauthenticate.doOAUTH(T4CTTIoauthenticate.java:1025)
at oracle.jdbc.driver.T4CConnection.logon(T4CConnection.java:743)
at oracle.jdbc.driver.PhysicalConnection.connect(PhysicalConnection.java:793)
at oracle.jdbc.driver.T4CDriverExtension.getConnection(T4CDriverExtension.java:57)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:747)
at oracle.jdbc.driver.OracleDriver.connect(OracleDriver.java:562)
at org.apache.commons.dbcp.DriverConnectionFactory.createConnection(DriverConnectionFactory.java:38)
at org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582)
at org.apache.commons.dbcp.BasicDataSource.validateConnectionFactory(BasicDataSource.java:1556)
I am answering my own question.
The problem was my Database was version Oracle 12.2.0.1 and my ojdbc jar was ojdbc8-19.3.0.0.
It did not matter when testing on widnows or linux machines.
However using it alongside activemq broker in WSL2 or Docker (virtual envs basically) then it caused 'Invalid username password logon error'.
had to replace the jar with ojdbc8-12.2.0.1 on client side

Can Spring JdbcTemplate connect to hive?

I am developing a java web project which is based on spring. And I want to use Spring JdbcTemplate to connect to hive. But when I tested my service, it came out this error message
"org.springframework.jdbc.CannotGetJdbcConnectionException: Could not
get JDBC Connection; nested exception is
org.apache.commons.dbcp.SQLNestedException: Cannot load JDBC driver
class 'org.apache.hadoop.hive.jdbc.HiveDrive'".
The project is created by idea maven, but the hive jdbc driver is a local jar(it is located at WEB-INF/lib). So I am not sure whether the error is caused by the problem that my project still can't recognize the local jdbc driver jar or just because JdbcTemplate does not support hive connection. Can someone help me figure it out? thank you in advance.
Here are my code:
JdbcTemplate definition:
<bean id="dataSourceTDW" class="org.apache.commons.dbcp.BasicDataSource"
destroy-method="close">
<property name="driverClassName" value="${driver}" />
<property name="url" value="${url}" />
<property name="username" value="${username}" />
<property name="password" value="${password}" />
</bean>
<bean id="jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate">
<property name="dataSource" ref="dataSourceTDW"/>
</bean>
DAO class:
#Repository(value = "tdwQueryImp")
public class QueryDAOImp implements QueryDAO {
#Autowired
JdbcTemplate jdbcTemplate;
public List<Map<String,Object>> execute(String sql) {
return jdbcTemplate.queryForList(sql);
}
}
Kerberos Example:
#Component
public class HiveDataSource extends SimpleDriverDataSource {
private static final Logger logger = LoggerFactory.getLogger(HiveDataSource.class);
private final Subject subject;
#Autowired
HiveDataSource(Subject subject, Driver hiveDriver, String jdbcUrl) {
this.subject = subject;
setUrl(jdbcUrl);
setDriver(hiveDriver);
}
#Override
protected Connection getConnectionFromDriver(final Properties props) throws SQLException {
try {
return Subject.doAs(subject, (PrivilegedExceptionAction<Connection>)() -> getDriver().connect(getUrl(), props));
} catch (Exception e) {
e.printStackTrace();
logger.error("Failed to get Hive JDBC connection.", e);
}
return null;
}
}
Here HiveDriver bean is defined as :
#Bean
HiveDriver hiveDriver() {
HiveDriver impl = new HiveDriver() {
#Override
public Connection connect(String url, Properties info) throws SQLException {
return acceptsURL(url) ? new HiveConnection(url, info) {
#Override
public void setAutoCommit(boolean autoCommit) throws SQLException {
/* do nothing */
};
} : null;
}
};
return impl;
}
I solved my problem by changing the class of datasource from org.apache.commons.dbcp.BasicDataSource to org.springframework.jdbc.datasource.SimpleDriverDataSource.
Here are the beans configuration:
<bean id="hiveDriver" class="org.apache.hadoop.hive.jdbc.HiveDriver"/>
<bean id="dataSourceTDW" class="org.springframework.jdbc.datasource.SimpleDriverDataSource">
<constructor-arg name="driver" ref="hiveDriver"/>
<constructor-arg name="url" value="${url}"/>
<constructor-arg name="username" value="${username}" />
<constructor-arg name="password" value="${password}" />
</bean>
<bean id="jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate">
<property name="dataSource" ref="dataSourceTDW"/>
</bean>
you can refer bellow link :
http://saurzcode.in/2015/01/connect-hiveserver2-service-jdbc-client/
and more
http://hadooptutorial.info/hive-jdbc-client-example/
https://community.hortonworks.com/articles/53629/writing-a-spring-boot-microservices-to-access-hive.html

TransactionTemplate and TransactionManager connection objects

The following code in my DAO works perfectly fine.
public void insert(final Person person) {
transactionTemplate.execute(new TransactionCallback<Void>() {
public Void doInTransaction(TransactionStatus txStatus) {
try {
getJdbcTemplate().execute("insert into person(username, password) values ('" + person.getUsername() + "','" + person.getPassword() + "')");
} catch (RuntimeException e) {
txStatus.setRollbackOnly();
throw e;
}
return null;
}
});
}
Following is my spring config.
<bean id="derbyds" class="org.apache.commons.dbcp.BasicDataSource">
<property name="driverClassName" value="org.apache.derby.jdbc.EmbeddedDriver" />
<property name="username" value="app" />
<property name="password" value="app" />
<property name="url" value="jdbc:derby:mytempdb" />
</bean>
<bean id="persondaojdbc" class="com.napp.dao.impl.PersonDaoJdbcImpl">
<property name="dataSource" ref="derbyds" />
<property name="transactionTemplate">
<bean class="org.springframework.transaction.support.TransactionTemplate">
<property name="transactionManager" ref="transactionManager" />
</bean>
</property>
</bean>
<bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="derbyds"/>
</bean>
What i wanted to know is how does the
<bean id="persondaojdbc" class="com.napp.dao.impl.PersonDaoJdbcImpl">
<property name="dataSource" ref="derbyds" />
<property name="transactionTemplate">
<bean class="org.springframework.transaction.support.TransactionTemplate">
<property name="transactionManager" ref="transactionManager" />
</bean>
</property>
</bean>
<bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="derbyds"/>
</bean>
Now, its imperative that both the TransactionManager and the code (in my case jdbc template) operate on the same connection. I am assuming both of them are getting the Connection objects from the DataSource. DataSource pools connections and its a chance when you call getConnection multiple times, you will get different Connection obejcts. How does spring make sure that the TransactionManager and JdbcTemplate end up getting the same connection objects. My understanding is that, if that doesn't happen, rollbacks, or commits wont work, correct? Could someone throw more light on this.
If you look at the code for JdbcTemplate (one of the execute(...) methods) you will see
Connection con = DataSourceUtils.getConnection(getDataSource());
Which tries to retrieve a Connection from a ConnectionHolder registered with a TransactionSynchronizationManager.
If there is no such object, it just gets a connection from the DataSource and registers it (if it is in a transactional environment, ie. you have a transaction manager). Otherwise, it immediately returns the registered object.
This is the code (stripped of logs and stuff)
ConnectionHolder conHolder = (ConnectionHolder) TransactionSynchronizationManager.getResource(dataSource);
if (conHolder != null && (conHolder.hasConnection() || conHolder.isSynchronizedWithTransaction())) {
conHolder.requested();
if (!conHolder.hasConnection()) {
conHolder.setConnection(dataSource.getConnection());
}
return conHolder.getConnection();
}
// Else we either got no holder or an empty thread-bound holder here.
Connection con = dataSource.getConnection();
// flag set by the TransactionManager
if (TransactionSynchronizationManager.isSynchronizationActive()) {
// Use same Connection for further JDBC actions within the transaction.
// Thread-bound object will get removed by synchronization at transaction completion.
ConnectionHolder holderToUse = conHolder;
if (holderToUse == null) {
holderToUse = new ConnectionHolder(con);
}
else {
holderToUse.setConnection(con);
}
holderToUse.requested();
TransactionSynchronizationManager.registerSynchronization(
new ConnectionSynchronization(holderToUse, dataSource));
holderToUse.setSynchronizedWithTransaction(true);
if (holderToUse != conHolder) {
TransactionSynchronizationManager.bindResource(dataSource, holderToUse);
}
}
return con;
You'll notice that the JdbcTemplate tries to
finally {
DataSourceUtils.releaseConnection(con, getDataSource());
}
release the Connection, but this only happens if you're in a non-transactional environment, ie.
if it is not managed externally (that is, not bound to the thread).
Therefore, in a transactional world, the JdbcTemplate will be reusing the same Connection object.

Very simple Spring transactions of JDBC not roll back (even log said yes)

I'm new to Spring transactions. I use Spring 3.2.2 and MySQL 5.5.20(InnoDB). I can see in the log file that it did roll back, but in the database, the record still being updated to 9. What did I miss? Thanks.
The config.xml:
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName" value="com.mysql.jdbc.Driver" />
<property name="url" value="jdbc:mysql://127.0.0.1:3306/javatest?useUnicode=true&characterEncoding=UTF-8" />
<property name="username" value="root" />
<property name="password" value="xxx" />
</bean>
<bean id="txManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="dataSource"/>
</bean>
<bean id="hello" class="com.xol.oss.HelloService">
<property name="dataSource" ref="dataSource"/>
</bean>
<tx:annotation-driven transaction-manager="txManager"/>
The Java code:
public void setDataSource(BasicDataSource dataSource) {
this.dataSource = dataSource;
}
#Transactional
public void getData() {
Connection con=null;
try {
con = dataSource.getConnection();
Statement stat = con.createStatement();
stat.executeUpdate("update testdata set foo=9 where id=1");
throw new RuntimeException("an Exception for test");
} catch (SQLException e) {
e.printStackTrace();
} finally{
try {
con.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
}
The log said it did roll back:
15:15:36,936 DEBUG DataSourceTransactionManager:366 - Creating new transaction with name [com.xol.oss.HelloService.getData]: PROPAGATION_REQUIRED,ISOLATION_DEFAULT; ''
15:15:37,525 DEBUG DataSourceTransactionManager:205 - Acquired Connection [jdbc:mysql://127.0.0.1:3306/javatest?useUnicode=true&characterEncoding=UTF-8, UserName=root#localhost, MySQL-AB JDBC Driver] for JDBC transaction
15:15:37,535 DEBUG DataSourceTransactionManager:222 - Switching JDBC Connection [jdbc:mysql://127.0.0.1:3306/javatest?useUnicode=true&characterEncoding=UTF-8, UserName=root#localhost, MySQL-AB JDBC Driver] to manual commit
15:15:37,581 DEBUG DataSourceTransactionManager:844 - Initiating transaction rollback
15:15:37,582 DEBUG DataSourceTransactionManager:280 - Rolling back JDBC transaction on Connection [jdbc:mysql://127.0.0.1:3306/javatest?useUnicode=true&characterEncoding=UTF-8, UserName=root#localhost, MySQL-AB JDBC Driver]
15:15:37,583 DEBUG DataSourceTransactionManager:323 - Releasing JDBC Connection [jdbc:mysql://127.0.0.1:3306/javatest?useUnicode=true&characterEncoding=UTF-8, UserName=root#localhost, MySQL-AB JDBC Driver] after transaction
15:15:37,583 DEBUG DataSourceUtils:327 - Returning JDBC Connection to DataSource
Exception in thread "main" java.lang.RuntimeException: an RuntimeException for test
at com.xol.oss.HelloService.getData(HelloService.java:31)
at com.xol.oss.HelloService$$FastClassByCGLIB$$3d7d84e8.invoke(<generated>)
The problem is that you are not using the connection managed by spring, instead you are opening a new connection. Change the code to the following and try.
import org.springframework.jdbc.datasource.DataSourceUtils;
#Transactional
public void getData() {
Connection con=null;
try {
// Get the connection associated with the transaction
con = DataSourceUtils.getConnection(dataSource);
Statement stat = con.createStatement();
stat.executeUpdate("update testdata set foo=9 where id=1");
throw new RuntimeException("an Exception for test");
} catch (SQLException e) {
e.printStackTrace();
} finally{
DataSourceUtils.releaseConnection(dataSource, con);
}
}
If you are writing new code you should be using the JdbcTemplate instead of raw jdbc.
class HelloService {
JdbcTemplate jdbcTemplate;
public setDataSource(DataSource dataSource) {
jdbcTemplate = new JDBCTemplate(dataSource);
}
#Transactional
public void getData() {
jdbcTemplate.update(update testdata set foo=9 where id=1);
throw new RuntimeException("an Exception for test");
}

BasicDataSource with JDBCTemplate not pooling connections as expected

I am trying to use BasicDataSource to pool connections with JDBCTemplate in a spring application. From everything I've read, this should be really simple: Just configure the BasicDataSource in XML, inject the data source into a bean, and in the setter method, create a new JDBCTemplate.
When I did this, I noticed my performance was terrible. So then I switched to Spring's SingleConnectionDataSource, just to see what would happen, and my performance got much better. I started investigating with a profiler tool, and I noticed that when using BasicDataSource, a new connection was being created for every query.
Investigating further, I can see where the connection is being closed after the query is finished. Specifically in Spring's DataSourceUtil class:
public static void doReleaseConnection(Connection con, DataSource dataSource) throws SQLException {
if (con == null) {
return;
}
if (dataSource != null) {
ConnectionHolder conHolder = (ConnectionHolder) TransactionSynchronizationManager.getResource(dataSource);
if (conHolder != null && connectionEquals(conHolder, con)) {
// It's the transactional Connection: Don't close it.
conHolder.released();
return;
}
}
// Leave the Connection open only if the DataSource is our
// special SmartDataSoruce and it wants the Connection left open.
if (!(dataSource instanceof SmartDataSource) || ((SmartDataSource) dataSource).shouldClose(con)) {
logger.debug("Returning JDBC Connection to DataSource");
con.close();
}
}
The thing that I notice is there is some special logic for 'SmartDataSource' which leaves the connection open. This partially explains the behavior I was seeing: Since SingleConnectionDataSource implements SmartDataSource, the connection is not closed. However, I thought by using BasicDataSource, the close() method on the connection would just return the connection to the pool. However, when I look at what is happening in my profiler, the close method is actually being called on my sybase connection: not any kind of 'Pooled Connection Wrapper' like I would expect to see.
One last thing (this is what I'm going to investigate now): I use TransactionTemplate for some of my queries (involving commits to the database), but simple queries are not inside a transactionTemplate. I don't know if that has anything to do with the problem or not.
EDIT 1:
Ok finally got some more time to investigate after getting pulled off the project a bit and, here is a very simple test that shows the problem
public class DBConnectionPoolTest {
#Autowired
#Qualifier("myDataSource")
private DataSource dataSource;
#Test
public void test() throws Exception{
JdbcTemplate template = new JdbcTemplate(dataSource);
StopWatch sw = new StopWatch();
sw.start();
for(int i=0; i<1000; i++){
template.queryForInt("select count(*) from mytable");
}
sw.stop();
System.out.println("TIME: " + sw.getTotalTimeSeconds() + " seconds");
}}
Here are my two datasource configurations:
<bean id="myDataSource" class="org.springframework.jdbc.datasource.SingleConnectionDataSource">
<property name="driverClassName" value="${db.driver}" />
<property name="url" value="${db.url}" />
<property name="username" value="${db.username}" />
<property name="password" value="${db.password}" />
</bean>
<bean id="myDataSource" class="org.apache.commons.dbcp.BasicDataSource">
<property name="driverClassName" value="${db.driver}" />
<property name="url" value="${db.url}" />
<property name="username" value="${db.username}" />
<property name="password" value="${db.password}" />
</bean>
When I run the test with the first configuration, it takes about 2.1 seconds. When I run it with the second configuration, it takes about 4.5 seconds. I have tried various parameters on BasicDataSource, such as setting maxActive=1 and testOnBorrow=false, but nothing makes a difference.
I think the problem in my case was that my jdbc libraries for sybase were outdated.

Resources