I have created an H2 alias function which is being called when I execute the corresponding spring jdbc StoredProcedure.
public static ResultSet dao_cashflow_read(Integer contractId, Integer contractVersion, Date rateDtmDate, String refSrc,
String rateStatus, String rateType, String message, Integer rowCount) throws SQLException, IOException {
File f = new File(FixingDaoIT.class.getResource("/").getFile().concat("csv/ldn/fixing.csv"));
return new Csv().read(new FileReader(f), null);
}
This part is working as expected as the associated row mapper function parses the result set correctly and generates the entity objects. The problem is, the JdbcTemplate class then attempts to extract the associated output parameters that were registered with the StoredProcedure implementation. Which in turn calls the method below in class JdbcCallableStatement.java
private JdbcResultSet getOpenResultSet() throws SQLException {
try {
checkClosed();
if (resultSet == null) {
throw DbException.get(ErrorCode.NO_DATA_AVAILABLE);
}
if (resultSet.isBeforeFirst()) {
resultSet.next();
}
return resultSet;
} catch (Exception e) {
throw logAndConvert(e);
}
}
This call fails as the resultSet reference is now null and it throws the DbException NO_DATA_AVAILBLE
If I run the application against Sybase, the output params are as expected. Is this an issue with H2 not supporting output params?
Related
I want to know whether it is possible to call stored procedure using Spring Data JPA which is having resultset and multiple out parameter.
I found Git issue for same https://github.com/spring-projects/spring-data-examples/issues/80
If it is resolved, could someone provide one example with Spring Boot?
The way I've accomplished this in the past is to add custom behavior to a Spring Data JPA repository (link). Inside that I get the EntityManager and use java.sql.Connection and CallableStatement
Edit: Adding high level sample code. Sample makes the assumption that you are using Hibernate but idea should be applicable to others as well
Assuming you have an EntityRepository
public interface EntityRepositoryCustom {
Result storedProcCall(Input input);
}
public class EntityRepositoryImpl implements EntityRepositoryCustom {
#PersistenceContext
private EntityManager em;
#Override
public Result storedProcCall(Input input) {
final Result result = new Result();
Session session = getSession();
// instead of anonymous class you could move this out to a private static class that implement org.hibernate.jdbc.Work
session.doWork(new Work() {
#Override
public void execute(Connection connection) throws SQLException {
CallableStatement cs = null;
try {
cs = connection.prepareCall("{call some_stored_proc(?, ?, ?, ?)}");
cs.setString(1, "");
cs.setString(2, "");
cs.registerOutParameter(3, Types.VARCHAR);
cs.registerOutParameter(4, Types.VARCHAR);
cs.execute();
// get value from output params and set fields on return object
result.setSomeField1(cs.getString(3));
result.setSomeField2(cs.getString(4));
cs.close();
} finally {
// close cs
}
}
});
return result;
}
private Session getSession() {
// get session from entitymanager. Assuming hibernate
return em.unwrap(org.hibernate.Session.class);
}
}
I have the following code snippet
ResultSet rs = stmt.executeQuery();
List<String> userIdList = new ArrayList<String>();
while(rs.next()){
userIdList.add(rs.getString(1));
}
Can I make use of Java streams/Lambda expressions to perform this iteration instead of a while loop to populate the List?
You may create a wrapper for the ResultSet making it an Iterable. From there you can iterate as well as create a stream. Of course you have to define a mapper function to get the iterated values from the result set.
The ResultSetIterable may look like this
public class ResultSetIterable<T> implements Iterable<T> {
private final ResultSet rs;
private final Function<ResultSet, T> onNext;
public ResultSetIterable(ResultSet rs, CheckedFunction<ResultSet, T> onNext){
this.rs = rs;
//onNext is the mapper function to get the values from the resultSet
this.onNext = onNext;
}
private boolean resultSetHasNext(){
try {
hasNext = rs.next();
} catch (SQLException e) {
//you should add proper exception handling here
throw new RuntimeException(e);
}
}
#Override
public Iterator<T> iterator() {
try {
return new Iterator<T>() {
//the iterator state is initialized by calling next() to
//know whether there are elements to iterate
boolean hasNext = resultSetHasNext();
#Override
public boolean hasNext() {
return hasNext;
}
#Override
public T next() {
T result = onNext.apply(rs);
//after each get, we need to update the hasNext info
hasNext = resultSetHasNext();
return result;
}
};
} catch (Exception e) {
//you should add proper exception handling here
throw new RuntimeException(e);
}
}
//adding stream support based on an iteratable is easy
public Stream<T> stream() {
return StreamSupport.stream(this.spliterator(), false);
}
}
Now that we have our wrapper, you could stream over the results:
ResultSet rs = stmt.executeQuery();
List<String> userIdList = new ResultSetIterable(rs, rs -> rs.getString(1)).stream()
.collect(Collectors.toList())
}
EDIT
As Lukas pointed out, the rs.getString(1) may throw a checked SQLException, therefor we need to use a CheckedFunction instead of a java Function that would be capable of wrapping any checked Exception in an unchecked one.
A very simple implementation could be
public interface CheckedFunction<T,R> extends Function<T,R> {
#Override
default R apply(T t) {
try {
return applyAndThrow(t);
} catch (Exception e) {
throw new RuntimeException(e);
}
}
R applyAndThrow(T t) throws Exception;
}
Alternatively you could use a library with such a function, i.e. jooλ or vavr
If using a third party library is an option, you could use jOOQ, which supports wrapping JDBC ResultSet in jOOQ Cursor types, and then stream them. For example, using DSLContext.fetchStream()
Essentially, you could write:
try (ResultSet rs = stmt.executeQuery()) {
DSL.using(con) // DSLContext
.fetchStream(rs) // Stream<Record>
.map(r -> r.get(0, String.class)) // Stream<String>
.collect(toList());
}
Disclaimer: I work for the vendor.
Try library: abacus-jdbc
List<String> userIdList = StreamEx.<String> rows(resultSet, 1).toList(); // Don't forget to close ResultSet
Or: If you want to close the ResultSet after toList.
StreamEx.<String> rows(resultSet, 1).onClose(() -> JdbcUtil.closeQuitely(resultSet)).toList();
Or: If you use the utility classes provided in abacus-jdbc:
String sql = "select user_id from user";
// No need to worry about closing Connection/Statement/ResultSet manually. It will be took care by the framework.
JdbcUtil.prepareQuery(dataSource, sql).stream(String.class).toList();
// Or:
JdbcUtil.prepareQuery(dataSource, sql).toList(String.class);
Disclaimer：I'm the developer of abacus-jdbc.
I want to bind a connection to a thread, and use that connection for any JdbcTemplate calls, to finally commit the changes or do a rollback.
I'm declaring all sentences from a Groovy script, so I can't control how many SQL query will be call, that's why I have to used this method and not a TransactionalTemplate or something like that. this script will call a helper class that will use that connection and JdbcTemplate, let's call that class SqlHelper.
Right now my non-working-as-needed solution is call from groovy script to that SqlHelper to initialize a transaction:
initTransaction(ecommerce)
which calls
public void initTransaction(DataSource dataSource) {
DefaultTransactionDefinition transactionDefinition = new DefaultTransactionDefinition();
transactionDefinition.setReadOnly(false);
transactionDefinition.setIsolationLevel(TransactionDefinition.ISOLATION_SERIALIZABLE);
// dataSourceTransactionManager.setDataSource(dataSource);
// dataSourceTransactionManager.setRollbackOnCommitFailure(true);
// dataSourceTransactionManager.setTransactionSynchronization(TransactionSynchronization.STATUS_COMMITTED);
// dataSourceTransactionManager.setDataSource(dataSource);
try {
Connection connection = DataSourceUtils.getConnection(dataSource);
connection.setAutoCommit(false);
DataSourceUtils.prepareConnectionForTransaction(connection, transactionDefinition);
} catch (CannotGetJdbcConnectionException e) {
throw new RuntimeException(e);
} catch (SQLException e) {
throw new RuntimeException(e);
}
}
After that the script will call some SQL operations, like
sqlUpdate(ecommerce, insertSentence, insertParams)
which calls
public Integer update(DataSource dataSource, String sql, Map<String, Object> paramMap) {
return new NamedParameterJdbcTemplate(dataSource).update(sql, paramMap);
}
Finally I want to finish the transaction committing the changes using
commitTransaction(dataSource)
which calls
public void commitTransaction(DataSource dataSource) {
Connection connection = DataSourceUtils.getConnection(dataSource);
try {
connection.commit();
} catch (Exception e) {
rollbackTransaction(dataSource);
}
// DataSourceUtils.resetConnectionAfterTransaction(connection, TransactionDefinition.ISOLATION_DEFAULT);
// SimpleTransactionStatus transactionStatus = new SimpleTransactionStatus(false);
// try {
// dataSourceTransactionManager.commit(transactionStatus);
// jta.commit(transactionStatus);
// } catch (TransactionException e) {
// dataSourceTransactionManager.rollback(transactionStatus);
// throw new RuntimeException(e);
// }
}
private void rollbackTransaction(DataSource dataSource) {
Connection connection = DataSourceUtils.getConnection(dataSource);
try {
connection.rollback();
} catch (SQLException e) {
throw new RuntimeException(e);
}
DataSourceUtils.resetConnectionAfterTransaction(connection, TransactionDefinition.ISOLATION_DEFAULT);
}
I left commented blocks of some testing to show you what approaches I tried. I don't know very well how Spring transaction works, so I'm just trying different things and trying to learn how all this stuff works... I will provide you more information if you want ;)
Thank you in advance.
UPDATE
As M. Denium suggested, that's what I have for now:
I declared the variable, using the TransactionStatus as ThreadSafe and finally solved as:
private DataSourceTransactionManager dataSourceTransactionManager = null;
private DefaultTransactionDefinition transactionDefinition = null;
private static final ThreadLocal<TransactionStatus> transactionStatus = new ThreadLocal<TransactionStatus>() {
#Override
protected TransactionStatus initialValue() {
return null;
}
};
And then using the same call from Groovy script, using the helper methods:
public void initTransaction(DataSource dataSource) {
dataSourceTransactionManager = new DataSourceTransactionManager();
transactionDefinition = new DefaultTransactionDefinition();
transactionDefinition.setReadOnly(false);
transactionDefinition.setIsolationLevel(TransactionDefinition.ISOLATION_SERIALIZABLE);
dataSourceTransactionManager.setRollbackOnCommitFailure(true);
dataSourceTransactionManager.setTransactionSynchronization(TransactionSynchronization.STATUS_UNKNOWN);
dataSourceTransactionManager.setDataSource(dataSource);
transactionStatus.set(dataSourceTransactionManager.getTransaction(null));
}
public void commitTransaction() {
try {
LOG.info("Finishing transaction...");
dataSourceTransactionManager.commit(transactionStatus.get());
dataSourceTransactionManager.getDataSource().getConnection().close();
LOG.info("Done.");
} catch (Throwable e) {
dataSourceTransactionManager.rollback(transactionStatus.get());
throw new RuntimeException("An exception during transaction. Rolling back.");
}
}
You are trying to reimplement the things that are already implemented by the transaction abstraction of Spring. Simply use the proper PlatformTransactionManager (you can probably grab that from an ApplicationContext) keep a reference to the TransactionStatus instead of a DataSource and use that to commit/rollback.
public TransactionStatus initTransaction() {
return transactionManager.getTransaction(null);
}
public void commit(TransactionStatus status) {
transactionManager.commit(status);
}
Instead of passing the TransactionStatus around you could also store it in a ThreadLocal and retrieve it in the commit method. This would ease the pain.
Another tip you shouldn't be creating JdbcTemplates and NamedParameterJdbcTemplates those are heavy objects to create. Upon construction they consult a connection to determine which database and version this is needed for the exception conversion. So performance wise this isn't a smart thing to do. Create a single instance and reuse, the templates are thread safe so you would only be needing a single instance.
However I would strongly argue that you should actually be using Groovy and not to try to work around it. Groovy has the Sql class that can help you. You already have access to the DataSource so doing something like this would be all that is needed.
def sql = new Sql(dataSource);
sql.withTransaction {
sql.execute "INSERT INTO city (name, state, founded_year) VALUES ('Minneapolis', 'Minnesota', 1867)"
sql.execute "INSERT INTO city (name, state, founded_year) VALUES ('Orlando', 'Florida', 1875)"
sql.execute "INSERT INTO city (name, state, founded_year) VALUES ('Gulfport', 'Mississippi', 1887)"
}
This is plain Groovy, no need to develop additional classes or to write extensive documentation to get it working. Just Groovy...
I'm newbie with spring framework. I used spring.xml to define the datasouce and DataSourceTransactionManager, so that I could insert data with jdbctemplate object.
And now I want to add the rollback to the transaction.
Unfortunately this rollback only works for JdbcTemplate.updata (String SQL), not for JdbcTemplate.update(PreparedStatementCreator, Keyholder), which I used to get the generated ID by insert.
#Override
#Transactional("txManagerTest")
public SQLQueryObjectIF process(SQLQueryObjectIF queryObj) {
KeyHolder keyHolder = new GeneratedKeyHolder();
for (final String query : queryObj.getQueries()) {
System.out.println(query);
// Rollback works fine for the "update" below.
//jdbcTemplate.update(query);
// Rollback doesn't work for the "update" below. Don't why...
jdbcTemplate.update(new PreparedStatementCreator() {
#Override
public PreparedStatement createPreparedStatement(Connection con) throws SQLException {
jdbcTemplate.getDataSource().getConnection().setAutoCommit(false);
PreparedStatement ps = jdbcTemplate.getDataSource().getConnection().prepareStatement(query,Statement.RETURN_GENERATED_KEYS);
return ps;
}
}, keyHolder);
//log.info(keyHolder.getKeys().toString());
}
//just for rollback test
if (keyHolder.toString().length()>-1){
throw new RuntimeException("Test Error");
}
return queryObj;
}
That code should be used like this (you need to use the connection given as parameter), otherwise with your code you will get a connection that Spring doesn't know about, by directly accessing the DataSource instance (if Spring doesn't know about it, it will not know to rollback in case of exception):
public PreparedStatement createPreparedStatement(Connection con) throws SQLException {
PreparedStatement ps = con.prepareStatement(query,Statement.RETURN_GENERATED_KEYS);
return ps;
}
I am using Guava to cache hot data. When the data does not exist in the cache, I have to get it from database:
public final static LoadingCache<ObjectId, User> UID2UCache = CacheBuilder.newBuilder()
//.maximumSize(2000)
.weakKeys()
.weakValues()
.expireAfterAccess(10, TimeUnit.MINUTES)
.build(
new CacheLoader<ObjectId, User>() {
#Override
public User load(ObjectId k) throws Exception {
User u = DataLoader.datastore.find(User.class).field("_id").equal(k).get();
return u;
}
});
My problem is when the data does not exists in database, I want it to return null and to not do any caching. But Guava saves null with the key in the cache and throws an exception when I get it:
com.google.common.cache.CacheLoader$InvalidCacheLoadException:
CacheLoader returned null for key shisoft.
How do we avoid caching null values?
Just throw some Exception if user is not found and catch it in client code while using get(key) method.
new CacheLoader<ObjectId, User>() {
#Override
public User load(ObjectId k) throws Exception {
User u = DataLoader.datastore.find(User.class).field("_id").equal(k).get();
if (u != null) {
return u;
} else {
throw new UserNotFoundException();
}
}
}
From CacheLoader.load(K) Javadoc:
Returns:
the value associated with key; must not be null
Throws:
Exception - if unable to load the result
Answering your doubts about caching null values:
Returns the value associated with key in this cache, first loading
that value if necessary. No observable state associated with this
cache is modified until loading completes.
(from LoadingCache.get(K) Javadoc)
If you throw an exception, load is not considered as complete, so no new value is cached.
EDIT:
Note that in Caffeine, which is sort of Guava cache 2.0 and "provides an in-memory cache using a Google Guava inspired API" you can return null from load method:
Returns:
the value associated with key or null if not found
If you may consider migrating, your data loader could freely return when user is not found.
Simple solution: use com.google.common.base.Optional<User> instead of User as value.
public final static LoadingCache<ObjectId, Optional<User>> UID2UCache = CacheBuilder.newBuilder()
...
.build(
new CacheLoader<ObjectId, Optional<User>>() {
#Override
public Optional<User> load(ObjectId k) throws Exception {
return Optional.fromNullable(DataLoader.datastore.find(User.class).field("_id").equal(k).get());
}
});
EDIT: I think #Xaerxess' answer is better.
Faced the same issue, cause missing values in the source was part of the normal workflow. Haven't found anything better than to write some code myself using getIfPresent, get and put methods. See the method below, where local is Cache<Object, Object>:
private <K, V> V getFromLocalCache(K key, Supplier<V> fallback) {
#SuppressWarnings("unchecked")
V s = (V) local.getIfPresent(key);
if (s != null) {
return s;
} else {
V value = fallback.get();
if (value != null) {
local.put(key, value);
}
return value;
}
}
When you want to cache some NULL values, you could use other staff which namely behave as NULL.
And before give the solution, I would suggest you not to expose LoadingCache to outside. Instead, you should use method to restrict the scope of Cache.
For example, you could use LoadingCache<ObjectId, List<User>> as return type. And then, you could return empty list when you could'n retrieve values from database. You could use -1 as Integer or Long NULL value, you could use "" as String NULL value, and so on. After this, you should provide a method to handler the NULL value.
when(value equals NULL(-1|"")){
return null;
}
I use the getIfPresent
#Test
public void cache() throws Exception {
System.out.println("3-------" + totalCache.get("k2"));
System.out.println("4-------" + totalCache.getIfPresent("k3"));
}
private LoadingCache<String, Date> totalCache = CacheBuilder
.newBuilder()
.maximumSize(500)
.refreshAfterWrite(6, TimeUnit.HOURS)
.build(new CacheLoader<String, Date>() {
#Override
#ParametersAreNonnullByDefault
public Date load(String key) {
Map<String, Date> map = ImmutableMap.of("k1", new Date(), "k2", new Date());
return map.get(key);
}
});