I need to update thousands of records in the database but i would like to commit after a batch of 5000 records.
#Service
#Transactional (rollbackFor=Throwable.class)
public class AttributeProcessorServiceImpl extends DataLoader implements
AttributeProcessorService
{
.....
private final TransactionTemplate transTemplate;
private final JdbcTemplate jdbcTemplate;
#Autowired private PlatformTransactionManager platformTransactionManager;
#Autowired
public BlockAttributeProcessorServiceImpl(
TransactionTemplate transTemplate,
JdbcTemplate jdbcTemplate,
.....)
{
super();
this.transTemplate = transTemplate;
this.jdbcTemplate=jdbcTemplate;
.....
}
#Async
#Transactional (propagation=Propagation.NOT_SUPPORTED)
public void reloadAttrs()
{
loadAttrs();
updateAttrs();
}
private void loadAttrs()
{
...some data fetching and processing, finally call db update.
updateDbInBatches(rowcount, sql);
}
private void updateAttrs()
{
...some data fetching and processing, finally call db update.
updateDbInBatches(rowcount, sql);
}
private void updateDbInBatches(long rowcount, String sql)
{
DefaultTransactionDefinition def;
boolean hasMore=true;
Integer from;
Integer to = 0;
int batchSize=5000; //gets from property
while (hasMore)
{
from = to+1;
to = batchSize;
def = new DefaultTransactionDefinition();
def.setName("backCommitTx");
def.setPropagationBehavior(TransactionDefinition.PROPAGATION_REQUIRED);
TransactionStatus status = platformTransactionManager.getTransaction(def);
int rows = jdbcTemplate.update(sql,paramValues,paramTypes);
logger.debug("Loaded ["+rows+"] records.");
platformTransactionManager.commit(status);
if (to > rowcount)
{
hasMore=false;
logger.debug("All records ["+rowcount+"] updated.");
}
}
}
}
If I put a breakpoint after loadAttrs(), it shows it loaded bunch of records to the database and issued a commit(), but database does not reflect that commit, until after entire public method completes. How do i ensure data is indeed written to the database after each commit. commit neither gives any error as well.
I missed an important piece of information that solved the problem.
I had another public method which is what was called from outside.
public void reloadAttrs(TransDetail trans)
{
reloadAttrs();
}
Above method was infact using default Transaction Propagation as i did not mention it specifically. Since this was the first public method that was called, spring was ignoring transaction demarcation on next public (async) method that was called. I changed above signature to:
#Transactional (propagation=Propagation.NOT_SUPPORTED)
public void reloadAttrs(TransDetail trans)
{
reloadAttrs();
}
It then worked. I was able to see changes in the database after every commit.
I am exporting a report in my code, I am using HibernateDAOSupport and the method is not annotated with #Transactional. So when the request comes from the UI then automatically Transaction is created and the report is exported in 2 mins.
But when I try to use thread, so I had to put annotation #Transactional, otherwise I get an error of LazyInitialization as no Transcaction is present.
So when I put #Transactional then the same report takes time 2.4 mins. This time keeps increasing and sometimes it take double of the time without #Transactional
I am not sure why it takes time when I put annotation #Transactional
The main class:
CommonDAO
public class CommonDAO extends HibernateDaoSupport
private HibernateTransactionManager txnManager;
public void setTxnManager(HibernateTransactionManager txnManager) {
this.txnManager = txnManager;
}
public List executeSQLQueryPaging(final String hql,
final Object[] params, final Integer[] pagingParam) throws ServiceException {
List results = null;
results = getHibernateTemplate().executeFind(new HibernateCallback() {
public Object doInHibernate(Session session) {
SQLQuery query = session.createSQLQuery(hql);
if (params != null) {
for (int i = 0; i < params.length; i++) {
query.setParameter(i, params[i]);
}
}
query.setFirstResult(pagingParam[0]);
query.setMaxResults(pagingParam[1]);
return query.list();
}
});
return results;
}
ModuleDAO extends CommonDAO
public List getReportData{
executeSQLQueryPaging();
...
return list}
Service
public List getReportData(){
.....
return ModuleDAO.getReportData();
}
If I put #Transactional at service layer then the performance detriorates, or else it is faster if executed from web.
I am trying to perform update with NamedQuery but updated values are not getting persisted in the DB though the update statement returning the updated count. This is happening only in tests but in the actual flow update is happening.
#Test
#Transactional(propagation = Propagation.REQUIRED, isolation = Isolation.READ_UNCOMMITTED)
public void test() throws Exception
messageHandler.process(message);
} catch (Exception e) {
throw new Exception();
}
assertEquals(new Integer(5), ServiceImpl.findById(100L).get().getStatus());
}
Class MessageHandler{
#Transactional
public void process(String message){
serviceImpl.update(5,100, some date, user);
}
}
class ServiceImpl {
#PersistenceContext
EntityManager entityManager;
#Modifying(flushAutomatically = true, clearAutomatically = true)
public void updateOrderStatus(Integer newOrderStatus, Long OrderId, String updateTs,
String updateUserId) {
Query query = entityManager.createNamedQuery(Order.UPDATE_ORDER_STATUS);
entityManager.flush();
query.setParameter(1, newOrderStatus);
query.setParameter(2, OrderId);
query.setParameter(3, updateTs);
query.setParameter(4, updateUserId);
int i = query.executeUpdate();
System.out.println("***************************************");
System.out.println(i);
}
}
Can anyone help me what I am doing wrong in testcases?
Thanks in advance!!!!!!
By default test transactions are rolled back. You'll need to explicitly use #Commit if you want your tests to commit the changes. If you haven't already, take a look at the spring docs.
Fighting with TestNG, Spring an Hibernate. I'm writing test for Service class, and it's always failure. But without test class works fine. So App is working, but tests don't want to.
Here is my test class
#Transactional
public class BorrowerServiceTest {
#Mock
BorrowerDAOImpl borrowerDAO;
#InjectMocks
BorrowerService borrowerService;
#BeforeClass
public void setUp() {
MockitoAnnotations.initMocks(this);
}
#Test
public void persistTest() {
Borrower borrower = new Borrower.BorrowerBuilder().firstName("Lars").lastName("Urlich").adress("LA")
.phoneNumber("900900990").build();
borrowerService.persist(borrower);
List<Borrower> borrowerList = borrowerService.getBorrowerByName("Lars Urlich");
Assert.assertEquals(true, borrower.equals(borrowerList.get(0)));
}
}
My BorrowerService:
#Service("borrowerService")
#Transactional
public class BorrowerService {
#Autowired
private BorrowerDAO borrowerDAO;
public List<Borrower> getBorrowers() {
return borrowerDAO.getBorrowers();
}
public List<Borrower> getBorrowerByName(String name) {
return borrowerDAO.getBorrowerByName(name);
}
public boolean removeBorrower(Borrower borrower) {
return borrowerDAO.removeBorrower(borrower);
}
public boolean persist(Borrower borrower) {
return borrowerDAO.persist(borrower);
}
}
My BorrowerDAOImpl:
#Repository("borrowerDAO")
#Transactional
public class BorrowerDAOImpl extends DAO implements BorrowerDAO {
#Override
public List<Borrower> getBorrowers() {
List<Borrower> borrowerList = null;
Query query = entityManager.createQuery("SELECT B FROM Borrower B");
borrowerList = query.getResultList();
return borrowerList;
}
#Override
public List<Borrower> getBorrowerByName(String name) {
List<Borrower> borrowerList = null;
String[] values = name.split(" ");
Query query = entityManager.createQuery("SELECT B FROM Borrower B WHERE B.firstName LIKE '" + values[0]
+ "' AND B.lastName LIKE '" + values[1] + "'");
borrowerList = query.getResultList();
return borrowerList;
}
#Override
public boolean removeBorrower(Borrower borrower) {
String firstName = borrower.getFirstName();
String lastName = borrower.getLastName();
Query query = entityManager
.createQuery("DELETE Borrower where FIRST_NAME LIKE :FirstName AND LAST_NAME LIKE :LastName");
query.setParameter("FirstName", firstName);
query.setParameter("LastName", lastName);
query.executeUpdate();
return true;
}
#Override
public boolean persist(Borrower borrower) {
entityManager.persist(borrower);
return true;
}
}
and abstract DAO:
#Repository
#Transactional
public abstract class DAO {
#PersistenceContext
protected EntityManager entityManager;
}
Maven returns failure:
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.LinkedList.checkElementIndex(LinkedList.java:555)
at java.util.LinkedList.get(LinkedList.java:476)
at com.me.service.test.BorrowerServiceTest.persistTest(BorrowerServiceTest.java:41)
I also had to fight with this. The problem here is that your test runs in it's own transaction, so nothing will be committed during method's execution. Now here is what I did:
public class IntegrationTest extends SomeTestBase
{
#Autowired
private PlatformTransactionManager platformTransactionManager;
private TransactionTemplate transactionTemplate;
#Autowired
private BeanToTest beanToTest;
#Override
#Before
public void setup()
{
super.setup();
this.transactionTemplate = new TransactionTemplate(this.platformTransactionManager);
}
#Test
public void fooTest()
{
// given
// when
boolean result = this.transactionTemplate.execute(new TransactionCallback<Boolean>()
{
#Override
public Boolean doInTransaction(TransactionStatus status)
{
return IntegrationTest.this.beanToTest.foo();
}
});
// then
}
}
This allows you to have methods execute within a separate transaction. Please note that you might declare some variables as final.
Hope that helps.
Check the Spring documentation: it looks your test class should extend AbstractTestNGSpringContextTests.
Use #Commit annotation on the whole test class or even method to persist changes made in the test. For more information https://docs.spring.io/spring/docs/current/spring-framework-reference/testing.html#commit
I'm trying to use something similar to org.springframework.cache.annotation.Cacheable :
Custom annotation:
#Target(ElementType.METHOD)
#Retention(RetentionPolicy.RUNTIME)
#Documented
public #interface CheckEntity {
String message() default "Check entity msg";
String key() default "";
}
Aspect:
#Component
#Aspect
public class CheckEntityAspect {
#Before("execution(* *.*(..)) && #annotation(checkEntity)")
public void checkEntity(JoinPoint joinPoint, CheckEntitty checkEntity) {
System.out.println("running entity check: " + joinPoint.getSignature().getName());
}
}
Service:
#Service
#Transactional
public class EntityServiceImpl implements EntityService {
#CheckEntity(key = "#id")
public Entity getEntity(Long id) {
return new Entity(id);
}
}
My IDE (IntelliJ) doesn't see anything special with the key = "#id" usage in contrast to similar usages for Cacheable where it's shown with different color than plain text. I'm mentioning the IDE part just as a hint in case it helps, it looks like the IDE is aware in advance about these annotations or it just realizes some connection which doesn't exist in my example.
The value in the checkEntity.key is '#id' instead of an expected number.
I tried using ExpressionParser but possibly not in the right way.
The only way to get parameter value inside the checkEntity annotation is by accessing the arguments array which is not what I want because this annotation could be used also in methods with more than one argument.
Any idea?
Adding another simpler way of doing it using Spring Expression. Refer below:
Your Annotation:
#Target(ElementType.METHOD)
#Retention(RetentionPolicy.RUNTIME)
#Documented
public #interface CheckEntity {
String message() default "Check entity msg";
String keyPath() default "";
}
Your Service:
#Service
#Transactional
public class EntityServiceImpl implements EntityService {
#CheckEntity(keyPath = "[0]")
public Entity getEntity(Long id) {
return new Entity(id);
}
#CheckEntity(keyPath = "[1].otherId")
public Entity methodWithMoreThanOneArguments(String message, CustomClassForExample object) {
return new Entity(object.otherId);
}
}
class CustomClassForExample {
Long otherId;
}
Your Aspect:
#Component
#Aspect
public class CheckEntityAspect {
#Before("execution(* *.*(..)) && #annotation(checkEntity)")
public void checkEntity(JoinPoint joinPoint, CheckEntitty checkEntity) {
Object[] args = joinPoint.getArgs();
ExpressionParser elParser = new SpelExpressionParser();
Expression expression = elParser.parseExpression(checkEntity.keyPath());
Long id = (Long) expression.getValue(args);
// Do whatever you want to do with this id
// This works for both the service methods provided above and can be re-used for any number of similar methods
}
}
PS: I am adding this solution because I feel this is a simpler/clearner approach as compared to other answers and this might be helpful for someone.
Thanks to #StéphaneNicoll I managed to create a first version of a working solution:
The Aspect
#Component
#Aspect
public class CheckEntityAspect {
protected final Log logger = LogFactory.getLog(getClass());
private ExpressionEvaluator<Long> evaluator = new ExpressionEvaluator<>();
#Before("execution(* *.*(..)) && #annotation(checkEntity)")
public void checkEntity(JoinPoint joinPoint, CheckEntity checkEntity) {
Long result = getValue(joinPoint, checkEntity.key());
logger.info("result: " + result);
System.out.println("running entity check: " + joinPoint.getSignature().getName());
}
private Long getValue(JoinPoint joinPoint, String condition) {
return getValue(joinPoint.getTarget(), joinPoint.getArgs(),
joinPoint.getTarget().getClass(),
((MethodSignature) joinPoint.getSignature()).getMethod(), condition);
}
private Long getValue(Object object, Object[] args, Class clazz, Method method, String condition) {
if (args == null) {
return null;
}
EvaluationContext evaluationContext = evaluator.createEvaluationContext(object, clazz, method, args);
AnnotatedElementKey methodKey = new AnnotatedElementKey(method, clazz);
return evaluator.condition(condition, methodKey, evaluationContext, Long.class);
}
}
The Expression Evaluator
public class ExpressionEvaluator<T> extends CachedExpressionEvaluator {
// shared param discoverer since it caches data internally
private final ParameterNameDiscoverer paramNameDiscoverer = new DefaultParameterNameDiscoverer();
private final Map<ExpressionKey, Expression> conditionCache = new ConcurrentHashMap<>(64);
private final Map<AnnotatedElementKey, Method> targetMethodCache = new ConcurrentHashMap<>(64);
/**
* Create the suitable {#link EvaluationContext} for the specified event handling
* on the specified method.
*/
public EvaluationContext createEvaluationContext(Object object, Class<?> targetClass, Method method, Object[] args) {
Method targetMethod = getTargetMethod(targetClass, method);
ExpressionRootObject root = new ExpressionRootObject(object, args);
return new MethodBasedEvaluationContext(root, targetMethod, args, this.paramNameDiscoverer);
}
/**
* Specify if the condition defined by the specified expression matches.
*/
public T condition(String conditionExpression, AnnotatedElementKey elementKey, EvaluationContext evalContext, Class<T> clazz) {
return getExpression(this.conditionCache, elementKey, conditionExpression).getValue(evalContext, clazz);
}
private Method getTargetMethod(Class<?> targetClass, Method method) {
AnnotatedElementKey methodKey = new AnnotatedElementKey(method, targetClass);
Method targetMethod = this.targetMethodCache.get(methodKey);
if (targetMethod == null) {
targetMethod = AopUtils.getMostSpecificMethod(method, targetClass);
if (targetMethod == null) {
targetMethod = method;
}
this.targetMethodCache.put(methodKey, targetMethod);
}
return targetMethod;
}
}
The Root Object
public class ExpressionRootObject {
private final Object object;
private final Object[] args;
public ExpressionRootObject(Object object, Object[] args) {
this.object = object;
this.args = args;
}
public Object getObject() {
return object;
}
public Object[] getArgs() {
return args;
}
}
I think you probably misunderstand what the framework is supposed to do for you vs. what you have to do.
SpEL support has no way to be triggered automagically so that you can access the actual (resolved) value instead of the expression itself. Why? Because there is a context and as a developer you have to provide this context.
The support in Intellij is the same thing. Currently Jetbrains devs track the places where SpEL is used and mark them for SpEL support. We don't have any way to conduct the fact that the value is an actual SpEL expression (this is a raw java.lang.String on the annotation type after all).
As of 4.2, we have extracted some of the utilities that the cache abstraction uses internally. You may want to benefit from that stuff (typically CachedExpressionEvaluator and MethodBasedEvaluationContext).
The new #EventListener is using that stuff so you have more code you can look at as examples for the thing you're trying to do: EventExpressionEvaluator.
In summary, your custom interceptor needs to do something based on the #id value. This code snippet is an example of such processing and it does not depend on the cache abstraction at all.
Spring uses internally an ExpressionEvaluator to evaluate the Spring Expression Language in the key parameter (see CacheAspectSupport)
If you want to emulate the same behaviour, have a look at how CacheAspectSupport is doing it. Here is an snippet of the code:
private final ExpressionEvaluator evaluator = new ExpressionEvaluator();
/**
* Compute the key for the given caching operation.
* #return the generated key, or {#code null} if none can be generated
*/
protected Object generateKey(Object result) {
if (StringUtils.hasText(this.metadata.operation.getKey())) {
EvaluationContext evaluationContext = createEvaluationContext(result);
return evaluator.key(this.metadata.operation.getKey(), this.methodCacheKey, evaluationContext);
}
return this.metadata.keyGenerator.generate(this.target, this.metadata.method, this.args);
}
private EvaluationContext createEvaluationContext(Object result) {
return evaluator.createEvaluationContext(
this.caches, this.metadata.method, this.args, this.target, this.metadata.targetClass, result);
}
I don't know which IDE you are using, but it must deal with the #Cacheable annotation in a different way than with the others in order to highlight the params.
Your annotation can be used with methods with more than 1 parameter, but that doesn't mean you can't use the arguments array. Here's a sollution:
First we have to find the index of the "id" parameter. This you can do like so:
private Integer getParameterIdx(ProceedingJoinPoint joinPoint, String paramName) {
MethodSignature methodSignature = (MethodSignature) joinPoint.getSignature();
String[] parameterNames = methodSignature.getParameterNames();
for (int i = 0; i < parameterNames.length; i++) {
String parameterName = parameterNames[i];
if (paramName.equals(parameterName)) {
return i;
}
}
return -1;
}
where "paramName" = your "id" param
Next you can get the actual id value from the arguments like so:
Integer parameterIdx = getParameterIdx(joinPoint, "id");
Long id = joinPoint.getArgs()[parameterIdx];
Of course this assumes that you always name that parameter "id". One fix there could be to allow to specify the parameter name on the annotation, something like
#Target(ElementType.METHOD)
#Retention(RetentionPolicy.RUNTIME)
#Documented
public #interface CheckEntity {
String message() default "Check entity msg";
String key() default "";
String paramName() default "id";
}