I encountered on following problem in spring boot application with rest api.
The problem is that when an user calls rest api to download csv file then the processing of file is executed in async process - downloadFile.
I would like to inform users about state of async process via next rest api end point - getState.
But here is the root of issue. There on downloadFile is used repository to saveAndFlush the information about progress into database table - TableStatus. However when calling getState then it doesnt see such row in database table TableStatus.
- Please what can be wrong and how we can manage it? Thanks
I know that it is probably caused by transaction that was not yet commited. I found out that transaction for downloadFile is commited after end of async process.
Some short code summary for better image:
Endpoint 1)
#Async
void downloadFile() {
//processing file
while (true) {
putInfoAboutState(50);
}
}
#Transactional
void putInfoAboutState(e) {
repo.saveAndFlush(e);
}
Endpoint 2)
Integer getState(i) {
return repo.getByOne(i);
}
Related
Imagine there is a method in appservice which upgrade data and finally a report is made in Pdf form through Ssrs. All methods are unit of work, so data is not committed in server until calling method uow.CompleteAsync(). therefore GenerateSampleReport() can not be called. To solve the problem we have used [unit of work (Isdisabled = true) so the method is written so:
[UnitOfWork(IsDisabled =true)]
public async Task Method()
{
using (var uow = UnitOfWorkManager.Begin())
{
//some data manipulation
await uow.CompleteAsync();
}
var pdfFile GenerateSampleReport();
}
But, while providing the report, if there is any exception, report can not be generated and data manipulation must be roll backed(problem about data integrity). How to solve the problem?
you shouldn't begin your own Unit Of Work. And don't disable UnitOfWork. Normally if Method() is an application service method then this should work as you want.
public async Task Method()
{
//some data manipulation...
var pdfFile GenerateSampleReport();//if any exception occurs in this line, the db transaction will be rolled back.
}
have you checked this before?
I need to implement multithreading background job for import file.
I have implemented it with background job(Hangfire). But if i use one thread it goes very slow.
The function look like this.
I using non-transaction unit to save changes to db immediately.
var contactFound = await _contactRepository.FirstOrDefaultAsync(x => x.Email.ToLower() == contact.Email.ToLower());
if (contactFound != null)
{
await _bjInfoManager.AddLog(args.JobId, "Found duplicated email: " + contact.Email);
}
else
{
contact.ContactListId = args.ContactListId;
contact.Email = contact.Email.ToLower();
await _contactRepository.InsertAsync(contact);
//Save changes in db
await CurrentUnitOfWork.SaveChangesAsync();
}
The problem occur when I tries to use this with Producer-Consumer Dataflow Pattern. I throws the exception "A second operation started on this context before a previous asynchronous operation completed."
The question is how to create isolated DbContext inside this method.
Please help me.
Transactions should not be multi-threaded. If you create a new task/thread in a UOW, you can create a seperated UOW using IUnitOfWork.Begin(TransactionScopeOption.RequiresNew) in a using block.
See the links
https://github.com/aspnetboilerplate/aspnetboilerplate/issues/619
Does Entity Framework support Multi-Threading?
Entity Framework and Multi threading
If you are using Microsoft SQL Server, then I recommend you to use bulk insert. It's super fast than entity framework.
https://learn.microsoft.com/en-us/sql/t-sql/statements/bulk-insert-transact-sql
I have a code where event is published in a method annotated with Spring #Transactional annotation.
#Override
#Transactional
public Task updateStatus(Integer taskId, ExecutionStatus newStatus) {
Task task = Task.builder().executionStatus(newStatus).build();
return updateStatusInternal(taskId, rteWithMetadata);
}
private TaskExecution updateStatusInternal(Integer taskId,
Task newStatus) {
Task task = taskService.findById(taskId);
TaskExecution te = task.getFirstExecution();
TaskExecution.ExecutionStatus oldStatus = te.getExecutionStatus();
TaskExecution.ExecutionStatus newStatus = newStatus.getExecutionStatus();
log.info(
"Task Execution status changed. Task id={}, from={}, to={}. Manual override : {}",
task.getId(), oldStatus, newStatus,
newStatus.isManualOverrideInitiated());
te.setExecutionStatus(newStatus);
if (te.getExecutionStatus() == ExecutionStatus.COMPLETED
|| te.getExecutionStatus() == ExecutionStatus.FAILED) {
te.setEndDate(DateTimeHelper.getUtcNow());
if (rte.isManualOverrideInitiated()) {
rte.setManualOverrideEndDate(DateTimeHelper.getUtcNow());
}
}
publisher.publishEvent(TaskStatusChanged.of(task, oldStatus, newStatus));
log.info("Published TaskStatusChanged event. task Id={}", task.getId());
// Send STOMP message
final Object payload = StompMessageHelper.getTaskExecutionUpdateMessage(task);
messageTemplate.convertAndSend(taskDestination(task), payload);
log.info("STOMP message for task status update sent. task Id={}",
task.getId());
return te;
}
There is a corresponding listener method for the application event which is annotated with #TransactionalEventListener.
#Async("changeEventExecutor")
#TransactionalEventListener(phase=TransactionPhase.AFTER_COMMIT)
public void taskStatusChanged(final TaskStatusChanged e) {
log.info("taskStatusChanged called");
}
Problem is listener is not fired on one of our production boxes. It works fine consistently on local dev environment but fails consistently in production.
Did somebody face this issue earlier? Only solution I can think of is to manually fire the application event.
Note: I have checked the existing similar posting. My scenario does not match with any of the existing posting.
The only thing I can think of comes from Spring's javadoc:
If the event is not published within the boundaries of a managed
transaction, the event is discarded unless the fallbackExecution()
flag is explicitly set. If a transaction is running, the event is
processed according to its TransactionPhase.
Could there be no transaction running? I assume your code sample isn't complete, so perhaps the transaction is being rolled-back when the event is fired or something along those lines.
In any case, you could try with the following (I know you are referring to a production box, so I'm not sure what are your options in trying things out):
#TransactionalEventListener(fallbackExecution=true, phase=TransactionPhase.AFTER_COMMIT)
I have a problem with async controllers in Grails. Consider the following controller:
#Transactional(readOnly=true)
class RentController {
def myService
UserProperties props
def beforeInterceptor = {
this.props = fetchUserProps()
}
//..other actions
#Transactional
def rent(Long id) {
//check some preconditions here, calling various service methods...
if (!allOk) {
render status: 403, text: 'appropriate.message.key'
return
}
//now we long poll because most of the time the result will be
//success within a couple of seconds
AsyncContext ctx = startAsync()
ctx.timeout = 5 * 1000 * 60 + 5000
ctx.start {
try {
//wait for external service to confirm - can take a long time or even time out
//save appropriate domain objects if successful
//placeRental is also marked with #Transactional (if that makes any difference)
def result = myService.placeRental()
if (result.success) {
render text:"OK", status: 200
} else {
render status:400, text: "rejection.reason.${result.rejectionCode}"
}
} catch (Throwable t) {
log.error "Rental process failed", t
render text: "Rental process failed with exception ${t?.message}", status: 500
} finally {
ctx.complete()
}
}
}
}
The controller and service code appear to work fine (though the above code is simplified) but will sometimes cause a database session to get 'stuck in the past'.
Let's say I have a UserProperties instance whose property accountId is updated from 1 to 20 somewhere else in the application while a rent action is waiting in the async block. As the async block eventually terminates one way or another (it may succeed, fail or time out), the app will sometimes get a stale UserProperties instance with accountId: 1. Let's say I refresh the updated user's properties page, I will see accountId: 1 about 1 time per 10 refreshes while the rest of the time it will be 20 - and this is on my development machine where noone else is accessing the application (though the same behaviour can be observed in production). My connection pool also holds 10 connections so I suspect there may be a correlation here.
Other strange things will happen - for example, I will get StaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect) from actions doing something as simple as render (UserProperties.list() as JSON) - after the response had already rendered (successfuly apart from the noise in the logs) and despite the action being annotated with #Transactional(readOnly=true).
A stale session doesn't seem to appear every time and so far our solution was to restart the server every evening (the app has few users for now), but the error is annoying and the cause was hard to pinpoint. My guess is that a DB transaction doesn't get committed or rolled back because of the async code, but GORM, Spring and Hibernate have many nooks and crannies where things could get stuck.
We're using Postgres 9.4.1 (9.2 on a dev machine, same problem), Grails 2.5.0, Hibernate plugin 4.3.8.1, Tomcat 8, Cache plugin 1.1.8, Hibernate Filter plugin 0.3.2 and the Audit Logging plugin 1.0.1 (other stuff too, obviously, but this feels like it could be relevant). My datasource config contains:
hibernate {
cache.use_second_level_cache = true
cache.use_query_cache = false
cache.region.factory_class = 'org.hibernate.cache.ehcache.SingletonEhCacheRegionFactory'
singleSession = true
flush.mode = 'manual'
format_sql = true
}
Grails bug. And a nasty one, everything seems OK until your app starts acting funny in completely unrelated parts of the app.
I have a session scoped class that contains an object to manage user statistics. When a user logs in(through SSO) an application scoped method checks the table for active sessions - if any are found the session is invalidated using the session id in the table.
A row is added to a userStats table in the session scoped class:
/**
* get all the info needed for collecting user stats, add userStats to the users session and save to userStats table
* this happens after session is created
* #param request
*/
private void createUserStats(HttpServletRequest request){
if (!this.sessionExists) {
this.userStats = new UserStats(this.user, request.getSession(true)
.getId(), System.getProperty("atcots_host_name"));
request.getSession().setAttribute("userstats", this.userStats);
Events.instance().raiseEvent("userBoundToSession", this.userStats);
this.sessionExists = true;
log.info("user " + this.user + " is now logged on");
// add this to the db
try {
this.saveUserStatsToDb();
} catch (Exception e) {
log.error("could not save " + this.userStats.getId().getPeoplesoftId() + " information to db");
e.printStackTrace();
}
}
}
When the user's session is destroyed this row is updated with a log off time.
For reasons I can't explain nor duplicate 2 users in the last 2 weeks have logged in and locked the row. When that happens any database calls by that user are no longer possible and the application is effectively unusable for this user.
[org.hibernate.util.JDBCExceptionReporter] (http-127.0.0.1-8180-3) SQL Error: 0, SQLState: null
2012-07-26 18:45:53,427 ERROR [org.hibernate.util.JDBCExceptionReporter] (http-127.0.0.1-8180-3) Transaction is not active: tx=TransactionImple < ac, BasicAction: -75805e7d:3300:5011c807:6a status: ActionStatus.ABORT_ONLY >; - nested throwable: (javax.resource.ResourceException: Transaction is not active: tx=TransactionImple < ac, BasicAction: -75805e7d:3300:5011c807:6a status: ActionStatus.ABORT_ONLY >)
The gathering of these stats is important, but not life and death, if I can't get the information I'd like to give up and keep it moving. But that's not happening. What is happening is that the entityManager is marking the transaction for rollback and any db call after that returns the above error. I originally saved the users stats at the application scope - so when the row locked it locked the entityManager for the ENTIRE APPLICATION (this did not go over well). When I moved the method to session scope it only locks out the offending user.
I tried setting the entityManger to a lesser scope(I tried EVENT and METHOD):
((EntityManager) Component.getInstance("entityManager", ScopeType.EVENT)).persist(this.userStats);
((EntityManager) Component.getInstance("entityManager", ScopeType.EVENT)).flush();
This doesn't make db calls at all.
I've tried manually rolling back the transaction, but no joy.
When I lock a row in a table that has data that is used at the conversation scope level the results are not nearly as catastrophic - no data is saved but it recovers.
ETA:
I tried raising an AsynchronousEvent - that works locally, but deployed to our remote test server - and this is odd - I get:
DEBUG [org.quartz.core.JobRunShell] (qtz_Worker-1) Calling execute on job DEFAULT.2d0badb3:139030aec6e:-7f34
INFO [com.mypkg.myapp.criteria.SessionCriteria] (qtz_Worker-1) observing predestroy for seam
DEBUG [com.mypkg.myapp.criteria.SessionCriteria] (qtz_Worker-1) destroy destroy destroy sessionCriteria
ERROR [org.jboss.seam.async.AsynchronousExceptionHandler] (qtz_Worker-1) Exeception thrown whilst executing asynchronous call
java.lang.IllegalArgumentException: attempt to create create event with null entity
at org.hibernate.event.PersistEvent.<init>(PersistEvent.java:45)
at org.hibernate.event.PersistEvent.<init>(PersistEvent.java:38)
at org.hibernate.impl.SessionImpl.persist(SessionImpl.java:619)
at org.hibernate.impl.SessionImpl.persist(SessionImpl.java:623)
...
The odd bit is that it appears to be going through the Quartz handler.
ETA again:
So, not so odd, I had set Quartz as the async handler - I thought it was only for scheduling jobs. Also asynchronous methods don't have access to the session context, so I had to add a parameter to my observing method to actually have an object to persist:
#Observer("saveUserStatsEvent")
#Transactional
public void saveUserStatsToDb(UserStats userstats) throws Exception {
if(userstats != null){
log.debug("persisting userstats to db");
this.getEntityManager().persist(userstats);
this.getEntityManager().flush();
}
}
How do I recover from this?
First of all, specifying a scope in Component.getInstance() does not have the result of creating the component in the scope specified. EntityManager instances always live in the conversation context (be it temporary or long-running). The scope parameter of getInstance() serves the sole purpose of hinting the context in which the component should be, in order to avoid an expensive search in all contexts (which is what happens if you don't specify a context or specify the wrong context).
The transaction is being marked for rollback because of the previous error. If the entityManager were to commit regardless, it would not be transactional (a transaction in fact guarantees that if an error happens nothing is persisted). If you want to isolate the login transaction from stats gathering, the simplest solution is to perform the saveUserStatsToDb method inside an asynchronous event (transactions are bound to the thread, so using a different thread guarantees that the event is handled in a separate transaction).
Something like this:
#Observer("saveUserStatsEvent")
#Transactional
public void saveUserStatsToDb(UserStats stats) {
((EntityManager)Component.getInstance("entityManager")).persist(stats);
}
And in your createUserStats method:
Events.instance().raiseAsynchronousEvent("saveUserStatsEvent", this.userStats);
However, this just circumvents the problem by dividing the transactions in two. What you really want to solve is the locking condition at the base of the problem.