I have the following method in an #Service class which has #Transactional defined:
#Override
public Result add(#NonNull final UserSaveRequest request) {
final Result<Email> emailResult = Email.create(request.getEmail());
final Result<UserFirstName> userFirstNameResult = UserFirstName.create(request.getFirstName());
final Result<UserLastName> userLastNameResult = UserLastName.create(request.getLastName());
final Result combinedResult = Result.combine(emailResult, userFirstNameResult, userLastNameResult);
if (combinedResult.isFailure()) {
return Result.fail(combinedResult.getErrorMessage());
}
final Result<User> userResult = User.create(emailResult.getValue(), userFirstNameResult.getValue(), userLastNameResult.getValue());
if (userResult.isFailure()) {
return Result.fail(userResult.getErrorMessage());
}
this.userRepository.save(userResult.getValue());
return Result.ok();
}
Now as you can see I utilize a Result class which can contain a return value or an error message as I don't think using exceptions for flow control is very clean.
The problem I now have is; the complete method is bound in one transaction and if one database call should fail the whole transaction will be rolled back. In my model however, after the this.userRepository.save(userResult.getValue()); call, if something would happen that would force me to return a failed result, I can't undo that save(userResult.getVlaue()); call seeing as I don't use exceptions for flow control.
Is this a problem that has an elegant solution, or is this a place where I need to make a trade-off between using exceptions as flow control and having to mentally keep track of the ordering of my statements in these kind of situations?
Yes, you can trigger rollback manually. Try this:
TransactionAspectSupport.currentTransactionStatus().setRollbackOnly();
More information: https://docs.spring.io/spring/docs/5.0.7.RELEASE/spring-framework-reference/data-access.html#transaction-declarative-rolling-back
Related
We are using java 8 parallel stream to process a task, and we are submitting the task through ForkJoinPool#submit. We are not using jvm wide ForkJoinPool.commonPool, instead we are creating our own custom pool to specify the parallelism and storing it as static variable.
We have validation framework, where we subject a list of tables to a List of Validators, and we submit this job through the custom ForkJoinPool as follows:
static ForkJoinPool forkJoinPool = new ForkJoinPool(4);
List<Table> tables = tableDAO.findAll();
ModelValidator<Table, ValidationResult> validator = ValidatorFactory
.getInstance().getTableValidator();
List<ValidationResult> result = forkJoinPool.submit(
() -> tables.stream()
.parallel()
.map(validator)
.filter(result -> result.getValidationMessages().size() > 0)
.collect(Collectors.toList())).get();
The problem we are having is, in the downstream components, the individual validators which run on separate threads from our static ForkJoinPool rely on tenant_id, which is different for every request and is stored in an InheritableThreadLocal variable. Since we are creating a static ForkJoinPool, the threads pooled by the ForkJoinPool will only inherit the value of the parent thread, when it is created first time. But these pooled threads will not know the new tenant_id for the current request. So for subsequent execution these pooled threads are using old tenant_id.
I tried creating a custom ForkJoinPool and specifying ForkJoinWorkerThreadFactory in the constructor and overriding the onStart method to feed the new tenant_id. But that doesnt work, since the onStart method is called only once at creation time and not during individual execution time.
Seems like we need something like the ThreadPoolExecutor#beforeExecute which is not available in case of ForkJoinPool. So what alternative do we have if we want to pass the current thread local value to the statically pooled threads?
One workaround would be to create the ForkJoinPool for each request, rather than make it static but we wouldn't want to do it, to avoid the expensive nature of thread creation.
What alternatives do we have?
I found the following solution that works without changing any underlying code. Basically, the map method takes a functional interface which I am representing as a lambda expression. This expression adds a preExecution hook to set the new tenantId in the current ThreadLocal and cleaning it up in postExecution.
forkJoinPool.submit(tables.stream()
.parallel()
.map((item) -> {
preExecution(tenantId);
try {
return validator.apply(item);
} finally {
postExecution();
}
}
)
.filter(validationResult ->
validationResult.getValidationMessages()
.size() > 0)
.collect(Collectors.toList())).get();
The best option in my view would be to get rid of the thread local and pass it as an argument instead. I understand that this could be a massive undertaking though. Another option would be to use a wrapper.
Assuming that your validator has a validate method you could do something like:
public class WrappingModelValidator implements ModelValidator<Table. ValidationResult> {
private final ModelValidator<Table. ValidationResult> v;
private final String tenantId;
public WrappingModelValidator(ModelValidator<Table. ValidationResult> v, String tenantId) {
this.v = v;
this.tenantId = tenantId;
}
public ValidationResult validate(Table t) {
String oldValue = YourThreadLocal.get();
YourThreadLocal.set(tenantId);
try {
return v.validate(t);
} finally {
YourThreadLocal.set(oldValue);
}
}
}
Then you simply wrap your old validator and it will set the thread local on entry and restore it when done.
I have an event handler class as follows:
#Component
#RepositoryEventHandler(DonationOffer.class)
public class DonationOfferEventHandler {
#Autowired
DonationOfferNotificationService notificationService;
#HandleAfterSave
public void handleAfterSave(DonationOffer donationOffer){
Integer donationOfferId = donationOffer.getDonationOfferId();
System.out.println("updating donation offer");
switch (donationOffer.getOfferStatus()){
case Accepted:
notificationService.generateAcceptedNotification(donationOfferId);
break;
case Rejected:
notificationService.generateRejectedNotification(donationOfferId);
break;
case Cancelled:
notificationService.generateCancelledNotification(donationOfferId);
break;
}
}
}
And spring repo as follows:
#Transactional
public interface DonationOfferRepo extends PagingAndSortingRepository<DonationOffer,Integer>{
#Modifying
#Query(
"update DonationOffer d " +
"set offerStatus = :offerStatus " +
"where d.donationOfferId = :donationOfferId"
)
#RestResource(path="updateStatus")
int updateStatus(
#Param("donationOfferId") Integer donationOfferId,
#Param("offerStatus") OfferStatus offerStatus
);
}
Event handler is called in case of PUT request by the problem is that #HandleAfterSave method is not being called when I call updateStatus() i.e.,
a get request to /api/donationOffers/search/updateStatus?donationOfferId=45&offerStatus=Cancelled
Can any one help if I am missing something or is it the default behavior that #HandleAfterSave would be called only with PUT request ?
UPDATE:
I have created a method to test event handler with repo save method as follows. Event handler is not called in this case also.
#GetMapping(value="/savetest")
#Transactional
public ResponseEntity saveTesT(){
DonationOffer donationOffer = donationOfferRepo.findOne(1);
donationOffer.setOfferedBottles(donationOffer.getOfferedBottles()+1);
donationOfferRepo.save(donationOffer);
return new ResponseEntity(new StringWrapper("OK"), HttpStatus.OK);
}
In order to trigger event from HandleBeforeSave you need to execute PUT request. If you are doing POST request than the correct event handler would be HandleBeforeCreate.
UPDATE:i would guess that the listener method does not get invoked for the same reason as to why Queries do not invoke JPA prePersist, postPersist, preUpdate ... and so on events. Queries have different execution route as to EntityManager.persis , update. Entity manager works on the foundation of identity , therefore it can guarantee that an entity has actually been updates. Same can not be told when you use a Query. Normally entity manager would execute one query to fetch the entity than it will be analysed for changes and then UPDATE will happen, or will not happen in case of no changes. There is no mechanism to tell if update has really happened when query is executed.
According to the documentation #Modifying annotation is not guaranteeing that modification will actually happen. What it is guaranteeing is that the session will be cleared after method invokation. The reasoning behind this is again that the queries have different execution route than the normal persist, merge methods and you can not guarantee that the session will keep integrity. Therefore #Modifying will guarantee it has been cleared.
In a microservice environment I see two main benefits from tracing requests through all microservice instances over an entire business process.
Finding latency gaps between or in service instances
Finding roots of failures, whether technical or regarding the business case
With Zipkin there is a tool, which addresses the first issue. But how can tracing be used to unveil failures in your microservice landscape? I definitely want to trace all error afflicted spans, but not each request, where nothing went wrong.
As mentioned here a custom Sampler could be used.
Alternatively, you may register your own Sampler bean definition and programmatically make the decision which requests should be sampled. You can make more intelligent choices about which things to trace, for example, by ignoring successful requests, perhaps checking whether some component is in an error state, or really anything else.
So I tried to implement that, but it doesn't work or I used it wrong.
So, as the blog post suggested I registered my own Sampler:
#Bean
Sampler customSampler() {
return new Sampler() {
#Override
public boolean isSampled(Span span) {
boolean isErrorSpan = false;
for(String tagKey : span.tags().keySet()){
if(tagKey.startsWith("error_")){
isErrorSpan = true;
}
}
return isErrorSpan ;
}
};
}
And in my controller I create a new Span, which is being tagged as an error if an exception raises
private final Tracer tracer;
#Autowired
public DemoController(Tracer tracer) {
this.tracer = tracer;
}
#RequestMapping(value = "/calc/{i}")
public String calc(#PathVariable String i){
Span span = null;
try {
span = this.tracer.createSpan("my_business_logic");
return "1 / " + i + " = " + new Float(1.0 / Integer.parseInt(i)).toString();
}catch(Exception ex){
log.error(ex.getMessage(), ex);
span.logEvent("ERROR: " + ex.getMessage());
this.tracer.addTag("error_" + ex.hashCode(), ex.getMessage());
throw ex;
}
finally{
this.tracer.close(span);
}
}
Now, this doesn't work. If I request /calc/a the method Sampler.isSampled(Span) is being called before the Controller method throws a NumberFormatException. This means, when isSampled() checks the Span, it has no tags yet. And the Sampler method is not being called again later in the process. Only if I open the Sampler and allow every span to be sampled, I see my tagged error-span later on in Zipkin. In this case Sampler.isSampled(Span) was called only 1 time but HttpZipkinSpanReporter.report(Span) was executed 3 times.
So what would the use case look like, to transmit only traces, which have error spans ? Is this even a correct way to tag a span with an arbitrary "error_" tag ?
The sampling decision is taken for a trace. That means that when the first request comes in and the span is created you have to take a decision. You don't have any tags / baggage at that point so you must not depend on the contents of tags to take this decision. That's a wrong approach.
You are taking a very custom approach. If you want to go that way (which is not recommended) you can create a custom implementation of a SpanReporter - https://github.com/spring-cloud/spring-cloud-sleuth/blob/master/spring-cloud-sleuth-core/src/main/java/org/springframework/cloud/sleuth/SpanReporter.java#L30 . SpanReporter is the one that is sending spans to zipkin. You can create an implementation that will wrap an existing SpanReporter implementation and will delegate the execution to it only when some values of tags match. But from my perspective it doesn't sound right.
I am in reference to Spring Roo In Action (book from Manning). Somewhere in the book it says "Roo marks the test class as #Transactional so that the unit tests automatically roll back any change.
Here is the illustrating method:
#Test
#Transactional
public void addAndFetchCourseViaRepo() {
Course c = new Course();
c.setCourseType(CourseTypeEnum.CONTINUING_EDUCATION);
c.setName("Stand-up Comedy");
c.setDescription(
"You'll laugh, you'll cry, it will become a part of you.");
c.setMaxiumumCapacity(10);
c.persist();
c.flush();
c.clear();
Assert.assertNotNull(c.getId());
Course c2 = Course.findCourse(c.getId());
Assert.assertNotNull(c2);
Assert.assertEquals(c.getName(), c2.getName());
Assert.assertEquals(c2.getDescription(), c.getDescription());
Assert.assertEquals(
c.getMaxiumumCapacity(), c2.getMaxiumumCapacity());
Assert.assertEquals(c.getCourseType(), c2.getCourseType());
}
However, I don't understand why changes in this method would be automatically rolled back if no RuntimeException occurs...
Quote from documentation:
By default, the framework will create and roll back a transaction for each test. You simply write code that can assume the existence of a transaction. [...] In addition, if test methods delete the contents of selected tables while running within a transaction, the transaction will roll back by default, and the database will return to its state prior to execution of the test. Transactional support is provided to your test class via a PlatformTransactionManager bean defined in the test's application context.
So, in other words, SpringJUnit4ClassRunner who runs your tests always do transaction rollback after test execution.
I'm trying to find a method that allows me to do a rollback when one of the elements of a list fails for a reason within the business rules established (ie: when throw my customize exception)
Example, (the idea is not recording anything if one element in list fails)
public class ControlSaveElement {
public void saveRecords(List<MyRecord> listRecords) {
Boolean status = true;
foreach(MyRecord element: listRecords) {
// Here is business rules
if(element.getStatus() == false) {
// something
status = false;
}
element.persist();
}
if(status == false) {
// I need to do roll back from all elements persisted before
}
}
...
}
Any idea? I'm working with Roo 1.2.2..
Boot Pros,
I recently started to program in spring-boot and I stumbled upon a question where I would like to get your opinion on.
What I try to achieve:
I created a Controller that exposes a GET endpoint, named nonBlockingEndpoint. This nonBlockingEndpoint executes a pretty long operation that is resource heavy and can run between 20 and 40 seconds.(in the attached code, it is mocked by a Thread.sleep())
Whenever the nonBlockingEndpoint is called, the spring application should register that call and immediatelly return an Operation ID to the caller.
The caller can then use this ID to query on another endpoint queryOpStatus the status of this operation. At the beginning it will be started, and once the controller is done serving the reuqest it will be to a code such as SERVICE_OK. The caller then knows that his request was successfully completed on the server.
The solution that I found:
I have the following controller (note that it is explicitely not tagged with #Async)
It uses an APIOperationsManager to register that a new operation was started
I use the CompletableFuture java construct to supply the long running code as a new asynch process by using CompletableFuture.supplyAsync(() -> {}
I immdiatelly return a response to the caller, telling that the operation is in progress
Once the Async Task has finished, i use cf.thenRun() to update the Operation status via the API Operations Manager
Here is the code:
#GetMapping(path="/nonBlockingEndpoint")
public #ResponseBody ResponseOperation nonBlocking() {
// Register a new operation
APIOperationsManager apiOpsManager = APIOperationsManager.getInstance();
final int operationID = apiOpsManager.registerNewOperation(Constants.OpStatus.PROCESSING);
ResponseOperation response = new ResponseOperation();
response.setMessage("Triggered non-blocking call, use the operation id to check status");
response.setOperationID(operationID);
response.setOpRes(Constants.OpStatus.PROCESSING);
CompletableFuture<Boolean> cf = CompletableFuture.supplyAsync(() -> {
try {
// Here we will
Thread.sleep(10000L);
} catch (InterruptedException e) {}
// whatever the return value was
return true;
});
cf.thenRun(() ->{
// We are done with the super long process, so update our Operations Manager
APIOperationsManager a = APIOperationsManager.getInstance();
boolean asyncSuccess = false;
try {asyncSuccess = cf.get();}
catch (Exception e) {}
if(true == asyncSuccess) {
a.updateOperationStatus(operationID, Constants.OpStatus.OK);
a.updateOperationMessage(operationID, "success: The long running process has finished and this is your result: SOME RESULT" );
}
else {
a.updateOperationStatus(operationID, Constants.OpStatus.INTERNAL_ERROR);
a.updateOperationMessage(operationID, "error: The long running process has failed.");
}
});
return response;
}
Here is also the APIOperationsManager.java for completness:
public class APIOperationsManager {
private static APIOperationsManager instance = null;
private Vector<Operation> operations;
private int currentOperationId;
private static final Logger log = LoggerFactory.getLogger(Application.class);
protected APIOperationsManager() {}
public static APIOperationsManager getInstance() {
if(instance == null) {
synchronized(APIOperationsManager.class) {
if(instance == null) {
instance = new APIOperationsManager();
instance.operations = new Vector<Operation>();
instance.currentOperationId = 1;
}
}
}
return instance;
}
public synchronized int registerNewOperation(OpStatus status) {
cleanOperationsList();
currentOperationId = currentOperationId + 1;
Operation newOperation = new Operation(currentOperationId, status);
operations.add(newOperation);
log.info("Registered new Operation to watch: " + newOperation.toString());
return newOperation.getId();
}
public synchronized Operation getOperation(int id) {
for(Iterator<Operation> iterator = operations.iterator(); iterator.hasNext();) {
Operation op = iterator.next();
if(op.getId() == id) {
return op;
}
}
Operation notFound = new Operation(-1, OpStatus.INTERNAL_ERROR);
notFound.setCrated(null);
return notFound;
}
public synchronized void updateOperationStatus (int id, OpStatus newStatus) {
iteration : for(Iterator<Operation> iterator = operations.iterator(); iterator.hasNext();) {
Operation op = iterator.next();
if(op.getId() == id) {
op.setStatus(newStatus);
log.info("Updated Operation status: " + op.toString());
break iteration;
}
}
}
public synchronized void updateOperationMessage (int id, String message) {
iteration : for(Iterator<Operation> iterator = operations.iterator(); iterator.hasNext();) {
Operation op = iterator.next();
if(op.getId() == id) {
op.setMessage(message);
log.info("Updated Operation status: " + op.toString());
break iteration;
}
}
}
private synchronized void cleanOperationsList() {
Date now = new Date();
for(Iterator<Operation> iterator = operations.iterator(); iterator.hasNext();) {
Operation op = iterator.next();
if((now.getTime() - op.getCrated().getTime()) >= Constants.MIN_HOLD_DURATION_OPERATIONS ) {
log.info("Removed operation from watchlist: " + op.toString());
iterator.remove();
}
}
}
}
The questions that I have
Is that concept a valid one that also scales? What could be improved?
Will i run into concurrency issues / race conditions?
Is there a better way to achieve the same in boot spring, but I just didn't find that yet? (maybe with the #Async directive?)
I would be very happy to get your feedback.
Thank you so much,
Peter P
It is a valid pattern to submit a long running task with one request, returning an id that allows the client to ask for the result later.
But there are some things I would suggest to reconsider :
do not use an Integer as id, as it allows an attacker to guess ids and to get the results for those ids. Instead use a random UUID.
if you need to restart your application, all ids and their results will be lost. You should persist them to a database.
Your solution will not work in a cluster with many instances of your application, as each instance would only know its 'own' ids and results. This could also be solved by persisting them to a database or Reddis store.
The way you are using CompletableFuture gives you no control over the number of threads used for the asynchronous operation. It is possible to do this with standard Java, but I would suggest to use Spring to configure the thread pool
Annotating the controller method with #Async is not an option, this does not work no way. Instead put all asynchronous operations into a simple service and annotate this with #Async. This has some advantages :
You can use this service also synchronously, which makes testing a lot easier
You can configure the thread pool with Spring
The /nonBlockingEndpoint should not return the id, but a complete link to the queryOpStatus, including id. The client than can directly use this link without any additional information.
Additionally there are some low level implementation issues which you may also want to change :
Do not use Vector, it synchronizes on every operation. Use a List instead. Iterating over a List is also much easier, you can use for-loops or streams.
If you need to lookup a value, do not iterate over a Vector or List, use a Map instead.
APIOperationsManager is a singleton. That makes no sense in a Spring application. Make it a normal PoJo and create a bean of it, get it autowired into the controller. Spring beans by default are singletons.
You should avoid to do complicated operations in a controller method. Instead move anything into a service (which may be annotated with #Async). This makes testing easier, as you can test this service without a web context
Hope this helps.
Do I need to make database access transactional ?
As long as you write/update only one row, there is no need to make this transactional as this is indeed 'atomic'.
If you write/update many rows at once you should make it transactional to guarantee, that either all rows are updated or none.
However, if two operations (may be from two clients) update the same row, always the last one will win.