I'm using Spring/Hibernate combination for my project, with standard CRUD operations.
I wonder, is it reasonable to validate object existence before its deletion or update? If it is, where is the most appropriate place to do - service or dao layer?
EDIT:
Sorry, I didn't make the point at first when I asked this question. The only motive for existence checking is to throw friendly exception to client of service (no DAO specific).
Problem is that I 'have to do' existence checking because my service method below is transactional, and besides that, I'm using HibernateTemplate and DaoSupport helper classes for session manipulation in DAO objects.
According to mentioned, Hibernate exception (in case of deleting non-existing instance for example) is thrown at commit time, which is out of my reach, because (I suppose) commit is executed by PlatformTransactionManager in proxy object, and I have no opportunity to handle that exception in my service method and re-throw some friendly exception to the client.
But even if I keep my strategy to check existence before deletion, bad stuff is that I have problems with NonUniqueObjectException in the case that instance exist, because I re-attach (in delete-time) already loaded instance (in read-time due existence checking).
For example:
//Existence checking in this example is not so important
public void delete(Employee emp){
Employee tempEmp=employeeDao.read(emp.getId());
if(tempEmp==null)
throw new SomeAppSpecificException();
}
//Existence checking in this example is maybe 'more logical'
public void save(Assignment a){
Task t=taskDao.read(a.getTask().getId());
if(t==null)
throw new FriendlyForeignKeyConstraintException();
Employee e=employeeDao.read(a.getEmployee().getId());
if(e==null)
throw new EmployeeNotFoundExc();
//...some more integrity checking and similar...
assignmentDao.save(a);
}
The point is that I just want to throw friendly exception with appropriate message in the case of mentioned situations (integrity violations and similar).
In Hibernate terms, both update and delete operations (and corresponding methods on Session) deal with persisted entities. Their existence is, therefore, implied and verifying it again in your code is kind of pointless - Hibernate will do that on its own and throw an exception if that's not the case. You can then catch and handle (or rethrow) that exception.
On a side note (based on your sample code), it's not necessary to explicitly read the instance you're going to delete. Session provides load() method that will return proxy instance suitable for passing to delete() method without actually hitting the database. It assumes that instance associated with given PK exists and will fail (throw an exception) later if that's not the case.
Edit (based on question clarification):
When you say you want to throw "friendly" exception to the client, the definitions of "friendly" and "client" become important. In most real-life scenarios your transaction would span across more than a simple atomic "save()" or "delete()" method on one of your services.
If the client is local and you don't need separate transactions within a single client interaction (typical scenario - web app running in the same VM with service layer), it's usually a good idea to initiate / commit transaction there (see Open Session In View, for example). You can then catch and properly handle (including wrapping / re-throwing, if needed) exceptions during commit. Other scenarios are more complicated, but ultimately the exception will be propagated to your "top level" client; it's just that unwrapping it may prove to be complicated if you need to present the "root" cause to the client in a "friendly" way.
The bottom line, though, is that it's up to you. If you'd rather fail fast with your own exception (at the expense of writing some boilerplate code), there's nothing really wrong with that.
Related
We are using the following frameworks and versions:
jOOQ 3.11.1
Spring Boot 2.3.1.RELEASE
Spring 5.2.7.RELEASE
I have an issue where some of our business logic is divided into logical units that look as follows:
Request containing a user transaction is received
This request contains various information, such as the type of transaction, which products are part of this transaction, what kind of payments were done, etc.
These attributes are then stored individually in the database.
In code, this looks approximately as follows:
TransactionRecord transaction = transactionRepository.create();
transaction.create(creationCommand);`
In Transaction#create (which runs transactionally), something like the following occurs:
storeTransaction();
storePayments();
storeProducts();
// ... other relevant information
A given transaction can have many different types of products and attributes, all of which are stored. Many of these attributes result in UPDATE statements, while some may result in INSERT statements - it is difficult to fully know in advance.
For example, the storeProducts method looks approximately as follows:
products.forEach(product -> {
ProductRecord record = productRepository.findProductByX(...);
if (record == null) {
record = productRepository.create();
record.setX(...);
record.store();
} else {
// do something else
}
});
If the products are new, they are INSERTed. Otherwise, other calculations may take place. Depending on the size of the transaction, this single user transaction could obviously result in up to O(n) database calls/roundtrips, and even more depending on what other attributes are present. In transactions where a large number of attributes are present, this may result in upwards of hundreds of database calls for a single request (!). I would like to bring this down as close as possible to O(1) so as to have more predictable load on our database.
Naturally, batch and bulk inserts/updates come to mind here. What I would like to do is to batch all of these statements into a single batch using jOOQ, and execute after successful method invocation prior to commit. I have found several (SO Post, jOOQ API, jOOQ GitHub Feature Request) posts where this topic is implicitly mentioned, and one user groups post that seemed explicitly related to my issue.
Since I am using Spring together with jOOQ, I believe my ideal solution (preferably declarative) would look something like the following:
#Batched(100) // batch size as parameter, potentially
#Transactional
public void createTransaction(CreationCommand creationCommand) {
// all inserts/updates above are added to a batch and executed on successful invocation
}
For this to work, I imagine I'd need to manage a scoped (ThreadLocal/Transactional/Session scope) resource which can keep track of the current batch such that:
Prior to entering the method, an empty batch is created if the method is #Batched,
A custom DSLContext (perhaps extending DefaultDSLContext) that is made available via DI has a ThreadLocal flag which keeps track of whether any current statements should be batched or not, and if so
Intercept the calls and add them to the current batch instead of executing them immediatelly.
However, step 3 would necessitate having to rewrite a large portion of our code from the (IMO) relatively readable:
records.forEach(record -> {
record.setX(...);
// ...
record.store();
}
to:
userObjects.forEach(userObject -> {
dslContext.insertInto(...).values(userObject.getX(), ...).execute();
}
which would defeat the purpose of having this abstraction in the first place, since the second form can also be rewritten using DSLContext#batchStore or DSLContext#batchInsert. IMO however, batching and bulk insertion should not be up to the individual developer and should be able to be handled transparently at a higher level (e.g. by the framework).
I find the readability of the jOOQ API to be an amazing benefit of using it, however it seems that it does not lend itself (as far as I can tell) to interception/extension very well for cases such as these. Is it possible, with the jOOQ 3.11.1 (or even current) API, to get behaviour similar to the former with transparent batch/bulk handling? What would this entail?
EDIT:
One possible but extremely hacky solution that comes to mind for enabling transparent batching of stores would be something like the following:
Create a RecordListener and add it as a default to the Configuration whenever batching is enabled.
In RecordListener#storeStart, add the query to the current Transaction's batch (e.g. in a ThreadLocal<List>)
The AbstractRecord has a changed flag which is checked (org.jooq.impl.UpdatableRecordImpl#store0, org.jooq.impl.TableRecordImpl#addChangedValues) prior to storing. Resetting this (and saving it for later use) makes the store operation a no-op.
Lastly, upon successful method invocation but prior to commit:
Reset the changes flags of the respective records to the correct values
Invoke org.jooq.UpdatableRecord#store, this time without the RecordListener or while skipping the storeStart method (perhaps using another ThreadLocal flag to check whether batching has already been performed).
As far as I can tell, this approach should work, in theory. Obviously, it's extremely hacky and prone to breaking as the library internals may change at any time if the code depends on Reflection to work.
Does anyone know of a better way, using only the public jOOQ API?
jOOQ 3.14 solution
You've already discovered the relevant feature request #3419, which will solve this on the JDBC level starting from jOOQ 3.14. You can either use the BatchedConnection directly, wrapping your own connection to implement the below, or use this API:
ctx.batched(c -> {
// Make sure all records are attached to c, not ctx, e.g. by fetching from c.dsl()
records.forEach(record -> {
record.setX(...);
// ...
record.store();
}
});
jOOQ 3.13 and before solution
For the time being, until #3419 is implemented (it will be, in jOOQ 3.14), you can implement this yourself as a workaround. You'd have to proxy a JDBC Connection and PreparedStatement and ...
... intercept all:
Calls to Connection.prepareStatement(String), returning a cached proxy statement if the SQL string is the same as for the last prepared statement, or batch execute the last prepared statement and create a new one.
Calls to PreparedStatement.executeUpdate() and execute(), and replace those by calls to PreparedStatement.addBatch()
... delegate all:
Calls to other API, such as e.g. Connection.createStatement(), which should flush the above buffered batches, and then call the delegate API instead.
I wouldn't recommend hacking your way around jOOQ's RecordListener and other SPIs, I think that's the wrong abstraction level to buffer database interactions. Also, you will want to batch other statement types as well.
Do note that by default, jOOQ's UpdatableRecord tries to fetch generated identity values (see Settings.returnIdentityOnUpdatableRecord), which is something that prevents batching. Such store() calls must be executed immediately, because you might expect the identity value to be available.
I am working on a new Spring MVC based application.
I have multiple flows where the controller will make request to business manager and further business manager will talk to DAO layer to retrieve data.
There can be possible cases where I don't get data back from the DAO.
I want to understand what is best way to deal with this situation.
1) When ever there is no data retrieved for a query then Throw back Custom Exception like 'Content Not Found' from DAO layer to Business Layer and then to Controller and let controller decide what to do.
2) Return blank/null Pojo object back to business manager and let manager throw the exception to Controller.
3) Controller receives null/blank from Manager and decides what to do with that.
I am finding 1st approach better as when the exception is thrown i have complete stack trace to understand where exactly the problem occurred but on downside I will end up cluttering my code with Exception in the signatures.
Number 3 will leave the code clean but I wont be able to pin point where exactly the data retrieval failed as there can be multiple calls to DAO from Business Layer.
Throw an exception on the level where the situation of not having matching records (in other words, no data to be processed) actually is exceptional.
This largely depends on the specifics of your domain, but it's often the best idea to simply return an empty container object from the DAO if there was no matching object in the database. That is: an Collections.emptyList(), Optional.empty() or something with similar semantics. Under no circumstances return null, it's 2015 after all.
If having no matching data is an exceptional situation in your business domain, translate that to a specific exception in the service layer and let the controller handle that by translating again: into an error HTML page, some specific XML or JSON response or whatever the interface your users use to interact with your system.
The DAO layer executes queries and returns the results. It doesn't care about the results, so "nothing found" cannot be an exceptional situation in the DAO layer. It can be in the business layer, but it doesn't have to.
I wont be able to pin point where exactly the data retrieval failed
If your use case is http://server/something/2 and something 2 doesn't exist in the database, then there simply is no failure on the server side. So if there is no exception, or only one in the controller, then you can be pretty confident that no data is returned to the client because no data exists.
I would suggest you to throw custom exceptions at each layer. Each layer should be aware of exception handling.
Its beautifully explained in the below link.
Handling Dao exceptions in service layer
There are various transaction propagations like
REQUIRED - This is the case for DML operation.
SUPPORTS - This is the case for querying database.
MANDATORY - ?
REQUIRES_NEW - ?
NOT_SUPPORTED - ?
NEVER - ?
NESTED - ?
What are some real life scenarios for these transaction propagations? Why these are perfect fit into that situation?
There are various usages and there is no simple answer but I'll try to be the most explainatory
MANDATORY (expecting transaction): Typically used when you expect that any "higher-context" layer started the transaction and you only want to continue in it. For example if you want one transaction for whole operation from HTTP request to response. Then you start transaction on JAX-RS resource level and lower layers (services) require transaction to run within (otherwise exception is thrown).
REQUIRES_NEW (always create new transaction): This creates a new transaction and suspends the current one if any exists. From above example, this is the level you set on your JAX-RS resource for example. Or generally if your flow somehow changes and you want to split your logic into multiple transactions (so your code have mutliple logic operations that should be separated).
REQUIRED (continue in transaction or create if needed): Some kind of mix between MANDATORY and REQUIRES_NEW. In MANDATORY you expect that the transaction exists, for this level you hope that it exists and if not, you create it. Typically used in DAO-like services (from my experience), but it really depends on logic of your app.
SUPPORTS (caller decides whether to not/run in transaction): Used if want to use the same context as caller (higher context), if your caller was running in transaction, then you run in too. If it didn't, you are also non-transactional. May also be used in DAO-like services if you want higher context to decide.
NESTED (sub-transaction): I must admit I didn't use this one in real code but generally you create a real sub-transaction that works as some kind of checkpoint. So it runs in the context of "parent" transaction but if it fails it returns to that checkpoint (start of nested transaction). This may be useful when you require this kind of logic in your application, for example if you want to insert large number of items to the database, commiting valid ones and keeping track of invalid ones (so you can catch exception when nested transaction fails but can still commit the whole transaction for valid ones)
NEVER (expecting there is no transaction): This is the case when you want to be sure that no transaction exists at all. If some code that runs in transaction reaches this code, exception is thrown. So typically this is for cases completely opposite to MANDARTORY. E.g. when you know that no transaction should be affected by this code - very likely because the transaction should not exist.
NOT_SUPPORTED (continue non-transactionally): This is weaker than NEVER, you want the code to be run non-transactionally. If somehow you enter this code from context where transaction is, you suspend this transaction and continue non-transactionally.
From my experience, you very often want one business action to be atomic. Thus you want only one transaction per request/... For example simple REST call via HTTP that does some DB operations all in one HTTP-like transaction. So my typical usage is REQUIRES_NEW on the top level (JAX-RS Resource) and MANDATORY on all lower level services that are injected to this resource (or even lower).
This may be useful for you. It describes how code behave with given propagation (caller->method)
REQUIRED: NONE->T1, T1->T1
REQUIRES_NEW: NONE->T1, T1->T2
MANDATORY: NONE->Exception, T1->T1
NOT_SUPPORTED: NONE->NONE, T1->NONE
SUPPORTS: NONE->NONE, T1->T1
NEVER: NONE->NONE, T1->Exception
I am using the play framework to develop a web application which accesses a postgres db using JOOQ and spring transactions.
Currently I am implementing the user signup which is structured in the following way:
The user posts the signup form
The request is routed to a controller action which maps all parameters like e-mail, password etc. on a DTO. The different fields of the DTO are annotated with JSR 303 constraints.
The e-mail field is annotated with a constraint validator that makes sure that the same address is not added twice. This validator has an autowired reference to the UserRepository, so that it can invoke it's isExistingEmail method.
The signup method of the user service is called, which basically looks as follows:
#Transactional(isolation = Isolation.SERIALIZABLE)
public User signupUser(UserDto userDto) {
validator.validate(userDto);
userRepository.add(userDto);
return tutor;
}
In case of a validation error the validator.validate(userDto) call inside of the service will throw a RuntimeException.
Please note that the repository's add method is annotated with #Transactional(propagation = Propagation.MANDATORY) while the isExistingEmail method does not have any annotations.
My problem is that when I post the signup form twice in succession, I receive a unique constraint error from the database since both times the userRepository.isExistingEmail call returns false. However, what I would expect is that the second signup call is not allowed to add the user to the repository, as I set the isolation level of the transaction to serializable.
Is this the expected behavior or might there be a JOOQ/spring transactions configuration issue?
I added a TransactionSynchronizationManager.isActualTransactionActive() call in the service to make sure a transaction is actually active. So this part seems to work.
After some more research and reading the documentation on transaction isolation in the postgres manual I have come to realize that my understanding of spring managed transactions was just lacking.
When setting the isolation level to SERIALIZABLE postgres won't really block any concurrent transactions. Instead it will make use of predicate locks to monitor if a committed transaction would produce a result that is different than actually running concurrent transactions one after another.
An exception will only be thrown by the underlying database driver if the state of the data is not valid when the second transaction tries to commit. I was able to verify this behavior and to force a serialization failure by temporarily removing the unique constraint on my e-mail field.
My final solution was to reduce the isolation level to READ_COMMITTED and to handle the unique constraint violation exception when invoking userRepository.add(userDto), since a SERIALIZABLE isolation level is not really necessary to deal with this particular use case.
Please let me know of any better ways of dealing with this kind of standard situation.
In a Grails app, the default behaviour of service methods is that they are transactional and the transaction is automatically rolled-back if an unchecked exception is thrown. However, in Groovy one is not forced to handle (or rethrow) checked exceptions, so there's a risk that if a service method throws a checked exception, the transaction will not be rolled back. On account of this, it seems advisable to annotate every Grails service class
#Transactional(rollbackFor = Throwable.class)
class MyService {
void writeSomething() {
}
}
Assume I have other methods in MyService, one of which only reads the DB, and the other doesn't touch the DB, are the following annotations correct?
#Transactional(readOnly = true)
void readSomething() {}
// Maybe this should be propagation = Propagation.NOT_SUPPORTED instead?
#Transactional(propagation = Propagation.SUPPORTS)
void dontReadOrWrite() {}
In order to answer this question, I guess you'll need to know what my intention is:
If an exception is thrown from any method and there's a transaction in progress, it will be rolled back. For example, if writeSomething() calls dontReadOrWrite(), and an exception is thrown from the latter, the transaction started by the former will be rolled back. I'm assuming that the rollbackFor class-level attribute is inherited by individual methods unless they explicitly override it.
If there's no transaction in progress, one will not be started for methods like dontReadOrWrite
If no transaction is in progress when readSomething() is called, a read-only transaction will be started. If a read-write transaction is in progress, it will participate in this transaction.
Your code is right as far as it goes: you do want to use the Spring #Transactional annotation on individual methods in your service class to get the granularity you're looking for, you're right that you want SUPPORTS for dontReadOrWrite (NOT_SUPPORTED will suspend an existing transaction, which won't buy you anything based on what you've described and will require your software to spend cycles, so there's pain for no gain), and you're right that you want the default propagation behavior (REQUIRED) for readSomething.
But an important thing to keep in mind with Spring transactional behavior is that Spring implements transaction management by wrapping your class in a proxy that does the appropriate transaction setup, invokes your method, and then does the appropriate transaction tear-down when control returns. And (crucially), this transaction-management code is only invoked when you call the method on the proxy, which doesn't happen if writeSomething() directly calls dontReadOrWrite() as in your first bullet.
If you need different transactional behavior on a method that's called by another method, you've got two choices that I know of if you want to keep using Spring's #Transactional annotations for transaction management:
Move the method being called by the other into a different service class, which will be accessed from your original service class via the Spring proxy.
Leave the method where it is. Declare a member variable in your service class to be of the same type as your service class's interface and make it #Autowired, which will give you a reference to your service class's Spring proxy object. Then when you want to invoke your method with the different transactional behavior, do it on that member variable rather than directly, and the Spring transaction code will fire as you want it to.
Approach #1 is great if the two methods really aren't related anyway, because it solves your problem without confusing whoever ends up maintaining your code, and there's no way to accidentally forget to invoke the transaction-enabled method.
Approach #2 is usually the better option, assuming that your methods are all in the same service for a reason and that you wouldn't really want to split them out. But it's confusing to a maintainer who doesn't understand this wrinkle of Spring transactions, and you have to remember to invoke it that way in each place you call it, so there's a price to it. I'm usually willing to pay that price to not splinter my service classes unnaturally, but as always, it'll depend on your situation.
I think that what you're looking for is more granular transaction management, and using the #Transactional annotation is the right direction for that. That said, there is a Grails Transaction Handling Plugin that can give you the behavior that you're looking for. The caveat is that you will need to wrap your service method calls in a DomainClass.withTransaction closure and supply the non-standard behavior that you're looking for as a parameter map to the withTransaction() method.
As a note, on the backend this is doing exactly what you're talking about above by using the #Transactional annotation to change the behavior of the transaction at runtime. The plugin documentation is excellent, so I don't think you'll find yourself without sufficient guidance.
Hope this is what you're looking for.