If I have an ATG Nucleus Repository Item that is not cacheable (ATG/Nucleus simple cache is disabled) AND I'm not in a transaction, the following results in two queries to the database.
The following code results in a db query for every property.
repositoryItem.getPropertyValue("columnA");
repositoryItem.getPropertyValue("columnB");
If debugging for the user entity is enabled you would see the following log statements ever each call:
repositoryItem.getPropertyValue("columnA");
DEBUG loadingPropertyFromDatabase(user:ID_1.columnA, column_a_value) property is not cacheable caching disabled for this transaction
DEBUG loadingPropertyFromDatabase(user:ID_1.columnB, column_b_value) property is not cacheable caching disabled for this transaction
DEBUG getPropertyValue(user:ID_1.columnA) -> "column_a_value" (value from database)
repositoryItem.getPropertyValue("columnB");
DEBUG loadingPropertyFromDatabase(user:ID_1.columnA, column_a_value) property is not cacheable caching disabled for this transaction
DEBUG loadingPropertyFromDatabase(user:ID_1.columnB, column_b_value) property is not cacheable caching disabled for this transaction
DEBUG getPropertyValue(user:ID_1.columnB) -> "column_b_value" (value from database)
We cannot enable caching, due to how the object is being access/updated by other systems.
I also do not want to create a transaction for a read only query of the entity.
If I was using Hibernate, the Hibernate session would keep a state within the session, even if I was not in a transaction. That doesn't seem to be the case with ATG/Nucleus. Is there any way I can get this type of behavior or a thread level cache?
In looking at documentation and walking through the code via debugger (which is difficult w/out source), I am not having any luck finding a work around.
Thanks!
You need to wrap the getPropertyValue calls with a transaction which will save the results of the database queries into the temporary transaction cache. That will prevent the repository from going back to the database for every getPropertyValue call.
You also want to ensure that all the properties you are accessing are part of the same property group (as described here). The first load of the item from the database will pull in the properties in the same group as the ID property. This combined with the transaction cache will significantly reduce the number of database queries.
I also do not want to create a transaction for a read only query of
the entity.
I don't understand why you wouldn't want to explicitly demarcate a transaction. Every getPropertyValue call will automatically create (and end) a transaction if one isn't already present. So in your example, you would have 2 transactions implicitly created for you. Why not just create 1 transaction explicitly?
Related
I have a list of ~10 000 objects.
I am trying to call an mysql update query (procedure) and then to get the updated objects inside same transaction.
Can this be achieved ?
When I call a delete statement + flush(), hibernate retrieves me correct objects (deleted objects are missing)
But when I try update statement + flush(), hibernate retrieves me the initial unchanged objects.
#Transactional
void test() {
//...
em.createQuery("delete from StorefrontProduct sp where sp in (:storefrontProducts)")
.setParameter("storefrontProducts", storefrontProductsToDelete)
.executeUpdate();
// example
em.createQuery("update StorefrontProduct sp set sp.orderIndex=0 where sp.id=90")
.executeUpdate();
em.flush();
//Simple JPA query
List<StorefrontProduct> result = repository.findAllByPreviousOrderIndexIsNotNull();
//additional code....
}
After running the code from above and putting a breakpoint after findAll call, provided objects from 1-st query were deleted and flushed, but the update query was not flushed.
That is known counterintuitive behaviour of Hibernate.
First of all, em.flush() call might be superfluous if flush mode set to AUTO (in that case Hibernate automatically synchronises persistence context (session-level cache) with underlying database prior executing update/delete queries).
Delete and successive Select case:
you issues delete then select, since select does not see deleted records anymore you do not see deleted records in resultset, however if you call findById you may find deleted records.
Update and successive Select case:
you issues update then select, when processing resultset Hibernate sees both records stored in database and records stored in persistence context and it assumes that persistence context is a source of truth, that is the reason why you see "stale" data.
There are following options to mitigate that counterintuitive behaviour:
do not perform direct updates, use "slow" find/save API instead
either detach or refresh stale entities after direct update, em.clear() may also help, however it completely cleans up persistence context, which might be undesirable
I am working on a Java 8 / Spring Boot 2 application and I have noticed that the security module of my app internally uses the findByEmail method of my UserRepostiory (which is a standard Spring Data JPA Repository). When I enabled Hibernate SQL logging, I discovered that these queries are performed multiple times within the same session (security uses it 3-4 times and then my business code uses it some more times). Each time the query hits the database.
This surprised me, as I expected it to be cached in the Hibernate's first level cache. After reading up about it a little bit more, I found out that the first level cache only caches the result of the findById query, not others.
Is there anyway that I can cache the result of the findByEmail query in the first level cache? (I don't want the cache to be shared between sessions, I don't want to use the 2nd level cache, as I think it should be invalidated right after the current session ends).
Yes, you can cache the results of a query on a unique property if you annotate the property with the #NaturalId annotation. If you then use the dedicated API to execute the query, the results will be stored in the 1st level cache. An example:
User user = entityManager
.unwrap(Session.class)
.bySimpleNaturalId(User.class)
.load("john#example.com");
I have the following code:
println "######## RUNNING ProfessionaCustomer - ${pcCounter} under ${accountCustomer.customerNumber} Professional SQLid ${it.id}"
def professionalCustomerId = it.customerId
def professionalCustomer = ProfessionalCustomer.findByCustomerNumber(professionalCustomerId)
I have SQL logging on and I get:
######## RUNNING ProfessionaCustomer - 31 under 106450 Professional SQLid 100759
Hibernate: update base_domain set version=?, account_name=?, address_line1=?, address_line2=?, city=?, customer_number=?, date_created=?, disabled=?, last_updated=?, postal_code=?, primary_phone=?, state_or_province=? where id=? and version=?
Hibernate: update base_domain set version=?, address1=?, address2=?, city=?, customer_number=?, date_created=?, disabled=?, first_name=?, last_name=?, last_updated=?, middle_name=?, phone_number=?, postal_code=?, state=? where id=? and version=?
Hibernate: insert into account_customer_professionals (account_customer_id, professional_customer_id) values (?, ?)
Hibernate: select this_.id as id1_3_0_, this_.version as version2_3_0_, this_.address1 as address70_3_0_, this_.address2 as address71_3_0_, this_.city as city7_3_0_, this_.customer_number as customer8_3_0_, this_.date_created as date_cre9_3_0_, this_.disabled as disable10_3_0_, this_.first_name as first_n19_3_0_, this_.last_name as last_na20_3_0_, this_.last_updated as last_up11_3_0_, this_.middle_name as middle_72_3_0_, this_.phone_number as phone_n73_3_0_, this_.postal_code as postal_12_3_0_, this_.state as state74_3_0_ from base_domain this_ where this_.class='com.eveo.nplate.model.ProfessionalCustomer' and this_.customer_number=? limit ?
Which is updating the DB. This would explain why this is so slow, but I can't see any reason for this to happen.
Why would 'findBy' cause an update?
Hibernate doesn't immediately execute creates, updates, or deletes until it thinks it has to - it waits as long as possible (although it's rather pessimistic) and only flushes these changes when you tell it to, or when it thinks it needs to. In general the only time it will flush without an explicit call is when running queries. This is because any of the new instances, updated instances, and deleted instances that are in-memory (cached in the Hibernate Session, the 1st-level cache) could affect the query results, so they must be flushed to the database so you get the proper results for your query.
One exception to this is calling save() on a new instance. Grails flushes this because typically the id is assigned by the database, either via an auto-increment column or a sequence. To ensure that the in-memory state is the same as the database, it flushes the save() call so it can retrieve the id and set it in the instance. But if you retrieve a persistence instance (e.g. with a get() call, or with a criteria query, finder, etc.) and modify it, calling save() on that does not get automatically flushed. The same goes for delete() calls - not flushed.
Think of delete() and save() calls on persistent instances as messages to Hibernate that the action should be performed "eventually".
So when you execute a finder, or a criteria, "where", or HQL query, Hibernate will flush any un-flushed changes for you. If you don't want that to happen (e.g. in a custom domain class validator closure) you can run the query in a separate session, e.g. with the withNewSession method.
If you don't flush the session at all, either explicitly on the Session instance or by adding flush:true to a save or delete call, the session will be flushed, since Grails registers an OpenSessionInView interceptor that starts a session at the beginning of each request, and flushes and closes it at the end. This helps with lazy loading; since there's a session open and bound to a ThreadLocal in a known location, Hibernate and GORM (via Spring's HibernateTemplate) can use that open session to retrieve lazy-loaded collections and instances on-demand after the query runs.
Note also that you do not need to flush in a transaction. The transaction manager is a Spring HibernateTransactionManager that flushes before committing.
Probably there was some transaction in the session that was not persisted in the database.
When you ran the findBy hibernate took advantage of the connection to run the two queries. I believe this is what happened.
First of all i would like to confirm is it the responsibility of developer to follow these properties or responsibilty of transaction Apis like JDBC?
Below is my understanding how we achieve acid properties in JDBC
Atomicity:- as there is one transaction associated with connection, so we do commit or rollback , there are no partial updation.Hence achieved
Consitency:- when some data integrity constraint is voilated (say some check constraint) then sqlexception will be thrown . Then programmer acieve the consistent database by rollbacking the transaction?
one question on above say we do transaction1 and sql excpetion is thrown during transaction 2 as explained above . Now we catch the exception and do the commit will first transaction be commited?
Isolation:- Provided by JDBC Apis.But this leads to the problem of concurrent update . so it has be dealt manually right?
Durability:- Provided by JDBC Apis.
Please let me if above understanding is right?
ACID principles of transactional integrity are implemented by the database not by the API (like JDBC) or by the application. Your application's responsibility is to choose a database and a database configuration that supports whatever transactional integrity you need and to correctly identify the transactional boundaries in your application.
When an exception is thrown, your application has to determine whether it is appropriate to rollback the entire transaction or to proceed with additional processing. It may be appropriate if your application is processing orders from a vendor, for example, to process the 99 orders that succeed and log the 1 order that failed somewhere for users to investigate. On the other hand, you may reject all 100 orders because 1 failed. It depends what your application is doing.
In general, you only have one transaction open at a time (or, more accurately, one transaction per connection). So if you are working in transaction 2, transaction 1 by definition has already completed-- it was either committed or rolled back previously. Exceptions thrown in transaction 2 have no impact on transaction 1.
Depending on the transaction isolation level your application requests (and the transaction isolation levels your database supports) as well as the mechanics of your application, lost updates are something that you may need to be concerned about. If you set your transaction isolation level to read committed, it is possible that you would read a value as 'A' in transaction 1, wait for a user to do something, update the value to 'B', and commit without realizing that transaction 2 updated the value to 'C' between the time you read the data and the time you wrote the data. This may be a problem that you need to deal with or it may be something where it is fine for the last person to update a row to "win".
Your database, on the other hand, should take care of the automatic locking that prevents two transactions from simultaneously updating the same row of the same table. It may do this by locking more than is strictly necessary but it will serialize the updates somehow.
I have a table and two databases which have the same table, but one is a symlink of the other one and only read is permitted on this table.
I have mapped the table to Java using Hibernate and I use spring to set the Entity Manager's data source as one of the two databases based on some input criteria.
I call only read only operations (selects) when I am connected to the second database, but it seems Hibernate tries to flush something back to the database and it fails telling update is not allowed on this view.
How do I disable this update only for the second datasource and keep it normal for the first one?
Update:
Looking at the stack trace, the flush seems to be started here:
at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:321)
at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:50)
at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1027)
at org.hibernate.impl.SessionImpl.managedFlush(SessionImpl.java:365)
at org.hibernate.ejb.AbstractEntityManagerImpl$1.beforeCompletion(AbstractEntityManagerImpl.java:504)
... 55 more
Is this related to hibernate.transaction.flush_before_completion property? Can I set it to false for the second data source?
Most probably your entities become "dirty" the same moment they are loaded from the database, and Hibernate thinks that it needs to store the changes. This happens, if your accessors (get and set methods) are not returning the exact same value or reference that had been set by Hibernate.
In our code, this happened with lists, developers created new list instances because they didn't like the type they got in the setter.
If you don't want to change the code, change the mapping to field access.
You can also prevent Hibernate of storing changes by setting FlushMode to never on the session, but this only hides the real problem which will still occur in other situations an will lead to unnecessary updates.
First you need to determine if this is DDL or DML. If you don't know, then I recommend you set hibernate.show_sql=true to capture the offending statement.
If it is DDL, then it's most likely going to be Hibernate updating the schema for you and you'd want to additionally configure the hibernate.hbm2ddl.auto setting to be either "update" or "none", depending on whether you're using the actual db or the symlinked (read-only) one, respectivley. You can use "validate" instead of none, too.
If it is DML, then I would first determine whether your code is for some reason making a change to an instance which is still attached to an active Hibernate Session. If so, then a subsequent read may cause a flush of these changes without ever explicitly saving the object (Grails?). If this is the case, consider evicting the instance causing the flush ( or using transport objects instead ).
Are you perhaps using any aspects or Hibernate lifecycle events to provide auditing of the objects? This, too, could cause access of a read-only to result in an insert or update being run.
It may turn out that you need to provide alternative mappings for the offending class should the updatability of a field come into play, but the code is doing everything exactly as you'd like ( this is unlikely ;0 ). If you are in an all-annotation world, this may be tricky. If working with hbm.xml, then providing an alternative mapping is easier.