I have two models: User and Base.
User model:
http://pastebin.com/WdLzBkHJ
Base model:
http://pastebin.com/tQrEUaSu
At first I want to put your mind on this notation in Base model:
#org.hibernate.annotations.Entity(dynamicInsert=true,dynamicUpdate=true)
It doesn't work (in SQL Debug shown that Hibernate generated queries with using unnecessary columns which in MySQL are configured as nullable). Tell me please why? What am I doing wrong?
And the main problem is there (method in which user uploads base and string in this base inserts in the MySQL table after parsing):
http://pastebin.com/yG3Mapze
Insertation is VERY SLOW. I have file with 70000 string per line and I can't wait until Hibernate insert this string into DB. Maximum I was waiting for 30 minutes and that wasn't the end.
If I'll use raw queries like that:
DB.execute("INSERT INTO bases (user_id,email,password) VALUES (1,'" + email.replaceAll("'", "\'") + "','" + password.replaceAll("'", "\'") + "')");
instead of
b.save();
After that insertation of 70000 strings to DB completes after ~10-20 seconds.
So I can't understand where is the problem and how to fix it?
Also you can see this code above method declaration:
#NoTransaction
If I uncomment it then I'll recieve this exception:
#689mbad1k
Internal Server Error (500) for request POST /checker/uploadnewbase
JPA error
A JPA error occurred (The JPA context is not initialized. JPA Entity Manager automatically start when one or more classes annotated with the #javax.persistence.Entity annotation are found in the application.):
play.exceptions.JPAException: The JPA context is not initialized. JPA Entity Manager automatically start when one or more classes annotated with the #javax.persistence.Entity annotation are found in the application.
at play.db.jpa.JPA.get(JPA.java:22)
at play.db.jpa.JPA.em(JPA.java:51)
at play.db.jpa.JPQL.em(JPQL.java:16)
at play.db.jpa.JPQL.find(JPQL.java:44)
at models.User.find(User.java)
at controllers.Security.getUser(Security.java:30)
at controllers.GlobalController.userStat(GlobalController.java:21)
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:502)
at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:476)
at play.mvc.ActionInvoker.invokeControllerMethod(ActionInvoker.java:471)
at play.mvc.ActionInvoker.handleBefores(ActionInvoker.java:320)
at play.mvc.ActionInvoker.invoke(ActionInvoker.java:140)
at Invocation.HTTP Request(Play!)
But in Play manual we can see that:
"If you want to prevent Play from starting any transaction at all, you can annotate the method with #play.db.jpa.NoTransaction.
To prevent transactions for all methods, you can annotate the Controller-class with #play.db.jpa.NoTransaction."
So I have three problems which I described:
About exception in NoTransaction.
About using dynamicInsert = true.
About improving perfomance for Hibernate like if I'll use raw queries.
The problem is the hibernate session, which must be cleared. Otherwise you get problems with memory and performance. You can found some information in http://docs.jboss.org/hibernate/core/3.3/reference/en/html/batch.html. Unfortunately I don't know how to get the HibernateSession. Perhaps you can get the EntityManager and work with it. But my experiences with Hibernate and Batch are really frustrating, so I would recommend to use your raw-solution.
Related
I have a spring boot application (based off spring-boot-starter-data-jpa. I have an absolute minimum of configuration going on, and only a single table and entity.
I'm using CrudRepository<Long, MyEntity> with a couple of findBy methods which all work. And I have a derived deleteBy method - which doesn't work. The signature is simply:
public interface MyEntityRepository<Long, MyEntity> extends CrudRespository<> {
Long deleteBySystemId(String systemId);
// findBy methods left out
}
The entity is simple, too:
#Entity #Table(name="MyEntityTable")
public class MyEntity {
#Id
#GeneratedValue(strategy=GenerationType.IDENTITY)
#Column(name="MyEntityPID")
private Long MyEntityPID;
#Column(name="SystemId")
private String systemId;
#Column(name="PersonIdentifier")
private String personIdentifier;
// Getters and setters here, also hashCode & equals.
}
The reason the deleteBy method isn't working is because it seems to only issue a "select" statement to the database, which selects all the MyEntity rows which has a SystemId with the value I specify. Using my mysql global log I have captured the actual, physical sql and issued it manually on the database, and verified that it returns a large number of rows.
So Spring, or rather Hibernate, is trying to select the rows it has to delete, but it never actually issues a DELETE FROM statement.
According to a note on Baeldung this select statement is normal, in the sense that Hibernate will first select all rows that it intends to delete, then issue delete statements for each of them.
Does anyone know why this derived deleteBy method would not be working? I have #TransactionManagementEnabled on my #Configuration, and the method calling is #Transactional. The mysql log shows that spring sets autocommit=0 so it seems like transactions are properly enabled.
I have worked around this issue by manually annotating the derived delete method this way:
public interface MyEntityRepository<Long, MyEntity> extends CrudRespository<> {
#Modifying
#Query("DELETE FROM MyEntity m where m.systemId=:systemId")
Long deleteBySystemId(#Param("systemId") String systemId);
// findBy methods left out
}
This works. Including transactions. But this just shouldn't have to be, I shouldn't need to add that Query annotation.
Here is a person who has the exact same problem as I do. However the Spring developers were quick to wash their hands and write it off as a Hibernate problem so no solution or explanation to be found there.
Oh, for reference I'm using Spring Boot 2.2.9.
tl;dr
It's all in the reference documentation. That's the way JPA works. (Me rubbing hands washing.)
Details
The two methods do two different things: Long deleteBySystemId(String systemId); loads the entity by the given constraints and ends up issuing EntityManager.delete(…) which the persistence provider is about to delay until transaction commits. I.e. code following that call is not guaranteed that the changes have already been synced to the database. That in turn is due to JPA allowing its implementations to actually do just that. Unfortunately that's nothing Spring Data can fix on top of that. (More rubbing, more washing, plus a bit of soap.)
The reference documentation justifies that behavior with the need for the EntityManager (again a JPA abstraction, not something Spring Data has anything to do with) to trigger lifecycle events like #PreDelete etc. which users expect to fire.
The second method declaring a modifying query manually is declaring a query to be executed in the database, which means that entity lifecycles do not fire as the entities do not get materialized upfront.
However the Spring developers were quick to wash their hands and write it off as a Hibernate problem so no solution or explanation to be found there.
There's detailed explanation why it works the way it works in the comments to the ticket. There are solutions provided even. Workarounds and suggestions to bring this up with the part of the stack that has control over this behavior. (Shuts faucet, reaches for a towel.)
I am very new for testng(unit testing) and mockito. I have read some articles and went through some code snippets on Internet. But still I have some doubts regarding unit testing with testng & mockito in spring framework.
For unit testing a service layer we mock a DAO. What if I want to test a function wich fetch some data from database and do some operations. How does mock DAO works here. From where mocked DAO will get some data for testing such a function.
If am doing a validation like Data not present in database and I want to test wheather it throws correct exception for that. So it needs some values in database and mocked DAO will check if data present in that predefined database(in-memory). How to provide such a data.
Does dataprovider helps to provide a data to used by DAO. If yes, How it can be done?
Please correct me if my understanding regarding unit testing is correct. Please let me know where I am getting it wrong if I miss understood a concept.
Thank you.
1) Besides UnitTests, you also need Integration and / or Acceptance-Tests.
The Unit-Tests will test that your SUT - Single Unit of Test, in this case a specific service class works as intended, without integrating it with other classes or systems (DB). However, additionally I would write an Integration Test for this Service that retrieves / manipulates test data from the database. This test should ideally not make any assumptions about the data in the database, so inserting the data you will be looking for, before executing the test, is recommended, e.g. using a #Before annotation and actually committing this test data into the test database. However, I further recommend you to do a proper cleanup of the database, in the #After test method. Auto rolling back the data could be done, but is not as optimal, especially if you have a persistence framework like Hibernate or JPA in between. Only when you work on committed data that is really in the physical (not virtual!) dabase, you can be 100% sure your test succeeded.
If I correctly understood your intend, this actually sounds like a perfect reason for mocking your DB / persistence object - make it throw the expected exception / return an empty result, that test that your code behaves as expected on this condition.
A TestNG Dataprovider actually does the opposite of what you are looking for - it is a way to provide an array of data to your test method:
org.testng.annotations.DataProvider
Annotation Type DataProvider
Mark a method as supplying data for a test method. The data provider name defaults to method name. The annotated method must return an Object[][] where each Object[] can be assigned the parameter list of the test method. The #Test method that wants to receive data from this DataProvider needs to use a dataProvider name equals to the name of this annotation.
How transaction is controlled while using JdbcTemplate/HibernateTemplateand HibernateDaoSupport/JdbcDaoSupport? I used to check source code and didn't find where the transaction is controlled by JdbcTemplate/HibernateTemplate and HibernateDaoSupport/JdbcDaoSupport.
And In source code HibernateDaoSupport/JdbcDaoSupport is using JdbcTemplate/HibernateTemplate, what's the role of HibernateDaoSupport/JdbcDaoSupport and what's the role of JdbcTemplate/HibernateTemplate?
Why do we use JdbcTemplate/HibernateTemplate and HibernateDaoSupport/JdbcDaoSupport? It seems all sample code is using them. What should I use if I don't want to use them, such as only using spring + hibernate?
If I'm using JdbcTemplate/HibernateTemplate and HibernateDaoSupport/JdbcDaoSupport, do I still need to config transaction proxy in xml? If I still need to config transaction proxy in xml, it means it's ok for me to put both getHibernateTemplate().saveOrUpdate(user)and getHibernateTemplate().saveOrUpdate(order) together, and they're invoked in the same transaction, is this right?
First off all please forget about HibernateTemplate and HibernateDaoSupport these classes should be considered deprecated since the release of hibernate 3.0.1 (which was somewhere in 2006!). You should be creating daos/repositories based on a plain hibernate API, as explained in the Spring Reference Guide. (The same goes for JpaTemplate and JpaDaoSupport).
JdbcTemplate (and all other *Template classes) intend is to make it easier to work with the underlying technology. Once upon a time this was also needed for Hibernate (< 3.0.1), now it isn't.
JdbcTemplate makes it easier to work with plain JDBC code. You don't have to get a connection, create a (Prepared)Statement, add the parameters, execute the query, iterate over the resultset and convert the ResultSet. With the JdbcTemplate much of this is hidden and most of it can be written in 1 to 3 lines of code, whereas plain JDBC would require a lot more.
The *Support classes make it easier to gain access to a template but aren't a must to use. Creating a JdbcTemplate is quite easy and you don't really need to extend JdbcDaoSupport. But you can if you want. For more information a lot is explained in the reference guide.
I wrote a web service with spring boot using spring data jpa for persistence.
The webservice has some static objects (in Singleton Bean) that regulary needs to be backed up to my database.
Sometimes! (This sucks...I dont' really know what happens) when I call
ObjectType updated = myRepository.save(existingObject)
I get an java.lang.NullPointerException - without usable stacktrace as the method doing this is running via #Scheduled.
I tried debugging and existingObject seems to be absolutely fine. The error only occurs, when existingObject is actually NOT a new object (i.e. when id != 0)
P.S. I am using Spring Boot therefore not really using EntityManager. I only use the #Autowired myRepository.
I'm seeing something similar happening. During save, it seems the object is re-fetched from DB (perhaps to see which fields were altered?) but a ManyToOne relationship is not loaded (even though the FetchType is explicitly set to EAGER).
For some reason, a compareTo is called on the relationship. The related object isn't null, but it only has its ID filled in (presumably because that was available in the object that was fetched from the DB). All other fields are null.
When the compareTo then does its stuff, a NullPointerException follows.
As to the actual solution, I don't know yet, as I would have expected the FetchType EAGER to make sure the relationship is loaded. Hopefully this helps someone to further find the root cause.
(I would have added this as a comment as it doesn't actually answer the question, but StackOverflow won't let me due to insufficient reputation...)
You haven't provided enough information. IF that line is where the NullPointerException is occurring, then the only possibilities are that myRepository is null, or existingObject is null. However, it's possible the NullPointerException is happening as a result of something in the save. Wrap the code in a try catch, and log the exception stacktrace to file.
If needed, checkout the logging customization notes here:
http://projects.spring.io/spring-boot/docs/spring-boot/README.html
I am writing a web application using spring 3 and hibernate 3.5 with JPA.
I am using the following code:
Buy buy = buyDAO.find(buyId);
buy.setNum(getNum());
buyDAO.merge(buy);
The find returns an entity (i see it in debug) but the buyDAO.merge throws the exception stated above.
I know that this exception is thrown only if i use query.getSingleResult() and there is no row in the DB for my query.
I am truly baffled by this, I tried every trick I know. I moved these lines to a different method, i tried using a transaction buy nothing works.