Save LocalDateTime with namedParameterJdbcTemplate - spring

I have a list of 4 million generated entities that I want to move into table. The entity have field with type LocalDateTime:
#Data
#AllArgsConstructor
#Entity
#Table(name = "invoices")
public class Invoice {
#Id
#GeneratedValue(strategy=GenerationType.IDENTITY)
Long id;
#Column(name = "exact_iss_time")
private LocalDateTime exactIssueTime;
#Column(name = "final_iss_time")
private LocalDateTime finalIssueTime;
#Column(name = "issuer")
private String issuer;
#Column(name = "groupid")
private Integer groupID;
protected Invoice() {
}
}
As it is a big number of entities I want to do it optimally - which I gues is with NamedParameterJdbcTemplate.batchUpdate() like this:
public int[] bulkSaveInvoices(List<Invoice> invoices){
String insertSQL = "INSERT INTO invoices VALUES (:id, :exactIssueTime, :finalIssueTime, :issuer, :groupID)";
SqlParameterSource[] sqlParams = SqlParameterSourceUtils.createBatch(invoices.toArray());
int[] insertCounts = namedParameterJdbcTemplate.batchUpdate(insertSQL, sqlParams);
return insertCounts;
}
However I keep getting error:
org.springframework.jdbc.BadSqlGrammarException: PreparedStatementCallback; bad SQL grammar [INSERT INTO invoices VALUES (?, ?, ?, ?, ?)]; nested exception is org.postgresql.util.PSQLException: Can't infer the SQL type to use for an instance of java.time.LocalDateTime. Use setObject() with an explicit Types value to specify the type to use.
Am I doing this right - if so how to fix this LocalDateTime issue in this case?
If this is the wrong way than what is the most optimal way to INSERT the 4 million generated test entities into the table with Spring-boot?
database?

JPA 2.1 was released before Java 8 and therefore doesn’t support the new Date and Time API.
You can try add convertor, for more check How to persist LocalDate and LocalDateTime with JPA
#Converter(autoApply = true)
public class LocalDateTimeAttributeConverter implements AttributeConverter<LocalDateTime, Timestamp> {
#Override
public Timestamp convertToDatabaseColumn(LocalDateTime locDateTime) {
return (locDateTime == null ? null : Timestamp.valueOf(locDateTime));
}
#Override
public LocalDateTime convertToEntityAttribute(Timestamp sqlTimestamp) {
return (sqlTimestamp == null ? null : sqlTimestamp.toLocalDateTime());
}
}
Or you can try not jdbc approach, but use custom JPA batch

Related

Unable to persist an object in mariadb, Caused by: org.mariadb.jdbc.internal.util.dao.QueryException: You have an error in your SQL syntax;

I am trying to persist an object using the entity manager in a springboot application.
but I am getting the below exception
Caused by: java.sql.SQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'interval, last_updated) values ('5MIN', '2023-02-01 00:56:16')' at line 1
Query is : insert into cms_kpi_config (interval, last_updated) values ('5MIN', '2023-02-01 00:56:16.0')
at org.mariadb.jdbc.internal.util.ExceptionMapper.get(ExceptionMapper.java:125)
at org.mariadb.jdbc.internal.util.ExceptionMapper.throwException(ExceptionMapper.java:69)
at org.mariadb.jdbc.MariaDbStatement.executeQueryEpilog(MariaDbStatement.java:242)
at org.mariadb.jdbc.MariaDbClientPreparedStatement.executeInternal(MariaDbClientPreparedStatement.java:210)
at org.mariadb.jdbc.MariaDbClientPreparedStatement.executeUpdate(MariaDbClientPreparedStatement.java:186)
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeUpdate(NewProxyPreparedStatement.java:384)
at org.hibernate.engine.jdbc.internal.ResultSetReturnImpl.executeUpdate(ResultSetReturnImpl.java:204)
... 223 more
Caused by: org.mariadb.jdbc.internal.util.dao.QueryException: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'interval, last_updated) values ('5MIN', '2023-02-01 00:56:16')' at line 1
Query is : insert into cms_kpi_config (interval, last_updated) values ('5MIN', '2023-02-01 00:56:16.0')
at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.getResult(AbstractQueryProtocol.java:939)
at org.mariadb.jdbc.internal.protocol.AbstractQueryProtocol.executeQueries(AbstractQueryProtocol.java:775)
at org.mariadb.jdbc.MariaDbClientPreparedStatement.executeInternal(MariaDbClientPreparedStatement.java:201)
... 226 more
I don't seem to understand what the problem is.
here is my table definition
CREATE TABLE IF NOT EXISTS `cms_kpi_config` (
`id` INT(11) NOT NULL AUTO_INCREMENT,
`interval` enum('5MIN','30MIN','60MIN') NOT NULL DEFAULT '5MIN',
`last_updated` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8;
here is my Entity
#Entity
#Table(name = "cms_kpi_config")
#JsonInclude(JsonInclude.Include.NON_NULL)
public class CmsKPIConfig {
private Integer id;
private String interval;
private Date lastUpdated;
#Id
#Column(name = "id", nullable = false, unique = true)
#GeneratedValue(strategy = GenerationType.IDENTITY)
public Integer getId() {
return id;
}
public void setId(Integer id) {
this.id = id;
}
#Column(name = "interval", nullable = false)
#NotNull
public String getInterval() {
return interval;
}
public void setInterval(String interval) {
this.interval = interval;
}
#Column(name = "last_updated")
#JsonFormat(pattern = "yyyy-MM-dd HH:mm:ss")
public Date getLastUpdated() {
return lastUpdated;
}
public void setLastUpdated(Date lastUpdated) {
this.lastUpdated = lastUpdated;
}
}
In my Dao service class, I'm using entitymanager as below.
private EntityManager manager;
this.manager.persist(obj);
Please help

How does insert ignore for batches work in Spring Boot repository

I am working in Spring Boot framework.
I have a working way to run batches of "insert ignore" but i don't fully understand how/why it works.
I am using the Persistable interface to have inserts be done in batches.
In addition i'm using SQLInsert to run insert ignore instead of insert.
The code below demonstrates it:
#Entity
#SQLInsert(sql = "insert ignore into my_table (created_at, data, id) VALUES (?, ?, ?)")
public class MyTable implements Persistable<String> {
#Id
#Column
private String id;
#Column
private String data;
#Column
private Timestamp createdAt;
#Id
#Column
private String data;
public String getId() {
calculateId();
return id;
}
#Override
public boolean isNew() {
return true;
}
}
===============
#Repository
public interface MyTableRepository extends JpaRepository<MyTable, String> {
#Transactional
#Modifying(clearAutomatically = true, flushAutomatically = true)
<S extends MyTable> List<S> saveAll(Iterable<S> entities);
}
In debug, and on the DB side, i see that the save is indeed done in batches.
The query shown in debug mode is of this sort:
["insert ignore into my_table (created_at, data, id) VALUES (?, ?, ?)"], Params:[(2022-06-08 17:44:35.041,data1,id1),(2022-06-08 17:44:35.042,data2,id2),(2022-06-08 17:44:35.042,data3,id3)]
I see that there are 3 new records created in the DB.
The question how does it work if the number of "?" is smaller than the numbers of parameters passed?
If you would try it directly on the DB you would get an error

Sequence generator in h2 does not provide unique values

For H2 database in my spring boot api test there is an insertion script that looks something like:
INSERT INTO STORES_TEMPLATE (ID, COUNTRY, TEMPLATE_NAME_1, STORES_1) VALUES(STORES_TEMPLATE_ID_SEQ.NEXTVAL,'country', 'template_name', 'stores');
INSERT INTO STORES_TEMPLATE (ID, COUNTRY, TEMPLATE_NAME_2, STORES_2) VALUES(STORES_TEMPLATE_ID_SEQ.NEXTVAL,'country', 'template_name', 'stores');
INSERT INTO STORES_TEMPLATE (ID, COUNTRY, TEMPLATE_NAME_3, STORES_3) VALUES(STORES_TEMPLATE_ID_SEQ.NEXTVAL,'country', 'template_name', 'stores');
The entity:
#Entity
#Getter
#Setter
#NoArgsConstructor
#Table(name = "STORES_TEMPLATE")
public class StoresTemplate {
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "STORES_TEMPLATE_ID_SEQ")
#SequenceGenerator(name = "STORES_TEMPLATE_ID_SEQ", sequenceName = "STORES_TEMPLATE_ID_SEQ", allocationSize = 1)
private Long id;
#Enumerated(EnumType.STRING)
private CountryEnum country;
private String templateName;
#Lob
private String stores;
public void setStores(List<String> stores) {
this.stores = String.join(",", stores);
}
#JsonIgnore
public List<String> getStoresAsList() {
return Arrays.stream(stores.split(","))
.distinct()
.collect(Collectors.toList());
}
}
On production oracle database everything works just fine, but on my test environment with h2 database when I try to save new entity the sequence generator gives me ids that are duplicated with the ids of the predefined by the sql migration data. What can I do to fix this issue?
What did the trick was to exclude the insertion script for the tests, but is there another possible solution?

Spring Data lock table to read while writing

I am using Spring data in my app (2.0.1). The app is REST-based service which add orders.
The order entity looks as follows:
#Entity
#Table(name = "orders")
#Data
#NoArgsConstructor
public class OrderEntity {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private long id;
#Version
private int version;
private Date date;
private String post;
private BigDecimal totalAmount;
#Enumerated(EnumType.STRING)
private OrderStatus status;
#OneToMany(fetch = FetchType.EAGER, cascade = CascadeType.ALL)
#JoinColumn(name = "order_id")
List<OrderSetEntity> sets;
private int dailyNumber;
private String documentNumber;
The posting or orders happens in OrderService:
public Long postOrder(OrderDTO orderDTO){
Date date = new Date();
OrderEntity order = new OrderEntity(date, orderDTO.getPost(), orderDTO.getPaymentMethod());
Integer dailyNumber = orderRepository.findDailyNumberByDate(date) + 1;
order.setDailyNumber(dailyNumber);
orderRepository.save(order);
return order.getId();
}
while findDailyNumberByDate is implemented this way:
#Override
public int findDailyNumberByDate(Date date) {
String sql = String.format("select max(o.dailyNumber) from OrderEntity o ");
Query query = entityManager.createQuery(sql);
Integer result = (Integer) query.getSingleResult();
if (result == null){
return 0;
}else {
return result.intValue();
}
Now I have a problem, that it get duplicated dailyNumber. The table isn't locked for reading when I am about to write into it.
How can I achieve it?
I tried lockin the table - #Lock(LockModeType.PESSIMISTIC_WRITE)
or query.setLockMode(LockModeType.PESSIMISTIC_FORCE_INCREMENT);
but is still isn't working.
Thanks a lot for help
LockModeType can work in different ways with different databases, especially with Oracle db it gets a little tricky. A similar issue was answered here jpa lockmode type npt working as expected

EntityNotFoundException in Hibernate Many To One mapping however data exist

I'm getting an error
Caused by: javax.persistence.EntityNotFoundException: Unable to find tn.entities.AgenceBnq with id 01
when I get AgenceBnq through Employee
Employee class:
#Table(name = "EMPLOYEE")
#NamedQuery(name = "Employee.findById", query = "SELECT e FROM Employee e WHERE e.employeMat = ?1"),
public class Employee implements Serializable {
private static final long serialVersionUID = 1L;
#Id
#Column(name = "EMPLOYEE_MAT", unique = true, nullable = false, length = 15)
private String employeeMat;
...
#ManyToOne
#JoinColumn(name = "AGENCE_COD")
private AgenceBnq agenceBnq;
}
#Entity
#Table(name="AGENCEBNQ")
public class AgenceBnq implements Serializable {
private static final long serialVersionUID = 1L;
#Id
#Column(name="AGENCE_COD", unique=true, nullable=false, length=10)
private String agenceCod;
...
//bi-directional many-to-one association to Employee
#OneToMany(mappedBy="agenceBnq")
private Set<Employee> employees;
}
I'm calling namedQuery Employee.findById in DAO to retrieve data and I have to get AgenceBnq from Employee but get this error while calling query.getResultList()
#NotFound( action = NotFoundAction.IGNORE) isn't useful for me because data exist in AGENCEBNQ table and I have to retrieve date through Employee.
Is this a bug in hibernate ? I'm using hibernate version 3.6.7.Final
Firstly, You dont need query for it, the EnityManger.find(Employee.class, YOUR_ID) will do the job.
Secondly dont use ? in your queries but names (e.employeMat = :id) as it is easier to debug and less error prones for complicated queries.
Finally, check your DB table if the AGENCE_COD column in Employee table really contains the valid ID for your entitity that crashes (and that it length matches the ID length of AgenceBnq). It should work, the typical reason why it doesnt will be that your Employe.AGENCE_COD has defualt value and when creatubg the new EMploye you add it only to the Agence but you did not set Agence in the Employ.

Resources