I came to a problem where I need to put ordering by multiplication of two columns of entity, for the sake of imagination entity is:
#Entity
public class Entity {
#Column(name="amount")
private BigDecimal amount;
#Column(name="unitPprice")
private BigDecimal unitPrice;
.
.
.
many more columns
}
My repo interface implements JpaRepository and QuerydslPredicateExecutor,
but I am struggling to find a way to order my data by "amount*unitPrice",
as I can't find a way to put it into
PageRequest (new Sort.Order(ASC, "amount * unitPrice"))
without having PropertyReferenceException: No property amount * unitPrice... thrown.
I can't user named query, as my query takes quite massive filter based on user inputs (can't put where clause into query, because if user hasn't selected any value, where clause can't just be in query).
To make it simple. I need something like findAll(Predicate, Pageable), but I need to force that query to order itself by "amount * unitPrice", but also have my Preditate (filter) and Pageable (offset, limit, other sortings) untouched.
Spring Sort can be used only for sorting by properties, not by expressions.
But you can create a unique sort in a Predicate, so you can add this sort-predicate to your other one before you call the findAll method.
Related
in our Spring application we have a table that contains a lot of "Payment" record. Now we need a query that pages the results sorted from the one with the largest total to the smallest, we are facing an error because sometimes the same record is contained in two successive pages.
We are creating a PageRequest passed to the repository. Here our implementation:
Repository:
public interface StagingPaymentEntityRepository extends JpaRepository<StagingPaymentEntity, Long> {
Page<StagingPaymentEntity> findAllByStatusAndCreatedDateLessThanEqualAndOperationTypeOrderByEffectivePaymentDesc(String status, Timestamp batchStartTimestamp, String operationType, Pageable pageable);
}
public class BatchThreadReiteroStorni extends ThreadAbstract<StagingPaymentEntity> {
PageRequest pageRequest = PageRequest.of (index, 170);
Page<StagingPaymentEntity> records = ((StagingPaymentEntityRepository) repository).findAllByStatusAndCreatedDateLessThanEqualAndOperationTypeOrderByEffectivePaymentDesc("REITERO", batchStartTimestamp, "STORNO", pageRequest) ;
}
where index is the index of the page we are requesting.
There is a way to understand why it is happening ? Thank for support
This can have multiple reasons.
Non deterministic ordering: If the ordering you are using isn't deterministic, i.e. there are rows that might com in any order that order might change between selects resulting in items getting skipped or returned multiple times. Fix: add the primary key as a last column to the ordering.
If you change the entities in a way that affects the ordering, or another process does that you might end up with items getting processed multiple times.
In this scenario I see a couple of approaches:
do value based pagination. I.e. don't select pages but select the next N rows after .
Instead of paging use a Stream this allows to use a single select but still processing the results an element at a time. You might have to flush and evict entities and I'm not 100% sure that works, but certainly worth a try.
Finally you can mark all all rows that you want to process in a separate column, then select N marked entities and unmark them once they are processed.
I have a domain object in Spring which I am saving using JpaRepository.save method and using Sequence generator from Postgres to generate id automatically.
#SequenceGenerator(initialValue = 1, name = "device_metric_gen", sequenceName = "device_metric_seq")
public class DeviceMetric extends BaseTimeModel {
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "device_metric_gen")
#Column(nullable = false, updatable = false)
private Long id;
///// extra fields
My use-case requires to do an upsert instead of normal save operation (which I am aware will update if the id is present). I want to update an existing row if a combination of three columns (assume a composite unique) is present or else create a new row.
This is something similar to this:
INSERT INTO customers (name, email)
VALUES
(
'Microsoft',
'hotline#microsoft.com'
)
ON CONFLICT (name)
DO
UPDATE
SET email = EXCLUDED.email || ';' || customers.email;
One way of achieving the same in Spring-data that I can think of is:
Write a custom save operation in the service layer that
Does a get for the three-column and if a row is present
Set the same id in current object and do a repository.save
If no row present, do a normal repository.save
Problem with the above approach is that every insert now does a select and then save which makes two database calls whereas the same can be achieved by postgres insert on conflict feature with just one db call.
Any pointers on how to implement this in Spring Data?
One way is to write a native query insert into values (all fields here). The object in question has around 25 fields so I am looking for an another better way to achieve the same.
As #JBNizet mentioned, you answered your own question by suggesting reading for the data and then updating if found and inserting otherwise. Here's how you could do it using spring data and Optional.
Define a findByField1AndField2AndField3 method on your DeviceMetricRepository.
public interface DeviceMetricRepository extends JpaRepository<DeviceMetric, UUID> {
Optional<DeviceMetric> findByField1AndField2AndField3(String field1, String field2, String field3);
}
Use the repository in a service method.
#RequiredArgsConstructor
public class DeviceMetricService {
private final DeviceMetricRepository repo;
DeviceMetric save(String email, String phoneNumber) {
DeviceMetric deviceMetric = repo.findByField1AndField2AndField3("field1", "field", "field3")
.orElse(new DeviceMetric()); // create new object in a way that makes sense for you
deviceMetric.setEmail(email);
deviceMetric.setPhoneNumber(phoneNumber);
return repo.save(deviceMetric);
}
}
A word of advice on observability:
You mentioned that this is a high throughput use case in your system. Regardless of the approach taken, consider instrumenting timers around this save. This way you can measure the initial performance against any tunings you make in an objective way. Look at this an experiment and be prepared to pivot to other solutions as needed. If you are always reading these three columns together, ensure they are indexed. With these things in place, you may find that reading to determine update/insert is acceptable.
I would recommend using a named query to fetch a row based on your candidate keys. If a row is present, update it, otherwise create a new row. Both of these operations can be done using the save method.
#NamedQuery(name="getCustomerByNameAndEmail", query="select a from Customers a where a.name = :name and a.email = :email");
You can also use the #UniqueColumns() annotation on the entity to make sure that these columns always maintain uniqueness when grouped together.
Optional<Customers> customer = customerRepo.getCustomersByNameAndEmail(name, email);
Implement the above method in your repository. All it will do it call the query and pass the name and email as parameters. Make sure to return an Optional.empty() if there is no row present.
Customers c;
if (customer.isPresent()) {
c = customer.get();
c.setEmail("newemail#gmail.com");
c.setPhone("9420420420");
customerRepo.save(c);
} else {
c = new Customer(0, "name", "email", "5451515478");
customerRepo.save(c);
}
Pass the ID as 0 and JPA will insert a new row with the ID generated according to the sequence generator.
Although I never recommend using a number as an ID, if possible use a randomly generated UUID for the primary key, it will qurantee uniqueness and avoid any unexpected behaviour that may come with sequence generators.
With spring JPA it's pretty simple to implement this with clean java code.
Using Spring Data JPA's method T getOne(ID id), you're not querying the DB itself but you are using a reference to the DB object (proxy). Therefore when updating/saving the entity you are performing a one time operation.
To be able to modify the object Spring provides the #Transactional annotation which is a method level annotation that declares that the method starts a transaction and closes it only when the method itself ends its runtime.
You'd have to:
Start a jpa transaction
get the Db reference through getOne
modify the DB reference
save it on the database
close the transaction
Not having much visibility of your actual code I'm gonna abstract it as much as possible:
#Transactional
public void saveOrUpdate(DeviceMetric metric) {
DeviceMetric deviceMetric = metricRepository.getOne(metric.getId());
//modify it
deviceMetric.setName("Hello World!");
metricRepository.save(metric);
}
The tricky part is to not think the getOne as a SELECT from the DB. The database never gets called until the 'save' method.
I have entity like:
#Id
#Column_name = "abc"
int pk;
#Column_name = "def"
int id;
And I have Repository as:
interface fetchDataRepository extends jpaRepository<className, int> {
#Query("Select S_Test.nextVal from dual");
Long generateId();
}
In above example S_Test is hardcoded sequence name.
But the problem is that I want to pass sequence name dynamically as follows:
Long generateId(#Param("sequenceName") String sequenceName)
and use inside #Query annotation as:
#Query("Select :sequenceName.nextVal from dual");
Is there anyway to do that? Any suggestion would be appreciated.
Edit: Isn't there possible to use #(#entityName). If yes, then please tell me how?
Unfortunately you can only substitute in things that you could do in JDBC anyway (so, pretty much just values in the INSERT and WHERE clauses). No dynamic table, column, schema names are supported.
There is one exception that may apply, and that is a limited subset of SpEL can be used. There is one variable available - #entityName. So, assuming that the #Entity annotation on your entity class is named identically to the sequence, you could use an #Query like so:
#Query("Select #{#entityName}.nextVal from dual");
Otherwise, since your query is simple and does not involve any object relational mapping, you would probably need to Create a custom repository implementation and inject a JdbcTemplate into it in order to run the query.
Else you could inject an EntityManager and try using the JPA Criteria API - but again you arent actualy trying to map a resultset to an entity so JdbcTemplate will be simpler.
I have in a Spring Repo something like this:
findTop10ItemsByCategIdInOrderByInsertDateDesc(List ids)
I want the first 10 items where category id in list of ids ordered by insert date.
Another similar query:
findTop10ItemsByDomainIdAndCategIdInOrderByInsertDateDesc(List ids, #Param Integer domainId)
Here I want that the domain id is equal to the given param and the categId to be in given list.
I managed to resolve it using #Query but I wonder if there is an one liner for the above queries.
thanks
EDIT
The top works fine. Initially I had findTop10ItemsByDomainIdAndCategIdOrderByInsertDateDesc. Now I want the results from a list of category ids. That's the new requirement.
SECOND EDIT
My query works for find the set o results where domain id is equal to a given param and categ id is contained in a given list. BUT I found out that HQL doesn't support a setMaxResult kind of thing as top or limit.
#Query("select i from Items i where i.domainId = :domainId and i.categId in :categoryIds order by i.insertDate desc")
The params for this method were (#Param("domainid") Integer domainid,List<Integer> categoryIds) but it seams that I'm alowed to use either #Param annotation to each parameter or no #Param at all ( except for Pageable return; not my case )
I still don't know how to achieve this think:
extract top n elements where field a eq to param, field b in set of param, ordered by another field.
ps: sorry for tags but there is no spring-crudrepository :)
The method to resolve your problem is:
List<MyClass> findTop10ByDomainIdAndCategIdInOrderByInsertDateDesc(Long domainId, List<Long> ids);
Top10 limits the results to first 10.
ByDomainId limits results to those that have passed domainId.
And adds another constraint.
CategIdIn limits results to those entries that have categId in the passed List.
OrderByInsertDateDesc orders results descending by insert date before limiting to TOP10.
I have tested this query on the following example:
List<User> findTop10ByEmailAndPropInOrderByIdDesc(String email, List<Long> props);
Where User is:
private Long id;
private String username;
private String password;
private String email;
private Long prop;
Currently I would recommend using LocalDate or LocalDateTime for storing dates using Spring Data JPA.
I've modified an existing RESTful/JDBC application i have to work with new features in Spring 4... specifically the JpaRepository. It will:
1) Retrieve a list of transactions for a specified date. This works fine
2) Retrieve a count of transactions by type for a specified date. This is not working as expected.
The queries are setup similarly, but the actual return types are very different.
I have POJOs for each query
My transactions JPA respository looks like:
public interface MyTransactionsRepository extends JpaRepository<MyTransactions, Long>
//My query works like a charm.
#Query( value = "SELECT * from ACTIVITI_TMP.BATCH_TABLE WHERE TO_CHAR(last_action, 'YYYY-MM-DD') = ?1", nativeQuery = true )
List< MyTransactions > findAllBy_ToChar_LastAction( String lastActionDateString );
This returns a list of MyTransactions objects as expected. Debugging, i see the returned object as ArrayList. Looking inside the elementData, I see that each object is, as expected, a MyTransactions object.
My second repository/query is where i'm having troubles.
public interface MyCountsRepository extends JpaRepository<MyCounts, Long>
#Query( value = "SELECT send_method, COUNT(*) AS counter FROM ACTIVITI_TMP.BATCH_TABLE WHERE TO_CHAR(last_action, 'YYYY-MM-DD') = ?1 GROUP BY send_method ORDER BY send_method", nativeQuery = true )
List<MyCounts> countBy_ToChar_LastAction( String lastActionDateString );
This DOES NOT return List as expected.
The object that holds the returned data was originally defined as List, but when I inspect this object in Eclipse, I see instead that it is holding an ArrayList. Drilling down to the elementData, each object is actually an Object[2]... NOT a MyCounts object.
I've modified the MyCountsRepository query as follows
ArrayList<Object[]> countBy_ToChar_LastAction( String lastActionDateString );
Then, inside my controller class, I create a MyCounts object for each element in List and then return List
This works, but... I don't understand why i have to go thru all this?
I can query a view as easily as a table.
Why doesn't JPA/Hibernate treat this as a simple 2 column table? send_method varchar(x) and count (int or long)
I know there are issues or nuances for how JPA treats queries with counts in them, but i've not seen anything like this referenced.
Many thanks for any help you can provide in clarifying this issue.
Anthony
That is the expected behaviour when you're doing a "group by". It will not map to a specific entity. Only way this might work is if you had a view in your database that summarized the data by send_method and you could map an entity to it.