I am trying to get a sum with a one to many relationship, illustrated by the following relationship (only parent shown):
#Entity
#Table(name = "Parent")
public class Parent implements Serializable {
private static final long serialVersionUID = -7348332185233715983L;
#Id
#Basic(optional = false)
#Column(name = "PARENT_ID")
private Long parentId;
#OneToMany
#JoinColumn(name="CHILDREN", referencedColumnName="PARENT_ID")
private List<Child> children;
#Formula("(select sum(select height from children))")
private BicDecimal totalHeight
}
It is pretty straight forward with no restrictions and even with static restrictions. I am having trouble when the children list is restricted dynamically though.
In my case, I am using spring data and jpa. I am using specifications to restrict the children and am getting the appropriate list of children, but obviously the sum is still for unrestricted children because there is no where clause in the #Formula tag.
I do not want to iterate over the list in java for performance reasons and because the results are paginated. Also, the sum is not of the paginated results, but of all results.
I am new to Spring Data/JPA. Historically, I could build this query dynamically or use hibernate criteria. I am OK running a completley separate query to make this calculation. it is not required that I use the #Formula annotation as there is only 1 aggregation per call. In a hibernate framework, I could just state the select clause as "sum(field)" and build the criteria. In the Spring Data/JPA framework, I can build the specifications fine which covers the criteria, but I have no idea how to manipulate the select part of the query since it seems tied so tightly to the entity.
Using the #Query annotation on the repository works as its own query if I know which fields I need to restrict on, but often the fields are null and need to be ignored for the query. There are 8 possible fields, leaving me with 256 possible combinations (2^8). That is too many methods for this in the repository.
Any ideas outside of switching frameworks?
Posting an answer to this old question since I had a somewhat similar problem recently.
I decided to go with a custom repository with a method that does the aggregation based on any Specification passed into it. Specifications can be combined to compose dynamic criteria (see org.springframework.data.jpa.domain.Specifications)
So my repository to above Child height problem would look like below:
package something
import org.springframework.data.jpa.domain.Specification;
import org.springframework.stereotype.Repository;
import javax.persistence.EntityManager;
import javax.persistence.PersistenceContext;
import javax.persistence.criteria.CriteriaBuilder;
import javax.persistence.criteria.CriteriaQuery;
import javax.persistence.criteria.Root;
#Repository
public class ChildHeightRepository {
#PersistenceContext
EntityManager entityManager;
public Long getTotalHeight(Specification<Child> spec) {
CriteriaBuilder cb = entityManager.getCriteriaBuilder();
CriteriaQuery query = cb.createQuery(Long.class);
Root root = query.from(Child.class);
return ((Long) entityManager.createQuery(
query.where(spec.toPredicate(root, query, cb))
.select(cb.sum(root.get("height")))).getSingleResult());
}
}
Have you tried in JPQL
select sum(d.height) from Parent a join a.children d
If you dont want to ignore nulls
select sum(d.height) from Parent a left join a.children d
I think other question you have is how to filter depending on the properties . I mean if you need to have a where statement with several combinations.
Why you don't try to use a List and adding to the list all the predicates you want to apply depending on the combinations you want to have. Example
Create query
CriteriaBuilder cb = em.getCriteriaBuilder();
CriteriaQuery cq = cb.createQuery(Double.class);
Root<Parent> root = cq.from(Parent.class)
Join<Parent, Child> join = cq.join(("children"));
cq.select(cb.<Double>sum(join.get("height")));
Create the list of predicates
List<Predicates> listOfpredicates = new ArrayList<Predicates>();
if(property1 != null && !"".equals(property1){
PatameterExpression<String> p = cb.parameter(String.class, "valueOfproperty1")
listOfpredicates.add(cb.equal(join.get("property1"),p);
}
Then add to the CriteriaQuery
if(listOfPredicates.size() == 1)
cq.where(listOfPredicates.get(0))
else
cq.where(cb.and(listOfPredicates.toArray(new Predicate[0])));
Finally execute the query.
TypedQuery<Double> q = em.createQuery(cq);
q.getResultList();
This will create dynamically your query with any combination.
6 years late but it still took me a while to get it to work, here is how I would do it for your mapping:
#Formula("(select sum(children.height) from children_table children inner join Parent p on children.parent_id=p.id where children.parent_id=parent_id)")
private BicDecimal totalHeight
Stuff you need to take care of:
add () to the beginning and end of your formula otherwise the syntax of the sql wont be translated correctly.
the formula query is a native SQL query to my understanding and not a JPQL some one might want to correct me on this?.
properties are those of the tables and not what you name your properties in Java so the column names and table names have to actually be the table names.
Related
I have a domain object in Spring which I am saving using JpaRepository.save method and using Sequence generator from Postgres to generate id automatically.
#SequenceGenerator(initialValue = 1, name = "device_metric_gen", sequenceName = "device_metric_seq")
public class DeviceMetric extends BaseTimeModel {
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "device_metric_gen")
#Column(nullable = false, updatable = false)
private Long id;
///// extra fields
My use-case requires to do an upsert instead of normal save operation (which I am aware will update if the id is present). I want to update an existing row if a combination of three columns (assume a composite unique) is present or else create a new row.
This is something similar to this:
INSERT INTO customers (name, email)
VALUES
(
'Microsoft',
'hotline#microsoft.com'
)
ON CONFLICT (name)
DO
UPDATE
SET email = EXCLUDED.email || ';' || customers.email;
One way of achieving the same in Spring-data that I can think of is:
Write a custom save operation in the service layer that
Does a get for the three-column and if a row is present
Set the same id in current object and do a repository.save
If no row present, do a normal repository.save
Problem with the above approach is that every insert now does a select and then save which makes two database calls whereas the same can be achieved by postgres insert on conflict feature with just one db call.
Any pointers on how to implement this in Spring Data?
One way is to write a native query insert into values (all fields here). The object in question has around 25 fields so I am looking for an another better way to achieve the same.
As #JBNizet mentioned, you answered your own question by suggesting reading for the data and then updating if found and inserting otherwise. Here's how you could do it using spring data and Optional.
Define a findByField1AndField2AndField3 method on your DeviceMetricRepository.
public interface DeviceMetricRepository extends JpaRepository<DeviceMetric, UUID> {
Optional<DeviceMetric> findByField1AndField2AndField3(String field1, String field2, String field3);
}
Use the repository in a service method.
#RequiredArgsConstructor
public class DeviceMetricService {
private final DeviceMetricRepository repo;
DeviceMetric save(String email, String phoneNumber) {
DeviceMetric deviceMetric = repo.findByField1AndField2AndField3("field1", "field", "field3")
.orElse(new DeviceMetric()); // create new object in a way that makes sense for you
deviceMetric.setEmail(email);
deviceMetric.setPhoneNumber(phoneNumber);
return repo.save(deviceMetric);
}
}
A word of advice on observability:
You mentioned that this is a high throughput use case in your system. Regardless of the approach taken, consider instrumenting timers around this save. This way you can measure the initial performance against any tunings you make in an objective way. Look at this an experiment and be prepared to pivot to other solutions as needed. If you are always reading these three columns together, ensure they are indexed. With these things in place, you may find that reading to determine update/insert is acceptable.
I would recommend using a named query to fetch a row based on your candidate keys. If a row is present, update it, otherwise create a new row. Both of these operations can be done using the save method.
#NamedQuery(name="getCustomerByNameAndEmail", query="select a from Customers a where a.name = :name and a.email = :email");
You can also use the #UniqueColumns() annotation on the entity to make sure that these columns always maintain uniqueness when grouped together.
Optional<Customers> customer = customerRepo.getCustomersByNameAndEmail(name, email);
Implement the above method in your repository. All it will do it call the query and pass the name and email as parameters. Make sure to return an Optional.empty() if there is no row present.
Customers c;
if (customer.isPresent()) {
c = customer.get();
c.setEmail("newemail#gmail.com");
c.setPhone("9420420420");
customerRepo.save(c);
} else {
c = new Customer(0, "name", "email", "5451515478");
customerRepo.save(c);
}
Pass the ID as 0 and JPA will insert a new row with the ID generated according to the sequence generator.
Although I never recommend using a number as an ID, if possible use a randomly generated UUID for the primary key, it will qurantee uniqueness and avoid any unexpected behaviour that may come with sequence generators.
With spring JPA it's pretty simple to implement this with clean java code.
Using Spring Data JPA's method T getOne(ID id), you're not querying the DB itself but you are using a reference to the DB object (proxy). Therefore when updating/saving the entity you are performing a one time operation.
To be able to modify the object Spring provides the #Transactional annotation which is a method level annotation that declares that the method starts a transaction and closes it only when the method itself ends its runtime.
You'd have to:
Start a jpa transaction
get the Db reference through getOne
modify the DB reference
save it on the database
close the transaction
Not having much visibility of your actual code I'm gonna abstract it as much as possible:
#Transactional
public void saveOrUpdate(DeviceMetric metric) {
DeviceMetric deviceMetric = metricRepository.getOne(metric.getId());
//modify it
deviceMetric.setName("Hello World!");
metricRepository.save(metric);
}
The tricky part is to not think the getOne as a SELECT from the DB. The database never gets called until the 'save' method.
I have entity like:
#Id
#Column_name = "abc"
int pk;
#Column_name = "def"
int id;
And I have Repository as:
interface fetchDataRepository extends jpaRepository<className, int> {
#Query("Select S_Test.nextVal from dual");
Long generateId();
}
In above example S_Test is hardcoded sequence name.
But the problem is that I want to pass sequence name dynamically as follows:
Long generateId(#Param("sequenceName") String sequenceName)
and use inside #Query annotation as:
#Query("Select :sequenceName.nextVal from dual");
Is there anyway to do that? Any suggestion would be appreciated.
Edit: Isn't there possible to use #(#entityName). If yes, then please tell me how?
Unfortunately you can only substitute in things that you could do in JDBC anyway (so, pretty much just values in the INSERT and WHERE clauses). No dynamic table, column, schema names are supported.
There is one exception that may apply, and that is a limited subset of SpEL can be used. There is one variable available - #entityName. So, assuming that the #Entity annotation on your entity class is named identically to the sequence, you could use an #Query like so:
#Query("Select #{#entityName}.nextVal from dual");
Otherwise, since your query is simple and does not involve any object relational mapping, you would probably need to Create a custom repository implementation and inject a JdbcTemplate into it in order to run the query.
Else you could inject an EntityManager and try using the JPA Criteria API - but again you arent actualy trying to map a resultset to an entity so JdbcTemplate will be simpler.
I came to a problem where I need to put ordering by multiplication of two columns of entity, for the sake of imagination entity is:
#Entity
public class Entity {
#Column(name="amount")
private BigDecimal amount;
#Column(name="unitPprice")
private BigDecimal unitPrice;
.
.
.
many more columns
}
My repo interface implements JpaRepository and QuerydslPredicateExecutor,
but I am struggling to find a way to order my data by "amount*unitPrice",
as I can't find a way to put it into
PageRequest (new Sort.Order(ASC, "amount * unitPrice"))
without having PropertyReferenceException: No property amount * unitPrice... thrown.
I can't user named query, as my query takes quite massive filter based on user inputs (can't put where clause into query, because if user hasn't selected any value, where clause can't just be in query).
To make it simple. I need something like findAll(Predicate, Pageable), but I need to force that query to order itself by "amount * unitPrice", but also have my Preditate (filter) and Pageable (offset, limit, other sortings) untouched.
Spring Sort can be used only for sorting by properties, not by expressions.
But you can create a unique sort in a Predicate, so you can add this sort-predicate to your other one before you call the findAll method.
I have a complex query(using multiple joins and subqueries) written in HQL which I have used in a Repository class. Similar to one below -
#Repository
public interface DataRepository extends PagingAndSortingRepository<Data,String> {
public List<Data> findByService(#Param("service")Service service, Pageable page);
#Query("SELECT DISTINCT d from Data d "
+" WHERE (d.working in (SELECT d1 from Data d1 "
+" JOIN d1.working d1w "
+" JOIN d1.service s WITH (s in (:serviceList)))"
+" OR d.cleared IS NOT NULL) AND [..several other CRITERIA]")
public Page<Data> findForServices(#Param("serviceList")Set<Service> serviceList, Pageable page);
....
Now I need to add criteria to it dynamically. These criteria are flexible in number which is holding me from including it into the HQL straightaway. Is it anyhow possible?
Sifting through the internet I have come across solutions for dynamic query. But, I guess they would be working only for cases where I do not have a custom query i.e.- no #Query at the query in the repository.
There was another interesting question I found. But that also suits for a case where you have a single table to query.
I do not want to be switching over to raw SQL queries. How do I solve this?
The mentioned Criteria API with specifications and predicates is a little bit difficult to get used to but it is a good way to handle dynamic conditions.
I don't think it is possible to mix the annotation based query with programmatic query creation.
I've modified an existing RESTful/JDBC application i have to work with new features in Spring 4... specifically the JpaRepository. It will:
1) Retrieve a list of transactions for a specified date. This works fine
2) Retrieve a count of transactions by type for a specified date. This is not working as expected.
The queries are setup similarly, but the actual return types are very different.
I have POJOs for each query
My transactions JPA respository looks like:
public interface MyTransactionsRepository extends JpaRepository<MyTransactions, Long>
//My query works like a charm.
#Query( value = "SELECT * from ACTIVITI_TMP.BATCH_TABLE WHERE TO_CHAR(last_action, 'YYYY-MM-DD') = ?1", nativeQuery = true )
List< MyTransactions > findAllBy_ToChar_LastAction( String lastActionDateString );
This returns a list of MyTransactions objects as expected. Debugging, i see the returned object as ArrayList. Looking inside the elementData, I see that each object is, as expected, a MyTransactions object.
My second repository/query is where i'm having troubles.
public interface MyCountsRepository extends JpaRepository<MyCounts, Long>
#Query( value = "SELECT send_method, COUNT(*) AS counter FROM ACTIVITI_TMP.BATCH_TABLE WHERE TO_CHAR(last_action, 'YYYY-MM-DD') = ?1 GROUP BY send_method ORDER BY send_method", nativeQuery = true )
List<MyCounts> countBy_ToChar_LastAction( String lastActionDateString );
This DOES NOT return List as expected.
The object that holds the returned data was originally defined as List, but when I inspect this object in Eclipse, I see instead that it is holding an ArrayList. Drilling down to the elementData, each object is actually an Object[2]... NOT a MyCounts object.
I've modified the MyCountsRepository query as follows
ArrayList<Object[]> countBy_ToChar_LastAction( String lastActionDateString );
Then, inside my controller class, I create a MyCounts object for each element in List and then return List
This works, but... I don't understand why i have to go thru all this?
I can query a view as easily as a table.
Why doesn't JPA/Hibernate treat this as a simple 2 column table? send_method varchar(x) and count (int or long)
I know there are issues or nuances for how JPA treats queries with counts in them, but i've not seen anything like this referenced.
Many thanks for any help you can provide in clarifying this issue.
Anthony
That is the expected behaviour when you're doing a "group by". It will not map to a specific entity. Only way this might work is if you had a view in your database that summarized the data by send_method and you could map an entity to it.