How to get spring neo4j cypher custom query to populate an array of child relationships - spring-boot

Built-in queries to Spring Data Neo4j (SDN) return objects populated with depth 1 by default. This means that "children" (related nodes) of an object returned by a query are populated. That's good - there are actual objects on the end of references from objects returned by these queries.
Custom queries are depth 0 by default. This is a hassle.
In this answer, it is described how to get springboot neo4j to populate a related element to the target of a custom query - to achieve an extra one level of depth of results from the query.
I am having trouble with this method when the related elements are in a list:
#NodeEntity
public class BoardPosition {
#Relationship(type="PARENT", direction = Relationship.INCOMING)
public List<BoardPosition> children;
I have a query returning a target BoardPosition and I need it's children to be populated.
#Query("MATCH (target:BoardPosition) <-[c:PARENT]- (child:BoardPosition)
WHERE target.play={Play}
RETURN target, c, child")
BoardPosition findActiveByPlay(#Param("Play") String play);
The problem is that the query appears to return one separate result for each child, and those results aren't being used to populate the array of children in the target.
Instead of Spring Neo collating the children into the array on the target, I get "only 1 result expected" error - as if the query is returning multiple results each with one child, rather than one result with the children in it.
org.springframework.dao.IncorrectResultSizeDataAccessException:
Incorrect result size: expected at most 1
How can I have a custom query to populate that target's children list?
(Note that the built-in findByPlay(play) does what I want - the built-in queries have a depth of 1 rather than 0, and it returns a target with populated children - but of course I need to make the query a bit more sophisticated than just "by Play"... that's why I need to solve this)
Versions:
org.springframework.data:spring-data-neo4j:5.1.3.RELEASE
neo4j 3.5.0

=== Edit ======
Your problem arises because you have self-relationship (relationship between nodes of the same label)
This is how Spring treat your query for single node:
org.springframework.data.neo4j.repository.query.GraphQueryExecution
#Override
public Object execute(Query query, Class<?> type) {
Iterable<?> result;
....
Object ret = iterator.next();
if (iterator.hasNext()) {
throw new IncorrectResultSizeDataAccessException("Incorrect result size: expected at most 1", 1);
}
return ret;
}
Spring passes your node class type Class<?> type to neo4j-ogm and have your data read back.
You know, neo4j server will returns multiple rows for your query, one for each matching path:
A <- PARENT - B
A <- PARENT - C
A <- PARENT - D
If your nodes are of different labels, i.e. of different class type then the ogm only return single node correspond to your query return type, no problem.
But your nodes are of the same labels, i.e. same class type => Neo4j OGM cannot distinguish which is the returned node -> All nodes A, B, C, D returned -> Exception
Regard this issue, I think you should file a bug report now.
For workaround, you can can change the query to return only the distinct target.your_identity_property (identity_property is 'primary key' of the node, which uniquely identify your node)
Then have your application call load with the that identity property:
public interface BoardRepository extends CrudRepository<BoardPos, Long> {
#Query("MATCH (target:B) <-[c:PARENT]- (child:B) WHERE target.play={Play} RETURN DISTINCT target.your_identity_property")
Long findActiveByPlay(#Param("Play") String play);
BoardPos findByYourIdentityProperty(xxxx);
}
=== OLD ======
Spring docs says that (highlighted by me):
Custom queries do not support a custom depth. Additionally, #Query does not support mapping a path to domain entities, as such, a path should not be returned from a Cypher query. Instead, return nodes and relationships to have them mapped to domain entities.
So clearly your use-case (populate children nodes by custom query) is supported. Spring framework already maps the results into a single node. (Indeed, my setup on local turnouts that the operation is working properly)
So your exception may be caused by several issues:
You have more than one target:BoardPosition with target.play={play}. So the exception refers to more than one target:BoardPosition instead of one BoardPosition with multiple child result
You have incorrect entity mapping. Do you have your mapping field annotated with #Relationship with correct direction attribute? You might post your entity here.
Here is my local setup:
#NodeEntity(label = "C")
#Data
public class Child {
#Id
#GeneratedValue
private long id;
private String name;
#Relationship(type = "PARENT", direction = "INCOMING")
private List<Parent> parents;
}
public interface ChildRepository extends CrudRepository<Child, Long> {
#Query("MATCH (target:C) <-[p:PARENT]- (child:P) "
+ "WHERE target.name={name} "
+ "RETURN target, p, child")
Child findByName(#Param("name") String name);
}
(:C) <-[:PARENT] - (:P)

Consider the alternative query
MATCH (target:BoardPosition {play:{Play}})
RETURN target, [ (target)<-[c:PARENT]-(child:BoardPosition) | [c, child] ]
which is using list comprehension to return not only the target but also its relations and related nodes of label BoardPosition within one result row. This ensures that the result will be a single row (as long as your attribute play is unique).
I didn't try it with your example but in my application this approach is working fine. Neo4j OGM hydrates the objects as expected. It is important to include the related nodes as well as the relations pointing to the nodes.
If you enable neo4j OGM logs, you can see that the build-in queries with depth 1 use the same approach.

Related

Spring Data / Hibernate save entity with Postgres using Insert on Conflict Update Some fields

I have a domain object in Spring which I am saving using JpaRepository.save method and using Sequence generator from Postgres to generate id automatically.
#SequenceGenerator(initialValue = 1, name = "device_metric_gen", sequenceName = "device_metric_seq")
public class DeviceMetric extends BaseTimeModel {
#Id
#GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "device_metric_gen")
#Column(nullable = false, updatable = false)
private Long id;
///// extra fields
My use-case requires to do an upsert instead of normal save operation (which I am aware will update if the id is present). I want to update an existing row if a combination of three columns (assume a composite unique) is present or else create a new row.
This is something similar to this:
INSERT INTO customers (name, email)
VALUES
(
'Microsoft',
'hotline#microsoft.com'
)
ON CONFLICT (name)
DO
UPDATE
SET email = EXCLUDED.email || ';' || customers.email;
One way of achieving the same in Spring-data that I can think of is:
Write a custom save operation in the service layer that
Does a get for the three-column and if a row is present
Set the same id in current object and do a repository.save
If no row present, do a normal repository.save
Problem with the above approach is that every insert now does a select and then save which makes two database calls whereas the same can be achieved by postgres insert on conflict feature with just one db call.
Any pointers on how to implement this in Spring Data?
One way is to write a native query insert into values (all fields here). The object in question has around 25 fields so I am looking for an another better way to achieve the same.
As #JBNizet mentioned, you answered your own question by suggesting reading for the data and then updating if found and inserting otherwise. Here's how you could do it using spring data and Optional.
Define a findByField1AndField2AndField3 method on your DeviceMetricRepository.
public interface DeviceMetricRepository extends JpaRepository<DeviceMetric, UUID> {
Optional<DeviceMetric> findByField1AndField2AndField3(String field1, String field2, String field3);
}
Use the repository in a service method.
#RequiredArgsConstructor
public class DeviceMetricService {
private final DeviceMetricRepository repo;
DeviceMetric save(String email, String phoneNumber) {
DeviceMetric deviceMetric = repo.findByField1AndField2AndField3("field1", "field", "field3")
.orElse(new DeviceMetric()); // create new object in a way that makes sense for you
deviceMetric.setEmail(email);
deviceMetric.setPhoneNumber(phoneNumber);
return repo.save(deviceMetric);
}
}
A word of advice on observability:
You mentioned that this is a high throughput use case in your system. Regardless of the approach taken, consider instrumenting timers around this save. This way you can measure the initial performance against any tunings you make in an objective way. Look at this an experiment and be prepared to pivot to other solutions as needed. If you are always reading these three columns together, ensure they are indexed. With these things in place, you may find that reading to determine update/insert is acceptable.
I would recommend using a named query to fetch a row based on your candidate keys. If a row is present, update it, otherwise create a new row. Both of these operations can be done using the save method.
#NamedQuery(name="getCustomerByNameAndEmail", query="select a from Customers a where a.name = :name and a.email = :email");
You can also use the #UniqueColumns() annotation on the entity to make sure that these columns always maintain uniqueness when grouped together.
Optional<Customers> customer = customerRepo.getCustomersByNameAndEmail(name, email);
Implement the above method in your repository. All it will do it call the query and pass the name and email as parameters. Make sure to return an Optional.empty() if there is no row present.
Customers c;
if (customer.isPresent()) {
c = customer.get();
c.setEmail("newemail#gmail.com");
c.setPhone("9420420420");
customerRepo.save(c);
} else {
c = new Customer(0, "name", "email", "5451515478");
customerRepo.save(c);
}
Pass the ID as 0 and JPA will insert a new row with the ID generated according to the sequence generator.
Although I never recommend using a number as an ID, if possible use a randomly generated UUID for the primary key, it will qurantee uniqueness and avoid any unexpected behaviour that may come with sequence generators.
With spring JPA it's pretty simple to implement this with clean java code.
Using Spring Data JPA's method T getOne(ID id), you're not querying the DB itself but you are using a reference to the DB object (proxy). Therefore when updating/saving the entity you are performing a one time operation.
To be able to modify the object Spring provides the #Transactional annotation which is a method level annotation that declares that the method starts a transaction and closes it only when the method itself ends its runtime.
You'd have to:
Start a jpa transaction
get the Db reference through getOne
modify the DB reference
save it on the database
close the transaction
Not having much visibility of your actual code I'm gonna abstract it as much as possible:
#Transactional
public void saveOrUpdate(DeviceMetric metric) {
DeviceMetric deviceMetric = metricRepository.getOne(metric.getId());
//modify it
deviceMetric.setName("Hello World!");
metricRepository.save(metric);
}
The tricky part is to not think the getOne as a SELECT from the DB. The database never gets called until the 'save' method.

Retrieve id of an entity based on which the stream is being mapped

I'm trying to construct a stream pipeline which is not really straightforward and it gets me puzzled.
The idea is that I have a class containing a set of entities I want to traverse. Application class defines a field Set<Document> documents. Inside of these docs I have a field DocumentFile documentFile. I filter the stream based on the name of this document file, but the result that I need is the id of Document entity.
So the method goes like this:
private long retrieveSmth(String docName, long applicationId) {
final Application application = this.applicationDao.get(applicationId);
final long docId = application.getDocuments()
.stream()
.map(Document::getDocumentFile)
.filter(doc -> doc.getDocumentFileName().equals(docName))
...
}
At this point I get stuck questioning myself how do I get the control back to Document level and retrieve the id of the document whose document file satisfies the condition. Is there a way to do this using Stream API?
If you map() to DocumentFile you cannot "go back" to the owner object of DocumentFile : the Stream<Document> was transformed into a Stream<DocumentFile>.
You should so not map and specify the object to test from Stream<Document>.filter() :
final long docId = application.getDocuments()
.stream()
.filter(doc -> doc.getDocumentFile().getDocumentFileName().equals(docName))
.map(Document::getId); // now it is possible
Note that you should avoid talking to strangers and this makes this bad smell :
doc.getDocumentFile().getDocumentFileName()
So it would be interesting to introduce a method matchesName() in Document that does the delegation and the equality test :
public boolean matchesName(String name){
return name.equals(getDocumentFile().getDocumentFileName());
}
In this way it sounds better :
.filter(doc -> doc.matchesName(docName))

Upgrade of spring data neo4j 3.x to 4.x Relationship Operations

In Spring data neo4j 3.x To create relation ship between two nodes and relationship contains set of properties earlier used to achieve this by apis
create :
n4jOperations.createRelationshipBetween(Object start, Object end, Class<R> relationshipEntityClass, String relationshipType, boolean allowDuplicates);
delete:
n4jOperations.deleteRelationshipBetween(Object start, Object end, String type);
get:
n4jOperations.getRelationshipBetween( from, to, relationshipClass, relationshipType );
But after migration i didnt't find above apis
as per docs says
#NodeEntity
public class Student {
private String name;
#Relationship(type = "ENROLLED")
private Set<Enrollment> enrollments;
}
By repo.save(Student);
//Relation creation was possible but new api's how can i achieve below use cases
1.How can avoid duplicate relation creation?
2.get Relation ship between two nodes ?
2.delete relation ship between two nodes ?
SDN 4 does not provide low-level graph-operations like setting nodes and relationships directly.
Relationships in the graph are modelled and manipulated using object references in your domain classes. They come in two flavours: implicit and explicit. Implicit relationships are described by simple references between two node entities, e.g. Customer and Address:
class Customer {
#Relationship(type="LIVES_AT")
Address address; // implied (:Customer)-[:LIVES_AT]->(:Address)
...
}
Explicit relationships are modelled using RelationshipEntity objects, and are allowed to have properties (but don't have to). They are still accessed as references in your domain model.
class Person {
#Relationship(type="RATED")
List<Rating> ratings
}
class Movie {
}
#RelationshipEntity(type="RATED")
class Rating {
#StartNode Person person;
#EndNode Movie movie;
int stars;
}
Note: If you don't need properties on a particular relationship, you don't need to use a RelationshipEntity.
To answer your specific questions:
1) SDN 4.0 doesn't create duplicate relationships. No matter how many times you persist a specific object reference, it will represented by only one relationship in the graph.
2) Hopefully that is clear now!
3) Setting an object reference to null and saving the parent object will remove the relationship. Or, if the reference is part of Collection, remove it from the collection. You must ensure that the object references are removed from both sides. For example if A holds a reference to B and B holds a reference to A, you must remove A's reference to B as well as B's reference to A.

Sum with one to many Spring Data JPA

I am trying to get a sum with a one to many relationship, illustrated by the following relationship (only parent shown):
#Entity
#Table(name = "Parent")
public class Parent implements Serializable {
private static final long serialVersionUID = -7348332185233715983L;
#Id
#Basic(optional = false)
#Column(name = "PARENT_ID")
private Long parentId;
#OneToMany
#JoinColumn(name="CHILDREN", referencedColumnName="PARENT_ID")
private List<Child> children;
#Formula("(select sum(select height from children))")
private BicDecimal totalHeight
}
It is pretty straight forward with no restrictions and even with static restrictions. I am having trouble when the children list is restricted dynamically though.
In my case, I am using spring data and jpa. I am using specifications to restrict the children and am getting the appropriate list of children, but obviously the sum is still for unrestricted children because there is no where clause in the #Formula tag.
I do not want to iterate over the list in java for performance reasons and because the results are paginated. Also, the sum is not of the paginated results, but of all results.
I am new to Spring Data/JPA. Historically, I could build this query dynamically or use hibernate criteria. I am OK running a completley separate query to make this calculation. it is not required that I use the #Formula annotation as there is only 1 aggregation per call. In a hibernate framework, I could just state the select clause as "sum(field)" and build the criteria. In the Spring Data/JPA framework, I can build the specifications fine which covers the criteria, but I have no idea how to manipulate the select part of the query since it seems tied so tightly to the entity.
Using the #Query annotation on the repository works as its own query if I know which fields I need to restrict on, but often the fields are null and need to be ignored for the query. There are 8 possible fields, leaving me with 256 possible combinations (2^8). That is too many methods for this in the repository.
Any ideas outside of switching frameworks?
Posting an answer to this old question since I had a somewhat similar problem recently.
I decided to go with a custom repository with a method that does the aggregation based on any Specification passed into it. Specifications can be combined to compose dynamic criteria (see org.springframework.data.jpa.domain.Specifications)
So my repository to above Child height problem would look like below:
package something
import org.springframework.data.jpa.domain.Specification;
import org.springframework.stereotype.Repository;
import javax.persistence.EntityManager;
import javax.persistence.PersistenceContext;
import javax.persistence.criteria.CriteriaBuilder;
import javax.persistence.criteria.CriteriaQuery;
import javax.persistence.criteria.Root;
#Repository
public class ChildHeightRepository {
#PersistenceContext
EntityManager entityManager;
public Long getTotalHeight(Specification<Child> spec) {
CriteriaBuilder cb = entityManager.getCriteriaBuilder();
CriteriaQuery query = cb.createQuery(Long.class);
Root root = query.from(Child.class);
return ((Long) entityManager.createQuery(
query.where(spec.toPredicate(root, query, cb))
.select(cb.sum(root.get("height")))).getSingleResult());
}
}
Have you tried in JPQL
select sum(d.height) from Parent a join a.children d
If you dont want to ignore nulls
select sum(d.height) from Parent a left join a.children d
I think other question you have is how to filter depending on the properties . I mean if you need to have a where statement with several combinations.
Why you don't try to use a List and adding to the list all the predicates you want to apply depending on the combinations you want to have. Example
Create query
CriteriaBuilder cb = em.getCriteriaBuilder();
CriteriaQuery cq = cb.createQuery(Double.class);
Root<Parent> root = cq.from(Parent.class)
Join<Parent, Child> join = cq.join(("children"));
cq.select(cb.<Double>sum(join.get("height")));
Create the list of predicates
List<Predicates> listOfpredicates = new ArrayList<Predicates>();
if(property1 != null && !"".equals(property1){
PatameterExpression<String> p = cb.parameter(String.class, "valueOfproperty1")
listOfpredicates.add(cb.equal(join.get("property1"),p);
}
Then add to the CriteriaQuery
if(listOfPredicates.size() == 1)
cq.where(listOfPredicates.get(0))
else
cq.where(cb.and(listOfPredicates.toArray(new Predicate[0])));
Finally execute the query.
TypedQuery<Double> q = em.createQuery(cq);
q.getResultList();
This will create dynamically your query with any combination.
6 years late but it still took me a while to get it to work, here is how I would do it for your mapping:
#Formula("(select sum(children.height) from children_table children inner join Parent p on children.parent_id=p.id where children.parent_id=parent_id)")
private BicDecimal totalHeight
Stuff you need to take care of:
add () to the beginning and end of your formula otherwise the syntax of the sql wont be translated correctly.
the formula query is a native SQL query to my understanding and not a JPQL some one might want to correct me on this?.
properties are those of the tables and not what you name your properties in Java so the column names and table names have to actually be the table names.

Grails many to many with 3 classes: sorting by the number of relationships

Let's say we have 3 domain classes: 2 classes related with each other through a 3rd class.
Ok, some code:
class A {
String subject
String description
static hasMany = [cs: C]
static transients = ['numberOfCs']
Long getNumberOfCs() {
return cs.size()
}
}
class B {
String title
}
class C {
A objectA
B objectB
static belongsTo = [a: A]
}
Pretty clear? I hope so. This work perfectly with my domain.
You can see the transient property numberOfCs, which is used to calculate the number of C instances related to my A object. And it works just fine.
The problem: listing all my A objects, I want to sort them by the number of relationships with C objects, but the transient property numberOfCs cannot be used for the scope.
How can I handle the situation? How can I tell GORM to sort the As list by numberOfCs as it would be a regular (non transient) field?
Thanks in advance.
I'm not sure that Grails' criteria do support this, as you need both to select the A object itself and aggregate by a child objects (C). That means grouping by all the A's fields, which is not done automatically.
If you only need some fields from A, you can group by them:
def instances = A.withCriteria {
projections {
groupProperty('subject')
count('cs', 'cCount')
}
order 'cCount'
}
otherwise you'll need to retrieve only ids and make a second query, like in this question.
Another way is to use derived properties like described here (not sure it will work though):
class A {
static mapping = {
numberOfCs formula: 'select count(*) from C where c.b_id = id'
}
}
I wouldn't consider your Problem GORM related but rather general Hibernate or even SQL related.
Take a look at the HQL Docu they are a lot of examples.
check the following HQL this pretty close what you are asking.
select mySortedAs from A mySortedAs left join mySortedAs.cs myCsOfA order by
count(myCsOfA)
I think I saw somewhere that you also can do something like this myCsOfA.length or myCsOfA.size

Resources