Parent/Child relationships in spring-data-elastic-search - elasticsearch

I'm using Spring-Data-Elastic-Search for searching/caching purposes.
I need to execute a query which uses child(TermCache) and parent(ConceptCache) properties
and return instances of child objects(this means i can't use nested objects).
i have the following structure:
#Document(indexName = "termweb" , type = "term")
public class TermCache {
#Id
private String id;
private String name;
private LanguageDTO language;
private String status;
private String definition;
#Field(type = FieldType.String, store = true)
#Parent(type = "concept")
private Long conceptId;
private String displayId;
private Map<Long, String> fields = new HashMap<>();
//todo think about storing it as a collection of nested objects
}
#Document( indexName = "termweb" , type = "concept")
public class ConceptCache implements ConceptDTO{
#Id
private String id;
private String displayId;
private Long dictionaryId;
private String dictionaryName;
private Map<Long, String> fields = new HashMap<>();
}
I need a hint on how to handle this type of tasks; should i use two separate queries or should i somehow fetch properties of a parent or maybe something else?

Agreed, We are lacking on documentation which we will be improving with upcoming release.
If you have any question about spring data elasticsearch stackoverflow probably is not best way to get answer(as we wont be notified for new thread), we have separate google group for question/queries https://groups.google.com/forum/#!forum/spring-data-elasticsearch-devs
Without having any idea about what exactly you are trying to achieve with above entities, i can give you an example of sample parent child entities as below
#Document(indexName = "parent-child", type = "parent-entity")
public class ParentEntity {
#Id
private String id;
#Field(type = FieldType.String, index = FieldIndex.analyzed, store = true)
private String name;
// setter/getter
public ParentEntity() {
}
public ParentEntity(String id, String name) {
this.id = id;
this.name = name;
}
}
#Document(indexName = "parent-child", type = "child-entity")
public class ChildEntity {
#Id
private String id;
#Field(type = FieldType.String, store = true)
#Parent(type = "parent-entity")
private String parentId;
#Field(type = FieldType.String, index = FieldIndex.analyzed, store = true)
private String name;
public ChildEntity() {
}
public ChildEntity(String id, String parentId, String name) {
this.id = id;
this.parentId = parentId;
this.name = name;
}
}
// indexing parent (you can use many other ways to index that includes using repositories)
ParentEntity parent1 = new ParentEntity("parent1", "First Parent");
IndexQuery parentIndex1 = new IndexQuery();
parentIndex1.setId(parent1.getId());
parentIndex1.setObject(parent1);
elasticsearchTemplate.index(parentIndex1);
ParentEntity parent2 = new ParentEntity("parent2", "Second Parent");
IndexQuery parentIndex2 = new IndexQuery();
parentIndex2.setId(parent2.getId());
parentIndex2.setObject(parent2);
elasticsearchTemplate.index(parentIndex2);
// indexing child
ChildEntity child1 = new ChildEntity("child1", parent1.getId(), "First");
IndexQuery childIndex1 = new IndexQuery();
childIndex1.setId(child1.getId());
childIndex1.setObject(child1);
childIndex1.setParentId(child1.getParentId());
elasticsearchTemplate.index(childIndex1);
ChildEntity child2 = new ChildEntity("child2", parent1.getId(), "Second");
IndexQuery childIndex2 = new IndexQuery();
childIndex2.setId(child2.getId());
childIndex2.setObject(child2);
childIndex2.setParentId(child2.getParentId());
elasticsearchTemplate.index(childIndex2);
// searching
there are several available option while searching on Parent/Child entities, that includes has children, has parent and top children queries.
QueryBuilder query = topChildrenQuery("child-entity", QueryBuilders.termQuery("name", child1name.toLowerCase()));
SearchQuery searchQuery = new NativeSearchQueryBuilder().withQuery(query).build();
List<ParentEntity> parents = elasticsearchTemplate.queryForList(searchQuery, ParentEntity.class);
Hope this small example will give you basic understanding how to use parent child. have a look at ParentChildTests for more.
If you still have more question please feel free to contact us.

You should simply use hasparent query of filter : http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-has-parent-filter.html#query-dsl-has-parent-filter
This will make a request on parent field and result in children documents of the matching parents documents. You can then use a filter on the returned child document :)

Related

Elasticsearch with spring boot when using index query return nullPointerExciption

I am using Elasticsearch with spring boot. A post request returns a null pointer exception, because index query is null value doesn't have index name or any things.
Look at my code
Service :
public List<Product> createProducts() {
List<Product>productList = new ArrayList<>();
for (Integer i = 0; i < 200; i++)
{
Product product = new Product();
product.setId(Long.parseLong(i.toString()));
product.setProductName(generateName());
product.setProductPrice(generatePrice());
product.setCategory(generateCategory());
if(!product.validation().equals(""))
{
throw new BadRequestAlertException(product.validation(),"Product","check input");
}
IndexQuery indexQuery=new IndexQueryBuilder().withId(i.toString()).build(); //return null
elasticsearchOperations.index(indexQuery); // here is error becouse index is null
productList.add(product);
}
return productList;
}
And this is the entity:
#JsonInclude(value = JsonInclude.Include.NON_NULL)
#Document(indexName = "product",type = "product")
public class Product implements Serializable {
private static final long serialVersionUID = 6320548148250372657L;
#Id
private Long id;
#Field(type = FieldType.Text)
private String productName;
#Field(type = FieldType.Text)
private String category;
#Field(type = FieldType.Double)
private Double productPrice;
This is the repostory:
public interface ProductSearchRepostory extends ElasticsearchRepository<Product,Long> {
List<Product> findByProductName(String name);
List<Product> findByCategory(String category);
}
You need to let the elasticsearchOperations.index method know which index to save the data in.
As per the documentation present for Spring Data Elasticsearch, in the newer versions, the index method expects an IndexCoordinates object which tells the client which index to put the data in. For the older versions ( < 4.0), this is inferred by the client based on the entity object that is being indexed.
In your code, can you please try to pass the entity while building the IndexQuery. Something like,
new IndexQueryBuilder().withId(i.toString()).withObject(product).build()

JPA Hibernate - Entity with #Loader and a function field in select, won't work properly

#Entity
#Table(name="cad_paciente")
#Loader(namedQuery = "selectInicial")
#NamedNativeQuery(
name="selectInicial",
query="select p.*, fu_obter_lista_convenios_pac(p.id) as ds_convenio from cad_paciente p where p.id = ?", resultClass = Paciente.class,
resultSetMapping = "sqlResult")
#SqlResultSetMapping(
name="sqlResult",
entities={
#EntityResult(entityClass = Paciente.class, fields={
#FieldResult(name="ds_convenio",column="ds_convenio")})})
public class Paciente {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#NotNull
#Column(name="id_empresa")
private Integer id_empresa;
...
#Transient
#Column(name="ds_convenio")
private String ds_convenio;
public String getDs_convenio() {
return ds_convenio;
}
public void setDs_convenio(String ds_convenio) {
this.ds_convenio = ds_convenio;
}
My Controller method "pacientes.findAll()" won't return "ds_convenio" field with the correct value, listing "null" always in my JSON return.
What do I have to do?
Try removing the annotation #Transient and provide the column as below :
#Column(name="ds_convenio")
private String ds_convenio;
#org.springframework.data.annotation.Transient specifically states to the spring framework that the Object Mapper you are using should not include this value when converting from Java Object to JSON. Also, it means that the value is not to be persisted into the database, which means you could not query over it.
Or if you want to keep it as transient itself but does not require the value to be serialized then register the object mapper as below :
#Bean
public ObjectMapper includeTransientObjectMapper() {
Hibernate5Module hibernate5Module = new Hibernate5Module();
hibernate5Module.disable(Hibernate5Module.Feature.USE_TRANSIENT_ANNOTATION);
ObjectMapper mapper = new ObjectMapper();
mapper.registerModule(hibernate5Module);
return mapper;
}
Or in your case since you want the result of the #NamedNativeQuer in which you aliased ds_convenio, using #FieldResult might be required to get the desired result as follows :
#Entity
#Table(name="cad_paciente")
#Loader(namedQuery = "selectInicial")
#NamedNativeQuery(name="selectInicial", query="select p.*, fu_obter_lista_convenios_pac(p.id) as ds_convenio from cad_paciente p where p.id = ?", resultClass = Paciente.class)
#SqlResultSetMapping(name="Results",
entities={
#EntityResult(entityClass=com.acme.Order.class, fields={
#FieldResult(name="id", column="id"),
#FieldResult(name="id_empresa", column="id_empresa"),
........
})
public class Paciente {
#Id
#GeneratedValue(strategy = GenerationType.IDENTITY)
private Long id;
#NotNull
#Column(name="id_empresa")
private Integer id_empresa;
...
#Transient
#Column(name="ds_convenio")
private String ds_convenio;
Read doc

Spring data elastic search with sort not working

I am using elastic search db and spring data.
Following is my document in which I am searching and that search result should return sorted and pageable list.
#Data
#EqualsAndHashCode(exclude = { "id" })
#Document(indexName = "job", type = "job")
public class JobDocument implements Serializable {
private static final long serialVersionUID = 1L;
#Id
private String id;
#Field(type = FieldType.String, index = FieldIndex.analyzed, store = true)
private String name;
#Field(type = FieldType.String, index = FieldIndex.analyzed, store = true)
private String desc;
#Field(type = FieldType.Date, store = true)
private LocalDateTime dateTime; // java.Time
}
I am searching text in desc field and want to sort it by dateTime field. My search service does following,
BoolQueryBuilder queryBuilder = boolQuery();
queryBuilder.must(QueryBuilders.queryStringQuery("*" + desc + "*").lenient(true).field("desc"));
NativeSearchQueryBuilder searchQuery = new NativeSearchQueryBuilder();
searchQuery.withPageable(pageable);
searchQuery.withSort(SortBuilders.fieldSort("dateTime")
.order(SortOrder.DESC))
searchQuery.withQuery(queryBuilder);
Page<JobDocument> jobs = jobRepo.search(searchQuery.build());
Following is my repo,
public interface JobDAO extends ElasticsearchRepository<JobDocument, String>
{}
Search and pageable is working but Sorting is not working.
Am I missing something?
Their is a sort option available in pageable object.Instead of using the sort and pagination separately, you can add them into a single object PageRequest.
BoolQueryBuilder queryBuilder = boolQuery();
queryBuilder.must(QueryBuilders.queryStringQuery("*" + desc + "*").lenient(true).field("desc"));
NativeSearchQueryBuilder searchQuery = new NativeSearchQueryBuilder();
searchQuery.withPageable(new PageRequest(0, 10, Sort.Direction.DESC, "dateTime"));
searchQuery.withQuery(queryBuilder);

Can't hibernate search sort in #OneToMany association?

I try to sort a list of Items for a customer by ordered Date. The Date is only avalable through Item.orderPositions.order.orderDate . But #IndexedEmbedded doesn't work. There's no Exeption or Error but the result is only sorted by HS-logic.
#Entity
#Indexed
public class Item{
#Id
private long id;
#Field(index = Index.YES, store = Store.YES, analyse = Analyse.YES, analyser = #Analyzer(definition = Constant.ANALYSER))
private String description;
#OneToMany(mappedBy = "item")
#IndexedEmbedded
private List<OrderPosition> orderPositions;
#ManyToOne
#IndexedEmbedded
private Company company;
//getter&setter
}
#Entity
public class OrderPosition{
#Id
private long id;
#ManyToOne
private Item item;
#ManyToOne
#IndexedEmbedded
private Order order;
//getter&setter
}
#Entity
public class Order{
#Id
private long id;
#ManyToOne
private Customer customer;
#Field(index = Index.NO, store = Store.NO, analyze = Analyze.NO)
#SortableField
private String orderDate;
//getter&setter
}
#Entity
public class Company{
#Id
private long id;
#Field(index = Index.NO, store = Store.NO, analyze = Analyze.NO)
#SortableField
private String name;
//getter&setter
}
If I sort the List by Item.company.name it works fine.
queryService.buildFullTextQuery("searchText", Item.class, "description", "company.name").getResultList();
If I sort the List by Item.orderPosition.order.orderDate it's sorted by default(HS-logic)
queryService.buildFullTextQuery("searchText", Item.class, "description", "orderPositions.order.orderDate").getResultList();
I build the FullTextQuery this way:
public FullTextQuery buildFullTextQuery(#NonNull String searchText, #NonNull Class<?> clazz, #NonNull String... fields) throws Exception {
FullTextEntityManager fullTextEntityManager = Search.getFullTextEntityManager(getEntityManager());
QueryBuilder qb = fullTextEntityManager.getSearchFactory().buildQueryBuilder().forEntity(clazz).get();
Query query = qb.keyword().onField(fields[0]).matching(searchText).createQuery();
SortField sortField = new SortField(fields[1], SortField.Type.STRING, false);
Sort sort = new Sort(sortField);
return fullTextEntityManager.createFullTextQuery(query, clazz).setSort(sort);
}
I think HS can't find the association for #OneToMany. Is there a way to solve this prob?
Thank you in advance
I can't tell you what's going on exactly without the results of your queries, but you're definitely doing something wrong here: you are trying to sort on a multi-valued field. One item is linked to multiple orders, each having its own date. So there is multiple dates per item.
When you ask to compare two items that each have three dates, what should Hibernate Search do? Compare only the latest dates? Compare only the earliest dates? You didn't say, so your query is bound to return inconsistently ordered results.
Thing is, there is no way to tell Hibernate Search which value to pick in multi-valued fields, so your easiest way out is to explicitly create a single-valued field to sort on.
For instance, you could add a getter on Item to return the latest order, and add the #IndexedEmbedded there:
#Entity
#Indexed
public class Item{
#Id
private long id;
#Field(index = Index.YES, store = Store.YES, analyse = Analyse.YES, analyser = #Analyzer(definition = Constant.ANALYSER))
private String description;
#OneToMany(mappedBy = "item")
#IndexedEmbedded
private List<OrderPosition> orderPositions;
#ManyToOne
#IndexedEmbedded
private Company company;
#javax.persistence.Transient
public Order getLatestOrder() {
Order latestOrder;
// ... compute the latest order ...
return latestOrder;
}
//getter&setter
}
Then sort on latestOrder.orderDate and you should be good.

How to search nested object by using Spring Data Solr?

I have two such Java object:
public class PSubject
{
#Column
#Field(index=Index.YES, analyze=Analyze.YES, store=Store.NO)
#org.apache.solr.client.solrj.beans.Field("name")
private String name;
#Column
#Field(index=Index.YES, analyze=Analyze.YES, store=Store.NO)
#org.apache.solr.client.solrj.beans.Field("type")
private String type;
#Column
#Field(index=Index.YES, analyze=Analyze.YES, store=Store.NO)
#org.apache.solr.client.solrj.beans.Field("uri")
private String uri;
#OneToMany(fetch=FetchType.EAGER,cascade=CascadeType.ALL)
#IndexedEmbedded
#org.apache.solr.client.solrj.beans.Field("attributes")
private Set<PAttribute> attributes = new HashSet<PAttribute>();
.....
}
#Entity
#Indexed
#Table(name="PAttribute")
#Inheritance(strategy = InheritanceType.TABLE_PER_CLASS)
public class PAttribute extends PEntity
{
private static final long serialVersionUID = 1L;
#Column
#Field(index=Index.YES, analyze=Analyze.YES, store=Store.YES)
#org.apache.solr.client.solrj.beans.Field("attr_name")
private String name;
#Column
#Field(index=Index.YES, analyze=Analyze.YES, store=Store.YES)
#org.apache.solr.client.solrj.beans.Field("attr_value")
private String value;
.....
}
And my Spring Data Solr query interface:
public interface DerivedSubjectRepository extends SolrCrudRepository<PSubject, String> {
Page<PSubject> findByName(String name, Pageable page);
List<PSubject> findByNameStartingWith(String name);
Page<PSubject> findBy(Pageable page);
#Query("name:*?0* or description:*?0* or type:*?0* or mac_address:*?0* or uri:*?0* or attributes:*?0*")
Page<PSubject> find(String keyword,Pageable page);
#Query("name:*?0* or description:*?0* or type:*?0* or mac_address:*?0* or uri:*?0* or attributes:*?0*")
List<PSubject> find(String keyword);
}
I can search any by name, description, type and mac_address, but can't search any result by attribute.
Update:
For example,when user search "ipod", it's probably means the type of subject or name of subject, or the name of attribute or the value of attribute. And I want get all the matched subject in one request. I know I can search the attribute object in a separate query. But that makes the code in the backend complex.
So, how can I search this nested object?
Update:
I flattened my data:
#Transient
#Field(index=Index.YES, analyze=Analyze.YES, store=Store.NO)
#org.apache.solr.client.solrj.beans.Field("attrs")
private String attrs;
public String getAttrs() {
return attrs;
}
public void setAttrs(Set<PAttribute> attributes) {
StringBuffer attrs = new StringBuffer();
if(attributes==null) {
attributes = this.getAttributes();
}
for(PAttribute attr:attributes){
attrs.append(attr.getName()+" " + attr.getValue()).append(" ");
}
this.attrs =attrs.toString();
}
The issue is resolved.
IIRC it is not possible to store nested data structures in solr - it depends how you flatten your data to fit into an eg. multivalue field - a little hard not knowing your schema.
see: http://lucene.472066.n3.nabble.com/Possible-to-have-Solr-documents-with-deeply-nested-data-structures-i-e-hashes-within-hashes-td4004285.html
How does the data look like in you index, and did you have a look at the http request sent by spring-data-solr?

Resources