I am new to elastic-search and i am trying to use spring data elastic search in the application.
I have a requirement where in there are two separate indexes and i want to fetch documents from both indexes in one query based on some condition.
I would try to explain it with sample example with the same scenario.
There are two Different classes for individual indexes.
#Document(indexName = "Book", type = "Book")
public class Book {
#Id
private String id;
#Field(type = FieldType.String)
private String bookName;
#Field(type = FieldType.Integer)
private int price;
#Field(type = FieldType.String)
private String authorName;
//Getters and Setters
}
There is one more class Author
#Document(indexName = "Author", type = "Author")
public Class Author{
#Id
private String id;
#Field(type = FieldType.String)
private String authorName;
//Getters and setters
}
So there are two indexes one Book and Other Author.
I want to fetch all the documents where authorName in Book index is equal to authorName in Author index.
Can i get the details from both the index as a single document like merged result.
It would be very helpful if anyone can suggest solution for this usecase.
Thanks & Regards
Sumanth
Related
I'm trying to do a join using spring data couchbase.
The documentation has this example
#Document
public class Author {
#Id
String id;
String name;
#N1qlJoin(on = "lks.name=rks.authorName")
List<Book> books;
#N1qlJoin(on = "lks.name=rks.name")
Address address;
...
}
I need to do the same, except my model uses the document key:
#Document
public class Author {
#Id
String id;
String addressId;
//#N1qlJoin(on = "lks.addressId=meta(rks).id") this doesn't work
#N1qlJoin(on = "lks.addressId=rks.id") //nor this
Address address;
...
}
I am trying to use Hibernate Search 6 and elastic search
A simple example of what I am trying to build is as follows.
I have a Book entity, which has information like title, authorName, genre, price
I have a Shop entity which has information like shopName, phone, email, location
I have a "joining table" which does a many to many mapping between nooks and shops. ( A book can be at many shops, and a shop can have many books)
I am trying to do a search by name and location, ideally to find a book at a location nearest to the input. The standard book-author example in the documentation requires a IndexedEmbedded annotation, which is not really possible in my case because I am using a joining table.
Is there an alternative approach to solve this problem
My entities
#Indexed
public class Book extends PanacheEntity{
public String title;
public String authorName;
#OneToMany(mappedBy = "book", fetch = FetchType.LAZY, cascade = { CascadeType.REMOVE })
public List<BookShopRelation> bookShopRelation = new ArrayList<>();
}
#Indexed
public class Shop extends PanacheEntity{
public String name;
public String city;
#OneToMany(mappedBy = "shop", fetch = FetchType.LAZY, cascade = { CascadeType.REMOVE })
private List<BookShopRelation> bookShopRelation = new ArrayList<>();
}
#Indexed
public class BookShopRelation extends PanacheEntity{
#JoinColumn(name = "shop_id")
#ManyToOne(fetch = FetchType.EAGER, optional = false)
#IndexedEmbedded
private Shop shop;
#JoinColumn(name = "offer_id")
#ManyToOne(fetch = FetchType.EAGER, optional = false)
#IndexedEmbedded
private Book book;
}
For me what was key was to understand, was that the relation table could be indexed and used as the basis for the search
List<BookShopRelation> result = Search.session(entityManager)
.search(BookShopRelation.class) .predicate(f ->
pattern == null || pattern.trim().isEmpty() ?
f.matchAll() :
f.simpleQueryString()
.fields("book.title").matching(pattern)
)
.fetchHits(size.orElse(20));
I try to sort a list of Items for a customer by ordered Date. The Date is only avalable through Item.orderPositions.order.orderDate . But #IndexedEmbedded doesn't work. There's no Exeption or Error but the result is only sorted by HS-logic.
#Entity
#Indexed
public class Item{
#Id
private long id;
#Field(index = Index.YES, store = Store.YES, analyse = Analyse.YES, analyser = #Analyzer(definition = Constant.ANALYSER))
private String description;
#OneToMany(mappedBy = "item")
#IndexedEmbedded
private List<OrderPosition> orderPositions;
#ManyToOne
#IndexedEmbedded
private Company company;
//getter&setter
}
#Entity
public class OrderPosition{
#Id
private long id;
#ManyToOne
private Item item;
#ManyToOne
#IndexedEmbedded
private Order order;
//getter&setter
}
#Entity
public class Order{
#Id
private long id;
#ManyToOne
private Customer customer;
#Field(index = Index.NO, store = Store.NO, analyze = Analyze.NO)
#SortableField
private String orderDate;
//getter&setter
}
#Entity
public class Company{
#Id
private long id;
#Field(index = Index.NO, store = Store.NO, analyze = Analyze.NO)
#SortableField
private String name;
//getter&setter
}
If I sort the List by Item.company.name it works fine.
queryService.buildFullTextQuery("searchText", Item.class, "description", "company.name").getResultList();
If I sort the List by Item.orderPosition.order.orderDate it's sorted by default(HS-logic)
queryService.buildFullTextQuery("searchText", Item.class, "description", "orderPositions.order.orderDate").getResultList();
I build the FullTextQuery this way:
public FullTextQuery buildFullTextQuery(#NonNull String searchText, #NonNull Class<?> clazz, #NonNull String... fields) throws Exception {
FullTextEntityManager fullTextEntityManager = Search.getFullTextEntityManager(getEntityManager());
QueryBuilder qb = fullTextEntityManager.getSearchFactory().buildQueryBuilder().forEntity(clazz).get();
Query query = qb.keyword().onField(fields[0]).matching(searchText).createQuery();
SortField sortField = new SortField(fields[1], SortField.Type.STRING, false);
Sort sort = new Sort(sortField);
return fullTextEntityManager.createFullTextQuery(query, clazz).setSort(sort);
}
I think HS can't find the association for #OneToMany. Is there a way to solve this prob?
Thank you in advance
I can't tell you what's going on exactly without the results of your queries, but you're definitely doing something wrong here: you are trying to sort on a multi-valued field. One item is linked to multiple orders, each having its own date. So there is multiple dates per item.
When you ask to compare two items that each have three dates, what should Hibernate Search do? Compare only the latest dates? Compare only the earliest dates? You didn't say, so your query is bound to return inconsistently ordered results.
Thing is, there is no way to tell Hibernate Search which value to pick in multi-valued fields, so your easiest way out is to explicitly create a single-valued field to sort on.
For instance, you could add a getter on Item to return the latest order, and add the #IndexedEmbedded there:
#Entity
#Indexed
public class Item{
#Id
private long id;
#Field(index = Index.YES, store = Store.YES, analyse = Analyse.YES, analyser = #Analyzer(definition = Constant.ANALYSER))
private String description;
#OneToMany(mappedBy = "item")
#IndexedEmbedded
private List<OrderPosition> orderPositions;
#ManyToOne
#IndexedEmbedded
private Company company;
#javax.persistence.Transient
public Order getLatestOrder() {
Order latestOrder;
// ... compute the latest order ...
return latestOrder;
}
//getter&setter
}
Then sort on latestOrder.orderDate and you should be good.
I'm using Spring-Data-Elastic-Search for searching/caching purposes.
I need to execute a query which uses child(TermCache) and parent(ConceptCache) properties
and return instances of child objects(this means i can't use nested objects).
i have the following structure:
#Document(indexName = "termweb" , type = "term")
public class TermCache {
#Id
private String id;
private String name;
private LanguageDTO language;
private String status;
private String definition;
#Field(type = FieldType.String, store = true)
#Parent(type = "concept")
private Long conceptId;
private String displayId;
private Map<Long, String> fields = new HashMap<>();
//todo think about storing it as a collection of nested objects
}
#Document( indexName = "termweb" , type = "concept")
public class ConceptCache implements ConceptDTO{
#Id
private String id;
private String displayId;
private Long dictionaryId;
private String dictionaryName;
private Map<Long, String> fields = new HashMap<>();
}
I need a hint on how to handle this type of tasks; should i use two separate queries or should i somehow fetch properties of a parent or maybe something else?
Agreed, We are lacking on documentation which we will be improving with upcoming release.
If you have any question about spring data elasticsearch stackoverflow probably is not best way to get answer(as we wont be notified for new thread), we have separate google group for question/queries https://groups.google.com/forum/#!forum/spring-data-elasticsearch-devs
Without having any idea about what exactly you are trying to achieve with above entities, i can give you an example of sample parent child entities as below
#Document(indexName = "parent-child", type = "parent-entity")
public class ParentEntity {
#Id
private String id;
#Field(type = FieldType.String, index = FieldIndex.analyzed, store = true)
private String name;
// setter/getter
public ParentEntity() {
}
public ParentEntity(String id, String name) {
this.id = id;
this.name = name;
}
}
#Document(indexName = "parent-child", type = "child-entity")
public class ChildEntity {
#Id
private String id;
#Field(type = FieldType.String, store = true)
#Parent(type = "parent-entity")
private String parentId;
#Field(type = FieldType.String, index = FieldIndex.analyzed, store = true)
private String name;
public ChildEntity() {
}
public ChildEntity(String id, String parentId, String name) {
this.id = id;
this.parentId = parentId;
this.name = name;
}
}
// indexing parent (you can use many other ways to index that includes using repositories)
ParentEntity parent1 = new ParentEntity("parent1", "First Parent");
IndexQuery parentIndex1 = new IndexQuery();
parentIndex1.setId(parent1.getId());
parentIndex1.setObject(parent1);
elasticsearchTemplate.index(parentIndex1);
ParentEntity parent2 = new ParentEntity("parent2", "Second Parent");
IndexQuery parentIndex2 = new IndexQuery();
parentIndex2.setId(parent2.getId());
parentIndex2.setObject(parent2);
elasticsearchTemplate.index(parentIndex2);
// indexing child
ChildEntity child1 = new ChildEntity("child1", parent1.getId(), "First");
IndexQuery childIndex1 = new IndexQuery();
childIndex1.setId(child1.getId());
childIndex1.setObject(child1);
childIndex1.setParentId(child1.getParentId());
elasticsearchTemplate.index(childIndex1);
ChildEntity child2 = new ChildEntity("child2", parent1.getId(), "Second");
IndexQuery childIndex2 = new IndexQuery();
childIndex2.setId(child2.getId());
childIndex2.setObject(child2);
childIndex2.setParentId(child2.getParentId());
elasticsearchTemplate.index(childIndex2);
// searching
there are several available option while searching on Parent/Child entities, that includes has children, has parent and top children queries.
QueryBuilder query = topChildrenQuery("child-entity", QueryBuilders.termQuery("name", child1name.toLowerCase()));
SearchQuery searchQuery = new NativeSearchQueryBuilder().withQuery(query).build();
List<ParentEntity> parents = elasticsearchTemplate.queryForList(searchQuery, ParentEntity.class);
Hope this small example will give you basic understanding how to use parent child. have a look at ParentChildTests for more.
If you still have more question please feel free to contact us.
You should simply use hasparent query of filter : http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-has-parent-filter.html#query-dsl-has-parent-filter
This will make a request on parent field and result in children documents of the matching parents documents. You can then use a filter on the returned child document :)
I have two such Java object:
public class PSubject
{
#Column
#Field(index=Index.YES, analyze=Analyze.YES, store=Store.NO)
#org.apache.solr.client.solrj.beans.Field("name")
private String name;
#Column
#Field(index=Index.YES, analyze=Analyze.YES, store=Store.NO)
#org.apache.solr.client.solrj.beans.Field("type")
private String type;
#Column
#Field(index=Index.YES, analyze=Analyze.YES, store=Store.NO)
#org.apache.solr.client.solrj.beans.Field("uri")
private String uri;
#OneToMany(fetch=FetchType.EAGER,cascade=CascadeType.ALL)
#IndexedEmbedded
#org.apache.solr.client.solrj.beans.Field("attributes")
private Set<PAttribute> attributes = new HashSet<PAttribute>();
.....
}
#Entity
#Indexed
#Table(name="PAttribute")
#Inheritance(strategy = InheritanceType.TABLE_PER_CLASS)
public class PAttribute extends PEntity
{
private static final long serialVersionUID = 1L;
#Column
#Field(index=Index.YES, analyze=Analyze.YES, store=Store.YES)
#org.apache.solr.client.solrj.beans.Field("attr_name")
private String name;
#Column
#Field(index=Index.YES, analyze=Analyze.YES, store=Store.YES)
#org.apache.solr.client.solrj.beans.Field("attr_value")
private String value;
.....
}
And my Spring Data Solr query interface:
public interface DerivedSubjectRepository extends SolrCrudRepository<PSubject, String> {
Page<PSubject> findByName(String name, Pageable page);
List<PSubject> findByNameStartingWith(String name);
Page<PSubject> findBy(Pageable page);
#Query("name:*?0* or description:*?0* or type:*?0* or mac_address:*?0* or uri:*?0* or attributes:*?0*")
Page<PSubject> find(String keyword,Pageable page);
#Query("name:*?0* or description:*?0* or type:*?0* or mac_address:*?0* or uri:*?0* or attributes:*?0*")
List<PSubject> find(String keyword);
}
I can search any by name, description, type and mac_address, but can't search any result by attribute.
Update:
For example,when user search "ipod", it's probably means the type of subject or name of subject, or the name of attribute or the value of attribute. And I want get all the matched subject in one request. I know I can search the attribute object in a separate query. But that makes the code in the backend complex.
So, how can I search this nested object?
Update:
I flattened my data:
#Transient
#Field(index=Index.YES, analyze=Analyze.YES, store=Store.NO)
#org.apache.solr.client.solrj.beans.Field("attrs")
private String attrs;
public String getAttrs() {
return attrs;
}
public void setAttrs(Set<PAttribute> attributes) {
StringBuffer attrs = new StringBuffer();
if(attributes==null) {
attributes = this.getAttributes();
}
for(PAttribute attr:attributes){
attrs.append(attr.getName()+" " + attr.getValue()).append(" ");
}
this.attrs =attrs.toString();
}
The issue is resolved.
IIRC it is not possible to store nested data structures in solr - it depends how you flatten your data to fit into an eg. multivalue field - a little hard not knowing your schema.
see: http://lucene.472066.n3.nabble.com/Possible-to-have-Solr-documents-with-deeply-nested-data-structures-i-e-hashes-within-hashes-td4004285.html
How does the data look like in you index, and did you have a look at the http request sent by spring-data-solr?