I have the following annotation based elastic search configuration, I've set the index not to be analyzed because I don't want these fields to be tokenized:
#Document(indexName = "abc", type = "efg")
public class ResourceElasticSearch {
#Id
private String id;
#Field(type = FieldType.String, index = FieldIndex.not_analyzed)
private String name;
#Field(type = FieldType.String, store = true)
private List<String> tags = new ArrayList<>();
#Field(type = FieldType.String)
private String clientId;
#Field(type = FieldType.String, index = FieldIndex.not_analyzed)
private String virtualPath;
#Field(type = FieldType.Date)
private Date lastModifiedTime;
#Field(type = FieldType.Date)
private Date lastQueryTime;
#Field(type = FieldType.String)
private String modificationId;
#Field(type = FieldType.String)
private String realPath;
#Field(type = FieldType.String)
private String extension;
#Field(type = FieldType.String)
private ResourceType type;
Is it possible by using annotations to make the searches on the name, virtualPath and tags to be case-insensitive?
The search looks like this, search by wildcard is required:
private QueryBuilder getQueryBuilderForSearch(SearchCriteria criteria) {
String virtualPath = criteria.getPath();
return boolQuery()
.must(wildcardQuery("virtualPath", virtualPath))
.must(wildcardQuery("name", criteria.getName()));
}
Not really possible what you want to do and it's not about Spring Data configuration, it's about Elasticsearch itself: you indexed data as not_analyzed and it will stay that way.
Also, if you wanted case insensitive data I suggest indexing with keyword analyzer combined with a lowercase token filter.
I've found something based on Andrei Stefan's suggestion which has a similar result to using the annotations:
#Bean
public Client client() throws IOException {
TransportClient client = new TransportClient();
TransportAddress address = new InetSocketTransportAddress(env.getProperty("elasticsearch.host"), Integer.parseInt(env.getProperty("elasticsearch.port")));
client.addTransportAddress(address);
XContentBuilder settingsBuilder = XContentFactory.jsonBuilder()
.startObject()
.startObject("analysis")
.startObject("analyzer")
.startObject("keyword")
.field("tokenizer", "keyword")
.array("filter", "lowercase")
.endObject()
.endObject()
.endObject()
.endObject();
if (!client.admin().indices().prepareExists("abc").execute().actionGet().isExists()) {
client.admin().indices().prepareCreate("abc").setSettings(settingsBuilder).get();
}
return client;
}
You can add #Setting, which consumes file path, after #Document, settings file should contain json like this:
{"analysis":{"analyzer":{"case_insensitive":{"type":"custom","tokenizer":"whitespace","char_filter":["html_strip"],"filter":["lowercase","asciifolding"]}}}}
and field annotation with analyzer #Field(type = FieldType.Keyword, analyzer = "case_insensitive")
Related
I am using Spring Elasticsearch. This is my java class :
#Entity
#Document(indexName = "shopindex")
public class Shop implements Serializable {
private #Id #GeneratedValue(strategy = GenerationType.IDENTITY) Long id;
private String imagePath;
#Field(type = FieldType.Text, name = "name")
private String name;
#Field(type = FieldType.Text, name = "description")
private String description;
#Field(type = FieldType.Text, name = "address")
private String address;
#Field(type = FieldType.Text, name = "locality")
private String locality;
#Field(type = FieldType.Keyword, name = "city")
private String city;
#Field(type = FieldType.Keyword, name = "state")
private String state;
private String timing;
#Field(type = FieldType.Nested, includeInParent = true)
private ArrayList<Listing> listings;
Shop () {}
}
I want to have two query strings, location and query. I want query to search through fields name, description, and listing and location to search through fields address, location, city, and state. I am using this query for search but I am getting exception :
NativeSearchQuery searchQuery = new NativeSearchQueryBuilder()
.withQuery(boolQuery.must(QueryBuilders.multiMatchQuery(location, "address", "locality", "city", "state"))
.must(boolQuery.should(QueryBuilders.multiMatchQuery(query, "name", "description"))
.should(QueryBuilders
.nestedQuery("listings",
QueryBuilders.multiMatchQuery(query, "listings.name", "listings.description"),ScoreMode.Avg))))
.build();
Iterable<Shop> itr = searchRepository.search(searchQuery);
#Document(indexName = "opportunity_data", type = "opportunities", createIndex = false)
#Setting(settingPath = "/search/settings.json")
#Data
#Accessors(chain = true)
#JsonIgnoreProperties(value = {"id"}, allowGetters = true, allowSetters = false)
public class OpportunityVo extends AbstractGenericVo<Opportunity> {
#Id
#Field(type = FieldType.Long)
private Long opportunityId;
#Field(type = FieldType.Long)
private Long prospectId;
#Field(type = FieldType.Text)
private String prospectName;
}
mapping to keyword snapshot
the opportunityId above is mapped to keyword other than the long type. Anyone knows how to map the opportunityId to long type when #Id annotated at the same time since I'd like to do stats metrics aggregation on this property?
Stats aggregation can be only applied to numerics. Change the mappings works fine. But #Id will be mapped to keyword , still no luck.
Having this SearchQuery:
final SearchQuery searchQuery = new NativeSearchQueryBuilder().withQuery(QueryBuilders.rangeQuery("updateTime").gte(LocalDate.now())).build();
final List<ActivityValue> listOf = elasticsearchTemplate.queryForList(searchQuery, ActivityValue.class);
With Entity ActivityValue:
#Document(indexName = "tracking1", type = "activity")
public class ActivityValue {
#Id
private String id;
#Field(type = FieldType.Date, index = false, store = true, format = DateFormat.custom, pattern = "yyyy-MM-dd")
#JsonFormat(shape = JsonFormat.Shape.STRING, pattern = "yyyy-MM-dd")
private LocalDate updateTime;
#Field(type = FieldType.Object, includeInParent = true)
private Vendor vendor;
#Field(type = FieldType.Object, includeInParent = true)
private QCriteria quality;
public ActivityValue() {
}
//setter and getter
}
If i run the query and try to receive the list i get following exception:
caused by: java.io.IOException: can not write type [class java.time.LocalDate]
The entity is stored before with the actual date as LocalDate.
I'm uncertain what is the best/easiest way to query elasticsearch and to resolve this error. Can anybody help?
final SearchQuery searchQuery = new NativeSearchQueryBuilder().withQuery(QueryBuilders.rangeQuery("updateTime").gte(LocalDate.now().toString())).build();
final List<ActivityValue> listOf = elasticsearchTemplate.queryForList(searchQuery, ActivityValue.class);
I solved this problem by using LocalDate.now().toString() instead of LocalDate.now()
I am using elastic search db and spring data.
Following is my document in which I am searching and that search result should return sorted and pageable list.
#Data
#EqualsAndHashCode(exclude = { "id" })
#Document(indexName = "job", type = "job")
public class JobDocument implements Serializable {
private static final long serialVersionUID = 1L;
#Id
private String id;
#Field(type = FieldType.String, index = FieldIndex.analyzed, store = true)
private String name;
#Field(type = FieldType.String, index = FieldIndex.analyzed, store = true)
private String desc;
#Field(type = FieldType.Date, store = true)
private LocalDateTime dateTime; // java.Time
}
I am searching text in desc field and want to sort it by dateTime field. My search service does following,
BoolQueryBuilder queryBuilder = boolQuery();
queryBuilder.must(QueryBuilders.queryStringQuery("*" + desc + "*").lenient(true).field("desc"));
NativeSearchQueryBuilder searchQuery = new NativeSearchQueryBuilder();
searchQuery.withPageable(pageable);
searchQuery.withSort(SortBuilders.fieldSort("dateTime")
.order(SortOrder.DESC))
searchQuery.withQuery(queryBuilder);
Page<JobDocument> jobs = jobRepo.search(searchQuery.build());
Following is my repo,
public interface JobDAO extends ElasticsearchRepository<JobDocument, String>
{}
Search and pageable is working but Sorting is not working.
Am I missing something?
Their is a sort option available in pageable object.Instead of using the sort and pagination separately, you can add them into a single object PageRequest.
BoolQueryBuilder queryBuilder = boolQuery();
queryBuilder.must(QueryBuilders.queryStringQuery("*" + desc + "*").lenient(true).field("desc"));
NativeSearchQueryBuilder searchQuery = new NativeSearchQueryBuilder();
searchQuery.withPageable(new PageRequest(0, 10, Sort.Direction.DESC, "dateTime"));
searchQuery.withQuery(queryBuilder);
I'm using Spring-Data-Elastic-Search for searching/caching purposes.
I need to execute a query which uses child(TermCache) and parent(ConceptCache) properties
and return instances of child objects(this means i can't use nested objects).
i have the following structure:
#Document(indexName = "termweb" , type = "term")
public class TermCache {
#Id
private String id;
private String name;
private LanguageDTO language;
private String status;
private String definition;
#Field(type = FieldType.String, store = true)
#Parent(type = "concept")
private Long conceptId;
private String displayId;
private Map<Long, String> fields = new HashMap<>();
//todo think about storing it as a collection of nested objects
}
#Document( indexName = "termweb" , type = "concept")
public class ConceptCache implements ConceptDTO{
#Id
private String id;
private String displayId;
private Long dictionaryId;
private String dictionaryName;
private Map<Long, String> fields = new HashMap<>();
}
I need a hint on how to handle this type of tasks; should i use two separate queries or should i somehow fetch properties of a parent or maybe something else?
Agreed, We are lacking on documentation which we will be improving with upcoming release.
If you have any question about spring data elasticsearch stackoverflow probably is not best way to get answer(as we wont be notified for new thread), we have separate google group for question/queries https://groups.google.com/forum/#!forum/spring-data-elasticsearch-devs
Without having any idea about what exactly you are trying to achieve with above entities, i can give you an example of sample parent child entities as below
#Document(indexName = "parent-child", type = "parent-entity")
public class ParentEntity {
#Id
private String id;
#Field(type = FieldType.String, index = FieldIndex.analyzed, store = true)
private String name;
// setter/getter
public ParentEntity() {
}
public ParentEntity(String id, String name) {
this.id = id;
this.name = name;
}
}
#Document(indexName = "parent-child", type = "child-entity")
public class ChildEntity {
#Id
private String id;
#Field(type = FieldType.String, store = true)
#Parent(type = "parent-entity")
private String parentId;
#Field(type = FieldType.String, index = FieldIndex.analyzed, store = true)
private String name;
public ChildEntity() {
}
public ChildEntity(String id, String parentId, String name) {
this.id = id;
this.parentId = parentId;
this.name = name;
}
}
// indexing parent (you can use many other ways to index that includes using repositories)
ParentEntity parent1 = new ParentEntity("parent1", "First Parent");
IndexQuery parentIndex1 = new IndexQuery();
parentIndex1.setId(parent1.getId());
parentIndex1.setObject(parent1);
elasticsearchTemplate.index(parentIndex1);
ParentEntity parent2 = new ParentEntity("parent2", "Second Parent");
IndexQuery parentIndex2 = new IndexQuery();
parentIndex2.setId(parent2.getId());
parentIndex2.setObject(parent2);
elasticsearchTemplate.index(parentIndex2);
// indexing child
ChildEntity child1 = new ChildEntity("child1", parent1.getId(), "First");
IndexQuery childIndex1 = new IndexQuery();
childIndex1.setId(child1.getId());
childIndex1.setObject(child1);
childIndex1.setParentId(child1.getParentId());
elasticsearchTemplate.index(childIndex1);
ChildEntity child2 = new ChildEntity("child2", parent1.getId(), "Second");
IndexQuery childIndex2 = new IndexQuery();
childIndex2.setId(child2.getId());
childIndex2.setObject(child2);
childIndex2.setParentId(child2.getParentId());
elasticsearchTemplate.index(childIndex2);
// searching
there are several available option while searching on Parent/Child entities, that includes has children, has parent and top children queries.
QueryBuilder query = topChildrenQuery("child-entity", QueryBuilders.termQuery("name", child1name.toLowerCase()));
SearchQuery searchQuery = new NativeSearchQueryBuilder().withQuery(query).build();
List<ParentEntity> parents = elasticsearchTemplate.queryForList(searchQuery, ParentEntity.class);
Hope this small example will give you basic understanding how to use parent child. have a look at ParentChildTests for more.
If you still have more question please feel free to contact us.
You should simply use hasparent query of filter : http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-has-parent-filter.html#query-dsl-has-parent-filter
This will make a request on parent field and result in children documents of the matching parents documents. You can then use a filter on the returned child document :)