I have an elastic search index that contains a certain field on which I want to perform a wildcard query. The issue is that the field is duplicated in many docs hence I want to use aggregation first to get unique values for that field and then perform a wildcard query on top of that. Is there a way I can perform the query on aggregation results in elastic search?
I believe you can find the results you need by collapsing your search results rather than using your strategy of first obtaining the aggregation results and then running a wildcard query.
Adding a working example with index data (with the default mapping), search query and search result.
Index Data:
{
"role": "example123",
"number": 1
}
{
"role": "example",
"number": 2
}
{
"role": "example",
"number": 3
}
Search Query:
{
"query": {
"wildcard": {
"role": "example*"
}
},
"collapse": {
"field": "role.keyword"
}
}
Search Result:
"hits": [
{
"_index": "72724517",
"_id": "1",
"_score": 1.0,
"_source": {
"role": "example",
"number": 1
},
"fields": {
"role.keyword": [
"example"
]
}
},
{
"_index": "72724517",
"_id": "3",
"_score": 1.0,
"_source": {
"role": "example123",
"number": 1
},
"fields": {
"role.keyword": [
"example123"
]
}
}
]
Related
ı want to find a field, Which field did find the search query?
this can be any query I am not writing a specific query
for example
ı searching dilo abinin phrase or any word, and found bellow document
{
"name":"dilo abinin",
"surname: "sürücü"
}
ı want to get name keyword
You can use highlighting, to see which field matched your query
Index API
{
"name":"dilo abinin",
"surname": "sürücü"
}
Search Query:
{
"query": {
"query_string": {
"query": "dilo abinin"
}
},
"highlight": {
"fields": {
"*": {}
}
}
}
Search Result:
"hits": [
{
"_index": "65325154",
"_type": "_doc",
"_id": "1",
"_score": 0.5753642,
"_source": {
"name": "dilo abinin",
"surname": "sürücü"
},
"highlight": {
"name": [ // note this
"<em>dilo</em> <em>abinin</em>"
],
"name.keyword": [
"<em>dilo abinin</em>"
]
}
}
]
I have documents in elasticsearch index with a "type" field, like this:
[
{
"id": 1,
"serviceDescription": "a bunch of text",
"serviceTitle": "title",
"serviceTags":["tag1","tag2"]
"type":"service"
},
{
"id": 2,
"companyDescription": "a bunch of text more",
"companyTitle": "title",
"companyTags":["tag1","tag2"]
"type":"company"
},...
]
I want to run a match query across all docs in my index, like this:
body = {
"query": {
"match": {
"_all":"sequencing"
}
}
}
but add a filter to only return results where the "type" field equals "service".
As far as I can understand your question, you want to query for sequencing query string, across all the fields, for that
you can use the multi_match query that builds on the match query to allow multi-field queries.
If no fields are provided, the multi_match query defaults to the
index.query.default_field index settings, which in turn defaults to *.
This extracts all fields in the mapping that are eligible to term queries and filters the metadata fields. All extracted fields are then
combined to build a query.
Search Query:
{
"query": {
"bool": {
"must": [
{
"multi_match": {
"query": "bunch of text"
}
}
],
"filter": {
"term": {
"type": "service"
}
}
}
}
}
Search Result:
"hits": [
{
"_index": "64867032",
"_type": "_doc",
"_id": "1",
"_score": 0.8630463,
"_source": {
"id": 1,
"serviceDescription": "a bunch of text",
"serviceTitle": "title",
"serviceTags": [
"tag1",
"tag2"
],
"type": "service"
}
}
]
I've been trying to add some fuzziness to my search as you type field type on Elasticsearch, but never got the needed query. Anyone have any idea to implement this?
Fuzzy Query returns documents that contain terms similar to the search term, as measured by a Levenshtein edit distance.
The fuzziness parameter can be specified as:
AUTO -- It generates an edit distance based on the length of the term.
For lengths:
0..2 -- must match exactly
3..5 -- one edit allowed Greater than 5 -- two edits allowed
Adding working example with index data and search query.
Index Data:
{
"title":"product"
}
{
"title":"prodct"
}
Search Query:
{
"query": {
"fuzzy": {
"title": {
"value": "prodc",
"fuzziness":2,
"transpositions":true,
"boost": 5
}
}
}
}
Search Result:
"hits": [
{
"_index": "test",
"_type": "_doc",
"_id": "1",
"_score": 2.0794415,
"_source": {
"title": "product"
}
},
{
"_index": "test",
"_type": "_doc",
"_id": "2",
"_score": 2.0794415,
"_source": {
"title": "produt"
}
}
]
Refer these blogs to get a detailed explaination on fuzzy query
https://www.elastic.co/blog/found-fuzzy-search
https://qbox.io/blog/elasticsearch-optimization-fuzziness-performance
Update 1:
Refer this ES official documentation
The fuzziness , prefix_length , max_expansions , rewrite , and
fuzzy_transpositions parameters are supported for the terms that are
used to construct term queries, but do not have an effect on the
prefix query constructed from the final term.
There are some open issues and discuss links that states that - Fuzziness not work with bool_prefix multi_match (search-as-you-type)
https://github.com/elastic/elasticsearch/issues/56229
https://discuss.elastic.co/t/fuzziness-not-work-with-bool-prefix-multi-match-search-as-you-type/229602/3
I know this question is asked long ago but I think this worked for me.
Since Elasticsearch allows a single field to be declared with multiple data types, my mapping is like below.
PUT products
{
"mappings": {
"properties": {
"title": {
"type": "text",
"fields": {
"product_type": {
"type": "search_as_you_type"
}
}
}
}
}
}
After adding some data to the index I fetched like this.
GET products/_search
{
"query": {
"bool": {
"should": [
{
"multi_match": {
"query": "prodc",
"type": "bool_prefix",
"fields": [
"title.product_type",
"title.product_type._2gram",
"title.product_type._3gram"
]
}
},
{
"multi_match": {
"query": "prodc",
"fuzziness": 2
}
}
]
}
}
}
I have a fuzzy query with customized AUTO:10,20 fuzziness value.
{
"query": {
"match": {
"name": {
"query": "nike",
"fuzziness": "AUTO:10,20"
}
}
}
}
How to convert it to a query_string query? I tried nike~AUTO:10,20 but it is not working.
It's possible with query_strng as well, let me show using the same example as OP provided, both match_query provided by OP matches and query_string fetches the same document with same score.
And according to this and this ES docs, Elasticsearch supports AUTO:10,20 format, which is shown in my example as well.
Also
Index mapping
{
"mappings": {
"properties": {
"name": {
"type": "text"
}
}
}
}
Index some doc
{
"name" : "nike"
}
Search query using match with fuzziness
{
"query": {
"match": {
"name": {
"query": "nike",
"fuzziness": "AUTO:10,20"
}
}
}
}
And result
"hits": [
{
"_index": "so-query",
"_type": "_doc",
"_id": "1",
"_score": 0.9808292,
"_source": {
"name": "nike"
}
}
]
Query_string with fuzziness
{
"query": {
"query_string": {
"fields": ["name"],
"query": "nike",
"fuzziness": "AUTO:10,20"
}
}
}
And result
"hits": [
{
"_index": "so-query",
"_type": "_doc",
"_id": "1",
"_score": 0.9808292,
"_source": {
"name": "nike"
}
}
]
Lucene syntax only allows you to specify "fuzziness" with the tilde symbol "~", optionally followed by 0, 1 or 2 to indicate the edit distance.
Elasticsearch Query DSL supports a configurable special value for AUTO which then is used to build the proper Lucene query.
You would need to implement that logic on your application side, by evaluating the desired edit distance based on the length of your search term and then use <searchTerm>~<editDistance> in your query_string-query.
I have a short question which seems to be simple, but I wasn't able to find any answer so far.
I want to retrieve on an Elasticsearch node, the last document given to a date field. But I want to have the last document, only for documents which contains a specific field.
For instance, let's say I want to get the last purchase which contains the field "promotionCode" :
Query :
http://elasticsearch:9200/store1/purchase/_search?q=vendor:Marie&size=1&sort=date:desc
where store1 is my index, purchase a document type.
Now let's say I have these two documents in my ElasticSearch :
"hits": [
{
"_index": "store1",
"_type": "purchase",
"_id": "1",
"_score": 1,
"_source": {
"date": "2016-03-16T12:53:16.000Z",
"vendor": "Marie",
"promotionCode": "XYZ123"
}
},
{
"_index": "store1",
"_type": "purchase",
"_id": "2",
"_score": 1,
"_source": {
"date": "2016-03-18T12:53:16.000Z",
"vendor": "Marie"
}
}
]
The above query will retrieve the document of id 2, but I will not have any field "promotionCode" in my result.
If I want to get the last document, containing a specific field, how do I do ?
I explored "fields" filter, but it only send back void document if the field is not contained, and I read about Source filtering but not sure it is doing what I want ...
Thanks a lot for any hint !
Yo can try with this query:
{
"query": {
"term": { "vendor": "Marie" }
},
"filter": {
"bool": {
"must_not": { "missing": { "field": "promotionCode" } }
}
},
"sort": { "date" : "desc" },
"size": 1
}
You can use Exists Query
GET /store1/purchase/_search?q=vendor:Marie&size=1&sort=date:desc
{
"query": {
"exists" : {
"field" : "promotionCode"
}
}
}
Hope it helps!!