I have an index called find and a type called song.
Song type structure :
"_index": "find",
"_type": "song",
"_id": "192108",
"_source": {
"id": 192108,
"artist": "Melanie",
"title": "Dark Night",
"lyrics": "Hot air hangs like a dead man\nFrom a white oak tree",
"downloadCount": 234
}
Because of multiple songs maybe has same field values, so I need to boost results by a popularity field such as downloadCount.
How can i change below query to optimize by downloadCount?
GET /search/song/_search
{
"query": {
"multi_match": {
"query": "like a dead hangs",
"type": "most_fields",
"fields": ["artist","title","lyrics"],
"operator": "or"
}
}
}
You can use field_value_factor feature of elastic_search to boost the result by downloadCount
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-function-score-query.html#function-field-value-factor
you can use function score query. Function score query provides api for scoring the document based on the document field through script_score functions.
{
"query": {
"function_score": {
"query": {
"bool": {
"must": [{
"term": {
"you_filter_field": {
"value": "VALUE"
}
}
}]
}
},
"functions": [{
"script_score": {
"script": "doc['downloadCount'].value"
}
}]
}
}
}
Thanks
Related
I'd like to implement a keyword blacklist using ElasticSearch. Basically I want to create a list of banned queries that a user is not allowed to search for. Then I want to be able to pass in a checked query and see which banned queries it matches (if any).
A checked query matches a banned query if the banned query has a subset of its keywords. To illustrate, let me provide an example:
Banned Queries:
"black lives"
"black lives matter"
"black lives
matters"
"black lives matter rulez"
Checked Query: "black lives
matter"
Matches:
"black lives"
"black lives matter"
Only the first two banned queries match, because they're strict subsets of the checked query. The third banned query doesn't match because it uses "matters", not "matter". The last banned query doesn't match because it isn't a strict subset of "black lives matter", because it has an additional keyword "rulez".
I've been told that the best way to implement this is a percolate index. My question is how do I create a percolate query that implements a subset match against a checked query (the incoming document)?
Here is the documentation page about percolate queries: https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-percolate-query.html
Here is a related answer about subset matching:
https://discuss.elastic.co/t/subset-in-an-array/237459
The best way to achieve your use case is to use Percolate query
Adding a working example with index data, mapping, search query, and search result
Index Mapping:
{
"mappings": {
"properties": {
"field": {
"type": "text"
},
"query": {
"type": "percolator"
}
}
}
}
Index Data:
{
"query": {
"match": {
"field": {
"query": "black lives matter rulez",
"operator": "AND"
}
}
}
}
{
"query": {
"match": {
"field": {
"query": "black lives matters",
"operator": "AND"
}
}
}
}
{
"query": {
"match": {
"field": {
"query": "black lives matter",
"operator": "AND"
}
}
}
}
{
"query": {
"match": {
"field": {
"query": "black lives",
"operator": "AND"
}
}
}
}
Search Query:
{
"query": {
"percolate": {
"field": "query",
"document": {
"field": "black lives matter"
}
}
}
}
Search Result:
"hits": [
{
"_index": "68734373",
"_type": "_doc",
"_id": "2",
"_score": 0.39229372,
"_source": {
"query": {
"match": {
"field": {
"query": "black lives matter",
"operator": "AND"
}
}
}
},
"fields": {
"_percolator_document_slot": [
0
]
}
},
{
"_index": "68734373",
"_type": "_doc",
"_id": "1",
"_score": 0.26152915,
"_source": {
"query": {
"match": {
"field": {
"query": "black lives",
"operator": "AND"
}
}
}
},
"fields": {
"_percolator_document_slot": [
0
]
}
}
]
I would like to do a search in an elasticsearch index but only for a list of ids. I can select the ids with a terms query
{
"query": {
"terms": {
"_id": list_of_ids
}
}
}
Now I want to search in the resulting list, which can be done with a query like this
{
"query": {
"bool": {
"must": {}
}
}
}
My question is how can I combine those two queries?
One solution I found is to add the ids into the must query like this
{
"query": {
"bool": {
"must": {}
"should": [{
"term": {
"_id": id1
},
"term": {
"_id": id2
}]
}
}
}
}
which works fine. However, if the list of ids is very large it can lead to errors.
elasticsearch.exceptions.RequestError: RequestError(400, 'search_phase_execution_exception', 'failed to create query:
I am wondering whether there is a more compact way to write such a query? I think the error above is caused by my query just being too long since I added thousands of term searches... there must be a way to just provide an array, like in the terms query?
solved it
{
"query": {
"bool": {
"must": {},
"filter": {
"terms": {
"_id": list_of_ids
}
}
}
}
}
sorry I am a bit of a newbie to elasticsearch...
You can also use IDs query, which returns documents based on their IDs.
Adding a working example with index data, search query, and search result.
Index Data:
{
"name":"buiscuit",
"cost":"55",
"discount":"20"
}
{
"name":"multi grain bread",
"cost":"55",
"discount":"20"
}
Search Query:
{
"query": {
"bool": {
"must": {
"match": {
"name": "bread"
}
},
"filter": {
"ids": {
"values": [
"1",
"2",
"4"
]
}
}
}
}
}
Search Result:
"hits": [
{
"_index": "65431114",
"_type": "_doc",
"_id": "1",
"_score": 0.5754429,
"_source": {
"name": "multi grain bread",
"cost": "55",
"discount": "20"
}
}
]
I'm trying to use the match_phrase_prefix query with an exclude query, so that it matches all terms except for the terms to be exclude. I have it figured out in a basic URI query, but not the regular JSON query. How do I convert this URI into a JSON type query?
"http://127.0.0.1:9200/topics/_search?q=name:"
+ QUERY + "* AND !name=" + CURRENT_TAGS
Where CURRENT_TAGS is a list of tags not to match with.
This is what I have so far:
{
"query": {
"bool": {
"must": {
"match_phrase_prefix": {
"name": "a"
}
},
"filter": {
"terms": {
"name": [
"apple"
]
}
}
}
}
}
However, when I do this apple is still included in the results. How do I exclude apple?
You are almost there, you can use must_not, which is part of boolean query to exclude the documents which you don't want, below is working example on your sample.
Index mapping
{
"mappings": {
"properties": {
"name": {
"type": "text"
}
}
}
}
Index sample docs as apple and amazon worlds biggest companies which matches your search criteria :)
Search query to exclude apple
{
"query": {
"bool": {
"must": {
"match_phrase_prefix": {
"name": "a"
}
},
"must_not": {
"match": {
"name": "apple"
}
}
}
}
}
Search results
"hits": [
{
"_index": "matchprase",
"_type": "_doc",
"_id": "2",
"_score": 0.6931471,
"_source": {
"name": "amazon"
}
}
]
I'm new to Elasticsearch and would greatly appreciate help on this
In the query below I only want the first document to be returned, but instead both documents are returned. How can I write a query to search for two wildcard strings on two separate fields, but only return documents that match?
I think what's being returned currently is score dependent, but I don't need the score.
POST /pr/_doc/1
{
"type": "Type ONE",
"currency":"USD"
}
POST /pr/_doc/2
{
"type": "Type TWO",
"currency":"USD"
}
GET /pr/_search
{
"query": {
"bool": {
"must": [
{
"simple_query_string": {
"query": "Type ON*",
"fields": ["type"],
"analyze_wildcard": true
}
},
{
"simple_query_string": {
"query": "US*",
"fields": ["currency"],
"analyze_wildcard":true
}
}
]
}
}
}
Use below query which uses the default_operator: AND and query string for in depth information and further reading.
Search query
{
"query": {
"query_string": {
"query": "(Type ON*) AND (US*)",
"fields" : ["type", "currency"],
"default_operator" : "AND"
}
}
}
Index your sample docs and it returns your expected doc only:
"hits": [
{
"_index": "multiplequery",
"_type": "_doc",
"_id": "1",
"_score": 2.1823215,
"_source": {
"type": "Type ONE",
"currency": "USD"
}
}
]
Considering the Elasticsearch query below, I was wondering if it is possible to have the score in percentage (0-100%), or how could I calculate it in such way, and have it only based on the multi_match part of the query, where fuzziness is enabled.
Therefore, I would like the score to ignore the filter part of it.
Thanks in advance.
{
"index": "myindex",
"type": "mytype",
"body": {
"_source": [
"author_mt",
...
"title_t",
],
"from": 0,
"size": 100,
"query": {
"bool": {
"must": {
"multi_match": {
"query": "test",
"fields": [
"title*"
],
"fuzziness": "AUTO"
}
},
"filter": {
"bool": {
"must": [
{
"term": {
"genre_t_s": 'test'
}
}
]
}
}
}
}
}
}
The elasticsearch score is based on the TF/IDF algorithm, therefore the score can be grater than 1(or 100%). Check the following link what is relevance:
https://www.elastic.co/guide/en/elasticsearch/guide/current/relevance-intro.html