Ignore term frequency but use positions - elasticsearch

Question

I have an index with a text field and i want to ignore term frequencies in scoring but keep positions to have match phrase search ability.
and my index define like this:
curl --location --request PUT 'localhost:9200/my-index-001' \
--header 'Content-Type: application/json' \
--data-raw '{
"mappings": {
"autocomplete": {
"properties": {
"title": {
"type": "text",
"analyzer": "row_autocomplete"
},
"name": {
"type": "text",
"analyzer": "row_autocomplete"
}
}
}
},
"settings": {
"analysis": {
"analyzer": {
"row_autocomplete": {
"tokenizer": "icu_tokenizer",
"filter": ["icu_folding", "autocomplete_filter", "lowercase"]
}
},
"filter": {
"autocomplete_filter": {
"type": "edge_ngram",
"min_gram": 1,
"max_gram": 20
}
}
}
}
}'
index data:
[
{
"title": "university",
"name": "london and EC london English"
},
{
"title": "city",
"name": "london"
}
]
I want city get high score when I execute match query like this:
POST _search
{
"query": {
"bool": {
"should": [
{
"match": {
"name": {
"query": "london"
}
}
},
{
"match_phrase": {
"name": {
"query": "london",
}
}
}
]
}
}
}
and they got different score(university is greater than city actually) because of term frequency, What I want is only count term frequency one time, and according to fieldLength, city's fieldLength is less than university's fieldLength, so if I can ignore repeat termFreq, the score of city will greater than university refer to elasticsearch's rule:
GET _explain
# city's _explain
{
"value": 2.0785222,
"description": "tfNorm, computed as (freq * (k1 + 1)) / (freq + k1 * (1 - b + b * fieldLength / avgFieldLength)) from:",
"details": [
{
"value": 6.0,
"description": "termFreq=6.0",
"details": []
},
{
"value": 2.0,
"description": "fieldLength",
"details": []
},
...
]
}
# university's explain
{
"value": 2.1087635,
"description": "tfNorm, computed as (freq * (k1 + 1)) / (freq + k1 * (1 - b + b * fieldLength / avgFieldLength)) from:",
"details": [
{
"value": 24.0,
"description": "termFreq=24.0",
"details": []
},
{
"value": 29.0,
"description": "fieldLength",
"details": []
},
...
]
}
There are something I tried, for example In index mapping i can set index_options=docs to ignore term frequencies but this disables term positions and i cant use match phrase query anymore.
Does anyone have any idea?
thanks in advance.

Answer 1

You can use constant score query that wraps a filter query and returns every matching document with a relevance score equal to the boost parameter value.
If you use a constant score query, then your match query will give no score apart from just 0 or 1. This is because it will just act like a filter that will just if the query matches or not. match query will not act like a matching based on full-text search.
A constant_score query takes a boost argument that is set as the score
for every returned document when combined with other queries. By
default boost is set to 1.
Refer this to get detailed explaination on bool filter and this SO answer to understand difference between constant score query and bool filter.
Adding a working example with index data, search query and search result.
Index data
{ "name": "london only" }
{ "name": "london and London" }
Search Query:
{
"query": {
"constant_score": {
"filter": {
"bool": {
"must": [
{
"match": {
"name": "london"
}
}
]
}
}
}
}
}
Search Result:
"hits": [
{
"_index": "my_index",
"_type": "_doc",
"_id": "1",
"_score": 1.0,
"_source": {
"name": "london and london"
}
},
{
"_index": "my_index",
"_type": "_doc",
"_id": "2",
"_score": 1.0,
"_source": {
"name": "london only"
}
}
]

Answer 2

I indexed both the sample documents what you provided using default index mapping, so both title and name fields are text fields. and used same your query and it returns me the high score for just doc which contains just london as shown below:
"hits": [
{
"_index": "matchrel",
"_type": "_doc",
"_id": "1",
"_score": 0.51518387,
"_source": {
"title": "city",
"name": "london"
}
},
{
"_index": "matchrel",
"_type": "_doc",
"_id": "2",
"_score": 0.41750965,
"_source": {
"title": "university",
"name": "london university and EC London English"
}
}
]
Also, As you have not explained your use case in details, and with limited info, It seems it can be easily achieved with below query with also returns more score for london doc:
{
"query": {
"match_phrase": {
"name": "london"
}
}
}
And its search result
"hits": [
{
"_index": "matchrel",
"_type": "_doc",
"_id": "1",
"_score": 0.25759193, // note score
"_source": {
"title": "city",
"name": "london"
}
},
{
"_index": "matchrel",
"_type": "_doc",
"_id": "2",
"_score": 0.20875482,
"_source": {
"title": "university",
"name": "london university and EC London English"
}
}
]