Elasticsearch changing similarity does not work - elasticsearch

Question

Changing the similarity algorithm of my index does not work. I wan't to compare BM25 vs. TF-IDF, but i always get the same results. I'm using Elasticsearch 5.x.
I have tried literally everything. Setting the similarity of a property to classic or BM25 or don't set anything
"properties": {
"content": {
"type": "text",
"similarity": "classic"
},
I also tried setting the default similarty of my index in the settings and using it in the properties
"settings": {
"index": {
"number_of_shards": "5",
"provided_name": "test",
"similarity": {
"default": {
"type": "classic"
}
},
"creation_date": "1493748517301",
"number_of_replicas": "1",
"uuid": "sNuWcT4AT82MKsfAB9JcXQ",
"version": {
"created": "5020299"
}
}
The query im testing looks something like this:
{
"query": {
"match": {
"content": "some search query"
}
}
}

Answer 1

I have created a sample below:
DELETE test
PUT test
{
"mappings": {
"book": {
"properties": {
"content": {
"type": "text",
"similarity": "BM25"
},
"subject": {
"type": "text",
"similarity": "classic"
}
}
}
}
}
POST test/book/1
{
"subject": "A neutron star is the collapsed core of a large (10–29 solar masses) star. Neutron stars are the smallest and densest stars known to exist.[1] Though neutron stars typically have a radius on the order of 10 km, they can have masses of about twice that of the Sun.",
"content": "A neutron star is the collapsed core of a large (10–29 solar masses) star. Neutron stars are the smallest and densest stars known to exist.[1] Though neutron stars typically have a radius on the order of 10 km, they can have masses of about twice that of the Sun."
}
POST test/book/2
{
"subject": "A quark star is a hypothetical type of compact exotic star composed of quark matter, where extremely high temperature and pressure forces nuclear particles to dissolve into a continuous phase consisting of free quarks. These are ultra-dense phases of degenerate matter theorized to form inside neutron stars exceeding a predicted internal pressure needed for quark degeneracy.",
"content": "A quark star is a hypothetical type of compact exotic star composed of quark matter, where extremely high temperature and pressure forces nuclear particles to dissolve into a continuous phase consisting of free quarks. These are ultra-dense phases of degenerate matter theorized to form inside neutron stars exceeding a predicted internal pressure needed for quark degeneracy."
}
GET test/_search?explain
{
"query": {
"match": {
"subject": "neutron"
}
}
}
GET test/_search?explain
{
"query": {
"match": {
"content": "neutron"
}
}
}
subject and content fields have different similarities definitions but in the two documents I provided (from wikipedia) they have the same text in them. Running the two queries you will see in the explanations something like this and also get different scores in results:
from the first query: "description": "idf, computed as log((docCount+1)/(docFreq+1)) + 1 from:"
from the second one: "description": "idf, computed as log(1 + (docCount - docFreq + 0.5) / (docFreq + 0.5)) from:",