I have an index that has several title fields.
main_title,
sub_titles,
preferred_titles
etc.
These texts fields also have a suggest field each where I run a custom analyzer that uses edge-n-gram tokenizer so we can search as we type.
I would like to value exact match over term frequency. And exact match in main_title is worth more than exact match in preferred_titles.
Anyone know how I can achieve this? Thanks in advance.
I have tried a bool_query with multi_match_query in the must clause. The multi_match is crossfields with no fields attached with the operator 'and'.
I have both the text fields and the suggest fields in the should cluase. Each text field is in a match_query with a boost and the operator 'and'. Each suggest field is in a match_phrase_query with a boost and the operator 'and'. The issue is that several boosts are added on top of the scores and I end up with very inflated scores.
Related
Let's say I have two texts:
Text 1 - "The fox has been living in the wood cabin for days."
Text 2 - "The wooden hammer is a dangerous weapon."
And I would like to search for the word "wood", without it matching me "wooden hammer". How would I do that in Elastic Search or nest?
Term query is used for exact matches search. However it's not recommended to use it against text fields, the following quote from term query documentation:
To better search text fields, the match query also analyzes your
provided search term before performing a search. This means the match
query can search text fields for analyzed tokens rather than an exact
term.
The term query does not analyze the search term. The term query only
searches for the exact term you provide. This means the term query may
return poor or no results when searching text fields.
The problem with text exact matches, as described in the Term query documentation:
By default, Elasticsearch changes the values of text fields as part of
analysis. This can make finding exact matches for text field values
difficult.
So, the documents data is modified (i.e., analyzed) before indexing. This depends on the index mapping definition for each field, defaults to the default index analyzer, or the standard analyzer.
But the default standard analyzer will not change the token "Wooden" to "Wood", this might happen if you used stemming for this field.
This means, if you don't use a different analyzer or stemming, querying with "Wood" shouldn't match "Wooden" token.
To summarize: Indexed data is modified/analyzed before indexing (based on the field mapping definition). Match query analyze the search query, while Term query doesn't analyze the search query. So you have to properly chose the field mapping and the search query to better suit your use case
For some use cases, like storing email addressed, phone numbers or keyword fields that always have the same value, consider using the Keyword type, which is suitable for exact matches in these use cases. However, ES recommends:
Avoid using keyword fields for full-text search. Use the text field
type instead.
So for better visibility and practical solution for your use case, it's better to elaborate more the field mapping you use and what you want to achieve.
Being new to ElasticSearch, need help in my understanding.
What I read about term vs match query is that term query is used for exact match and match query is used when we are searching for a term and want result based on a relevancy score.
But if we already defined a mapping for a field as a keyword, why anyone has to decide upon between term vs match, wouldn't it be always a term query in case mapping is defined as a keyword?
What are the use cases where someone will make a match query on the keyword mapping field?
The same confusion is vice versa.
A text field will be analyzed (transformed, split) to generate N tokens, and the keyword itself will become a token with no transformations. At the end, you have N tokens referencing a document.
Then.
By doing a match query, you will treat your query as a text as well, by analyzing it before performing the matching (transforming it), and the term will not.
You can create a field with a term mapping, but then perform a match query on top of it (for example if you want to be case insensitive), and you can create a text mapping for a n-gram and perform a term query to match exactly what you're asking for.
The Wildcard Query offers a rewrite parameter to influence how Lucene calculates the relevance scores. On keyword fields this works as expected but it does not seem to work with the new wildcard field type which belongs to the keyword family. Is this an expected behavior or a bug?
As confirmed by Elastic staff, the rewrite parameter is unsupported. Unlike keyword fields, the wildcard field doesn't have a single indexed token for each term so it has no pre-built count for the document frequency of whole values. Instead it uses an ngram index which obviously has different frequencies for the multiple terms a search string can be broken down into.
We have an ES index which has a field which stores its data as an array. In this field, we include the original text, plus text without any punctuation, special characters, etc. The problem is, when searching on the field, the multiple values appears to be skewing the score.
For example, if we search on the term 'up', the document which has the array ['up, up and away', 'up up and away'] is scoring higher with a multi_match (we are using because we may search more than one field) than the document with the array as simply ['up'].
In the end, I guess what I am looking for is a score that emulates calculating a score for each item in the array and returning me the highest. I believe in this case, comparing 'up' to 'Up' and 'Up, Up and Away' will give me a higher score for 'Up'.
With my research, I believe I may need to do custom scoring on this field...? If that is true, am I looking at "score_mode": "max" as what I want?
I think you slightly over-engineered your index. You don't need to create duplicate fields for the same information and remove punctuation, lowercase fields yourself.
I'd recommend you to read what are elasticsearch token filters and how to create multiple analyzers for the same field.
For your exact use case, if you provided a document sample, it would certainly help. But in any case looking at what you are dealing with - index your array of strings with default analyzer and with a custom one that you'll build yourself. Then you can use the same field, but with different analyzers (differently processed text) to control your score.
What is the scenario when you would need to use a mapping of Keyword-Analyzer compared to marking it as not_analyzed with Doc values turned on for the field.
From the elasticsearch documentation, it seems that if a field is not analyzed , then it is better to turn doc_Values on for this field.
The elasticsearch documentation also states specifically that [sic] Note, when using mapping definitions, it might make more sense to simply mark the field as not_analyzed
I am a bit confused as to why the Keyword Analyzer will ever be used ?
According to a core committer, both are equivalent.
That wouldn't be the case of the keyword tokenizer, though, which can be combined with other filters (lowercase, etc) and thus participate in many different ways of tokenizing your input.
I had the exact usecase for keyword analyzer, analyzers allow you to add character filters in addition to common filters and that's where they are used.
I had a field containing numeric values including whitespaces. Numbers may be persian , for ex ۱۲۳۴۵۶۷۸۹ or English, so i needed a char filter to normalize them to english ones with a char filter without any other tokenizing process, when I marked the field not_analyzed I was unable to use my char filter.