I have a scenario where i want to search for 'bank of india' and documents retrieved have hits for 'reserve bank of india', 'state bank of india', etc. Basically the search string named entity is part of another named entity as well.
What are the ways to avoid it in elasticsearch?
If you use keyword type instead of text as the mapping for your entity field you will no longer have those partial matches. keyword says treat this text like a single unit (named entities are like this), while text says treat each word as a unit and consider the field as a bag of words, So the query looks for the most word matches, regardless of order or if all of the words are there. There are different queries that can get at that requiring order (match_phrase) and requiring all words to be matches (minimum_should_match parameter), but I like to use the term query if you follow the keyword mapping strategy. Does that make sense?
Related
Let's say I have two texts:
Text 1 - "The fox has been living in the wood cabin for days."
Text 2 - "The wooden hammer is a dangerous weapon."
And I would like to search for the word "wood", without it matching me "wooden hammer". How would I do that in Elastic Search or nest?
Term query is used for exact matches search. However it's not recommended to use it against text fields, the following quote from term query documentation:
To better search text fields, the match query also analyzes your
provided search term before performing a search. This means the match
query can search text fields for analyzed tokens rather than an exact
term.
The term query does not analyze the search term. The term query only
searches for the exact term you provide. This means the term query may
return poor or no results when searching text fields.
The problem with text exact matches, as described in the Term query documentation:
By default, Elasticsearch changes the values of text fields as part of
analysis. This can make finding exact matches for text field values
difficult.
So, the documents data is modified (i.e., analyzed) before indexing. This depends on the index mapping definition for each field, defaults to the default index analyzer, or the standard analyzer.
But the default standard analyzer will not change the token "Wooden" to "Wood", this might happen if you used stemming for this field.
This means, if you don't use a different analyzer or stemming, querying with "Wood" shouldn't match "Wooden" token.
To summarize: Indexed data is modified/analyzed before indexing (based on the field mapping definition). Match query analyze the search query, while Term query doesn't analyze the search query. So you have to properly chose the field mapping and the search query to better suit your use case
For some use cases, like storing email addressed, phone numbers or keyword fields that always have the same value, consider using the Keyword type, which is suitable for exact matches in these use cases. However, ES recommends:
Avoid using keyword fields for full-text search. Use the text field
type instead.
So for better visibility and practical solution for your use case, it's better to elaborate more the field mapping you use and what you want to achieve.
Being new to ElasticSearch, need help in my understanding.
What I read about term vs match query is that term query is used for exact match and match query is used when we are searching for a term and want result based on a relevancy score.
But if we already defined a mapping for a field as a keyword, why anyone has to decide upon between term vs match, wouldn't it be always a term query in case mapping is defined as a keyword?
What are the use cases where someone will make a match query on the keyword mapping field?
The same confusion is vice versa.
A text field will be analyzed (transformed, split) to generate N tokens, and the keyword itself will become a token with no transformations. At the end, you have N tokens referencing a document.
Then.
By doing a match query, you will treat your query as a text as well, by analyzing it before performing the matching (transforming it), and the term will not.
You can create a field with a term mapping, but then perform a match query on top of it (for example if you want to be case insensitive), and you can create a text mapping for a n-gram and perform a term query to match exactly what you're asking for.
I am using Elasticsearch to allow a user to type in a term to search. I have the following property 'name' I'd like to search, for instance:
'name': 'The car is black'
I'd like to have this document returned if the following is used to search black car or car black.
I've tried doing a bool must and doing multiple terms ['black', 'car'] but it seems like it only works if the entire string is a match.
So what I'd really like to do is more of a, does the term contain both words in any order.
Can someone please get me on the right track? I've been banging my head on this one for a while.
If it seems like it only works if the entire string is a match, first make sure that in index mapping your string property name is analysed, i.e. mapping for this property doesn't contain "index": "not_analyzed". If it isn't so, you'll need to reindex your index in order to be able to search for tokens rather than for the whole phrase only.
Once you're sure your strings are analysed you can use:
Terms query with "minimum_should_match" parameter equalling to the number of words entered.
Bool query with must clause containing term queries per each word.
Common terms query which has a nice clean syntax for this purpose (you don't need to break down input string and construct more complex query structure in your app like with previous two) in addition to taking a smarter approach to stopwords analysing.
How do you return a match for facets that begin with a certain letter? For instance, Elasticsearch suggests you use the query "T*" to return all titles that begin with the letter T. However, this seems to return titles that contain any words that begin with the letter T. For example, "Trees and Shrubs" as well as "How to Prune Trees." I only want to return "Trees and Shrubs". I tried using "^T*", but that returns SearchPhraseExecutionException.
By default, elasticsearch analyzes the fields using the default analyzer, which splits the field up into tokens split generally on English-language word boundaries. So "How to Prune Trees" gets split into ("How", "to", "Prune", "Trees") and the token "Trees" subsequently matches your query, "T*". To avoid this, you have to specify that the field is not_analyzed in the object mapping. This makes elasticsearch effectively treat the field value as one giant token, which it will then match correctly against your prefix query.
You might also check out the prefix query, but this is effectively just another way of issuing the same query. You still need to specify that the field is not analyzed in the mapping.
it is too simple to describe:
q=mydynamicfield_txt:"video"
I want only hits when mydynamicfield is exact "video.
Other way round, how to supress hits, where "video" is only part of the field (like "home video").
Is this supported with Solr3.1 out of the box, or do I have to add my own special brackets like "SOLRSTARTSOLR video SOLRENDSOLR" in my index, to retrieve later my term between "START" and "END". Kind of manual regex anchoring.
This is PITA cause it needs special handling in index/gui and breaks highlighting.
Where is the way to go?
regards
Peter
(=PA=)
One solution to create untokenized(KeywordAnalyzed) field and search within it - all your text will be distinct token in Solr index.
Other solution is to write filter which will read token count from index and compare to query tokens i.e. filter entities where doc_tokens > query_tokens assuming that all query tokens are matched.