For example, judgement and judgment are actually the same, but there is a difference in the spelling. But when the query is
{
"query":{
"match_phrase":{
"name":{
"query":"Judgement"
}
}
}
}
Or
{
"query":{
"match_phrase":{
"name":{
"query":"Judgment"
}
}
}
}
I should get the same result. How can I achieve this? Thank you.
You can use the fuzzy query to achieve your use-case, there are several params to customize the default behaviour of it.
Related
GET qnaindexfinal/_search
{
"query":{
"bool":{
"must":[
{
"common":{
"question.questionText":{
"query":"showrroom",
"cutoff_frequency":0.001
}
}
}
],
"filter":[
{
"term":{
"modelId":{
"value":78
}
}
}
]
}
}
}
Please help me with the above dsl.
With queries in elasticsearch it's best to break things down a little.
common same as a term but for more than one keyword
term value == 78
must in this context is checking that the only documents returned match both common and term.
I am using elasticsearch 6 and have the following query
{
"query":{
"bool":{
"should":[
{
"match_phrase":{
"fieldOne":{
"query":"One two three",
"slop":10
}
}
},
{
"match_phrase":{
"fieldTwo":{
"query":"one two three",
"slop":10
}
}
}
]
}
}
}
This works well when I want to match on the two fields with the terms in the query.
However if I have a document which has term 'one' and 'two' in fieldOne the above does not return results as 'three' is required
I cannot seems to find a way of making the terms in the query optional e.g. what I wanted is to say find any of the terms in those two fields
The reason I went with match_phrase is the use of the slop which allows the terms to be in different positions in the field which i also require
if the order is not important to use, you don't need to use match_phrase, a simple match query does the job
{
"match":{
"fieldOne":{
"query":"one two three"
}
}
},
Then if you need at least two terms to match you can do so using minimum_should_match:
{
"match":{
"fieldOne":{
"query":"one two three",
"minimum_should_match": 2
}
}
},
I am very new to Elasticsearch and I have to perform the following query:
GET book-lists/book-list/_search
{
"query":{
"filtered":{
"filter":{
"bool":{
"must":[
{
"term":{
"title":"Sociology"
}
},
{
"term":{
"idOwner":"17xxxxxxxxxxxx45"
}
}
]
}
}
}
}
}
According to the Elasticsearch API, it is equivalent to pseudo-SQL:
SELECT document
FROM book-lists
WHERE title = "Sociology"
AND idOwner = 17xxxxxxxxxxxx45
The problem is that my document looks like this:
{
"_index":"book-lists",
"_type":"book-list",
"_id":"AVBRSvHIXb7carZwcePS",
"_version":1,
"_score":1,
"_source":{
"title":"Sociology",
"books":[
{
"title":"The Tipping Point: How Little Things Can Make a Big Difference",
"isRead":true,
"summary":"lorem ipsum",
"rating":3.5
}
],
"numberViews":0,
"idOwner":"17xxxxxxxxxxxx45"
}
}
And the Elasticsearch query above doesn't return anything.
Whereas, this query returns the document above:
GET book-lists/book-list/_search
{
"query":{
"filtered":{
"filter":{
"bool":{
"must":[
{
"term":{
"numberViews":"0"
}
},
{
"term":{
"idOwner":"17xxxxxxxxxxxx45"
}
}
]
}
}
}
}
}
This makes me suspect that the fact that "title" is the same name for the two fields is for something.
Is there a way to fix this without having to rename any of the fields. Or am I missing it somewhere else?
Thanks for anyone trying to help.
Your problem is described in the documentation.
I suspect that you don't have any explicit mapping on your index, which means elasticsearch will use dynamic mapping.
For string fields, it will pass the string through the standard analyzer which lowercases it (among other things). This is why your query doesn't work.
Your options are:
Specify an explicit mapping on the field so that it isn't analyzed before storing in the index (index: not_analyzed).
Clean your term query before sending it to elasticsearch (in this specific query lowercasing will work, but note that the standard analyzer also does other things like remove stop words, so depending on the title you may still have issues).
Use a different query type (e.g., query_string instead of term which will analyze the query before running it).
Looking at the sort of data you are storing you probably need to specify an explicit not_analyzed mapping.
For option three your query would look something like this:
{
"query":{
"filtered":{
"filter":{
"bool":{
"must":[
{
"query_string":{
"fields": ["title"],
"analyzer": "standard",
"query": "Sociology"
}
},
{
"term":{
"idOwner":"17xxxxxxxxxxxx45"
}
}
]
}
}
}
}
}
Note that the query_string query has special syntax (e.g., OR and AND are not treated as literals) which means you have to be careful what you give it. For this reason explicit mapping with a term filter is probably more appropriate for your use case.
I have described this issue in this blog.
The issue is coming due to default tokenization in Elasticsearch.
In the same , I have outlined 2 solutions.
One is enabling not_analyzed flag on the required field and other is to use keyword tokenizer.
To expand on solarissmoke's solution, while the contents of that field will be passed through the standard analyzer, your query will not. If you refer to the Elasticsearch documentation on the term query, you will see that term queries are not analyzed.
The match query is probably more appropriate for your case. What you query will be analyzed in the same way as the contents of the title field by default. The query_string query brings a lot more to the table and you should review the documentation if you plan on using that.
So again pretty much what you had with the small tweak:
GET book-lists/book-list/_search
{
"query":{
"filtered":{
"filter":{
"bool":{
"must":[
{
"match":{
"title":"Sociology"
}
},
{
"term":{
"idOwner":"17xxxxxxxxxxxx45"
}
}
]
}
}
}
}
}
It is important to note passing lowercase version of the terms to the term query (hack - does not seem like a good idea given what solarissmoke describe about the other features of the Standard analyzer like the stop filter), using the query_string query, or using the match query is still very different from the SQL query you described:
SELECT document
FROM book-lists
WHERE title = "Sociology"
AND idOwner = 17xxxxxxxxxxxx45
With those Elasticsearch queries, you can match records where idOwner might be the same but title might be something like "Another Sociology Title" which is different from what you would expect with that SQL. Here is some great stuff from the documentation and another stackoverflow post that will elaborate on what was going on, where term queries and filters are appropriate, and getting exact matches:
Elasticsearch : Finding Exact Values
Stackoverflow : Exact (not substring) matching in Elasticsearch
I try to make a query with filtering but it fails, Bad Request comes as reponse
{
"query":{
"filtered":{
"query":{
"logdate":{
"gte":"01-01-2014"
}
}
}
}
}
I search documentation online and see it works as same part of my code but something in there is not true that I cant figure out
you seem "query" tag in filter comment in online documentation of elasticsearch or elsewhere ? lol never go there. Use "filter" tag in filtered query and also you must add "range" field. here This is the true form of your query
{
"query":{
"filtered":{
"filter":{
"range":{
"logdate":{
"gte":"01-01-2014"
}
}
}
}
}
}
Elastic Search 1.6
I want to index text that contains hyphens, for example U-12, U-17, WU-12, t-shirt... and to be able to use a "Simple Query String" query to search on them.
Data sample (simplified):
{"title":"U-12 Soccer",
"comment": "the t-shirts are dirty"}
As there are quite a lot of questions already about hyphens, I tried the following solution already:
Use a Char filter: ElasticSearch - Searching with hyphens in name.
So I went for this mapping:
{
"settings":{
"analysis":{
"char_filter":{
"myHyphenRemoval":{
"type":"mapping",
"mappings":[
"-=>"
]
}
},
"analyzer":{
"default":{
"type":"custom",
"char_filter": [ "myHyphenRemoval" ],
"tokenizer":"standard",
"filter":[
"standard",
"lowercase"
]
}
}
}
},
"mappings":{
"test":{
"properties":{
"title":{
"type":"string"
},
"comment":{
"type":"string"
}
}
}
}
}
Searching is done with the following query:
{"_source":true,
"query":{
"simple_query_string":{
"query":"<Text>",
"default_operator":"AND"
}
}
}
What works:
"U-12", "U*", "t*", "ts*"
What didn't work:
"U-*", "u-1*", "t-*", "t-sh*", ...
So it seems the char filter is not executed on search strings?
What could I do to make this work?
The answer is really simple:
Quote from Igor Motov: Configuring the standard tokenizer
By default the simple_query_string query doesn't analyze the words
with wildcards. As a result it searches for all tokens that start with
i-ma. The word i-mac doesn't match this request because during
analysis it's split into two tokens i and mac and neither of these
tokens starts with i-ma. In order to make this query find i-mac you
need to make it analyze wildcards:
{
"_source":true,
"query":{
"simple_query_string":{
"query":"u-1*",
"analyze_wildcard":true,
"default_operator":"AND"
}
}
}
the Quote from Igor Motov is true, you have to add "analyze_wildcard":true, in order to make it worked with regex. But it is important to notice that the hyphen actually tokenizes "u-12" in "u" "12", two separated words.
if preserve the original is important do not use Mapping char filter. Otherwise is kind of useful.
Imagine that you have "m0-77", "m1-77" and "m2-77", if you search m*-77 you are going to have zero hits. However you can remplace "-" (hyphen) with AND in order to connect the two separed words and then search m* AND 77 that is going to give you a correct hit.
you can do it in the client front.
In your problem u-*
{
"query":{
"simple_query_string":{
"query":"u AND 1*",
"analyze_wildcard":true
}
}
}
t-sh*
{
"query":{
"simple_query_string":{
"query":"t AND sh*",
"analyze_wildcard":true
}
}
}
If anyone is still looking for a simple workaround to this issue, replace hyphen with underscore _ when indexing data.
For eg, O-000022334 should indexed as O_000022334.
When searching, replace underscore back to hyphen again when displaying results. This way you can search for "O-000022334" and it will find a correct match.