I am new to Elasticsearch and please forgive me if the answer is obvious.
Here is what I have for the mapping of the field in question:
"condition" : { "type" : "string", "store" : "no", "index": "not_analyzed", "omit_norms" : "true" }
I need search on this field, but I need 100% string match (no stemming, etc.) on a sub-string (blank separated). An example of this field in a document is as follows:
{
"condition": "abc xyz"
}
An example query is:
/_search?q=condition:xyz
Is the above mapping correct? I also used omit_norms (true). Is this a correct thing to do in my case?
How can I disable scoring on this field? Can I do it in mapping? What is the best way of doing it? (Actually I need to disable scoring on more than one. I do have fields that need scoring)
Thanks and regards!
Using omit_norms:true will not take the length of the field into consideration for the scoring, Elasticsearch won't index the norms information. So if you don't want to use scoring that is a good thing to do as it will save you some disk space.
If you're not interested in scoring in your queries use a filtered query:
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": {
"term": {
"condition": "abc xyz"
}
}
}
}
}
}
}
The new syntax for a filtered query is now:
{
"query": {
"bool": {
"must": {
"match_all": {}
},
"filter": {
"term": {
"condition": "abc"
}
}
}
}
}
Related
I like to find an document in elastic search an highlight terms based on an query string.
Is this possible?
I tried to run an query-string elastic search and filter the result based on ID. But those sounds not very efficient, because elastic first generates an huge list of all document matched the querystring (which could by millions) an pic only one document based on the filter.
Is there a way or query-contstruct to combine querystring and "search for term in _id field" in one boolean search?
Something like this (which is not working):
"query": {
"bool": {
"must": {
"query_string": {
"query": "red*",
"fields": [
"text",
"title"
]
},
"term": {
"_id":"fda72434fa172"
}
}
}
},
"highlight": {
"fields": {
[...]
I made a small example that can be a starting point.
Use filter to perform your query and retrieve the doc by id.
Then I used match and highlight to highlight the term I want.
POST test/_doc/fda72434fa172
{
"text": "I like to find an document in elastic search an highlight terms based on an query string. Is this possible?"
}
GET test/_search
{
"query": {
"bool": {
"filter": [
{
"term": {
"_id": "fda72434fa172"
}
}
],
"must": [
{
"match": {
"text": {
"query": "elastic search"
}
}
}
]
}
},
"highlight": {
"fields": {
"text": {}
}
}
}
I have a requirement where I need to return only those records whose comments donot start with a String. PFB the query and this approach is not working. Need help
{
"size": 0,
"fields": ["id","comment"],
"query": {
"bool": {
"must_not": [
{
"wildcard": {
"comment":
"AG//*"
}
}
]
}
}
}
First, you should remove the "size": 0 from your query (or set the required size) to see the results.
Now, the best way to implement 'Starts with' in elasticsearch is by using the Prefix Query as follows:
{
"fields": ["id", "comment"],
"query": {
"bool": {
"must_not": [
{
"prefix": {
"comment": "AG" <-- No need for any wildcards
}
}
]
}
}
}
Note: The Prefix Query and Wildcard Query makes sense only on not_analyzed fields, so make sure your "comment" field has the same mapping.
I want to exactly match the string ":Feed:" in a message field and go back a day pull all such records. The json I have seems to also match the plain word " feed ". I am not sure where I am going wrong. Do I need to add "constant_score" to this query JSON? The JSON I have currently is as shown below:
{
"query": {
"bool": {
"must": {
"query_string": {
"fields": ["message"],
"query": "\\:Feed\\:"
}
},
"must": {
"range": {
"timestamp": {
"gte": "now-1d",
"lte": "now"
}
}
}
}
}
}
As stated here: Finding Exact Values, since the field has been analyzed when indexed - you have no way of exact-matching its tokens (":"). Whenever the tokens should be searchable the mapping should be "not_analyzed" and the data needs to be re-indexed.
If you want to be able to easily match only ":feed:" inside the message field you might want to costumize an analyzer which doesn't tokenize ":" so you will be able to query the field with a simple "match" query instead of wild characters.
Not able to do this with query_string but managed to do so by creating a custom normalizer and then using a "match" or "term" query.
The following steps worked for me.
create a custom normalizer (available >V5.2)
"settings": {
"analysis": {
"normalizer": {
"my_normalizer": {
"type": "custom",
"filter": ["lowercase"]
}
}
}
}
Create a mapping with type "keyword"
{
"mappings": {
"default": {
"properties": {
"title": {
"type": "text",
"fields": {
"normalize": {
"type": "keyword",
"normalizer": "my_normalizer"
},
"keyword" : {
"type": "keyword"
}
}
}
}
}
}
use match or term query
{
"query": {
"bool": {
"must": [
{
"match": {
"title.normalize": "string to match"
}
}
]
}
}
}
Use match phrase
GET /_search
{
"query": {
"match_phrase": {
"message": "7000-8900"
}
}
}
In java use matchPhraseQuery of QueryBuilder
QueryBuilders.matchPhraseQuery(fieldName, searchText);
Simple & Sweet Soln:
use term query..
GET /_search
{
"query": {
"term": {
"message.keyword": "7000-8900"
}
}
}
use term query instead of match_phrase,
match_phrase this find/match with ES-document stored sentence, It will not exactly match. It matches with those sentence words!
I am new to Elasticsearch. I have a mapping which has a boolean field:
{
...
"bool_field": {
"type": "boolean"
},
...
}
How possible to find documents on a boolean value without specifying the name of the field?
I tried one of the following, but without result:
{
"query": {
"match_all": {}
},
"filter": {
"query": {
"query_string": {
"query": "true"
}
}
}
}
Thanks!
Not that easy, by default, because boolean fields are not include_in_all (the _all field is used by default by query_string). This explains why your query doesn't work.
What you can do, though, is to use copy_to to create your own custom _all field and use that in the query_string.
Something like this:
"bool_field": {
"type": "boolean",
"copy_to": "_all_booleans"
}
And then
"query_string": {
"default_field": "_all_booleans",
"query": "true"
}
or
"query_string": {
"query": "_all_booleans:true"
}
I thought I had figured out Elasticsearch but I suspect I have failed to grok something, and hence this problem:
I am indexing products, which have a huge number of fields, but the ones in question are:
{
"show_in_catalogue": {
"type": "boolean",
"index": "no"
},
"prices": {
"type": "object",
"dynamic": false,
"properties": {
"site_id": {
"type": "integer",
"index": "no"
},
"currency": {
"type": "string",
"index": "not_analyzed"
},
"value": {
"type": "float"
},
"gross_tax": {
"type": "integer",
"index": "no"
}
}
}
}
I am trying to return all documents where "show_in_catalogue" is true, and there is a price with site_id 1:
{
"filter": {
"term": {
"prices.site_id": "1",
"show_in_catalogue": true
}
},
"query": {
"match_all": {}
}
}
This returns zero results. I also tried an "and" filter with two separate terms - no luck.
A subset of one of the documents returned if I have no filters looks like:
{
"prices": [
{
"site_id": 1,
"currency": "GBP",
"value": 595,
"gross_tax": 1
},
{
"site_id": 2,
"currency": "USD",
"value": 745,
"gross_tax": 0
}
]
}
I hope I am OK to omit so much of the document here; I don't believe it to be contingent but I cannot be certain, of course.
Have I missed a vital piece of knowledge, or have I done something terminally thick? Either way, I would be grateful for an expert's knowledge at this point. Thanks!
Edit:
At the suggestion of J.T. I also tried reindexing the documents so that prices.site_id was indexed - no change. Also tried the bool/must filter below to no avail.
To clarify, the reason I'm using an empty query is that the web interface may supply a query string, but the same code is used to simply filter all products. Hence I left in the query, but empty, since that's what Elastica seems to produce with no query string.
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": [
{
"term": {
"show_in_catalogue": true
}
},
{
"term": {
"prices.site_id": 1
}
}
]
}
}
}
}
}
You have site_id set as {"index": "no"}. This tells ElasticSearch to exclude the field from the index which makes it impossible to query or filter on that field. The data will still be stored. Likewise, you can set a field to only be in the index and searchable, but not stored.
I'm new to ElasticSearch as well and can't always grok the questions! I'm actually confused by you query. If you are going to "just filter" then you don't need a query. What I don't understand is your use of two fields inside the term filter. I've never done this. I guess it acts as an OR? Also, if nothing matches, it seems to return everything. If you wanted a query with the results of that query filtered, then you would want to use a
-d '{
"query": {
"filtered": {
"query": {},
"filter": {}
}
}
}'
If you just want to apply filters is the filter that should work without any "query" necessary
-d '{
"filter": {
"bool": {
"must": [
{
"term": {
"show_in_catalogue": true
}
},
{
"term": {
"prices.site_id": 1
}
}
]
}
}
}'