How to filter with multiple fields and values in elasticsearch? - elasticsearch

I've been reading through the docs and been trying to achieve a solution to filter results through multiple fields and columns, however I keep getting errors; malformed query.
I want to filter the result with exact equal values, such as the following:
is_active: true
category_id: [1,2,3,4]
brand: "addidas"
gender: "male"
To make it more clear what I intend to do, this is how I'd like it to run if it would be written in SQL:
SELECT .... WHERE
is_active= 1 AND category_id IN(1,2,3,4)
AND brand='addidas' AND gender='male'
My query in DSL goes as following:
{
"body": {
"query": {
"nested": {
"query": {
"bool": {
"must": {
"terms": {
"category_id": [
1,
2,
3
]
},
"term": {
"is_active": true
},
"term": {
"brand": "addidas"
}
}
}
}
}
}
}
}
How do I filter multiple fileds and values as described, in elasticsearch?
If you need extra information from me that is required to answer the question, leave a comment. If you add a link to the docs, please also provide an example (with query dsl) of how my current, or similar situations should be solved.

Use the following code:
The clause (query) must appear in matching documents and will contribute to the score.
"query": {
"bool": {
"must" : [
{"term" : { "is_active" : true}},
{"term" : { "gender" : "female"}},
{"term" : { "brand" : "addidas"}},
{"terms": { "categoryId": [1,2,3,4]}}
]
}
}
Queries specified under the filter element have no effect on scoring
"query": {
"bool": {
"filter" : [
{"term" : { "is_active" : true}},
{"term" : { "gender" : "female"}},
{"term" : { "brand" : "addidas"}},
{"terms": { "categoryId": [1,2,3,4]}}
]
}
}

Related

AND query nested objects

Full disclaimer: elasticsearch noob here.
I'm using the nested field value_per_id for an array of objects. Each object has the properties: value and id.
E.g.
"value_per_id": [
{
"id": 2,
"value": "positive"
},
{
"id": 23,
"value": "positive"
},
{
"id": 65,
"value": "neutral"
}
]
I have a query that looks like this (edited for clarity):
{
"query" : {
"bool" : {
"filter" : [
{
"bool" : {
"must" : {
"nested" : {
"path" : "value_per_id",
"query" : [
{"terms" : {"value_per_id.value" : <MY_VALUES>}},
{"terms" : {"value_per_id.id" : <MY_IDS>}},
]
}
}
}
}
]
}
}
}
With this query, I get all the elements that have an object with value in MY_VALUES OR id in MY_IDS.
I want all the elements that have value in MY_VALUES for any id in MY_IDS.
(E.g. if MY_VAULES = ['positive', 'neutral'] and MY_IDS = [1, 2], I want those that have an object with 1 - positive, 1 - negative, 2 - positive or 2 - negative).
What's the syntax for this?
EDIT - Wrong format
You need to use the bool within the nested query like this:
{
"query": {
"bool": {
"filter": [{
"nested": {
"path": "value_per_id",
"query": {
"bool": {
"must": [
{"term": {"value_per_id.value": 1}},
{"term": {"value_per_id.id": 2}}
]
}
}
}
}]
}
}
}
Use must or filter in the bool query.
If you use must, the sub queries will contribute to the score, if you use filter they won't. Also, if you use filter elasticsearch will be able to cache the results.
{
"bool": {
"must": [
{
"nested": {
"path": "value_per_id",
"query": {
"terms": {
"value_per_id.value": <MY_VALUES>
}
}
}
},
{
"nested": {
"path": "value_per_id",
"query": {
"terms": {
"value_per_id.id": <MY_IDS>
}
}
}
}
]
}
}

Nested Elasticsearch queries

I'm trying to create a nested query which would filter out some documents with specific terms. In this case I'm trying to filter out documents which have matching terms in user.first. Data example:
{
"group" : "fans",
"user" : [
{
"first" : "John",
"last" : "Smith"
},
{
"first" : "Tim",
"last" : "White"
}
]
},
{
"group" : "fans",
"user" : [
{
"first" : "Jim",
"last" : "Smith"
},
{
"first" : "Alice",
"last" : "Black"
}
]
}
My query doesn't get the desired result as it returns me all records which are not filtered. I tried querying using:
"query": {
"bool": {
"must_not": [
{
"nested": {
"query": {
"terms": {
"user.first": [
"John",
"Thomas"
]
}
},
"path": "user"
}
}
]
}
}
I expect here to get documents which don't match with the filter. In this case it should return only the second document. What is the right way to do this?
After a lot of trial and error I've managed to find a solution for the problem. This is the query that filters out documents we don't need.
"query": {
"bool": {
"must_not": [
{
"bool": {
"filter": [
{
"nested": {
"query": {
"terms": {
"user.first": [
"John",
"Thomas"
]
}
},
"path": "user"
}
}
]
}
}
]
}
}

How to return both a scripted field as well as document fields in Elasticsearch?

I am trying to build a query that will return both its normal document values as well as one scripted field which I need to add some string to (enclose it in a anchor tag).
A friend told me to add _source: true but it just won't work. If I add _source: true, it only returns my regular fields. If I remove _source: true it only returns the scripted_field. If I name all my fields in _source, it will ignore the scripted_field and only return the regular ones. How can I make it so that both regular and scripted fields are displayed?
PS. I am running elastic6 on AWS and this query is being run on Redash for a report.
{
"index": "notifications_production",
"sort" : {"created_at" : {"order" : "desc"}},
"query": {
"bool": {
"must": [
{ "match_all": {} }
],
"filter": [
{ "match": { "company_id": 5054 }},
{
"range" : {
"created_at" : {
"gte" : "now-30d",
"lte" : "now"
}
}
}
]
}
},
"script_fields" : {
"sequence2" : {
"script" : {
"lang": "painless",
"source": "'<a href=#>' + params._source.file_name + '</a>'"
}
}
},
"_source": true
}

Elastic search filter

I am new to Elastic search . Please help me in finding the filter/query to be written to match the exact records using Java API.
Below is the mongodb record .I need to get both the record matching the word 'Jerry' using elastic search.
{
"searchcontent" : [
{
"key" : "Jerry",
"sweight" : "1"
},{
"key" : "Kate",
"sweight" : "1"
},
],
"contentId" : "CON_4",
"keyword" : "TEST",
"_id" : ObjectId("55ded619e4b0406bbd901a47")
},
{
"searchcontent" : [
{
"key" : "TOM",
"sweight" : "2"
},{
"key" : "Kruse",
"sweight" : "2"
}
],
"contentId" : "CON_3",
"keyword" : "Jerry",
"_id" : ObjectId("55ded619e4b0406ccd901a47")
}
And if you would like to search in all the fields.
Then you can just do a match _all query,
POST <index name>/<type name>/_search.
{
"query": {
"match" : {
"_all" : "Jerry"
}
}
}
This searches for 'Jerry' in all the fields.
A Multi-Match query is what you need to search across multiple fields. Below query will search for the word "jerry" in both the fields "searchcontent.key" and "keyword" which is what you want.
POST <index name>/<type name>/_search
{
"query": {
"multi_match": {
"query": "jerry",
"fields": [
"searchcontent.key",
"keyword"
]
}
}
}
There is no single solution, it depends how you map your data in elastic search and what you are indexing
GET /intu/_settings
You can use: query string.
If you don't need to combine filter you can remove bool and should.
From the documentation: "The bool query takes a more-matches-is-better approach, so the score from each matching must or should clause will be added together to provide the final _score for each document."
For example:
GET /yourstaff/_search
{
"query": {
"filtered": {
"query": {
"bool": {
"should":
{
"query_string": {
"query": "jerry"
}
}
}
}
}
}
}
Take a look to the documentation:
Query string
Term vs full-search
Bool query
Use Sense to figure out what results you want to have
Using filter is a better option as it caches the results..
{
"query":
{
"bool":
{
"should":
[
{
"term":
{
"searchcontent.key":"jerry"
}
},
{
"term":
{
"keyword":"jerry"
}
}
]
}
}
}
https://www.elastic.co/blog/found-optimizing-elasticsearch-searches
A suggested read for better search.

Elastic Search Nested Object mapping and Query for search

I am trying to use Elastic Search and I am stuck trying to query for the nested object.
Basically my object is of the following format
{
"name" : "Some Name",
"field2": [
{
"prop1": "val1",
"prop2": "val2"
},
{
"prop1": "val3",
"prop2":: "val4"
}
]
}
Mapping I used for the nested field is the following.
PUT /someval/posts/_mapping
{
"posts": {
"properties": {
"field2": {
"type": "nested"
}
}
}
}
Say now i insert elements for /field/posts/1 and /field/posts/2 etc. I have k values for field2.prop1 and i want a query which gets the posts sorted based on most match of field2.prop1 among the K values i have. What would be the appropriate query for that.
Also I tried a simple filter but even that doesnt seem to work right.
GET /someval/posts/_search
{
"query": {
"filtered": {
"query": {
"match_all": {}
}
},
"filter" : {
"nested" : {
"path" : "field2",
"filter" : {
"bool" : {
"must" : [
{
"term" : {"field2.prop1" : "val1"}
}
]
}
},
"_cache" : true
}
}
}
}
The above query should match atleast the first post. But it returns no match. Can anyone help to clarify whats wrong here ?
There was problem in your json structure, you used filtered query , but filter(object) was in different level than query.
Find the difference.
POST /someval/posts/_search
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"nested": {
"path": "field2",
"filter": {
"bool": {
"must": [
{
"term": {
"field2.prop1": "val1"
}
}
]
}
},
"_cache": true
}
}
}
}
}

Resources