I would like to match within a boolean query in Elasticsearch. I have the match query and boolean query working as expected now, but I am not sure how to have a AND to combine them.
nested boolean
{
"query": {
"constant_score" : {
"filter":{
"bool":{
"must":[
{"terms":{"address.keyword": addr}},
{"bool":{
"should":[
{"terms": {"state.keyword": state}}
,{"terms": {"city.keyword": city}}
]
}}
]
}
}
}}}
match
{"query": {
"match": {
"auct_title": {
"query": keyword,
"operator": "and"
}
}
}
, "collapse" : {
"field" : "id"
}
,"sort" : [
{ sort_field: {"order" : sort_order} }]
,"size":20
}
You can move natch to the must clause . So document has to satisfy three conditions
1.address
2.either of state/city
2.match on auct_title
It will then return one document per Id based on sort order passed
GET <index>/_search
{
"query": {
"constant_score": {
"filter": {
"bool": {
"must": [
{
"term": {
"address.keyword": "addr"
}
},
{
"bool": {
"should": [
{
"term": {
"state.keyword": "state"
}
},
{
"term": {
"city.keyword": "city"
}
}
]
}
},
{
"match": {
"auct_title": {
"query": "keyword",
"operator": "and"
}
}
}
]
}
}
}
},
"collapse": {
"field": "id"
},
"sort": [
{
"FIELD": {
"order": "desc"
}
}
],
"size": 20
}
Related
There are 2 indexes: categories, posts.
categories
name
body
posts
name
body
publish_at
publish_until
I want to do a query on both indexes with a filter on publish_at and publish_until for the posts index.
http://localhost:9200/categories,posts/_search
{
"query": {
"bool": {
"must": {
"multi_match": {
"query": "keyword",
"fields": [
"name^3",
"body"
]
}
},
"filter": [{
"bool": {
"must": [
{
"range": {
"publish_at": {
"lte" : "now"
}
}
},
{
"range": {
"publish_until": {
"gt" : "now"
}
}
}
]
}
}]
}
}
}
This query only gives me posts as results. I also want categories in my results.
How do I apply the date range filters to only indexes with publish_at and publish_until fields and skip the date range filters for the other indexes?
Ok after a day of fiddling with bool I got it working:
{
"query": {
"bool" : {
"must" : [
{
"multi_match": {
"query": "keyword",
"fields": [
"name^3",
"body"
]
}
},
{
"bool": {
"should": [
{
"bool": {
"must": [
{
"range": {
"publish_at": {
"lte" : "now"
}
}
},
{
"range": {
"publish_until": {
"gt" : "now"
}
}
}
]
}
},
{
"bool": {
"must_not": [
{
"exists": {
"field": "publish_at"
}
},
{
"exists": {
"field": "publish_until"
}
}
]
}
}
]
}
}
]
}
}
}
Consider the following query:
{
"query" : {
"match_phrase" : {
"_all" : "Smith"
}
}
}
How would I specify in which fields of which types it may search, instead of searching in everything? (field names may be non-unique across types)
I've tried the query below, but it didn't work (it doesn't return results, it does when I remove person. from all fields):
{
"query": {
"multi_match": {
"query": "Smith",
"fields": [
"person.first_name",
"person.last_name",
"person.age"
],
"lenient": true
}
}
}
I'm sending these queries to http://localhost:9200/tsf-model/_search.
If you can build your query dynamically, I think you can use a combination of your multi_match query and a type query for each type, in order to achieve what you want:
{
"query": {
"bool": {
"minimum_should_match": 1,
"should": [
{
"bool": {
"filter": [
{
"type": {
"value": "type1"
}
},
{
"multi_match": {
"query": "Smith",
"fields": [
"field1",
"field3",
"field5"
]
}
}
]
}
},
{
"bool": {
"filter": [
{
"type": {
"value": "type2"
}
},
{
"multi_match": {
"query": "Smith",
"fields": [
"field2",
"field4",
"field6"
]
}
}
]
}
}
]
}
}
}
I have my request:
{
"size": 10,
"query": {
"filtered": {
"query": {
"bool": {
"must": [
{"term": {"event": "matchmaking_done"}}
]
}
},
"filter": {
"range": {
"#timestamp": {
"gt" : "2016-06-01T00:00:00.000Z",
"lte" : "2016-06-01T00:05:00.000Z"
}
}
}
}
},
"aggs" : {
"user-ids" : {
"terms" : { "field" : "user_id",
"size": 0
}
}
}
}
And I need to add into this request parameter - does not contain field pvp_league! I tried add must_not but can't understand how to do this correct.
Help please!
You answered it yourself, but the ES 2.x way to do this is to not use the filtered query because it has been deprecated and it will be removed in ES 5.0. ES 2.x introduces the concept of the "filter" context rather than every query being either just a query or a filter; now every query is both a filter or a query (scored), just depending on the context it's used in.
For your query, this therefore becomes a little simpler because of the simplified bool / filter syntax:
{
"size":10,
"query":{
"bool":{
"must":[
{
"term":{
"event":"matchmaking_done"
}
}
],
"must_not":[
{
"exists":{
"field":"pvp_league"
}
}
],
"filter":[
{
"range":{
"#timestamp":{
"gt":"2016-06-01T00:00:00.000Z",
"lte":"2016-06-01T00:05:00.000Z"
}
}
}
]
}
},
"aggs":{
"user-ids":{
"terms":{
"field":"user_id",
"size":0
}
}
}
}
As a very big aside, specifying "size" : 0 for the terms aggregation, you are requesting all unique terms, up to INT_MAX. That is not a scalable request (works great with 10 user_ids, or even 100, but not 10000 users).
As a not-so-bad aside, your request doesn't need a query context at all because nothing about the search side of it cares about relevance. Your term query ("event" : "matchmaking_done") either matches or it doesn't. Since you either want it to match or not, but you don't really care about order inherently, you should use this in the filter context. This changes the request to:
{
"size": 10,
"query": {
"bool": {
"must_not": [
{
"exists": {
"field": "pvp_league"
}
}
],
"filter":[
{
"range": {
"#timestamp": {
"gt":"2016-06-01T00:00:00.000Z",
"lte":"2016-06-01T00:05:00.000Z"
}
}
},
{
"term": {
"event": "matchmaking_done"
}
}
]
}
},
"aggs": {
"user-ids": {
"terms": {
"field": "user_id",
"size": 0
}
}
}
}
I've found solution! It looks like this:
{
"size": 10,
"query": {
"filtered": {
"query": {
"bool": {
"must": [
{"term": {"event": "matchmaking_done"}}
],
"must_not": [
{"filtered": {
"filter": {
"exists": {
"field": "pvp_league"
}
}
}
}
]
}
},
"filter": {
"range": {
"#timestamp": {
"gt" : "2016-06-01T00:00:00.000Z",
"lte" : "2016-06-01T00:05:00.000Z"
}
}
}
}
},
"aggs" : {
"user-ids" : {
"terms" : { "field" : "user_id",
"size": 0
}
}
}
}
I'm new to Elasticsearch. I don't think I fully understand the concept of query and filters. In my case I just want to use filters as I don't want to use advance feature like scoring.
How would I convert the following SQL statement into elasticsearch query?
select * from tablename where (name="d" and time>1231312) or (name="ds" and time>21)
{
"filter" : {
"or":[
{ "and" : [
{"range": {"time": {"gt": 1231312}}},
{"term" : {"name":"d"}}
]},
{ "and" : [
{"range": {"time": {"gt": 21}}},
{"term" : {"name":"ds"}}
]}
]
}
}
Here is the query DSL which is equivalent to your sql query. The query_string/query filter is not cached by default that's why I have use _cache:true performance wise it will works good.
curl -XPOST http://localhost:9200/index_name/_search '{
"filter": {
"or": {
"filters": [
{
"fquery": {
"query": {
"bool": {
"must": [
{
"term": {
"name": "d"
}
},
{
"range": {
"time": {
"gte":1231312
}
}
}
]
}
},
"_cache": true
}
},
{
"fquery": {
"query": {
"bool": {
"must": [
{
"term": {
"name": "ds"
}
},
{
"range": {
"time": {
"gte":21
}
}
}
]
}
},
"_cache": true
}
}
]
}
}
}'
How can i create Elasticsearch curl query to get the field value which are not null and not empty(""),
Here is the mysql query:
select field1 from mytable where field1!=null and field1!="";
A null value and an empty string both result in no value being indexed, in which case you can use the exists filter
curl -XGET 'http://127.0.0.1:9200/test/test/_search?pretty=1' -d '
{
"query" : {
"constant_score" : {
"filter" : {
"exists" : {
"field" : "myfield"
}
}
}
}
}
'
Or in combination with (eg) a full text search on the title field:
curl -XGET 'http://127.0.0.1:9200/test/test/_search?pretty=1' -d '
{
"query" : {
"filtered" : {
"filter" : {
"exists" : {
"field" : "myfield"
}
},
"query" : {
"match" : {
"title" : "search keywords"
}
}
}
}
}
'
As #luqmaan pointed out in the comments, the documentation says that the filter exists doesn't filter out empty strings as they are considered non-null values.
So adding to #DrTech's answer, to effectively filter null and empty string values out, you should use something like this:
{
"query" : {
"constant_score" : {
"filter" : {
"bool": {
"must": {"exists": {"field": "<your_field_name_here>"}},
"must_not": {"term": {"<your_field_name_here>": ""}}
}
}
}
}
}
On elasticsearch 5.6, I have to use command below to filter out empty string:
GET /_search
{
"query" : {
"regexp":{
"<your_field_name_here>": ".+"
}
}
}
Wrap a Missing Filter in the Must-Not section of a Bool Filter. It will only return documents where the field exists, and if you set the "null_value" property to true, values that are explicitly not null.
{
"query":{
"filtered":{
"query":{
"match_all":{}
},
"filter":{
"bool":{
"must":{},
"should":{},
"must_not":{
"missing":{
"field":"field1",
"existence":true,
"null_value":true
}
}
}
}
}
}
}
You can do that with bool query and combination of must and must_not like this:
GET index/_search
{
"query": {
"bool": {
"must": [
{"exists": {"field": "field1"}}
],
"must_not": [
{"term": {"field1": ""}}
]
}
}
}
I tested this with Elasticsearch 5.6.5 in Kibana.
The only solution here that worked for me in 5.6.5 was bigstone1998's regex answer. I'd prefer not to use a regex search though for performance reasons. I believe the reason the other solutions don't work is because a standard field will be analyzed and as a result have no empty string token to negate against. The exists query won't help on it's own either since an empty string is considered non-null.
If you can't change the index the regex approach may be your only option, but if you can change the index then adding a keyword subfield will solve the problem.
In the mappings for the index:
"myfield": {
"type": "text",
"fields": {
"keyword": {
"ignore_above": 256,
"type": "keyword"
}
}
}
Then you can simply use the query:
{
"query": {
"bool": {
"must": {
"exists": {
"field": "myfield"
}
},
"must_not": {
"term": {
"myfield.keyword": ""
}
}
}
}
}
Note the .keyword in the must_not component.
You can use not filter on top of missing.
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"not": {
"filter": {
"missing": {
"field": "searchField"
}
}
}
}
}
}
Here's the query example to check the existence of multiple fields:
{
"query": {
"bool": {
"filter": [
{
"exists": {
"field": "field_1"
}
},
{
"exists": {
"field": "field_2"
}
},
{
"exists": {
"field": "field_n"
}
}
]
}
}
}
You can use a bool combination query with must/must_not which gives great performance and returns all records where the field is not null and not empty.
bool must_not is like "NOT AND" which means field!="", bool must exist means its !=null.
so effectively enabling: where field1!=null and field1!=""
GET IndexName/IndexType/_search
{
"query": {
"bool": {
"must": [{
"bool": {
"must_not": [{
"term": { "YourFieldName": ""}
}]
}
}, {
"bool": {
"must": [{
"exists" : { "field" : "YourFieldName" }
}]
}
}]
}
}
}
ElasticSearch Version:
"version": {
"number": "5.6.10",
"lucene_version": "6.6.1"
}
ES 7.x
{
"_source": "field",
"query": {
"bool": {
"must": [
{
"exists": {
"field":"field"
}
}
],
"must_not": [
{
"term": {
"field.keyword": {
"value": ""
}
}
}
]
}
}
}
We are using Elasticsearch version 1.6 and I used this query from a co-worker to cover not null and not empty for a field:
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"bool": {
"must": [
{
"exists": {
"field": "myfieldName"
}
},
{
"not": {
"filter": {
"term": {
"myfieldName": ""
}
}
}
}
]
}
}
}
}
}
You need to use bool query with must/must_not and exists
To get where place is null
{
"query": {
"bool": {
"must_not": {
"exists": {
"field": "place"
}
}
}
}
}
To get where place is not null
{
"query": {
"bool": {
"must": {
"exists": {
"field": "place"
}
}
}
}
}
Elastic search Get all record where condition not empty.
const searchQuery = {
body: {
query: {
query_string: {
default_field: '*.*',
query: 'feildName: ?*',
},
},
},
index: 'IndexName'
};