All, I am using ES 6.7 and trying to return the results from a single index while querying two indices( customer, payment) and doing a terms lookup against user-customers index. The index I want data from(customer) has more fields than the second index. But for some reason, I only see results from the payment index. The fields customerName, customerNumber, state, address only exist on customer index. But I only want customers that has totalCredits > 0(This exists only on payment index) ordered by the logic in the sort array. I tried adding an _index filter( setting this to customer) but dint help. Adding source filtering doesn't help either. Is this doable in ES 6.7?. Am I left with the option of adding the fields in the sort array to the payment index or are there some other options?
ES query
GET customer,payment/_search
{
"sort": [
{
"customerName": {
"order": "asc",
"unmapped_type": "keyword"
}
},
{
"customerNumber": {
"order": "asc"
}
},
{
"state": {
"order": "asc",
"unmapped_type": "keyword"
}
},
{
"address": {
"order": "asc",
"unmapped_type": "keyword"
}
}
],
"query": {
"bool": {
"filter": [
{
"bool": {
"must_not": [
{
"terms": {
"status": [
"pending"
]
}
}
]
}
}
],
"must": [
{
"terms": {
"customerNumber": {
"index": "user-customers",
"type": "_doc",
"id": "rennish#emial.com",
"path": "users"
}
}
},
{
"range": {
"totalCredits": {
"gt": 0
}
}
}
]
}
}
}
Related
I am using elasticsearch in my application and I am new to Elasticsearch.
I have an index called files with some tags associated to it. I want to query them using tags. something like this may be
{
"query": {
"terms": {
"tags": [
"xxx",
"yyy"
]
}
},
"sort": [
{
"created_at": {
"order": "desc"
}
}
]
}
The above query results only matched ones. But I need all the results with matched results on top. And also sort by created_at. How to do it?
I TRIED THIS:
{
"query": {
"bool": {
"should": [
{
"terms": {
"name": [
"cool",
"co"
]
}
}
],
"minimum_should_match": 0
}
},
"sort": [
{
"_score": {
"order": "desc"
}
},
{
"created_at": {
"order": "desc"
}
}
]
}
But results zero always.
You can use bool queries with should.
Since you want all the docs, you can use a match_all. should only affects the scoring and not whether documents are included or not.
{
"query": {
"bool": {
"must" :
{
"match_all": { }
}
},
"should": [
{ "terms" : {
"tags": [
"xxx",
"yyy"
]
} }]
},
"sort": [
{ "_score":
{ "order": "desc"
}
},
{ "created_at":
{ "order": "desc"
}
}
]
}
Also, sort can take an array so you can pass in your multiple parameters basis which the results should be sorted.
i have multiple document in each channel id and i want sort descending order in view column by channel id.
{
"query":
{
"match":{
"channel_id":"UCQOd1f6pYldvhgvdQ_ktpGA"
}
},
"aggs":{
"video_views":{
"sort": {
"views": "desc"
},
"_source": ["channel_id", "views"]
}
}
}
You can get top 1 document sorted on views column
{
"size":1, --> get top 1 documnet
"query": {
"term": { --> term query to filter on channelId
"channel_id.keyword": {
"value": "UCQOd1f6pYldvhgvdQ_ktpGA"
}
}
},
"sort": [ ---> sort on views column
{
"views": {
"order": "desc"
}
}
]
}
Use Max aggregation
{
"query": {
"term": {
"channel_id.keyword": {
"value": "UCQOd1f6pYldvhgvdQ_ktpGA"
}
}
},
"aggs": {
"views": {
"max": {
"field": "views",
"missing": 0
}
}
}
}
All,
I am working on building a NEST 6.x query that takes a serach term and looks in different fields in different indices.
This is the one I got so far but is not returning any results that I am expecting.
Please see the details below
Indices used
dev-sample-search
user-agents-search
The way the search should work is as follows.
The value in the query field(27921093) is searched against the
fields agentNumber, customerName, fileNumber, documentid(These are all
analyzed fileds).
The search should limit the documents to the agentNumbers the user
sampleuser#gmail.com has access to( sample data for
user-agents-search) is added below.
agentNumber, customerName, fileNumber, documentid and status are
part of the index dev-sample-search.
status field is defined as a keyword.
The fields in the user-agents-search index are all keywords
Sample user-agents-search index data:
{
"id": "sampleuser#gmail.com"",
"user": "sampleuser#gmail.com"",
"agentNumber": [
"123.456.789",
"1011.12.13.14"
]
}
Sample dev-sample-search index data:
{
"agentNumber": "123.456.789",
"customerName": "Bank of america",
"fileNumber":"test_file_1123",
"documentid":"1234456789"
}
GET dev-sample-search/_search
{
"from": 0,
"size": 10,
"query": {
"bool": {
"must": [
{
"multi_match": {
"type": "best_fields",
"query": "27921093",
"operator": "and",
"fields": [
"agentNumber",
"customerName",
"fileNumber",
"documentid^10"
]
}
}
],
"filter": [
{
"bool": {
"must": [
{
"terms": {
"agentNumber": {
"index": "user-agents-search",
"type": "_doc",
"user": "sampleuser#gmail.com",
"path": "agentNumber"
}
}
},
{
"bool": {
"must_not": [
{
"terms": {
"status": {
"value": "pending"
}
}
},
{
"term": {
"status": {
"value": "cancelled"
}
}
},
{
"term": {
"status": {
"value": "app cancelled"
}
}
}
],
"should": [
{
"term": {
"status": {
"value": "active"
}
}
},
{
"term": {
"status": {
"value": "terminated"
}
}
}
]
}
}
]
}
}
]
}
}
}
I see a couple of things that you may want to look at:
In the terms lookup query, "user": "sampleuser#gmail.com", should be "id": "sampleuser#gmail.com",.
If at least one should clause in the filter clause should match, set "minimum_should_match" : 1 on the bool query containing the should clause
I have the following model class:
public class NewsItem
{
public String Language { get; set; }
public DateTime DateUpdated { get; set; }
public List<String> Tags { get; set; }
}
I index it with NEST using the automapping, resulting in the mapping below:
{
"search": {
"mappings": {
"news": {
"properties": {
"dateUpdated": {
"type": "date",
"format": "strict_date_optional_time||epoch_millis"
},
"language": {
"type": "string"
},
"tags": {
"type": "string"
},
}
}
}
}
}
I then run a query on language which works fine:
{
"query": {
"constant_score": {
"filter": [
{
"terms": {
"language": [
"en"
]
}
}
]
}
},
"sort": {
"dateUpdated": {
"order": "desc"
}
}
}
But running the same query on the tags property doesn't work. Is there any special tricks to query an array field? I read the docs again and again and I don't understand why this query gives no results:
{
"query": {
"constant_score": {
"filter": [
{
"terms": {
"tags": [
"Hillary"
]
}
}
]
}
},
"sort": {
"dateUpdated": {
"order": "desc"
}
}
}
The document returned from another query:
{
"_index": "search",
"_type": "news",
"_score": 0.12265198,
"_source": {
"tags": [
"Hillary"
],
"language": "en",
"dateUpdated": "2016-11-07T15:41:00Z"
}
}
Your tags field is analyzed, hence Hillary has been indexed to hillary. So you have two ways out:
A. Use a match query instead (since terms query does not analyze the token
{
"query": {
"bool": {
"filter": [
{
"match": { <--- use match here
"tags": "Hillary"
}
}
]
}
},
"sort": {
"dateUpdated": {
"order": "desc"
}
}
}
B. Keep the terms query but lowercase the token:
{
"query": {
"bool": {
"filter": [
{
"terms": {
"tags": [
"hillary" <--- lowercase here
]
}
}
]
}
},
"sort": {
"dateUpdated": {
"order": "desc"
}
}
}
Elasticsearch by default runs an analyzer on all strings but Terms filter on other hand computer exact match. So this implies that ES is storing 'Hillary' as 'hillary' while you are querying for 'Hillary'. So, there are 2 ways to fix this. Either you use a match query instead of terms query or you don't automap and rather create an index and analyze the tags field as you want. You can also query 'hillary' but this would be a solution for this one case because if tag was something like 'us elections' us and elections both will be stored separately.
have a situation where treatment has a price and hospital may or may not want to display it.
so there is a price field PLUS a lp_low_priority field.
value of lp_low_priority is 1(true) when price is not set(price is_null).
hospital doc is saved with its nested treatments.
when user searches for treatment he get list of hospitals with minimum price of the treatment.
now the sort works fine.
BUT i want the hospital with that treatment with the lp_low_priority = 1 to come at last.
Code to search is like
{
"sort": [
{
"treatments.lowest_price": {
"nested_filter": {
"term": {
"treatments.treatment_slug": "heart-surgery"
}
},
"mode": "avg",
"order": "asc"
}
},
{
"treatments.lp_low_priority": {
"order": "asc",
"nested_filter": {
"term": {
"treatments.treatment_slug": "heart-surgery"
}
},
"mode": "max"
}
}
],
"query": {
"filtered": {
"filter": [
{
"term": {
"treatments.treatment_slug": "heart-surgery"
}
},
{
"term": {
"treatments.status": "active"
}
},
{
"term": {
"treatments.treatment_status": "active"
}
},
{
"term": {
"hospital_status": "active"
}
},
{
"terms": {
"location.country": [
"India"
]
}
}
]
}
}
}
the result is way too weird.
if I only use
{
"sort": [
{
"treatments.lowest_price": {
"nested_filter": {
"term": {
"treatments.treatment_slug": "heart-surgery"
}
},
"mode": "avg",
"order": "asc"
}
}
The sorting is in order but then you see the lp_low_priority come first in order, which is OK(but not the requirement).
Can i even use more than one sorts for nested fields.