ElasticSearch 6.7 painless, how to access nested document - elasticsearch

When I use ES 5.5 update to 6.7.
Painless script does’t work
This is 5.5
If I want get a nested document [transFilter]
I do this
params['_source’]['carFilter’]
It works very well。
But
When I used 6.7 version
params['_source’]['carFilter’]
I found it does’t work
All params['_source’] is null
my mappings
carFilter": {
"type": "nested",
"properties": {
"time": {
"type": "long"
}
}
}
my data example
"carFilter" : [
{
"time" : 20200120
},
{
"time" : 20200121
}
]
and my query script example
{
"query" : {
"bool" : {
"must" : [
{
"script" : {
"script" : {
"inline" : "if(params['_source']!=null){
if(params['_source']['carFilter']!=null){
for(def item:params['_source']['carFilter'] ){
if (item.time>1) { return true; }
}
}
}
return false;",
"lang" : "painless",
"params" : {
"rentTime" : 1000
}
}
}
}
]
}
}
}
even no error
but fact
if(params['_source']!=null){
this line already return
The simple painless above is just to illustrate the problem, and a relatively real one is attached below.
double carPrice=0.00;if(!params['_source'].empty){"+
" def days=params['_source']['everyDayPrice'];if(params['_source']['everyDayPrice']!=null){int size=days.length;" +
" if(size>0){for(int i=0;i<size;i++){String day = days[i]['day'];Double price = days[i]['price'];"+
" if(price!=null&&params.get(day)!=null){carPrice=carPrice+params.get(day)*price;}}}}}" +
" return carPrice/params.total"

Looking at your query, you would want to filter the documents having carFilter.time > 1 and why not use a simple Nested Query:
POST <your_index_name>/_search
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "carFilter",
"query": {
"range": {
"carFilter.time": {
"gte": 1
}
}
}
}
}
]
}
}
}
Note that I've made use of Range Query to evaluate the time based on what you are looking for.
I'd suggest you go through this answer if the above doesn't help.
Let me know if you have any queries.

Related

How to filter data using Elastic Search by adding two fields as a condition?

How to filter data using Elastic Search by adding two fields as a condition?
Like the following sql statement
select * from hello where hello.a + hello.b > 10;
Elastic search gives best performance when things are precomputed. If sum was already in index then a simple range query would give result.
{
"query": {
"bool" : {
"filter" : {
"range": {
"sum_field": {
"gte": 10
}
}
}
}
}
}
For your current scenario, you need to use script query
{
"query": {
"bool" : {
"filter" : {
"script" : {
"script" : {
"source": "doc['field1'].value + doc['field2'].value > params.limit",
"lang": "painless",
"params": {
"limit": 10
}
}
}
}
}
}
}
script is slow it will run for each document , compute sum and filter result.

Elasticsearch NEST nested query not working c#

I have an ElasticSearch query issue I cannot resolve that happened when upgrading from 6.4 to 6.8:
My Schema has this
...
"levels" : {
"type" : "nested",
"dynamic" : "false",
"properties" : {
"curriculaCount" : {
"type" : "integer"
},
"hoursEarned" : {
"type" : "half_float"
},
"levelCode" : {
"type" : "keyword"
}
}
},
...
I have a query that works just fine in Kibana dev tools:
{
"query": {
"nested": {
"path": "banner.levels",
"query": {
"bool": {
"filter": [
{
"match": {
"banner.levels.levelCode": "UG"
}
},
{
"range": {
"banner.levels.curriculaCount": {
"gt": 0
}
}
},
{
"range": {
"banner.levels.hoursEarned": {
"lte": 29
}
}
}
]
}
}
}
}
}
My corresponding c# code used to work in 6.4 but is failing in 6.8 and no amount of trying yields me a successful query:
queries.Add(new NestedQuery()
{
Path = Infer.Field<BannerPerson>(x => x.banner.levels),
Query = new BoolQuery()
{
Must = new List<QueryContainer>()
{
new NumericRangeQuery()
{
Field = Infer.Field<BannerPerson>(x => x.banner.levels.First().hoursEarned),
LessThan = 30
},
new NumericRangeQuery()
{
Field = Infer.Field<BannerPerson>(x => x.banner.levels.First().curriculaCount),
GreaterThan = 0
},
new TermQuery()
{
Field = Infer.Field<BannerPerson>(x => x.banner.levels.First().levelCode.Suffix("keyword")),
Value = "UG"
}
}
}
});
Anything I do yields me the dreaded "Invalid NEST response built from a unsuccessful low level call on POST..."
I wish there were a way to get a more detailed error message...
Any help is much appreciated!
UPDATE
I finally fixed the issue and it was unrelated to the code above. Clearing all indices, reloading the schema and then fresh data fixed all the wonky behavior
If updating elasticsearch to a new version, I recommend clearing out the old indices, reloading the schema and reloading fresh data. That fixed the issue.

Elasticsearch query all documents where keyword value is greater than X [7.2]

I am trying to find all documents that have a name that is over 32 characters in length.
This is the mapping of the document.
export const boards = {
handle: {
type: "text"
},
name: {
type: "keyword"
},
};
I tried to use painless to query the size of the field but the following query did not return any results despite the fact that there are.
Query
GET /_search
{
"query": {
"bool" : {
"filter" : {
"script" : {
"script" : {
"source": "doc['name'].size() > 32",
"lang": "painless"
}
}
}
}
}
}
I am thinking that it is perhaps related to the keyword type being used.
I figured out from this forum post that when using keyword size is not the correct method. Instead you need to use:
.value.length() making my final query look like the following:
{
"query": {
"bool" : {
"filter" : {
"script" : {
"script" : {
"source": "doc['name'].value.length() > 32",
"lang": "painless"
}
}
}
}
}
}

Search for documents with exactly different fields values

I'm adding documents with the following strutucte
{
"proposta": {
"matriculaIndicacao": 654321,
"filial": 100,
"cpf": "12345678901",
"idStatus": "3",
"status": "Reprovada",
"dadosPessoais": {
"nome": "John Five",
"dataNascimento": "1980-12-01",
"email": "fulanodasilva#fulano.com.br",
"emailValidado": true,
"telefoneCelular": "11 99876-9999",
"telefoneCelularValidado": true,
"telefoneResidencial": "11 2211-1122",
"idGenero": "1",
"genero": "M"
}
}
}
I'm trying to perform a search with multiple field values.
I can successfull search for a document with a specific cpf atribute with the following search
{
"query": {
"term" : {
"proposta.cpf" : "23798770823"
}
}
}
But now I need to add an AND clause, like
{
"query": {
"term" : {
"proposta.cpf" : "23798770823"
,"proposta.dadosPessoais.dataNascimento": "1980-12-01"
}
}
}
but it's returning an error message.
P.S: If possible I would like to perform a search where if the field doesn't exist, it returns the document that matches only the proposta.cpf field.
I really appreciate any help.
The idea is to combine your constraints within a bool/should query
{
"query": {
"bool": {
"should": [
{
"term": {
"proposta.cpf": "23798770823"
}
},
{
"term": {
"proposta.dadosPessoais.dataNascimento": "1980-12-01"
}
}
]
}
}
}

Elasticsearch query on array index

How do I query/filter by index of an array in elasticsearch?
I have a document like this:-
PUT /edi832/record/1
{
"LIN": [ "UP", "123456789" ]
}
I want to search if LIN[0] is "UP" and LIN[1] exists.
Thanks.
This might look like a hack , but then it will work for sure.
First we apply token count type along with multi field to capture the the number of tokens as a field.
So the mapping will look like this -
{
"record" : {
"properties" : {
"LIN" : {
"type" : "string",
"fields" : {
"word_count": {
"type" : "token_count",
"store" : "yes",
"analyzer" : "standard"
}
}
}
}
}
}
LINK - http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-core-types.html#token_count
So to check if the second field exists , its as easy as checking if this field value is more than or equal to 2.
Next we can use the token filter to check if the token "up" exists in position 0.
We can use the scripted filter to check this.
Hence a query like below should work -
{
"query": {
"filtered": {
"query": {
"range": {
"LIN.word_count": {
"gte": 2
}
}
},
"filter": {
"script": {
"script": "for(pos : _index['LIN'].get('up',_POSITIONS)){ if(pos.position == 0) { return true}};return false;"
}
}
}
}
}
Advanced scripting - http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-advanced-scripting.html
Script filters - http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-script-filter.html

Resources