I have an ElasticSearch query issue I cannot resolve that happened when upgrading from 6.4 to 6.8:
My Schema has this
...
"levels" : {
"type" : "nested",
"dynamic" : "false",
"properties" : {
"curriculaCount" : {
"type" : "integer"
},
"hoursEarned" : {
"type" : "half_float"
},
"levelCode" : {
"type" : "keyword"
}
}
},
...
I have a query that works just fine in Kibana dev tools:
{
"query": {
"nested": {
"path": "banner.levels",
"query": {
"bool": {
"filter": [
{
"match": {
"banner.levels.levelCode": "UG"
}
},
{
"range": {
"banner.levels.curriculaCount": {
"gt": 0
}
}
},
{
"range": {
"banner.levels.hoursEarned": {
"lte": 29
}
}
}
]
}
}
}
}
}
My corresponding c# code used to work in 6.4 but is failing in 6.8 and no amount of trying yields me a successful query:
queries.Add(new NestedQuery()
{
Path = Infer.Field<BannerPerson>(x => x.banner.levels),
Query = new BoolQuery()
{
Must = new List<QueryContainer>()
{
new NumericRangeQuery()
{
Field = Infer.Field<BannerPerson>(x => x.banner.levels.First().hoursEarned),
LessThan = 30
},
new NumericRangeQuery()
{
Field = Infer.Field<BannerPerson>(x => x.banner.levels.First().curriculaCount),
GreaterThan = 0
},
new TermQuery()
{
Field = Infer.Field<BannerPerson>(x => x.banner.levels.First().levelCode.Suffix("keyword")),
Value = "UG"
}
}
}
});
Anything I do yields me the dreaded "Invalid NEST response built from a unsuccessful low level call on POST..."
I wish there were a way to get a more detailed error message...
Any help is much appreciated!
UPDATE
I finally fixed the issue and it was unrelated to the code above. Clearing all indices, reloading the schema and then fresh data fixed all the wonky behavior
If updating elasticsearch to a new version, I recommend clearing out the old indices, reloading the schema and reloading fresh data. That fixed the issue.
Related
When I use ES 5.5 update to 6.7.
Painless script does’t work
This is 5.5
If I want get a nested document [transFilter]
I do this
params['_source’]['carFilter’]
It works very well。
But
When I used 6.7 version
params['_source’]['carFilter’]
I found it does’t work
All params['_source’] is null
my mappings
carFilter": {
"type": "nested",
"properties": {
"time": {
"type": "long"
}
}
}
my data example
"carFilter" : [
{
"time" : 20200120
},
{
"time" : 20200121
}
]
and my query script example
{
"query" : {
"bool" : {
"must" : [
{
"script" : {
"script" : {
"inline" : "if(params['_source']!=null){
if(params['_source']['carFilter']!=null){
for(def item:params['_source']['carFilter'] ){
if (item.time>1) { return true; }
}
}
}
return false;",
"lang" : "painless",
"params" : {
"rentTime" : 1000
}
}
}
}
]
}
}
}
even no error
but fact
if(params['_source']!=null){
this line already return
The simple painless above is just to illustrate the problem, and a relatively real one is attached below.
double carPrice=0.00;if(!params['_source'].empty){"+
" def days=params['_source']['everyDayPrice'];if(params['_source']['everyDayPrice']!=null){int size=days.length;" +
" if(size>0){for(int i=0;i<size;i++){String day = days[i]['day'];Double price = days[i]['price'];"+
" if(price!=null&¶ms.get(day)!=null){carPrice=carPrice+params.get(day)*price;}}}}}" +
" return carPrice/params.total"
Looking at your query, you would want to filter the documents having carFilter.time > 1 and why not use a simple Nested Query:
POST <your_index_name>/_search
{
"query": {
"bool": {
"must": [
{
"nested": {
"path": "carFilter",
"query": {
"range": {
"carFilter.time": {
"gte": 1
}
}
}
}
}
]
}
}
}
Note that I've made use of Range Query to evaluate the time based on what you are looking for.
I'd suggest you go through this answer if the above doesn't help.
Let me know if you have any queries.
I'm adding documents with the following strutucte
{
"proposta": {
"matriculaIndicacao": 654321,
"filial": 100,
"cpf": "12345678901",
"idStatus": "3",
"status": "Reprovada",
"dadosPessoais": {
"nome": "John Five",
"dataNascimento": "1980-12-01",
"email": "fulanodasilva#fulano.com.br",
"emailValidado": true,
"telefoneCelular": "11 99876-9999",
"telefoneCelularValidado": true,
"telefoneResidencial": "11 2211-1122",
"idGenero": "1",
"genero": "M"
}
}
}
I'm trying to perform a search with multiple field values.
I can successfull search for a document with a specific cpf atribute with the following search
{
"query": {
"term" : {
"proposta.cpf" : "23798770823"
}
}
}
But now I need to add an AND clause, like
{
"query": {
"term" : {
"proposta.cpf" : "23798770823"
,"proposta.dadosPessoais.dataNascimento": "1980-12-01"
}
}
}
but it's returning an error message.
P.S: If possible I would like to perform a search where if the field doesn't exist, it returns the document that matches only the proposta.cpf field.
I really appreciate any help.
The idea is to combine your constraints within a bool/should query
{
"query": {
"bool": {
"should": [
{
"term": {
"proposta.cpf": "23798770823"
}
},
{
"term": {
"proposta.dadosPessoais.dataNascimento": "1980-12-01"
}
}
]
}
}
}
I have a template with a mapping like this:
"template" : "database*",
"mappings": {
"_default_" : {
"dynamic_templates" : [
{
// default string mapping
"string" : {
"match" : "*",
"match_mapping_type": "string",
"mapping": {
"type": "text",
"fields": {
"raw" : {
"type": "keyword"
}
}
}
}
},
]
}
}
The idea behind this template is to generate for every new field a "type : keyword" field for exact searches.
When adding documents (in an empty index using this template) with JS API client.index() everything works fine, this way i am able to query like:
{
"query": {
"match": {
"fooBar": "bla"
}
}
}
Or for exact searches like:
{
"query": {
"match": {
"fooBar.raw": "bla"
}
}
}
But adding documents (again empty index) in bulk with client.bulk() like this:
client.bulk({
body: [
// FIRST DOCUMENT
{ index: { _index: 'myindex', _type: 'mytype', _id: 1 } },
// the first document to index
{ fooBar: 'foo' },
// SECOND DOCUMENT
{ index: { _index: 'myindex', _type: 'mytype', _id: 2 } },
// the second document to index
{ fooBar: 'bar' },
(...)
]
}, function (err, resp) {
// ...
});
The same "raw" request results in an error:
"No mapping found for [fooBar.raw] (...)"
while:
{
"query": {
"match": {
"fooBar": "bla"
}
}
}
delivers results. This brings me to the conclusion, that the document is indeed indexed but the field "raw" has not been created.
Is that right? Is in fact bulk Indexing not using mapping?
How can I use bulk import and map the documents?
Thanks! :)
Im trying to search documents with wildcard and _all. But It does not seem like it's possible to get boosted result with wildcard on _all ?
MappingRequest:
"theboostingclass": {
"properties": {
"Important": {
"boost": 2.0,
"type": "string"
},
"LessImportant": {
"type": "string"
},
"Garbage": {
"type": "string"
}
}
}
}
Indexing:
{
"index" :
{
"_index":"boosting",
"_type":"theboostingclass"
}
}
{
"Important":"bomb",
"LessImportant":"kruka",
"Garbage":"kalkon"
}
{
"index" :
{
"_index":"boosting",
"_type":"theboostingclass"
}
}
{
"Important":"kalkon",
"LessImportant":"bomb",
"Garbage":"bomber"
}
{
"index" :
{
"_index":"boosting",
"_type":"theboostingclass"
}
}
{
"Important":"kruka",
"LessImportant":"bomber",
"Garbage":"bomb"
}
Query
"query": {
"wildcard": {
"_all": {
"value": "*bomb*"
}
}
}
The result returs all hits with a Score of 1 and a seemingly random order. Which is not really what Im after. I want the hit on "Important"field to yield a higher score.
If I do a wildcard search on all 3 fields the scoring seems correct. However I want to use it on _all. Any ideas?
Please see documentation here:
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-multi-term-rewrite.html. Note that the reason it works with a constant scoring by default is for performance.
I believe you need to modify your query as follows:
"query": {
"wildcard": {
"_all": {
"value": "*bomb*",
"rewrite": "scoring_boolean"
}
}
}
How do I query/filter by index of an array in elasticsearch?
I have a document like this:-
PUT /edi832/record/1
{
"LIN": [ "UP", "123456789" ]
}
I want to search if LIN[0] is "UP" and LIN[1] exists.
Thanks.
This might look like a hack , but then it will work for sure.
First we apply token count type along with multi field to capture the the number of tokens as a field.
So the mapping will look like this -
{
"record" : {
"properties" : {
"LIN" : {
"type" : "string",
"fields" : {
"word_count": {
"type" : "token_count",
"store" : "yes",
"analyzer" : "standard"
}
}
}
}
}
}
LINK - http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-core-types.html#token_count
So to check if the second field exists , its as easy as checking if this field value is more than or equal to 2.
Next we can use the token filter to check if the token "up" exists in position 0.
We can use the scripted filter to check this.
Hence a query like below should work -
{
"query": {
"filtered": {
"query": {
"range": {
"LIN.word_count": {
"gte": 2
}
}
},
"filter": {
"script": {
"script": "for(pos : _index['LIN'].get('up',_POSITIONS)){ if(pos.position == 0) { return true}};return false;"
}
}
}
}
}
Advanced scripting - http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/modules-advanced-scripting.html
Script filters - http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl-script-filter.html