I am new in Elasticsearch. I am trying to convert query below in MongoDB to elastic:
let TXs = await db.collection("transactions").find({
$or: [
{"sender.wallet.address": {$in: addresses}, currency: currency},
{"receiver.wallet.address": {$in: addresses}, currency: currency}
]
}).sort({unix: 1}).toArray().catch(console.error);
addresses is an array of strings. I used code block below, but I got error:
let TXs = await elasticClient.search({
index: elasticIndex,
"query": {
"terms": {
"should": [
{"bool": {"must": [{"term": {"sender.wallet.address": addresses}}, {"term": {"currency": currency}}]}},
{"bool": {"must": [{"term": {"receiver.wallet.address": addresses}}, {"term": {"currency": currency}}]}}
]
}
}
}).catch(console.error);
Error:
ResponseError: parsing_exception: [parsing_exception] Reason: Unknown
key for a END_ARRAY in [query].
Could someone help how can I wrote it in Elastic?
At the beginning of your search query, you need to use the should clause, instead of the terms clause. Modify your query as
{
"query": {
"bool": {
"should": [
{
"bool": {
"must": [
{
"term": {
"sender.wallet.address": "addresses"
}
},
{
"term": {
"currency": currency
}
}
]
}
},
{
"bool": {
"must": [
{
"term": {
"receiver.wallet.address": "addresses"
}
},
{
"term": {
"currency": currency
}
}
]
}
}
]
}
}
}
Update 1:
Based on your MongoDB query,
{"sender.wallet.address": {$in: addresses}, currency: currency},
since you are using $in, you should replace the term query inside the bool must clause with the terms query, which will allow you to use arrays of strings, instead of a string.
I have changed the code block to:
let TXs = await elasticClient.search({
index: elasticIndex,
query: {
bool: {
filter: {term: {currency: currency}},
should: [
{terms: {"sender.wallet.address": addresses}},
{terms: {"receiver.wallet.address": addresses}}
]
}
}
}).catch(console.error);
I don't get any errors now, but it seems "filter" and "should" cannot work in a harmony. If I remove the filter, I get some documents, but when I add the filter, I get nothing. I think I don't combine these two in a right way.
Related
I was providing the below query to ElasticSearch:
{
"bool": {
"filter": [
[
"000TBT-E",
"07N3P3-E",
"BNTX",
"PFE"
],
{
"terms": {
"query_uuid": [
"0284EFDB592041B7BEC9867C096FD881",
"051AA9969F3140BFABA50B2B195249CF",
"079EF8038042418EA3A219A62A09845A",
"0890230614E14DE59473C9A2ADA72FC9",
"0A8B26197C034154B516652E30100D4D",
"110D2E40E2E04DB6845D0CFC62FA537B",
"12529A8CB715483F98333818FCBD54C8",
"1379AD490F914C8893B68E41ACF115C0",
"137C1B522A37441884B566F914127836",
"13A0E9DCD4DE4350985802D1826252DF",
"FE4F0480A0D04B7FB67B3E9B8BC96496"
]
}
}
]
}
}
When this query was executed the threw the error:
'{"col":106,"line":1,"reason":"[_na] query malformed, must start with start_object","root_cause":[{"col":106,"line":1,"reason":"[_na] query malformed, must start with start_object","type":"parsing_exception"}],"type":"parsing_exception"}'
May I know what I missing here?
There are two issues in your query:
You need to put everything inside the query section
The first filter is not acting on any field
Here is a working query:
{
"query": { <-- add this
"bool": {
"filter": [
{
"terms": { <-- add this
"fieldname": [ <-- add this
"000TBT-E",
"07N3P3-E",
"BNTX",
"PFE"
]
}
},
{
"terms": {
"query_uuid": [
"0284EFDB592041B7BEC9867C096FD881",
"051AA9969F3140BFABA50B2B195249CF",
"079EF8038042418EA3A219A62A09845A",
"0890230614E14DE59473C9A2ADA72FC9",
"0A8B26197C034154B516652E30100D4D",
"110D2E40E2E04DB6845D0CFC62FA537B",
"12529A8CB715483F98333818FCBD54C8",
"1379AD490F914C8893B68E41ACF115C0",
"137C1B522A37441884B566F914127836",
"13A0E9DCD4DE4350985802D1826252DF",
"FE4F0480A0D04B7FB67B3E9B8BC96496"
]
}
}
]
}
}
}
I am encountering an issue trying to correctly combine elastic search queries, in SQL my query would look something like this:
Select * from ABPs where (PId = 10 and PUId = 1130) or (PId = 30 and PUId = 2000) or (PlayerID = '12345')
I can achieve each of these by themselves and get correct results.
Query A) (PId = 10 and PUId = 1130)
translates to
{
"query": {
"bool": {
"must": [
{
"term": {
"PId": "1366"
}
},
{
"term": {
"PUId": "10"
}
}
]
}
}
}
Query B) (PId = 10 and PUId = 1130)
translates the same as above just with different values
Query C) (PlayerID = '12345')
translates to
{
"query": {
"match": {
"PlayerUuid": "62fe0832-7881-477c-88bb-9cbccdbfb3c3"
}
}
}
I have been trying to figure out how to get all of these into the same ES search query and I am just not having any luck at all and was hoping someone with more extensive ES experience would be able to give me a hand.
You can make use of Bool query using should(Logical OR) and must(Logical AND) clause.
Below is the ES query representation of the clause Select * from ABPs where (PId = 10 and PUId = 1130) or (PId = 30 and PUId = 2000) or (PlayerID = '12345')
POST <your_index_name>/_search
{
"query": {
"bool": {
"should": [
{
"bool": {
"must": [
{
"term": {
"PId": "10"
}
},
{
"term": {
"PUId": {
"value": "1130"
}
}
}
]
}
},
{
"bool": {
"must": [
{
"term": {
"PId": "30"
}
},
{
"term": {
"PUId": "2000"
}
}
]
}
},
{
"bool": {
"must": [
{
"term": {
"PlayerId": "12345"
}
}
]
}
}
]
}
}
}
Note that I'm assuming the fields PId, PUId and PlayerId are all of type keyword.
Wrap all your queries into a should-clause of a bool-query which you put in the filter-clause of another top-level bool-query.
Pseudo-code (as I’m typing on a cell phone):
“bool”: {
“filter”: {
“bool”: {
“should”: [
{query1},
{query2},
{query3}
]
}
}
}
In a bool- query made up of only should-clauses, will make it a requirement that at least one of the queries in the should-clause has to match (minimum_should_match-will be in such a scenario).
Update with the actual query (additional explanation):
POST <your_index_name>/_search
{
"query": {
"bool": {
"filter": {
"bool": {
"should": [
{"bool": {{"must": [ {"term": {"PId": "10"}},{"term": {"PUId": "1130"}} ]}}},
{"bool": {{"must": [ {"term": {"PId": "30"}},{"term": {"PUId": "2000"}} ]}}},
{"term": {"PlayerId": "12345"}}
]
}
}
}
}
}
The example above is wrapping your actual bool-query in a filer-clause of another top-level bool-query to follow best-practices and guarantee for a better performance: whenever you don't care about the score, especially when it's always about exact-matching queries, you should put them into filter-clauses. For those queries Elasticsearch will not calculate a score and therefore can even potentially cache the results of that query for even better performance.
I have a query
GET index/_search
{
"query": {
"bool": {
"should": [
{
"match": {
"key1": "value"
}
},
{
"wildcard": {
"key2": "*match*"
}
}
]
}
}
}
I want to make the same call with elasticsearch_dsl package
I tried with
s = Search(index=index).query({
"bool": {
"should": [
{
"match": {
"key1": "value"
}
},
{
"wildcard": {
"key2": "*match*"
}
}
]
}
})
s.using(self.client).scan()
But the results are not same, am I missing something here
Is there a way to represent my query with elasticsearch_dsl
tried this, no results
s = Search(index=index).query('wildcard', key2='*match*').query('match', key1=value)
s.using(self.client).scan()
it seems to me that you forgot the stars in the query.
s = Search(index=index).query('wildcard', key='*match*').query('match', key=value)
This query worked for me
s = Search(index=index).query('match', key1=value)
.query('wildcard', key2='*match*')
.source(fields)
also, if key has _ like key_1 elastic search behaves differently and query matches results even which do not match your query. So try to choose your key which do not have underscores.
In our Elasticsearch collection of products, we have an an array of hashes, called "nutrients". A partial example of the data would be:
"_source": {
"quantity": "150.0",
"id": 1001,
"barcode": "7610809001066",
"nutrients": [
{
"per_hundred": "1010.0",
"name_fr": "Énergie",
"per_portion": "758.0",
"name_de": "Energie",
"per_day": "9.0",
"name_it": "Energia",
"name_en": "Energy"
},
{
"per_hundred": "242.0",
"name_fr": "Énergie (kCal)",
"per_portion": "181.0",
"name_de": "Energie (kCal)",
"per_day": "9.0",
"name_it": "Energia (kCal)",
"name_en": "Energy (kCal)"
},
{
"per_hundred": "18.0",
"name_fr": "Matières grasses",
"per_portion": "13.5",
"name_de": "Fett",
"per_day": "19.0",
"name_it": "Grassi",
"name_en": "Fat"
},
In the search, we are trying to bring back the products based on an exact match of two of the fields contained in the nutrients array. What I am finding is the conditions seemed to be OR and not AND.
The two attempts have been:
"query": {
"bool": {
"must": [
{ "match": { "nutrients.name_fr": "Énergie" } },
{ "match": { "nutrients.per_hundred": "242.0" } }
]
}
}
}
and
"query": {
"filtered": {
"filter": {
"and": [
{ "term": { "nutrients.name_fr": "Énergie" } },
{ "term": { "nutrients.per_hundred": "242.0" } }
]
}
}
}
Both of these are in fact bringing back entries with Énergie and 242.0, but are also match on different name_fr, eg:
{
"per_hundred": "242.0",
"name_fr": "Acide folique",
"per_portion": "96.0",
"name_de": "Folsäure",
"per_day": "48.0",
"name_it": "Acido folico",
"name_en": "Folic acid"
},
They are also matching on a non exact match, i.e: matching also on "Énergie (kCal)" when we want to match only on "Énergie"
On your first problem:
You have to make the nutrients field nested, so you can query each object inside it for itself Elasticsearch Nested Objects.
I have such index:
{
"id":2,
"name":"test home",
"city_id":"31",
"county_id":"1",
"zip_code":"123",
"residencePlans":[
{
"id" : 1,
"unit_price_from":480240,
"bathrooms_count":3,
"interior_area_sqrft":23,
"floor_range_hight":5,
"bedrooms_count":5,
"elevator_type_id":4,
"price_psqft":3756,
},
{
"id" : 2,
"unit_price_from":123456,
"bathrooms_count":1,
"interior_area_sqrft":12,
"floor_range_hight":4,
"bedrooms_count":2,
"elevator_type_id":3,
"price_psqft":1234,
}
],
}
And then I use some filters. Some of them are applied to the top object, and some to nesting.
I need to query residencePlans, that match filter, applied for their. eg filter on residencePlans.bathrooms_count >= 3 should return only residence with id = 1 and not 2.
{
"id": [2],
"residencePlans.id": [1]
}
I marked residencePlans as nested mapping, but it doesn't help.
Checkout the documentation here: https://www.elastic.co/guide/en/elasticsearch/guide/current/nested-query.html
And here: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-inner-hits.html
Something like this should do it
{
"query": {
"bool": {
"must": [
{ "match": { "id": 1 }},
{
"nested": {
"path": "residencePlans",
"query": {
"bool": {
"must": [
{ "gte": { "residencePlans.unit_price_from": 3 }}
]
}
}
}
}
]
},
inner_hits: {}
}
}
I've revised my answer to take into account the particulars of filtering your top level document and your nested documents. Please let me know if it works for you!