I was providing the below query to ElasticSearch:
{
"bool": {
"filter": [
[
"000TBT-E",
"07N3P3-E",
"BNTX",
"PFE"
],
{
"terms": {
"query_uuid": [
"0284EFDB592041B7BEC9867C096FD881",
"051AA9969F3140BFABA50B2B195249CF",
"079EF8038042418EA3A219A62A09845A",
"0890230614E14DE59473C9A2ADA72FC9",
"0A8B26197C034154B516652E30100D4D",
"110D2E40E2E04DB6845D0CFC62FA537B",
"12529A8CB715483F98333818FCBD54C8",
"1379AD490F914C8893B68E41ACF115C0",
"137C1B522A37441884B566F914127836",
"13A0E9DCD4DE4350985802D1826252DF",
"FE4F0480A0D04B7FB67B3E9B8BC96496"
]
}
}
]
}
}
When this query was executed the threw the error:
'{"col":106,"line":1,"reason":"[_na] query malformed, must start with start_object","root_cause":[{"col":106,"line":1,"reason":"[_na] query malformed, must start with start_object","type":"parsing_exception"}],"type":"parsing_exception"}'
May I know what I missing here?
There are two issues in your query:
You need to put everything inside the query section
The first filter is not acting on any field
Here is a working query:
{
"query": { <-- add this
"bool": {
"filter": [
{
"terms": { <-- add this
"fieldname": [ <-- add this
"000TBT-E",
"07N3P3-E",
"BNTX",
"PFE"
]
}
},
{
"terms": {
"query_uuid": [
"0284EFDB592041B7BEC9867C096FD881",
"051AA9969F3140BFABA50B2B195249CF",
"079EF8038042418EA3A219A62A09845A",
"0890230614E14DE59473C9A2ADA72FC9",
"0A8B26197C034154B516652E30100D4D",
"110D2E40E2E04DB6845D0CFC62FA537B",
"12529A8CB715483F98333818FCBD54C8",
"1379AD490F914C8893B68E41ACF115C0",
"137C1B522A37441884B566F914127836",
"13A0E9DCD4DE4350985802D1826252DF",
"FE4F0480A0D04B7FB67B3E9B8BC96496"
]
}
}
]
}
}
}
Related
Elastic search version is old 7.9 and Cypress is trying to connect via plugins file using the javascript client. Problem overserved is that the code to fetch the data from required index works all good and logging shows the data fetched and printed on the console when test is executed using the cypress open. But same test fails to fetch the data and just lets the test pass without printing any data on console when test is executed using cypress run.
Code to fetch the data:
const result = await client.search({ index: 'product-v2.0', body: {"query": query}, searchType: "query_then_fetch", trackScores: true, size: 2 })
The query object is as following:
{
"bool": {
"must_not": [
{
"bool": {
"filter": [
{
"exists": {
"field": "gp"
}
}
]
}
}
],
"filter": [
{
"term": {
"webstatus": "1"
}
}
]
}
},
"_source": [
"stockcode"
]
}
When using cypress open -- Following is the part of the trace logging level for the request being fired:
-> POST https://elasticsearch.xxxxxxxxxx.net:443/product-v2.0/_search?search_type=query_then_fetch&track_scores=true&size=2
{
"query": {
"bool": {
"must_not": [
{
"bool": {
"filter": [
{
"exists": {
"field": "gp"
}
}
]
}
}
],
"filter": [
{
"term": {
"webstatus": "1"
}
}
]
}
}
}
That clearly showed that all the search parameters I used are part of the request itself -
POST https://elasticsearch.xxxxxxxxxxxxxx.net:443/product-v2.0/_search?search_type=query_then_fetch&track_scores=true&size=2
My requirement - I need to run it not as ‘cypress open’ - but as ‘cypress run’ process
When I run the same code as cypress run and pass the spec file to be executed - it somehow messes with the query being created and adds and additional ‘query’ to the body of the query instead of placing all the parameters to the request as in case when ‘cypress open’ was used.
Following is the part of the trace logging level for the request being fired via cypress run:
starting request {
"method": "POST",
"path": "/product-v2.0/_search",
"body": {
"query": {
"bool": {
"must_not": [
{
"bool": {
"filter": [
{
"exists": {
"field": "gp"
}
}
]
}
}
],
"filter": [
{
"term": {
"webstatus": "1"
}
}
]
}
}
},
"query": {
"search_type": "query_then_fetch",
"track_scores": true,
"size": 2
}
}
This doesn't fetch any data as well as doens't fail the test and doesn't show much in the logging to convey what happened over there.
Any help to sort this or is there any where else I should raise this issue ?
I have tried using ES JS client for version 8 also but in that case also same observation is seen. Cypress open works but cypress run doesn't.
I want to delete all records from a specified index except for a specific set. Is there such a function in elastic_search?
I tried to use delete_by_query function but could not get it to work as desired. Below is a snippet of what I tried. I basically want to have an array of ids instead of only one id at a time.
POST /myindex/_delete_by_query
{
"query": {
"bool": {
"must_not": [
{
"match": {
"id": {
"query": [12345,67890]
}
}
}
]
}
}
}
I am new to elastic_search, but in SQL terms I want to something like the following query:
DELETE * FROM <my-index> WHERE <id> != <listOfIds>
Good start!! You can do it like you suggest with a terms query:
POST /myindex/_delete_by_query
{
"query": {
"bool": {
"must_not": [
{
"terms": {
"id": [
12345,
67890
]
}
}
]
}
}
}
I am new in Elasticsearch. I am trying to convert query below in MongoDB to elastic:
let TXs = await db.collection("transactions").find({
$or: [
{"sender.wallet.address": {$in: addresses}, currency: currency},
{"receiver.wallet.address": {$in: addresses}, currency: currency}
]
}).sort({unix: 1}).toArray().catch(console.error);
addresses is an array of strings. I used code block below, but I got error:
let TXs = await elasticClient.search({
index: elasticIndex,
"query": {
"terms": {
"should": [
{"bool": {"must": [{"term": {"sender.wallet.address": addresses}}, {"term": {"currency": currency}}]}},
{"bool": {"must": [{"term": {"receiver.wallet.address": addresses}}, {"term": {"currency": currency}}]}}
]
}
}
}).catch(console.error);
Error:
ResponseError: parsing_exception: [parsing_exception] Reason: Unknown
key for a END_ARRAY in [query].
Could someone help how can I wrote it in Elastic?
At the beginning of your search query, you need to use the should clause, instead of the terms clause. Modify your query as
{
"query": {
"bool": {
"should": [
{
"bool": {
"must": [
{
"term": {
"sender.wallet.address": "addresses"
}
},
{
"term": {
"currency": currency
}
}
]
}
},
{
"bool": {
"must": [
{
"term": {
"receiver.wallet.address": "addresses"
}
},
{
"term": {
"currency": currency
}
}
]
}
}
]
}
}
}
Update 1:
Based on your MongoDB query,
{"sender.wallet.address": {$in: addresses}, currency: currency},
since you are using $in, you should replace the term query inside the bool must clause with the terms query, which will allow you to use arrays of strings, instead of a string.
I have changed the code block to:
let TXs = await elasticClient.search({
index: elasticIndex,
query: {
bool: {
filter: {term: {currency: currency}},
should: [
{terms: {"sender.wallet.address": addresses}},
{terms: {"receiver.wallet.address": addresses}}
]
}
}
}).catch(console.error);
I don't get any errors now, but it seems "filter" and "should" cannot work in a harmony. If I remove the filter, I get some documents, but when I add the filter, I get nothing. I think I don't combine these two in a right way.
I have a database model for a user as:
{
"referenceId": "123",
"name": "Frank",
"department": "CS",
"negativeUserList": [ "980", "739" ]
}
I am writing an ES 2.4 Query with which I can perform FTS query on user name with given department but the result should not contain a user with his ID in negativeUserList array. So, in the above object, even if User ID 980 has the same name and department, he (980) should not appear in searching user's (123) FTS. I have successfully written the following query which is working good:
{
"query": {
"bool": {
"must": {
"multi_match": {
"query": "A",
"fields": [
"name.ngram"
],
"minimum_should_match": "100%"
}
},
"filter": {
"term": {
"department": "cs"
}
},
"must_not": {
"terms": {
"referenceId": [
"980",
"739"
]
}
}
}
}
}
But when I integrate the same with Spring ElasticsearchRepository as:
#Query("{ \"query\": { \"bool\": { \"must\": { \"multi_match\": { \"query\": \"?0\", \"fields\": [ \"name.ngram\" ], \"minimum_should_match\": \"100%\" } }, \"filter\": { \"term\": { \"department\": \"?1\" } }, \"must_not\": { \"terms\": { \"referenceId\": [ \"?2\" ] } } } } }")
Page<UserSummary> getFreeTextSearchForUser(String freeText, String currentUserDepartment, List<String> negativeUserList, Pageable pageable);
Then I do not get any results when currentUserDepartment is cs and negativeUserList is [ "980", "739" ] despite of getting results when I run the native query on ES REST endpoint directly.
As of now, most of the users have an empty array in negativeUserList and it should run normally.
EDIT:
When I modify the Spring query with:
{ \"referenceId\": \"?2\" }
i.e. without the square brackets around \"?2\", I get an error as:
Caused by: org.elasticsearch.index.query.QueryParsingException: [terms] query does not support [referenceId]
at org.elasticsearch.index.query.TermsQueryParser.parse(TermsQueryParser.java:155)
If I now remove the double-quotes:
{ \"referenceId\": ?2 }
I simply do not get back any results but no error. What am I doing wrong?
I have such index:
{
"id":2,
"name":"test home",
"city_id":"31",
"county_id":"1",
"zip_code":"123",
"residencePlans":[
{
"id" : 1,
"unit_price_from":480240,
"bathrooms_count":3,
"interior_area_sqrft":23,
"floor_range_hight":5,
"bedrooms_count":5,
"elevator_type_id":4,
"price_psqft":3756,
},
{
"id" : 2,
"unit_price_from":123456,
"bathrooms_count":1,
"interior_area_sqrft":12,
"floor_range_hight":4,
"bedrooms_count":2,
"elevator_type_id":3,
"price_psqft":1234,
}
],
}
And then I use some filters. Some of them are applied to the top object, and some to nesting.
I need to query residencePlans, that match filter, applied for their. eg filter on residencePlans.bathrooms_count >= 3 should return only residence with id = 1 and not 2.
{
"id": [2],
"residencePlans.id": [1]
}
I marked residencePlans as nested mapping, but it doesn't help.
Checkout the documentation here: https://www.elastic.co/guide/en/elasticsearch/guide/current/nested-query.html
And here: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-inner-hits.html
Something like this should do it
{
"query": {
"bool": {
"must": [
{ "match": { "id": 1 }},
{
"nested": {
"path": "residencePlans",
"query": {
"bool": {
"must": [
{ "gte": { "residencePlans.unit_price_from": 3 }}
]
}
}
}
}
]
},
inner_hits: {}
}
}
I've revised my answer to take into account the particulars of filtering your top level document and your nested documents. Please let me know if it works for you!