I need to delete information that match exactly the word of the keys in the query of elastic-search, but I have problems with the request and I deleted information with the same prefix. What I have to do to fix my script and delete only the correct ones? (Exact Match on two conditions)
curl -X POST elasticDomain/index/_delete_by_query -d '{"query": {
"bool": {
"must": [
{
"term": {
"component.name": {
"query" : "prefix-component-one"
}
}
},
{
"term": {
"enviroment": "qa"
}
}
]
}}}'
Data example, when I want to delete only information about component-one:
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 2,
"max_score": 1,
"hits": [
{
"_index": "index",
"_type": "event",
"_id": "c04b0f94-4995-11e8-a9f5-a22f517abdda",
"_score": 1,
"_source": {
"component": {
"name": "prefix-component-two",
"qualifier": "TRK"
},
"enviroment": "history",
"timestamp": "2018-04-26T16:06:54.000Z"
}
},
{
"_index": "index",
"_type": "event",
"_id": "bf80d63e-4995-11e8-a9f5-a22f517abdda",
"_score": 1,
"_source": {
"component": {
"name": "prefix-component-one",
"qualifier": "TRK"
},
"enviroment": "qa",
"timestamp": "2018-04-26T16:06:54.000Z"
}
}
]
}
}
I fixed with match_phrase
curl -X POST elasticDomain/index/_delete_by_query -d '{"query": {
"bool": {
"must": [
{
"match_phrase": {
"component.name": {
"query" : "prefix-component-one"
}
}
},
{
"term": {
"enviroment": "qa"
}
}
]
}}}
Related
I'm using ElasticSearch 7.0
Given the mapping:
{
"searchquestion": {
"mappings": {
"properties": {
"server": {
"properties": {
"hostname": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
}
}
I have put the following documents into this index:
{
"server": {
"hostname": "server1-windows.loc2.uk"
}
}
{
"server": {
"hostname": "server1-windows.loc2.uk"
}
}
{
"server": {
"hostname": "server1-linux.loc1.uk"
}
}
I would like to query the exact text of the hostname. Luckily, this can be done because there is an additional keyword type field on this field.
Successful query :
{
"query": {
"bool": {
"must": [
{
"match": {
"server.hostname.keyword": {
"query": "server1-windows.loc2.uk"
}
}
}
]
}
}
}
However, I would like to extend this query string, to include another hostname to search for. In my results, I expect to have both documents returned.
My attempt:
{
"query": {
"bool": {
"must": [
{
"match": {
"server.hostname.keyword": {
"query": "server1-windows.loc2.uk server1-linux.loc1.uk",
"operator": "or"
}
}
}
]
}
}
}
This returns no hits, I suspect because the default analyser is splitting this query up into sections, but I'm actually searching the keyword field which is a full string. I cannot add analyzer: keyword to this query search, as server1-windows.loc2.uk server1-linux.loc1.uk as an exact string won't match anything either.
How can I search for both these strings, as their complete selves?
i.e. "query": ["server1-windows.loc2.uk", "server1-linux.loc1.uk"]
I would also like to use wildcards to match any loc. I would expect
"query": ["server1-windows.*.uk"] to match both windows servers, but I get no hits.
What am I missing?
you can use Query_String to get your desired result
Case 1:
Query:
GET server/_search
{
"query": {
"query_string": {
"query": "(server1-windows.loc2.uk) OR (server1-linux.loc1.uk)",
"default_field": "server.hostname.keyword"
}
}
}
Output:
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 3,
"relation": "eq"
},
"max_score": 0.9808291,
"hits": [
{
"_index": "server",
"_id": "3",
"_score": 0.9808291,
"_source": {
"server": {
"hostname": "server1-linux.loc1.uk"
}
}
},
{
"_index": "server",
"_id": "1",
"_score": 0.4700036,
"_source": {
"server": {
"hostname": "server1-windows.loc2.uk"
}
}
},
{
"_index": "server",
"_id": "2",
"_score": 0.4700036,
"_source": {
"server": {
"hostname": "server1-windows.loc2.uk"
}
}
}
]
}
}
Case 2: with wildcard(*)
Query:
GET server/_search
{
"query": {
"query_string": {
"query": "server1-windows.*.uk",
"default_field": "server.hostname.keyword"
}
}
}
Output:
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": 1,
"hits": [
{
"_index": "server",
"_id": "1",
"_score": 1,
"_source": {
"server": {
"hostname": "server1-windows.loc2.uk"
}
}
},
{
"_index": "server",
"_id": "2",
"_score": 1,
"_source": {
"server": {
"hostname": "server1-windows.loc2.uk"
}
}
}
]
}
}
I have an index, which stores a nested document. I wanna see this nested documents, for this purpose I used 'inner_hits' in request, but elastic returns nullPointerException. Do anyone meet with this problem?)
Request to elasticsearch using Postman:
GET http://localhost/my-index/_search
{
"query": {
"nested": {
"path": "address_object",
"query": {
"bool": {
"must": {
"term": {"address_object.city": "Paris"}
}
}
},
"inner_hits" : {}
}
}
}
Response with status code 200:
{
"took": 161,
"timed_out": false,
"_shards": {
"total": 2,
"successful": 1,
"skipped": 0,
"failed": 1,
"failures": [
{
"shard": 0,
"index": "my-index",
"node": "DWdD83KaTmUiodENQkGDww",
"reason": {
"type": "null_pointer_exception",
"reason": null
}
}
]
},
"hits": {
"total": 6500039,
"max_score": 2.1761138,
"hits": []
}
}
Elasticsearch version: 6.2.4
Lucene version: 7.2.1
Update:
Mapping:
{
"my-index": {
"mappings": {
"mytype": {
"dynamic": "false",
"_source": {
"enabled": false
},
"properties": {
"adverts_count": {
"type": "integer",
"store": true
},
...
"address_object": {
"type": "nested",
"properties": {
"adverts_count": {
"type": "integer",
"store": true
},
"city": {
"type": "keyword",
"store": true
}
}
},
...
Sample document:
{
"_index": "my-index",
"_type": "mytype",
"_id": "XDWrGncBdwNBWGEagAM2",
"_score": 2.1587489,
"fields": {
"is_target_page_shown": [
0
],
"updated_at": [
1612264276
],
"is_shown": [
0
],
"nb_queries": [
1
],
"search_query": [
"phone"
],
"target_category": [
15
],
"adverts_count": [
1
]
}
}
Extra information:
If I remove the "inner_hits": {} from search request, elastic returns nested documents(_index, _type, _id, _score), but ain't other fields(e.g city)
Also, as suggested in the comments, I tried setting to true ignore_unmapped, but it doesn't helped. The same nullPointerException.
I tried reproducing your issue, but as you have not provided the proper sample documents(one which you provided doesn't have the address_object properties), I used your mapping and below sample documents.
PUT index-name/_doc/1
{
"address_object" :{
"adverts_count" : 1,
"city": "paris"
}
}
PUT index-name/_doc/2
{
"address_object" :{
"adverts_count" : 1,
"city": "blr"
}
}
And when I use the same search provided by you.
POST 71907588/_search
{
"query": {
"nested": {
"path": "address_object",
"query": {
"bool": {
"must": {
"term": {
"address_object.city": "paris"
}
}
}
},
"inner_hits": {}
}
}
}
I get a proper response, matching paris as city as shown in the search response.
"hits": [
{
"_index": "71907588",
"_id": "1",
"_score": 0.6931471,
"_source": {
"address_object": {
"adverts_count": 1,
"city": "paris"
}
},
"inner_hits": {
"address_object": {
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 0.6931471,
"hits": [
{
"_index": "71907588",
"_id": "1",
"_nested": {
"field": "address_object",
"offset": 0
},
"_score": 0.6931471,
"_source": {
"city": "paris",
"adverts_count": 1
}
}
]
}
}
}
}
]
I am new to ElasticSearch and can't quite figure out what I want is possible or not.
I can query like this:
GET entity/_search
{
"query": {
"bool": {
"must": [
{ "match": { "searchField": "searchValue" }}
]
}
},
"aggs" : {
"uniq_Id" : {
"terms" : { "field" : "Id", "size":500 }
}
}
}
and it will return top search results and the term aggregation buckets. But ideally what I would like for the search results to return, is only one (perhaps the top one, does not matter) for each of unique Id's defined in the aggregation terms.
You can make use of Terms Aggregation along with the Top Hits Aggregation to give you the result you are looking for.
Now once you do that, specify the size as 1 in the Top Hits Aggregation
Based on your query I've created sample mapping,documents, aggregation query and the response for your reference.
Mapping:
PUT mysampleindex
{
"mappings": {
"mydocs": {
"properties": {
"searchField":{
"type": "text"
},
"Id": {
"type": "keyword"
}
}
}
}
}
Sample Documents:
POST mysampleindex/mydocs/1
{
"searchField": "elasticsearch",
"Id": "1000"
}
POST mysampleindex/mydocs/2
{
"searchField": "elasticsearch is awesome",
"Id": "1000"
}
POST mysampleindex/mydocs/3
{
"searchField": "elasticsearch is awesome",
"Id": "1001"
}
POST mysampleindex/mydocs/4
{
"searchField": "elasticsearch is pretty cool",
"Id": "1001"
}
POST mysampleindex/mydocs/5
{
"searchField": "elasticsearch is pretty cool",
"Id": "1002"
}
Query:
POST mysampleindex/_search
{
"size": 0,
"query": {
"bool": {
"must": [
{
"match": {
"searchField": "elasticsearch"
}
}
]
}
},
"aggs": {
"myUniqueIds": {
"terms": {
"field": "Id",
"size": 10
},
"aggs": {
"myDocs": {
"top_hits": { <---- Top Hits Aggregation
"size": 1 <---- Note this
}
}
}
}
}
}
Sample Response:
{
"took": 7,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 5,
"max_score": 0,
"hits": []
},
"aggregations": {
"myUniqueIds": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "1000",
"doc_count": 2,
"myDocs": {
"hits": {
"total": 2,
"max_score": 0.2876821,
"hits": [
{
"_index": "mysampleindex",
"_type": "mydocs",
"_id": "1",
"_score": 0.2876821,
"_source": {
"searchField": "elasticsearch",
"Id": "1000"
}
}
]
}
}
},
{
"key": "1001",
"doc_count": 2,
"myDocs": {
"hits": {
"total": 2,
"max_score": 0.25316024,
"hits": [
{
"_index": "mysampleindex",
"_type": "mydocs",
"_id": "3",
"_score": 0.25316024,
"_source": {
"searchField": "elasticsearch is awesome",
"Id": "1001"
}
}
]
}
}
},
{
"key": "1002",
"doc_count": 1,
"myDocs": {
"hits": {
"total": 1,
"max_score": 0.2876821,
"hits": [
{
"_index": "mysampleindex",
"_type": "mydocs",
"_id": "5",
"_score": 0.2876821,
"_source": {
"searchField": "elasticsearch is pretty cool",
"Id": "1002"
}
}
]
}
}
}
]
}
}
}
Notice that I am not returning any bool results in the above, the search result you are looking for comes in the form of Top Hits Aggregation.
Hope this helps!
I am trying to do a search within elasticsearch using the regexp filters. Following is my query:
{
"from": 0,
"size": 10,
"_source":["CODE"],
"query": {
"bool": {
"must": [
{
"bool": {
"should": [
{
"regexp" : {
"CODE" : {
"value" : "[0]?[0]?[0]?[0]?3410086456[0-9]?",
"flags_value" : 0,
"boost" : 20.0
}
}
},
{
"regexp" : {
"CODE" : {
"value" : "[0]?[0]?[0]?[0]?83560900204[0-9]?",
"flags_value" : 0,
"boost" : 20.0
}
}
}
]
}
},
{
"terms": {
"CODETYPE": [
"TYPE1", "TYPE2", "TYPE3"
]
}
}
]
}
}
}
Below is the result of the query:
{
"took": 5,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 2,
"max_score": 20.091797,
"hits": [
{
"_index": "index1",
"_type": "type1",
"_id": "142242",
"_score": 20.091797,
"_source": {
"CODE": "003410086456"
}
},
{
"_index": "index1",
"_type": "type1",
"_id": "375897",
"_score": 20.091797,
"_source": {
"CODE": "083560900204"
}
}
]
}
}
What I need to get additionally in my output is the input term against which each result has matched. Something like this:
{
"took": 5,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 2,
"max_score": 20.091797,
"hits": [
{
"_index": "index1",
"_type": "type1",
"_id": "142242",
"_score": 20.091797,
"_source": {
"CODE": "003410086456",
"INPUT": "3410086456"
}
},
{
"_index": "index1",
"_type": "type1",
"_id": "375897",
"_score": 20.091797,
"_source": {
"CODE": "083560900204",
"INPUT": "83560900204"
}
}
]
}
}
Notice the additional INPUT field above. That way I can map what pattern has mapped to which result. Is there any possibility in elasticsearch I can do this? I am currently unable to find any way of achieving this.
Appreciate your help on this. Let me know if I need to furnish any more information.
you could use highlighting, though it won't in _source, it would create a separate field highlight which gives the field value.
{
"from": 0,
"size": 10,
"_source": [
"CODE"
],
"query": {
"bool": {
"must": [
{
"bool": {
"should": [
{
"regexp": {
"CODE": {
"value": "[0]?[0]?[0]?[0]?3410086456[0-9]?",
"flags_value": 0,
"boost": 20
}
}
},
{
"regexp": {
"CODE": {
"value": "[0]?[0]?[0]?[0]?83560900204[0-9]?",
"flags_value": 0,
"boost": 20
}
}
}
]
}
},
{
"terms": {
"CODETYPE": [
"TYPE1",
"TYPE2",
"TYPE3"
]
}
}
]
}
},
"highlight": {
"fields": {
"CODE": {}
}
}
}
Refer: https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-highlighting.html#search-request-highlighting
I cannot seem to aggregate my query results when using my custom query parser. I get a result set by these are not aggregated. When using a standard query parser like match everything turns out well.
What works:
GET pages/_search
{
"query": {
"match": {
"text": "binomial"
}
},
"aggs": {
"docs": {
"terms": {
"field": "rooturl"
}
}
}
}
returns a nice aggregated result:
{
"took": 3,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"failed": 0
},
"hits": {
"total": 10,
"max_score": 11.11176,
"hits": [
...
{
"_index": "pages",
"_type": "doc",
"_id": "AVcq6z6lzDazctHi91RE",
"_score": 3.3503218,
"_source": {
"rooturl": "document",
"type": "equation",
"url": "document:poly",
"text": "coefficient"
}
},
{
"_index": "pages",
"_type": "doc",
"_id": "AVcq6z6xzDazctHi91RF",
"_score": 3.3503218,
"_source": {
"rooturl": document",
"type": "equation",
"url": "document:poly",
"text": "dot"
}
}
...
]
},
"aggregations": {
"docs": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "document",
"doc_count": 10
}
]
}
}
}
But when using my custom query parser, The result is not aggregated.
Query:
GET pages/_search
{
"query": {
"my_custom_query_parser": {
"query": "binomial"
}
},
"aggs": {
"docs": {
"terms": {
"field": "rooturl"
}
}
}
}
Can anyone point me into the right direction?