Elastcisearch copy two fields to the object for every document - elasticsearch

How to copy two fields in the the object for the same document.
I have an object in the elasticsearch
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "test_index",
"_type": "doc",
"_id": "AXWTrVr6LIkj1JVvPnDX",
"_score": 1,
"_source": {
"field1": 1,
"field2": 2
}
}
]
}
}
I want to copy field1 and field2 into the test_object for every document.
Expected result
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "test_index",
"_type": "doc",
"_id": "AXWTrVr6LIkj1JVvPnDX",
"_score": 1,
"_source": {
"field1": 1,
"field2": 2,
"test_object":{
"field1": 1,
"field2": 2
}
}
}
]
}
}
I am trying to do it via next script, but I don't understand what is wrong
POST test_index/doc/update
{
"query":{
"match":{
"field1":1
}
},
"script" : {
"inline": "ctx._source.test_field.field1 = ctx._source.field1 ctx._source.test_field.field2 = ctx._source.field2"
}
}

First you need to hit the _update_by_query endpoint
Then, since test_field doesn't exist in your document, you need to create it:
This should work for you:
POST test_index/_update_byquery
{
"query":{
"match":{
"field1":1
}
},
"script" : {
"inline": "ctx._source.test_field = ['field1': ctx._source.field1, 'field2': ctx._source.field2]"
}
}

Related

Finding numbers with decimal zeros in elasticsearch

I'm displaying numbers with decimal zeros like this: 25785 --> 25'785.00
I want to copy & paste this displayed number in the search field and find my actual number.
When I do it my query looks like this "query": "(25785.00 OR 25785.00*)", but the indexed number is 25785 and it doesn't get found.
Can I index this field differently so it'll also find the numbers with the decimal zeros?
Mapping:
"my-money" : {
"type" : "text",
"fields" : {
"raw" : {
"type" : "double"
}
}
},
You can use matchphrase query. Details can be found here
Mappings:
PUT /mstest
{
"mappings": {
"test": {
"properties": {
"money": {
"type": "text",
"fields": {
"raw": {
"type": "double"
}
}
}
}
}
}
}
Existing data:
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 2,
"max_score": 1,
"hits": [
{
"_index": "mstest",
"_type": "test",
"_id": "AXlhj0RUNamWTgl090_3",
"_score": 1,
"_source": {
"money": 257851111
}
},
{
"_index": "mstest",
"_type": "test",
"_id": "AXlhjR3f7ALnT2aUN_qN",
"_score": 1,
"_source": {
"money": 25785
}
}
]
}
}
Search query for number '25785':
GET mstest/test/_search
{
"query": {
"match_phrase": {
"money.raw": "25785.00"
}
}
}
Output:
{
"took": 3,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "mstest",
"_type": "test",
"_id": "AXlhjR3f7ALnT2aUN_qN",
"_score": 1,
"_source": {
"money": 25785
}
}
]
}
}
See if this unblocks you.

mysql field="value" in elasticsearch

I want to display only the items that contain the word itself when "google" searches
How can I only search for items that have only the word "google"?
Request body
(Request created in postman)
{
"query": {
"bool": {
"must": [
{
"match": {
"body": "google"
}
}
]
}
}
}
Response body
(Request created in postman)
{
"took": 0,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 3,
"relation": "eq"
},
"max_score": 0.6587735,
"hits": [
{
"_index": "s_t",
"_type": "_doc",
"_id": "3",
"_score": 0.6587735,
"_source": {
"body": "google"
}
},
{
"_index": "s_t",
"_type": "_doc",
"_id": "4",
"_score": 0.5155619,
"_source": {
"body": "google map"
}
},
{
"_index": "s_t",
"_type": "_doc",
"_id": "5",
"_score": 0.5155619,
"_source": {
"body": "google-map"
}
}
]
}
}
I need this output
(Request created in postman)
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 3,
"relation": "eq"
},
"max_score": 0.69381464,
"hits": [
{
"_index": "s_t",
"_type": "_doc",
"_id": "3",
"_score": 0.69381464,
"_source": {
"body": "google"
}
}
]
}
}
In mysql with this query I reach my goal.
Similar query in mysql:
select * from s_t where body='google'
well i assume you automap or use a text in your mappings.
specify .keyword in your query. Note this is case sensitive.
{
"query": {
"bool": {
"must": [
{
"match": {
"body.keyword": "google"
}
}
]
}
}
}
If you only want to query your body field using exact match. You need to reindex it using keyword. Take a look at: Exact match in elastic search query

Exact match Query in Elastic Search issue

I have a index in ElasticSearch with 4 datas
Here's the Data:
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 4,
"relation": "eq"
},
"max_score": 1,
"hits": [
{
"_index": "sample4",
"_type": "logs",
"_id": "UQBMOHABHstawU4w4_z3",
"_score": 1,
"_source": {
"date": "2020-02-12T07:28:48",
"target": {
"http://localhost/wordpress/index.php/2020/01/13/hello-world/": {
"clicks": {
"868 278": 12
}
}
}
}
},
{
"_index": "sample4",
"_type": "logs",
"_id": "UgBNOHABHstawU4wT_wn",
"_score": 1,
"_source": {
"date": "2020-02-12T07:29:15",
"target": {
"http://localhost/wordpress/": {
"clicks": {
"958 250": 5
}
}
}
}
},
{
"_index": "sample4",
"_type": "logs",
"_id": "UABMOHABHstawU4wC_y9",
"_score": 1,
"_source": {
"date": "2020-02-12T07:27:52",
"target": {
"http://localhost/wordpress/": {
"clicks": {
"880 257": 6
}
}
}
}
},
{
"_index": "sample4",
"_type": "logs",
"_id": "UwBOOHABHstawU4wFvxV",
"_score": 1,
"_source": {
"date": "2020-02-12T07:30:06",
"target": {
"http://localhost/wordpress/index.php/2020/01/13/hello-world/": {
"clicks": {
"389 60": 33
}
},
"http://localhost/wordpress/": {
"clicks": {
"657 235": 8
}
}
}
}
}
]
}
}
I want to match the target key in the index with the value http://localhost/wordpress/. If the given value exactly matches the value in target key in ES index, I would get 3 data. Inside the target key, it was like an object. So i don't know how make a match query.
Here's the query i tried:
{
"query": {
"wildcard": {
"target.http://localhost/wordpress/": {
"value": "*"
}
}
}
}
But it returns 0 results.
Output I got:
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 0,
"relation": "eq"
},
"max_score": null,
"hits": []
}
}
Required Output:
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 3,
"relation": "eq"
},
"max_score": 1,
"hits": [
{
"_index": "sample4",
"_type": "logs",
"_id": "UgBNOHABHstawU4wT_wn",
"_score": 1,
"_source": {
"date": "2020-02-12T07:29:15",
"target": {
"http://localhost/wordpress/": {
"clicks": {
"958 250": 5
}
}
}
}
},
{
"_index": "sample4",
"_type": "logs",
"_id": "UABMOHABHstawU4wC_y9",
"_score": 1,
"_source": {
"date": "2020-02-12T07:27:52",
"target": {
"http://localhost/wordpress/": {
"clicks": {
"880 257": 6
}
}
}
}
},
{
"_index": "sample4",
"_type": "logs",
"_id": "UwBOOHABHstawU4wFvxV",
"_score": 1,
"_source": {
"date": "2020-02-12T07:30:06",
"target": {
"http://localhost/wordpress/index.php/2020/01/13/hello-world/": {
"clicks": {
"389 60": 33
}
},
"http://localhost/wordpress/": {
"clicks": {
"657 235": 8
}
}
}
}
}
]
}
}
Help me to solve this problem.....
Since you're checking on a field name and not a value, you should try this query instead
{
"query": {
"exists": {
"field": "target.http://localhost/wordpress/"
}
}
}

What does total value shows inside the _search query result in elasticsearch?

When we call the elasticsearch, say as follows:
POST https:////_search with body:
{
"from": 0,
"size": 1,
"query": {
"bool": {
"must": [
{
"range": {
"createdAt": {
"gt": "2019-11-11T10:00:00"
}
}
}
]
}
},
"sort": [
{
"createdAt" : {
"order" : "desc"
}
}
]
}
I see that I get only 1 result as pagination is set to 1 but total inside hits in response shows 2. This is the response I get:
{
"took": 4,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": null,
"hits": [
{
"_index": “<index-name>”,
"_type": "_doc",
"_id": "5113c843-dff3-499f-a12e-44c7ac103bcf_0",
"_score": null,
"_source": {
"oId": "5113c843-dff3-499f-a12e-44c7ac103bcf",
"oItemId": 0,
"createdAt": "2019-11-13T11:00:00"
},
"sort": [
1573642800000
]
}
]
}
}
Doesn’t total doesn’t capture the pagination part? And it only cares about the query report? It should show the total count of items matching the query irrespective of the pagination set, right?
Yes, You are right that total doesn't capture the pagination part and just cares about the query report ie. whatever the total no of the document matches for a given query.
To be precise, it is as explained in official ES docs .
total (Object) Metadata about the number of returned documents.
Returned parameters include:
value: Total number of returned documents. relation: Indicates whether
the number of documents returned. Returned values are:
eq: Accurate gte: Lower bound, including returned documents
It means its the total no of returned documents, but as pagination is set to 1 in your example, inner hits have just 1 document.You can cross-check this understanding easily by creating a sample example as below:
Create a sample index with just 1 text field:
URL:- http://localhost:9200/{your-index-name}/ --> PUT method
{
"mappings": {
"properties": {
"name": {
"type": "text"
}
}
},
"settings": {
"index": {
"number_of_shards": "1",
"number_of_replicas": "1"
}
}
}
Once the above index is created index below 4 documents:
URL:- http://localhost:9200/{your-index-name}/_doc/{1,2,like..} --> POST method
{
"name": "foo 1"
}
{
"name": "foo bar"
}
{
"name": "foo"
}
{
"name": "foo 2"
}
Now when you hit below search query without pagination:
{
"query": {
"bool": {
"must": [
{
"match": {
"name": "foo"
}
}
]
}
}
}
It gives below response:
{
"took": 9,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 4, --> Note 4 here
"relation": "eq"
},
"max_score": 0.12199639,
"hits": [
{
"_index": "59638303",
"_type": "_doc",
"_id": "1",
"_score": 0.12199639,
"_source": {
"name": "foo"
}
},
{
"_index": "59638303",
"_type": "_doc",
"_id": "3",
"_score": 0.12199639,
"_source": {
"name": "foo"
}
},
{
"_index": "59638303",
"_type": "_doc",
"_id": "2",
"_score": 0.09271725,
"_source": {
"name": "foo bar"
}
},
{
"_index": "59638303",
"_type": "_doc",
"_id": "4",
"_score": 0.09271725,
"_source": {
"name": "foo 1"
}
}
]
}
}
But when you hit a search query with pagination:
{
"from": 0,
"size": 1,--> note size 1
"query": {
"bool": {
"must": [
{
"match": {
"name": "foo"
}
}
]
}
}
}
it gives below response
{
"took": 23,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 4, --> this is still 4
"relation": "eq"
},
"max_score": 0.12199639,
"hits": [
{
"_index": "59638303",
"_type": "_doc",
"_id": "1",
"_score": 0.12199639,
"_source": {
"name": "foo"
}
}
]
}
}
Now in the above query, you can change the size and check only inner-hits array gets change but the outer hits object which contains total always remains same as 4, this confirms your understanding is correct.

elasticsearch max aggregation return more than one result

Im running the following query:
POST myindex/_search
{
"aggs": {
"minSamp": {
"min": {
"field": "sample"
}
}
}
}
part of the result:
{
"took": 15,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 91,
"max_score": 1,
"hits": [
{
"_index": "myindex",
"_type": "myindex",
"_id": "HyYmY2oB06bGDsjT4C7Z",
"_score": 1,
"_source": {
"sample": 119267,
"age": 6,
"comp": 11
}
},
{
"_index": "myindex",
"_type": "myindex",
"_id": "HyYmY2oB06bGDsjT4C79",
"_score": 1,
"_source": {
"sample": 117100,
"age": 9,
"comp": 7
}
}
]
}
}
....
and I want to get only one response (what is the smallest "sample" value")
but I get lot of documents as response, full documents ,
1. what is wrong?
2. can I get one response for multiple indices? for example: if my query is for all indices that start with "my":
Thanks
POST my*/_search
In hits, it will be returning default 10 documents. You need to give size:0 in your query if you don't want to return documents i.e only aggregation is needed
"size":0,
"aggs": {
"minSamp": {
"min": {
"field": "sample"
}
}
}
link for reference.

Resources