I have a data in elasticsearch which has a time field as Created. Below is the data:
{
"_index": "machine",
"_type": "health",
"_id": "30",
"_score": 1,
"_source": {
"Data": {
"DataId": "46",
"Created": "2018-06-11T07:31:33.739575"
},
"Datacount": 2,
"hostname": "username",
"health": "running"
}
}
As in the above data, I am using Data.Created as my time field in the elasticsearch. Now I want to query the data for which I open the Dev Tools and enter the below command:
GET machine/health/_search?
This gives me all the data belonging to index as machine and type as health. How can I sort this data to descending order on the basis of Data.Created so that the latest data should come first. Also with this, how can we only get data between two time range.?
Thanks
Simply add the sort query string parameter
GET machine/health/_search?sort=Data.Created:desc
In order to add a range you can do it like this:
GET machine/health/_search?sort=Data.Created:desc&q=Data.Created:[2018-06-10T00:00:00 TO 2018-06-11T00:00:00]
Related
I search for key word machine4 in my ES . My python client is simply:
result = es.search('machine4', index='machines')
Result look like this
[
{
"_score": 0.13424811,
"_type": "person",
"_id": "2",
"_source": {
"date": "**20180601**",
"deleted": [],
"changed": [
"machine1",
"machine2",
"machine3"
],
"**added**": [
"**machine4**",
"machine5"
]
},
"_index": "contacts"
},
{
"_score": 0.13424811,
"_type": "person",
"_id": "3",
"_source": {
"date": "**20180701**",
"deleted": [
"machine2"
],
"**changed**": [
"machine1",
"**machine4**",
"machine3"
],
"added": [
"machine7"
]
},
"_index": "contacts"
}
]
So we can easily see:
In date 20180601 , machine4 belonged to added.
In date 20180701 , machine4 belonged to changed.
I can write another function to analyze the result. Basically loop through every key,value of each items and check if the searched keyword belong, like this:
for result in search_results['hits']['hits']:
source_result = result['_source']
for key,value in source_result.items():
if 'machine4' in value:
print key
However, I wonder if ES having API to detect which key/mapping/field that the searched keywords belonged to ? In this case is added of the 1st result, and changed in 2nd result
Thank you so much
Alex
The simple answer seems to be that no, Elasticsearch doesn't have a way to do this out of the box, because Lucene doesn't have it, as per this thread
Elasticsearch has the concept of highlights, however. These could be useful, but they do require you to have some idea about which fields the match may be in.
The ES Python search documentation suggests there's no way to do that as a parameter to search, but you could create a custom query and pass it on as the q argument. It would look something like:
q = {"query" : {"match": { "content": "'machine4'" }}, "highlight" : {"fields" : {"added" : {}, "updated": {}}}}
result = es.search(index='machines', q=q)
Hope this is helpful!
On searching with routing it giving me data of different routing keys also.
Please help me out.
I have set routing in v2.0 and i queried query with routing key,below is example:
GET myindex/mytype/_search?routing=5
{
"query": {
"match_all": {}
}
}
I have search data with routing key=5 but output i get was:
hits": [
{
"_index": "goqii",
"_type": "nazar",
"_id": "2047",
"_score": 1,
"_routing": "10",
"_source": {
"userId": "111239",
"activityId": "765982",
"activityUserId": "111239",
"activityType": "water",
"commentText": "kinu juice",
"status": "delievered",
"createdTime": "2016-01-13 13:28:54"
}
},
{
"_index": "goqii",
"_type": "nazar",
"_id": "2046",
"_score": 1,
"_routing": "5",
"_source": {
"userId": "110554",
"activityId": "251449",
"activityUserId": "110554",
"activityType": "activity",
"commentText": "did home cycling yesterday for 20mins",
"status": "delievered",
"createdTime": "2016-01-13 12:04:31"
}
}
It is giving me routing key:5 & routing key:10 boths data.please help me out if i am doing something wrong.
Routing doesnt gaurantee that all the items on a shard will have the same routing key, it will make sure that all the docs that share a routing key are all on the same shard.
Routing is the process of determining which shard that document will reside in.
How many shards do you have in your index? It defaults to 5.
The scenario seems that you have the default number of shards in your index.
Routing scheme hashes the ID of a document and uses that to find a
shard. Routing ensures that documents of a particular routing value all go to the same shard…but that doesn’t mean that other documents aren’t routed to the shard too.
In your case both userIds "110554" and "111239" may be getting assigned to the same shard, hence the behavior.
I'm pretty new to ElasticSearch. I'm using v2.0.0. I would like to know how to focus a search query on one portion of the document in order to answer the question "Get me (a page of 50) People who are a member of Group "Developers".
The document structure of a single person might look like something like this:
{
"_index": "people",
"_type": "employee",
"_id": "8725",
"_source": {
"id": 43470,
"firstName": "John",
"lastName": "Smith",
"groups": [
{
"id": 345,
"name": "Developers"
},
{
"id": 75432,
"name": "Scrummasters"
},
{
"id": 5789,
"name": "UX"
}
]
}
}
So what I want to do is look at the name of each group of each person to see if it matches what I'm looking for and if so, select the whole person. The position of what I'm looking for is obviously not static, and I can't do something simpler like
q=roles:developer
Indeed, I can select employees that are in group Developers by specifying the query string as "q=groups.name:Developers". Indeed, I could also use a wildcard to get anybody in group Admin or Administrators or Admins using "q=groups.name:Admin*"
Here is my search object:
{
"_index": "search",
"_type": "product",
"_id": "2",
"_version": 1,
"found": true,
"_source": {
"datePublished": "2014-01-01T00:00:00-08:00",
"published": true,
"name": "Winter Coat",
"description": "A nice warm winter coat. If you get cold with this on, you've probably done something wrong.",
"brand": {
"name": "Burton"
}
}
}
and here is the query i'm trying to perform:
$query = new Query\QueryString('*' . 'winter' .'*');
$boolFilter = new Bool();
$brandFilter = new Term(['brand.name' => 'Burton']);
$boolFilter->addMust($brandFilter);
$filtered = new Filtered($baseQuery, $boolFilter);
return Query::create($filtered);
but i'm getting nothing back. What am i doing wrong?
After some trial and error i found that terms is a somewhat unreliable way to do exact matching because of the tokenizer. For any field you want to exact match you need to make sure it is not analyzed. Alternatively, you can just use integer values (such and an object id) for exact matching. I ended up using ids in the cases of exact matching.
I have json data as follows
{
"_index": "project",
"_type": "ts_order_snapshot",
"_id": "1",
"_version": 1,
"exists": true,
"_source": {
"order_id": "100000001",
"order_data": "Order Data:{\"entity_id\":\"1\",\"state\":\"canceled\"}"
}
}
Now I would like to get the list of all the order ids having state as canceled. Any suggestions?
order_data is a serialized, json-encoded string, not a structured json map. If you promoted it to actual json, you could construct a query like ?q=order_data.state:canceled&fields=order_id.