3-level parent-child relationships: documents have a _parent field, but a has_child query returns no results - elasticsearch

Question

I have three types of documents: request, job, and driver. There's a parent/child relationship between request and job, and a parent/child relationship between job and driver. This query:
`GET batch-val-weekly-2015-02-v1/driver/_search
{
"query": {
"match_all": {}
},
"fields": ["_parent"]
}`
returns a list of documents. The parent field is present in each and gives the id of the appropriate job document. But this query:
'GET batch-val-weekly-2015-02-v1/job/_search
{
"query": {
"has_child": {
"type": "driver",
"query": {
"match_all": {}
}
}
}
}'
returns no hits. I also get no hits from a has_parent query.
EDIT: This was answered in Elasticsearch deeper level Parent-child relationship (grandchild). See my explanation below.

Answer 1

When I take out the property "driver" of the type "driver" it seems to work fine. If that was a mistake, then removing it seems to solve the problem. More concretely, first I create the index (notice the difference in the definition of "driver" from what you posted above):
DELETE /test_index
PUT /test_index
{
"mappings": {
"request": {
"_all": {
"enabled": false
},
"_timestamp": {
"enabled": true
}
},
"job": {
"_all": {
"enabled": false
},
"_timestamp": {
"enabled": true
},
"_parent": {
"type": "request"
}
},
"driver": {
"_all": {
"enabled": false
},
"_parent": {
"type": "job"
},
"properties": {
"rules": {
"type": "nested"
}
}
}
}
}
add three docs related to each other:
PUT /test_index/_bulk
{"index": {"_index": "test_index", "_type": "request", "_id": 1}}
{}
{"index": {"_index": "test_index", "_type": "job", "_id": 1, "_parent":1}}
{}
{"index": {"_index": "test_index", "_type": "driver", "_id": 1, "_parent":1}}
{}
first query works:
POST /test_index/driver/_search
{
"query": {
"match_all": {}
},
"fields": [
"_parent"
]
}
...
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "test_index",
"_type": "driver",
"_id": "1",
"_score": 1,
"fields": {
"_parent": "1"
}
}
]
}
}
second query works:
POST /test_index/job/_search
{
"query": {
"has_child": {
"type": "driver",
"query": {
"match_all": {}
}
}
}
}
...
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "test_index",
"_type": "job",
"_id": "1",
"_score": 1,
"_source": {}
}
]
}
}
Here is the code I used:
http://sense.qbox.io/gist/602af489fbfb6595c65cd27cff7a1926642ea205

Answer 2

It turns out that has_parent and has_child queries are problematic with more than two generations. That's because these queries rely on the parent and child being on the same shard. By default, documents are routed to shards using their ids (more on this here). When you establish a parent-child relationship, child documents are routed based on their parent IDs, to ensure that they end up on the same shard.
So if you have three generations -- parents, children, grandchildren -- then the children are routed using the parents' IDs, and the grandchildren are routed using the childrens' IDs. But the children weren't routed using their own IDs, they were routed using the parents' IDs; so the grandchildren don't end up on the right shards and has_parent/has_child queries can't find them.
The way to fix this is to route the grandchildren based on the parents' IDs. This is configurable, see the Elasticsearch routing documentation here. You can also set up custom routing via the Java API. That's what I did, and it fixed the problem.
Check out this stackoverflow answer for more: Elasticsearch deeper level Parent-child relationship (grandchild)