ElasticSearch _parent query - elasticsearch

Elastic documentation states that one can use the _parent field in a query (see https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-parent-field.html).
However, I have been unable to get it to work. Here's the simple test:
PUT /company
{
"mappings": {
"branch": {},
"employee": {
"_parent": {
"type": "branch"
}
}
}
}
POST /company/branch/_bulk
{ "index": { "_id": "london" }}
{ "name": "London Westminster", "city": "London", "country": "UK" }
{ "index": { "_id": "liverpool" }}
{ "name": "Liverpool Central", "city": "Liverpool", "country": "UK" }
{ "index": { "_id": "paris" }}
{ "name": "Champs Élysées", "city": "Paris", "country": "France" }
PUT /company/employee/1?parent=london
{
"name": "Alice Smith",
"dob": "1970-10-24",
"hobby": "hiking"
}
Verifying that the employees have a _parent field:
GET /company/employee/_search
{
"query": {
"match_all": {}
}
}
returns
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "company",
"_type": "employee",
"_id": "1",
"_score": 1,
"_routing": "london",
"_parent": "london",
"_source": {
"name": "Alice Smith",
"dob": "1970-10-24",
"hobby": "hiking"
}
}
]
}
}
But the following:
GET /company/employee/_search
{
"query": {
"term": {
"_parent":"london"
}
}
}
returns:
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 0,
"max_score": null,
"hits": []
}
}
Using "has_parent" works, but why doesn't using _parent work, as stated in the docs.
Here's the query using has_parent that works:
GET /company/employee/_search
{
"query": {
"has_parent": {
"parent_type":"branch",
"query":{
"match_all": {}
}
}
}
}
What am I missing? Using ElasticSearch 5.0.2.

It's a documentation bug. According to the breaking changes in 5.0, the _parent field is no longer indexed and hence it is not possible to run a term query on that field. You either need to use the has_parent query or the new parent_id query to find that child document:
POST /company/employee/_search
{
"query": {
"parent_id": {
"type": "employee",
"id": "london"
}
}
}
For those who want to follow, I've filed an issue to report this and it got fixed. The updated documentation will soon be available.

Related

Elastic Search query match on keyword 'OR'

I'm using ElasticSearch 7.0
Given the mapping:
{
"searchquestion": {
"mappings": {
"properties": {
"server": {
"properties": {
"hostname": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword",
"ignore_above": 256
}
}
}
}
}
}
}
}
}
I have put the following documents into this index:
{
"server": {
"hostname": "server1-windows.loc2.uk"
}
}
{
"server": {
"hostname": "server1-windows.loc2.uk"
}
}
{
"server": {
"hostname": "server1-linux.loc1.uk"
}
}
I would like to query the exact text of the hostname. Luckily, this can be done because there is an additional keyword type field on this field.
Successful query :
{
"query": {
"bool": {
"must": [
{
"match": {
"server.hostname.keyword": {
"query": "server1-windows.loc2.uk"
}
}
}
]
}
}
}
However, I would like to extend this query string, to include another hostname to search for. In my results, I expect to have both documents returned.
My attempt:
{
"query": {
"bool": {
"must": [
{
"match": {
"server.hostname.keyword": {
"query": "server1-windows.loc2.uk server1-linux.loc1.uk",
"operator": "or"
}
}
}
]
}
}
}
This returns no hits, I suspect because the default analyser is splitting this query up into sections, but I'm actually searching the keyword field which is a full string. I cannot add analyzer: keyword to this query search, as server1-windows.loc2.uk server1-linux.loc1.uk as an exact string won't match anything either.
How can I search for both these strings, as their complete selves?
i.e. "query": ["server1-windows.loc2.uk", "server1-linux.loc1.uk"]
I would also like to use wildcards to match any loc. I would expect
"query": ["server1-windows.*.uk"] to match both windows servers, but I get no hits.
What am I missing?
you can use Query_String to get your desired result
Case 1:
Query:
GET server/_search
{
"query": {
"query_string": {
"query": "(server1-windows.loc2.uk) OR (server1-linux.loc1.uk)",
"default_field": "server.hostname.keyword"
}
}
}
Output:
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 3,
"relation": "eq"
},
"max_score": 0.9808291,
"hits": [
{
"_index": "server",
"_id": "3",
"_score": 0.9808291,
"_source": {
"server": {
"hostname": "server1-linux.loc1.uk"
}
}
},
{
"_index": "server",
"_id": "1",
"_score": 0.4700036,
"_source": {
"server": {
"hostname": "server1-windows.loc2.uk"
}
}
},
{
"_index": "server",
"_id": "2",
"_score": 0.4700036,
"_source": {
"server": {
"hostname": "server1-windows.loc2.uk"
}
}
}
]
}
}
Case 2: with wildcard(*)
Query:
GET server/_search
{
"query": {
"query_string": {
"query": "server1-windows.*.uk",
"default_field": "server.hostname.keyword"
}
}
}
Output:
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": 1,
"hits": [
{
"_index": "server",
"_id": "1",
"_score": 1,
"_source": {
"server": {
"hostname": "server1-windows.loc2.uk"
}
}
},
{
"_index": "server",
"_id": "2",
"_score": 1,
"_source": {
"server": {
"hostname": "server1-windows.loc2.uk"
}
}
}
]
}
}

Is there a way to exclude a particular term from elastic search highlights?

I'm trying out a query in elastic search(version 6.0) where I have a base query and on top of that, I have filters applied to narrow down the search. It is as follows:
GET target_index/_search
{
"from": {start},
"size": {offset},
"_source": [
"id",
"name",
"email",
"company",
"created_at",
],
"query": {
"bool": {
"filter": {
"bool": {
"filter": [
{ "terms":{"name.raw": ["test","test2"] }},
{ "terms":{"email.raw": ["test#test.com","test2#test.com"] }}
]
}
},
"must": {
"query_string": {
"query": "test",
}
}
}
},
"highlight": {
"fields": {
"*":{
"type":"plain"
}
}
}
}
Current result -
{
"took": 5,
"timed_out": false,
"_shards": {
"total": 3,
"successful": 3,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1.90374,
"hits": [
{
"_index": "index_name",
"_id": "my_id",
"_score": 1.90374,
"_source": {
"id": 2,
"name": "test",
"email": "test#test.com",
"company": "test company"
},
"highlight": {
"name.raw": [
"<em>test</em>"
],
"name": [
"<em>test</em>"
],
"company": [
"<em>test</em> company"
]
}
}
]
}
}
Desired result -
{
"took": 5,
"timed_out": false,
"_shards": {
"total": 3,
"successful": 3,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1.90374,
"hits": [
{
"_index": "index_name",
"_id": "my_id",
"_score": 1.90374,
"_source": {
"id": 2,
"name": "test",
"email": "test#test.com",
"company": "test company"
},
"highlight": {
"company": [
"<em>test</em> company"
]
}
}
]
}
}
Here, in the highlights in the desired result, I don't want the data for "name" and "name.raw". This field should not be searched only for this particular query , so I cannot disable the field entirely from searching.
I have a lot of terms and cannot specify every term to include in the query. Is there a way to exclude only a few fields from query search?
related ES doc -
https://www.elastic.co/guide/en/elasticsearch/reference/6.0/index.html
Instead of excluding certain fields, you could include only those that you need:
{
"query": {
...
},
"highlight": {
"fields": {
"company":{ <---
"type":"plain"
}
}
}
}

Query Vs Filter in Elastic Search

I am trying to index a document which has three fields first_name, last_name, occupation of type "keyword" and has values XYZ, ABC, DEF respectively.
I have written query using filter for an exact match with AND condition as follows,
"query": {
"bool": {
"filter": [
{"term": {"first_name": "XYZ"}},
{"term": {"last_name": "ABC"}}
]
}
}
This has to return one document, but returns nothing.
I have another query for the same operation,
"query": {
"bool": {
"must": [
{"match": {"first_name": "XYZ"}},
{"match": {"last_name": "ABC"}}
]
}
}
This returns one document.
According to Elasticsearch documentation, I understand that the difference between query and filter is that filter does not score the result. I am not sure why the first query does not return any result. Is my understanding correct?
As documentation states there is no difference between query and filter except scoring. Of course this applies to the situation when both query and filters uses the same query type. Here you are using two different types - term and match. term is designed for exact comparison while match is analyzed and used as full-text search.
Take a look at the example below.
Your mapping:
PUT /index_53053054
{
"mappings": {
"_doc": {
"properties": {
"first_name": {
"type": "text"
},
"last_name": {
"type": "text"
},
"occupation": {
"type": "keyword"
}
}
}
}
}
Your document:
PUT index_53053054/_doc/1
{
"first_name": "XYZ",
"last_name": "ABC",
"occupation": "DEF"
}
filter query:
GET index_53053054/_search
{
"query": {
"bool": {
"filter": [
{
"match": {
"first_name": "XYZ"
}
},
{
"match": {
"last_name": "ABC"
}
},
{
"term": {
"occupation": "DEF"
}
}
]
}
}
}
and result:
{
"took": 7,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 0,
"hits": [
{
"_index": "index_53053054",
"_type": "_doc",
"_id": "1",
"_score": 0,
"_source": {
"first_name": "XYZ",
"last_name": "ABC",
"occupation": "DEF"
}
}
]
}
}
Similar must query:
GET index_53053054/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"first_name": "XYZ"
}
},
{
"match": {
"last_name": "ABC"
}
},
{
"term": {
"occupation": "DEF"
}
}
]
}
}
}
and response:
{
"took": 5,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 0.8630463,
"hits": [
{
"_index": "index_53053054",
"_type": "_doc",
"_id": "1",
"_score": 0.8630463,
"_source": {
"first_name": "XYZ",
"last_name": "ABC",
"occupation": "DEF"
}
}
]
}
}
As you can see hits are almost the same. The only difference is that in filter score is not calculated while in must query is.
Read more: https://www.elastic.co/guide/en/elasticsearch/reference/6.4/query-filter-context.html

What is the query required for fetching full-text with delimiter in elasticsearch

Assuming I have a document like this in elasticSearch :
{
"videoName": "taylor.mp4",
"type": "long"
}
I tried full-text search using the DSL query:
{
"query": {
"match":{
"videoName": "taylor"
}
}
}
I need to get the above document, but I don't get it .If I specify taylor.mp4, it returns the document.
So, I would like to know, how to make full-text search with delimiters.
Edit after KARTHEEK answer:
The regexp fetches the taylor.mp4 document. Take the situation, where the document in video index are:
{
"videoName": "Akon - smack that.mp4",
"type": "long"
}
So, the query for retrieving this document can be ,
{
"query": {
"match":{
"videoName": "smack that"
}
}
}
In this case, the document will be retrieved, since we use smack in the query string. match does the full-text search and gets us the document. But, say I only know the that keyword and the match, doesn't get the document. I need to use regexp for that.
{
"query": {
"regexp":{
"videoName": "smack.* that.*"
}
}
}
On the Other hand, if i take up regexp and make all my query strings to smack.* that.*, this will also not retrieve any documents. And, we dont know which word will have its suffix .mp4. So, my question is we need to do the full-text search with match, and it should also detect the delimiters. Is there any other way ?
Edit after Richa asked the mapping of index
for http://localhost:9200/example/videos/_mapping
{
"example": {
"mappings": {
"videos": {
"properties": {
"query": {
"properties": {
"match": {
"properties": {
"videoName": {
"type": "string"
}
}
}
}
},
"type": {
"type": "string"
},
"videoName": {
"type": "string"
}
}
}
}
}
}
Depending upon above query you mentioned right we can use regular expression in order get the result.Please find attached result for your perusal and let me know if there are anything else you want.
curl -XGET "http://localhost:9200/test/sample/_search" -d'
{
"query": {
"regexp":{
"videoName": "taylor.*"
}
}
}'
Result:
{
"took": 22,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "test",
"_type": "sample",
"_id": "1",
"_score": 1,
"_source": {
"videoName": "taylor.mp4",
"type": "long"
}
}
]
}
}
Please use this mapping
PUT /test_index
{
"settings": {
"number_of_shards": 1
},
"mappings": {
"doc": {
"properties": {
"videoName": {
"type": "string",
"term_vector": "yes"
}
}
}
}
}
After that you need to index a document that you mentioned earlier:
PUT test_index/doc/1
{
"videoName": "Akon - smack that.mp4",
"type": "long"
}
Output:
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 0.15342641,
"hits": [
{
"_index": "test_index",
"_type": "doc",
"_id": "1",
"_score": 0.15342641,
"_source": {
"videoName": "Akon - smack that.mp4",
"type": "long"
}
}
]
}
}
Query to get results:
GET /test_index/doc/1/_termvector?fields=videoName
Results:
{
"_index": "test_index",
"_type": "doc",
"_id": "1",
"_version": 1,
"found": true,
"took": 1,
"term_vectors": {
"videoName": {
"field_statistics": {
"sum_doc_freq": 3,
"doc_count": 1,
"sum_ttf": 3
},
"terms": {
"akon": {
"term_freq": 1
},
"smack": {
"term_freq": 1
},
"that.mp4": {
"term_freq": 1
}
}
}
}
}
By using this we will search based on "smack"
POST /test_index/_search
{
"query": {
"match": {
"_all": "smack"
}
}
}
Result:
{
"took": 1,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 0.15342641,
"hits": [
{
"_index": "test_index",
"_type": "doc",
"_id": "1",
"_score": 0.15342641,
"_source": {
"videoName": "Akon - smack that.mp4",
"type": "long"
}
}
]
}
}

Filtering nested objects in elasticsearch 2.2 not working

I've read some articles and documentation about queries/filters in nested objects but I can not make this sample work. Hopefully you can help me to check what is wrong. Bellow is the index and mapping settings I am using:
# Create Index
PUT agency
# Mapping
PUT agency/site/_mapping
{
"site": {
"properties": {
"name":{
"type":"string"
},
"phones": {
"type": "nested",
"properties":{
"is_confidential": { "type": "string" },
"number": { "type": "string" },
"description": {"type" : "string"}
}
}
}
}
}
# Indexing one document
PUT agency/site/1
{
"site":{
"name":"Burger Queen",
"phones":[
{
"is_confidential":"true",
"number":"10000000000",
"description":"Manager Phone"
},
{
"is_confidential":"false",
"number":"10000000001",
"description":"Public Line"
},
{
"is_confidential":"false",
"number":"10000000002",
"description":"Public Line 2"
},
{
"is_confidential":"false",
"number":"10000000003",
"description":"Complains Phone"
}
]
}
}
# Query the nested document (https://www.elastic.co/guide/en/elasticsearch/guide/current/nested-query.html)
GET /agency/site/_search
{
"query": {
"bool": {
"must": [
{ "match": { "site.name": "Burger" }},
{
"nested": {
"path": "phones",
"query": {
"bool": {
"must": [
{ "match": { "phones.is_confidential": "false" }}
]
}}}}
]
}}}
# Results
{
"took": 4,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 0,
"max_score": null,
"hits": []
}
}
Why I can not get any results ?
What I am trying to do is filter the parent documents by some term, for example: name = Burger and also filter nested documents to get only those phones where is_confidential = false.
Sample results without any filter applied in nested documents:
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "agency",
"_type": "site",
"_id": "1",
"_score": 1,
"_source": {
"site": {
"name": "Burger Queen",
"phones": [
{
"is_confidential": "true",
"number": "10000000000",
"description": "Manager Phone"
},
{
"is_confidential": "false",
"number": "10000000001",
"description": "Public Line"
},
{
"is_confidential": "false",
"number": "10000000002",
"description": "Public Line 2"
},
{
"is_confidential": "false",
"number": "10000000003",
"description": "Complains Phone"
}
]
}
}
}
]
}
}
If the site contains several nested objects in the phones array, only those phones that are not confidential should be returned by elasticsearch.
Sample result when is_confidential = false filter is applied:
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "agency",
"_type": "site",
"_id": "1",
"_score": 1,
"_source": {
"site": {
"name": "Burger Queen",
"phones": [
{
"is_confidential": "false",
"number": "10000000001",
"description": "Public Line"
},
{
"is_confidential": "false",
"number": "10000000002",
"description": "Public Line 2"
},
{
"is_confidential": "false",
"number": "10000000003",
"description": "Complains Phone"
}
]
}
}
}
]
}
}
Sample result when is_confidential = true:
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 1,
"hits": [
{
"_index": "agency",
"_type": "site",
"_id": "1",
"_score": 1,
"_source": {
"site": {
"name": "Burger Queen",
"phones": [
{
"is_confidential": "true",
"number": "10000000000",
"description": "Manager Phone"
}
]
}
}
}
]
}
}
Is it possible to get those sample results shown above with elasticsearch nested filter (queries)? If is possible could you please show me sample?
You need to update your PUT clause
Indexing one document
PUT agency/site/1
{
"site":{ // <-- need to remove this as it will alter the mapping definition
"name":"Burger Queen",
"phones":[
{
"is_confidential":"true",
"number":"10000000000",
"description":"Manager Phone"
},
{
"is_confidential":"false",
"number":"10000000001",
"description":"Public Line"
},
{
"is_confidential":"false",
"number":"10000000002",
"description":"Public Line 2"
},
{
"is_confidential":"false",
"number":"10000000003",
"description":"Complains Phone"
}
]
}
}
GET agency/site/_search
{
"query": {
"bool": {
"must": [
{
"match": {
"name": "burger"
}
},
{
"nested": {
"path": "phones",
"query": {
"term": {
"phones.is_confidential": {
"value": "true"
}
}
},
"inner_hits":{}
}
}
]
}
}
}
The matching nested-documents will be present in the inner hits response.
Sample Response :
{
"took": 2,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 2.2019112,
"hits": [
{
"_index": "agency",
"_type": "site",
"_id": "1",
"_score": 2.2019112,
"_source": {
"name": "Burger Queen",
"phones": [
{
"is_confidential": "true",
"number": "10000000000",
"description": "Manager Phone"
},
{
"is_confidential": "false",
"number": "10000000001",
"description": "Public Line"
},
{
"is_confidential": "false",
"number": "10000000002",
"description": "Public Line 2"
},
{
"is_confidential": "false",
"number": "10000000003",
"description": "Complains Phone"
}
]
},
"inner_hits": {
"phones": {
"hits": {
"total": 1,
"max_score": 1.9162908,
"hits": [
{
"_index": "agency",
"_type": "site",
"_id": "1",
"_nested": {
"field": "phones",
"offset": 0
},
"_score": 1.9162908,
"_source": {
"is_confidential": "true",
"number": "10000000000",
"description": "Manager Phone"
}
}
]
}
}
}
}
]
}
}

Resources