when writing a search query with a script, I can access fields using "doc['myfield']"
curl -XPOST 'http://localhost:9200/index1/type1/_search' -d '
{
"query": {
"filtered": {
"query": {
"match_all": {}
},
"filter": {
"script": {
"script": "doc[\"myfield\"].value>0",
"params": {},
"lang":"python"
}
}
}
}
}'
how do I go about accessing the _id or _parent fields?
The "ctx" object does not seem to be available in a search query (while it is accessible in an update API request, why?).
Mind you, I am using the python language instead of mvel, but both of them pose the same question.
By default, both document id and parent id are indexed in uid format: type#id. Elasticsearch provides a few methods that can be used to extract type and id from uid string. Here is an example of using these methods in MVEL:
curl -XDELETE localhost:9200/test
curl -XPUT localhost:9200/test -d '{
"settings": {
"index.number_of_shards": 1,
"index.number_of_replicas": 0
},
"mappings": {
"doc": {
"properties": {
"name": {
"type": "string"
}
}
},
"child_doc": {
"_parent": {
"type": "doc"
},
"properties": {
"name": {
"type": "string"
}
}
}
}
}'
curl -XPUT "localhost:9200/test/doc/1" -d '{"name": "doc 1"}'
curl -XPUT "localhost:9200/test/child_doc/1-1?parent=1" -d '{"name": "child 1-1 of doc 1"}'
curl -XPOST "localhost:9200/test/_refresh"
echo
curl "localhost:9200/test/child_doc/_search?pretty=true" -d '{
"script_fields": {
"uid_in_script": {
"script": "doc[\"_uid\"].value"
},
"id_in_script": {
"script": "org.elasticsearch.index.mapper.Uid.idFromUid(doc[\"_uid\"].value)"
},
"parent_uid_in_script": {
"script": "doc[\"_parent\"].value"
},
"parent_id_in_script": {
"script": "org.elasticsearch.index.mapper.Uid.idFromUid(doc[\"_parent\"].value)"
},
"parent_type_in_script": {
"script": "org.elasticsearch.index.mapper.Uid.typeFromUid(doc[\"_parent\"].value)"
}
}
}'
echo
Related
I have a document that has the structure with an field object with a nested field internally. The nested field is responsible for storing all interactions that occurred in an internal communication.
It happens that I need to create a new field inside the nested field, with a new type that will now be used to store the old field with a new parser.
How can I copy the data from the old field to the new field inside the nested field?
My document:
curl -XPUT 'localhost:9200/problems?pretty' -H 'Content-Type: application/json' -d '
{
"settings": {
"number_of_shards": 1
},
"mappings": {
"problem": {
"properties": {
"problemid": {
"type": "long"
},
"subject": {
"type": "text",
"index": true
},
"usermessage": {
"type": "object",
"properties": {
"content": {
"type": "nested",
"properties": {
"messageid": {
"type": "long",
"index": true
},
"message": {
"type": "text",
"index": true
}
}
}
}
}
}
}
}
}'
My New Field:
curl -XPUT 'localhost:9200/problems/_mapping/problem?pretty' -H 'Content-Type: application/json' -d '
{
"properties": {
"usermessage": {
"type": "object",
"properties": {
"content": {
"type": "nested",
"properties": {
"message_accents" : {
"type" : "text",
"analyzer" : "ignoreaccents"
}
}
}
}
}
}
}
'
Data Example:
{
"problemid": 1,
"subject": "Test",
"usermessage": [
{
"messageid": 1
"message": "Hello"
},
{
"messageid": 2
"message": "Its me"
},
]
}'
My script to copy fields:
curl -XPOST 'localhost:9200/problems/_update_by_query' -H 'Content-Type: application/json' -d '
{
"query": {
"match_all": {
}
},
"script": "ctx._source.usermessage.content.message_accents = ctx._source.usermessage.content.message"
}'
I tried the code below but it didn't work, it returns an error.
curl -XPOST 'localhost:9200/problems/_update_by_query' -H 'Content-Type: application/json' -d '
{
"query": {
"match_all": {
}
},
"script": "ctx._source.usermessage.content.each { elm -> elm.message_accents = elm.message }"
}
'
Error:
"script":"ctx._source.usermessage.content.each { elm -> elm.message_accents = elm.message }","lang":"painless","caused_by":{"type":"illegal_argument_exception","reason":"unexpected token ['{'] was expecting one of [{, ';'}]."}},"status":500}%
I tried a few different ways of doing a simple get request, filtering on two different attributes, example:
"query": {
"filtered": {
"filter": {
"bool": {
"must": [
{
"term": {
"email": "erik.landvall#example.com"
}
},
{
"term": {
"password": "bb3810356e9b60cf6..."
}
}
]
}
},
"query": {
"match_all": []
}
}
}
The problem is that I get nothing back in return. As I understand it, this is because ElasticSearch analyzes the email field, making the query fail. So if I however would use the term erik.landvall instead of the complete email address, it will match the document - which confirms that's what's going on.
I can define the attribute as type:string and index:not_analyzed when I create the index. But what if I wanna be able to search on the email attribute in a different context? So there should, to my mind, be a way to specify that I wanna filter on the actual value of the attribute in a query. I can however not find how such a query would look.
Is it possible to force Elasticsearch to use "not_analyze" when querying? If so, then how?
You can use scripting for this purpose. You would have to directly access the JSON you have stored with _source. Try following query
{
"query": {
"bool": {
"filter": {
"script": {
"script": {
"inline" : "_source.email==param1 && _source.password==param2",
"params" : {
"param1" : "erik.landvall#example.com",
"param2" : "bb3810356e9b60cf6"
}
}
}
}
}
}
}
You would need to enable dynamic scripting. Add script.inline: on to your yml file and restart the node.
If this kind of query is fairly regular then It would be much better to reindex the data as others have suggested in the comments.
Its not possible to turn on/off analyzed or not, the way to do it to "transform" your field to analysis you need by using fields.
curl -XPUT 'localhost:9200/my_index?pretty' -d'
{
"mappings": {
"my_type": {
"properties": {
"city": {
"type": "string",
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
}
}'
curl -XPUT 'localhost:9200/my_index/my_type/1?pretty' -d'
{
"city": "New York"
}'
curl -XPUT 'localhost:9200/my_index/my_type/2?pretty' -d'
{
"city": "York"
}'
curl -XGET 'localhost:9200/my_index/_search?pretty' -d'
{
"query": {
"match": {
"city": "york"
}
},
"sort": {
"city.raw": "asc"
},
"aggs": {
"Cities": {
"terms": {
"field": "city.raw"
}
}
}
}'
ElasticSearch 1.2.1
I am trying to query documents using weighted tags.
curl -X PUT 'http://localhost:9200/test'
curl -X PUT 'http://localhost:9200/test/thing/_mapping' - '{
"thing": {
"properties": {
"tags": {
"type": "nested",
"properties": {
"name": { "type": "string" },
"weight": { "type": "integer" }
}
}
}
}}'
Adding a document:
curl -X PUT 'http://localhost:9200/test/thing/1', -d '{
"tags": [
{ "name": "a", "weight": 2 }
]
}'
Now I am searching for documents having a tag a and boost the score based on the weight.
Note: to run these examples you have to enable dynamic scripting in ElasticSearch: add script.disable_dynamic: false to elasticsearch.yml
curl -X GET 'http://localhost:9200/test/thing/_search?pretty' -d '{
"query": {
"function_score": {
"boost_mode": "replace",
"query": {
"match_all": {}
},
"functions": [
{
"filter": {
"nested": {
"path": "tags",
"filter": {
"term": {
"tags.name": "a"
}
}
}
},
"script_score": {
"script": "doc.weight.value"
}
}
]
}
}
}'
The document is found, as expected, however the score is 0. It seems as if the property doc.weight was empty.
Let's test this by replacing the script with doc.weight.empty ? 50 : 100. The hit now has a score of 50 indicating that the field doc.weight is empty. It is found though, because using a non-existant field name (e.g. doc.foobar) gives an error.
Background: The match_all part would be replaced by a real query. I want to use the tags to boost results matching the tags before the ones not matching the tags.
What am I missing?
I have a bunch of documents with an array field like this:
{ "feed_uids": ["math.CO", "cs.IT"] }
I would like to find all documents that contain some subset of these values i.e. treat them as tags. Documentation leads me to believe a terms filter should work:
{ "query": { "filtered": { "filter": { "terms": { "feed_uids": [ "cs.IT" ] } } } } }
However, the query matches nothing. What am I doing wrong?
The terms-filter works as you expect. I guess your problem here is that you have a mapping where feed_uids is using the standard analyzer.
This is quite a common problem which is described in a bit more depth here: Troubleshooting Elasticsearch searches, for Beginners
Here is a runnable example showcasing how it works if you specify "index": "not_analyzed" for the field: https://www.found.no/play/gist/bc957d515597ec8262ab
#!/bin/bash
export ELASTICSEARCH_ENDPOINT="http://localhost:9200"
# Create indexes
curl -XPUT "$ELASTICSEARCH_ENDPOINT/play" -d '{
"mappings": {
"type": {
"properties": {
"feed_uids": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}'
# Index documents
curl -XPOST "$ELASTICSEARCH_ENDPOINT/_bulk?refresh=true" -d '
{"index":{"_index":"play","_type":"type"}}
{"feed_uids":["math.CO","cs.IT"]}
{"index":{"_index":"play","_type":"type"}}
{"feed_uids":["cs.IT"]}
'
# Do searches
curl -XPOST "$ELASTICSEARCH_ENDPOINT/_search?pretty" -d '
{
"query": {
"filtered": {
"filter": {
"terms": {
"feed_uids": [
"cs.IT"
]
}
}
}
}
}
'
I have an index with 2 types of document: District and Street. I would like to boost rank of District documents.
Is there any way to do that without the script option?
You can use a custom_filters_score to boost documents matching a filter, and your filter can be a type-filter.
Here's a runnable example you can play with: https://www.found.no/play/gist/8744808
#!/bin/bash
export ELASTICSEARCH_ENDPOINT="http://localhost:9200"
# Index documents
curl -XPOST "$ELASTICSEARCH_ENDPOINT/_bulk?refresh=true" -d '
{"index":{"_index":"play","_type":"district"}}
{"title":"there"}
{"index":{"_index":"play","_type":"street"}}
{"title":"there"}
'
# Do searches
curl -XPOST "$ELASTICSEARCH_ENDPOINT/_search?pretty" -d '
{
"query": {
"custom_filters_score": {
"query": {
"match": {
"title": {
"query": "there"
}
}
},
"filters": [
{
"boost": 2,
"filter": {
"type": {
"value": "district"
}
}
}
]
}
}
}
'