I'm trying to write query with custom script in elasticsearch:
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-script-filter.html#query-dsl-script-filter
https://www.elastic.co/guide/en/elasticsearch/reference/master/modules-scripting.html.
This is useful when you need to compare two document fields.
Everything worked fine, until I decide to use special document field (ex: _id, _uid, etc). The query always returns empty results and there is no errors if I use it like this: doc['_id'].value.
So how to use, for example, "_id" field of a document in a custom script?
The _id is indexed in the uid field, using this format: type#id.
So, your script should look like this (for a type called my_type and an ID of 1):
{
"query": {
"filtered": {
"filter": {
"script" : {
"script" : "doc['_uid'].value == 'my_type#1'"
}
}
}
}
}
A more elaborate solution, to take out the id ES-way is like this:
{
"query": {
"filtered": {
"filter": {
"script": {
"script": "org.elasticsearch.index.mapper.Uid.splitUidIntoTypeAndId(new org.apache.lucene.util.BytesRef(doc['_uid'].value))[1].utf8ToString() == '1'"
}
}
}
}
}
where org.elasticsearch.index.mapper.Uid.splitUidIntoTypeAndId(new org.apache.lucene.util.BytesRef(doc['_uid'].value))[1] is the id and org.elasticsearch.index.mapper.Uid.splitUidIntoTypeAndId(new org.apache.lucene.util.BytesRef(doc['_uid'].value))[0] is the type.
Related
In my elastic search index, each document will have two date fields createdDate and modifiedDate. I'm trying to add a filter in kibana to fetch the documents where the modifiedDate is greater than createdDate. How to create this filter in kibana?
Tried Using below query instead of greater than it is considering as gte and fetching all records
GET index/_search
{
"query": {
"bool": {
"filter": {
"script": {
"script" : {
"inline" : "doc['modifiedTime'].value.getMillis() > doc['createdTime'].value.getMillis()",
"lang" : "painless"
}
}
}
}
}
}
There are a few options.
Option A: The easiest and most performant one is to store the difference of the two fields inside a new field of your document, e.g.
{
"createDate": "2022-01-11T12:34:56Z",
"modifiedDate": "2022-01-11T12:34:56Z",
"diffMillis": 0
}
{
"createDate": "2022-01-11T12:34:56Z",
"modifiedDate": "2022-01-11T12:35:58",
"diffMillis": 62000
}
Then, in Kibana you can query on diffMillis > 0 and figure out all documents that have been modified after their creation.
Option B: You can use a script query
GET index/_search
{
"query": {
"bool": {
"filter": {
"script": {
"script": """
return doc['createdDate'].value.millis < doc['modifiedDate'].value.millis;
"""
}
}
}
}
}
Note: depending on the amount of data you have, this option can potentially have disastrous performance, because it needs to be evaluated on ALL of your documents.
Option C: If you're using ES 7.11+, you can use runtime fields directly from the Kibana Discover view.
You can use the following script in order to add a new runtime field (e.g. name it diffMillis) to your index pattern:
emit(doc['modifiedDate'].value.millis - doc['createdDate'].value.millis)
And then you can add the following query into your search bar
diffMillis > 0
We are using elastic version 7.10.2. I want to compare two fields from a same document.Scripting is disabled in my organization.
Kindly help in building below query without using script.
Here my query is : nickname is null or nickname is empty or nickname is equal to firstname.
Hard part is how to build query to get the records which have nickname is equal to firstname
Relevant script query to be converted to normal query :
{
"query": {
"bool": {
"must": [{
"script": {
"script": {
"inline": "doc['nickname.keyword'].value==null || doc['nickname.keyword'].value =='' || doc['nickname.keyword'].value == doc['firstname.keyword'].value",
"lang": "painless",
}
}
}]
}
}
}
I see you are already comparing the nickname.keyword to your firstname also mentioned this is the hard part, for this why you need a script, you can simply use the search query on this keyword field and get the result you want.
You can use below term query for it.
{
"query": {
"term": {
"nickname.keyword": {
"value": "your-nickname", // provide your nickname as value
}
}
}
}
I have a document which looks like this:
{
"foo": {
"orgnr": "1"
},
"bar": {
"orgnr" : "2"
},
"created": "2015-02-12",
...
}
I have an API where a user can query for:
orgnr (required)
role (optional) - ANY by default ANY means must match at least one of bar.orgnr or foo.orgnr, but could also be role:BAR, and then it must match bar.orgnr:
created (optional)
query (optional)
Orgnr must match foo.orgnr OR bar.orgnr, and can then have a lot of other field and text queries. I match this doing a query string query. So for a request where orgnr is 1, the following query string would be generated:
(foo.orgnr:1 OR bar.orgnr:1) AND (rest of query)
Where rest of query can be for example
created:[2015-01-01 TO *]
created:[2015-01-01 TO *] AND *query*
But i'm not sure this is actually the correct way or doing this. Reading https://www.elastic.co/guide/en/elasticsearch/guide/current/_queries_and_filters.html makes me insecure.
I could also use a boolean match, with must for the orgnr.
With role:BAR the following query would be generated:
(bar.orgnr:1) AND (rest of query)
The most important here is that orgnr is actually matched towards foo OR bar orgnr.
Or should I use a filter for this instead?
A bool query in filter context is a fine way of doing it. The Should requires at least one clause to match.
POST _search
{
"query": {
"constant_score": {
"filter": {
"bool": {
"should": [
{
"term": {
"foo.orgnr": "1"
}
},
{
"term": {
"bar.orgnr": "1"
}
}
]
}
}
}
}
}
My data has a field localities which is an array of strings.
"localities": [
"Mayur Vihar Phase 1",
"Paschim Vihar",
"Rohini",
"",
"Laxmi Nagar",
"Vasant Vihar",
"Dwarka",
"Karol Bagh",
"Inderlok" ]
What query should I write to filter the documents by a specific locality such as "Rohini"?
A simple match query will be enough (if you don't know the mapping of your localities field).
POST <your index>/_search
{
"query": {
"match": {
"localities": "Rohini"
}
}
}
If the localities field is set as a string type and index as not_analyzed, the best way to query this is to use a term filter, wrapped in a filtered query (you can't use directly filters) :
POST <your index>/_search
{
"query": {
"filtered": {
"filter": {
"term": {
"localities": "Rohini"
}
}
}
}
}
If you doesn't need the score, the second solution is the way to go as filters doesn't compute score, are faster and cached.
Check the documentation for information about analysis which is a very important subject in ElasticSearch, heavily influencing the way you query.
POST /_search
{
"query": {
"match": {
"localities": "Rohini"
}
}
}
Or you can simply query:
GET /_search?q=localities:Rohini
This query is returning fine only one item "steve_jobs".
{
"query": {
"constant_score": {
"filter": {
"term": {
"name":"steve_jobs"
}
}
}
}
}
So, now I want to get all people with name prefix steve_. So I try this:
{
"query": {
"constant_score": {
"filter": {
"term": {
"name": "steve_"
}
}
}
}
}
This is returning nothing. Why?
I'm confused about when to use term query / term filter / terms filter / querystring query.
What you need is Prefix Query.
If you are indexing your document like so:
POST /testing_nested_query/class/
{
"name": "my name is steve_jobs"
}
And you are using the default analyzer, then the problem is that the term steve_jobs will be indexed as one term. So your Term Query will never be able to find any docs matching the term steve as there is no term like in the index. Prefix Query helps you solve your problem by searching for a prefix in all the indexed terms.
You can solve the same problem by making your custom analyzers (read this and this) so that steve_jobs is stored as steve and jobs.