Experiencing an issue with ES,
I have a mapping for a user type, specifying a field as keyword
GET _template/user_template
Returns:
{
...
"primary_user": {
"type": "keyword"
}
}
The following filter request will return with hits
GET users/user/_search
{
"query": {
"bool": {
"filter": {
"term": {
"primary_user.keyword": "AWBFyulcxxxxxxxx"
}
}
}
}
}
The following request will return with 0 hits.
GET users/user/_search
{
"query": {
"bool": {
"filter": {
"term": {
"primary_user": "AWBFyulcxxxxxxxx"
}
}
}
}
}
From the Dev tools autocomplete, I can see the ES regards the primary_user as text.
What am I missing?
Check the name of the index with the template index pattern: the template will be applied only to index with name matching the index pattern.
In addition templates are only applied at index creation time and changing a template will have no impact on existing indices: if you have updated the template, you have to create a new index (ord deleting and recreating an existing one) for viewing the changes in the mapping.
Related
In my elastic search index, each document will have two date fields createdDate and modifiedDate. I'm trying to add a filter in kibana to fetch the documents where the modifiedDate is greater than createdDate. How to create this filter in kibana?
Tried Using below query instead of greater than it is considering as gte and fetching all records
GET index/_search
{
"query": {
"bool": {
"filter": {
"script": {
"script" : {
"inline" : "doc['modifiedTime'].value.getMillis() > doc['createdTime'].value.getMillis()",
"lang" : "painless"
}
}
}
}
}
}
There are a few options.
Option A: The easiest and most performant one is to store the difference of the two fields inside a new field of your document, e.g.
{
"createDate": "2022-01-11T12:34:56Z",
"modifiedDate": "2022-01-11T12:34:56Z",
"diffMillis": 0
}
{
"createDate": "2022-01-11T12:34:56Z",
"modifiedDate": "2022-01-11T12:35:58",
"diffMillis": 62000
}
Then, in Kibana you can query on diffMillis > 0 and figure out all documents that have been modified after their creation.
Option B: You can use a script query
GET index/_search
{
"query": {
"bool": {
"filter": {
"script": {
"script": """
return doc['createdDate'].value.millis < doc['modifiedDate'].value.millis;
"""
}
}
}
}
}
Note: depending on the amount of data you have, this option can potentially have disastrous performance, because it needs to be evaluated on ALL of your documents.
Option C: If you're using ES 7.11+, you can use runtime fields directly from the Kibana Discover view.
You can use the following script in order to add a new runtime field (e.g. name it diffMillis) to your index pattern:
emit(doc['modifiedDate'].value.millis - doc['createdDate'].value.millis)
And then you can add the following query into your search bar
diffMillis > 0
Default Elasticsearch adds any new attribute into the index mapping with type text that contains the string value but I need multi fields support(text and keyword)
Your mapping is correct and it can be used for match and term queries, you can find more info about multi-fields here.
For fulltext query you can use this query:
{
"query": {
"match": {
"dynamic_field001": "search"
}
}
}
and for term query:
{
"query": {
"term": {
"dynamic_field001.keyword": "search"
}
}
}
I have a document which looks like this:
{
"foo": {
"orgnr": "1"
},
"bar": {
"orgnr" : "2"
},
"created": "2015-02-12",
...
}
I have an API where a user can query for:
orgnr (required)
role (optional) - ANY by default ANY means must match at least one of bar.orgnr or foo.orgnr, but could also be role:BAR, and then it must match bar.orgnr:
created (optional)
query (optional)
Orgnr must match foo.orgnr OR bar.orgnr, and can then have a lot of other field and text queries. I match this doing a query string query. So for a request where orgnr is 1, the following query string would be generated:
(foo.orgnr:1 OR bar.orgnr:1) AND (rest of query)
Where rest of query can be for example
created:[2015-01-01 TO *]
created:[2015-01-01 TO *] AND *query*
But i'm not sure this is actually the correct way or doing this. Reading https://www.elastic.co/guide/en/elasticsearch/guide/current/_queries_and_filters.html makes me insecure.
I could also use a boolean match, with must for the orgnr.
With role:BAR the following query would be generated:
(bar.orgnr:1) AND (rest of query)
The most important here is that orgnr is actually matched towards foo OR bar orgnr.
Or should I use a filter for this instead?
A bool query in filter context is a fine way of doing it. The Should requires at least one clause to match.
POST _search
{
"query": {
"constant_score": {
"filter": {
"bool": {
"should": [
{
"term": {
"foo.orgnr": "1"
}
},
{
"term": {
"bar.orgnr": "1"
}
}
]
}
}
}
}
}
My data has a field localities which is an array of strings.
"localities": [
"Mayur Vihar Phase 1",
"Paschim Vihar",
"Rohini",
"",
"Laxmi Nagar",
"Vasant Vihar",
"Dwarka",
"Karol Bagh",
"Inderlok" ]
What query should I write to filter the documents by a specific locality such as "Rohini"?
A simple match query will be enough (if you don't know the mapping of your localities field).
POST <your index>/_search
{
"query": {
"match": {
"localities": "Rohini"
}
}
}
If the localities field is set as a string type and index as not_analyzed, the best way to query this is to use a term filter, wrapped in a filtered query (you can't use directly filters) :
POST <your index>/_search
{
"query": {
"filtered": {
"filter": {
"term": {
"localities": "Rohini"
}
}
}
}
}
If you doesn't need the score, the second solution is the way to go as filters doesn't compute score, are faster and cached.
Check the documentation for information about analysis which is a very important subject in ElasticSearch, heavily influencing the way you query.
POST /_search
{
"query": {
"match": {
"localities": "Rohini"
}
}
}
Or you can simply query:
GET /_search?q=localities:Rohini
This query is returning fine only one item "steve_jobs".
{
"query": {
"constant_score": {
"filter": {
"term": {
"name":"steve_jobs"
}
}
}
}
}
So, now I want to get all people with name prefix steve_. So I try this:
{
"query": {
"constant_score": {
"filter": {
"term": {
"name": "steve_"
}
}
}
}
}
This is returning nothing. Why?
I'm confused about when to use term query / term filter / terms filter / querystring query.
What you need is Prefix Query.
If you are indexing your document like so:
POST /testing_nested_query/class/
{
"name": "my name is steve_jobs"
}
And you are using the default analyzer, then the problem is that the term steve_jobs will be indexed as one term. So your Term Query will never be able to find any docs matching the term steve as there is no term like in the index. Prefix Query helps you solve your problem by searching for a prefix in all the indexed terms.
You can solve the same problem by making your custom analyzers (read this and this) so that steve_jobs is stored as steve and jobs.