We have a few similar queries and I wanted to do some templating based on parameters. Recently I've found that elastic supports search templates so I'm wondering whether this is supported by spring-data-elasticearch.
Currently my query looks something like:
final Query query = new NativeSearchQueryBuilder().addAggregation(aggregationBuilder)
.withPageable(EmptyPage.INSTANCE)
.withQuery(queryBuilder)
.build();
I'm wondering if I can somehow pass the template that I've stored in application and get the result from elastic. Or if I can store the template in elastic and get the result based on parameters.
No, Spring Data Elasticsearch currently does not support search templates.
Edit 16.03.2021: search template support has been added to the ReactiveElasticsearchClientin Spring Data Elasticsearch by a pull request from bilak. Thanks for that.
Related
How to convert below query to Spring Data elasticsearch
GET indexName/_doc/{id}?_source_includes=jsonField1,jsonField2
source filtering for get requests is currently not supported in Spring Data Elasticsearch.
Edit 09.12.2021:
If you don't need source filtering, to get a Document by id you can either use
ElasticsearchOperations#get(java.lang.String, java.lang.Class<T>)
or
ElasticsearchRepository#findById(ID)
We are using ElasticSearch to save and manage information on complex transactions. We might need to add more information for every transaction, on the near future.
How about including JSON doc version?
Is it possible for elastic search, to include different versions of JSON docs, to save and to search?
How does this affects performance on ElasticSearch?
It's completely possible, By default elastic uses the dynamic mappings for every new documents such as your JSON documents to index them. For each field in your documents elastic creates a table called inverted_index and the search queries executed against them so regardless of your field variation as long as you know which field you want to execute query the data throughput and performance will not be affected.
Mostly what I do is to assemble the mapping by hand. Choosing the correct types myself.
Is there any tool which facilitates this?
For example which will read a class (c#,java..etc) and choosing the closest ES types accordingly.
I've never seen such a tool, however I know that ElasticSearch has a REST API over HTTP.
So you can create a simple HTTP query with JSON body that will depict your object with your fields: field names + types (Strings, numbers, booleans) - pretty much like a Java/C# class that you've described in the question.
Then you can ask the ES to store the data in the non-existing index (to "index" your document in ES terms). It will index the document, but it will also create an index, and the most importantly for your question, will create a mapping for you "dynamically", so that later you will be able to query the mapping structure (again via REST).
Here is the link to the relevant chapter about dynamically created mappings in the ES documentation
And Here you can find the API for querying the mapping structure
At the end of the day you'd still want to retain some control over how your mapping is generated. I'd recommend:
syncing some sample documents w/o a mapping
investigating what mapping was auto generated and
dropping the index & using dynamic_templates to pseudo-auto-generate / update the mapping as new documents come in.
This GUI could help too.
Currently, there is no such tool available to generate the mapping for elastic.
It is a kind of similar thing as we have to design a database in MySQL.
But if we want such kind of thing then we use Mongo DB which requires no predefined schema.
But Elastic comes with its very dynamic feature, which allows us to play around it. One of the most important features of Elasticsearch is that it tries to get out of your way and let you start exploring your data as quickly as possible like the mongo schema which can be manipulated dynamically.
To index a document, you don’t need to first define a mapping or schema and define your fields along with their data type .
You can just index a document and the index, type, and fields will be created automatically.
For further details you can go through the below documentation:
Elastic Dynamic Mapping
I'd like to have a properties set up to adjust fuzziness of elasticsearch search request as a whole application set up, i.e not changing this per #Query of the individual MyEntitySearchRepository. Is there a way to specify this using 1) some SpringBoot properties to be picked up by the Spring Data ElasticSearch 2) using ElasticsearchTemplate to prepopulate it with the fuzzy value from the homegrown spring boot property, while the other part of the app queries to go to ElasticSearch should go from the Spring data definitions (index names, by/in/like parameters). Is it ever possible, or for now the only way it to set up individual #Query to form the request json, containing fuzzy parameter like is described there and I can only paste the fuzzy value there being taken from the homegrown SpringBoot property?
This is at the moment not possible, and I'm not sure if I understand you right: You want to define a global fuzzy setting that should be applied to all queries? On which fields of your document? All String fields?
There is no global fuzzy setting in Elasticsearch itself, so it would be necessary to build custom queries internally.
At the moment the only way to go is with #Query annotated custom repository methods.
I am exploring Kibana for data exploration, and have created a query using KQL:
title:biologist or description:biology
I would like to use this same query as a saved filter that I would use in my dashboards. However, I can not find a way to convert this search to a filter.
I see that filters can be created using "Query DSL" that is a JSON config. I believe I can find a way to write my KQL search in Query DSL, but it seems there should be a way to convert my KQL search directly to a filter.
Or my understanding of filters vs queries needs some updating.
Thank you very much,
Jay, at the moment, Filters in Kibana use the AND syntax, there isn't an option for Or filters. You have a few options to do what you want:
Create a filter with one of the items, e.g. title:biologist and a query for the other. You could also edit a filter using DSL. Here's a link to the filters docs: https://www.elastic.co/guide/en/kibana/current/field-filter.html. For DSL, you can have a look here https://www.elastic.co/guide/en/elasticsearch/reference/7.3/query-dsl-bool-query.html#