How to delete data based on condition from elastic search index using RestHighLevelClient in spring boot
The stackoverflow above describe how to remove main_phone1=1 from companyphones[] array inside the ES document via 'painless'
Anyone know how to remove data from fields[] from ES document without using 'painless' script inside the java code. For example construct the QueryBuilder and using esresttemplate to invoke the delete data?
currently I am using springboot + springdataelasticsearch jar + mvn
Thank you
Related
Wondering how to do this in java using spring data elasticsearch library.
GET /my-index-000001/_mapping/field/user
This is not supported by Spring Data Elasticsearch. You'll have to get the mapping for the index and extract the part you need from the returned Map<String, Object>.
How to convert below query to Spring Data elasticsearch
GET indexName/_doc/{id}?_source_includes=jsonField1,jsonField2
source filtering for get requests is currently not supported in Spring Data Elasticsearch.
Edit 09.12.2021:
If you don't need source filtering, to get a Document by id you can either use
ElasticsearchOperations#get(java.lang.String, java.lang.Class<T>)
or
ElasticsearchRepository#findById(ID)
I have a java code which connects to Elasticsearch DB using Spring-data-elasticsearch and fetches all the index data by connecting to the repository and executing the findAll() method. The data received from ES is being processed by a seperate application. When new data is inserted into elastic search, I have the below queries
1. How can I fetch only the newly inserted data Programatically ?
2. Apart from using the DSL queries, Is there a way to Asyncronously get the new records as and when new data is inserted into elasticsearch DB.
I dont want to execute the findAll() method again. Because it returns the entire data ( including the previously processed records as well) .
Any help on this is much appreciated.
You will need to add a field (I call it createdAt here) to your entities that contains the timestamp when your application inserts into Elasticsearch. One possibility would be to use the auditing support of Spring Data Elasticsearch to have the value set automatically, or you set the value in your application. If the data is inserted by some other application you need to make sure that it contains a timestamp in a format that maps the field type definition of this field in your application.
Then you'd need to define a method in your repository like
SearchHits<T> findByCreatedAtAfter(Timestamp referenceValue);
As for getting a notification in some form when new data is inserted: I'm not aware that Elasticsearch offers something like that. You will probably need to regularly call the method that retrieves the data.
We have a few similar queries and I wanted to do some templating based on parameters. Recently I've found that elastic supports search templates so I'm wondering whether this is supported by spring-data-elasticearch.
Currently my query looks something like:
final Query query = new NativeSearchQueryBuilder().addAggregation(aggregationBuilder)
.withPageable(EmptyPage.INSTANCE)
.withQuery(queryBuilder)
.build();
I'm wondering if I can somehow pass the template that I've stored in application and get the result from elastic. Or if I can store the template in elastic and get the result based on parameters.
No, Spring Data Elasticsearch currently does not support search templates.
Edit 16.03.2021: search template support has been added to the ReactiveElasticsearchClientin Spring Data Elasticsearch by a pull request from bilak. Thanks for that.
I'd like to have a properties set up to adjust fuzziness of elasticsearch search request as a whole application set up, i.e not changing this per #Query of the individual MyEntitySearchRepository. Is there a way to specify this using 1) some SpringBoot properties to be picked up by the Spring Data ElasticSearch 2) using ElasticsearchTemplate to prepopulate it with the fuzzy value from the homegrown spring boot property, while the other part of the app queries to go to ElasticSearch should go from the Spring data definitions (index names, by/in/like parameters). Is it ever possible, or for now the only way it to set up individual #Query to form the request json, containing fuzzy parameter like is described there and I can only paste the fuzzy value there being taken from the homegrown SpringBoot property?
This is at the moment not possible, and I'm not sure if I understand you right: You want to define a global fuzzy setting that should be applied to all queries? On which fields of your document? All String fields?
There is no global fuzzy setting in Elasticsearch itself, so it would be necessary to build custom queries internally.
At the moment the only way to go is with #Query annotated custom repository methods.