I have to upgrade our ES from 6.2 (TCP client) to 7.4.2 (REST API)
And I have a little problem with UpdateByQueryRequestBuilder, that looks like need to be change to UpdateByQueryRequest (doc). The old code looks like this:
BoolQueryBuilder dateQueryBuilder = ...
QueryBuilder a = ...
BoolQueryBuilder b = ...
UpdateByQueryRequestBuilder updateByQuery = new UpdateByQueryRequestBuilder(tcpClient, UpdateByQueryAction.INSTANCE);
updateByQuery.filter(dateQueryBuilder.filter(a).filter(b)).script(updateScript);
As I wrote, I'm understanding UpdateByQueryRequestBuilder (that using the oldest client) should be replace with UpdateByQueryRequest but this new API haven't filter method (just setQuery that will replace the current query in chain case...)
UpdateByQueryRequest updateRequest = new UpdateByQueryRequest();
updateRequest.setQuery(dateQueryBuilder)
// .setQuery(a) - will replace dateQueryBuilder instead of chain new filter...
// .filter - not exist in the new API
So the question, how should I replace this code with newest ES REST API (or chain the queries)?
Related
Steps to write elastic search count api using spring boot.?
how to use countRequest to get count of documents...??
need code snippets.
Using new Java API Client:
var countRequestResponse = client.count(CountRequest.of(c -> c.index("idx_name")));
System.out.println(countRequestResponse.count());
Using Java High Client Library
var countRequest = new CountRequest();
countRequest.indices("idx_name");
var response = getClient().count(countRequest, RequestOptions.DEFAULT);
System.out.println(response.getCount());
We wanted to create IndexRequest, DeleteRequest, UpdateRequest and BulkRequest in Elasticsearch version 8 using JAVA APIs. But I don't see any java documentation in elasticsearch v8 official website. Previously in elasticsearch version 7, we used below code in order to perform operations.
IndexRequest indexRequest = Requests.indexRequest(index).id(key).source(source);
BulkRequest bulkRequest = Requests.bulkRequest();
bulkRequest.add(indexRequest);
Also following Elasticsearch Java API Client [8.1] , but no luck.
Problem arises when we try to do Requests.indexRequest(), this Request class is not available in version 8.
So, Is it possible to create similar request in ES version 8 also?
Update 1:-
My point here is that I need to keep a list of request operation which maybe arbitrary ( maybe 1st five are inserts, next 2 are update and next 2 are delete requests and at the end 1 insert operation ). And that list needed to be flushed via Bulk maintaining the type of request received. I am using BulkRequest.Builder bulkRequestBuilder = new BulkRequest.Builder();
But my issue is with bulk update. I am unable to find any update API for bulkrequest for elasticsearch version 8.
For Insert:-
bulkRequestBuilder.operations(op -> op.index(idx -> idx.index(index).id(key).document(source)));
For Delete:-
bulkRequestBuilder.operations(op -> op.delete(d -> d.index(index).id(key)));
And flushing the bulk operation:-
BulkResponse bulkResponse = client.bulk(bulkRequestBuilder.build());
I am looking for something similar to above mentioned insert and delete operation.
Like, bulkRequestBuilder.operations(op->op.update(u->u.index(index).id(key)....))
You can use the refer the following code which I used to bulk index.
String PRODUCT_INDEX="product"
final BulkResponse bulkResponse = esUtil.getESClient().bulk(builder -> {
for (Product product : products) {
builder.index(PRODUCT_INDEX)
.operations(ob -> {
ob.index(ib -> ib.document(product).pipeline("score").id(product.getId())));
return ob;
});
}
return builder;
});
You can use Fluent DSL like below as mentioned here:
List<Product> products = fetchProducts();
BulkRequest.Builder br = new BulkRequest.Builder();
for (Product product : products) {
br.operations(op -> op
.index(idx -> idx
.index("products")
.id(product.getSku())
.document(product)
)
);
}
BulkResponse result = esClient.bulk(br.build());
You can use Classic Builder like below (Not Recommndate):
IndexRequest.Builder<Product> indexReqBuilder = new IndexRequest.Builder<>();
indexReqBuilder.index("product");
indexReqBuilder.id("id");
indexReqBuilder.document(product);
List<BulkOperation> list = new ArrayList<BulkOperation>();
list.add(indexReqBuilder);
BulkRequest.Builder br = new BulkRequest.Builder();
br.index("");
br.operations(list);
client.bulk(br.build());
Update Request:
As mentioned here in document, UpdateRequest class support TDocument and TPartialDocument as parameter. when you want to index document as parial document (means only update) then you can use TPartialDocument and when you want to index document as upsert then you can use TDocument. you can pass Void class for other parameter.
You can check this discussion as well which give you some understanding.
client.update(b -> b
.index("")
.id("")
.doc(product),
Product.class
);
client.update(new UpdateRequest.Builder<Void, Product>()
.index("product")
.id("")
.doc(product)
.build(),
Void.class
);
Bulk Update request:
BulkRequest.Builder br = new BulkRequest.Builder();
for (Product product : products) {
ObjectMapper mapper = new ObjectMapper();
String json = mapper.writeValueAsString(product);
br.operations(op -> op
.update(idx -> idx.index("products").id("").withJson(new StringReader(json))));
}
I am trying to migrate my es 2.2.1 with spring data elastic 2 to ES 5.6.1 with spring Data elastic 3 but when i am trying to use scan scroll method for large dataaset i am not able to access them, its look like ElasticsearchTemaplate class do not have these function any more in the newer version Can you please let me know what is the substitute of scan and scroll in SDA 3.x:-
ElasticsearchTemplate rep=null;
String scrollId = rep.scan(searchQuery, (long)25000, false);
//List<SampleEntity> sampleEntities = new ArrayList<SampleEntity>();
boolean hasRecords = true;
You now can use startScroll method instead of scan() and scroll().
It's not presented in current docs.
Here are example ElasticsearchTemplate retrieve big data sets
While exploring aggregation in elasticsearch I found out that aggregation functionality can be implemented via JSON query in HTTP based JEST client but not in TCP based Java client.
I am using Jest Client and implemented aggregation through query String that works fine. But I feel it gets quiet cumbersome as the filters increase.
I want to know if there is a way to implement aggregations other than using JSON query in JEST client (Something like aggregation builders in TCP client) and how do we implement it?
This was my solution to implement aggregations, don't forget to add the query first.
org.elasticsearch.search.builder.SearchSourceBuilder searchSourceBuilder = new SearchSourceBuilder();
TermsAggregationBuilder termsAggregationBuilder =
AggregationBuilders.terms(fieldName).field("keyword");
termsAggregationBuilder.minDocCount(minDocCount);
termsAggregationBuilder.size(size);
searchRequestBuilder.aggregation(termsAggregationBuilder);
Search search = new io.searchbox.core.Search.Builder(searchSourceBuilder.toString())
.addIndex(indexNameBuilder.getIndexName(searchContext.getCompanyId()))
.build();
SearchResult result = jestConnectionManager.getClient().execute(search);
Here's what I landed on:
SearchSourceBuilder searchBuilder = SearchSourceBuilder
.searchSource()
.size(0)
.query(QueryBuilders.termQuery("field", "my value"))
.aggregation(
AggregationBuilders
.sum("number_field_sum")
.field("number_field")
);
Search search = new Search.Builder(searchBuilder.toString())
.addIndex("my-index")
.build();
SearchResult result = jestClient.execute(search);
This is largely the same as what you came up with, but I put the query into the SearchSourceBuilder and I simplified the example a bit.
I am trying to remove an alias mapping for an index in ES using jest.
Here is what I have tried :
// create Jest Client.
JestClient client = factory.getObject();
// create RemoveAliasMapping Object.
RemoveAliasMapping removeAliasMapping = new RemoveAliasMapping.Builder("oldIndex", "alias").build();
After creating the removeAliasMapping object, I couldn't find a way to execute it.
If I use the api : client.execute(removeAliasMapping), it says : The method execute(Action<T>) in the type JestClient is not applicable for the arguments (RemoveAliasMapping)
Also, I couldn't find any other api exposed to execute AliasMapping.
Can anyone help me out with this here? If possible, please put an example too.
Try this:
ModifyAliases modifyAliases = new ModifyAliases.Builder(new RemoveAliasMapping.Builder("oldIndex", "alias").build()).build();
JestResult result = client.execute(modifyAliases);