Elasticsearch: bulk update multiple documents saved in a Java String? - elasticsearch

I can create the following string saved in a Java String object called updates.
{ "update":{ "_index":"myindex", "_type":"order", "_id":"1"} }
{ "doc":{"field1" : "aaa", "field2" : "value2" }}
{ "update":{ "_index":"myindex", "_type":"order", "_id":"2"} }
{ "doc":{"field1" : "bbb", "field2" : "value2" }}
{ "update":{ "_index":"myindex", "_type":"order", "_id":"3"} }
{ "doc":{"field1" : "ccc", "field2" : "value2" }}
Now I want to do bullk update within a Java program:
Client client = getClient(); //TransportClient
BulkRequestBuilder bulkRequest = client.prepareBulk();
//?? how to attach updates variable to bulkRequest?
BulkResponse bulkResponse = bulkRequest.execute().actionGet();
I am unable to find a way to attach the above updates variable to bulkRequest before execute.
I notice that I am able to add UpdateRequest object to bulkRequest, but it seems to add only one document one time. As indicated above, I have multiple to-be-updated document in one string.
Can someone enlighten me on this? I have a gut feel that I may do things wrong way.
Thanks and regards.

The following code should work fine for you.
For each document updation , you need to create a separate update request as below and keep on adding it to the bulk requests.
Once the bulk requests is ready , execute a get on it.
JSONObject obj = new JSONObject();
obj.put("field1" , "value1");
obj.put("field2" , "value2");
UpdateRequest updateRequest = new UpdateRequest(index, indexType, id1).doc(obj.toString());
BulkRequestBuilder bulkRequest = client.prepareBulk();
bulkRequest.add(updateRequest);
obj = new JSONObject();
obj.put("fieldX" , "value1");
obj.put("fieldY" , "value2");
updateRequest = new UpdateRequest(index, indexType, id2).doc(obj.toString());
bulkRequest = client.prepareBulk();
bulkRequest.add(updateRequest);
bulkRequest.execute().actionGet();

I ran into the same problem where only 1 document get updated in my program. Then I found the following way which worked perfectly fine. This uses spring java client. I have also listed the the dependencies I used in the code.
import org.elasticsearch.action.update.UpdateRequest;
import org.elasticsearch.index.query.QueryBuilder;
import org.springframework.data.elasticsearch.core.query.UpdateQuery;
import org.springframework.data.elasticsearch.core.query.UpdateQueryBuilder;
private UpdateQuery updateExistingDocument(String Id) {
// Add updatedDateTime, CreatedDateTime, CreateBy, UpdatedBy field in existing documents in Elastic Search Engine
UpdateRequest updateRequest = new UpdateRequest().doc("UpdatedDateTime", new Date(), "CreatedDateTime", new Date(), "CreatedBy", "admin", "UpdatedBy", "admin");
// Create updateQuery
UpdateQuery updateQuery = new UpdateQueryBuilder().withId(Id).withClass(ElasticSearchDocument.class).build();
updateQuery.setUpdateRequest(updateRequest);
// Execute update
elasticsearchTemplate.update(updateQuery);
}

Related

More Like This Query Not Getting Serialized - NEST

I am trying to create an Elasticsearch MLT query using NEST's object initializer syntax. However, the final query when serialized, is ONLY missing the MLT part of it. Every other query is present though.
When inspecting the query object, the MLT is present. It's just not getting serialized.
I wonder what I may be doing wrong.
I also noticed that when I add Fields it works. But I don't believe fields is a mandatory property here that when it is not set, then the MLT query is ignored.
The MLT query is initialized like this;
new MoreLikeThisQuery
{
Like = new[]
{
new Like(new MLTDocProvider
{
Id = parameters.Id
}),
}
}
MLTDocProvider implements the ILikeDocument interface.
I expect the serialized query to contain the MLT part, but it is the only part that is missing.
This looks like a bug in the conditionless behaviour of more like this query in NEST; I've opened an issue to address. In the meantime, you can get the desired behaviour by marking the MoreLikeThisQuery as verbatim, which will override NEST's conditionless behaviour
var client = new ElasticClient();
var parameters = new
{
Id = 1
};
var searchRequest = new SearchRequest<Document>
{
Query = new MoreLikeThisQuery
{
Like = new[]
{
new Like(new MLTDocProvider
{
Id = parameters.Id
}),
},
IsVerbatim = true
}
};
var searchResponse = client.Search<Document>(searchRequest);
which serializes as
{
"query": {
"more_like_this": {
"like": [
{
"_id": 1
}
]
}
}
}

Spring boot custom query MongoDB

I have this MongoDb query:
db.getCollection('user').find({
$and : [
{"status" : "ACTIVE"},
{"last_modified" : { $lt: new Date(), $gte: new Date(new Date().setDate(new Date().getDate()-1))}},
{"$expr": { "$ne": ["$last_modified", "$time_created"] }}
]
})
It works in Robo3T, but when I put this in spring boot as custom query, it throws error on project start.
#Query("{ $and : [ {'status' : 'ACTIVE'}, {'last_modified' : { $lt: new Date(), $gte: new Date(new Date().setDate(new Date().getDate()-1))}}, {'$expr': { '$ne': ['$last_modified', '$time_created']}}]}")
public List<User> findModifiedUsers();
I tried to make query with Criteria in spring:
Query query = new Query();
Criteria criteria = new Criteria();
criteria.andOperator(Criteria.where("status").is(UserStatus.ACTIVE), Criteria.where("last_modified").lt(new Date()).gt(lastDay), Criteria.where("time_created").ne("last_modified"));
but it doesn't work, it returns me all users like there is no this last criteria not equal last_modified and time_created.
Does anyone know what could be problem?
I think that this feature is not supported yet by Criteria - check this https://jira.spring.io/browse/DATAMONGO-1845 .
One workaround is to pass raw query via mongoTemplate like this:
BasicDBList expr = new BasicDBList();
expr.addAll(Arrays.asList("$last_modified","$time_created"));
BasicDBList and = new BasicDBList();
and.add(new BasicDBObject("status","ACTIVE"));
and.add(new BasicDBObject("last_modified",new BasicDBObject("$lt",new Date()).append("$gte",lastDate)));
and.add(new BasicDBObject("$expr",new BasicDBObject("$ne",expr)));
Document document = new Document("$and",and);
FindIterable<Document> result = mongoTemplate.getCollection("Users").find(document);

spring jpa won't update new data

I would like to update new data from request.
Request JSON data looks like this.
[{"media_id : 1, "path" : "some path", ...}, {"media_id : 2, "path" : "some path", ...}]
These primary keys already exist in database
So It will update those rows and It should be updated
But update sql on debug log only update old data
I checked out media object that It contains new data from request
But jpa still try update with old data
What is my mistake?
private List<Media> upsertMedia(SquarePostDetailResource postToUpsert) {
List<Media> media = postToUpsert.getContent().getMedia();
media.forEach((item) -> {
item.setCreatedAt(item.getId() == null ? new Date() : item.getCreatedAt());
item.setModifiedAt(new Date());
item.setMember(Member.builder().id(postToUpsert.getMemberId()).build());
item.setSquare(Square.builder().id(postToUpsert.getSquareId()).build());
item.setSquarePost(SquarePost.builder().id(postToUpsert.getPostId()).build());
});
return (List<Media>) mediaRepo.save(media);
}
If you mean that updates inside foreach do not update the database, you can try the following.
private List<Media> upsertMedia(SquarePostDetailResource postToUpsert) {
List<Media> media = postToUpsert.getContent().getMedia();
List<Media> updatedMedia = new ArrayList<>();
media.forEach((item) -> {
item.setCreatedAt(item.getId() == null ? new Date() : item.getCreatedAt());
item.setModifiedAt(new Date());
item.setMember(Member.builder().id(postToUpsert.getMemberId()).build());
item.setSquare(Square.builder().id(postToUpsert.getSquareId()).build());
item.setSquarePost(SquarePost.builder().id(postToUpsert.getPostId()).build());
updatedMedia.add(item);
});
return (List<Media>) mediaRepo.save(updatedMedia);
}

MongoDB how update element in array using Spring Query Update

In my project I'm using SpringBoot 1.3.2 and org.springframework.data.mongodb.core.query.*
I'm trying to update element in array, in my main object i have array looking like this:
"sections" : [
{
"sectionId" : "56cc3c908f5e6c56e677bd2e",
"name" : "Wellcome"
},
{
"sectionId" : "56cc3cd28f5e6c56e677bd2f",
"name" : "Hello my friends"
}
]
Using Spring I want to update name of record with sectionId 56cc3c908f5e6c56e677bd2e
I was trying to to this like that but it didn't work
Query query = Query.query(Criteria
.where("sections")
.elemMatch(
Criteria.where("sectionId").is(editedSection.getId())
)
);
Update update = new Update().set("sections", new BasicDBObject("sectionId", "56cc3c908f5e6c56e677bd2e").append("name","Hi there"));
mongoTemplate.updateMulti(query, update, Offer.class);
It create something like:
"sections" : {
"sectionId" : "56cc3c908f5e6c56e677bd2e",
"name" : "Hi there"
}
But this above is object { } I want an array [ ], and I don't want it remove other elements.
Can any body help me how to update name of record with sectionId 56cc3c908f5e6c56e677bd2e using Spring
You essentially want to replicate this mongo shell update operation:
db.collection.update(
{ "sections.sectionId": "56cc3c908f5e6c56e677bd2e" },
{
"$set": { "sections.$.name": "Hi there" }
},
{ "multi": true }
)
The equivalent Spring Data MongoDB code follows:
import static org.springframework.data.mongodb.core.query.Criteria.where;
import static org.springframework.data.mongodb.core.query.Query;
import static org.springframework.data.mongodb.core.query.Update;
...
WriteResult wr = mongoTemplate.updateMulti(
new Query(where("sections.sectionId").is("56cc3c908f5e6c56e677bd2e")),
new Update().set("sections.$.name", "Hi there"),
Collection.class
);
Can use BulkOperations approach to update list or array of document objects
BulkOperations bulkOps = mongoTemplate.bulkOps(BulkMode.UNORDERED, Person.class);
for(Person person : personList) {
Query query = new Query().addCriteria(new Criteria("id").is(person.getId()));
Update update = new Update().set("address", person.setAddress("new Address"));
bulkOps.updateOne(query, update);
}
BulkWriteResult results = bulkOps.execute();
Thats my solution for this problem:
public Mono<ProjectChild> UpdateCritTemplChild(
String id, String idch, String ownername) {
Query query = new Query();
query.addCriteria(Criteria.where("_id")
.is(id)); // find the parent
query.addCriteria(Criteria.where("tasks._id")
.is(idch)); // find the child which will be changed
Update update = new Update();
update.set("tasks.$.ownername", ownername); // change the field inside the child that must be updated
return template
// findAndModify:
// Find/modify/get the "new object" from a single operation.
.findAndModify(
query, update,
new FindAndModifyOptions().returnNew(true), ProjectChild.class
)
;
}

Querying with Completion Suggesters in Elasticsearch with Java API

I have my indices created, and mapping type for my 'suggest' field set to completion. I can't figure out how to configure the query for completion suggestions in elastic-search (Java API)
I'm trying to use this Query to base my implementation off of.
"song-suggest" : {
"text" : "n",
"completion" : {
"field" : "suggest"
}
}
Here's what I have so far,
CompletionSuggestionBuilder compBuilder = new CompletionSuggestionBuilder("complete");
compBuilder.text("n");
compBuilder.field("suggest");
SearchResponse searchResponse = localClient.prepareSearch(INDEX_NAME)
.setTypes("completion")
.setQuery(QueryBuilders.matchAllQuery())
.addSuggestion(compBuilder)
.execute().actionGet();
CompletionSuggestion compSuggestion = searchResponse.getSuggest().getSuggestion("complete");
Am I missing something, doing something wrong? Thanks!
Not sure if this is the best thing to do. But this works for me. Hope it helps.
#Override
public List<SuggestionResponse> findSuggestionsFor(String suggestRequest) {
CompletionSuggestionBuilder suggestionsBuilder = new CompletionSuggestionBuilder("completeMe");
suggestionsBuilder.text(suggestRequest);
suggestionsBuilder.field("suggest");
SuggestRequestBuilder suggestRequestBuilder =
client.prepareSuggest(MUSIC_INDEX).addSuggestion(suggestionsBuilder);
logger.debug(suggestRequestBuilder.toString());
SuggestResponse suggestResponse = suggestRequestBuilder.execute().actionGet();
Iterator<? extends Suggest.Suggestion.Entry.Option> iterator =
suggestResponse.getSuggest().getSuggestion("completeMe").iterator().next().getOptions().iterator();
List<SuggestionResponse> items = new ArrayList<>();
while (iterator.hasNext()) {
Suggest.Suggestion.Entry.Option next = iterator.next();
items.add(new SuggestionResponse(next.getText().string()));
}
return items;
}
paramsMap = req.getParameterMap();
String prefix = getParam("prefix");
if (prefix == null) {
EndpointUtil.badRequest("Autocomplete EndPoint: prefix parameter is missing", resp);
return;
}
SearchRequest searchRequest;
SearchSourceBuilder searchSourceBuilder;
searchRequest = new SearchRequest("section");
searchSourceBuilder = new SearchSourceBuilder();
searchSourceBuilder.timeout(new TimeValue(60, TimeUnit.SECONDS));
searchSourceBuilder.from(0);
searchSourceBuilder.size(MAX_HITS);
CompletionSuggestionBuilder suggestionBuilder = new CompletionSuggestionBuilder("text.completion")
.prefix(prefix, Fuzziness.AUTO).size(MAX_HITS);
SuggestBuilder suggestBuilder = new SuggestBuilder();
suggestBuilder.addSuggestion(SUGGEST_NAME, suggestionBuilder);
searchSourceBuilder.suggest(suggestBuilder);
searchRequest.source(searchSourceBuilder);
SearchResponse searchResponse = getElasticClient().search(searchRequest);
Suggest suggest = searchResponse.getSuggest();
List<Document> results = new ArrayList<Document>();
Suggest.Suggestion<Suggest.Suggestion.Entry<Suggest.Suggestion.Entry.Option>> suggestion
= suggest.getSuggestion(SUGGEST_NAME);
List<Suggest.Suggestion.Entry<Suggest.Suggestion.Entry.Option>> list = suggestion.getEntries();
for(Suggest.Suggestion.Entry<Suggest.Suggestion.Entry.Option> entry :list) {
List<Suggest.Suggestion.Entry.Option> options = entry.getOptions();
for(Suggest.Suggestion.Entry.Option option : options) {
Document doc = new Document();
doc.append("text",option.getText().toString());
results.add(doc);
}
}
sendJsonResult(results, resp);
But I'm running into the error "field "suggest" doesn't have type 'completion'. My mapping looks like this: code .field("suggest") .startObject() .field("type", "completion") .field("index_analyzer","simple") .field("search_analyzer","simple") .endObject()
It sounds like, that your mapping is not applied correctly. Did you check it out?
Based on the mapping you provided, I think you are missing the properties around your mapping. Try the following mapping:
XContentFactory.jsonBuilder().startObject()
.startObject("properties")
.startObject("suggest")
.field("type", "completion")
.endObject()
.endObject()
.endObject()
Btw, SimpleAnalyzer is the default Analyzer for the suggestions. Thus, you need not define it explicitly.
To anyone who still needs this. The code snippet below works with ES v 6.3:-
CompletionSuggestionBuilder suggestionBuilder = new CompletionSuggestionBuilder("<field_name>").prefix("<search_term>");
SearchRequestBuilder requestBuilder =
oaEsClient.client().prepareSearch("<index_name>").setTypes("<type_name>")
.suggest(new SuggestBuilder().addSuggestion("<suggestion_name>",suggestionBuilder))
.setSize(20)
.setFetchSource(true)
.setExplain(false)
;
SearchResponse response = requestBuilder.get();
Suggest suggest = response.getSuggest();

Resources