Hi i am using elasticsearch java API for updating a document with script. But i am getting below exception
Exception in thread "main" MapperParsingException[object mapping for [content] tried to parse field [content] as object, but found a concrete value]
at org.elasticsearch.index.mapper.DocumentParser.parseObject(DocumentParser.java:215)
at org.elasticsearch.index.mapper.DocumentParser.parseObjectOrField(DocumentParser.java:308)
at org.elasticsearch.index.mapper.DocumentParser.parseValue(DocumentParser.java:438)
at org.elasticsearch.index.mapper.DocumentParser.parseObject(DocumentParser.java:264)
at org.elasticsearch.index.mapper.DocumentParser.parseDocument(DocumentParser.java:124)
at org.elasticsearch.index.mapper.DocumentMapper.parse(DocumentMapper.java:309)
at org.elasticsearch.index.shard.IndexShard.prepareIndex(IndexShard.java:580)
at org.elasticsearch.index.shard.IndexShard.prepareIndexOnPrimary(IndexShard.java:559)
at org.elasticsearch.action.index.TransportIndexAction.prepareIndexOperationOnPrimary(TransportIndexAction.java:211)
at org.elasticsearch.action.index.TransportIndexAction.executeIndexRequestOnPrimary(TransportIndexAction.java:223)
at org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:157)
at org.elasticsearch.action.index.TransportIndexAction.shardOperationOnPrimary(TransportIndexAction.java:66)
at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.java:657)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:287)
at org.elasticsearch.action.support.replication.TransportReplicationAction$PrimaryOperationTransportHandler.messageReceived(TransportReplicationAction.java:279)
at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:77)
at org.elasticsearch.transport.TransportService$4.doRun(TransportService.java:376)
at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Below is the existing document in the ES
{
"_index": "index1",
"_type": "type1",
"_id": "1",
"_version": 8,
"found": true,
"_source": {
"content": {
"contentId": 1,
"metadata": {
"title": "content one",
"duration": 4500
}
},
"custom": {
"field1": "value1"
}
}
}
I would like to update the "content" field as below
"content": {
"contentId": 1,
"metadata": {
"duration": 900
}
}
when i am updating with REST call (localhost:9200/index1/type1/1/_update), it is working fine. I am getting error in java API prepareUpdate.
I have 3 java classes.
DTO class has Content object
Content class has Metadata object and contentId as long
Metadata class has title (String) and duration(long).
Below is the code to update
Map<String, Object> params = new HashMap<>();
params.put("contentScript", dto.toString());
Script s = new Script("ctx._source.content=contentScript",ScriptType.INLINE,null,params);
UpdateResponse resp = client.prepareUpdate("index1", "type1", "1").setScript(s).setScriptedUpsert(true).get();
dto is object of DTO class and values are set accordingly.
Please help.
params.put("contentScript", dto.toString());
You are passing a string where it is expecting an object. The below code might help.
String script = "ctx._source.pete = jsonMap";
Map<String, Object> jsonMap = new ObjectMapper().readValue(json, HashMap.class);
Map<String, Object> params = ImmutableMap.of("jsonMap", jsonMap);
return new Script(script, ScriptService.ScriptType.INLINE, null, params);
https://discuss.elastic.co/t/how-to-update-nested-objects-in-elasticsearch-2-2-script-via-java-api/43135/2
Related
I have a lambda function the receives an S3Event object when a file is put into an S3 Bucket. When the lambda fails, it goes to a dead letter queue set up in Amazon SQS.
When I pull these messages, this this the body:
{
"Records": [
{
"eventVersion": "2.1",
"eventSource": "aws:s3",
"awsRegion": "us-east-1",
"eventTime": "d",
"eventName": "d:Put",
"userIdentity": {
"principalId": ""
},
"requestParameters": {
"sourceIPAddress": "2"
},
"responseElements": {
"x-amz-request-id": "",
"x-amz-id-2": "g"
},
"s3": {
"s3SchemaVersion": "1.0",
"configurationId": "",
"bucket": {
"name": "",
"ownerIdentity": {
"principalId": ""
},
"arn": ""
},
"object": {
"key": "",
"size": 12502,
"eTag": "",
"sequencer": ""
}
}
}
]
}
That looks quite a bit like the S3Event object which contains a list of S3EventNotification records. I have tried to deserialize it to the S3 Event Object using the following:
S3Event event = new GsonBuilder().serializeNulls().create().fromJson(s3EventString, S3Event.class);
This results in a null object like so:
{"records":null}
I noticed in the json return from SQS, the "R" in Records is capitalized. I wasn't sure if that made a difference so I changed it to a lowercase "r" and it throws this error:
java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING
I'm really not sure what type of object this actually is.
Any help would be greatly appreciated.
Strange. Using Jackson it works perfectly so I will use this for now..
import com.fasterxml.jackson.databind.ObjectMapper;
import com.amazonaws.services.sqs.model.Message;
private S3Event extractS3Event(Message message){
ObjectMapper objectMapper = new ObjectMapper();
return objectMapper.readValue(message.getBody(), S3Event.class)
}
//Then to get the S3 Details
S3Event event = extractS3Event(message);
S3Entity entity = event.getRecords().get(0).getS3();
String bucketName = entity.getBucket().getName();
String s3Key = entity.getObject().getKey();
Re:BEGIN_OBJECT but was STRING.
This is because AWS uses JodaTime for EventTime. You can avoid the issue by removing the field from the JSON text (assuming you do not need it)
I'm issuing a Multi-Get request via the Java High Level REST Client and I'm receiving the following exception:
"Unable to parse response body for Response{requestLine=POST /_mget HTTP/1.1, host=http://localhost:9200, response=HTTP/1.1 200 OK}"
I pulled the following JSON from the logs that was sent to Elastic:
{
"docs": [
{
"_index": "blah",
"_type": null,
"_id": "some-id-232332",
"routing": null,
"stored_fields": null,
"version": -3,
"version_type": "internal",
"_source": {
"includes": [],
"excludes": []
}
}
]
}
I sent the above JSON to Elastic via Postman and I'm seeing the following response (which is the same I see in the logs):
{
"docs": [
{
"_index": "blah",
"_type": null,
"_id": "some-id-232332",
"found": false
}
]
}
Isn't that a valid response? Is this an issue w/ the elasticsearch-rest-high-level-client?
Elastic 7.5.0, org.elasticsearch.client:elasticsearch-rest-high-level-client:7.5.2
I needed to check the getCause()'s of the exception for the real problem. I was passing null to Jackson on mapper.readValue(nullBytes, Customer.class); was the real problem.
Interestingly enough a NPE shows itself 🤷♂️.
Stack trace:
java.util.concurrent.ExecutionException: java.io.IOException: Unable to parse response body for Response{requestLine=POST /_mget HTTP/1.1, host=http://localhost:9200, response=HTTP/1.1 200 OK}
...
...
THE REAL PROBLEM IS HERE!!! 🚨🚨
Caused by: java.lang.IllegalArgumentException: argument "src" is null
at com.fasterxml.jackson.databind.ObjectMapper._assertNotNull(ObjectMapper.java:4429)
restHighLevelClient.mgetAsync(multiGetRequest, RequestOptions.DEFAULT, new ActionListener<MultiGetResponse>() {
#Override
public void onResponse(MultiGetResponse response) {
for (var responseItem : response.getResponses()) {
try {
// simulating a null source
byte[] nullBytes = null;
customer = mapper.readValue(nullBytes, Customer.class);
} catch (IOException e) {
result.completeExceptionally(e);
}
}
result.complete(true);
}
#Override
public void onFailure(Exception ex) {
//the real problem!!!
//log.error("ex.cause", ex.getCause());
log.error("error with mget", ex);
result.completeExceptionally(ex);
}
});
Full source of repro, Full log file
Below is a portion of my code, that I've taken from here. I had to make few changes to make work with the new v6.X of Elasticsearch.
It runs without any errors and creates new documents BUT with empty field values. If I take the same JSON payload and PUT in Elasticsearch using Postman, the document gets indexed fine, with all fields populated. Please let me know if I am using the right Elasticsearch API methods, and using them right.
string strJsonMessage = #"
{
message : {
content: 'Test Message Content'
}
}";
ConnectionSettings connectionSettings = new ConnectionSettings(new Uri("xxx")).BasicAuthentication("xx", "xx");
ElasticClient client = new ElasticClient(connectionSettings);
JObject msg = JObject.Parse(strJsonMessage);
var result = await client.LowLevel.IndexAsync<BytesResponse>("events-2018.03.27", "event", PostData.Serializable(msg));
if (result.Success)
{
log.Info("Data successfully sent.");
log.Verbose(result.Body);
}
else
{
log.Error("Failed to send data.");
}
****OUTPUT TRACE:****
2018-03-27T18:37:18.961 [Info] Data successfully sent.
2018-03-27T18:37:18.961 [Verbose] {"_index":"events-2018.03.27","_type":"event","_id":"u9HPaGIBBm3ZG7GB5jM_","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":29,"_primary_term":1}
Querying elasticsearch for this document by its ID gives me this:
{
"_index": "events-2018.03.27",
"_type": "event",
"_id": "u9HPaGIBBm3ZG7GB5jM_",
"_version": 1,
"found": true,
"_source": {
"message": {
"content": []
}
}
}
In case someone comes across the same issue, I got the solution here.
I have an elasticsearch document
{
"_index": "testindex",
"_type": "testtype",
"_id": "doc1",
"_version": 1,
"found": true,
"_source": {
"array": [
"abc",
"def",
"ghi"
]
}
}
How do I reverse all the strings inside the array in the document? I tried using update api with the following script
temp = []; for(String item : ctx._source.array) temp << item.reverse(); ctx._source.array = temp;
Update api in java :
new org.elasticsearch.action.update.UpdateRequest(index, type, docId).script(script);
I am getting an exception like method call not allowed inside the script.
Bulk request failure, id: [doc4], message: ElasticsearchIllegalArgumentException[failed to execute script]; nested: GroovyScriptCompilationException[MultipleCompilationErrorsException[startup failed:
General error during canonicalization: Method calls not allowed on [java.lang.String]
try this
temp=[];
ctx._source.array.each{
it -> temp.add(it.reverse())
}
ctx._source.array = temp
I'm working with a json file containing the following:
{
"objects":[
{"object": 1,
"data":{
"name": "object 1",
"priority_threshold": "6000",
"email_threshold": "2000"
}},
{"object": 3,
"data":{
"name": "object 3",
"priority_threshold": "5000",
"email_threshold": "2000"
}},
{"object": 5,
"data":{
"name": "object 5",
"priority_threshold": "5000",
"email_threshold": "1000"
}},
{"object": 6,
"data": {
"name": "object 6",
"priority_threshold": "4000",
"email_threshold": "2000"
}
}
]}
the .json file is an embedded file and is being returned as a string.
Then from the string I am deserializing the object using System.Web.Script.Serialization.JavaScriptSerializer to do the following:
Dictionary<string, object> toConfigGraph = (Dictionary<string, object> toSerializer.DeserializeObject(psJsonString);
object[] toEventServiceConfig = (object[])toConfigGraph["objects"];
The problem running into is that I only want to return the data for a particular object using the object ID, but I'm unsure as to the best process. The I would like to implement a Linq solution, but as of now I'm not even sure if that will work since toConfigGraph["applications"] returns an array of objects based on the structure of the json.
Any suggestions would be greatly appreciated.
I'd rather NOT have to iterate through the object array.
Thanks.
Dictionary<string, object> toObj = (Dictionary<string, object>)toEventServiceConfig.Where(o => Int32.Parse(((Dictionary<string, object>)o)["object"].ToString()) == 1).First<object>();
Dictionary<string, object> toData = (Dictionary<string, object>)toObj["data"];