I am using this expression to update a nested document:
curl -XPOST 'localhost:9200/event/docs/cPd4cfqGTe2Hw9sq0qs_NQ/_update' -d '{
"script": "foreach (item : ctx._source.to) { item['read'] = true }"
}'
But it always says classcastexception boolean cannot be casted to String. I tried putting the true in a param, tried 'T', 'true', 'TRUE', TRUE, 1.
Running out of ideas.
The document sample:
{
"prop":"test"
"to": [{"id": "1", "read":false},
{"id":"2","read": true}]
}
I also tried changing "id" just to test, and it's telling me I can't cast HashMap into string
curl -XPOST 'localhost:9200/event/docs/cPd4cfqGTe2Hw9sq0qs_NQ/_update' -d '{
"script": "foreach (item : ctx._source.to) { item['id'] = '3' }",
}'
The script sent to UPDATE API is not right.
I have corrected your script -
curl -XPOST 'http://localhost:9200/vm/vm/vm' -d '{
"prop":"test",
"to": [{"id": "1", "read":false},
{"id":"2","read": true}]
}'
And on updating , use the below script -
curl -XPOST 'localhost:9200/vm/vm/vm/_update' -d '{
"script": "for (item in ctx._source.to) { item.id = 4 }"
}'
Related
I have a bunch of documents in Elasticsearch that don't have a geo_point attribute.
Now I want to add it to all of them.
With some research I found the command bellow but was originally used to update a string attribute.
curl -XPOST "http://localhost:9200/products/_update_by_query" -H 'Content-Type: application/json' -d'
{
"script": {
"source": "ctx._source.location = {'lat': 0.0, 'lon':0.0}",
"lang": "painless"
},
"query": {
"match_all": {}
}
}'
Thought I'd just replace the string with geo_point but it gives me this error:
{
"error":{
"root_cause":[{
"type":"parse_exception",
"reason":"expected one of [inline], [file] or [stored] fields, but found none"
}],
"type":"parse_exception",
"reason":"expected one of [inline], [file] or [stored] fields, but found none"
},
"status":400
}
I appreciate any help.
Good job so far!
It looks like you're running an older version of ES. Try the command below which simply replaces source by inline as it was the norm in older versions:
curl -XPOST "http://localhost:9200/products/_update_by_query" -H 'Content-Type: application/json' -d'
{
"script": {
"inline": "ctx._source.location = ['lat': 0.0, 'lon':0.0]",
"lang": "painless"
},
"query": {
"match_all": {}
}
}'
Note, however, that if your location field is already of type text or string you cannot change it to geo_point with this command. You'll need to either create a new field named differently than location and of type geo_point or create a new index with the proper mapping for the location field.
Edit: If the above doesn't work, try replacing single quote ' with \" like so
curl -XPOST "http://localhost:9200/products/_update_by_query" -H 'Content-Type: application/json' -d'
{
"script": {
"inline": "ctx._source.location = [\"lat\": 0.0, \"lon\":0.0]",
"lang": "painless"
},
"query": {
"match_all": {}
}
}'
I have the following curl request
curl -H "Content-Type: application/json" -X POST http://localhost:9200/_reindex\?wait_for_completion\=true -d '{"source": {"index": "analytics-prod-2019.12.30", "size":1000 }, "dest": {"index": "analytics-prod-2019.12"}, "conflicts": "proceed", "script": { "lang": "painless","source: """ctx._source.index = ctx._index; def eventData = ctx._source["event.data"]; if(eventData != null) { eventData.remove("realmDb.size"); eventData.remove("realmDb.format"); eventData.remove("realmDb.contents"); }""" } }'
but this fails with the following error:
{"error":{"root_cause":[{"type":"x_content_parse_exception","reason":"[1:166] [script] failed to parse object"}],"type":"x_content_parse_exception","reason":"[1:166] [reindex] failed to parse field [script]","caused_by":{"type":"x_content_parse_exception","reason":"[1:166] [script] failed to parse object","caused_by":{"type":"json_parse_exception","reason":"Unexpected character ('\"' (code 34)): was expecting a colon to separate field name and value\n at [Source: org.elasticsearch.common.bytes.BytesReference$MarkSupportingStreamInputWrapper#51c48433; line: 1, column: 177]"}}},"status":400}
if i remove the script field from the request this works just fine:
curl -H "Content-Type: application/json" -X POST http://localhost:9200/_reindex\?wait_for_completion\=true -d '{"source":{"index":"analytics-prod-2019.12.30","size":1000},"dest":{"index":"test-index"},"conflicts":"proceed"}}'
using the kibana UI works fine.
what is the correct way to run this in curl?
Use a single " to surround your script value and \u0027 to escape in your Painless script.
curl -H "Content-Type: application/json" -X POST http://localhost:9200/_reindex\?wait_for_completion\=true -d '
{
"source": {
"index": "analytics-prod-2019.12.30",
"size": 1000
},
"dest": {
"index": "analytics-prod-2019.12"
},
"conflicts": "proceed",
"script": {
"lang": "painless",
"source": "ctx._source.index = ctx._index; def eventData = ctx._source[\u0027event.data\u0027]; if(eventData != null) { eventData.remove(\u0027realmDb.size\u0027); eventData.remove(\u0027realmDb.format\u0027); eventData.remove(\u0027realmDb.contents\u0027);"
}
}
'
You can also see an example of this here, click on the Copy as cURL link and review the example in that format.
Your source was missing a double quote:
Corrected:
curl -H "Content-Type: application/json" \
-X POST http://localhost:9200/_reindex\?wait_for_completion\=true \
-d '{"source": {"index": "analytics-prod-2019.12.30", "size":1000 }, "dest": {"index": "analytics-prod-2019.12"}, "conflicts": "proceed", "script": { "lang": "painless","source": "ctx._source.index = ctx._index; def eventData = ctx._source[\"event.data\"]; if (eventData != null) { eventData.remove(\"realmDb.size\"); eventData.remove(\"realmDb.format\"); eventData.remove(\"realmDb.contents\"); }" } }'
You can either use single quotes like #Zsolt pointed out but even Kibana itself, when clicking "Copy as cURL", uses escaped double quotes.
curl -XPOST "http://elasticsearch:9200/_reindex?requests_per_second=115&wait_for_completion=true" -H 'Content-Type: application/json' -d'
{
"source": {
"index": "analytics-prod-2019.12.30",
"size": 1000
},
"dest": {
"index": "analytics-prod-2019.12"
},
"script": {
"lang": "painless",
"source": " ctx._source.index = ctx._index;\n def eventData = ctx._source[\"event.data\"];\n if (eventData != null) {\n eventData.remove(\"realmDb.size\");\n eventData.remove(\"realmDb.format\");\n eventData.remove(\"realmDb.contents\");\n }"
}
}'
had to escape \"
I create a simple mapping:
curl -XPUT 'localhost:9200/ficherosindex?pretty=true' -d '{
"mappings": {
"items": {
"dynamic": "strict",
"properties" : {
"title" : { "type": "string" },
"body" : { "type": "string" },
"attachments" : { "type": "attachment" }
}}}}'
I make PUT the title and the body, leaving attachments empty.
curl -XPUT 'localhost:9200/ficherosindex/items/1' -d '{
"title": "This is a test title",
"body" : "This is the body of the java",
"attachments" : ""
}'
And then I make the following script to update the attachments fields with the content of the MY_PDF.pdf file, converting it to base64.
#!/bin/sh
coded=`cat MY_PDF.pdf | perl -MMIME::Base64 -ne 'print encode_base64($_)'`
curl -X POST 'localhost:9200/ficherosindex/items/1/_update?pretty=true' -d '{
"doc" : {
"attachments" : \"${coded}\"
}}'
When I run the script I'm getting the following error:
{
"error" : {
"root_cause" : [ {
"type" : "json_parse_exception",
"reason" : "Unexpected character ('\\' (code 92)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')\n at [Source: [B#6c8caddf; line: 3, column: 30]"
} ],
"type" : "json_parse_exception",
"reason" : "Unexpected character ('\\' (code 92)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')\n at [Source: [B#6c8caddf; line: 3, column: 30]"
},
"status" : 500
}
What I'm doing wrong? Maybe I've to change the following line?
{
"doc" : {
"attachments" : \"${coded}\"
}}'
I also tried this solution with no luck. I have to mantain the order I'm showing. First create the item without the attachments and then use the _update to append the content of the .PDF to it.
Thanks in advance
Something like this should do:
#!/bin/sh
coded=`cat MY_PDF.pdf | perl -MMIME::Base64 -ne 'print encode_base64($_)'`
curl -XPOST 'localhost:9200/ficherosindex/items/1/_update?pretty=true' -H "Content-Type: application/json" -d #- <<CURL_DATA
{ "doc": { "attachments": "$coded" }}
CURL_DATA
I'm trying to build an elasticsearch query to return documents for times between midnight and the current time of day, for all dates. For example, if I run the query at 09:00:00, then the query will return any document with a timestamp between midnight and 09:00:00 regardless of the date.
Here's an example dataset:
curl -XPUT localhost:9200/test/dt/_mapping -d '{"dt" : {"properties" : {"created_at" : {"type" : "date", "format": "YYYY-MM-DD HH:mm:ss" }}}}'
curl -XPOST localhost:9200/test/dt/1 -d '{ "created_at": "2014-10-09 07:00:00" }'
curl -XPOST localhost:9200/test/dt/2 -d '{ "created_at": "2014-10-09 14:00:00" }'
curl -XPOST localhost:9200/test/dt/3 -d '{ "created_at": "2014-10-08 08:00:00" }'
curl -XPOST localhost:9200/test/dt/4 -d '{ "created_at": "2014-10-08 15:00:00" }'
curl -XPOST localhost:9200/test/dt/5 -d '{ "created_at": "2014-10-07 09:00:00" }'
curl -XPOST localhost:9200/test/dt/6 -d '{ "created_at": "2014-10-07 16:00:00" }'
and an example filter:
curl -XPOST localhost:9200/test/dt/_search?pretty -d '{
"query": {
"filtered" : {
"filter" : {
"bool": {
"must": [
{
"script" : {
"script" : "doc[\"created_at\"].date.getMinuteOfDay() < 600"
}
}
]
}
}
}
}
}'
where 600 is a static parameter that I want to replace with a dynamic minutes parameter depending on the time of day when the query is run.
My best attempt is to use the getMinuteOfDay() method to filter each doc, but my problem is how to get getMinuteOfDay() for the current time. I've tried variations using time() instead of the hard-coded parameter 600 in the above query, but can't figure it out.
Any ideas?
This is pretty late but this can be done with DateTime.now(), you should change your query to
GET test/dt/_search
{
"query": {
"filtered": {
"filter": {
"bool": {
"must": [
{
"script": {
"script": "doc[\"created_at\"].date.getMinuteOfDay() < DateTime.now().getMinuteOfDay()"
}
}
]
}
}
}
}
}
Hope this helps!
I have created a mapping for a tweetb type in a twitter index:
curl -XPUT http://www.mydomain:9200/twitter/tweetb/_mapping -d '{
"twitter": {
"mappings": {
"tweetb": {
"properties": {
"message": {
"type": "string",
"null_value": "NA"
}
}
}
}
}
}'
Then, I put one document:
curl -XPUT http://www.mydomain.com:9200/twitter/tweetb/1 -d '{"message": null}'
Then, I tried to get the inserted doc back:
curl -XGET http://www.mydomain:9200/twitter/tweetb/1
And that returned:
{
"_index": "twitter",
"_type": "tweetb",
"_id": "1",
"_version": 2,
"found" : true,
"_source" : { "message": null }
}
I was expecting "message" : "NA" in the _source field. However, it looks like "null_value" isn't working. Am I missing something?
The "null_value" field mapping does not change the value stored, rather it changes the value that is used in searches.
If you try searching for your "message" using "NA", then it should appear in the results:
curl -XPOST http://www.mydomain.com:9200/twitter/tweetb/_search -d '{
"query" : {
"match" : { "message" : "NA" }
}
}'
Of interest, it should respond with the actual value being null. Now, if you add a new document whose raw value is literally "NA" and perform the search, then you should see both results returned for the above query--one with a value and the other with null defined.
Perhaps of similar interest, this works for other queries as well based on how it is indexed, which is why a lowercase n.* matches, but N.* semi-surprisingly will not match:
curl -XPOST http://www.mydomain.com:9200/twitter/tweetb/_search -d '{
"query" : {
"regexp" : { "message" : "n.*" }
}
}'