I am trying to get the fielddata from a not_analyzed field in Multi Get query. It is working fine in _search queries.
This is what I've tried with no luck:
curl -XGET "http://es:9200/articles/article/_mget/?pretty&fielddata_fields=url" -d '{"ids" : ["5763197951"]}'
curl -XGET "http://es:9200/articles/article/_mget/?pretty" -d '{"fielddata_fields": ["url"], "ids" : ["5763197951"]}'
curl -XGET "http://es:9200/articles/article/_mget/?pretty" -d '{"docs" : [{"_id" : "5763197951", "fielddata_fields": ["url"]}]}'
It looks like fielddata_fields is completely ignored, since I always get this result:
{
"docs" : [ {
"_index" : "articles",
"_type" : "article",
"_id" : "5763197951",
"_version" : 1,
"found" : true
} ]
}
I'm running ES version 1.4.4 with JVM: 1.8.0_31
Edit: I just tried the above with a test database running ES 2.2.2 with the same results...
Related
I'm a beginner with Elasticsearch and am following an "Essential Training" in LinkedIn Learning. I'm trying to follow with bulk loading API and the instructor is using Linux, I'm on Windows. He created a text file to read in with data using "VI". I just created a text file and pasted the data and removed the ".txt". The contents of the file, called reqs, is this:
{
"index":{
"_index":"my-test",
"_type":"my-type",
"_id":"1"
}
}{
"col1":"val1"
}{
"index":{
"_index":"my-test",
"_type":"my-type",
"_id":"2"
}
}{
"col1":"val2"
}{
"index":{
"_index":"my-test",
"_type":"my-type",
"_id":"3"
}
}{
"col1":"val3"
}
I've tried saving it with a carriage return (new line) after the last line and without. I saved this into my elasticsearch folder (C:\elasticsearch-7.12.0) which is the same directory I'm running the following command from:
c:\elasticsearch-7.12.0>curl -s -H "Content-Type: application/x-ndjson" -XPOST localhost:9200/_bulk --data-binary "#reqs"; echo
When I do this, I'm getting the following error:
{"error":{"root_cause":[{"type":"parse_exception","reason":"request body is required"}],"type":"parse_exception","reason":"request body is required"},"status":400}
Use this below curl command
curl -H 'Content-Type: application/x-ndjson' -XPOST 'localhost:9200/index-name/_bulk?pretty' --data-binary #reqs.json
reqs.json should look like this
{"index" : {"_index" : "my-test", "_type" : "my-type", "_id" : "1"}}
{"col1" : "val1"}
{"index" : {"_index" : "my-test", "_type" : "my-type", "_id" : "2"}}
{"col1" : "val2"}
{"index" : {"_index" : "my-test", "_type" : "my-type", "_id" : "3"}}
{"col1" : "val3"}
I am using an old version on ElasticSearch - 1.5.
Problem: I need to delete a lot of documents, like few hundred thousands up to few millions. I have all the info about the records, including it's _ids - so array of _ids is what I want to use.
Scale problem: I had this deletion in the loop before, but ES is inconsistent when performing a lot of subsequent operations in a high speed. Thus I decided to look for a bulk delete.
I am trying to make use of delete by query API.
Docs states:
curl -XDELETE 'http://localhost:9200/twitter/tweet/_query' -d '{
"query" : {
"term" : { "user" : "kimchy" }
}
}
'
What I'm doing:
curl -XDELETE 'http://localhost:9200/my_index/logs/_query' -d '{
"query" : {
"terms" : { "_id" : ["AVTD6fhLAn35BG25xbZz", "AVTD6fhLAn35BG25xbaC"] }
}
}
'
The response is:
{
"found":false,
"_index":"my_index",
"_type":"logs",
"_id":"_query",
"_version":1,
"_shards":{"total":2, "successful":1, "failed":0}
}
And it does not remove any of documents. How do I make it work and actually delete these records?
Not sure about the delete_by_query API in elasticsearch 1.5. Seems to me that elasticsearch is unable to understand your query as it is looking for "_id": "_query" (as evident from the response you posted).
What you can do is, use the Bulk API as documented here:
https://www.elastic.co/guide/en/elasticsearch/reference/1.5/docs-bulk.html
As in the example in the doc page, you can do:
curl -s -XPOST localhost:9200/_bulk --data-binary #requests; echo
{ "delete" : { "_index" : "test", "_type" : "type1", "_id" : "2" } }
{ "delete" : { "_index" : "test", "_type" : "type1", "_id" : "3" } }
...
You need to make a file by any name ("requests" here) and add individual delete requests, each separated by a newline character.
I had some data already residing in ElasticSearch. I added a term {"oncall" :"true"} to some document based on some conditions by python post requests.When I am going to Kibana and trying to search it on discover page. I am not getting any results. But when I am doing following curl request I am getting the results.
curl -XPOST "http://localhost:9200/logstash*/logs/_search?pretty" -d '
{
"query" : {
"term" : {"oncall" :"true"}
}
}'
results
"hits" : {
"total" : 47,
"max_score" : 12.706658,
"hits" : [ {
"_index" : "logstash-2015.10.20",
"_type" : "logs",
.......
.......
I want to ask that why i am not able to see the results in kibana and what setting do i need to change.
the query which I am writing on discover-page text box is
oncall:true #this is giving me no results
Is there a way to retrieve from ElasticSearch information on when a specific index was last updated?
My goal is to be able to tell when it was the last time that any documents were inserted/updated/deleted in the index. If this is not possible, is there something I can add in my index modification requests that will provide this information later on?
You can get the modification time from the _timestamp
To make it easier to return the timestamp you can set up Elasticsearch to store it:
curl -XPUT "http://localhost:9200/myindex/mytype/_mapping" -d'
{
"mytype": {
"_timestamp": {
"enabled": "true",
"store": "yes"
}
}
}'
If I insert a document and then query on it I get the timestamp:
curl -XGET 'http://localhost:9200/myindex/mytype/_search?pretty' -d '{
> fields : ["_timestamp"],
> "query": {
> "query_string": { "query":"*"}
> }
> }'
{
"took" : 7,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"failed" : 0
},
"hits" : {
"total" : 1,
"max_score" : 1.0,
"hits" : [ {
"_index" : "myindex",
"_type" : "mytype",
"_id" : "1",
"_score" : 1.0,
"fields" : {
"_timestamp" : 1417599223918
}
} ]
}
}
updating the existing document:
curl -XPOST "http://localhost:9200/myindex/mytype/1/_update" -d'
{
"doc" : {
"field1": "data",
"field2": "more data"
},
"doc_as_upsert" : true
}'
Re-running the previous query shows me an updated timestamp:
"fields" : {
"_timestamp" : 1417599620167
}
I don't know if there are people who are looking for an equivalent, but here is a workaround using shards stats for > Elasticsearch 5 users:
curl XGET http://localhost:9200/_stats?level=shards
As you'll see, you have some informations per indices, commits and/or flushs that you might use to see if the indice changed (or not).
I hope it will help someone.
Just looked into a solution for this problem. Recent Elasticsearch versions have a <index>/_recovery API.
This returns a list of shards and a field called stop_time_in_millis which looks like it is a timestamp for the last write to that shard.
Lets say I'm updating user data
curl -XPOST 'localhost:9200/test/type1/1/_update' -d '{
"doc" : {
"name" : "new_name"
},
"fields": ["_source"]
}'
Heres an example of what I'm getting back when I perform an update
{
"_index" : "test",
"_type" : "type1",
"_id" : "1",
"_version" : 4
}
How do I perform an update that returns the given document post update?
The documentation is a little misleading with regards to returning fields when performing an Elasticsearch update. It actually uses the same approach that the Index api uses, passing the parameter on the url, not as a field in the update.
In your case you would submit:
curl -XPOST 'localhost:9200/test/type1/1/_update?fields=_source' -d '{
"doc" : {
"name" : "new_name"
}
}'
In my testing in Elasticsearch 1.2.1 it returns something like this:
{
"_index":"test",
"_type":"testtype",
"_id":"1","_version":9,
"get": {
"found":true,
"_source": {
"user":"john",
"body":"testing update and return fields",
"name":"new_name"
}
}
}
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-index_.html
http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-update.html