Recently I use Kibana console to update a field of an existing document in ElasticSearch, I saw this error while using the PUT method for this, which confuse me a lot.
{
"error": "Incorrect HTTP method for uri [/product/_doc/1/_update] and method [PUT], allowed: [POST]",
"status": 405
}
The query I used is
PUT /product/_doc/1/_update
{
"doc": {"price": 95, "tags": ["Elasticsearch"]}
}
Which I believe should be idempotent. Could someone help me understand why only POST method can be used here? My thinking is PUT method is for idempotent operations so to me, PUT should be the only candidate rather than POST.
Depending on how you read the semantics of HTTP a PUT would replace a resource entirely and you would need a PATCH for an update (which isn't supported by Elasticsearch). Also the _update endpoint will either accept doc or script and the later one isn't necessarily idempotent — for example doing a scripted upsert.
Generally Elasticsearch is as RESTful as possible, but will make pragmatic choices where required.
Related
I'm new to the elastic-cloud interface. It allows to chooose operations get, post, put and del. I'm trying to submit queries, but I don't know the precise syntax. For instance:
tweet/_search?q=something
works, but:
tweet/_search?q={ "match_all": {} }
does not, returning a parser error. I have tried with double quotes, but it seems that then it searches for the query as a string.
The preferred way to test the search APIs are using the POST method, GET API in some case, gives even incorrect search results as it ignores the search and brings the top 10 search results for match_all query.
Elasticsearch supports both methods GET and POST to search but using the GET method which has payload information isn't common on modern app-severs, although Elasticsearch implemented it requires carefully crafting your queries.
Still, if you want to use the GET API, then for complex queries its better to send it as part of request body, I know it sounds weird to send a body to GET request but it works 😀 .
Like at this elastic get query I see below example where per my understanding query_string is passed under request body in GET request . Is n't it ? But I believe we can't pass request body with GET request then how come this example is true ?
GET /_search
{
"query": {
"query_string" : {
"default_field" : "content",
"query" : "this AND that OR thus"
}
}
}
In fact when I used the option COPY as CURL from the stated link I see below copied text
curl -X GET "localhost:9200/_search" -H 'Content-Type: application/json' -d'
{
"query": {
"query_string" : {
"default_field" : "content",
"query" : "this AND that OR thus"
}
}
}
'
Am I missing anything here or something wrong in example? In fact I do not see the way to send the request body under Postman tool.
The fact is that you can send a GET request with a body. The current HTTP standard rfc7231 (obsoletes rfc2616 and updates rfc2817) does not strictly define what must happen to a GET request with a body. The previous versions were different in this regard. For that reason, some HTTP servers allow it, but some others don't, I'm afraid. This case is mentioned in the latest standard as follows:
A payload within a GET request message has no defined semantics;
sending a payload body on a GET request might cause some existing
implementations to reject the request.
In terms of Elasticsearch, using using GET for a search request is a design decision. They feel it makes more sense semantically. Because it represents a data retrieving action better than the POST verb.
On the other hand, as mentioned above, a GET request with a body is not supported universally. That's why Postman does not allow you to do so, although Kibana > Dev Tool does it by using cURL. Therefore, the Elasticsearch search API also supports POST requests to search and retrieve information. So, when you cannot make a GET request with a body, you can obtain exactly the same result by making a POST request.
This is actually very interested question. In fact, a lot of HTTP clients aren’t supporting GET requests with body (i just recently face, that iOS client in Cocoa isn’t able to do so).
I also had a lot of discussions with my colleagues - to me after using Elasticsearch for a long time GET with a body sounds like a perfectly fine HTTP request, however some may argue, that GET shouldn’t go with body at all according to HTTP standard. However, I will leave this discussion out of this answer.
In general this leads to a situation, that if you’re using client which not supporting GET, you could either change it to POST or switch to something else - I used to use cURL all the time or Kibana Dev Tools if I needed to construct complex query on the fly
So I can successfully do request like:
localhost:9200/filebeat-*/_count?filter_path=-_shards
{"query": {
"match_phrase" : {
"message" : "o hohoho"
}
}}
How I can move filter_path=-_shards into the request body to make it work?
According to the documentation code, still not possible in Elasticsearch 6.2:
All REST APIs accept a filter_path parameter that can be used to
reduce the response returned by Elasticsearch
and it's impossible to include it into the request body, that's just not supported (to be honest, I'm not sure if it will ever be supported).
However, for some scenarios, you could limit response returned by Elasticsearch by using source filtering (unfortunately, it's only applicable to returned fields of documents)
I'm using ES 5.1.2 and I want to do some simple authentication key lookup for every /_search request.
I don't find a very detailed plugin development guide on elastic.co, so far the only document I found is this http://david.pilato.fr/blog/2016/10/19/adding-a-new-rest-endpoint-to-elasticsearch-updated-for-ga/, but it is about create another endpoint.
I found search-guard https://github.com/floragunncom/search-guard and from source code it feels like I can create my own plugin extends Plugin implements ActionPlugin, but then I'm stuck and don't know where to go.
From the source code I know I can add my own ActionFilter and add it into action chain in Plugin, such that all request go thru /_search endpoint will also go thru my ActionFilter. But I don't have a complete list of possible action, which might be indices:data/read/search (search) or indices:admin/delete (delete index). There are too many to use try and error.
Another thing is, in ActionFilter, how do I get POST request payload from a Request object? When in /_search request, I got SearchRequest, but it doesn't have http request headers from browser.
== update ==
Found I could use use stack trace to get the invoke history, so for ActionFilter the call stack is like
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:171)
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:145)
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:87)
at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:75)
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:64)
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:403)
at org.elasticsearch.client.support.AbstractClient.search(AbstractClient.java:530)
at org.elasticsearch.rest.action.search.RestSearchAction.lambda$prepareRequest$0(RestSearchAction.java:83)
at org.elasticsearch.rest.action.search.RestSearchAction$$Lambda$1405/1241306571.accept(Unknown Source)
at org.elasticsearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:82)
in RestSearchAction#prepareRequest, ES use parseSearchRequest and convert RestRequest data to SearchRequest, which means I can't get RestRequest in my ActionFitler. Should there be another way to pass this data? Because I want to extend an existed /_search not add another endpoint, so I think I should not create any more RestHandler
I found a temporary solution and source code is here.
Long thing short, when implement your plugin as RestHandler (usually with new endpoints), you can skip registerHandler and do registerFilter only. I implement my logic in RestFilter. This works because in ElasticSearch source code, when RestController gets a new request, it will check if you have any RestFilter, if you do then it will go through whole chain of filters, then dispatch to a particular handler based on request URI.
In search guard 5 there seems to be a way to directly register a RestFilter in RestController without creating a RestHandler, but I don't understand the whole flow so I didn't use it.
My main references:
http://david.pilato.fr/blog/2016/10/19/adding-a-new-rest-endpoint-to-elasticsearch-updated-for-ga/
https://github.com/floragunncom/search-guard/tree/es-5.1.2
The official reference states that one can send _search requests also through POST instead of GET because not all clients support sending bodys with GET (see https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-body.html). You can then insert the query parameters from the URL also as JSON directly in the body.
Now I wonder: is this true for all GET requests that Elasticsearch offers that need query parameters?
For example, the _stat endpoint (https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-stats.html) is documented as a GET request (which makes sense), but supports URI parameters. Is it safe to use POST in this case as well and pass the parameters in the body using JSON?
No, the _search endpoint is one of a few special cases. If you look at the source code for the _stats endpoint in RestIndicesStatsAction.java, you can see that only the GET HTTP method is supported.
Using the POST method usually makes sense only when the payload to be sent can be substantially big, which is not the case for the few parameters such as the ones accepted by the _stats endpoint. In that case, sending those parameters in the query string is usually more than sufficient.