how the query URL is formed in elastic search and why we have a ? as below.Does this ? have any importance.Sample =” /kibana-int/dashboard/log-analytics-s?1431600189554
1431600189554 appears to be a timestamp (UNIX timestamp w/milliseconds), likely used to prevent caching of requests. It's not necessary for ElasticSearch (which will ignore it entirely), but it means the browser treats every request as a new URL and fetches new data.
The "?" states that the rest of the URL will be parameters. As an example it's often used with the parameter "pretty" in ElasticSearch, when you query a mapping (like http://localhost:8080//_mapping?pretty). The "pretty" argument forces the return to be formatted in a way that is eye-friendly.
As a side note, the "&" character is used to separate arguments. For exemple http://localhost:8080//?lastname=foo&firstname=bar
Related
I'm trying to use Spring's ServletUriComponentsBuilder to create paging next and prev links from the current request.
The problem I'm having is that the ServletUriComponentsBuilder.fromCurrentRequest() is not unencoding percent-encoded values like:
http://example.com/articles?page%5Bnumber%5D=2
The problem is uses could have called the page with unencoded square brackets like http://example.com/articles?page[number]=2 without any problems.
Spring Data is accepting both variants (both unencoded square brackets and encoded square brackets) in it's pageable argument resolver.
This to the fact that under water the Coyote web request get parameter is used which contains the unencoded param names.
Also Spring's #RequestParam("page[number]") accepts without any problem the encoded request like http://example.com/articles?page%5Bnumber%5D=2.
From the server side I always want to return percent encoded url's as per RFC 3986.
But there does not seem a way to this as the UriComponents query params might contain both encoded en uncoded names. Because to that, if I would call encode() on the builder the already encoded query params get encoded another time, but if would contain unencoded names toURI() will fail as an unencoded [ is not allowed.
Note that the url's might contain multiple query params besides paging, e.g. for filtering.
A request could come in like:
http://example.com/articles?filter[category]=food
And would return a response with a encoded next link like:
http://example.com/articles?page%5Bnumber%5D=2&filter%5Bcategory%5D=food
My workaround it to ignore ServletUriComponentsBuilder and simply get the request url and do so custom regexp replacing.
I know this is an older question and you also found a workaround. But did you try to use ServletUriComponentsBuilder's build() method?
Some kind of the following:
ServletUriComponentsBuilder.fromCurrentRequest().build().toUriString();
I had some issues when handling JSON Strings and this helped.
When doing a search operation in elasticsearch i want the metadata to be filtered out and return only "_source" in the response. I'm able to achieve the same through "search" in the following way:
out1 = es.search(index='index.com', filter_path=['hits.hits._id',
'hits.hits._source'])
But when i do the same with scan method it just returns an empty list:
out2 = helpers.scan(es, query, index='index.com',
doc_type='2016-07-27',filter_path= ['hits.hits._source'])
The problem may be with the way i'm processing the response of 'scan' method or with the way i'm passing the value to filter_path. To check the output i parse out2 to a list.
The scan helper currently doesn't allow passing extra parameters to the scroll API so your filter_path doesn't apply to it. It does, however, get applied to the initial search API call which is used to initiate the scan/scroll cycle. This means that the scroll_id is stripped from the response causing the entire operation to fail.
In your case even passing the filter_path parameter to the scroll API calls would cause the helper to fail because it would strip the scroll_id which is needed for this operation to work and also because the helper relies on the structure of the response.
My recommendation would be to use source filtering if you need to limit the size of the response or use smaller size parameter than the default 1000.
Hope this helps,
Honza
You could pass filter_path=['_scroll_id', '_shards', 'hits.hits._source'] to the scan helper to get it to work. Obviously that leaves some metadata in the response but it removes as much as possible while allowing the scroll to work. _shards is required because it is used internally by the scan helper.
For some reason my customers GSA always returns 10 results no matter what we send with the "num" parameter:
http://www.google.com/support/enterprise/static/gsa/docs/admin/72/gsa_doc_set/xml_reference/request_format.html#1076882
Is there a global configuration that we are missing that overrides what is sent as a parameter?
In GSA, there isn't a globally configuration which can override the num-parameter. If you use the num-parameter in the search query, it should return that number of results you assign in the parameter (with a maximum of 1000). So, you are sure that there are more than 10 results for the query you used?
The other thing I can think of, is that a proxy handles the search requests and responses. The parameters can then be controlled by this proxy, so whatever parameter you add in the URL is ignored since the proxy / the backend handles these.
We had an issue when trying to transform the parameters for the pagination and the sorting of an HTTP Request into a Pageable object in Spring Data/Spring MVC. Some of our sort parameters had an underscore character and when sorting on them we were getting an error. In the logs, these parameters were trimmed at the underscore character.
We have discovered that the org.springframework.data.web.PageableHandlerMethodArgumentResolver was using the underscore as default value for the qualifier delimiters. Now, we wonder if it's safe to modify this default value because we don't know what this qualifier delimiter is used for, and what consequence this would have.
The documentation of setQualifierDelimiter in the class PageableHandlerMethodArgumentResolver mentions:
The delimiter to be used between the qualifier and the actual page number
and size properties
So our questions: what is exactly the "qualifier delimiter" ? When is it used ?
Thanks for your help.
Searching into the documentation and code of Spring Data Web gave me the answer.
The qualifier delimiter is used in conjunction with the prefix. In the pageable arguments, it is the delimiter between the prefix and the page number argument, or between the prefix and the size argument. By default, the prefix is an empty string and, thus, the qualifier delimiter is not used. But, if you set the prefix, the qualifier delimiter will be used to parse the parameters of the http request.
Setting the prefix is useful if you have multiple tables with pagination in the same web page and these tables are updated via standard HTTP requests (not an AJAX query for each table separately). In that situation, the prefix allows to distinguish the http parameters for each table.
setQualifierDelimiter(String qualifierDelimiter):
the delimiter to be used between the qualifier and the actual page number and size properties.
setPrefix(String prefix):
configures a general prefix to be prepended to the page number and page size parameters.
I would like to run an elasticsearch query to find items within 10mi of a given point.
I know how to do it with a post, but I would like to use a get with everything in the uri.
I found the below example but it does not work.
http://localhost:9200/items/item/_search?{%22query%22:{%22filtered%22:{%22query%22:{%22match_all%22:{}},%22filter%22:{%22geo_distance%22:{%22distance%22:%220.1km%22,%22location%22:{%22lat%22:46.884106,%22lon%22:-71.377042}}}}}}
Any way to do this or am I stuck using a post?
The key is the source= parameter. Not to be confused with _source.
http://localhost:9200/items/item/_search?source={%22query%22:{%22filtered%22:{%22query%22:{%22match_all%22:{}},%22filter%22:{%22geo_distance%22:{%22distance%22:%220.1km%22,%22location%22:{%22lat%22:46.884106,%22lon%22:-71.377042}}}}}}
I had tried ?q= and a few other parameters listed on http://www.elasticsearch.org/guide/reference/api/search/uri-request/ with no luck (source is not listed).
I found http://www.elasticsearch.org/guide/reference/api/ and at the very bottom it says
request body in query string
For libraries that don’t accept a request body for non-POST requests,
you can pass the request body as the source query string parameter
instead.
So structure your query/filter request, set it all on one line and send it into the source parameter.
Do not use the q= parameter with source= or it will conflict and break the query, however I tried size= and from= and they work with source just fine.