I have keys that I want to setup routing for in elasticsearch. I have some keys that have special characters (eg: &, ") - how can I setup routing for such keys. Examples:
"A&B"
You simply need to URL-encode the routing parameter value since & is a reserved character in URLs that introduces a new parameter. In your case, the routing parameter has the value "A and then another parameter called B" is introduced, which ES doesn't know about, hence why it complains.
/partb-2017-06-*/_search?routing="A&B"
^
|
this introduces a new parameter
The right way to do it is to URL-encode "A&B" into %22A%26B%22 and ES will not complain anymore.
/partb-2017-06-*/_search?routing=%22A%26B%22
Related
I am trying to get organizations resource for Name = 'G&A' using the following API
https://xxx/hcmCoreSetupApi/resources/11.13.18.02/organizations/?onlyData=true&q=Name='G&A'
But getting an error "URL request parameter A' cannot be used in this context."
Thank you for the help in advance
The ampersand & character is used as a separator between query parameters. If you want to pass an ampersand as part of a query parameter's value then use the equivalent hexidecimal code %26 instead of &:
https://xxx/hcmCoreSetupApi/resources/11.13.18.02/organizations/?onlyData=true&q=Name='G%26A'
However, that is still invalid as you have too many equals = characters in that string; so did you intend to have three parameters named onlyData, q and Name? Then you would encode them like this:
https://xxx/hcmCoreSetupApi/resources/11.13.18.02/organizations/?onlyData=true&q=&Name='G%26A'
Or, if you really had intended to have two parameters named onlyData, q=Name then you would need to encode the equals = character in the parameter name as well:
https://xxx/hcmCoreSetupApi/resources/11.13.18.02/organizations/?onlyData=true&q%3DName='G%26A'
Or, if Name= is part of the value not the key then:
https://xxx/hcmCoreSetupApi/resources/11.13.18.02/organizations/?onlyData=true&q=Name%3D'G%26A'
Was attempting to search our directory based on an attribute whose value is a DN. However, our user RDNs are of the form CN=Surname, GivenName, which requires that the comma be quoted in the full DN. But given an attribute like manager whose value is the DN of another user, I was unable to search for all users having specific manager. I tried (manager=CN=Surname\, GivenName,CN=users,DC=mydomain,DC=com), but got a syntax error "Bad search filter". I tried various options for quoting the DN, but all either gave me a syntax error or failed to match any objects. What am I doing wrong?
(Note that if I were looking for user objects directly, I could search for simply (CN=Surname, GivenName), with no quoting required, but I was searching for users having a specific manager. The comma-containing attribute value only becomes a problem when part of a Distinguished Name.)
The problem is that quoting the comma in the Common Name is not for the benefit of the filter parser, but for the benefit of the DN parser; the attribute value passed to that by the filter has to literally contain the backslash character. Unfortunately, the backslash is also (differently) special in LDAP filters, thus the syntax errors.
The solution is simple, but it isn't as obvious as doubling the backslash; backslash in LDAP filters works like % in URIs, so you have to use a literal backslash followed by the 2-digit hexadecimal code point for a backslash:
(manager=CN=Surname\5c, Givenname,OU=org,DC=mydomain,DC=com)
It turns out there's an example of this specific use case at the very bottom of https://docs.oracle.com/cd/E19424-01/820-4811/gdxpo/index.html#6ng8i269q.
I'm looking over Section 3.4 of RFC 3986 trying to understand what constitutes a valid URI query parameter key, but I'm not seeing a clear answer.
The reason I'm asking is because I'm writing a Ruby class that composes a URI with query parameters. When a new parameter is added I want to validate the key. Based on experience, it seems like the key will be invalid if it requires any escaping.
I should also say that I plan to validate the key. I'm not sure how to go about validating this data either, but I do know that in all cases I should escape this value.
Advice is appreciated. Advice in the context of how validation might already be possible through say a Ruby Gem would also be a plus.
I could well be wrong, but that spec seems to say that anything following '?' or '#' is valid as long. I wonder if you should be looking more at the spec for 'application/x-www-form-urlencoded' (ie. the key/value pairs we're all used to)?
http://www.w3.org/TR/html401/interact/forms.html#h-17.13.4.1
This is the default content type. Forms submitted with this content
type must be encoded as follows:
Control names and values are escaped. Space characters are replaced by +', and then reserved characters are escaped as described in [RFC1738], section 2.2: Non-alphanumeric characters are replaced by %HH', a percent sign and two hexadecimal digits representing the ASCII code of the character. Line breaks are represented as "CR LF" pairs (i.e., `%0D%0A').
The control names/values are listed in the order they appear in the document. The name is separated from the value by =' and name/value pairs are separated from each other by &'.
I don't believe key=value is part of the RFC, it's a convention that has emerged. Wikipedia suggests this is an 'W3C recommendation'.
Seems like some good stuff to be found searching on the application/x-www-form-urlencoded content type.
http://www.w3.org/TR/REC-html40/interact/forms.html#form-data-set
How do I specify AND operation in URI based query? I'm looking for something like:
http://localhost:9200/_search?q="profiletype:student AND username:s*"
For URI search in ElasticSearch, you can use either the AND operator profiletype:student AND username:s, as you say but without quotes:
_search?q=profiletype:student AND username:s*
You can also set the default operator to AND
_search?q=profiletype:student username:s*&default_operator=AND
Or you can use the + operator for terms that must be present, i.e. one would use +profiletype:student +username:s as query string. This doesn't work without URL encoding, though. In URL encoding + is %2Band space is %20, therefore the alternative would be
_search?q=%2Bprofiletype:student%20%2Busername:s*
According to documentation, it should work as you described it. See http://www.elasticsearch.org/guide/reference/query-dsl/query-string-query.html
That said, you can also use the following:
http://localhost:9200/_search?q="+profiletype:student +username:s*"
You can try bellow line.
http://localhost:9200/_search?q=profiletype:student%20AND%20username:s*
http://localhost:9200/_search?q=profiletype:student AND username:s*
i had to combine the default operator and + sign to make it work
curl -XGET 'localhost:9200/apm/inventory/_search?default_operator=AND&q=tenant_id:t2+_all:node_1'
Which characters are and are not allowed in a key (i.e. example in example: "Value") in YAML?
According to the YAML 1.2 specification simply advises using printable characters with explicit control characters being excluded (see here):
In constructing key names, characters the YAML spec. uses to denote syntax or special meaning need to be avoided (e.g. # denotes comment, > denotes folding, - denotes list, etc.).
Essentially, you are left to the relative coding conventions (restrictions) by whatever code (parser/tool implementation) that needs to consume your YAML document. The more you stick with alphanumerics the better; it has simply been our experience that the underscore has worked with most tooling we have encountered.
It has been a shared practice with others we work with to convert the period character . to an underscore character _ when mapping namespace syntax that uses periods to YAML. Some people have similarly used hyphens successfully, but we have seen it misconstrued in some implementations.
Any character (if properly quoted by either single quotes 'example' or double quotes "example"). Please be aware that the key does not have to be a scalar ('example'). It can be a list or a map.