I would like to do the Elasticsearch equivalent of the following SQL statement:
SELECT * FROM Users WHERE UserName NOT LIKE '%something%'
I don't care about efficiency or scoring... This only gets executed on occasion. I am using request body syntax.
Use a normal wildcard query, and negate it using a bool must_not query.
GET Users/_search
{
"query": {
"bool": {
"must_not": [{
"wildcard": {
"UserName": {
"value": "*something*"
}
}
}]
}
}
}
I'm not sure if your Users are your index in you Elastic - but this is the main idea anyway:
You could go with the regexp query by using Complement - "~" for negation:
GET Users/_search
{
"query": {
"regexp":{
"UserName": {
"value": ".*~(something).*"
}
}
}
}
For more useful reference, you can check here
P.S:
You will not get the best performance but it will do the job
Related
I am trying to create a elastic search query for one of my Library projects. I am trying to use regex but I do not get any result. I am trying to enter the following regex query.
GET /manifestation_v1/_search
{
"query": {
"regexp": {
"bibliographicInformation.title": {
"value": "python access*"
}
}
}
}
access is a wildcard so i want to create a query which takes as python access* not python access
Can anyone help me out who already has some experience in kibana?
you can try wildcard query
{
"query": {
"wildcard": {
"bibliographicInformation.title": {
"value": "saba safavi*"
}
}
}
}
You need to run regex query on keyword field and use .* instead of *
ex.
GET /manifestation_v1/_search
{
"query": {
"regexp": {
"bibliographicInformation.title": {
"value": "python access.*"
}
}
}
}
Regex is slower , you can also try prefix query
{
"query": {
"prefix": {
"bibliographicInformation.title": {
"value": "python access"
}
}
}
}
If field is of nested type then you need to use nested query
Update
For "text" type , field is stored as tokens. i.e
"python access" is stored as ["python","access"]. You query is trying to match "phython access*" with each of these tokens individually. You need to query against keyword field , which is stored as single value "phython access".
I run the query below on a large elastic search cluster. The cluster bcomes unresponsive
{
"size": 10000,
"query": {
"bool": {
"must": [
{
"regexp": {
"message": {
"value": ".*exception.*"
}
}
},
{
"bool": {
"should": [
{
"term": {
"beat.hostname": "ip-xxx-xx-xx-xx"
}
}
]
}
},
{
"range": {
"#timestamp": {
"lt": 1518459660000,
"format": "epoch_millis",
"gte": 1518459600000
}
}
}
]
}
}
}
When I remove the wildcarded .*exception.* and replace it with any non wildcarded string like xyz it returns fast. Though the query uses a wildcarded expression, it also looks for a small time range and a specific host. I would think this is a very simple query. Any reason why elasticsearch server can't handle this query? The cluster has 10 nodes and 20 TB of data.
See the documentation for Regexp Query. It clearly states the following:
Note: The performance of a regexp query heavily depends on the regular
expression chosen. Matching everything like .* is very slow
What would be ideal is to change the text analysis on the message field with a WordDelimiterTokenFilter and set split_on_case_change to true. Then something like NullPointerException will get indexed as three separate tokens [Null, Pointer, Exception]. This can help you search on exception without using a regex. Caveat is you need to reindex all your documents.
Another quick thing to try might be to keep your filter conditions on the hostname and timestamp in a filter context, which will prefilter documents before running your regexp query. This may be a short-term solution for you until you fix the text analysis.
new to ElasticSearch - start loving it. I am working on a Rails application (using elasticsearch-rails / elasticsearch-model).
I have two fields - both strings consisting of Tags.
about_me & about_you
Now I was to query the about_you of another user with the current users about_me.
At the same time, I wish to query the about_me of the other users with the about_you of the current user.
Does this make sense? Like two fields, two queries and each query is aimed at a particular field.
I just need a hint how this can be achieved in ES. For the sake of completeness, here is the part method I created in my rails model - it is incomplete:
def home_search(query_you, query_me)
search_definition =
{
query: {
multi_match: {
query: query_me,
fields: ['about_you']
}
..... SOMETHINGs MISSING HERE ..... ?
},
suggest: {
text: query,
about_me: {
term: {
size: 1,
field: :about_me
}
},
about_you: {
term: {
size: 1,
field: :about_you
}
}
}
}
self.class.__elasticsearch__.search(search_definition)
end
Any help, link or donations are welcome. Thank you!
I'm not sure I've understood your question but I can suggest two options:
First Use a bool query of type should and minimum_should_match=1. In this case you can write two queries for you'r searches. and If you want to distinguish between results you can pass a _name parameter in each query. something like this:
{
"query": {
"bool": {
"minimum_should_match": 1,
"should": [
{
"multi_match": {
"query": "query_me",
"fields": [
"about_you"
],
"_name": "about_you"
}
},
{
"multi_match": {
"query": "query_you",
"fields": [
"about_me"
],
"_name": "about_you"
}
}
]
}
}
}
By providing _name you can see which queries are hitted in your search result.
The second approach could be a _msearch query which in which you can pass multiple queries to the endpoint and get the results back.
Here are some useful links:
Bool Query
Named Queries
My data has a field localities which is an array of strings.
"localities": [
"Mayur Vihar Phase 1",
"Paschim Vihar",
"Rohini",
"",
"Laxmi Nagar",
"Vasant Vihar",
"Dwarka",
"Karol Bagh",
"Inderlok" ]
What query should I write to filter the documents by a specific locality such as "Rohini"?
A simple match query will be enough (if you don't know the mapping of your localities field).
POST <your index>/_search
{
"query": {
"match": {
"localities": "Rohini"
}
}
}
If the localities field is set as a string type and index as not_analyzed, the best way to query this is to use a term filter, wrapped in a filtered query (you can't use directly filters) :
POST <your index>/_search
{
"query": {
"filtered": {
"filter": {
"term": {
"localities": "Rohini"
}
}
}
}
}
If you doesn't need the score, the second solution is the way to go as filters doesn't compute score, are faster and cached.
Check the documentation for information about analysis which is a very important subject in ElasticSearch, heavily influencing the way you query.
POST /_search
{
"query": {
"match": {
"localities": "Rohini"
}
}
}
Or you can simply query:
GET /_search?q=localities:Rohini
This query is returning fine only one item "steve_jobs".
{
"query": {
"constant_score": {
"filter": {
"term": {
"name":"steve_jobs"
}
}
}
}
}
So, now I want to get all people with name prefix steve_. So I try this:
{
"query": {
"constant_score": {
"filter": {
"term": {
"name": "steve_"
}
}
}
}
}
This is returning nothing. Why?
I'm confused about when to use term query / term filter / terms filter / querystring query.
What you need is Prefix Query.
If you are indexing your document like so:
POST /testing_nested_query/class/
{
"name": "my name is steve_jobs"
}
And you are using the default analyzer, then the problem is that the term steve_jobs will be indexed as one term. So your Term Query will never be able to find any docs matching the term steve as there is no term like in the index. Prefix Query helps you solve your problem by searching for a prefix in all the indexed terms.
You can solve the same problem by making your custom analyzers (read this and this) so that steve_jobs is stored as steve and jobs.