I've looked at the other examples on StackOverflow and on the Elastic site, but I cannot get this combined Elasticsearch query to work.
Individually projectName and timestamp queries work, but not this combined query:
curl -XGET "http://localhost/jenkins/_search/exists" -d'{"query" : {"bool": {"must": [{"match": {"data.projectName": {"query": "QA_Deployment","type": "phrase"}}}]},{"range": {"#timestamp": {"gte": "now-30d","lte": "now"}}}}}'
I changed two things, there was a space missing between the -d', not sure if that is a problem though. The other thing is the second query. This should be within the bool>must part as well. This should work:
curl -XGET "http://localhost/jenkins/_search/exists" -d '
{
"query" : {
"bool": {
"must": [
{
"match": {
"data.projectName": {
"query": "QA_Deployment",
"type": "phrase"
}
}
},
{
"range": {
"#timestamp": {
"gte": "now-30d",
"lte": "now"
}
}
}
]
}
}
}'
Related
I tried a few different ways of doing a simple get request, filtering on two different attributes, example:
"query": {
"filtered": {
"filter": {
"bool": {
"must": [
{
"term": {
"email": "erik.landvall#example.com"
}
},
{
"term": {
"password": "bb3810356e9b60cf6..."
}
}
]
}
},
"query": {
"match_all": []
}
}
}
The problem is that I get nothing back in return. As I understand it, this is because ElasticSearch analyzes the email field, making the query fail. So if I however would use the term erik.landvall instead of the complete email address, it will match the document - which confirms that's what's going on.
I can define the attribute as type:string and index:not_analyzed when I create the index. But what if I wanna be able to search on the email attribute in a different context? So there should, to my mind, be a way to specify that I wanna filter on the actual value of the attribute in a query. I can however not find how such a query would look.
Is it possible to force Elasticsearch to use "not_analyze" when querying? If so, then how?
You can use scripting for this purpose. You would have to directly access the JSON you have stored with _source. Try following query
{
"query": {
"bool": {
"filter": {
"script": {
"script": {
"inline" : "_source.email==param1 && _source.password==param2",
"params" : {
"param1" : "erik.landvall#example.com",
"param2" : "bb3810356e9b60cf6"
}
}
}
}
}
}
}
You would need to enable dynamic scripting. Add script.inline: on to your yml file and restart the node.
If this kind of query is fairly regular then It would be much better to reindex the data as others have suggested in the comments.
Its not possible to turn on/off analyzed or not, the way to do it to "transform" your field to analysis you need by using fields.
curl -XPUT 'localhost:9200/my_index?pretty' -d'
{
"mappings": {
"my_type": {
"properties": {
"city": {
"type": "string",
"fields": {
"raw": {
"type": "string",
"index": "not_analyzed"
}
}
}
}
}
}
}'
curl -XPUT 'localhost:9200/my_index/my_type/1?pretty' -d'
{
"city": "New York"
}'
curl -XPUT 'localhost:9200/my_index/my_type/2?pretty' -d'
{
"city": "York"
}'
curl -XGET 'localhost:9200/my_index/_search?pretty' -d'
{
"query": {
"match": {
"city": "york"
}
},
"sort": {
"city.raw": "asc"
},
"aggs": {
"Cities": {
"terms": {
"field": "city.raw"
}
}
}
}'
I am using logstash to store logs in elasticsearch database. I want to get logs having a particular severitylabel and are between certain time stamps and matches to some specific message. The curl query I wrote is :
curl -XPOST 'localhost:9200/logstash-2015.06.19/_search/?pretty' -d '{
"query": {
"filtered": {
"query": {
"bool": {
"must": [
{
"match": {
"#message": "session"
}
}
]
}
},
"filter": {
"bool": {
"must": [
{
"range": {
"#timestamp": {
"gte": "2015-06-19T10:11:44.000Z",
"lte": "2015-06-19T11:11:44.000Z"
}
}
},
{
"term": {
"#app": "sparta"
}
},
{
"terms": {
"#severityLabel": [
"INFO",
"WARN",
"ERROR",
"FATAL",
"OFF"
]
}
}
]
}
}
}
} } '
It always shows zero documents, matched. I am using term filter as a sibling of terms filter, is that a problem?
Here's an example of my data :
{
"MOD_DATE_START": "2010-04-20T15:05:49Z",
"MOD_DATE_END": null,
"MOD_ID": "123456789",
}
I'm having some issues with my Elasticsearch query. I have a couple of date fields where I am doing a range based filtering to make sure that my date is in between the start and end dates.
My first query (which works well) is filtering on the :
curl -s -XPOST http://server:9200/myindex/mytype/_search?pretty=true -d '
{
"fields": ["MOD_ID", "MOD_DATE_START", "MOD_DATE_END"],
"query": {
"bool": {
"must": [
{"term": {"MOD_ID": "123456789"}},
{"range": {"MOD_DATE_START": {"lte": "2012-04-20T15:05:49Z"}}}
]
}
}
}
'
The MOD_DATE_START field always contains information, so the first query works well.
Since the second date field, MOD_DATE_END, is null in most cases I would like to modify my query too add the following test :
IF "MOD_DATE_END" NOT NULL then
{"range": {"MOD_DATE_END": {"gte": "2012-04-20T15:05:49Z"}}}
ELSE skip "MOD_DATE_END"
I am, however, not quite able to figure out how to modify my query to add the third condition to be able to perform the gte test successfully.
Thanks in advance for your help.
One way to achieve this is by using a missing filter in a filtered query.
Example below :
curl -s -XPOST http://server:9200/myindex/mytype/_search?pretty=true -d '
{
"fields": ["MOD_ID", "MOD_DATE_START", "MOD_DATE_END"],
"query": {
"filtered": {
"filter": {
"bool": {
"must": {
"range": {
"MOD_DATE_START": {
"lte": "2012-04-20T15:05:49Z"
}
}
},
"should": [
{
"missing": {
"field": "MOD_DATE_END",
"null_value": true,
"existence": true
}
},
{
"range": {
"MOD_DATE_START": {
"gte": "2012-04-20T15:05:49Z"
}
}
}
]
}
},
"query": {
"term": {
"MOD_ID": "123456789"
}
}
}
}
}
'
I'm trying to query a nested properties with multiple values.
Here is an example that will be clearer.
Create an index with a nested field
curl -X DELETE "http://localhost:9200/testing_nested_query/"
curl -X POST "http://localhost:9200/testing_nested_query/" -d '{
"mappings": {
"class": {
properties: {
title: {"type": "string"},
"students": {
"type": "nested",
"properties": {
"name": {"type": "string"}
}
}
}
}
}
}'
Add some values
curl -XPUT 'http://localhost:9200/testing_nested_query/class/1' -d '{
"title": "class1",
"students": [{"name": "john"},{"name": "jack"},{"name": "jim"}]
}'
curl -XPUT 'http://localhost:9200/testing_nested_query/class/2' -d '{
"title": "class2",
"students": [{"name": "john"},{"name": "chris"},{"name": "alex"}]
}'
Query for all classes where john is (2 hits as expected)
curl -XGET 'http://localhost:9200/testing_nested_query/class/_search' -d '{
"query": {
"nested": {
"path":"students",
"query": {
"bool": {
"must": [
{"match": {"students.name": "john"}}
]
}
}
}
}
}'
Query for classes where both john and jack are attending ( 0 results instead of 1)
curl -XGET 'http://localhost:9200/testing_nested_query/class/_search' -d '{
"query": {
"nested": {
"path":"students",
"query": {
"bool": {
"must": [
{"match": {"students.name": "john"}},
{"match": {"students.name": "jack"}}
]
}
}
}
}
}'
I've tried with match and filter but I can never get the query to return the expected values.
It just needs a bit change:
{
"query": {
"bool": {
"must": [
{
"nested": {
"path":"students",
"query": {
"bool": {
"must": [
{"match": {"name": "john"}}
]
}
}
}
},
{
"nested": {
"path":"students",
"query": {
"bool": {
"must": [
{"match": {"name": "jack"}}
]
}
}
}
}
]
}
}
}
Why?
Basically, in a nested query, the query and the filter are collectively executed on a single nested document - in your case one name. So your query would pick up every nested document and try to find every document that has name equal to john and jack at the same time - which is impossible.
My query tries to find an indexed document that has one nested document with name equal to john and another nested document with name equal to jack. So basically one nested query tries to match one nested document completely.
To prove what I am suggesting, try this:
Create the same index with same mapping as you did
** Then index the following documents **
curl -XPUT 'http://localhost:9200/testing_nested_query/class/1' -d '{
"title": "class1",
"students": [{"name": "john", "age": 4},{"name": "jack", "age": 1},{"name": "jim", "age": 9}]
}'
curl -XPUT 'http://localhost:9200/testing_nested_query/class/2' -d '{
"title": "class1",
"students": [{"name": "john", "age": 5},{"name": "jack", "age": 4},{"name": "jim", "age": 9}]
}'
Now execute the following queries:
{
"query": {
"nested": {
"path":"students",
"query": {
"bool": {
"must": [
{"match": {"name": "john"}},
{"match": {"age": 4}}
]
}
}
}
}
}
According to your expectations, this should match 2 documents but it actually matches just only one. Because there is only one nested document that has both name equal to john and age equal to 4.
Hope that helps.
You could also do following way. where you do not need to repeat bool again in a nested block , since there is a only one to match within that block , you can just do term match without bool
{
"query": {
"bool": {
"must": [{
"nested": {
"path": "students",
"query": {
{
"term": {
"name": "john"
}
}
}
}
}, {
"nested": {
"path": "students",
"query": {
{
"term": {
"name": "jack"
}
}
}
}
}]
}
}
}
I'm trying to filter by terms within an array on elasticsearch documents. This is what the documents look like:
{
"name": "Foo",
"id": 10,
"industries": ["Tech", "Fashion"],
...
}
But for the various filter-based queries I try, I've gotten zero results. e.g.:
$ curl -XGET 'http://localhost:9200/_search?pretty=true' -d '
{
"query": {
"filtered": {
"filter": {
"bool": {
"must": [{
"terms": {
"industries": ["Tech"],
"execution": "or"
}
}]
}
},
"query": {"match_all": {}}
}
},
"from": 0,
"size": 20
}
'
I've tried about a dozen different queries against various simplifications and filter clauses, e.g. here's a simplified one:
$ curl -XGET 'http://localhost:9200/_search?pretty=true' -d '
{
"query": {
"filtered": {
"filter": {
"terms": {
"industries": ["Tech"],
"execution": "or"
}
}
}
},
"from": 0,
"size": 20
}
'
What am I missing here?
What analyzer are you using for the industries field? If you are using the default, it will actually lower case and split your stings, which would explain why your filters aren't picking those documents up (e.g., it's looking for "Tech" when only "tech" exists). If you set the mapping to not_analyzed (or use the multi fields option), that might solve your problem.