I am looking for the last time any user has logged in.
I have two authentication types ( Admin, Client) and am trying to do a query to find the event: Authentication succeeded Where authentication_type: Client
closest I have come is:
params='{ "query":{"terms":{"event_type":["authentication_succeeded"]}}}'
data=$(curl -XGET "localhost:9200/logstash-*/_search?scroll=10m&size=500&pretty" -H 'Content-Type: application/json' -d"${params}")
After many variations and alternate query types but I have not been able to successfully append the filter for '{"authentication_type":"Client"}'
You need to define your authentication_type and event_type as keyword and then use the term filters on these fields in your search query.
You can read with an example filter with multiple terms in this official ES link.
Below is a step by step example which uses your data to show the expected search results.
Index mapping
{
"mappings": {
"properties": {
"authentication_type": {
"type": "keyword"
},
"event_type" :{
"type" : "keyword"
}
}
}
}
Index sample docs
{
"authentication_type" : "Client",
"event_type" : "authentication_succeeded"
}
{
"authentication_type" : "Client",
"event_type" : "authentication_failed"
}
{
"authentication_type" : "Admin",
"event_type" : "authentication_succeeded"
}
{
"authentication_type" : "Admin",
"event_type" : "authentication_failed"
}
Search query
{
"query": {
"bool": {
"filter": [
{
"term": {
"event_type": "authentication_succeeded"
}
},
{
"term": {
"authentication_type": "Client"
}
}
]
}
}
}
Search Result
"hits": [
{
"_index": "so_60750542",
"_type": "_doc",
"_id": "1",
"_score": 0.0,
"_source": {
"authentication_type": "Client",
"event_type": "authentication_succeeded"
}
}
]
Related
I want to implement a simple username search within Elasticsearch. I don't want weighted username searches yet, so I would expect it wouldn't be to hard to find resources on how do this. But in the end, I came across NGrams and lot of outdated Elasticsearch tutorials and I completely lost track on the best practice on how to do this.
This is now my setup, but it is really bad because it matches so much unrelated usernames:
{
"settings": {
"index" : {
"max_ngram_diff": "11"
},
"analysis": {
"analyzer": {
"username_analyzer": {
"tokenizer": "username_tokenizer",
"filter": [
"lowercase"
]
}
},
"tokenizer": {
"username_tokenizer": {
"type": "ngram",
"min_gram": "1",
"max_gram": "12"
}
}
}
},
"mappings": {
"properties": {
"_all" : { "enabled" : false },
"username": {
"type": "text",
"analyzer": "username_analyzer"
}
}
}
}
I am using the newest Elasticsearch and I just want to query similar/exact usernames. I have a user db and users should be able to search for eachother, nothing to fancy.
If you want to search for exact usernames, then you can use the term query
Term query returns documents that contain an exact term in a provided field. If you have not defined any explicit index mapping, then you need to add .keyword to the field. This uses the keyword analyzer instead of the standard analyzer.
There is no need to use an n-gram tokenizer if you want to search for the exact term.
Adding a working example with index data, index mapping, search query, and search result
Index Mapping:
{
"mappings": {
"properties": {
"username": {
"type": "text",
"fields": {
"keyword": {
"type": "keyword"
}
}
}
}
}
}
Index Data:
{
"username": "Jack"
}
{
"username": "John"
}
Search Query:
{
"query": {
"term": {
"username.keyword": "Jack"
}
}
}
Search Result:
"hits": [
{
"_index": "68844541",
"_type": "_doc",
"_id": "1",
"_score": 0.2876821,
"_source": {
"username": "Jack"
}
}
]
Edit 1:
To match for similar terms, you can use the fuzziness parameter along with the match query
{
"query": {
"match": {
"username": {
"query": "someting",
"fuzziness":"auto"
}
}
}
}
Search Result will be
"hits": [
{
"_index": "68844541",
"_type": "_doc",
"_id": "3",
"_score": 0.6065038,
"_source": {
"username": "something"
}
}
]
I have a field called that is inside a nested field "name" that is a "Keyword" in elastic search.
Name field contains 2 values.
Jagannathan Rajagopalan
Rajagopalan.
If I query "Rajagopalan", I should get only the item #2.
If I query the complete Jagannathan Rajagopalan, I should get #1.
How do I achieve it?
You need to use the term query which is used for exact search. Added a working example according to your use-case.
Index mapping
{
"mappings": {
"properties": {
"name": {
"type": "nested",
"properties": {
"keyword": {
"type": "keyword"
}
}
}
}
}
}
Index sample docs
{
"name" : {
"keyword" : "Jagannathan Rajagopalan"
}
}
And another doc
{
"name" : {
"keyword" : "Jagannathan"
}
}
And search query
{
"query": {
"nested": {
"path": "name",
"query": {
"bool": {
"must": [
{
"match": {
"name.keyword": "Jagannathan Rajagopalan"
}
}
]
}
}
}
}
}
Search result
"hits": [
{
"_index": "key",
"_type": "_doc",
"_id": "2",
"_score": 0.6931471,
"_source": {
"name": {
"keyword": "Jagannathan Rajagopalan"
}
}
}
]
I am getting results from following Elastic Search query:
"query": {
"bool": {
"should": [
{"match_phrase_prefix": {"title": keyword}},
{"match_phrase_prefix": {"second_title": keyword}}
]
}
}
The result is good, but I want to change the order of the result so that the results with matching title comes top.
Any help would be appreciated!!!
I was able to reproduce the issue with sample data and My solution is using a query time boost, as index time boost is deprecated from the Major version of ES 5.
Also, I've created sample data in such a manner, that without boost both the sample data will have a same score, hence there is no guarantee that one which has match comes first in the search result, this should help you understand it better.
1. Index Mapping
{
"mappings": {
"properties": {
"title": {
"type": "text"
},
"second_title" :{
"type" :"text"
}
}
}
}
2. Index Sample docs
a)
{
"title": "opster",
"second_title" : "Dimitry"
}
b)
{
"title": "Dimitry",
"second_title" : "opster"
}
Search query
{
"query": {
"bool": {
"should": [
{
"match_phrase_prefix": {
"title": {
"query" : "dimitry",
"boost" : 2.0 <-- Notice the boost in `title` field
}
}
},
{
"match_phrase_prefix": {
"second_title": {
"query" : "dimitry"
}
}
}
]
}
}
}
Output
"hits": [
{
"_index": "60454337",
"_type": "_doc",
"_id": "1",
"_score": 1.3862944,
"_source": {
"title": "Dimitry", <-- Dimitry in title field has doube score
"second_title": "opster"
}
},
{
"_index": "60454337",
"_type": "_doc",
"_id": "2",
"_score": 0.6931472,
"_source": {
"title": "opster",
"second_title": "Dimitry"
}
}
]
Let me know if you have any doubt understanding it.
I'm making a subscription system for notifications using the percolate type of property of Elasticsearch 7.x. The problem is that I can't make a percolate query with certain types of fields.
This is an example of the indexed data. As you can see, I have a query indexed to be able to perform a percolate query. The difference I would like to mention is the name of the field in the query which can be state or created_by.full_name.raw
{
"_index": "widgets_2020",
"_type": "widget",
"_score": 1.0,
"_source": {
"created_at": "2020-01-09T21:58:14.123Z",
"query": {
"bool": {
"must": [],
"filter": [
{
"terms": {
"created_by.full_name.raw": [
"Ivan Ledner"
]
}
}
]
}
}
}
},
{
"_index": "widgets_2020",
"_type": "widget",
"_score": 1.0,
"_source": {
"created_at": "2020-01-09T22:02:24.133Z",
"query": {
"bool": {
"must": [],
"filter": [
{
"terms": {
"state": [
"done"
]
}
}
]
}
}
}
}
When I do something like this, Elasticsearch returns the documents I expect.
widgets_2020/_search
{
"query" : {
"percolate" : {
"field" : "query",
"document" : {
"state": ["created"]
}
}
}
}
But when I search this, It returns nothing.
widgets_2020/_search
{
"query" : {
"percolate" : {
"field" : "query",
"document" : {
"created_by.full_name.raw": ["Ivan Ledner"]
}
}
}
}
Is there a different way of dealing with these types of names? Thanks in advance!
The problem was that I enabled the option map_unmapped_fields_as_text and this mapped all my fields as text as the options say. The way I solved this is mapping all the attributes manually and the percolator started to work as expected.
I'm trying to filter my elasticsearch index by specific fields, the "country" field to be exact. However, I keep getting loads of other results (other countries) back that are not exact.
Please could someone point me in the right direction.
I've tried the following searches:
GET http://127.0.0.1:9200/decision/council/_search
{
"query": {
"filtered": {
"filter": {
"term": {
"country": "Algeria"
}
}
}
}
}
Here is an example document:
{
"_index": "decision",
"_id": "54290140ec882c6dac5ae9dd",
"_score": 1,
"_type": "council",
"_source": {
"document": "DEV DOCUMENT"
"id": "54290140ec882c6dac5ae9dd",
"date_updated": 1396448966,
"pdf_file": null,
"reported": true,
"date_submitted": 1375894031,
"doc_file": null,
"country": "Algeria"
}
}
You can use the match_phrase query instead
POST http://127.0.0.1:9200/decision/council/_search
{
"query" : {
"match_phrase" : { "country" : "Algeria"}
}
}