I am trying to create a Slack action with a dynamic attachment. My _source looks like this:
{
"user.url": "https://api.github.com/users/...",
"user.gists_url": "https://api.github.com/users/.../gists{/gist_id}",
"user.repos_url": "https://api.github.com/users/.../repos",
"date": "2018-04-27T14:34:10Z",
"user.followers_url": "https://api.github.com/users/.../followers",
"user.following_url": "https://api.github.com/users/.../following{/other_user}",
"user.id": 123456,
"user.avatar_url": "https://avatars0.githubusercontent.com/u/123456?v=4",
"user.events_url": "https://api.github.com/users/.../events{/privacy}",
"user.site_admin": false,
"user.html_url": "https://github.com/...",
"user.starred_url": "https://api.github.com/users/.../starred{/owner}{/repo}",
"user.received_events_url": "https://api.github.com/users/.../received_events",
"metric": "stars",
"user.login": "...",
"user.type": "User",
"user.subscriptions_url": "https://api.github.com/users/.../subscriptions",
"user.organizations_url": "https://api.github.com/users/.../orgs",
"user.gravatar_id": ""
}
and here is my Slack action
"actions": {
"notify-slack": {
"throttle_period_in_millis": 240000,
"slack": {
"account": "monitoring",
"message": {
"from": "Elasticsearch Watcher",
"to": [
"#watcher"
],
"text": "We have {{ctx.payload.new.hits.total}} new stars! And {{ctx.payload.old.hits.total}} in total.",
"dynamic_attachments" : {
"list_path" : "ctx.payload.new.hits.hits",
"attachment_template" : {
"title" : "{{_source.[\"user.login\"]}}",
"text" : "Users Count: {{count}}",
"color" : "{{color}}"
}
}
}
}
}
I can't seem to figure out how to access my _source fields since they have dots in them. I have tried:
"{{_source.[\"user.login\"]}}"
"{{_source.user.login}}"
"{{_source.[user.login]}}"
"{{_source.['user.login']}}"
The answer to my question is that you can't access _source keys with dots in them directly using mustache, you must first transform your data.
Update:
I was able to get this working by using a transform to build a new object. Mustache might not be able to access fields with dots in their names, but painless can! I added this transform to my slack object:
"transform" : {
"script" : {
"source" : "['items': ctx.payload.new.hits.hits.collect(user -> ['userName': user._source['user.login']])]",
"lang" : "painless"
}
}
and now in the slack action dynamic attachments, I can access the items array:
"dynamic_attachments" : {
"list_path" : "ctx.payload.items",
"attachment_template" : {
"title" : "{{userName}}",
"text" : "{{_source}}"
}
}
Old Answer:
So according to this Watcher uses mustache.
and according to this mustache can't access fields with dots in the names.
Related
Is it possible to trigger an action for each hit of a given query in a Kibana Monitor? I would like to use a foreach loop to do this as demonstrated here. However, it's unclear how to implement this on the Kibana Monitor page. On the page there is an input field for Trigger Conditions but I'm unsure how to format the foreach within it or if this is supported.
Consider using Elasticsearch watcher (require at least gold licesnse): https://www.elastic.co/guide/en/elasticsearch/reference/current/how-watcher-works.html
Watcher will run on a certain interval and will perform a query against indices (according to your configuration). You will need to create a condition (e.g. hits number is greater than 5) that when it evaluates to true an action will be performed. Elasticsearch allows you to use multiple actions. For example, you can use webhook and receive the data from the last watcher run (you can also use watcher api to transform the data). If you don't have Gold license you can mimic watcher behavior by a script/program that uses Elasticsearch Search API.
Herbeby is a simple example of a watcher checking index named test every minute and sends a webhook with the entire search context in case there is at least one document.
{
"trigger" : {
"schedule" : { "interval" : "1m" }
},
"input" : {
"search" : {
"request" : {
"indices" : [ "test" ],
"body" : {
"query" : {
"bool": {
"must": {
"range": {
"updatedAt": {
"gte": "now-1m"
}
}
}
}
}
}
}
}
},
"condition" : {
"compare" : { "ctx.payload.hits.total" : { "gt" : 0 }}
},
"actions" : {
"sample_webhook" : {
"webhook" : {
"method" : "POST",
"url": "http://b4022015b928.ngrok.io/UtilsService/api/elasticHandler/watcher",
"body" : "{{#toJson}}ctx.payload{{/toJson}}",
"auth": {
"basic": {
"user": "user",
"password": "pass"
}
}
}
}
}
}
An alternative way would be to use Kibana Alerts and Actions.
https://www.elastic.co/guide/en/kibana/current/alerting-getting-started.html
This feature is slightly different from Watcher but basically allows you to perfrom actions upon a query against Elasticsearch. This featrue is only part of Kibana opposing to watcher which is part of Elasticsearch (though it is accessible from Kibana stack management).
I am developing a webapp, for which I am pushing my search templates to ES during startup and using them to form the elastic search queries at runtime. I have a requirement wherein, I don't know the number of filters to be applied. Created a search template like -
{
"filters" : {
{{#toJson}}
clauses
{{/toJson}}"
}
}
And search will be made like this -
GET _search/template
{
"id": "template-id",
"params": {
"clauses": {
"filters" : {
{ "match": { "user" : "foo" } },
{ "match": { "user" : "bar" } }
}
}
}
which will render result as -
{
"filters":{
"filters":{
"match" : {
"user" : "foo"
}
},
{
"match" : {
"user" : "bar"
}
}
}
}
as suggested by ES documentation-
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-template.html
But, since it's an invalid JSON, it doesn't allow me to push the template to ES.
My template works well when I use it as stored template in elastic-home/config/scripts. But I want to manage my templates with JAVA and push all templates during startup only.
Can I get any help?
I want to access a document in my index directly using its _id field and I want to highlight a word in messageTextfield, for this, I created below query but highlight attribute does not appear in result response.
{
"query":{
"term":{
"_id": "1006382869737"
}
},
"highlight" : {
"tags_schema" : "styled",
"fields" : {
"messageText" : {
"highlight_query":{
"term": {
"messageText":"car"
}
}
}
}
}
}
I'm sure that car is occurred in messageText field for document by Id 1006382869737. so I'm sure highlights must exist in response, but it's not.
if it is important I'm using 2.3.4 version of elasticsearch. and query has been created according to this documentation. I'm not sure what rescore_query is in this documentation, if it is important please tell me how to edit my query, else give me another suggestion.
tnx :)
Additional Information
I also try this below query:
{
"query":{
"bool": {
"must":[
{
"term":{
"_id": "1006382869737"
}
},
{
"term": {
"messageText":"car"
}
}
]
}
},
"highlight" : {
"fields" : {
"messageText" : {}
}
}
}
but it causes no document hits.
I found solution:
My data comes from a server that I haven't access to its code. After many debugging I figure out that in some cases server send the text data in another field named caption.
Unfortunately The server has not a good documentation, that causes this problem.
Finally I find problem and add a more highlight_query for caption field and it's work fine now.
I've a large configuration with queries and filters, which works fine.
Now, I'm adding a new script filter with Groovy, which works fine too:
doc['age'].value >= 18;
But I'm wondering how to do the following with Groovy:
Add a temporary boolean field to current document. See example below.
Example document in my result:
{
"name": "foo",
"age": 20
}
But I want to add the result of the script filter in my result, like so:
{
"name": "foo",
"age": 20,
"age_ok": true
}
age_ok is not indexed, but set by the Groovy filter.
AFAIK, You can't inject a script filter into the search result, but you can use scriptable field in order to inject scriptable data. You'll have to duplicate some part of the script.
From the documentation :
{
"query" : {
...
},
"script_fields" : {
"test1" : {
"script" : "doc['my_field_name'].value * 2"
},
"test2" : {
"script" : {
"inline": "doc['my_field_name'].value * factor",
"params" : {
"factor" : 2.0
}
}
}
}
}
See https://www.elastic.co/guide/en/elasticsearch/reference/current/search-request-script-fields.html
I have the currenct structure:
localhost:9200/objects/content
{
id:1,
author:{
name:"john"
},
body:"abc"
}
localhost:9200/objects/reaction
{
content_id:1
message:'I like it'
}
How can I query to get all reactions of contents writed by "john"?
This means a query on reactions, checking the content specified by id, if author is "someone".
This Elasticsearch blog post describes how to manage relationships inside Elasticsearch.
You will need to set a parent mapping between your reactions and your content.
{
"reaction" : {
"_parent" : {
"type" : "content"
}
}
}
You will then index your reaction as a child of content id 1:
curl -XPOST localhost:9200/test/homes?parent=1 -d'
{
message:'I like it'
}
You can then use a Has Parent Query to retrieve all reactions to authors named john:
{
"has_parent" : {
"parent_type" : "content",
"query" : {
"term" : {
"author.name" : "john"
}
}
}
}
I would recommend you to add some redundancy to your model:
localhost:9200/objects/reaction
{
content_id:1
message:'I like it'
author_name:'john'
}
This will increase the index, of course. On the other hand the query to get all reactions of contents writed by "john" will be simple and fast.