I just tried working with elasticsearch and now trying to create first watcher
There are some information I have read in elasticsearch documentation : https://www.elastic.co/guide/en/x-pack/current/watcher-getting-started.html
And now I trty to create one :
https://es.origin-test.cloud.rccf.ru/apiconnect508/_xpack/watcher/watch/audit_watch
PUT method + auth headers
I put in :
{ "trigger" : {
"schedule": {
"interval": "1h"
}
}, "actions" : { "send_email" : {
"email" : {
"to" : "ext_avolkova#rencredit.ru",
"subject" : "Watcher Notification",
"body" : "{{ctx.payload.hits.total}} logs found"
} } } }
But now I see mistake :
No handler found for uri
[/apiconnect508/_xpack/watcher/watch/log_audit] and method [PUT]
Please, help me to create one simple watcher
Based on the support matrix, elasticsearch 2.x is not compatible with x-pack.
You might want to install Watcher as a separate plugin using this document.
Related
Is it possible to trigger an action for each hit of a given query in a Kibana Monitor? I would like to use a foreach loop to do this as demonstrated here. However, it's unclear how to implement this on the Kibana Monitor page. On the page there is an input field for Trigger Conditions but I'm unsure how to format the foreach within it or if this is supported.
Consider using Elasticsearch watcher (require at least gold licesnse): https://www.elastic.co/guide/en/elasticsearch/reference/current/how-watcher-works.html
Watcher will run on a certain interval and will perform a query against indices (according to your configuration). You will need to create a condition (e.g. hits number is greater than 5) that when it evaluates to true an action will be performed. Elasticsearch allows you to use multiple actions. For example, you can use webhook and receive the data from the last watcher run (you can also use watcher api to transform the data). If you don't have Gold license you can mimic watcher behavior by a script/program that uses Elasticsearch Search API.
Herbeby is a simple example of a watcher checking index named test every minute and sends a webhook with the entire search context in case there is at least one document.
{
"trigger" : {
"schedule" : { "interval" : "1m" }
},
"input" : {
"search" : {
"request" : {
"indices" : [ "test" ],
"body" : {
"query" : {
"bool": {
"must": {
"range": {
"updatedAt": {
"gte": "now-1m"
}
}
}
}
}
}
}
}
},
"condition" : {
"compare" : { "ctx.payload.hits.total" : { "gt" : 0 }}
},
"actions" : {
"sample_webhook" : {
"webhook" : {
"method" : "POST",
"url": "http://b4022015b928.ngrok.io/UtilsService/api/elasticHandler/watcher",
"body" : "{{#toJson}}ctx.payload{{/toJson}}",
"auth": {
"basic": {
"user": "user",
"password": "pass"
}
}
}
}
}
}
An alternative way would be to use Kibana Alerts and Actions.
https://www.elastic.co/guide/en/kibana/current/alerting-getting-started.html
This feature is slightly different from Watcher but basically allows you to perfrom actions upon a query against Elasticsearch. This featrue is only part of Kibana opposing to watcher which is part of Elasticsearch (though it is accessible from Kibana stack management).
I fail at indexing timestamp fields with ElasticSearch (version 7.10.2) and I do not understand why.
So I create an index with the following mapping. You can copy & paste it directly to Kibana:
PUT /my-dokumente
{
"mappings" : {
"properties" : {
"aufenthalt" : {
"properties" : {
"aufnahme" : {
"properties" : {
"zeitpunkt" : {
"type" : "date",
"format": "yyyy-MM-dd HH:mm:ss",
"ignore_malformed": true
}
}
},
"entlassung" : {
"properties" : {
"zeitpunkt" : {
"type" : "date",
"format": "yyyy-MM-dd HH:mm:ss",
"ignore_malformed": true
}
}
}
}
}
}
}
}
Then I try to index a document:
PUT /my-dokumente/dokumente/1165963
{
"aufenthalt" :
{
"aufnahme" :
{
"zeitpunkt" : "2019-08-18 15:02:13"
},
"entlassung" :
{
"zeitpunkt" : "2019-08-20 10:29:22"
}
}
}
Now, i get this error:
"mapper [aufenthalt.entlassung.zeitpunkt] cannot be changed from type [date] to [text]
Why is elastic search not parsing my date?
I also tried with many different mapping settings like strict_date_hour_minute_second or to send the timestamp as "2019-08-18T15:02:13" or "2019-08-18T15:02:13Z" also, I converted it to epoch millis, but I always get some different error message, for example Cannot update parameter [format] from [strict_date_hour_minute_second] to [strict_date_optional_time||epoch_millis].
So the basic question is just: How can I send a timestamp value to ElasicSearch? (with Kibana/CURL).
PS: I am not using a Client SDK like Java High Level Rest Client. Why are talking about basic Kibana/CURL.
It can't be that complicated. What am I missing?
Thank you!
Mapping types are removed in 7.x. Refer to this official documentation
You need to add _doc in URL when indexing a document to Elasticsearch
Modify the URL as PUT /my-dokumente/_doc
I have an alias which is pointing to my OLD_INDEX. I have a requirement where I am creating a new index and after creation I need to point my alias A to the NEW_INDEX. I need to do this in Java.
I have looked almost everywhere but I cannot find any java implementation for this.
Would really appreciate some help. If possible, it would be great to have a sample code as well.
Thanks.
Refer
You can add7remove an alias.
To remove,
POST /_aliases { "actions" : [ { "remove" : { "index" : "test1", "alias" : "alias1" } } ] }
To add an alias,
POST /_aliases { "actions" : [ { "add" : { "index" : "test1", "alias" : "alias1" } } ] }
List of supported actions
You can use java low or high level clients to do this. Refer
You have to initialize Rest client and make a call by using above Json requests and end points.
I am trying to create a Slack action with a dynamic attachment. My _source looks like this:
{
"user.url": "https://api.github.com/users/...",
"user.gists_url": "https://api.github.com/users/.../gists{/gist_id}",
"user.repos_url": "https://api.github.com/users/.../repos",
"date": "2018-04-27T14:34:10Z",
"user.followers_url": "https://api.github.com/users/.../followers",
"user.following_url": "https://api.github.com/users/.../following{/other_user}",
"user.id": 123456,
"user.avatar_url": "https://avatars0.githubusercontent.com/u/123456?v=4",
"user.events_url": "https://api.github.com/users/.../events{/privacy}",
"user.site_admin": false,
"user.html_url": "https://github.com/...",
"user.starred_url": "https://api.github.com/users/.../starred{/owner}{/repo}",
"user.received_events_url": "https://api.github.com/users/.../received_events",
"metric": "stars",
"user.login": "...",
"user.type": "User",
"user.subscriptions_url": "https://api.github.com/users/.../subscriptions",
"user.organizations_url": "https://api.github.com/users/.../orgs",
"user.gravatar_id": ""
}
and here is my Slack action
"actions": {
"notify-slack": {
"throttle_period_in_millis": 240000,
"slack": {
"account": "monitoring",
"message": {
"from": "Elasticsearch Watcher",
"to": [
"#watcher"
],
"text": "We have {{ctx.payload.new.hits.total}} new stars! And {{ctx.payload.old.hits.total}} in total.",
"dynamic_attachments" : {
"list_path" : "ctx.payload.new.hits.hits",
"attachment_template" : {
"title" : "{{_source.[\"user.login\"]}}",
"text" : "Users Count: {{count}}",
"color" : "{{color}}"
}
}
}
}
}
I can't seem to figure out how to access my _source fields since they have dots in them. I have tried:
"{{_source.[\"user.login\"]}}"
"{{_source.user.login}}"
"{{_source.[user.login]}}"
"{{_source.['user.login']}}"
The answer to my question is that you can't access _source keys with dots in them directly using mustache, you must first transform your data.
Update:
I was able to get this working by using a transform to build a new object. Mustache might not be able to access fields with dots in their names, but painless can! I added this transform to my slack object:
"transform" : {
"script" : {
"source" : "['items': ctx.payload.new.hits.hits.collect(user -> ['userName': user._source['user.login']])]",
"lang" : "painless"
}
}
and now in the slack action dynamic attachments, I can access the items array:
"dynamic_attachments" : {
"list_path" : "ctx.payload.items",
"attachment_template" : {
"title" : "{{userName}}",
"text" : "{{_source}}"
}
}
Old Answer:
So according to this Watcher uses mustache.
and according to this mustache can't access fields with dots in the names.
POST /_aliases
{
"actions" : [
{ "add" : { "index" : "test*", "alias" : "all_test" } }
]
}
1)Based on the above configuration all the indices that start with test and also that are present in the cluster at that point of time will be kept in all_test alias.
2)If i create alias in Template by using below configuration
PUT localhost:9200/_template/test -d '
{
"template" : "test*",
"aliases" : {
"all_test" : {}
}
}
This will send the future indices which starts with test* to all_test alias.
Can i able to do 1+2 = I want to send the existing indices and the future indices to particular alias which i will define in ES Template?
If it is possible can you please share the sample template code for the above usecase?
Thanks