elastalert sends multiple email alerts instead of sending an aggregated email - elasticsearch

Instead of sending one alert, ElastAlert sends email for each document which mapped. Below is my rule file. It works but I want alerts in one email. Please help any suggestion will be appreciated.
skynet.yaml: |-
---
name: skynet
type: frequency
limit_execution: "0/10 * * * *"
index: wpng-httpd-perf-*
num_events: 1
top_count_keys: ["Host_Id", "Host_Group"]
timeframe:
minutes: 15
filter:
- query:
query_string:
query: "Host_Group.keyword:ZOOKEEPER_ZK1_QA"
alert:
- "email"
email_format: html
aggregation:
minutes: 15
aggregation_key: 'Host_Id'
email:
- "johndoe#skynet.com"
from_addr: "sam#skynet.com"
alert_subject: "PLOT1 at {0}."
alert_subject_args:
- "#timestamp"
alert_text: "Hi Team,<br><br/> {0} ERROR event(s) detected in last 15 minutes <br/><br>Hosts where errors are detected :</br> Host_Id is {1} <br></br><br></br> <br>Here are a few of those :</br><br> messages {2} </br><br> </br><br/><br>bye.</br><br></br><br>Thanks <br></br> "
alert_text_type: alert_text_only
alert_text_args:
- num_matches
- Host_Id
- message
- top_count_keys

Below code worked for me.
PLOTTHREE.yaml: |-
---
name: PLOTTHREE
type: frequency
limit_execution: "0/15 * * * *"
index: home-*
num_events: 1
aggregation:
minutes: 10
include:
- Host_Group
- Host_Id
timeframe:
minutes: 15
filter:
- query:
query_string:
query: "Host_Group.keyword:fatal"
alert:
- "email"
email:
- "john#doe.com"
from_addr: "yyy#doe.com"
alert_subject: "PLOTTHREE - ERROR detected in Kafka Zookeeper logs of host group fatal at {0}."
alert_subject_args:
- "#timestamp"
alert_text: "Hello Team, ERROR event(s) detected in last 15 minutes. Hosts where errors are detected in {0}. Here is the num events {1} . "
alert_text_type: alert_text_only
alert_text_args:
- Host_Id
- num_matches

Related

Lookup table not working after training the model in rasa

I am new to rasa. I am training a model to recognize certain entities using lookup tables. I have multiple entities in the single sentence and I am trying to extract them.
nlu.yml
version: "2.0"
nlu:
- intent: intent_1
examples : |
- how many deaths were there last year in [Ohio](Filter-State)?
- death count of [Florida](Filter-State) this year
- death count of [Texas](Filter-State) this year
- what's the death count for this quarter in [CA](Filter-State)?
- lookup: Filter-State
examples: |
- Alabama
- AL
- Alaska
- AK
- Arizona
- AZ
- Arkansas
- AR
- California
- CA
- Colorado
- CO
- Connecticut
- CT
- Delaware
- DE
- District of Columbia
- DC
- Florida
- FL
- Georgia
- GA
config.yml
language: en
pipeline:
- name: WhitespaceTokenizer
- name: RegexFeaturizer
- name: LexicalSyntacticFeaturizer
- name: CountVectorsFeaturizer
- name: CountVectorsFeaturizer
analyzer: "char_wb"
min_ngram: 1
max_ngram: 4
- name: DIETClassifier
epochs: 150
random_seed: 1
- name: FallbackClassifier
threshold: 0.7
- name: DucklingEntityExtractor
url: http://duckling.rasa.com:8000
dimensions:
- email
- time
- name: EntitySynonymMapper
policies:
- name: AugmentedMemoizationPolicy
max_history: 4
- name: TEDPolicy
max_history: 4
epochs: 100
- name: RulePolicy
core_fallback_threshold: 0.4
core_fallback_action_name: "action_default_fallback"
enable_fallback_prediction: True
When I train the model and try using the api, It doesn't recognize cases from the states in the lookup table and as a result can't assign it to slot filter_state.
Can anyone advise me as to what am I doing wrong here for making the lookup table work!
I'm new to Rasa and searching for another issue, but I just ran into and solved this issue last night.
For lookup tables to work, you need to add "RegexEntityExtractor" to your pipeline and possibly remove RegexFeaturizer. You also need to enable lookup tables in the RegexEntityExtractor config.
config.yml
pipeline:
- name: WhitespaceTokenizer
- name: LexicalSyntacticFeaturizer
- name: CountVectorsFeaturizer
- name: RegexEntityExtractor
case_sensitive: False
use_lookup_tables: True
use_regexes: True
...
Could you, please, post on the Rasa forum and include more details of your setup? In particular, what version of Rasa Open Source are you using? Is the above your complete NLU data? (I think that you need at least 2 intents for an intent classifier to train.) I also recommend that you test the system using rasa interactive --debug and share e.g. a screenshot, this will help everyone see the exact input message and how it gets processed by Rasa. I'm sure we'll trace the issue down to its roots :-)

How to use the intent inform in Rasa?

I am trying to create a chatbot for banking. Flow is,
User: hi
Bot: Welcome to ABC bank services. Let me know your name?
User: Kabeer
Bot: Hi kabeer, how can i help you?
Buttons:
1: Loan (/inform{“service_type”:“Loan”})
2: Deposit (/inform{“service_type”:“Deposit”})
3: Account Types (/inform{“service_type”:“Account Types”})
4: Customer Support No (/inform{“service_type”:“Customer Support No”})
5: Service Request (/inform{“service_type”:“Service Request”})
6: Careers (/inform{“service_type”:“Careers”})
Type out your own message…
User: 1: Loan (/inform{“service_type”:“Loan”})
Bot:Please select the required Loan type from the list
Buttons:
1: Personal Loan (/inform{“loan_type”:“Personal Loan”})
2: Home Loan (/inform{“loan_type”:“Home Loan”})
3: Two-wheeler Loan (/inform{“loan_type”:“Two-wheeler Loan”})
4: Car Loan (/inform{“loan_type”:“Car Loan”})
5: Gold Loan (/inform{“loan_type”:“Gold Loan”})
Type out your own message…
User:Car Loan (/inform{“loan_type”:“Car Loan”})
Bot: Uttering home loan here.
Issue is the bot always utter home loan only. The bot is not recognise the stories for other loans using entity. How can i make this work? I used inform intent because of i thought it will cause similar intnent problem
Here is my code.
## say goodbye
* goodbye
- utter_goodbye
## bot challenge
* bot_challenge
- utter_iamabot
## happy_path_personal_loan_01
* greet
- utter_greet
* PERSON{"PERSON": "kabeer"}
- slot{"PERSON": "kabeer"}
- utter_ask_help
* inform{"service_type": "Loan"}
- slot{"service_type": "Loan"}
- utter_ask_loan_type
* inform{"loan_type": "Personal Loan"}
- slot{"loan_type": "Personal Loan"}
- utter_personal_loan
* goodbye
- utter_goodbye
## happy_path_home_loan_01
* greet
- utter_greet
* PERSON{"PERSON": "kabeer"}
- slot{"PERSON": "kabeer"}
- utter_ask_help
* inform{"service_type": "Loan"}
- slot{"service_type": "Loan"}
- utter_ask_loan_type
* inform{"loan_type": "Home Loan"}
- slot{"loan_type": "Home Loan"}
- utter_home_loan
* goodbye
- utter_goodbye
## happy_path_car_loan_01
* greet
- utter_greet
* PERSON{"PERSON": "kabeer"}
- slot{"PERSON": "kabeer"}
- utter_ask_help
* inform{"service_type": "Loan"}
- slot{"service_type": "Loan"}
- utter_ask_loan_type
* inform{"loan_type": "Car Loan"}
- slot{"loan_type": "Car Loan"}
- utter_car_loan
## interactive_story_1
* greet
- utter_greet
* PERSON{"PERSON": "kiran"}
- slot{"PERSON": "kiran"}
- utter_ask_help
* inform{"service_type": "Loan"}
- slot{"service_type": "Loan"}
- utter_ask_loan_type
* inform{"loan_type": "Home Loan"}
- slot{"loan_type": "Home Loan"}
- utter_home_loan
## interactive_story_2
* greet
- utter_greet
* PERSON{"PERSON": "kumar"}
- slot{"PERSON": "kumar"}
- utter_ask_help
* inform{"service_type": "Loan"}
- slot{"service_type": "Loan"}
- utter_ask_loan_type
* inform{"loan_type": "Personal Loan"}
- slot{"loan_type": "Personal Loan"}
- utter_personal_loan
- action_restart
## interactive_story_3
* greet
- utter_greet
* PERSON{"PERSON": "kiran"}
- slot{"PERSON": "kiran"}
- utter_ask_help
* inform{"service_type": "Loan"}
- slot{"service_type": "Loan"}
- utter_ask_loan_type
* inform{"loan_type": "Car Loan"}
- slot{"loan_type": "Car Loan"}
- utter_car_loan
- action_restart
## interactive_story_4
* greet
- utter_greet
* PERSON{"PERSON": "kiran"}
- slot{"PERSON": "kiran"}
- utter_ask_help
* inform{"service_type": "Loan"}
- slot{"service_type": "Loan"}
- utter_ask_loan_type
* inform{"loan_type": "Personal Loan"}
- slot{"loan_type": "Personal Loan"}
- utter_personal_loan
- action_restart
domain.yml
actions:
- utter_ask_help
- utter_ask_loan_type
- utter_car_loan
- utter_cheer_up
- utter_did_that_help
- utter_goodbye
- utter_greet
- utter_happy
- utter_home_loan
- utter_iamabot
- utter_personal_loan
entities:
- PERSON
- loan_type
- service_type
intents:
- greet
- inform
- PERSON
- goodbye
- affirm
- deny
- bot_challenge
slots:
PERSON:
type: text
loan_type:
type: text
service_type:
type: text
templates:
utter_ask_help:
- buttons:
- payload: /inform{"service_type":"Loan"}
title: Loan
- payload: /inform{"service_type":"Deposit"}
title: Deposit
- payload: /inform{"service_type":"Account Types"}
title: Account Types
- payload: /inform{"service_type":"Customer Support No"}
title: Customer Support No
- payload: /inform{"service_type":"Service Request"}
title: Service Request
- payload: /inform{"service_type":"Careers"}
title: Careers
text: Hi {PERSON}, how can i help you?
utter_ask_loan_type:
- buttons:
- payload: /inform{"loan_type":"Personal Loan"}
title: Personal Loan
- payload: /inform{"loan_type":"Home Loan"}
title: Home Loan
- payload: /inform{"loan_type":"Two-wheeler Loan"}
title: Two-wheeler Loan
- payload: /inform{"loan_type":"Car Loan"}
title: Car Loan
- payload: /inform{"loan_type":"Gold Loan"}
title: Gold Loan
text: Please select the required Loan type from the list
utter_car_loan:
- text: LTV up to 100% on select models, faster processing of loans, easy documentation.
utter_cheer_up:
- image: https://i.imgur.com/nGF1K8f.jpg
text: 'Here is something to cheer you up:'
utter_did_that_help:
- text: Did that help you?
utter_goodbye:
- text: Bye
utter_greet:
- text: Welcome to Voxtron bank services. Let me know your name?
utter_happy:
- text: Great, carry on!
utter_home_loan:
- text: Loans from 25,000 to 50 lakhs. Attractive interest rates, zero prepayment
charges, affordable EMIs.
utter_iamabot:
- text: I am a bot, powered by Rasa.
utter_personal_loan:
- text: Loans up to 20 lakhs at attractive interest, no collateral & flexible repayment.
Documents Required to apply, KYC and Income Documents are mandatory for applying
a Personal Loan. Additional documents may be requested on a case to case basis.
nlu.md
## intent:inform
- How can i apply for a [loan](service_type)
- i want to get a [car loan](loan_type)
- i want to apply for [personal loan](loan_type)
- please let me know how can i apply for a [loan](service_type)
- [gold loan](loan_type) details please
- how to get a [home loan](loan_type)
- I would like to know about the [gold loan](loan_type)
- what are the conditions to get a [home loan](loan_type)
- please send me the details about [vehicle loans](loan_type)
- I would like to take a [housing loan](loan_type)
- How am I supposed to apply for a [two wheeler loans](loan_type)
- [car loan](loan_type)
- [personal loan](loan_type)
- [gold loan](loan_type)
- [home loan](loan_type)
- [vehicle loan](loan_type)
- [housing loan](loan_type)
- [loan](service_type)
- [loans](service_type)
slots of type text are featurized in a way that it only affects the story flow if they are set or not (docs for it here)
I think in your case you rather want to use a categorical slot. In this case the stories will go down different paths depending on the slot value (docs for it here)

Curator Allocation action does not change ES index box_type setting from "hot" to "warm"

I'm using Elasticsearch Hot Warm Architecture for large time data analytics.
My curator job will set box_type of indices older than 2 days from "hot" to "warm" nodes. But when I run it at 18pm, 30th september, the 28th september indices' box_type is still "hot".
My curator action setting:
actions:
1:
action: open
description: Open indices younger than warm days (based on index name), for logstash-
prefixed indices.
options:
ignore_empty_list: true
disable_action: false
filters:
- filtertype: age
source: name
direction: younger
timestring: '%Y.%m.%d'
unit: days
unit_count: 30
- filtertype: pattern
kind: prefix
value: logstash-
exclude:
2:
action: allocation
description: Apply shard allocation to hot nodes
options:
key: box_type
value: hot
allocation_type: require
wait_for_completion: true
timeout_override:
continue_if_exception: false
ignore_empty_list: true
disable_action: false
filters:
- filtertype: age
source: name
direction: younger
timestring: '%Y.%m.%d'
unit: days
unit_count: 2
- filtertype: pattern
kind: prefix
value: logstash-
3:
action: allocation
description: Apply shard allocation to warm nodes
options:
key: box_type
value: warm
allocation_type: require
wait_for_completion: true
timeout_override:
continue_if_exception: false
ignore_empty_list: true
disable_action: false
filters:
- filtertype: age
source: name
direction: younger
timestring: '%Y.%m.%d'
unit: days
unit_count: 30
- filtertype: age
source: name
direction: older
timestring: '%Y.%m.%d'
unit: days
unit_count: 2
- filtertype: pattern
kind: prefix
value: logstash-
Logs:
2018-09-30 18:38:26,053 - curator.py:55 - INFO - Action ID: 1, "open" completed.
2018-09-30 18:38:26,053 - curator.py:55 - INFO - Preparing Action ID: 2, "allocation"
2018-09-30 18:38:26,053 - curator.py:55 - INFO - Trying Action ID: 2, "allocation": Apply shard allocation to hot nodes
2018-09-30 18:38:26,053 - curator.py:55 - INFO - Updating index setting {'index.routing.allocation.require.box_type': 'hot'}
2018-09-30 18:38:26,053 - curator.py:55 - INFO - Health Check for all provided keys passed.
2018-09-30 18:38:26,053 - curator.py:55 - INFO - Action ID: 2, "allocation" completed.
2018-09-30 18:38:26,053 - curator.py:55 - INFO - Preparing Action ID: 3, "allocation"
2018-09-30 18:38:26,053 - curator.py:55 - INFO - Trying Action ID: 3, "allocation": Apply shard allocation to warm nodes
2018-09-30 18:38:26,053 - curator.py:55 - INFO - Updating index setting {'index.routing.allocation.require.box_type': 'warm'}
2018-09-30 18:38:26,053 - curator.py:55 - INFO - Health Check for all provided keys passed.
2018-09-30 18:38:26,053 - curator.py:55 - INFO - Health Check for all provided keys passed.
2018-09-30 18:38:26,053 - curator.py:55 - INFO - Health Check for all provided keys passed.
2018-09-30 18:38:26,053 - curator.py:55 - INFO - Action ID: 3, "allocation" completed.
The logs say the action is completed. Shouldn't 28th of sept indices box_type be "warm"?
filtertype: age
source: name
direction: older
timestring: '%Y.%m.%d'
unit: days
unit_count: 2
Does this filter change all the 28th sept indices box_type to warm when I run it at 18PM, 30th of sept?
Time is measured from UTC 00:00 of the current day, not from the time of execution, so older than 2 days would depend on when UTC 00:00 is in your time zone. You can see how the time is calculated and measured in the curator log files if you enable DEBUG logging.

Curator not finding my index

Trying to have curator clean up an index I have named "syslog"
I have this action yml:
actions:
1:
action: delete_indices
description: >-
Delete indices older than hour
options:
ignore_empty_list: True
timeout_override:
continue_if_exception: False
disable_action: False
filters:
- filtertype: pattern
kind: prefix
value: syslog
exclude:
- filtertype: age
source: name
direction: older
timestring: '%Y.%m.%d'
unit: hours
unit_count: 1
exclude:
I tried both "hours" and "days" but I keeping getting this message from curator:
Skipping action "delete_indices" due to empty list: <class 'curator.exceptions.NoIndices'>
You are telling Curator that you want it to identify indices that have a prefix value of syslog, and also have a Year.Month.Day timestring in the index name. This would match, for example, if the index name were syslog-2017.03.14. But if the name of the index is only syslog, this filter set will not match.
Perhaps you want to catch the index age relative to when the index was created. In this case you'd set
actions:
1:
action: delete_indices
description: >-
Delete indices older than hour
options:
ignore_empty_list: True
timeout_override:
continue_if_exception: False
disable_action: False
filters:
- filtertype: pattern
kind: regex
value: '^syslog$'
- filtertype: age
source: creation_date
direction: older
unit: hours
unit_count: 1
This will match only an index named syslog whose creation_date is at least one hour older than execution time.

symfony2 translating errormessages

I want to translate errormessages inside validation.yml.
If I have a normal "NotBlank" rule, it works like following:
- NotBlank: { message: not.blank.firstname }
But what if there are some further rules like:
- NotBlank: { message: not.blank.username }
- Length:
min: 7
max: 50
minMessage: "Your Username must be at least {{ limit }} characters length"
This works, but how should I handle the minMessage? Also for the reason that I want to give USers some hints about the min Length of the input.
You can do something like this:
- NotBlank: { message: not.blank.username }
- Length:
min: 7
max: 50
minMessage: 'username.minLength'
maxMessage: 'username.maxLength'
Your validators.LANG.yml:
username:
minLength: "Your Username must be at least 7 characters length"
maxLength: "Your Username must be at least 50 characters length"

Resources