So i have an alert rule that gets fired in prometheus when a queue length has been long for a certain period of time.
Through the alert manger, I am able to create and receive e-mails.
My question now is, as part of my e-mail body, I want to have the Date and Time that either the alert manager triggered the e-mail, or of when the alert was fired.
I am unsure how to do this. Whether I can create a label in the alert and populate it somehow with the current date/time, or what? Any ideas?
- alert: Alert
for: 5m
expr: ...
annotations:
timestamp: >
time: {{ with query "time()" }}{{ . | first | value | humanizeTimestamp }}{{ end }}
I still find iterating alerts and getting the value of timeseries or timestamp in alert text as difficult. So I have solved this problem in above way. It works, and I am able to get timestamp / timeseries of alert in email body. Cheers.!
The alerts in the Alertmanager templates have a StartsAt attribute you could use.
Related
I have EC2 service (elasticbeanstalk) which my project is located on it. Now Is there any way to see how many requests a specific API handles every day. I'm storing the error, access, ... logs to the cloud watch, Maybe somehow we could use the access logs to see how many requests each API handled every day. But I need to define a chart for it so in one look I could understand, For example, this new endpoint api/user/allowance that I have made, there are some customers started using it. So eventually what I need is something like this
Api | Total number of requests | filter_start_date | filter_end_date
Actually I have dig into the problem more, and I've found a solution for it, At least it's works for me. So I went to cloudWatch and then Insights panel, From there I could define a query to group my log messages by their request Url, The log message I have is sth like this
0.0.0.0 (11.11.11.11) - - [18/Oct/2019:13:33:49 +0000] "GET api/user/allowance HTTP/1.1" 200 2575 "-" "okhttp/3.6.0"
Then I have defined the query to get and group by the Request URL, And after grouping, I counted the grouped.
FIELDS #message
| PARSE #message "* [*] * * *" as ipAddresses, requestTime, RequestAction, RequestUrl, RestOfTheLog
| stats count(*) by RequestUrl
This way you'll have list of endpoints with total number of requests.
As you said, your app is writing the logs to CloudWatch Logs. You can log an unique entrance msg of the restful API you want to trace. Then create a custom metrics for the CloudWatch Log groups with filter matching the entrance msg of API.
See official doc for how to create custom metrics for CloudWatch Log groups.
I'm trying to figure out a way to export some of the events I can see in the security dashboard and alert center. The Customer Reports API only gives me the # of mail received per day, and # or spam messages per day, but is more than 24 hrs behind.
I've tried to create an alert in the security alerts center for whenever my domain gets a relevant email, but I just get an email once a minute that says the the threshold was exceeded, and I have to click into the investigation tool to actually get the relevant data.
Is there a place I can request # of phishing emails per hour, or be alerted whenever new phishing emails are found. Or Malware, etc.
The Reports API method UserUsageReport: get allows you to retrieve received spam emails for a certain date by specifying the parameter gmail:num_spam_emails_received
However, if you want to retrieve e.g. the emails from the last hour, there is no prebuilt functionality for this.
You can write a Google Apps Script that would browse your Gmail Inbox for new Spam Emails and set the script on a time-driven trigger
Sample:
function setmeOnHourlyTimer() {
var now = new Date();
var oneHourAgoinSeconds = Math.round(now.getTime()/1000 - 1200 *60);
var query = '"after:'+ oneHourAgoinSeconds +'"';
var spamMessages = Gmail.Users.Messages.list("YOU_EMAIL", {"labelIds": ["SPAM"] , "q": query}).messages;
if (spamMessages.length > 0){
GmailApp.sendEmail("paste your email here", "You have new Spam emails", "You got " + spamMessages.length + " new spam message(s) within the last hour.")
}
}
I'm having some difficulties setting up an elastalert rule. It's quite a basic one, and I've read the documentation but clearly not understood it and I'm after some help.
I have a basic test rule that i want to alert when my data input to elastic from certain devices stops for more that 5 minutes.
es_host: localhost
es_port: 9200
name: Example rule
type: flatline
index: test_mapping-*
threshold: 1
timeframe:
minutes: 5
filter:
- term:
device: "ggYthy767b"
alert:
- command
command: ["/bin/test"]
realert:
minutes: 10
This works, so when data stops i get an alert, then that alert is silenced until 10 minutes later it realerts again. The issue is that it realerts every 10 minutes and i don't know how to stop it. Is there a way to get it to realert just once and then stop? Or have i misunderstood? Also I have 10+ different devices, and i want the same alert to apply if any of them stop sending data for 5 minutes, is that possible within one rule? Thanks very much in advance.
The question you need to ask to yourself is how often do you want to get alerted. Once a lifetime, a year, a month or fortnightly or what? So "realert" is the part you want to edit. You might want to change it to something like below. So even if the alert is triggered multiple times you'll only get it once a day. It uses simple English terms so you can update it how you like it (weeks, hours etc.).
realert:
days: 1
But if you're getting alerted much more than you want, either you're system is too unstable or your alerts are too paranoid. For example for this alert every 5 minutes you're looking for one record which actually doesn't get populated. You should raise your period or add less selective filters because it's a 'flatline' alert. You can also use it with "query_key" so it will be applied on a per key basis.
I am trying to get all the messages from a particular group. I am getting the json feed back. The only problem is, its returning only 20 messages. Is this set as default or something. Is there any way by by which while doing the request, I can specify whether I want all the messages, by default just 20 or even messages posted between the start and the end date?
My RestApi call is:
https://www.yammer.com/api/v1/messages/in_group/[id].json
From Yammer Developer Documentation
<
Autocomplete: 10 requests in 10 seconds.
Messages: 10 requests in 30 seconds.
Notifications: 10 requests in 30 seconds.
All Other Resources: 10 requests in 10 seconds.
These limits are independent e.g. in the same 30 seconds period, you could make 10 message calls and 10 notification calls. The specific rate limits are subject to change but following the guidelines below will ensure that your app is not blocked.>>
I have tried using limit as the parameter to change the number of message more than 20. But it doesnt seem to be working?
Is this problem because of Rate Limit. If not, what's the problem?
Official documentation from Yammers Developer documentation
Messages - Viewing Messages
Endpoints:
1) All public messages in the user’s (whose access token is being used to make the API call henceforth referred to as current user) Yammer network. Corresponds to “All” conversations in the Yammer web interface.
GET https://www.yammer.com/api/v1/messages.json
2) The user’s feed, based on the selection they have made between “Following” and “Top” conversations.
GET https://www.yammer.com/api/v1/messages/my_feed.json
3) The algorithmic feed for the user that corresponds to “Top” conversations, which is what the vast majority of users will see in the Yammer web interface.
GET https://www.yammer.com/api/v1/messages/algo.json
4) The “Following” feed which is conversations involving people, groups and topics that the user is following.
GET https://www.yammer.com/api/v1/messages/following.json
5) All messages sent by the user. Alias for /api/v1/messages/from_user/logged-in_user_id.format.
GET https://www.yammer.com/api/v1/messages/sent.json
6) Private messages received by the user.
GET https://www.yammer.com/api/v1/messages/private.json
7) All messages received by the user.
GET https://www.yammer.com/api/v1/messages/received.json
Parameters:
The messages API endpoints return a similar structure and support the following query parameters:
older_than - Returns messages older than the message ID specified as a numeric string. This is useful for paginating messages. For example, if you’re currently viewing 20 messages and the oldest is number 2912, you could append “?older_than=2912″ to your request to get the 20 messages prior to those you’re seeing.
newer_than - Returns messages newer than the message ID specified as a numeric string. This should be used when polling for new messages. If you’re looking at messages, and the most recent message returned is 3516, you can make a request with the parameter “?newer_than=3516″ to ensure that you do not get duplicate copies of messages already on your page.
threaded=[true | extended] - threaded=true will only return the first message in each thread. This parameter is intended for apps which display message threads collapsed. threaded=extended will return the thread starter messages in order of most recently active as well as the two most recent messages, as they are viewed in the default view on the Yammer web interface.
limit - Return only the specified number of messages. Works for threaded=true and threaded=extended.
Noted the limit parameter that you can set on your GET request - so based on this documentation if it is correct (I'm not a Yammer Developer but I do use it) you should be able to do
https://www.yammer.com/api/v1/messages.json?limit=50
That is in theory but reading through the documentation there is a section on Search that has
page - Only 20 results of each type will be returned for each page, but a total count is returned with each query. page=1 (the default) will return items 1-20, page=2 will return items 21-30, etc.
Which says to me they are limited to 20 results returned.
UPDATE
After testing this with https://www.yammer.com/api/v1/messages.json?limit=50 and it not returning 50 messages but doing https://www.yammer.com/api/v1/messages.json?limit=5 will return only 5 messages I would say that Yammer restrict the number of messages to 20 Also after reading through the documents a bit more I read
For example, if you’re currently viewing 20 messages and the oldest is number 2912, you could append “?older_than=2912″ to your request to get the 20 messages prior to those you’re seeing"
This says to me that they will only return a max of 20. So I think you are stuck with 20 messages at a time.
Hope this helps.
You need to use Parameters:
The messages API endpoints return a similar structure and support the following query parameters:
older_than - Returns messages older than the message ID specified as a numeric string. This is useful for paginating messages. For example, if you’re currently viewing 20 messages and the oldest is number 2912, you could append “?older_than=2912″ to your request to get the 20 messages prior to those you’re seeing.
newer_than - Returns messages newer than the message ID specified as a numeric string. This should be used when polling for new messages. If you’re looking at messages, and the most recent message returned is 3516, you can make a request with the parameter “?newer_than=3516″ to ensure that you do not get duplicate copies of messages already on your page.
threaded=[true | extended] - threaded=true will only return the first message in each thread. This parameter is intended for apps which display message threads collapsed. threaded=extended will return the thread starter messages in order of most recently active as well as the two most recent messages, as they are viewed in the default view on the Yammer web interface.
limit - Return only the specified number of messages. Works for threaded=true and threaded=extended.
Example : GET https://www.yammer.com/api/v1/messages.json?older_than=2912
while older can be ID of message number 20 and so on you can get 20 by 20
I solved by requesting subsequent pages in a recursive manner.
You can simply increase the page parameter until the response is empty, or update the older_than parameter until the property meta.older_available is false.
We have been implementing an instant message service and want to use HBEASE to store message history (and using redis to caching ongoing conversation). The incoming message for a user looks like
Userid (to whom)
time
message body (combined with from, message body ....)
Regarding Schema design:
Option A: one message per row
Row key: md5(userid) + timesample
column/valye: null / message
Option B: one user per row
Row key: md5(userid)
column/valye: time / message
could you help me to figure out pro and cont? thanks
chatting type include: peer-2-peer, group chating
As far as I know Facebook has done a great job on message system use hbase; Maybe these links will help you:http://www.slideshare.net/brizzzdotcom/facebook-messages-hbase
http://sites.computer.org/debull/A12june/facebook.pdf