I am very new playing with syslog.
We have decided to use syslog to track some special events in our Rails application.
The problem is that I don't want to use the default /var/log/system.log file but use a custom one like /var/log/myapp_events.log.
I see that for that I have to define my own facility in /etc/syslog.conf like this:
myapp_events.* /var/log/myapp_events.log
After restarting syslogd I see that I can play with it directly into the bash console:
syslog -s -k Facility myapp_events Message "this is my message"
The message appears into the /var/log/myapp_events.log as expected, but I cannot reproduce this behavior using the syslog ruby gem. I have tried:
require 'syslog'
Syslog.open('myapp_events', Syslog::LOG_PID | Syslog::LOG_CONS) { |s| s.warning 'this is my message' } # sends the message to system.log
Syslog.open('myapp_events', Syslog::LOG_PID | Syslog::LOG_CONS, 'myapp_events') { |s| s.warning 'this is my message' } # error because 'myapp_event' can't be converted to int.
I see that Syslog.open has a third argument which is the facility but it has to be an integer and what I have is a string.
Any suggestion?
Definitely the syslog ruby implementation doesn't allow us to use custom facilities.
The syslog ruby implementation is using the syslog [C implementation] (http://github.com/ruby/ruby/blob/trunk/ext/syslog/syslog.c#L36).
The syslog C implementation only allows us to use a very short list of facility names: LOG_USER, LOG_MAIL, LOG_DAEMON, LOG_AUTH, LOG_SYSLOG, LOG_LPR, LOG_NEWS, LOG_UUCP, UUCP , LOG_CRON, LOG_AUTHPRIV, LOG_FTP, LOG_LOCAL0, LOG_LOCAL1, LOG_LOCAL2, LOG_LOCAL3, LOG_LOCAL4, LOG_LOCAL5, LOG_LOCAL6, LOG_LOCAL7.
So in the end what I did was to use one of the LOG_LOCALX facilities that are already there for personal use.
Now I can configure syslog like this:
# /etc/syslog.conf
local5.* /var/log/myapp_events.log
And in Ruby do this:
Syslog.open('myapp', Syslog::LOG_PID, Syslog::LOG_LOCAL5) { |s| s.info 'this is my message' }
I think is the way that syslog wants you to define custom facilities.
Logger isn't useful when you have multiple workers or threads, since the log messages get interleaved. This is why the default logger in Rails 3 is BufferedLogger.
This is also why you must use a buffering syslogger like rsyslog, or it will kill your performance. (I believe syslogd uses fsync() which is a synchronous call that waits to return.)
Take a look at Lumberjack.
It has support for Syslog as a log device, and you can specify facility as an option:
require 'lumberjack'
require 'lumberjack_syslog_device'
syslog_device = Lumberjack::SyslogDevice.new(:facility => 'myapp_events')
logger = Lumberjack::Logger.new(syslog_device)
logger.info "Hello, Syslog!"
Lumberjack also has quite a few other features that makes it worth a look.
You might want to look into using Logger, which is very similar to syslog, only it is a bit more user-friendly. It has multiple levels of criticality, like Syslog, and offers log rolling based on size or age.
Related
Accurately, I want to filter logs and send some warning email.
Firstly, I tried ommail, but unfortunately, this module only support mail server which do not need authentication, but my mail server needs.
So I tried to use omprog, I wrote a python script to logon to my mail server, it will recieve one parameter which is the log and send it as mail body.
Then I got the problem, I cannot pass the log to my script, if I try like this, $msg will be recognized as a string .
if $fromhost-ip == "x.x.x.x" then {
action(type="omprog"
binary="/usr/bin/python3 /home/elancao/Python/sendmail.py $msg")
}
I tried to search the official doc.
module(load="omprog")
action(type="omprog"
binary="/path/to/log.sh p1 p2 --param3=\"value 3\""
template="RSYSLOG_TraditionalFileFormat")
but in the sample, what they are using is a string "p1", not a dynamic parameter.
Can you please help? Thanks a lot!
The expected use of omprog is for your program to read stdin and there it will find the full default RSYSLOG_FileFormat template data (with date, host, tag, msg). This is useful, as it means you can write your program so that it is started only once, and then it can loop and handle all messages as they arrive.
This cuts down on the overhead of restarting your program for each message, and makes it react faster. However, if you prefer, your program can exit after reading one line, and then rsyslog will restart it for the next message. (You may want to implement confirmMessages=on).
If you just want the msg part as data, you can use template=... in the action to specify your own minimal template.
If you really must have the msg as an argument, you can use the legacy filter syntax:
^program;template
This will run program once for each message, passing it as argument the output of the template. This is not recommended.
if omprog script is not doing or not saving to a file the problem is that :
rsyslog is sending the full message to that script so you need to define or use a template
your script needs to listen to and return an
example in perl whit omprog
#my $input = join( '-', #ARGV ); ///not working I lost 5 hours of my life
my $input = ; now this is what you need
Hope this what the perl/python/rsyslog community needs.
I am using the google-api-client gem to interact with Google Calendar.
When saving a list of events like this:
#events = calendar.list_events(#tmpcalendar)
The output of calendar.list_events(#tmpcalendar) is huge. I just want to save the result to #events (don't need the huge content displayed).
I have tried: $output.close, redirecting to /dev/null, appending ;nil; but the huge result is displayed anyway.
To replicate this (with a solution that works) you can
large_text = <<~EOS
...huge text here
...huge text here
...huge text here
...
...huge text here
EOS
eventos_good = large_text ; nil # Assigns the content, does not display it
eventos_annoying = large_text # Assigns the content, but displays the huge text
This works for that case, but does not work with the above scenario (API call).
Is it possible to avoid that huge output just for that variable assignment above?
Thanks.
According to the docs, the Google API gem has debug logging enabled by default. To decrease the output, set the logging lever to something higher:
Google::Apis.logger.level = Logger::FATAL
# INFO, WARN, ERROR, FATAL each decrease the output.
# FATAL is the most restrictive.
From your code, it looks like you're using the Google API Client gem? Which version?
If you're not using this gem in a Rails application, then it will default all logging to $stdout: https://github.com/googleapis/google-api-ruby-client/blob/1135e74c4896d4ec8aa02c14e3532d9a14514815/lib/google/apis.rb#L34-L40
If you want to completely silence the logging, then override the logger:
require 'logger'
Google::Apis.logger = Logger.new(nil)
I've crafted a basic TCP client using EventMachine. Code:
# run.rb
EventMachine::run do
EventMachine::connect $config_host, $config_port, Bot
end
# bot.rb
module Bot
def post_init
# log us in and do any other spinup.
sleep(1)
send_data $config_login + "\n"
EventMachine.add_periodic_timer($config_keepalive_duration) { send_data $config_keepalive_str + "\n" }
#valid_command = /^<#{$config_passphrase}:([^>:]+):(#\d+)>(.*)$/
end
def receive_data(data)
if(ma = #valid_command.match(data))
command, user, args = ma[1,3]
args.strip!
command.downcase!
p "Received: #{command}, #{user}, #{args}"
# and.. here we handle the command.
end
end
end
This all works quite well. The basic idea is that it should connect, listen for specially formatted commands, and execute the command; in executing the command, there may be any number of "actions" taken that result in various data sent by the client.
But, for my next trick, I need to add the ability to actually handle the commands that Bot receives.
I'd like to do this using a dynamic library of event listeners, or something similar to that; ie, I have an arbitrary number of plugins that can register to listen for a specific command and get a callback from bot.rb. (Eventually, I'd like to be able to reload these plugins without restarting the bot, too.)
I've looked at the ruby_events gem and I think this makes sense but I'm having a little trouble of figuring out the best way to architect things. My questions include...
I'm a little puzzled where to attach ruby_events listeners to - it just extends Object so it doesn't make it obvious how to implement it.
Bot is a module, so I can't just call Bot.send_data from one of the plugins to send data - how can I interact with the EM connection from one of the plugins?
I'm completely open to any architectural revisions or suggestions of other gems that make what I'm trying to do easier, too.
I'm not sure exactly what you're trying to do, but one common pattern of doing this in EM is to define the command handlers as callbacks. So then the command logic can be pushed up out of the Bot module itself, which just handles the basic socket communication and command parsing. Think of how a web server dispatches to an application - the web server doesn't do the work, it just dispatches.
For example something like this
EM::run do
bot = EM::connect $config_host, $config_port, Bot
bot.on_command_1 do |user, *args|
# handle command_1 in a block
end
# or if you want to modularize it, something like this
# where Command2Handler = lambda {|user, *args| #... }
bot.on_command_2(&Command2Handler)
end
So then you just need to implement #on_command_1, #on_command_2, etc. in your Bot, which is just a matter of storing the procs to instance vars; then calling them once you parse the passed commands out of your receive_data.
A good, very readable example of this in production is TwitterStream.
I'd like to use Ruby's logger in command-line apps; it beats puts and has good flexibility for logging things.
One thing that I would like is to be able to have error/fatal messages go to the standard error (as is customary) in addition to where the logger's messages are configured to go.
logger = Logger.new(some_file)
logger.debug("This goes to some_file, if debug is set")
logger.info("This goes to some_file, if info is set")
logger.error("This goes to some_file, AND stderr")
One way I've done this is to hack the formatter:
logger.formatter = Proc.new do |severity,time,progname,msg|
message = format_message(severity,time,progname,msg)
if severity == ERROR
$stderr.puts message
end
message
end
This seems hacky. Another way might be to create a Logger that proxies its calls to an underlying real logger, but intercepting the error messages.
Anyone done this, and is there maybe something already that does this?
Anyone done this, and is there maybe something already that does this?
Ruby is cool, and flexible, and powerful, and stuff. Just with these monkey-patching things and abilities to open a class and add methods, it's too easy to forget about plain old inheritance...
class CopyLogger < Logger
def error message
# Print to standard error...
$stderr.puts message
# ...*and* to wherever you specified as well
super message
end
end
logger = CopyLogger.new(some_file)
# ...
You may put additional parameters into the constructor of your new class, such as the desired severity to tee messages of, etc.
I usually tail the log file to monitor while developing and put custom, user friendly messages for the masses.
Are there any logging frameworks in ruby that allow you to log a specific event type only once?
logger = IdealLogger.new
logger.log(:happy_path, "We reached the happy path") # => logs this message
logger.log(:happy_path, "We reached the happy path yet again") # => Doesn't log this
logger.log(:sad_path, "We've encountered a sad path!") # => logs this message
Also, is there a term for the concept of logging a certain event type only once?
Edit: I'm using Plain Old Ruby Objects, not Rails. I had in mind "once per time the script is run" for "once".
I'm not aware of one, but extending Logger to make your own isn't too difficult. It's essentially implementing caching for your logging, but instead of fetching from the cache and returning it like you would with a normal app, you quash it when it's been cached. Implementation and expiration strategy of this log cache are left as an exercise for the reader.
something like:
class IdealLogger < Logger
def info(event = nil, progname = nil, &block)
super(progname, &block) unless event_is_cached(event)
end
# define debug, warn, error, fatal, and unknown the same way, override others
# as you wish.
end