I'm currently using rsyslog to send logs from a Linux server to QRadar (IBM's SIEM).
However, the server sends a lot of logs and I would like to filter them directly in the rsyslog.conf file. But if I write someting else than
*.* #MyServerIp
no logs are sent. Can anyone help me ?
Thank you !
Related
I have an rsyslog server and ELK stack running on the same server.
Our application is forwarding logs to rsyslog and is forwarding it to localhost.
We now want to split up our logging (frontend and backend logging).
Our frontend dev has added a tag [frontend] that will be added to the message.
Is it possible to filter this out in rsyslog and forward this to another logstash while keeping the backend logging?
i have this in my configuration at the moment but it keeps forwarding all messages to that logstash:
*.* ##localhost:5555
:msg, contains, "\[frontend\]" stop
*.* ##localhost:5544
:programname, contains, "backend" ~
We are sending the frontend logs through the backend so program name 'backend' is in every message we receive
did some more research and found a working solution:
*.* {
:msg, contains, "\[frontend\]"
##localhost:5555
}
*.* {:programname, contains, "backend"
##localhost:5544
stop
}
There is a problem with Graphite Docker images I try to run on my PC. Containers start up gracefully but I'm not able to send any message so that it would be displayed under "Metrics" tab. Volumes Mounting doesn't help either. Default storage-schema.conf should accept all messages.
The message used for testing is such:
echo "test.bash.stats 42 date +%s" | nc localhost 2003.
Moreover, most of the time (but not always) after sending above listed message "400 Bad request" error is responded.
Following images has been tested:
https://hub.docker.com/r/hopsoft/graphite-statsd/
https://hub.docker.com/r/kamon/grafana_graphite/
Any ideas, I'm missing something to configure additionally?
Despite the above explained issue there is is question related to Spring Boot Metrics export to Graphite over StatsD.
As described here http://docs.spring.io/spring-boot/docs/current/reference/html/production-ready-metrics.html in section 49.8.3 Example: Export to Statsd there is a requirement only to add com.timgroup:java-statsd-client as dependency and add property spring.metrics.export.statsd.host.
Unfortunately nothing is send to Graphite (docker image running on local PC https://github.com/kamon-io/docker-grafana-graphite). I have checked network with Wireshark (udp.port=8125). Is there maybe something missing to add into Spring boot project with metrics?
I've setup the logstash. All of sudden the logstash-forwarder stops to send the logs to Logstash. While checking the logs, its show the below error, can anyone help me to fix this ?
Read error looking for ack: read tcp 10.11.12.13:1223 i/o timeout
Thanks.
The error shows there is no connection between logstash forwarder (or client) to logstash indexer (or server). Common reasons:
Logstash service is down on logstash indexer. If down, start the service up.
connection between logstash forwarder and indexer is blocked by firewall.
How to check?
login forwarder
telnet 10.11.12.13 1223
If you can't, that's the firewall issue. You need fix or rollback some recent changes.
I want anyone who send a email to test#example.com, and the server could get the mail and so I can use some server side scripting language like Ruby to parser the content of the email.
Ruby solution would be great.
If you've never set up an smtp server, and you're not willing to dig in and learn a lot, you might want to check out some of the services that offer this kind of ability. I have never used any of these, not sure about quality or pricing aspects.
cloudmailin
mailgun
dispatch
postmark
An alternative to setting up and configuring a mail server is to use a service such as sendgrid or postmark. You point your DNS mx records at them and they will turn inbound messages into JSON documents and post them to you
You can install qmail on your server, and configure it to pipe incoming mail to your domain to the STDIN of your ruby script.
If you haven't done so already, create an MX record for your domain that points to your mailserver's IP. Then, login to the server as root, then install qmail by following the instructions at http://lifewithqmail.org/lwq.html#installation
Configure qmail to accept incoming mail for *#yourdomain.tld (catchall), and forward all incoming mail to the script at /home/yourhomedirectory/yourscript.sc:
add the following line to /var/qmail/control/me:
hostname.yourdomain.tld
add the following line to /var/qmail/control/defaultdomain:
yourdomain.tld
addd the following line to /var/qmail/control/virtualdomains
yourdomain.tld:yourusername
add the following line to /var/qmail/control/rcphosts:
yourdomain.tld
restart qmail:
qmailctl stop
qmailctl stat
Logout of the server, then log back in again as yourusername. Create a file /home/yourusername/.qmail-default, consisting of the following line:
| /home/yourhomedirectory/yourscript.sc
Create /home/yourhomedirectory/yourscript.sc script to process incoming mail. Enable executable permissions on the script for all users:
chmod a+x /home/yourhomedirectory/yourscript.sc
That's all there is to it. Incoming mail to your domain will be piped by qmail to this script's STDIN.
Install and configure an SMTP server. SMTP servers can usually be configured quite extensively, what to do with mails that should be deliverd locally. One such option would be to deliver the mail locally by invoking a ruby script. Popular SMTP servers are postfix, courier and qmail. The details about how to setup the mail server depends on which one you use.
I would like to generate a report using /var/logs/xferlog showing the traffic for a particular user, for a particular month.
First you configure your log file setup, then you can use awstats normally with FTP files just as you would monitor your regular http traffic.
http://awstats.sourceforge.net/docs/awstats_faq.html#FTP
Hope it helps.