I created a dropwizard application that uses syslog to logging. I am using Mac OS version 10.10.3. My appenders in Configuration.yml file are as follows.
logging:
level: INFO
appenders:
- type: file
currentLogFilename: /var/log/myapp.log
threshold: INFO
archive: true
archivedLogFilenamePattern: /var/log/myapp-%d.log
archivedFileCount: 5
- type: syslog
host: localhost
port: 514
threshold: INFO
The file myapp.log is populated correctly. But when I do
sudo tail -f /var/log/system.log
I am not able to see the messages. I followed answer from How to start Syslogd server on Mac to accept remote logging messages? but still I was not able to see the messages. However if I do
sudo tcpdump -i lo0 host 127.0.0.1 and udp port 514
I am able see the packets. My syslog.conf looks like this
install.* #127.0.0.1:32376
What am I missing here?
Related
So I have rsyslog on an Ubuntu machine receiving log messages from another machine. The machine is sending tls on port 6514. I have another machine sending to this rsyslog server udp messages on port 514. Only the tls messages are getting to my /var/log/syslog file. The udp messages do not even though I do see the messages when I do a
sudo tcpdump port 514 -v
The related parts of my /etc/rsyslog.conf file are below
$DefaultNetstreamDriver gtls
$DefaultNetstreamDriverCAFile /etc/rsyslog-keys/ca.pem
$DefaultNetstreamDriverCertFile /etc/rsyslog-keys/xxx.pem
$DefaultNetstreamDriverKeyFile /etc/rsyslog-keys/yyy.pem
$ModLoad imudp
$UDPServerRun 514
$ModLoad imtcp
$InputTCPServerStreamDriveAuthMode anon
$ActionSendStreamDriverAuthMode anon
$InputTCPServerStreamDriverMode 1
$InputTCPServerRun 6514
What can I be missing that the udp messages are not getting to the syslog folder?
Any help will be greatly appreciated.
I'm completely new to ELK and trying to install the stack with some beats for our servers.
Elasticsearch, Kibana and Logstash are all installed (on server A). I followed this guide here https://www.elastic.co/guide/en/elastic-stack/current/installing-elastic-stack.html.
Filebeat template was installed as well.
I also installed filebeat on another server (server B), and was trying to test the connection
$ /usr/share/filebeat/bin/filebeat test output -c
/etc/filebeat/filebeat.yml -path.home /usr/share/filebeat -
path.config /etc/filebeat -path.data /var/lib/filebeat -path.logs
/var/log/filebeat
logstash: my-own-domain:5044...
connection...
parse host... OK
dns lookup... OK
addresses: 163.172.167.147
dial up... OK
TLS...
security: server's certificate chain verification is enabled
handshake... OK
TLS version: TLSv1.2
dial up... OK
talk to server... OK
Things seems to be ok, yet data from filebeat on server B doesn't seem to be sending data to logstash.
Accessing Kibana keeps redirecting me back to Create Index pattern, with the message
Couldn't find any Elasticsearch data
Any direction pointing would be really appreciated.
Can you check your filebeat.yml file and see if configuration for logs are activated :
filebeat.prospectors:
- type: log
enabled: true
paths:
- /var/log/*.log
I am using AWS EC2 CentOS based system. I also have a website is running on it at the domain mydomain.com
I have installed VLC successfully for my server. I can do SSH and remote to my server (I can use GUI application with gnome)
I start the vlc server with the below command:
vlc --ttl 34 -vvv --color -I telnet --telnet-password vlc --rtsp-host 0.0.0.0 --rtsp-port=554
and setup the stream on it by:
root#whm [~]# telnet localhost 4212
Trying ::1...
Connected to localhost.
Escape character is '^]'.
VLC media player 2.0.8 Twoflower
Password:
Welcome, Master
> new sample01 vod enabled
new
> setup sample01 input file:///home/myuser/public_html/mywebsite/folder/video.mp4
setup
The output of the log is good.
[0x7f24a002c5b8] stream_out_rtp vod server debug: RTSP stream at /sample01
[0x7f24a002c5b8] main vod server debug: net: listening to 0.0.0.0 port 554
[0x7f24a002c5b8] stream_out_rtp vod server debug: RTSP: adding /sample01/trackID=0
[0x7f24a002c5b8] stream_out_rtp vod server debug: RTSP: adding /sample01/trackID=1
[0x7fce4802d8c8] [Media: vod] main input debug: `file:///home/myuser/public_html/video.mp4' successfully opened
If I stay on the server it self, using GUI, I can use the VLC player to open network stream URL:
rtsp://localhost:554/sample01 (it works)
If I use vlc player from another network to connect to the stream by the below URL
rtsp://mydomain.com:554/sample01 (this is the domain that is working for my website)
rtsp://my-server-public-ip.com:554/sample01 (does not work too)
I guess there is something wrong with the port, then I went to AWS console and added more port 554 and 4212 into Inbound list in the Security Group of my instance (like what I'd done for other ports), but it does not help.
The error of the vlc client is:
[000000010050e4c8] core libvlc: Running vlc with the default interface. Use 'cvlc' to use vlc without interface.
[000000010022a7e8] [http] lua interface: Lua HTTP interface
[0000000104a00978] live555 demux error: Failed to connect with rtsp://mydomain.com:554/sample01
[00000001002b2dc8] core access error: connection failed: Connection refused
[00000001002b2dc8] access_realrtsp access error: cannot connect to mydomain.com:554
[0000000100618e58] core input error: open of `rtsp://mydomain.com:554/sample01' failed
CoreAnimation: warning, deleted thread with uncommitted CATransaction; set CA_DEBUG_TRANSACTIONS=1 in environment to log backtraces.
Please help me on this to troubleshoot this problem.
My problem is solved. I have done following things to resolve it, but I am not sure which one is correct. If anyone got same issue, you could try one of these
I enable to Elastic Load Balance serivice of Amazon and add port fowarding for port 554 (RTSP default port)
The streaming server was running well on localhost on server itself but it is inaccessible from outside, I guess it is something about firewall and port.
I scan mydomain.com to see the port is open or not
nmap -v -Pn -sT mydomain.com
Temporarily disable firewall on AWS EC2 instance OR add a line into iptables to open port 554.
Open and edit file by
vi /etc/sysconfig/iptables
Add below line into it
-A cP-Firewall-1-INPUT -p tcp -m state --state NEW -m tcp --dpport 554 -j ACCEPT
or shutdown the firewall absotelutely
sudo service iptables save
sudo service iptables stop
sudo chkconfig iptables off
Optional, if you are using firewall of Cpanel, you have to check the same with the allowed port of that firewall.
I opened couple client nodes with http 9200 to sever ElasticSearch queries/indices. I wanna log the access log from clients via http 9200, just like Http-Apache has the access.log. How should I enable this in ES please.
There's no such thing in Elasticsearch itself.
However, if you install the Shield plugin, you can enable auditing by adding this to your elasticsearch.yml configuration file.
shield.audit.enabled: true
You'll then get a new file called elasticsearch-access.log in your ES logs folder.
UPDATE by #lucabelluccini: Shield audit logs to syslog
In case you are interested in forwarding such audit logs to syslog, you can thanks to log4j SyslogAppender class which allows to forward logs to syslog via local socket.
Edit your logging.yml (customize the format etc...)
appender:
syslog:
type: org.apache.log4j.net.SyslogAppender
syslogHost: localhost
facility: local0
layout:
type: org.apache.log4j.PatternLayout
conversionPattern: "%d{ISO8601} %t %p %c %M %m %n"
Ensure rsyslog configuration allows UDP sources.
Associate this appender to the shield audit topic.
I just want to run packetbeat and get packet sniff from MySQL and output to file or console ,so that I no need Elastic system
I tried to run it but no thing output
root#localhost~: packetbeat -c packetbeat.yml
root#localhost~:
Following are my config file
procs:
enabled: true
monitored:
- process: mysqld
cmdline_grep: mysqld
output:
### Console output
console:
# Pretty print json event
pretty: false
How can I do that ?
Packetbeat works by capturing the network traffic that Mysql creates, so you need to also configure from which device to capture the traffic and on which tcp ports Mysql is running. For example:
interface:
device: any
protocols:
mysql:
ports: [3306]
procs:
enabled: true
monitored:
- process: mysqld
cmdline_grep: mysqld
output:
### Console output
console:
# Pretty print json event
pretty: false
Your console output configuration looks good to me. You can also output to rotating files, if you prefer.