I'm having an ec2 linux server, and am tracking the logs of my application server using rsyslog so that I can push these logs to loggly.
The problem is, rsyslog is also logging these in /var/log/messages which I don't want. Is there any way to avoid this? Can I filter out certain messages in /etc/rsyslog.conf so that these are not pushed to var/log/messages?
****** UPDATE *******
I tried adding the following lines in rsyslog.conf:
if $programname == 'programName' then {
*.err /var/log/messages
} else {
*.info;mail.none;authpriv.none;cron.none /var/log/messages
}
However, upon restarting rsyslog, I see the following error:
Dec 11 08:01:46 <hostname> rsyslogd: the last error occured in /etc/rsyslog.conf, line 37:"if $programname == 'programName' then {"
Dec 11 08:01:46 <hostname> rsyslogd: warning: selector line without actions will be discarded
Dec 11 08:01:46 <hostname> rsyslogd-3000: unknown priority name "" [try http://www.rsyslog.com/e/3000 ]
Dec 11 08:01:46 <hostname> rsyslogd: the last error occured in /etc/rsyslog.conf, line 39:"} else {"
Dec 11 08:01:46 <hostname> rsyslogd: warning: selector line without actions will be discarded
Dec 11 08:01:46 <hostname> rsyslogd-3000: unknown priority name "" [try http://www.rsyslog.com/e/3000 ]
Dec 11 08:01:46 <hostname> rsyslogd: the last error occured in /etc/rsyslog.conf, line 41:"}"
Dec 11 08:01:46 <hostname> rsyslogd: warning: selector line without actions will be discarded
I suppose my version of rsyslog (5.8.10) doesn't support if / else. Is there any other way to do this?
Thanks.
first send the message to the file that you want.
then use stop to prevent further actions.
if $programname == 'apache2' then {
action(type="omfile" file="/var/log/apache2/rewrite.log" name="action-omfile-apache2-rewrite")
stop
}
Related
When I start a linux server with Cloud-init, I have a few scripts in /etc/cloud/cloud.cfg.d/ and they run in reverse alphabetical order
# ll /etc/cloud/cloud.cfg.d/
total 28
-rw-r--r-- 1 root root 173 Dec 10 12:38 00-cloudinit-lifecycle-hook.cfg
-rw-r--r-- 1 root root 2120 Jun 1 2021 05_logging.cfg
-rw-r--r-- 1 root root 590 Oct 26 17:55 10_aws_yumvars.cfg
-rw-r--r-- 1 root root 29 Dec 1 18:22 20_amazonlinux_repo_https.cfg
-rw-r--r-- 1 root root 586 Dec 10 12:38 50-cloudinit-tomcat.cfg
-rw-r--r-- 1 root root 585 Dec 10 12:40 60-cloudinit-newrelic.cfg
The last to execute is 00-cloudinit-lifecycle-hook.cfg, in which I complete the lifecycle for the Auto Scaling Group with a CONTINUE. The ASG fails if it doesn't receive this signal after a given time out.
The issue is that even if there's an error in 50-cloudinit-tomcat.cfg, it still runs 00-cloudinit-lifecycle-hook.cfg instead of stopping
How can I ensure cloud-init stops and never reaches the last script? I would like the ASG to never receive the CONTINUE signal if there's any error.
Here are the files:
EC2 instance user-data:
#cloud-config
bootcmd:
- [cloud-init-per, once, "app-volume", mkfs, -t, "ext4", "/dev/nvme1n1"]
mounts:
- ["/dev/nvme1n1", "/app-volume", "ext4", "defaults,nofail", "0", "0"]
merge_how:
- name: list
settings: [append]
- name: dict
settings: [no_replace, recurse_list]
50-cloudinit-tomcat.cfg
#cloud-config
merge_how:
- name: list
settings: [append]
- name: dict
settings: [no_replace, recurse_list]
runcmd:
- "#!/bin/bash -e"
- set +x
- echo ' '
- echo '# ===================================='
- echo '# Tomcat Cloud Init '
- echo '# /etc/cloud/cloud.cfg.d/'
- echo '# ===================================='
- echo ' '
- echo '#===================================='
- echo '# Run Ansible'
- echo '#===================================='
- echo ' '
- set -x
- ansible-playbook /opt/init-config/tomcat/tomcat-config.yaml
when I run ansible-playbook /opt/init-config/tomcat/tomcat-config.yaml directly in the instance I get an error, and I know it returns 2
ansible-playbook /opt/init-config/tomcat/tomcat-config.yaml #shows errors
echo $? # shows "2"
00-cloudinit-lifecycle-hook.cfg
#cloud-config
merge_how:
- name: list
settings: [append]
- name: dict
settings: [no_replace, recurse_list]
runcmd:
- "/opt/lifecycles/lifecycle-hook-continue.sh"
An alternative I can think of, is to send a ABANDON signal instead of CONTINUE as soon as there's en error in one of the cloud-init config. But I can't find in the documentation on to define if there's an error
I have a syslog-ng server configured to send all apache log messages to a remote rsyslog server. Here are the pertinent part of my syslog-ng server's config:
source s_http {
file("/var/log/httpd/access_log" flags(no-parse));
};
...
destination loghost { tcp("10.0.0.48" port(514)); };
...
log { source(s_http); destination(loghost); };
I was hoping to find on the remote rsyslog server (10.0.0.48) the file: /apps/log/my-web-server/access_log. but instead I find several files in the /apps/log/my-web-server/ named for the IP address of the clients that hit my-web-server with a .log extension.
[root#10.0.0.48]# pwd
/apps/log/my-web-server
[root#10.0.0.48]# ls -l
total 140
-rw-------. 1 root root 4862 Aug 14 16:39 10.0.0.97.log
-rw-------. 1 root root 193 Aug 14 15:45 10.0.0.201.log
Why aren't the log messages going into one file named access_log?
Update:
On the rsyslog server at 10.0.0.48 I see these lines in the /etc/rsyslog.conf
$template RemoteStore, "/apps/log/%HOSTNAME%/%PROGRAMNAME%.log"
$template RemoteStoreFormat, "%msg%\n"
:source, !isequal, "localhost" -?RemoteStore;RemoteStoreFormat
:source, isequal, "last" STOP
what does that mean?
I needed to change ...
source s_http {
file("/var/log/httpd/access_log" flags(no-parse));
};
... to this ...
source s_http {
file("/var/log/httpd/access_log" program-override("apache_access_log"));
};
I made a small Bash-script to make my life easier. But I encountered a problem which I can't fix.
What I want
I've made a small script which will check for php-errors in a file each time that file gets saved/changed. This is done without me needing to run a command each time. So I run the Bash-script once on my second screen, and than each time when I save my PHP-file on screen one; I get the eventual errors shown on screen two automatically.
Basic algorithm
Get the hash of the file
Compare it to it's previous hash
If it differs, the file is changed/saved: Check if there are errors using the php -l command
Print out the result from php -l
Problem:
The result from php -l gets printed out before my code asked for it.
Code
#!/bin/bash
#Declaring basic variables here
fileToCheck="$1"
oldHash=("")
checksum=("")
#Function to get a striped line as long as the terminal width
function lineAmount {
cwidth=`tput cols`
lines=""
for i in $(seq $(expr $cwidth - 33)); do lines="$lines-";done
echo $lines
}
#Function to show the actual error
function showError {
msg=$1
time=`date +"%c"`
l=$(lineAmount)
if [ "$msg" == "No" ]
then
msg="No errors detected."
fi
printf "\n\n$time $l \n$msg\n"
}
#Start-screen------------------------------------------------
printf "Starting session for $1 at $time \n$(lineAmount)\n"
if [ ! -f $1 ]
then
echo "[Error] File $1 not found."
exit
fi
printf "\n\n\n"
#------------------------------------------
#Printing out error when file changed
while true
do
sleep 0.6
checksum=($(sha256sum $fileToCheck))
checksum=${checksum[0]}
if [ "$checksum" != "$oldHash" ]
then
error=$(php -l $fileToCheck)
oldHash=$checksum
showError $error
fi
done
Test file (test.php):
<?php
function foo() {
}
?>
Output of script:
Starting session for /home/name/Desktop/test.php at
-----------------------------------------------
Thu 11 Aug 2016 08:16:15 PM CEST -----------------------------------------------
No errors detected.
Now, in test.php I delete line 4:
<?php
function foo() {
?>
This will of course give an error, and my script shows that error:
Starting session for /home/name/Desktop/test.php at
-----------------------------------------------
Thu 11 Aug 2016 08:16:15 PM CEST -----------------------------------------------
No errors detected.
PHP Parse error: syntax error, unexpected end of file in /home/name/Desktop/test.php on line 6
Thu 11 Aug 2016 08:19:37 PM CEST ----------------------------------------------------------------------------------
Parse
But like you can see, this is not a nice output.
PHP Parse error: syntax error, unexpected end of file in /home/name/Desktop/test.php on line 6 should be printed below the second dotted line. Not below "No errors found." (The first output).
Expected output:
Starting session for /home/name/Desktop/test.php at
-----------------------------------------------
Thu 11 Aug 2016 08:16:15 PM CEST -----------------------------------------------
No errors detected.
Thu 11 Aug 2016 08:19:37 PM CEST ----------------------------------------------------------------------------------
PHP Parse error: syntax error, unexpected end of file in /home/name/Desktop/test.php on line 6
I tried a lot, I tried to change my algorithm a bit, searched up a lot; but it ain't working.
I guess the problem is somewhere on line 51, or 29. But I really can't see what's wrong.
Thanks!
Here's a stripped down and simplified version of your problem:
Why does this print an error message immediately instead of assigning it to the variable?
$ error=$(php -l test.php)
PHP Parse error: syntax error, unexpected end of file in test.php on line 5
php -l prints error messages to stderr like a good Unix citizen should. $(..) only captures stdout.
If you want to capture stdout and stderr together, you can use:
error=$(php -l $fileToCheck 2>&1)
You should also quote your variables so that the message is passed as a single parameter, since you're currently throwing away most of it (shellcheck is helpful):
showError "$error"
Being a good citizen, php also returns a useful exit code, so instead of trying to match a "No" to see if it's successful, you can just check the status directly:
if error=$(php -l $fileToCheck 2>&1)
then
echo "No problems"
else
echo "It failed with these messages: $error"
fi
I need a shell script to parse a log file and look for a certain pattern. if that paatern found, then take key values from that line and put it into a csv.
Example:
Here is the log file i have :
*webauthRedirect: Mar 24 08:57:50.903: #EMWEB-6-PARSE_ERROR: webauth_redirect.c:1034 parser exited. client mac= a0:88:b4:d3:55:8c bytes parsed = 0 and bytes read = 213
*webauthRedirect: Mar 24 08:57:50.903: #EMWEB-6-HTTP_REQ_BEGIN_ERR: http_parser.c:579 http request should begin with a character
***ewmwebWebauth1: Mar 04 11:33:46.870: #PEM-6-GUESTIN: pem_api.c:7851 Guest user logged in with user account (mrathi_dev) MAC address 00:1e:65:39:10:8e, IP address 192.168.133.146.**
*ewmwebWebauth1: Mar 04 11:33:46.870: #AAA-5-AAA_AUTH_NETWORK_USER: aaa.c:2178 Authentication succeeded for network user 'mrathi_dev'
*ewmwebWebauth1: Mar 04 11:33:46.858: #APF-6-USER_NAME_CREATED: apf_ms.c:6532 Username entry (mrathi_dev) with length (10) created for mobile 00:1e:65:39:10:8e
*mmListen: Mar 24 08:57:49.030: #APF-6-RADIUS_OVERRIDE_DISABLED: apf_ms_radius_override.c:1085 Radius overrides disabled, ignoring source 4
*webauthRedirect: Mar 24 08:57:47.008: #EMWEB-6-PARSE_ERROR: webauth_redirect.c:1034 parser exited. client mac= 5c:a:5b:60:f1:a7 bytes parsed = 0 and bytes read = 440
*webauthRedirect: Mar 24 08:57:47.008: #EMWEB-6-HTTP_REQ_BEGIN_ERR: http_parser.c:579 http request should begin with a character
*webauthRedirect: Mar 24 08:57:45.453: #EMWEB-6-PARSE_ERROR: webauth_redirect.c:1034 parser exited. client mac= 5c:a:5b:60:f1:a7 bytes parsed = 0 and bytes read = 440
*webauthRedirect: Mar 24 08:57:45.453: #EMWEB-6-HTTP_REQ_BEGIN_ERR: http_parser.c:579 http request should begin with a character
All I am interested in is the #PEM-6-GUESTIN line. I need to take the user id , mac and IP address from this line and put it in a csv. Only log lines with that status are required.
This is my first time working with shell scripts and all your help would be appreciated.
I think it is easier using grep to filter + sed to get groups using regex:
grep "#PEM-6-GUESTIN" log.txt | sed -r "s/.*user account \((.*)\).* MAC address (.*), IP address (.*)\.\*\*.*/\1,\2,\3/"
And the output is in CSV format:
mrathi_dev,00:1e:65:39:10:8e,192.168.133.146
guys,i have a problem on using xinetd,the error message is 'xinetd[20126]: execv( /home/fulu/download/mysqlchk_status2.sh ) failed: Exec format error (errno = 8)'
the system operation is : CentOS release 6.2;
i installed the xinetd by the command 'sudo yum install xinetd'
i edited the /etc/services, add my port 6033 for my service named 'mysqlchk'
the service 'mysqlchk' in /etc/xinetd.d/mysqlchk is
service mysqlchk
{
disable = no
flags = REUSE
socket_type = stream
port = 6033
wait = no
user = fulu
server = /home/fulu/download/mysqlchk_status2.sh
log_on_failure += USERID
}
the shell file /home/fulu/download/mysqlchk_status2.sh content is
echo 'test'
6.i can run the command /home/fulu/download/mysqlchk_status2.sh straightly and get the result 'test'
when i telnet 127.0.0.1 6033,i get the output
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
Connection closed by foreign host.
then i tail the log file /var/log/messages,it shows
Apr 22 22:01:47 AY1304111122016 xinetd[20001]: START: mysqlchk pid=20126 from=127.0.0.1
Apr 22 22:01:47 AY1304111122016 xinetd[20126]: execv( /home/fulu/download/mysqlchk_status2.sh ) failed: Exec format error (errno = 8)
Apr 22 22:01:47 AY1304111122016 xinetd[20001]: EXIT: mysqlchk status=0 pid=20126 duration=0(sec)
i don't know why,can anybody help me ?
I'm sorry, after questioning it i suddenly found the answer. If you want the shell to be run in other program you need add '#!/bin/echo' at the first line of the shell file (of course the echo can be changed)