I've a set of applications installed on a server that I want to start sending logs thru syslog to a remote Logstash server. For that, I've created a single external Logback configuration file containing a SyslogAppender pointing to the desired remote server.
As multiple applications will log to the same server, I've changed the log pattern to be the following:
<suffixPattern>%d{yyyy-MM-dd HH:mm:ss.SSS} [${server_instance}] [${application_name}] %p ${PID:- } --- [%15.15t] %logger : %m%n</suffixPattern>
Where server_instance and application_name are command-line option arguments provided at startup.
Now I just simply set the logging.config property of all my apps pointing to the same Logback config file, and all the applications start sending the logs using the specified pattern to the desired server. That part works like a charm.
But the only problem I've is that Logback isn't able to determine the server_instance and the application_name properties and they appear as [server_instance_IS_NOT_DEFINED] and [application_name_IS_NOT_DEFINED] respectively.
Can I achieve this somehow using a single external configuration file?
Logback isn't able to see command-line arguments; those are seen by Spring alone.
Moved those command-line options arguments to system properties (i.e. "-Dserver_instance" and "-Dapplication_name" instead of "--server_instance" and "--application_name") and now everything's working as expected.
Related
Is there any way to locate, copy, or manipulate logs of task execution, in SCDF, in local?
I'm currently seeing logs whenever I execute batch (or not batch) task in cmdline of shell where I've started dataflow server locally. In both CentOS 7 and Windows 10, it says that it located their stdout/stderr logs inside
/tmp (temp in windows)/[SOME_NUMBER_I_DON'T_KNOW]/${task_name_i_defined}_${SOME_HEX_CODE_RELATED_TO_TASK_EXECUTION_ID}
I want to use that information whenever I need.
Passing properties to dataflow jar doesn't work. It just creates a file, writes that file over and over at each task execution, unlike storing each task execution at different folder.
Modifying properties like loggig.file.path at task lauching configurations doesn't work, either. Only stdout of task is made with the name of 'spring.log', at specific location i designated. Behavior is same as above case.
Spring Cloud Data Flow Task logs
I looked at this answer, but it does not work, either...
I know there are a lot of parameters that I could pass to dataflow or tasks. I don't think none of them could satisfy this condition. Please enlighten me.
The only configuration property available to effect the log location is the working-directories-root deployer property.
Because it is a deployer property, it can not simply be set as spring.cloud.deployer.local.working-directories-root.
It can set at task launch time and prefixed w/ deployer.*.local (details).
It can also be configured globally via the "task platform" properties (details).
When configured at the platform level, it can be done in yml:
spring:
cloud:
dataflow:
task:
platform:
local:
accounts:
default:
working-directories-root: /Users/foo/logz
or via an env var:
SPRING_CLOUD_DATAFLOW_TASK_PLATFORM_LOCAL_ACCOUNTS_DEFAULT_WORKING_DIRECTORIES_ROOT=/Users/foo/logz
Details
The STDOUT log location is created at <task-work-dir>/stdout.log (details).
The <task-work-dir> is defined as:
<working-directories-root> / System.nanoTime() / <taskLaunchId>
(details)
The <working-directories-root> is the value of the
working-directories-root local deployer property or the "java.io.tmpdir" system property when the local deployer property is not specified.
Spring web application with log4j2 implemented in tomcat 8 running in windows and linux servers.
In windows server
CATALINA_OPTS specified in setenv.bat file
set "CATALINA_OPTS=-DLOG_FOLDER=D:\apache-tomcat-9.0.22\customlog"
log4j2.properties file is inside WEB-INF\classes\config\
appender.rolling.fileName=${sys:LOG_FOLDER}\logger.log
appender.rolling.filePattern=${sys:LOG_FOLDER}\logging-%d{MM-dd-yyyy}-%i.log.gz
In linux server
CATALINA_OPTS specified in setenv.sh file
set "CATALINA_OPTS=-DLOG_FOLDER=/opt/tomcat01/customlog"
log4j2.properties file is inside WEB-INF\classes\config\
appender.rolling.fileName=${env:LOG_FOLDER}/logger.log
appender.rolling.filePattern=${env:LOG_FOLDER}/logging-%d{MM-dd-yyyy}-%i.log.gz
log folder is always created like '${env:LOG_FOLDER}' or '${sys:LOG_FOLDER}'
You should use ${sys:LOG_FOLDER} in both cases:
${sys:property_name} retrieves the Java system property of that name,
${env:variable_name} retrieves the OS environment variable of that name.
See Log4j 2 Lookups for more details.
There are also other problems in your configuration:
if you start Tomcat on Windows as a service, setenv.bat is not used. You should add -DLOG_FOLDER=D:\apache-tomcat-9.0.22\customlog to the Java tab of the Tomcat Monitor application (bin\tomcat<version>w.exe).
on Linux you are using the wrong syntax to modify an environment variable (Microsoft's syntax), use:
export CATALINA_OPTS="-DLOG_FOLDER=/opt/tomcat01/customlog"
instead.
Remark: If you want to use environment variables instead of system properties, you can use:
export LOG_FOLDER="/opt/tomcat01/customlog"
on Linux, while on Windows you need to run:
tomcat10.exe //US ++Environment="LOG_FOLDER=D:\apache-tomcat-9.0.22\customlog"
See Procrun documentation for more details.
I need, for performance reasons, get rid of org.jooq.tools.LoggerListener DEBUG log messages in Spring Boot application running inside Docker. None of the Spring Boot options like (Docker) env variable LOGGING_LEVEL_ORG_JOOQ=INFO in docker-compose.yml or Java system property -Dlogging.level.org.jooq=INFO passed to docker container in entry.sh do not remove these DEBUG messages reporting query execution details. Both option have been checked at Docker container level.
Even custom logback-perf.xml conf file, as in https://github.com/jOOQ/jOOQ/blob/master/jOOQ-examples/jOOQ-spring-boot-example/src/main/resources/logback.xml with DEBUG->INFO, pointed by LOGGING_CONFIG env var from docker-compose.yml does not prevent these debug messages. I have verified that the custom logback-perf.xml conf file is in use by changing appender patterns.
The best way to remove those messages in jOOQ directly, is to specify Settings.executeLogging = false, see here.
Obviously, there are also ways to set up loggers correctly, but I cannot see what you did from your description, or why that failed.
Need to know where can I specify the spring boot port aa when I check my service status I get this:
"Serverkt.log started at server-name with process id"
But I don't get the port number.
Also wanted to know where I can find the spring logs in the server
spring boot has its configuration file. I guess that you have it included in your jar/war file. So basically unzip that file and look inside it (try to search for application.properties or application.yaml|yml).
property server.port defines the port on which application is running. It defaults to 8080.
If you are using spring-boot 2 then with property logging.path you can change the path where output file will be placed. However I don't know if this works when you have logback/log4j/... configuration.
if you run your application you can override those properties specified in appplication.properties|yaml by providing command line properties. For example you can change the port with command java -jar your-boot.jar --server.port=9090
Since you're starting spring boot from a service file, you can set the port using command line arguments e.g.port 8083 would look like this:
ExecStart=/usr/bin/java -jar /opt/corda/spring.jar --server.port=8083
In our projects we have a strange problem with duplicate log entries in the log file.
We have multiple appenders but a single logger.
If the spring boot application is started on local machine using java -jar the problem is not reproducible.
The problem occurs only when the application started as a service.
How can i solve the problem.
The problem occurs only if a file appender configured and if the spring boot application started using /etc/init.d/ symlink.
The spring boot's default start script redirects all console logs into the configured log file.
As a result both the logback logger and start scripts writes in the same file, thus we see duplicate entries in the log file.
Using systemctl (or setting the LOG_FILE or LOG_FOLDER environment variables) will solve this problem.
If you cannot switch to systemd you can set the environment variables so that all stdout&stderr messages redirected to /dev/null:
export LOG_FOLDER=/dev
export LOG_FILENAME=null