How do consoleSource and consoleLogLevel relate to the log settings in server.xml? - websphere-liberty

The documentation at https://www.ibm.com/support/knowledgecenter/en/SSEQTP_liberty/com.ibm.websphere.wlp.doc/ae/rwlp_logging.html lists the following config snippet for configuring logs to go to the console in JSON format (a useful configuration when Liberty is running inside a Linux container like in a kubernetes pod):
com.ibm.ws.logging.console.format=json
com.ibm.ws.logging.console.log.level=info
com.ibm.ws.logging.console.source=message,trace,accessLog,ffdc,audit
However, I noticed that when I increase the log level inside my server.xml (say to FINER to enable tracing), the trace messages are also emitted to the console. I assume its because the source is configured to pick up messages from the trace, but what I don't understand is whether/how that interacts with the consoleLogLevel setting.
Why specify a com.ibm.ws.logging.console.log.level if its just going to include everything from message and trace log files anyway?

The com.ibm.ws.logging.console.source bootstrap property controls which sources the console handler will subscribe to. This means, if there is any trace (typically levels FINE / FINER / FINEST) enabled by the traceSpecification, that content will be consumed by the console handler and output as JSON to console.
So, setting...
com.ibm.ws.logging.console.source=message,trace,accessLog,ffdc,audit
means that the console handler will receive any log, trace, access log, ffdc, and audit events that are generated by the system.
If com.ibm.ws.logging.console.log.level is specified, it controls which log events are filtered out by the console handler. By setting com.ibm.ws.logging.console.log.level=WARNING you won't see INFO level messages in the console output unless you also explicitly set consoleLogLevel=info in the server.xml.
Note that logging settings explicitly set in the <logging> element in the server.xml (which is processed when the config manager is ready) override any configuration set using environment variables or the bootstrap.properties file (which are processed very early in the server startup process). In practice this means you should use bootstrap.properties or environment variables for logging configuration, and just use the logging element in the server.xml file to specify things you want to override while the server is running.

Related

Tomcat on windows log shipping to logstash

I am trying to configure log shipping/ consolidation using logstash. My tomcat servers run on Windows. I am running into a few problems with my configuration - Tomcat on windows, logging using log4j, redis consolidator/ elasticsearch/ logstash/ kibana running on a single linux server.
Fewer log shippers available on Windows. It looks like nxlog does not work with redis out of the box. So, I have reverted to using logstash to ship. I would like to learn what others prefer to use
Rather use custom appenders I would rather have tomcat use log4j to log to file and then feed the file as input to be shipped to Redis. I don't want to the log formats.
No json-event format for me - http://spredzy.wordpress.com/2013/03/02/monitor-your-cluster-of-tomcat-applications-with-logstash-and-kibana/. I can't seem to get the right file config in the shipper.conf
Any sample config for log4j files - fed to logstash via redis would help.
Thanks
I'm currently writing a Java library to send logs to Logstash using ZeroMQ (no central redis broker required).
Disclaimer: it's not quite perfect yet, but may be worth keeping an eye on.
https://github.com/stuart-warren/logit
You can setup the standard juli log configuration (or log4j if you are using that), plus with the tomcat-valve jar you can send access logs as well by configuring the server.xml.
It does however send it in json-event format by default.
I'm confused as to why you wouldn't want to save all the processing on the Logstash server? You can (and currently probably should) log to file in standard format as well.
logging.properties file.
# "handlers" specifies a comma separated list of log Handler
# classes. These handlers will be installed during VM startup.
# Note that these classes must be on the system classpath.
# By default we only configure a ConsoleHandler, which will only
# show messages at the INFO and above levels.
handlers= com.stuartwarren.logit.jul.ZmqAppender
# handlers= com.stuartwarren.logit.jul.ZmqAppender, java.util.logging.ConsoleHandler
# Default global logging level.
# This specifies which kinds of events are logged across
# all loggers. For any given facility this global level
# can be overriden by a facility-specific level.
# Note that the ConsoleHandler also has a separate level
# setting to limit messages printed to the console.
.level=INFO
# Limit the messages that are printed on the console to INFO and above.
com.stuartwarren.logit.jul.ZmqAppender.level=INFO
com.stuartwarren.logit.jul.ZmqAppender.socketType=PUSHPULL
com.stuartwarren.logit.jul.ZmqAppender.endpoints=tcp://localhost:2120
com.stuartwarren.logit.jul.ZmqAppender.bindConnect=CONNECT
com.stuartwarren.logit.jul.ZmqAppender.linger=1000
com.stuartwarren.logit.jul.ZmqAppender.sendHWM=1000
com.stuartwarren.logit.jul.ZmqAppender.layout=com.stuartwarren.logit.jul.Layout
com.stuartwarren.logit.jul.Layout.layoutType=logstashv1
com.stuartwarren.logit.jul.Layout.detailThreshold=WARNING
com.stuartwarren.logit.jul.Layout.tags=tag1,tag2,tag3
com.stuartwarren.logit.jul.Layout.fields=field1:value1,field2:value2,field3:value3
java.util.logging.ConsoleHandler.level = FINE
java.util.logging.ConsoleHandler.formatter = java.util.logging.SimpleFormatter
server.xml
<Valve className="com.stuartwarren.logit.tomcatvalve.ZmqAppender"
layout="com.stuartwarren.logit.tomcatvalve.Layout"
socketType="PUSHPULL"
endpoints="tcp://localhost:2120"
bindConnect="CONNECT"
linger="1000"
sendHWM="1000"
layoutType="logstashv1"
iHeaders="Referer,User-Agent"
oHeaders=""
cookies=""
tags="tag1,tag2,tag3"
fields="field1:value1,field2:value2,field3:value3" />

How does one run Spring XD in distributed mode?

I'm looking to start Spring XD in distributed mode (more specifically deploying it with BOSH). How does the admin component communicate to the module container?
If it's via TCP/HTTP, surely I'll have to tell the admin component where all the containers are? If it's via Redis, I would've thought that I'll need to tell the containers where the Redis instance is?
Update
I've tried running xd-admin and Redis on one box, and xd-container on another with redis.properties updated to point to the admin box. The container starts without reporting any exceptions.
Running the example stream submission curl -d "time | log" http://{admin IP}:8080/streams/ticktock yields no output to either console, and not output to the logs.
If you are using the xd-container script, then the redis.properties is expected to be under "XD_HOME/config" where XD_HOME points the base directory where you have bin, config, lib & modules of xd.
Communication between the Admin and Container runtime components is via the messaging bus, which by default is Redis.
Make sure the environment variable XD_HOME is set as per the documentation; if it is not you will see a logging message that suggests the properties file has been loaded correctly when it has not:
13/06/24 09:20:35 INFO support.PropertySourcesPlaceholderConfigurer: Loading properties file from URL [file:../config/redis.properties]

Websphere application server logging in SystemOut.log

By default, log, error, and trace information for all processes and applications on a process server is written to the SystemOut.log, But our requirement is to only log Request and Responses . Is there any setup in admin console to do this?
Thanks in advance.....
When WebSphere starts, it designates the SystemOut.log file as the file into which all System.out prints will go to. Therefore, whenever any code is issuing, for example, System.out.println, the output will end up going into SystemOut.log - and that is true for both your application code and WebSphere's internal code.
To achieve the effect that you're looking for, consider using some logging framework, such as Log4J, SLF4J or Java's standard logging API.

Websphere Name Space Bindings affect the JNDI scope resolution order

Here is the situation:
I'm on websphere Network Deployment v8.0.0.3
I've an application which use 2 queuese
One Queue is for internal application usage (Publisher and Consumer are inside the same application) the other queue is used by other modules deployed on Other application server inside the cell.
So I have configured the 1st queue at Cell Scope level and the second queue at Cluster scope level.
Everything was working until I added a Name Space Binding.
After that every jms jndi object Cluster Scoped are not present anymore inside a dumpNameSpace.sh output.
Seams like the resolution of the scopes are modified by the presence of a Name Space Binding.
Which indeed is really odd but I got the same behaviour on 2 different installations of WAS.
Thanks for anyone which knows this.
Update
This is the diff between the jndi dump that works and the one which not.
--- clsdumpOk 2012-08-07 11:49:43.000000000 +0200
+++ clsdumpKo2 2012-08-07 11:49:59.000000000 +0200
## -454,28 +454,12 ##
(top)/clusters/TestCluster/jdbc/modulobase
(top)/clusters/TestCluster/jms
(top)/clusters/TestCluster/jms/as
-(top)/clusters/TestCluster/jms/as/BatchRequest
-(top)/clusters/TestCluster/jms/as/BatchResponse
(top)/clusters/TestCluster/jms/as/ciccio
-(top)/clusters/TestCluster/jms/as/FSCleaner
(top)/clusters/TestCluster/jms/as/License
(top)/clusters/TestCluster/jms/as/Mailer
-(top)/clusters/TestCluster/jms/as/Plans
-(top)/clusters/TestCluster/jms/as/RiaResponse
-(top)/clusters/TestCluster/jms/ConnectionFactory
-(top)/clusters/TestCluster/jms/pac
-(top)/clusters/TestCluster/jms/pac/as
-(top)/clusters/TestCluster/jms/pac/as/Events
(top)/clusters/TestCluster/jms/queue
-(top)/clusters/TestCluster/jms/queue/batch-request
-(top)/clusters/TestCluster/jms/queue/batch-response
-(top)/clusters/TestCluster/jms/QueueConnectionFactory
-(top)/clusters/TestCluster/jms/queue/events
-(top)/clusters/TestCluster/jms/queue/filesystem-cleaner
(top)/clusters/TestCluster/jms/queue/license
(top)/clusters/TestCluster/jms/queue/mailer
-(top)/clusters/TestCluster/jms/queue/plans
-(top)/clusters/TestCluster/jms/TopicConnectionFactory
(top)/clusters/TestCluster/jta
(top)/clusters/TestCluster/jta/usertransaction
(top)/clusters/TestCluster/SecurityServer
## -495,8 +479,10 ##
(top)/clusters/TestCluster/url/casCfgFile
(top)/clusters/TestCluster/UserRegistry
(top)/clusters/TestCluster/wb25
-(top)/clusters/TestCluster/wb25/topic
-(top)/clusters/TestCluster/wb25/topic/ria-response
+(top)/clusters/TestCluster/wb25/conf
+(top)/clusters/TestCluster/wb25/conf/locking
+(top)/clusters/TestCluster/wb25/conf/locking/lockingEnabled
+(top)/clusters/TestCluster/wb25/conf/rootFolder
(top)/clusters/TestCluster/wm
(top)/clusters/TestCluster/wm/ard
(top)/clusters/TestCluster/wm/default
as you can see once the
+(top)/clusters/TestCluster/wb25/conf/locking/lockingEnabled
is added
all the rest is removed.
It's really weird.
The problem in WebSphere for Environment -> Naming -> Namespace Bindings
is the following:
If you set some JNDI nodes with the naming:
url/someVariable
or generically
something/someVar
and than you use the same "something" for some other object like:
URL
or
JDBC
or
JMS
The Name space binding put "something" in read-only mode and when WebSphere try to configure other resources will fail.
You cannot spot this during configuration because only at the first reboot of the Application Server you will get this.
So be carefull when chose names inside jndi.

Can I dynamically update the websphere 's log configuration in my own code?

In websphere admin console, we can change the log configuration , for example, the log file name, rolloverSize and log level and so on. After we saved the change in console, if we choose run time and chose write the file to xml. In this case, the log configuraiton change will be write to config file (such as: \config\cells\SENLI-XPNode01Cell\nodes\SENLI-XPNode01\servers\server1\server.xml)
You know websphere support the dynamically change the log configuraiton, after we changed it, it will take effect immidiately.
My quesiton is that in my own code, is there any websphere public api we can invoke to implement the dynamically change the log configuraiton?
You use the AdminClient to look up the TraceService MBean and update the trace string via that interface.
To modify the values saved in xml you can use the ConfigService to programmatically. I'll warn you that this approach isn't for the faint of heart.

Resources