I have a Spring boot + sleuth based application. All works as expected. I have for now logs like this:
2017-05-04 17:55:52.226 INFO [alert,692d0eeca479e216,c3c8b680dc29ad02,false] 17292 --- [cTaskExecutor-1] c.k.a.b.s.alert.impl.AlertServiceImpl : Alert state to process: xxx
Now, I want to add custom MDC to my log like the contract reference for example. I want to have logs like this:
2017-05-04 17:55:52.226 INFO [alert,692d0eeca479e216,c3c8b680dc29ad02,false] [CONTRACT_REF] 17292 --- [cTaskExecutor-1] c.k.a.b.s.alert.impl.AlertServiceImpl : Alert state to process: xxx
I tried various things with no success:
Use the Spring Sleuth Tracer to add a tag;
Add logging.pattern.level=%5p %mdc to my application.properties file with MDC.put(xxx, xxx)
How can I add custom MDC/tags to my log?
For versions before 2.x, You have to create your own implementation of a SpanLogger. The easiest way will be to extend the Slf4jSpanLogger and provide your own code to add / update and remove the entries from MDC context. Then you can change your logging pattern and that way your logs will contain what they need.
I was able to add data to the MDC fairly easily by doing MDC.put("yourCoolKey", "your cool value") (see MDC.put JavaDoc).
Once you put the value into the MDC, you can use the sequence %X{yourCoolKey} in your logging pattern (in my case, the value of logging.pattern.console) to print the string "your cool value" as part of each log statement.
Optionally, you can specify a default value in the pattern string by adding :-<defaultValue> after the key, such as %X{yourCoolKey:-N/A}, which will print the string "N/A" whenever the MDC does not have an entry for "yourCoolKey". The default, if not specified, is a blank string ("")
Related
I'm working in a Spring Boot Application using Gradle. I'm probably going to be giving a lot of unnecessary context here, but I'm not sure where my problem is.
I have an SLF4j logger that's resolving to a org.slf4j.impl.Log4jLoggerAdapter. I'm trying to change the log pattern layout for the console logs. The default pattern looks like:
2022-03-02 17:42:48.892 [ INFO] 19296 --- [ main] ggestions.jvm.JVMLatestVersionSuggestion
In the console output, the level, timestamp, and class path are all highlighted. However, I really don't like the format, and would like to overwrite it.
I've added the following to my application.yml:
logging.pattern.console: '%d [%t] %-5level %logger{36} - %m%n'
Which is giving me the layout that I want, but the highlighting disappears and it's all default color. I've seen advice to add %highlight, but that doesn't correctly display - it just shows up as the word %highlight in the output, like so:
2022-03-02 17:50:09,525 [main] %highlight(INFO ) com.indeed.common.boot.suggestions.jvm.JVMLatestVersionSuggestion
The logger is still resolving to org.slf4j.impl.Log4jLoggerAdapter. My guess is that it's because Log4j doesn't support highlighting, but then I'm not clear what was making it work in the first place. Is it possible to get it back?
(note: this is work code, and I don't have the ability to change the Slf4j implementation to Log4j2)
logging:
level:
root: info
com:
demo:
mapper: debug
insertBigData: info
I want to see my mybatis sql in debug model, but there a function which inser a huge value, and I want to ignore this special function, but yml seem don't support config this style
I know move the function to another mapper is a way. what else can I do to ignore log of function insertBigData
It might not be ideal, but specifying partially-qualified key seems to be working.
logging:
level:
root: info
com:
demo:
mapper: debug
mapper.insertBigData: info
I'm working on a POC and was able to integrate 2 microservices with JaegerUI.
Request to an endpoint in serviceA calls an endpoint in serviceB and returns a response.
I have used below dependencies:
spring.boot.version : 2.1.4.RELEASE
<groupId>io.opentracing.contrib</groupId>
<artifactId>opentracing-spring-jaeger-web-starter</artifactId>
<version>3.1.1</version>
<groupId>io.opentracing.contrib</groupId>
<artifactId>opentracing-spring-cloud-starter</artifactId>
<version>0.4.0</version>
Spring autoconfiguration takes care of everything so just added the required properties:
opentracing.jaeger.http-sender.url
opentracing.jaeger.service-name
opentracing.jaeger.enabled
opentracing.spring.cloud.async.enabled
I want to achieve the below:
I want to add application logs to the span so that they are visible in JaegerUI.
I want to add some fields to span tags so that it's easy to search in JaegerUI.
Also, I want the spanId and traceId to the application log.
Is it possible to search in JaegerUI based on spanId/traceId? If yes, how?
Based on the answer in below SO question:
How to enrich Jaeger opentracing data with the application logs (produced by slf4j) for Spring Boot?
opentracing-spring-cloud-starter dependency should automatically take care of sending app logs to span in JaegerUI.
I have a log statement like below in serviceA:
logger.info("sending request to serviceB.");
But above log is not getting captured in corresponding span and not visible in JaegerUI.
Any suggestions on how to achieve the above scenarios are appreciated!
I was studying Opentracing and Jeager and I've used this tutorial to get familiar with the basic possibilities:
https://github.com/yurishkuro/opentracing-tutorial/tree/master/java
If you take a look in the case 1 (Hello World), it explains how to "Annotate the Trace with Tags and Logs".
That would answer your questions 1, 2 and 3, as with that you can add all the info that you would like within spans and logs.
Here is a snippet from the repository (but I'd recommend checking there, as it has a more detailed explanation):
Span span = tracer.buildSpan("say-hello").start();
span.setTag("hello-to", helloTo);
In this case helloTo is a variable containing a name, to whom the app will say hello. It would create a span tag called hello-to with the value that is coming from the execution.
Below we have an example for the logs case, where the whole helloStr message is added to the logs:
// this goes inside the sayHello method
String helloStr = String.format("Hello, %s!", helloTo);
span.log(ImmutableMap.of("event", "string-format", "value", helloStr));
System.out.println(helloStr);
span.log(ImmutableMap.of("event", "println"));
Regarding the last question, that would be easier, you can use the Jaeger UI to search for the trace you would like, there is a field for that on the top left corner:
There you go.
I want to add application logs to the span so that they are visible in JaegerUI.
Span span = tracer.buildSpan("my-span-name").start();
span.setTag("my-tag-name", "my-tag-value");
There are various overloaded methods as follows
Span setTag(String key, String value);
Span setTag(String key, boolean value);
Span setTag(String key, Number value);
I want to add some fields to span tags so that it's easy to search in JaegerUI.
Jaeger API provides log method to log multiple fields that needs to be added to a map, the method signature is as follows,
Span log(Map<String, ?> fields);
eg:
span.log(
ImmutableMap.Builder<String, Object>()
.put("event", "soft error")
.put("type", "cache timeout")
.put("waited.millis", 1500)
.build()
);
Also, I want the spanId and traceId to the application log.
spanId and traceId are stored in JaegerSpanContext class, which can be obtained from context() method of Span class.
JaegerSpanContext spanContext = (JaegerSpanContext)sprintSpan.context();
long spanId = spanContext.getSpanId();
long traceId = spanContext.getTraceId();
Is it possible to search in JaegerUI based on spanId/traceId? If yes, how?
There is a search box in the navigation bar of Jaeger UI where you can search traces by trace ID.
I am using Sleuth 2.1.3.
I want to add a custom "trace ID" as "correlation id" with alpha numeric value and want to spit in logs with spanid and parent id.
If i use below implementation for creating new custom trace id. does it get printed in logs ?
I tried below implementation but does not see any custom trace in log
https://github.com/openzipkin/zipkin-aws/blob/release-0.11.2/brave-propagation-aws/src/main/java/brave/propagation/aws/AWSPropagation.java
Tracing.newBuilder().propagationFactory(
ExtraFieldPropagation.newFactoryBuilder(B3Propagation.FACTORY)
.addField("x-vcap-request-id")
.addPrefixedFields("x-baggage-", Arrays.asList("country-code", "user-id"))
.build()
);
I tried with above code from https://cloud.spring.io/spring-cloud-sleuth/reference/html/#propagation but didnt see any custom trace id in log
You've passed in the B3Propagation.FACTORY as the implementation of the propagation factory so you're explicitly stating that you want the default B3 headers. You've said that you want some other field that is alphanumeric to be also propagated. Then in a log parsing tool you can define that you want to use your custom field as the trace id, but it doesn't mean that the deafult X-B3-TraceId field will be changed. If you want to use your custom field as trace id that Sleuth understands, you need to change the logging format and implement a different propagation factory bean.
One of the way which worked for me is
using ExtraFieldPropagation
and adding those keys in sleuth properties under propagation-keys
and whitelisted-keys
sample code
' #Autowired Tracer tracer;
Span currentSpan = tracer.nextSpan().start();
ExtraFieldPropagation.set(
"customkey", "customvalue");
sleuth:
log:
slf4j:
whitelisted-mdc-key : customkey
propagation:
tag:
enabled: true
propagation-keys : customkey '
I have configured spring security with a ldap server (but continue reading, it's not a problem if you have no knowledge about it, this is really a spring problem). All runs like a charm. Here is the line I use for that:
<ldap-server ldif="" root="" manager-dn="" manager-password="" url="" id="ldapServer" />
If I fill ldif and root attributes, it will run an embeded server:
<ldap-server ldif="classpath://ldap.ldif" root="dc=springframework,dc=org" manager-dn="" manager-password="" url="" id="ldapServer" />
If I fill other fields, it will run a distant server:
<ldap-server ldif="" root="" manager-dn="dc=admin,dc=springframeworg,dc=org" manager-password="password" url="ldap://myldapserver.com/dc=springframeworg,dc=org" id="ldapServer" />
All this stuff run correctly. Now I want to use Spring mechanism to load such parameters from a property file:
So I replace attribute values like this:
<ldap-server ldif="${ldap.ldif.path}" root="${ldap.ldif.root}" manager-dn="${ldap.server.manager.dn}" manager-password="${ldap.server.manager.password}" url="${ldap.server.url}" id="ldapServer" />
and create a property file with:
ldap.server.url=
ldap.server.manager.dn=
ldap.server.manager.password=
ldap.ldif.path=
ldap.ldif.root=
Now, the funny part of the problem. If I fill the following properties in the file:
ldap.server.url=ldap://myldapserver.com/dc=springframeworg,dc=org
ldap.server.manager.dn=dc=admin,dc=springframeworg,dc=org
ldap.server.manager.password=password
ldap.ldif.path=
ldap.ldif.root=
It runs a distant server as expected.
If I fill the property file like this:
ldap.server.url=
ldap.server.manager.dn=
ldap.server.manager.password=
ldap.ldif.path= classpath:ldap.ldif
ldap.ldif.root= dc=springframeworg,dc=org
It does not run, complaining that the ldap url is missing. But the problem is that if I change the spring configuration from:
<ldap-server ldif="${ldap.ldif.path}" root="${ldap.ldif.root}" manager-dn="${ldap.server.manager.dn}" manager-password="${ldap.server.manager.password}" url="${ldap.server.url}" id="ldapServer" />
to (by just removing the reference to the variable ${ldap.server.url})
<ldap-server ldif="${ldap.ldif.path}" root="${ldap.ldif.root}" manager-dn="${ldap.server.manager.dn}" manager-password="${ldap.server.manager.password}" url="" id="ldapServer" />
It runs !
My thoughs are that spring does not replace the attribute value with the property config one if this one is empty. But I find it strange.
Can you give me some clue to understand that ? And what's the best to do to configure my ldap server via a property file ?
EDIT: this is due to a poor design choice (look at accepted answer), an issue has been opened on jira :
https://jira.springsource.org/browse/SEC-1966
Ok, I think this is a spring security bug.
If I debug and look at the class LdapServerBeanDefinition, there is a method called "parse". Here is an extract:
public BeanDefinition parse(Element elt, ParserContext parserContext) {
String url = elt.getAttribute(ATT_URL);
RootBeanDefinition contextSource;
if (!StringUtils.hasText(url)) {
contextSource = createEmbeddedServer(elt, parserContext);
} else {
contextSource = new RootBeanDefinition();
contextSource.setBeanClassName(CONTEXT_SOURCE_CLASS);
contextSource.getConstructorArgumentValues().addIndexedArgumentValue(0, url);
}
contextSource.setSource(parserContext.extractSource(elt));
String managerDn = elt.getAttribute(ATT_PRINCIPAL);
String managerPassword = elt.getAttribute(ATT_PASSWORD);
if (StringUtils.hasText(managerDn)) {
if(!StringUtils.hasText(managerPassword)) {
parserContext.getReaderContext().error("You must specify the " + ATT_PASSWORD +
" if you supply a " + managerDn, elt);
}
contextSource.getPropertyValues().addPropertyValue("userDn", managerDn);
contextSource.getPropertyValues().addPropertyValue("password", managerPassword);
}
...
}
If I debug here, all variables (url, managerDn, managerPassword...) are not replaced by the value specified in the property file. And so, url has the value ${ldap.server.url}, managerDn has the value ${ldap.server.manager.dn} and so on.
The method parse creates a bean, a context source that will be used further. And when this bean will be used, place holders will be replaced.
Here, we got the bug. The parse method check if url is empty or not. The problem is that url is not empty here because it has the value ${ldap.server.url}. So, the parse method creates a context source as a distant server.
When the created source will be used, it will replace the ${ldap.server.url} by empty value (like specified in the property file). And....... Bug !
I don't know really how to solve this for the moment, but I now understand why it bugs ;)
I cannot explain it, but I think you can fix your problem using defaulting syntax, available since Spring 3.0.0.RC1 (see).
In the chageg log you can read: PropertyPlaceholderConfigurer supports "${myKey:myDefaultValue}" defaulting syntax
Anyway, I think that the problem is because "" is valid value, but no value in the property file don't.
I think that url="" works because url attribute is of type xs:token in spring-security XSD and empty string is converted to null (xs:token is removing any leading or trailing spaces, so "" can be recognized as no value). Maybe the value of ${ldap.server.url} is resolved as empty string and that is why you've got an error.
You can try use Spring profiles to define different configurations of ldap server (see Spring Team Blog for details about profiles)
I believe there is an issue here while using place holders. The following will most probably solve the problem:
Create a class which extends PropertyPlaceHolderConfigurer and override its method convertPropertyValue()
in the method you can return the property as empty string if you find anything other than a string which is of type LDAP url i.e. ldap://myldapserver.com/dc=springframeworg,dc=org
Also you need to configure your new specialization of class PropertyPlaceHolderConfigurer in the context file.
Hope this helps.
You can define empty String in the application.properties file as following:
com.core.estimation.stopwords=\ \