Micrometer filter is ignored with CompositeMeterRegistry - spring-boot

I use Spring Boot 2.1.2.RELEASE, and I try to use Micrometer with CompositeMeterRegistry. My goal is to publish some selected meters to ElasticSearch. The code below shows my sample config. The problem is, that the filter is completely ignored (so all metrics are sent to ElasticSearch), although I can see in the logs that it was processed ("filter reply of meter ..." lines).
Strangely, if I define the MeterFilter as a Spring bean, then it's applied to ALL registries (however, I want it to be applied only on "elasticMeterRegistry").
Here is a sample configuration class:
#Configuration
public class AppConfiguration {
#Bean
public ElasticConfig elasticConfig() {
return new ElasticConfig() {
#Override
#Nullable
public String get(final String k) {
return null;
}
};
}
#Bean
public MeterRegistry meterRegistry(final ElasticConfig elasticConfig) {
final CompositeMeterRegistry registry = new CompositeMeterRegistry();
registry.add(new SimpleMeterRegistry());
registry.add(new JmxMeterRegistry(new JmxConfig() {
#Override
public Duration step() {
return Duration.ofSeconds(10);
}
#Override
#Nullable
public String get(String k) {
return null;
}
}, Clock.SYSTEM));
final ElasticMeterRegistry elasticMeterRegistry = new ElasticMeterRegistry(elasticConfig, Clock.SYSTEM);
elasticMeterRegistry.config().meterFilter(new MeterFilter() {
#Override
public MeterFilterReply accept(Meter.Id id) {
final MeterFilterReply reply =
id.getName().startsWith("logback")
? MeterFilterReply.NEUTRAL
: MeterFilterReply.DENY;
log.info("filter reply of meter {}: {}", id.getName(), reply);
return reply;
}
});
registry.add(elasticMeterRegistry);
return registry;
}
}
So, I expect ElasticSearch to receive only "logback" metrics, and JMX to receive all metrics.
UPDATE:
I have played with filters and found a "solution", but I don't really understand why the code above doesn't work.
This works:
elasticMeterRegistry.config().meterFilter(new MeterFilter() {
#Override
public MeterFilterReply accept(Meter.Id id) {
final MeterFilterReply reply =
id.getName().startsWith("logback")
? MeterFilterReply.ACCEPT
: MeterFilterReply.DENY;
log.info("filter reply of meter {}: {}", id.getName(), reply);
return reply;
}
});
The difference is: I return ACCEPT instead of NEUTRAL.
Strangely, the following code does not work (ES gets all metrics):
elasticMeterRegistry.config().meterFilter(
MeterFilter.accept(id -> id.getName().startsWith("logback")));
But this works:
elasticMeterRegistry.config().meterFilter(
MeterFilter.accept(id -> id.getName().startsWith("logback")));
elasticMeterRegistry.config().meterFilter(
MeterFilter.deny());
CONCLUSION:
So, it seems that instead of NEUTRAL, the filter should return ACCEPT. But for meters not starting with "logback", my original filter (with NEUTRAL) returns DENY. Then why are those metrics published to ElasticSearch registry?
Can someone explain this?

This is really a composite of questions. I'll just point out a few points.
For the MeterRegistry bean you defined, Spring Boot will auto-configure an ElasticMeterRegistry bean as there's no ElasticMeterRegistry bean. Instead of creating a CompositeMeterRegistry bean on your own, just define a custom ElasticMeterRegistry bean which is applied the MeterFilter you want and let Spring Boot create one (CompositeMeterRegistry bean) for you.
For MeterFilterReply, ACCEPT will accept the meter immediately, DENY will deny the meter immediately, and NEUTRAL will postpone the decision to next filter(s). Basically meters will be accepted unless there's any DENY.

Related

Add tags to every span of application using spring sleuth

I am trying to add some tags to all of my traces going though my application. I tried without success to extends the DefaultMessageSpanCustomizer as showed in the docs like
#Component
class SpanCustomizerConfig extends DefaultMessageSpanCustomizer {
#Value("${spring.application.name}")
private String appName;
#Override
public Span customizeHandle(
final Span spanCustomizer, final Message<?> message, final MessageChannel messageChannel) {
return super.customizeHandle(spanCustomizer, message, messageChannel)
.tag("app", this.appName)
.tag("container", this.appName)
.tag("component", "api")
.tag("channelName", this.channelName(messageChannel));
}
#Override
public Span.Builder customizeSend(
final Span.Builder builder, final Message<?> message, final MessageChannel messageChannel) {
return super.customizeSend(builder, message, messageChannel)
.tag("app", this.appName)
.tag("container", this.appName)
.tag("component", "api")
.tag("channelName", this.channelName(messageChannel));
}
}
But unfortunately I could not make it work. Does anyone had success trying to add some tags for every application trace ?
Thanks a lot for the help.
Jonathan.
If you're using Sleuth with Brave (default option) you can register your own SpanHandler in the following way to add a tag to every span
#Bean
SpanHandler mySpanHandler() {
return new SpanHandler() {
#Override
public boolean end(TraceContext context, MutableSpan span, Cause cause) {
span.tag("add this tag", "for every span");
return super.end(context, span, cause);
}
};
}
If you're using Spring Cloud Sleuth OTel please register a bean of type SpanExporter and mutate the span. You can check https://github.com/spring-projects-experimental/spring-cloud-sleuth-otel/blob/v1.1.0-M4/spring-cloud-sleuth-otel/src/main/java/org/springframework/cloud/sleuth/otel/bridge/ArrayListSpanProcessor.java for reference. You'll need to override the onStart method where you have access to ReadWriteSpan. There you can modify all of your spans whenever they are started

Implementing axon snapshot with springboot 2.3.3 and axon 4.4.2

can anyone suggest any tutorial/sample project for Implementing Snapshot in AXON 4.4.2 with springBoot 2.3.3.
i went through the documentation(https://docs.axoniq.io/reference-guide/axon-framework/tuning/event-snapshots#snapshotting) and did below:
The AxonConfig.class
#Configuration
public class AxonConfig {
#Bean
public SnapshotTriggerDefinition app1SnapshotTrigger(Snapshotter snapshotter) {
return new EventCountSnapshotTriggerDefinition(snapshotter, 10);
}
}
The Aggregate
#Aggregate(snapshotTriggerDefinition = "app1SnapshotTrigger")
public class MyAggregate {
#AggregateIdentifier
private String id;
private String name;
#AggregateMember
private List<Address> addresses = new ArrayList<>();
private MyAggregate () {
}
#CommandHandler
private MyAggregate (CreateNameCommand createNameCommand) {
-----
}
#EventSourcingHandler
private void on(NameCreatedEvent nameCreatedEvent) {
----
}
Am i missing something. Will it create a snapshot at the threshold value 10.
Thanks.
unfortunately we have no sample demo ready to show in this case.
From your code snippet looks that all is in place. Maybe there is some other configuration that is taking over your annotation.
To give a try, I applied your configuration to our https://github.com/AxonIQ/giftcard-demo/
First note that can guide is the following
if you declared a Repository as we did in https://github.com/AxonIQ/giftcard-demo/blob/master/src/main/java/io/axoniq/demo/giftcard/command/GcCommandConfiguration.java#L17 this configuration will take over Annotation placed into your aggregate. If you prefer annotation, you can remove this Bean definition.
Here the piece of code, instead, to have this configured as a Bean
#Bean
public Repository<GiftCard> giftCardRepository(EventStore eventStore, SnapshotTriggerDefinition giftCardSnapshotTrigger) {
return EventSourcingRepository.builder(GiftCard.class)
.snapshotTriggerDefinition(giftCardSnapshotTrigger)
.eventStore(eventStore)
.build();
}
#Bean
public SpringAggregateSnapshotterFactoryBean snapshotter() {
var springAggregateSnapshotterFactoryBean = new SpringAggregateSnapshotterFactoryBean();
//Setting async executors
springAggregateSnapshotterFactoryBean.setExecutor(Executors.newSingleThreadExecutor());
return springAggregateSnapshotterFactoryBean;
}
#Bean("giftCardSnapshotTrigger")
public SnapshotTriggerDefinition giftCardSnapshotTriggerDefinition(Snapshotter snapshotter) {
return new EventCountSnapshotTriggerDefinition(snapshotter, 10);
}
You can check that your snapshot is working fine looking at client log : after 10 events on the same agggregateId, you should find this info log entry
o.a.a.c.event.axon.AxonServerEventStore : Snapshot created
To check you can use the REST api to retrieve the events from an aggregate
curl -X GET "http://localhost:8024/v1/events?aggregateId=A01"
This will produce a stream containing events starting from the latest Snapshot: you will have nine events listed until the tenth event will be processed. After that, the endpoint will list events from the snapshot.
You can also check /actuator/health endpoint: it will shows the last snapshot token if the showDetails is enabled (enabled by default in EE, not enabled by default in SE).
Corrado.

Cache Kafka Records using Caffeine Cache Springboot

I am trying to cache Kafka Records within 3 minutes of interval post that it will get expired and removed from the cache.
Each incoming records which is fetched using kafka consumer written in springboot needs to be updated in cache first then if it is present i need to discard the next duplicate records if it matches the cache record.
I have tried using Caffeine cache as below,
#EnableCaching
public class AppCacheManagerConfig {
#Bean
public CacheManager cacheManager(Ticker ticker) {
CaffeineCache bookCache = buildCache("declineRecords", ticker, 3);
SimpleCacheManager cacheManager = new SimpleCacheManager();
cacheManager.setCaches(Collections.singletonList(bookCache));
return cacheManager;
}
private CaffeineCache buildCache(String name, Ticker ticker, int minutesToExpire) {
return new CaffeineCache(name, Caffeine.newBuilder().expireAfterWrite(minutesToExpire, TimeUnit.MINUTES)
.maximumSize(100).ticker(ticker).build());
}
#Bean
public Ticker ticker() {
return Ticker.systemTicker();
}
}
and my Kafka Consumer is as below,
#Autowired
CachingServiceImpl cachingService;
#KafkaListener(topics = "#{'${spring.kafka.consumer.topic}'}", concurrency = "#{'${spring.kafka.consumer.concurrentConsumers}'}", errorHandler = "#{'${spring.kafka.consumer.errorHandler}'}")
public void consume(Message<?> message, Acknowledgment acknowledgment,
#Header(KafkaHeaders.RECEIVED_TIMESTAMP) long createTime) {
logger.info("Recieved Message: " + message.getPayload());
try {
boolean approveTopic = false;
boolean duplicateRecord = false;
if (cachingService.isDuplicateCheck(declineRecord)) {
//do something with records
}
else
{
//do something with records
}
cachingService.putInCache(xmlJSONObj, declineRecord, time);
and my caching service is as below,
#Component
public class CachingServiceImpl {
private static final Logger logger = LoggerFactory.getLogger(CachingServiceImpl.class);
#Autowired
CacheManager cacheManager;
#Cacheable(value = "declineRecords", key = "#declineRecord", sync = true)
public String putInCache(JSONObject xmlJSONObj, String declineRecord, String time) {
logger.info("Record is Cached for 3 minutes interval check", declineRecord);
cacheManager.getCache("declineRecords").put(declineRecord, time);
return declineRecord;
}
public boolean isDuplicateCheck(String declineRecord) {
if (null != cacheManager.getCache("declineRecords").get(declineRecord)) {
return true;
}
return false;
}
}
But Each time a record comes in consumer my cache is always empty. Its not holding the records.
Modifications Done:
I have added Configuration file as below after going through the suggestions and more kind of R&D removed some of the earlier logic and now the caching is working as expected but duplicate check is failing when all the three consumers are sending the same records.
`
#Configuration
public class AppCacheManagerConfig {
public static Cache<String, Object> jsonCache =
Caffeine.newBuilder().expireAfterWrite(3, TimeUnit.MINUTES)
.maximumSize(10000).recordStats().build();
#Bean
public CacheLoader<Object, Object> cacheLoader() {
CacheLoader<Object, Object> cacheLoader = new CacheLoader<Object, Object>() {
#Override
public Object load(Object key) throws Exception {
return null;
}
#Override
public Object reload(Object key, Object oldValue) throws Exception {
return oldValue;
}
};
return cacheLoader;
}
`
Now i am using the above cache as manual put and get.
I guess you're trying to implement records deduplication for Kafka.
Here is the similar discussion:
https://github.com/spring-projects/spring-kafka/issues/80
Here is the current abstract class which you may extend to achieve the necessary result:
https://github.com/spring-projects/spring-kafka/blob/master/spring-kafka/src/main/java/org/springframework/kafka/listener/adapter/AbstractFilteringMessageListener.java
Your caching service is definitely incorrect: Cacheable annotation allows marking the data getters and setters, to add caching through AOP. While in the code you clearly implement some low-level cache updating logic of your own.
At least next possible changes may help you:
Remove #Cacheable. You don't need it because you work with cache manually, so it may be the source of conflicts (especially as soon as you use sync = true). If it helps, remove #EnableCaching as well - it enables support for cache-related Spring annotations which you don't need here.
Try removing Ticker bean with the appropriate parameters for other beans. It should not be harmful as per your configuration, but usually it's helpful only for tests, no need to define it otherwise.
Double-check what is declineRecord. If it's a serialized object, ensure that serialization works properly.
Add recordStats() for cache and output stats() to log for further analysis.

Micrometer - Common tags for specific metrics

I'm trying to figure out how to set common tags for specific metrics. NOTE: I'm using the Cloudwatch monitoring system. Here is what I have:
#Bean
public MeterRegistryCustomizer<MeterRegistry> metricsCommonTags() {
return new MeterRegistryCustomizer<MeterRegistry>() {
#Override
public void customize(MeterRegistry registry) {
registry.config()
.meterFilter(MeterFilter.denyNameStartsWith("jvm.gc.pause"))
.meterFilter(MeterFilter.denyNameStartsWith("logback"))
.meterFilter(MeterFilter.denyNameStartsWith("process"))
.meterFilter(MeterFilter.denyNameStartsWith("system.cpu"))
.meterFilter(MeterFilter.denyNameStartsWith("jvm.buffer"))
.meterFilter(MeterFilter.denyNameStartsWith("jvm.classes")
.commonTags(Arrays.asList(Tag.of("instanceId", instanceId)));
}
};
}
I'm thinking of a MeterFilter method similar to MeterFilter.allow("metric.name").tags("tag1","tag2")
Micrometer does allow me to set the tags at the creation of the Meter, however that does not help me with the Spring enabled Meters.
It appears the only way to do that is by creating two MeterRegistryCustomizer objects, one for the Spring metrics and any custom metrics I create that DO need the common tags and the other for those that don't.
Is there a way to accomplish this that I'm missing?
For posterity sake, here's my solution with code. The chosen answer suggested an #Autowired MeterFilter bean, but that wasn't necessary for my specific use-case.
In order to differentiate between meters that I do and do not want to have the instanceId tag, I set an "AGG" tag-key on those that I don't want to have the instanceId tag (i.e. they are metrics that will be aggregated from all instances) and then remove it.
#Bean
public MeterRegistryCustomizer<MeterRegistry> buildMeterRegistry() {
return new MeterRegistryCustomizer<MeterRegistry>() {
#Override
public void customize(MeterRegistry registry) {
registry.config()
.meterFilter(new MeterFilter() {
#Override
public Meter.Id map(Meter.Id id) {
// Check for the "AGG" tag
if (id.getTag("AGG") != null) {
log.debug("Setting an aggregate meter: {} :: {}", id.getName(), id.getTags());
// Remove the "AGG" tag
List<Tag> tags = id.getTags().stream()
.filter(tag -> !StringUtils.equalsIgnoreCase(tag.getKey(), "AGG"))
.collect(Collectors.toList());
// Create a new Meter.Id
return new Meter.Id(id.getName(), tags, id.getBaseUnit(), id.getDescription(), id.getType());
}
// Create a new Meter.Id with the instanceId tag
return new Meter.Id(id.getName(), Arrays.asList(Tag.of("instanceId", instanceId)), id.getBaseUnit(), id.getDescription(), id.getType());
}
})
.meterFilter(MeterFilter.denyNameStartsWith("jvm.gc.pause"))
.meterFilter(MeterFilter.denyNameStartsWith("logback"))
.meterFilter(MeterFilter.denyNameStartsWith("process"))
.meterFilter(MeterFilter.denyNameStartsWith("system.cpu"))
.meterFilter(MeterFilter.denyNameStartsWith("jvm.buffer"))
.meterFilter(MeterFilter.denyNameStartsWith("jvm.classes"));
}
};
}
If you want to add tags to specific meters, register MeterFilter as a bean. For an example, see the following code: https://github.com/izeye/sample-micrometer-spring-boot/blob/so-53925641/src/main/java/com/izeye/sample/config/MetricsConfig.java#L40-L52

Spring WS (DefaultWsdl11Definition) HTTP status code with void

We have a (working) SOAP web service based on Spring WS with DefaultWsdl11Definition.
This is basically what it looks like:
#Endpoint("name")
public class OurEndpoint {
#PayloadRoot(namespace = "somenamespace", localPart = "localpart")
public void onMessage(#RequestPayload SomePojo pojo) {
// do stuff
}
}
It is wired in Spring and it is correctly processing all of our SOAP requests. The only problem is that the method returns a 202 Accepted. This is not what the caller wants, he'd rather have us return 204 No Content (or if that is not possible an empty 200 OK).
Our other endpoints have a valid response object, and do return 200 OK. It seems void causes 202 when 204 might be more appropriate?
Is it possible to change the response code in Spring WS? We can't seem to find the correct way to do this.
Things we tried and didn't work:
Changing the return type to:
HttpStatus.NO_CONTENT
org.w3c.dom.Element <- not accepted
Adding #ResponseStatus <- this is for MVC, not WS
Any ideas?
Instead of what I wrote in the comments it is possibly the easiest to create a delegation kind of solution.
public class DelegatingMessageDispatcher extends MessageDispatcher {
private final WebServiceMessageReceiver delegate;
public DelegatingMessageDispatcher(WebServiceMessageReceiver delegate) {
this.delegate = delegate;
}
public void receive(MessageContext messageContext) throws Exception {
this.delegate.receive(messageContext);
if (!messageContext.hasResponse()) {
TransportContext tc = TransportContextHolder.getTransportContext();
if (tc != null && tc.getConnection() instanceof HttpServletConnection) {
((HttpServletConnection) tc.getConnection()).getHttpServletResponse().setStatus(200);
}
}
}
}
Then you need to configure a bean named messageDispatcher which would wrap the default SoapMessageDispatcher.
#Bean
public MessageDispatcher messageDispatcher() {
return new DelegatingMessageDispatcher(soapMessageDispatcher());
}
#Bean
public MessageDispatcher soapMessageDispatcher() {
return new SoapMessageDispatcher();
}
Something like that should do the trick. Now when response is created (In the case of a void return type), the status as you want is send back to the client.
When finding a proper solutions we've encountered some ugly problems:
Creating custom adapters/interceptors is problematic because the handleResponse method isn't called by Spring when you don't have a response (void)
Manually setting the status code doesn't work because HttpServletConnection keeps a boolean statusCodeSet which doesn't get updated
But luckily we managed to get it working with the following changes:
/**
* If a web service has no response, this handler returns: 204 No Content
*/
public class NoContentInterceptor extends EndpointInterceptorAdapter {
#Override
public void afterCompletion(MessageContext messageContext, Object o, Exception e) throws Exception {
if (!messageContext.hasResponse()) {
TransportContext tc = TransportContextHolder.getTransportContext();
if (tc != null && tc.getConnection() instanceof HttpServletConnection) {
HttpServletConnection connection = ((HttpServletConnection) tc.getConnection());
// First we force the 'statusCodeSet' boolean to true:
connection.setFaultCode(null);
// Next we can set our custom status code:
connection.getHttpServletResponse().setStatus(204);
}
}
}
}
Next we need to register this interceptor, this can be easily done using Spring's XML:
<sws:interceptors>
<bean class="com.something.NoContentInterceptor"/>
</sws:interceptors>
A big thanks to #m-deinum for pointing us in the right direction!
To override the afterCompletion method really helped me out in the exact same situation. And for those who use code based Spring configuration, hereĀ“s how one can add the interceptor for a specific endpoint.
Annotate the custom interceptor with #Component, next register the custom interceptor to a WsConfigurerAdapter like this:
#EnableWs
#Configuration
public class EndpointConfig extends WsConfigurerAdapter {
/**
* Add our own interceptor for the specified WS endpoint.
* #param interceptors
*/
#Override
public void addInterceptors(List<EndpointInterceptor> interceptors) {
interceptors.add(new PayloadRootSmartSoapEndpointInterceptor(
new NoContentInterceptor(),
"NAMESPACE",
"LOCAL_PART"
));
}
}
NAMESPACE and LOCAL_PART should correspond to the endpoint.
If someone ever wanted to set custom HTTP status when returning non-void response, here is solution:
Spring Boot WS-Server - Custom Http Status

Resources