How to exclude some uri to be observed using springboot3/micrometer - spring-boot

Hy
I am using springboot 3 with the new micrometer observation. Is there a way to prevent generating a trace_id/span_id for some paths like /actuator/prometheus? Observation add a trace id for each call to /actuator/*.
Thank you

You need to give more information about a problem, but i think you have set this line in application.propeties like:
management.endpoints.web.exposure.include=/actuator/*
But exists option like:
management.endpoints.web.exposure.exclude=/actuator/prometheus

I managed to find a half solution to the problem, by defining the ObservationRegistry this way:
#Bean
#ConditionalOnMissingBean
ObservationRegistry observationRegistry() {
PathMatcher pathMatcher = new AntPathMatcher("/");
ObservationRegistry observationRegistry = ObservationRegistry.create();
observationRegistry.observationConfig().observationPredicate((name, context) -> {
if(context instanceof ServerRequestObservationContext) {
return !pathMatcher.match("/actuator/**", ((ServerRequestObservationContext) context).getCarrier().getRequestURI());
} else {
return true;
}
});
return observationRegistry;
}
This doesn't completely ignore the actuator requests, only the first span. So if you have for example Spring Security on your classpath, those spans are left intact.
EDIT: Looks like you don't need to redefine the entire observation registry, you can use a bean like the one show here: https://docs.spring.io/spring-security/reference/servlet/integrations/observability.html
EDIT2: From what I can tell, you need to include these 2 beans, and no actuator call will be traced (completely disables tracing for spring security too):
#Bean
ObservationRegistryCustomizer<ObservationRegistry> skipActuatorEndpointsFromObservation() {
PathMatcher pathMatcher = new AntPathMatcher("/");
return (registry) -> registry.observationConfig().observationPredicate((name, context) -> {
if (context instanceof ServerRequestObservationContext observationContext) {
return !pathMatcher.match("/actuator/**", observationContext.getCarrier().getRequestURI());
} else {
return true;
}
});
}
#Bean
ObservationRegistryCustomizer<ObservationRegistry> skipSecuritySpansFromObservation() {
return (registry) -> registry.observationConfig().observationPredicate((name, context) ->
!name.startsWith("spring.security"));
}
Also, you might want to keep an eye out on this issue: https://github.com/spring-projects/spring-framework/issues/29210

Related

Why my Spring redis cache doesn't work even with anntations

I follow the instructions on this tutorial (https://www.baeldung.com/spring-boot-redis-cache)
#Cacheable(value = "itemCache")
public UserInfo getUserInfo(String id) {
// without explicit manipulate cache, the value can't be retrieved from Redis
UserInfo res = cacheManager.getCache("itemCache").get(id, UserInfo.class);
if (res != null) {
return res;
}
try {
... retrieve from database ...
res = convertMapToUserInfo(id, userInfoMap);
// without explicit manipulate cache, the value can't be stored in Redis
cacheManager.getCache("itemCache").put(id, res);
return res;
}
} catch (Exception e) {
...
} finally {
...
}
return null;
}
The weird thing is, I have to put/get items from Cache manualy, even I use Cacheable annotation. Without explicit manipulate cache, the returned value of getUserInfo can't be cached.
The RedisConfiguration contains code
#Bean
public RedisCacheManagerBuilderCustomizer redisCacheManagerBuilderCustomizer() {
return (builder) -> builder
.withCacheConfiguration("itemCache",
this.cacheConfiguration());
}
#Bean
public RedisCacheConfiguration cacheConfiguration() {
return RedisCacheConfiguration.defaultCacheConfig()
.disableCachingNullValues()
.serializeValuesWith(RedisSerializationContext.SerializationPair.fromSerializer(new GenericJackson2JsonRedisSerializer()));
}
And I also add #EnableCaching to my Application class. Could anyone help me figure out why my cache doesn't take effect. Thanks!
I've recently found out that if we use just one entry with the builder it does'nt work. But if you have more than one cache in your project and include all these entries in this method or even if you add a 'dummy' entry, it works. Maybe a bug or a known issue in Spring implementation, I don't know.
#Bean
public RedisCacheManagerBuilderCustomizer redisCacheManagerBuilderCustomizer() {
return (builder) -> builder
.withCacheConfiguration("itemCache", this.cacheConfiguration())
.withCacheConfiguration("dummy", this.cacheConfiguration());
}
I hope helps.

StreamingResponseBodyReturnValueHandler does not use applicationTaskExecutor

I have Spring Data REST custom controller that returns ResponseEntity<StreamingResponseBody>
In my application.yaml file, I have defined a custom task executor to be used for the StreamingResponseBody.
spring:
task:
execution:
pool:
max-size: 16
queue-capacity: 100
However, mvc is still using the SimpleAsyncTaskExecutor, instead of the one defined above.
An Executor is required to handle java.util.concurrent.Callable return values.
Please, configure a TaskExecutor in the MVC config under "async support".
The SimpleAsyncTaskExecutor currently in use is not suitable under load.
After a little debugging, I found out that the StreamingResponseBodyReturnValueHandler does not set the applicationTaskExecutor on the WebAsyncTask type i.e. StreamingResponseBodyTask and as a result the SimpleAsyncTaskExecutor is used.
Callable<Void> callable = new StreamingResponseBodyTask(outputMessage.getBody(), streamingBody);
WebAsyncUtils.getAsyncManager(webRequest).startCallableProcessing(callable, mavContainer);
The WebMvcAutoConfiguration is picking up the applicationTaskExecutor and setting it correctly, when I debugged this method in WebMvcAutoConfiguration
#Override
public void configureAsyncSupport(AsyncSupportConfigurer configurer) {
if (this.beanFactory.containsBean(TaskExecutionAutoConfiguration.APPLICATION_TASK_EXECUTOR_BEAN_NAME)) {
Object taskExecutor = this.beanFactory
.getBean(TaskExecutionAutoConfiguration.APPLICATION_TASK_EXECUTOR_BEAN_NAME);
if (taskExecutor instanceof AsyncTaskExecutor) {
configurer.setTaskExecutor(((AsyncTaskExecutor) taskExecutor));
}
}
Duration timeout = this.mvcProperties.getAsync().getRequestTimeout();
if (timeout != null) {
configurer.setDefaultTimeout(timeout.toMillis());
}
}
Am I missing anything? How can I apply the ThreadPoolTaskExecutor for a StreamingResponseBody?
spring-data-rest Custom Repositories (#RepositoryRestResource or #BasePathAwareController) Ignore AsyncSupportConfigurer
see RepositoryRestMvcConfiguration#repositoryExporterHandlerAdapter here.
But spring-webmvc applies AsyncSupportConfigurer see WebMvcConfigurationSupport#requestMappingHandlerAdapter here
AsyncSupportConfigurer configurer = getAsyncSupportConfigurer();
if (configurer.getTaskExecutor() != null) {
adapter.setTaskExecutor(configurer.getTaskExecutor());
}
if (configurer.getTimeout() != null) {
adapter.setAsyncRequestTimeout(configurer.getTimeout());
}
adapter.setCallableInterceptors(configurer.getCallableInterceptors());
adapter.setDeferredResultInterceptors(configurer.getDeferredResultInterceptors());
If it's possible, use a simple #RestController or #Controller,
If not, you can create an issue here to add support for that

Springdoc GroupedOpenApi not following global parameters set with OperationCustomizer

When using GroupedOpenApi to define an API group, the common set of parameters that are added to every endpoint is not present in the parameters list.
Below are the respective codes
#Bean
public GroupedOpenApi v1Apis() {
return GroupedOpenApi.builder().group("v1 APIs")
// hide all v2 APIs
.pathsToExclude("/api/v2/**", "/v2/**")
// show all v1 APIs
.pathsToMatch("/api/v1/**", "/v1/**")
.build();
}
And the class to add the Standard Headers to all the endpoints
#Component
public class GlobalHeaderAdder implements OperationCustomizer {
#Override
public Operation customize(Operation operation, HandlerMethod handlerMethod) {
operation.addParametersItem(new Parameter().$ref("#/components/parameters/ClientID"));
operation.addSecurityItem(new SecurityRequirement().addList("Authorization"));
List<Parameter> parameterList = operation.getParameters();
if (parameterList!=null && !parameterList.isEmpty()) {
Collections.rotate(parameterList, 1);
}
return operation;
}
}
Actual Output
Expected Output
Workaround
Adding the paths to be included/excluded in the application properties file solves the error. But something at the code level will be much appreciated.
Attach the required OperationCustomizerobject while building the Api Group.
#Bean
public GroupedOpenApi v1Apis(GlobalHeaderAdder globalHeaderAdder) {
return GroupedOpenApi.builder().group("v1 APIs")
// hide all v2 APIs
.pathsToExclude("/api/v2/**", "/v2/**")
// show all v1 APIs
.pathsToMatch("/api/v1/**", "/v1/**")
.addOperationCustomizer(globalHeaderAdded)
.build();
}
Edit: Answer updated with reference to #Value not providing values from application properties Spring Boot
Alternative to add and load OperationCustomizer in the case you declare yours open api groups by properties springdoc.group-configs[0].group= instead definition by Java code in a Spring Configuration GroupedOpenApi.builder().
#Bean
public Map<String, GroupedOpenApi> configureGroupedsOpenApi(Map<String, GroupedOpenApi> groupedsOpenApi, OperationCustomizer operationCustomizer) {
groupedsOpenApi.forEach((id, groupedOpenApi) -> groupedOpenApi.getOperationCustomizers()
.add(operationCustomizer));
return groupedsOpenApi;
}

Micrometer filter is ignored with CompositeMeterRegistry

I use Spring Boot 2.1.2.RELEASE, and I try to use Micrometer with CompositeMeterRegistry. My goal is to publish some selected meters to ElasticSearch. The code below shows my sample config. The problem is, that the filter is completely ignored (so all metrics are sent to ElasticSearch), although I can see in the logs that it was processed ("filter reply of meter ..." lines).
Strangely, if I define the MeterFilter as a Spring bean, then it's applied to ALL registries (however, I want it to be applied only on "elasticMeterRegistry").
Here is a sample configuration class:
#Configuration
public class AppConfiguration {
#Bean
public ElasticConfig elasticConfig() {
return new ElasticConfig() {
#Override
#Nullable
public String get(final String k) {
return null;
}
};
}
#Bean
public MeterRegistry meterRegistry(final ElasticConfig elasticConfig) {
final CompositeMeterRegistry registry = new CompositeMeterRegistry();
registry.add(new SimpleMeterRegistry());
registry.add(new JmxMeterRegistry(new JmxConfig() {
#Override
public Duration step() {
return Duration.ofSeconds(10);
}
#Override
#Nullable
public String get(String k) {
return null;
}
}, Clock.SYSTEM));
final ElasticMeterRegistry elasticMeterRegistry = new ElasticMeterRegistry(elasticConfig, Clock.SYSTEM);
elasticMeterRegistry.config().meterFilter(new MeterFilter() {
#Override
public MeterFilterReply accept(Meter.Id id) {
final MeterFilterReply reply =
id.getName().startsWith("logback")
? MeterFilterReply.NEUTRAL
: MeterFilterReply.DENY;
log.info("filter reply of meter {}: {}", id.getName(), reply);
return reply;
}
});
registry.add(elasticMeterRegistry);
return registry;
}
}
So, I expect ElasticSearch to receive only "logback" metrics, and JMX to receive all metrics.
UPDATE:
I have played with filters and found a "solution", but I don't really understand why the code above doesn't work.
This works:
elasticMeterRegistry.config().meterFilter(new MeterFilter() {
#Override
public MeterFilterReply accept(Meter.Id id) {
final MeterFilterReply reply =
id.getName().startsWith("logback")
? MeterFilterReply.ACCEPT
: MeterFilterReply.DENY;
log.info("filter reply of meter {}: {}", id.getName(), reply);
return reply;
}
});
The difference is: I return ACCEPT instead of NEUTRAL.
Strangely, the following code does not work (ES gets all metrics):
elasticMeterRegistry.config().meterFilter(
MeterFilter.accept(id -> id.getName().startsWith("logback")));
But this works:
elasticMeterRegistry.config().meterFilter(
MeterFilter.accept(id -> id.getName().startsWith("logback")));
elasticMeterRegistry.config().meterFilter(
MeterFilter.deny());
CONCLUSION:
So, it seems that instead of NEUTRAL, the filter should return ACCEPT. But for meters not starting with "logback", my original filter (with NEUTRAL) returns DENY. Then why are those metrics published to ElasticSearch registry?
Can someone explain this?
This is really a composite of questions. I'll just point out a few points.
For the MeterRegistry bean you defined, Spring Boot will auto-configure an ElasticMeterRegistry bean as there's no ElasticMeterRegistry bean. Instead of creating a CompositeMeterRegistry bean on your own, just define a custom ElasticMeterRegistry bean which is applied the MeterFilter you want and let Spring Boot create one (CompositeMeterRegistry bean) for you.
For MeterFilterReply, ACCEPT will accept the meter immediately, DENY will deny the meter immediately, and NEUTRAL will postpone the decision to next filter(s). Basically meters will be accepted unless there's any DENY.

Micrometer - Common tags for specific metrics

I'm trying to figure out how to set common tags for specific metrics. NOTE: I'm using the Cloudwatch monitoring system. Here is what I have:
#Bean
public MeterRegistryCustomizer<MeterRegistry> metricsCommonTags() {
return new MeterRegistryCustomizer<MeterRegistry>() {
#Override
public void customize(MeterRegistry registry) {
registry.config()
.meterFilter(MeterFilter.denyNameStartsWith("jvm.gc.pause"))
.meterFilter(MeterFilter.denyNameStartsWith("logback"))
.meterFilter(MeterFilter.denyNameStartsWith("process"))
.meterFilter(MeterFilter.denyNameStartsWith("system.cpu"))
.meterFilter(MeterFilter.denyNameStartsWith("jvm.buffer"))
.meterFilter(MeterFilter.denyNameStartsWith("jvm.classes")
.commonTags(Arrays.asList(Tag.of("instanceId", instanceId)));
}
};
}
I'm thinking of a MeterFilter method similar to MeterFilter.allow("metric.name").tags("tag1","tag2")
Micrometer does allow me to set the tags at the creation of the Meter, however that does not help me with the Spring enabled Meters.
It appears the only way to do that is by creating two MeterRegistryCustomizer objects, one for the Spring metrics and any custom metrics I create that DO need the common tags and the other for those that don't.
Is there a way to accomplish this that I'm missing?
For posterity sake, here's my solution with code. The chosen answer suggested an #Autowired MeterFilter bean, but that wasn't necessary for my specific use-case.
In order to differentiate between meters that I do and do not want to have the instanceId tag, I set an "AGG" tag-key on those that I don't want to have the instanceId tag (i.e. they are metrics that will be aggregated from all instances) and then remove it.
#Bean
public MeterRegistryCustomizer<MeterRegistry> buildMeterRegistry() {
return new MeterRegistryCustomizer<MeterRegistry>() {
#Override
public void customize(MeterRegistry registry) {
registry.config()
.meterFilter(new MeterFilter() {
#Override
public Meter.Id map(Meter.Id id) {
// Check for the "AGG" tag
if (id.getTag("AGG") != null) {
log.debug("Setting an aggregate meter: {} :: {}", id.getName(), id.getTags());
// Remove the "AGG" tag
List<Tag> tags = id.getTags().stream()
.filter(tag -> !StringUtils.equalsIgnoreCase(tag.getKey(), "AGG"))
.collect(Collectors.toList());
// Create a new Meter.Id
return new Meter.Id(id.getName(), tags, id.getBaseUnit(), id.getDescription(), id.getType());
}
// Create a new Meter.Id with the instanceId tag
return new Meter.Id(id.getName(), Arrays.asList(Tag.of("instanceId", instanceId)), id.getBaseUnit(), id.getDescription(), id.getType());
}
})
.meterFilter(MeterFilter.denyNameStartsWith("jvm.gc.pause"))
.meterFilter(MeterFilter.denyNameStartsWith("logback"))
.meterFilter(MeterFilter.denyNameStartsWith("process"))
.meterFilter(MeterFilter.denyNameStartsWith("system.cpu"))
.meterFilter(MeterFilter.denyNameStartsWith("jvm.buffer"))
.meterFilter(MeterFilter.denyNameStartsWith("jvm.classes"));
}
};
}
If you want to add tags to specific meters, register MeterFilter as a bean. For an example, see the following code: https://github.com/izeye/sample-micrometer-spring-boot/blob/so-53925641/src/main/java/com/izeye/sample/config/MetricsConfig.java#L40-L52

Resources