Is there a way to turn off some of the returned metric values in Actuator/Micrometer? Looking at them now I'm seeing around 1000 and would like to whittle them down to a select few say 100 to actually be sent to our registry.
Let me elaborate on the answer posted by checketts with a few examples. You can enable/disable certain metrics in your application.yml like this:
management:
metrics:
enable:
tomcat: true
jvm: false
process: false
hikaricp: false
system: false
jdbc: false
http: false
logback: true
Or in code by defining a MeterFilter bean:
#Bean
public MeterFilter meterFilter() {
return new MeterFilter() {
#Override
public MeterFilterReply accept(Meter.Id id) {
if(id.getName().startsWith("tomcat.")) {
return MeterFilterReply.DENY;
}
if(id.getName().startsWith("jvm.")) {
return MeterFilterReply.DENY;
}
if(id.getName().startsWith("process.")) {
return MeterFilterReply.DENY;
}
if(id.getName().startsWith("system.")) {
return MeterFilterReply.DENY;
}
return MeterFilterReply.NEUTRAL;
}
};
}
Meter filters can help in 3 ways that have been discussed with the Micrometer slack channel:
Disabling metrics
Combining dimensions
High cardinality capping filter
Micrometer comes with the first type of meter filter built in. It also support hierarchical enabling/disabling similar to how logging works (As in if have meter like my.request.total and my.request.latency you can disable all metrics that start with my.request.
I've implemented my own example of a combining filter which is useful if you have a metrics with high cardinality and want to combine them in new dimensions. Take for example you have a dimension on status codes, this lets you combine 200, 201, 203 response codes as a tag 2xx. This is similar to Netflix Spectator 'placeholder' support. I would love to contribute that code upstream, but currently it is pretty custom, so it would need some work to make it generally usable.
The last type for catching high cardinality dimensions, hasn't been created yet, but would exist as a safety valve ensuring that if a metric can potentially have a high number of tag value, it will count the number of unique tags and once a max value is hit, either disable or combine the additional tags into a common bucket, so that value doesn't explode and potentially overwhelm your monitoring (or cost you lots of $$$, if paying per-metric)
The property naming in Mzzl's answer has changed in Spring Boot 2. For example, to disable the JVM metrics it's now:
management.metrics.binders.jvm.enabled=false
See this class for the other options. The Spring team have refactored yet again in 2.1.x and those inner factory bean classes are now lifted out into standalone files but the property naming remains the same as 2.0.x.
Related
After migration to Spring Boot 2.3.0 I need to create a custom implementation of StatusAggregator.
However, I don't know exactly how to do it properly after when the HealthAggregator is deprecated. I've got several external systems and I check their availability and in the previous version I had method Health aggregate(Map<String, Health> healths) but now it's Status getAggregateStatus(Set<Status> statuses) from interface StatusAggregator where as an argument it has Set of Statuses. I need to recognize from with system Status object comes, so during creation Health, I use the approach where I create Status with description, something like this:
Health.status(new Status((Status.DOWN.getCode(), "TEST_SYSTEM")).build())
Then I want to recognize it in method Status getAggregateStatus(Set<Status> statuses) by this description however when I go to Status.class and check equals it not include description field.
public boolean equals(Object obj) {
if (obj == this) {
return true;
} else {
return obj instanceof Status ? ObjectUtils.nullSafeEquals(this.code, ((Status)obj).code) : false;
}
}
so Set<Status> will return only Statuses with different code.
I was also looking to do something similar and did not find a way to identify the component sending the status. I defined custom health indicators for each API and checked the dependencies for external APIs within those individual health indicators. For custom health indicator, implement the health method in the HealthIndicator and annotate with 'component' to register in the application context as explained in the documentation.
I have a service, which receives different structured messages from different message queues. Having #StreamListener conditions we can choose at every message type how that message should be handled. As an example:
We receive two different types of messages, which have different header fields and values e.g.
Incoming from "order" queue:
Order1: { Header: {catalog:groceries} }
Order2: { Header: {catalog:tools} }
Incoming from "shipment" queue:
Shipment1: { Header: {region:Europe} }
Shipment2: { Header: {region:America} }
There is a binding for each queue, and with according #StreamListener I can process the messages by catalog and region differently
e.g.
#StreamListener(target = OrderSink.ORDER_CHANNEL, condition = "headers['catalog'] == 'groceries'")
public void onGroceriesOrder(GroceryOder order){
...
}
So the question is, how to achieve this with the new Spring Cloud Function approach?
At the documentation https://cloud.spring.io/spring-cloud-static/spring-cloud-stream/3.0.2.RELEASE/reference/html/spring-cloud-stream.html#_event_routing it is mentioned:
Also, for SpEL, the root object of the evaluation context is Message so you can do evaluation on individual headers (or message) as well ….routing-expression=headers['type']
Is it possible to add the routing-expression to the binding like (in application.yml)
onGroceriesOrder-in-0:
destination: order
routing-expression: "headers['catalog']==groceries"
?
EDIT after first answer
If the above expression at this location is not possible, what the first answer implies, than my question goes as follows:
As far as I understand, an expression like routing-expression: headers['catalog'] must be set globally, because the result maps to certain (consumer) functions.
How can I control that the 2 different messages on each queue will be forwarted to their own consumer function, e.g.
Order1 --> MyOrderService.onGroceriesOrder()
Order2 --> MyOrderService.onToolsOrder()
Shipment1 --> MyShipmentService.onEuropeShipment()
Shipment2 --> MyShipmentService.onAmericaShipment()
That was easy with #StreamListener, because each method gets their own #StreamListener annotation with different conditions. How can this be achieved with the new routing-expression setting?
?
Aside from the fact that the above is not a valid expression, but I think you meant headers['catalog']==groceries. If so, what would you expect to happen from evaluating it as the only two option could be true/false. Anyway, these are rhetorical but helps to understand the problem and how to fix it.
The expression must result in a value of a function to route TO. So. . .
routing-expression: headers['catalog'] - assumes that the actual value of catalog header is the name of the function to invoke
routing-expression: headers['catalog']==groceries ? 'processGroceries' : 'processOther' - maps value 'groceries' to 'processGroceries' function.
For a specific routing, you can use MessageRoutingCallback strategy:
MessageRoutingCallback
The MessageRoutingCallback is a strategy to assist with determining
the name of the route-to function definition.
public interface MessageRoutingCallback {
FunctionRoutingResult routingResult(Message<?> message);
. . .
}
All you need to do is implement and register it as a bean to be picked
up by the RoutingFunction. For example:
#Bean
public MessageRoutingCallback customRouter() {
return new MessageRoutingCallback() {
#Override
FunctionRoutingResult routingResult(Message<?> message) {
return new FunctionRoutingResult((String) message.getHeaders().get("func_name"));
}
};
}
Spring Cloud Function
I'm trying to understand reactive part of spring 5. I have created simple rest endpoint for finding all entities using spring web-flux and spring data reactive (mongo) but don't see any way how to implement pagination.
Here is my simple example in Kotlin:
#GetMapping("/posts/")
fun getAllPosts() = postRepository.findAll()
Does it mean that reactive endpoint does not require pagination? Is some way to implement pagination from server side using this stack?
The reactive support in Spring Data does not provide means of a Page return type. Still, the Pageable parameter is supported in method signatures passing on limit and offset to the drivers and therefore the store itself, returning a Flux<T> that emits the range requested.
Flux<Person> findByFirstname(String firstname, Pageable pageable);
For more information please have a look at the current Reference Documentation for 2.0.RC2 and the Spring Data Examples.
Flux provides skip and take methods to get pagination support, and you also can use filter and sort to filter and sort the result. The filter and sort below is not a good example, but use skip and Pageable as 2nd parameter are no different.
The following codes work for me.
#GetMapping("")
public Flux<Post> all(
//#RequestParam(value = "q", required = false) String q,
#RequestParam(value = "page", defaultValue = "0") long page,
#RequestParam(value = "size", defaultValue = "10") long size) {
return this.postRepository.findAll()
//.filter(p -> Optional.ofNullable(q).map(key -> p.getTitle().contains(key) || p.getContent().contains(key)).orElse(true))//(replace this with query parameters)
.sort(comparing(Post::getCreatedDate).reversed())
.skip(page * size).take(size);
}
Update: The underlay drivers should be responsible for handling the result in the reactivestreams way.
And as you see in the answer from Christoph, if using a findByXXX method, Spring Data Mongo Reactive provides a variant to accept a pageable argument, but the findAll(reactive version) does not include such a variant, you have to do skip in the later operations if you really need the pagination feature. When switching to Flux instead of List, imagine the data in Flux as living water in the rivers or oil in the pipes, or the tweets in twitter.com.
I have tried to compare the queries using Pageale and not in the following case.
this.postRepository.findByTitleContains("title")
.skip(0)
.limitRequest(10)
.sort((o1, o2) -> o1.getTitle().compareTo(o2.getTitle()))
this.postRepository.findByTitleContains("title", PageRequest.of(0, 10, Sort.by(Sort.Direction.ASC, "title")))
When enabling logging for logging.level.org.springframework.data.mongodb.core.ReactiveMongoTemplate=DEBUG and found they print the same log for queries.
find using query: { "title" : { "$regularExpression" : { "pattern" : ".*title.*", "options" : ""}}} fields: Document{{title=1}} for class: class com.example.demo.Post in collection: post
//other logging...
find using query: { "title" : { "$regularExpression" : { "pattern" : ".*title.*", "options" : ""}}} fields: Document{{title=1}} for class: class com.example.demo.Post in collection: post
Keep in mind, all these operations should be DELEGATED to the underlay R2dbc drivers which implemented the reactive streams spec and performed on the DB side, NOT in the memory of your application side.
Check the example codes.
The early sample code I provided above maybe is not a good sample of filter and sort operations(MongoDB itself provides great regularexpression operations for it). But pagination in the reactive variant is not a good match with the concept in the reactive stream spec. When embracing Spring reactive stack, most of the time, we just move our work to a new collection of APIs. In my opinion, the realtime update and elastic response scene could be better match Reactive, eg. using it with SSE, Websocket, RSocket, application/stream+json(missing in the new Spring docs) protocols, etc
This is not efficient but it works for me while I look for another solution
Service
public Page<Level> getPage(int page, int size, Sort.Direction direction, String properties) {
var pageRequest = PageRequest.of(page, size, direction, properties);
var count = levelRepository.count().block();
var levels = levelRepository.findAllLevelsPaged(pageRequest).collectList().block();
return new PageImpl<>(Objects.requireNonNull(levels), pageRequest, Objects.requireNonNull(count));
}
Repo
#Repository
public interface LevelRepository extends ReactiveMongoRepository<Level, String> {
#Query("{ id: { $exists: true }}")
Flux<Level> findAllLevelsPaged(final Pageable page);
}
Ref example
I'm trying to understand the proper way to use Windows.Foundation.Diagnostics.LoggingChannel. In particular I'd like to understand the purpose behind the Level property and when is this property set.
As described in the MSDN documentation of LoggingChannel, the Level property is read-only. So how can I set the level that a channel accepts messages at?
Currently what I have designed as a logger for my app is something like below:
public class Logger
{
public LoggingLevel LoggerLoggingLevel { get; set; }
private LoggingSession _session;
private LoggingChannel _channel;
public Logger()
{
_channel = new LoggingChannel("MyChannel");
_session = new LoggingSession("MySession");
_session.AddLoggingChannel(_channel);
}
public void LogMessage(string msg, LoggingLevel level)
{
if (level >= LoggerLoggingLevel)
{
_channel.LogMessage(msg, level);
}
}
.
.
.
}
// The consumer of the Logger class will instantiate an instance of it,
// sets the LoggerLoggingLevel, and then starts logging messages at various levels.
// At any point, the consumer can change LoggerLoggingLevel to start accepting
// messages at different levels.
IS this the right approach or is there a better way (for example by somehow setting the level of _channel and then passing the message & level to the channel, letting the channel decide whether it should filter out the message or accept and log it)?
LoggingChannel.Level tells you "somebody has expressed interest in receiving messages from you that are of severity 'Level' or higher". This property will be set automatically by the runtime when somebody subscribes to events from your LoggingChannel instance. (Within your app, you can subscribe to your app's events using the LoggingSession class; outside of your app, you can record your app's events using a tool like tracelog or xperf.)
In simple scenarios, you don't need to worry about the value of LoggingChannel.Level. The LoggingChannel.LogMessage function already checks the value of LoggingChannel.Level. It also checks the value of LoggingChannel.Enabled, which tells you whether anybody is subscribed to your events at any level. (Note that the value of LoggingChannel.Level is UNDEFINED and MEANINGLESS unless LoggingChannel.Enabled is true.) In normal use, you don't need to worry about LoggingChannel.Enabled or LoggingChannel.Level -- just call LogMessage and let LoggingChannel check the levels for you.
LoggingChannel exposes the Enabled and Level properties to support a more complex scenario where it is expensive to gather the data you are about to log. In this case, you would probably like to skip gathering the data if nobody is listening for your event. You would then write code like this:
if (channel.Enabled && channel.Level <= eventLevel)
{
string expensiveData = GatherExpensiveData();
channel.LogMessage(expensiveData, eventLevel);
}
Note that the Windows 10 version of LoggingChannel added a bunch of new methods to make life a bit easier. If your program will run on Windows 10 or later, you can use the IsEnabled method instead of separate checks for Enabled and Level:
if (channel.IsEnabled(eventLevel))
{
string expensiveData = GatherExpensiveData();
channel.LogMessage(expensiveData, eventLevel);
}
A bunch of other stuff was also added to LoggingChannel for Windows 10. You can now log complex events (strongly-typed fields) instead of just strings, you can define keywords and opcodes (look up ETW documentation for more information), and you can basically have your LoggingChannel act like a first-class ETW citizen.
What is the equivalent of destination-type from jms:listener-container in JavaConfig?
I have checked in the API these two following classes without results.
DefaultMessageListenerContainer
MessageListenerAdapter
I am trying to create consumers for a topic, many tutorials in the web use destination-type="topic"
According with the 23.6 JMS Namespace Support section, there is the Table 23.2. Attributes of the JMS element table. Where for the destination-type attribute says:
The JMS destination type for this listener: queue, topic or durableTopic. The default is queue.
For the audience: consider the two following links if you are trying to do a migration from jms:listener-container and jms:listener for JavaConfig.
complete jms:listener migration to JavaConfig
How to add multiple JMS MessageListners in a single MessageListenerContainer for Spring Java Config
When in doubt, look at the parser (in this case AbstractListenerContainerParser); that attribute doesn't map to a single property, it maps to pubSubDomain and subscriptionDurable...
String destinationType = ele.getAttribute(DESTINATION_TYPE_ATTRIBUTE);
boolean pubSubDomain = false;
boolean subscriptionDurable = false;
if (DESTINATION_TYPE_DURABLE_TOPIC.equals(destinationType)) {
pubSubDomain = true;
subscriptionDurable = true;
}
else if (DESTINATION_TYPE_TOPIC.equals(destinationType)) {
pubSubDomain = true;
}
else if ("".equals(destinationType) || DESTINATION_TYPE_QUEUE.equals(destinationType)) {
// the default: queue
}
else {
parserContext.getReaderContext().error("Invalid listener container 'destination-type': " +
"only \"queue\", \"topic\" and \"durableTopic\" supported.", ele);
}
configDef.getPropertyValues().add("pubSubDomain", pubSubDomain);
configDef.getPropertyValues().add("subscriptionDurable", subscriptionDurable);
Though this is a bit late, I would suggest to use the following approach for anyone who is still searching for the answer.
I have created a new Class DefaultMessageListenerContainerExtended which extends DefaultMessageListenerContainer and I have added one more method as setDestinationType. This does the trick in a nice and familiar way.
Following is the link to source code, which can be found on git:
https://github.com/HVT7/spring-jms-set-destination-type/blob/master/DefaultMessageListenerContainerExtended.java
Also to add, try to use spring version 4.2.5, as there are minor updates in that version (Had to dig a lot due to version issues as I was using 4.1.5 and Listener Containers did not had function to set "ReplyPubSubDomain" property).