I am using Actuator in springboot 2 to expose /actuator/prometheus endpoint from which a prometheus instance will pull metrics.
Everithing works perfect except because I am in need of tweak the metric names. I mean not the suffix (_count, _total, _bucket, ...) which are meaningful for Prometheus but something like:
http_server_requests_seconds_count -> http_server_requests_count
http_server_requests_seconds_max -> latency_seconds_max
http_server_requests_seconds_sum -> latency_seconds_sum
http_server_requests_seconds_bucket -> latency_seconds_bucket
Is there any better approach to this?
P.S.
I know i can use
management.metrics.web.server.requests-metric-name=different
to get
different_seconds_count
different_seconds_max
different_seconds_sum
different_seconds_bucket
but it would be difficult to:
1º remove the _seconds suffix
2º use a different base name for only one of them
I am guessing i could write an alternative PrometheusRenameFilter but not sure how to configure it to the default registry.
you can override this method and update the naming convention:
#Configuration
public class MetricsConfiga {
#Bean
MeterRegistryCustomizer<MeterRegistry> configurer(String applicationName) {
return (registry) -> registry.config().namingConvention(new NamingConvention() {
#Override
public String name(String name, Meter.Type type, String baseUnit) {
return "PREFIX" + Arrays.stream(name1.split("\\."))
.filter(Objects::nonNull)
.collect(Collectors.joining("_"));
}
});
}
}
Now I know how I can customize the global registry:
e.g. to set a custom meter filter:
#Configuration
public class MetricsConfig {
#Bean
MeterRegistryCustomizer<MeterRegistry> metricsConfig() {
return registry -> registry.config().meterFilter(new CustomRenameFilter());
}
}
However, setting a custom rename filter in the registry only allow to rename the base metric name.
It does not act on the suffixes nor allow to act on specific metric belonging a set e.g. generated by the summary.
with a custom NamingConvention I can add suffixes to convention base name ... I could even alter existing suffixes or replace convention base name.
Finally please note that Histogram prometheus metric type expects the creation of
<basename>_bucket
<basename>_sum
<basename>_count
with those specific names so it might be incorrect to tweak the component in the way I want because that would be a diferent component.
Related
I have a service, which receives different structured messages from different message queues. Having #StreamListener conditions we can choose at every message type how that message should be handled. As an example:
We receive two different types of messages, which have different header fields and values e.g.
Incoming from "order" queue:
Order1: { Header: {catalog:groceries} }
Order2: { Header: {catalog:tools} }
Incoming from "shipment" queue:
Shipment1: { Header: {region:Europe} }
Shipment2: { Header: {region:America} }
There is a binding for each queue, and with according #StreamListener I can process the messages by catalog and region differently
e.g.
#StreamListener(target = OrderSink.ORDER_CHANNEL, condition = "headers['catalog'] == 'groceries'")
public void onGroceriesOrder(GroceryOder order){
...
}
So the question is, how to achieve this with the new Spring Cloud Function approach?
At the documentation https://cloud.spring.io/spring-cloud-static/spring-cloud-stream/3.0.2.RELEASE/reference/html/spring-cloud-stream.html#_event_routing it is mentioned:
Also, for SpEL, the root object of the evaluation context is Message so you can do evaluation on individual headers (or message) as well ….routing-expression=headers['type']
Is it possible to add the routing-expression to the binding like (in application.yml)
onGroceriesOrder-in-0:
destination: order
routing-expression: "headers['catalog']==groceries"
?
EDIT after first answer
If the above expression at this location is not possible, what the first answer implies, than my question goes as follows:
As far as I understand, an expression like routing-expression: headers['catalog'] must be set globally, because the result maps to certain (consumer) functions.
How can I control that the 2 different messages on each queue will be forwarted to their own consumer function, e.g.
Order1 --> MyOrderService.onGroceriesOrder()
Order2 --> MyOrderService.onToolsOrder()
Shipment1 --> MyShipmentService.onEuropeShipment()
Shipment2 --> MyShipmentService.onAmericaShipment()
That was easy with #StreamListener, because each method gets their own #StreamListener annotation with different conditions. How can this be achieved with the new routing-expression setting?
?
Aside from the fact that the above is not a valid expression, but I think you meant headers['catalog']==groceries. If so, what would you expect to happen from evaluating it as the only two option could be true/false. Anyway, these are rhetorical but helps to understand the problem and how to fix it.
The expression must result in a value of a function to route TO. So. . .
routing-expression: headers['catalog'] - assumes that the actual value of catalog header is the name of the function to invoke
routing-expression: headers['catalog']==groceries ? 'processGroceries' : 'processOther' - maps value 'groceries' to 'processGroceries' function.
For a specific routing, you can use MessageRoutingCallback strategy:
MessageRoutingCallback
The MessageRoutingCallback is a strategy to assist with determining
the name of the route-to function definition.
public interface MessageRoutingCallback {
FunctionRoutingResult routingResult(Message<?> message);
. . .
}
All you need to do is implement and register it as a bean to be picked
up by the RoutingFunction. For example:
#Bean
public MessageRoutingCallback customRouter() {
return new MessageRoutingCallback() {
#Override
FunctionRoutingResult routingResult(Message<?> message) {
return new FunctionRoutingResult((String) message.getHeaders().get("func_name"));
}
};
}
Spring Cloud Function
I'm trying to load Configuration from YML. I can load value and I can also load list if these are comma seperated values. But i can't load a typical YML List.
Configuration Class
#Component
#PropertySource("classpath:routing.yml")
#ConfigurationProperties
class RoutingProperties(){
var angular = listOf("nothing")
var value: String = ""
}
Working routing.yml
angular: /init, /home
value: Hello World
Not Working routing.yml
angular:
- init
- home
value: Hello World
Why can't i load the second version of yml / do I have a syntaxt error?
ENV: Kotlin, Spring 2.0.0.M3
As #flyx say, #PropetySource not worked with yaml files. But in spring you may override almost everything :)
PropertySource has additional parameter: factory. It's possible to create your own PropertySourceFactory base on DefaultPropertySourceFactory
open class YamlPropertyLoaderFactory : DefaultPropertySourceFactory() {
override fun createPropertySource(name: String?, resource: EncodedResource?): org.springframework.core.env.PropertySource<*> {
if (resource == null)
return super.createPropertySource(name, resource)
return YamlPropertySourceLoader().load(resource.resource.filename, resource.resource, null)
}
}
And when use this factory in propertysource annotation:
#PropertySource("classpath:/routing.yml", factory = YamlPropertyLoaderFactory::class)
Last that you need is to initialized variable angular with mutableList
Full code sample:
#Component
#PropertySource("classpath:/routing.yml", factory = YamlPropertyLoaderFactory::class)
#ConfigurationProperties
open class RoutingProperties {
var angular = mutableListOf("nothing")
var value: String = ""
override fun toString(): String {
return "RoutingProperties(angular=$angular, value='$value')"
}
}
open class YamlPropertyLoaderFactory : DefaultPropertySourceFactory() {
override fun createPropertySource(name: String?, resource: EncodedResource?): org.springframework.core.env.PropertySource<*> {
if (resource == null)
return super.createPropertySource(name, resource)
return YamlPropertySourceLoader().load(resource.resource.filename, resource.resource, null)
}
}
#SpringBootApplication
#EnableAutoConfiguration(exclude = arrayOf(DataSourceAutoConfiguration::class))
open class Application {
companion object {
#JvmStatic
fun main(args: Array<String>) {
val context = SpringApplication.run(Application::class.java, *args)
val bean = context.getBean(RoutingProperties::class.java)
println(bean)
}
}
}
Kinda old post, i know. But i am at the very same topic right now.
As of now, it seems that PropertySource does indeed work with yaml Files. Given the restriction that it only allows for primitive types (it seems) and it cant handle nested elements. I'm probably gonna dig a bit deeper and update my answer accordingly, but as of now, the accepted answer seems like a functioning workaround.
Well, according to the docs, your YAML file will be rewritten into a property file. The first YAML file becomes:
angular=/init, /home
value=Hello World
While the second one becomes:
angular[0]=init
angular[1]=home
value=Hello World
These are obviously two very different things and therefore behave differently.
Moreover, later in the docs, it is stated that YAML does not even work with #PropertySource:
24.6.4 YAML shortcomings
YAML files can’t be loaded via the #PropertySource annotation. So in the case that you need to load values that way, you need to use a properties file.
That makes me kind of wonder why the first case works for you at all.
The docs say this about the generated …[index] properties:
To bind to properties like that using the Spring DataBinder utilities (which is what #ConfigurationProperties does) you need to have a property in the target bean of type java.util.List (or Set) and you either need to provide a setter, or initialize it with a mutable value, e.g. this will bind to the properties above
So, let's have a look at Kotlin docs: listOf returns a new read-only list of given elements. So the list is not mutable as required by the docs, which I assume is why it doesn't work. Try using a mutable list (since I have never used Kotlin, I cannot give you working code). Also try to declare it as java.util.List if that's possible in Kotlin.
I am using spring integration inbound channel adapter as below
inboundAdapter(csf).preserveTimestamp(true)//
.remoteDirectory(feed.getRemoteDirectory())//
.regexFilter(feed.getRegexFilter())// regex expression
.filter(ftpRemoteFileFilter)// remote filter
.deleteRemoteFiles(feed.getDeleteRemoteF
So I am using a remote filter and the out of the box regex filter . I wanted to know what is the order in which the regex filter and the remote filter are applied . From initial analysis looks like the regex filter comes first , can some one tell me the clas where this decision is made so I can be sure .
If there is no way of knowing the only other alternative will be to use the
CompositeFileListFilter .
The code you looking for is in the FtpInboundChannelAdapterSpec and looks like:
#Override
public FtpInboundChannelAdapterSpec regexFilter(String regex) {
return filter(composeFilters(new FtpRegexPatternFileListFilter(regex)));
}
#SuppressWarnings("unchecked")
private CompositeFileListFilter<FTPFile> composeFilters(FileListFilter<FTPFile> fileListFilter) {
CompositeFileListFilter<FTPFile> compositeFileListFilter = new CompositeFileListFilter<>();
compositeFileListFilter.addFilters(fileListFilter,
new FtpPersistentAcceptOnceFileListFilter(new SimpleMetadataStore(), "ftpMessageSource"));
return compositeFileListFilter;
}
So, as you see when you declare regexFilter, it is composed together with the FtpPersistentAcceptOnceFileListFilter to the the CompositeFileListFilter, where regexFilter is definitely first. And it is first because FtpPersistentAcceptOnceFileListFilter is persistence and that wouldn't be good to store files which might not match the regexp afterwards.
If you need some more complicated logic, you really should go CompositeFileListFilter way and inject it via filter() option only. I mean you have to combine your regexpFilter into the CompositeFileListFilter instead of regexFilter().
Note: after moving Java DSL into Core in 5.0, the .filter() option looks like:
public S filter(FileListFilter<F> filter) {
this.synchronizer.setFilter(filter);
return _this();
}
It overrides any previously provided filters, including regexp. That is done to avoid confusing with the chain of .filter() in favor of CompositeFileListFilter or ChainFileListFilter configured externally.
I'm trying to build a Gradle plugin that would allow the following:
myPluginConfig {
something1 {
// this is a closure
}
somethingElse {
// this is another closure
}
// more closures here
}
To achieve this I'm fairly certain I need to use a NamedDomainObjectContainer to wrap a Closure collection, so I've set up the following plugin:
class SwitchDependenciesPlugin implements Plugin<Project> {
void apply(Project project) {
// add the extension
project.getExtensions().myPluginConfig = project.container(Closure)
// read the current configuration
NamedDomainObjectContainer<Closure> config = project.myPluginConfig
// test it out, always returns []
System.out.println(config)
}
}
What am I doing wrong, do I need to use project.extensions.create instead? If so, how?
EDIT: my use case consists in adding dependencies according to some variables defined in the project hierarchy. For example, the following configuration would add the red project if the variable red is defined on project.ext, or gson otherwise:
myPluginConfig {
redTrue {
compile project(':red')
}
redFalse {
compile 'com.google.code.gson:gson:2.4'
}
greenTrue {
compile project(':green')
}
}
For this use case I need to have dynamic names for myPluginConfig, and therefore either a Map or a NamedDomainObjectContainer.
Can you elaborate what you try to model here? I think you have two options. One is to use NamedDomainObjectContainer. Here you need a class that represents "something". Have a look at the Gradle userguide chapter about maintaining multiple domain objects (see https://docs.gradle.org/current/userguide/custom_plugins.html#N175CF) in the sample of the userguide, the "thing" is 'Book'. The build-in configuration syntax like you described above comes for free.
If you want to have a syntax like above without the need for maintaining multiple domain objects, you can simply add a method that takes a Closure as a parameter to your Extension class:
void somethingToConfigure(Closure) {
}
You cannot have Closure as a type for NamedDomainObjectContainer simply because the type you use must have a property called name and a public constructor with a single String parameter.
To overcome this, you may create a wrapper around Closure with a name field added.
I'd like to change the representation of C# Doubles to rounded Int64 with a four decimal place shift in the serialization C# Driver's stack for MongoDB. In other words, store (Double)29.99 as (Int64)299900
I'd like this to be transparent to my app. I've had a look at custom serializers but I don't want to override everything and then switch on the Type with fallback to the default, as that's a bit messy.
I can see that RegisterSerializer() won't let me add one for an existing type, and that BsonDefaultSerializationProvider has a static list of primitive serializers and it's marked as internal with private members so I can't easily subclass.
I can also see that it's possible to RepresentAs Int64 for Doubles, but this is a cast not a conversion. I need essentially a cast AND a conversion in both serialization directions.
I wish I could just give the default serializer a custom serializer to override one of it's own, but that would mean a dirty hack.
Am I missing a really easy way?
You can definitely do this, you just have to get the timing right. When the driver starts up there are no serializers registered. When it needs a serializer, it looks it up in the dictionary where it keeps track of the serializers it knows about (i.e. the ones that have been registered). Only it it can't find one in the dictionary does it start figuring out where to get one (including calling the serialization providers) and if it finds one it registers it.
The limitation in RegisterSerializer is there so that you can't replace an existing serializer that has already been used. But that doesn't mean you can't register your own if you do it early enough.
However, keep in mind that registering a serializer is a global operation, so if you register a custom serializer for double it will be used for all doubles, which could lead to unexpected results!
Anyway, you could write the custom serializer something like this:
public class CustomDoubleSerializer : BsonBaseSerializer
{
public override object Deserialize(BsonReader bsonReader, Type nominalType, Type actualType, IBsonSerializationOptions options)
{
var rep = bsonReader.ReadInt64();
return rep / 100.0;
}
public override void Serialize(BsonWriter bsonWriter, Type nominalType, object value, IBsonSerializationOptions options)
{
var rep = (long)((double)value * 100);
bsonWriter.WriteInt64(rep);
}
}
And register it like this:
BsonSerializer.RegisterSerializer(typeof(double), new CustomDoubleSerializer());
You could test it using the following class:
public class C
{
public int Id;
public double X;
}
and this code:
BsonSerializer.RegisterSerializer(typeof(double), new CustomDoubleSerializer());
var c = new C { Id = 1, X = 29.99 };
var json = c.ToJson();
Console.WriteLine(json);
var r = BsonSerializer.Deserialize<C>(json);
Console.WriteLine(r.X);
You can also use your own serialization provider to tell Mongo which serializer to use for certain types, which I ended up doing to mitigate some of the timing issues mentioned when trying to override existing serializers. Here's an example of a serialisation provider that overrides how to serialize decimals:
public class CustomSerializationProvider : IBsonSerializationProvider
{
public IBsonSerializer GetSerializer(Type type)
{
if (type == typeof(decimal)) return new DecimalSerializer(BsonType.Decimal128);
return null; // falls back to Mongo defaults
}
}
If you return null from your custom serialization provider, it will fall back to using Mongo's default serialization provider.
Once you've written your provider, you just need to register it:
BsonSerializer.RegisterSerializationProvider(new CustomSerializationProvider());
I looked through the latest iteration of the driver's code and checked if there's some sort of backdoor to set custom serializers. I am afraid there's none; you should open an issue in the project's bug tracker if you think this needs to be looked at for future iterations of the driver (https://jira.mongodb.org/).
Personally, I'd open a ticket -- and if a quick workaround is necessary or required, I'd subclass DoubleSerializer, implement the new behavior, and then use Reflection to inject it into either MongoDB.Bson.Serialization.Serializers.DoubleSerializer.__instance or MongoDB.Bson.Serialization.BsonDefaultSerializationProvider.__serializers.