I decided to go back to my idea of integrating Kafka Metrics with Spring Boot Actuator, which I've mentioned here:
https://github.com/spring-projects/spring-boot/issues/6227
As of now I have a separate "sandbox" project with working code, which I want to merge into Spring Boot. And now I'm a bit confused. Part of my tests require powermock to Mock Kafka's "super-secured" class:
package org.apache.kafka.common.metrics;
// some imports...
public final class KafkaMetric implements Metric {
private MetricName metricName;
private final Object lock;
private final Time time;
private final Measurable measurable;
private MetricConfig config;
KafkaMetric(Object lock, MetricName metricName, Measurable measurable, MetricConfig config, Time time) {
this.metricName = metricName;
this.lock = lock;
this.measurable = measurable;
this.config = config;
this.time = time;
}
// some good code here, but no public constructor and the class is final... of course.
}
But powermock is not used in Spring Boot.
What should I do?
Add latest stable powermock - both in spring-boot-dependencies and spring-boot-actuator.
Add latest stable powermock to spring-boot-actuator only - let it be a hidden secret helper.
Exclude the tests which require powermock.
Forget about Kafka Metrics, they're huge and scary and nobody wants them in our nice and friendly Spring Boot Actuator.
Having your confirmation in the comments, I'm just copying here my comment as a valid answer to close the question properly.
You can try to use Metrics.addMetric() to create KafkaMetric instance and than react to it via your KafkaStatisticsProvider.metricChange(), which you will register in the Metrics instance before testing. Instead of Powermock, of course.
I will try to find time next week to contribute to your actuator project that part to avoid Powermock.
Related
I would like to record the startup information about my application during a Spring Boot test. I have the startup actuator configured and working in Spring Boot 'bootrun' mode. However, when I try to access that actuator during a test using a TestRestTemplate, I get a 404 error.
I have written an example program that demonstrates the problem. The issue isn't with acutators overall as I have the metrics and health actuators working in the same test. Just the startup actuator.
The example code is on GitHub
I have a solution for this so I thought I would post it. For complete details, see the original repo in GitHub and check the solution branch.
One possible way to enable ApplicationStartup data collection during a Spring Boot Test is to create a ContextCustomizer. This allows you to get into the testing context early enough to record all of the data that you are looking for. The ContextCustomizer should have a single static BufferingApplicationStartup that it registers as a singleton bean into the test context's bean factory. It also needs to set the bean factory's ApplicationStartup because that will be passed to the SpringApplication just before it is run.
Here is the snippet of the customizer that holds the key:
#Override
public void customizeContext(ConfigurableApplicationContext context, MergedContextConfiguration mergedConfig) {
ConfigurableListableBeanFactory beanFactory = context.getBeanFactory();
Object possibleSingleton = beanFactory.getSingleton(BEAN_NAME);
// The only way it wouldn't be an instance of a BufferingApplicationStartup is if it is null or we haven't
// run yet (and it is the DefaultApplicationStartup). In either case, jam our BufferingApplicationStartup
// in here.
if(!(possibleSingleton instanceof BufferingApplicationStartup)) {
beanFactory.registerSingleton(BEAN_NAME, APPLICATION_STARTUP);
beanFactory.setApplicationStartup(APPLICATION_STARTUP);
}
}
When you do this, make sure you implement a good equals and hashCode for your customizer or else you will break the test context caching and you will refresh your test context with every test class. Since the only relevant part of the customizer is the static BufferingApplicationStartup, I chose to return its hashcode.
Finally, don't forget to add your ContextCustomizerFactory to the src/test/resources/META-INF/spring.factories or else the rest of the Spring Boot testing support won't see your customizer.
Once this is all setup, you can access the Startup Actuator endpoint just like you would any other actuator.
This is regarding CDI spec of quarkus. Would want to understand is there a configuration bean for quarkus? How does one do any sort of configuration in quarkus?
If I get it right the original question is about #Configuration classes that can contain #Bean definitions. If so then CDI producer methods and fields annotated with #javax.enterprise.inject.Produces are the corresponding alternative.
Application configuration is a completely different question though and Jay is right that the Quarkus configuration reference is the ultimate source of information ;-).
First of all reading how cdi spec of quarkus differs from spring is important.
Please refer this guide:
https://quarkus.io/guides/cdi-reference
The learnings from this guide is there is #Produces which is an alternative to #Configuration bean in Quarkus.
Let us take an example for libs that might require a configuration through code. Example: Microsoft Azure IOT Service Client.
public class IotHubConfiguration {
#ConfigProperty(name="iothub.device.connection.string")
String connectionString;
private static final Logger LOG = Logger.getLogger(IotHubConfiguration.class);
#Produces
public ServiceClient getIot() throws URISyntaxException, IOException {
LOG.info("Inside Service Client bean");
if(connectionString==null) {
LOG.info("Connection String is null");
throw new RuntimeException("IOT CONNECTION STRING IS NULL");
}
ServiceClient serviceClient = new ServiceClient(connectionString, IotHubServiceClientProtocol.AMQPS);
serviceClient.open();
LOG.info("opened Service Client Successfully");
return serviceClient;
}
For all libs vertically intergrated with quarkus application.properties can be used and then you will get a driver obj for that broker/dbs available directly through #Inject in your #applicationScoped/#Singleton bean So, Why is that?
To Simplify and Unify Configuration
To Make Sure no code is required for configuring anything i.e. database config, broker config , quarkus config etc.
This drastically reduces the amount of code written for configuring and also Junits needed to cover that code.
Let us take an example where kafka producer configuration needs to be added: in application.properties
kafka.bootstrap.servers=${KAFKA_BROKER_URL:localhost:9092}
mp.messaging.outgoing.incoming_kafka_topic_test.topic=${KAFKA_INPUT_TOPIC_FOR_IOT_HUB:input_topic1}
mp.messaging.outgoing.incoming_kafka_topic_test.connector=smallrye-kafka
mp.messaging.outgoing.incoming_kafka_topic_test.value.deserializer=org.apache.kafka.common.serialization.StringDeserializer
mp.messaging.outgoing.incoming_kafka_topic_test.key.deserializer=org.apache.kafka.common.serialization.StringDeserializer
mp.messaging.outgoing.incoming_kafka_topic_test.health-readiness-enabled=true
For full blown project reference: https://github.com/JayGhiya/QuarkusExperiments/tree/initial_version_v1/KafkaProducerQuarkus
Quarkus References for Config:
https://quarkus.io/guides/config-reference
Example for reactive sql config: https://quarkus.io/guides/reactive-sql-clients
Now let us talk about a bonus feature that quarkus provides which improves developer experience by atleast an order of magnitude that is profile driven development and testing.
Quarkus provides three profiles:
dev - Activated when in development mode (i.e. quarkus:dev)
test - Activated when running tests
prod - The default profile when not running in development or test
mode
Let us just say that in the given example you wanted to have different topics for development and different topics for production. Let us achieve that!
%dev.mp.messaging.outgoing.incoming_kafka_topic_test.topic=${KAFKA_INPUT_TOPIC_FOR_IOT_HUB:input_topic1}
%prod.mp.messaging.outgoing.incoming_kafka_topic_test.topic=${KAFKA_INPUT_TOPIC_FOR_IOT_HUB:prod_topic}
This is how simple it is. This is extremely useful in cases where your deployments run with ssl enabled brokers/dbs etc and for dev purposes you have unsecure local brokers/dbs. This is a game changer.
The OpenApi documentation says that it supports micrometer. How does the integration works? I could not find anything except this little documentation.
I have a FeignClient in a spring boot application
#FeignClient(name = "SomeService", url = "xxx", configuration = FeignConfiguration.class)
public interface SomeService {
#GET
#Path("/something")
Something getSomething();
}
with the configuration
public class FeignConfiguration {
#Bean
public Capability capability() {
return new MicrometerCapability();
}
}
and the micrometer integration as a dependency
<dependency>
<groupId>io.github.openfeign</groupId>
<artifactId>feign-micrometer</artifactId>
<version>10.12</version>
</dependency>
The code makes a call but I could not find any new metrics via the actuator overview, expecting some general information about my HTTP requests. What part is missing?
Update
I added the support for this to spring-cloud-openfeign. After the next release (2020.0.2), if micrometer is set-up, the only thing you need to do is putting feign-micrometer onto your classpath.
Old answer
I'm not sure if you do but I recommend to use spring-cloud-openfeign which autoconfigures Feign components for you. Unfortunately, it seems it does not autoconfigure Capability (that's one reason why your solution does not work) so you need to do it manually, please see the docs how to do it.
I was able to make this work combining the examples in the OpenFeign and Spring Cloud OpenFeign docs:
#Import(FeignClientsConfiguration.class)
class FooController {
private final FooClient fooClient;
public FooController(Decoder decoder, Encoder encoder, Contract contract, MeterRegistry meterRegistry) {
this.fooClient = Feign.builder()
.encoder(encoder)
.decoder(decoder)
.contract(contract)
.addCapability(new MicrometerCapability(meterRegistry))
.target(FooClient.class, "https://PROD-SVC");
}
}
What I did:
Used spring-cloud-openfeign
Added feign-micrometer (see feign-bom)
Created the client in the way you can see above
Importing FeignClientsConfiguration and passing MeterRegistry to MicrometerCapability are vital
After these, and calling the client, I had new metrics:
feign.Client
feign.Feign
feign.codec.Decoder
feign.codec.Decoder.response_size
I am using spring boot, and I have two external properties files, so that I can easily change its value.
But I hope spring app will reload the changed value when it is updated, just like reading from files. Since property file is easy enough to meet my need, I hope I don' nessarily need a db or file.
I use two different ways to load property value, code sample will like:
#RestController
public class Prop1Controller{
#Value("${prop1}")
private String prop1;
#RequestMapping(value="/prop1",method = RequestMethod.GET)
public String getProp() {
return prop1;
}
}
#RestController
public class Prop2Controller{
#Autowired
private Environment env;
#RequestMapping(value="/prop2/{sysId}",method = RequestMethod.GET)
public String prop2(#PathVariable String sysId) {
return env.getProperty("prop2."+sysId);
}
}
I will boot my application with
-Dspring.config.location=conf/my.properties
I'm afraid you will need to restart Spring context.
I think the only way to achieve your need is to enable spring-cloud. There is a refresh endpoint /refresh which refreshes the context and beans.
I'm not quite sure if you need a spring-cloud-config-server (its a microservice and very easy to build) where your config is stored(Git or svn). Or if its also useable just by the application.properties file in the application.
Here you can find the doc to the refresh scope and spring cloud.
You should be able to use Spring Cloud for that
Add this as a dependency
compile group: 'org.springframework.cloud', name: 'spring-cloud-starter', version: '1.1.2.RELEASE'
And then use #RefreshScope annotation
A Spring #Bean that is marked as #RefreshScope will get special treatment when there is a configuration change. This addresses the problem of stateful beans that only get their configuration injected when they are initialized. For instance if a DataSource has open connections when the database URL is changed via the Environment, we probably want the holders of those connections to be able to complete what they are doing. Then the next time someone borrows a connection from the pool he gets one with the new URL.
Also relevant if you have Spring Actuator
For a Spring Boot Actuator application there are some additional management endpoints:
POST to
/env to update the Environment and rebind #ConfigurationProperties and log levels
/refresh for re-loading the boot strap context and refreshing the #RefreshScope beans
Spring Cloud Doc
(1) Spring Cloud's RestartEndPoint
You may use the RestartEndPoint: Programatically restart Spring Boot application / Refresh Spring Context
RestartEndPoint is an Actuator EndPoint, bundled with spring-cloud-context.
However, RestartEndPoint will not monitor for file changes, you'll have to handle that yourself.
(2) devtools
I don't know if this is for a production application or not. You may hack devtools a little to do what you want.
Take a look at this other answer I wrote for another question: Force enable spring-boot DevTools when running Jar
Devtools monitors for file changes:
Applications that use spring-boot-devtools will automatically restart
whenever files on the classpath change.
Technically, devtools is built to only work within an IDE. With the hack, it also works when launched from a jar. However, I may not do that for a real production application, you decide if it fits your needs.
I know this is a old thread, but it will help someone in future.
You can use a scheduler to periodically refresh properties.
//MyApplication.java
#EnableScheduling
//application.properties
management.endpoint.refresh.enabled = true
//ContextRefreshConfig.java
#Autowired
private RefreshEndpoint refreshEndpoint;
#Scheduled(fixedDelay = 60000, initialDelay = 10000)
public Collection<String> refreshContext() {
final Collection<String> properties = refreshEndpoint.refresh();
LOGGER.log(Level.INFO, "Refreshed Properties {0}", properties);
return properties;
}
//add spring-cloud-starter to the pom file.
Attribues annotated with #Value is refreshed if the bean is annotated with #RefreshScope.
Configurations annotated with #ConfigurationProperties is refreshed without #RefreshScope.
Hope this will help.
You can follow the ContextRefresher.refresh() code implements.
public synchronized Set<String> refresh() {
Map<String, Object> before = extract(
this.context.getEnvironment().getPropertySources());
addConfigFilesToEnvironment();
Set<String> keys = changes(before,
extract(this.context.getEnvironment().getPropertySources())).keySet();
this.context.publishEvent(new EnvironmentChangeEvent(context, keys));
this.scope.refreshAll();
return keys;
}
I am upgrading my project from spring-boot 1.5.12.release to 2.1.9.release. I am unable to find LoggersMvcEndpoint (https://docs.spring.io/spring-boot/docs/1.5.12.RELEASE/api/org/springframework/boot/actuate/endpoint/mvc/LoggersMvcEndpoint.html) in latest version.
In one of my controller I had this. Can some one help me to fix this.
public class LoggerController extends CloudRestTemplate {
#Autowired
LoggersMvcEndpoint loggerAPI;
#Override
public Object getFromInternalApi(final String param) {
return StringUtils.isEmpty(param) ? loggerAPI.invoke() : loggerAPI.get(param);
}
#Override
public Object postToInternalApi(final String param, final Object request) {
return loggerAPI.set(param, (Map<String, String>) request);
}
}
As per Spring docs here
Endpoint infrastructure
Spring Boot 2 brings a brand new endpoint
infrastructure that allows you to define one or several operations in
a technology independent fashion with support for Spring MVC, Spring
WebFlux and Jersey! Spring Boot 2 will have native support for Jersey
and writing an adapter for another JAX-RS implementation should be
easy as long as there is a way to programmatically register resources.
The new #Endpoint annotation declares this type to be an endpoint with
a mandatory, unique id. As we will see later, a bunch of properties
will be automatically inferred from that. No additional code is
required to expose this endpoint at /applications/loggers or as a
org.springframework.boot:type=Endpoint,name=Loggers JMX MBean.
Refer to documentation, it will help you further
and for your info LoggersMvcEndpoint was there until 2.0.0.M3 https://docs.spring.io/spring-boot/docs/2.0.0.M3/api/org/springframework/boot/actuate/endpoint/mvc/LoggersMvcEndpoint.html however there is no reference of deprecation in subsequent version's release notes of 2.0.0.M4
https://docs.spring.io/spring-boot/docs/2.0.0.M4/api/deprecated-list.html#class