I have a Spring Boot 2, Java 8 application using Resilience4j (version 1.3) CircuitBreaker (CB). The default CB properties in my application yaml look like this -
resilience4j.circuitbreaker:
configs:
default:
registerHealthIndicator: true
slidingWindowSize: 10
minimumNumberOfCalls: 5
permittedNumberOfCallsInHalfOpenState: 3
automaticTransitionFromOpenToHalfOpenEnabled: true
waitDurationInOpenState: 10000
failureRateThreshold: 50
eventConsumerBufferSize: 10
recordExceptions:
- org.springframework.web.client.HttpServerErrorException
- java.util.concurrent.TimeoutException
- java.io.IOException
ignoreExceptions:
- org.springframework.web.client.HttpClientErrorException
I was hoping every CB I create would end up with these default values but it does not appear to be the case.
// Case 1: Does not load defaults from yaml
#Bean
CircuitBreaker xCircuitBreaker(CircuitBreakerRegistry circuitBreakerRegistry) {
return circuitBreakerRegistry.circuitBreaker("xCircuitBreaker");
}
// Case 2: Does not load defaults from yaml
#Bean
CircuitBreaker yCircuitBreaker() {
return CircuitBreaker.ofDefaults("yCircuitBreaker");
}
// Case 3: Works
#Bean
public CircuitBreaker zCircuitBreaker(CircuitBreakerRegistry circuitBreakerRegistry,
#Value("${resilience4j.circuitbreaker.configs.default.slidingWindowSize}") Integer slidingWindowSize,
#Value("${resilience4j.circuitbreaker.configs.default.permittedNumberOfCallsInHalfOpenState}") Integer permittedNumberOfCallsInHalfOpenState,
#Value("${resilience4j.circuitbreaker.configs.default.minimumNumberOfCalls}") Integer minimumNumberOfCalls,
#Value("${resilience4j.circuitbreaker.configs.default.failureRateThreshold}") Integer failureRateThreshold,
#Value("${resilience4j.circuitbreaker.configs.default.waitDurationInOpenState}") Integer waitDurationInOpenState,
#Value("${resilience4j.circuitbreaker.configs.default.automaticTransitionFromOpenToHalfOpenEnabled}") Boolean automaticTransitionFromOpenToHalfOpenEnabled) {
CircuitBreakerConfig cfg = CircuitBreakerConfig.custom()
.slidingWindowSize(slidingWindowSize)
.minimumNumberOfCalls(minimumNumberOfCalls)
.permittedNumberOfCallsInHalfOpenState(permittedNumberOfCallsInHalfOpenState)
.failureRateThreshold(failureRateThreshold)
.waitDurationInOpenState(Duration.ofMillis(waitDurationInOpenState))
.automaticTransitionFromOpenToHalfOpenEnabled(automaticTransitionFromOpenToHalfOpenEnabled)
.recordException(recordFailurePredicate())
.recordExceptions(HttpServerErrorException.class)
.ignoreExceptions(HttpClientErrorException.class)
.build();
return circuitBreakerRegistry.circuitBreaker("zCircuitBreaker", cfg);
}
I am hoping I don't have to resort to case 3. Is there something I am doing incorrectly in Case 1 and 2? For the purposes of this testing, I am using an InMemoryCircuitBreakerRegistry. I appreciate your help.
Thank you
Please use the Spring Boot starter from Resilience4j and don't create a CircuitBreaker manually via a bean. Everything is done automatically for you on-the-fly.
Related
When using Spring Cloud Gateway (v3.1.3), how would one go about defining a default filter to perform retries for all routes, but then disable it for individual routes? I would like something as intuitive as this:
spring:
cloud:
gateway:
default-filters:
- name: Retry
args:
retries: 3
statuses: BAD_GATEWAY
methods: GET,POST,PUT,DELETE
backoff:
firstBackoff: 10ms
maxBackoff: 50ms
factor: 2
basedOnPreviousValue: false
routes:
- id: retry_disabled
uri: http://localhost:8080/retry_disabled
filters:
- name: Retry
args:
retries: 0
- id: retry_enabled
uri: http://localhost:8080/retry_enabled
I see in the RetryGatewayFilterFactory class that the RetryConfig.validate() method will fail when the number of retries is less than 1 or the other config options are not defined properly:
public void validate() {
Assert.isTrue(this.retries > 0, "retries must be greater than 0");
Assert.isTrue(!this.series.isEmpty() || !this.statuses.isEmpty() || !this.exceptions.isEmpty(),
"series, status and exceptions may not all be empty");
Assert.notEmpty(this.methods, "methods may not be empty");
if (this.backoff != null) {
this.backoff.validate();
}
}
Edit: I'm considering to implement it like this in code:
#Bean
public Function<GatewayFilterSpec, UriSpec> defaultRetryGatewayFilter() {
return gatewayFilterSpec -> gatewayFilterSpec
.retry(retryConfig -> {
RetryGatewayFilterFactory.BackoffConfig backoffConfig = new RetryGatewayFilterFactory.BackoffConfig();
backoffConfig.setFirstBackoff(Duration.ofMillis(10));
backoffConfig.setMaxBackoff(Duration.ofMillis(50));
backoffConfig.setFactor(2);
backoffConfig.setBasedOnPreviousValue(false);
retryConfig
.setRetries(3)
.allMethods()
.setSeries(HttpStatus.Series.SERVER_ERROR)
.setStatuses(HttpStatus.BAD_GATEWAY)
.setBackoff(backoffConfig);
});
}
#Bean
public RouteLocator routes(RouteLocatorBuilder builder, Function<GatewayFilterSpec, UriSpec> defaultRetryGatewayFilter) {
return builder.routes()
.route("retry_enabled", r -> r
.path("/retry_enabled")
.filters(defaultRetryGatewayFilter)
.uri("lb://foo"))
.route("retry_disabled", r -> r
.path("/retry_disabled")
// not retryable
.uri("lb://foo"))
.build();
}
Will the singleton defaultRetryGatewayFilter be thread safe?
I'm mapping json array response to reactor world but have an issue like:
val responses = configurationClient.getData() // return json array object
.flatMap { it.bodyToMono(object : ParameterizedTypeReference<GeneralResponse<Array<ObjectResponse>>>() {})
}
.map { it.data }
.blockOptional() // exception this line
.orElse(emptyArray())!!
This snipcode doesn't work if I add this property of spring actuator
management.endpoints.enabled-by-default=true
Netty server cannot start without any exceptions.
But It works when I change to
management.endpoints.enabled-by-default=false
The Netty started well
Any ideas this issue please?
** Updated **
When I add some timeout value .blockOptional(Duration.ofSeconds(60)) //60 seconds
val responses = configurationClient.getData() // return json array object
.flatMap { it.bodyToMono(object : ParameterizedTypeReference<GeneralResponse<Array<ObjectResponse>>>() {})
}
.map { it.data }
.blockOptional(Duration.ofSeconds(60))
.get()
I pretty sure conflict somewhere between Mono and Spring actuator management.endpoints.enabled-by-default=true
at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:886)
at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:790)
... 20 common frames omitted
Caused by: java.lang.IllegalStateException: Timeout on blocking read for 60000 MILLISECONDS
at reactor.core.publisher.BlockingOptionalMonoSubscriber.blockingGet(BlockingOptionalMonoSubscriber.java:162)
at reactor.core.publisher.Mono.blockOptional(Mono.java:1755)
Note
configurationClient.getData() this just a GET request return 200-[{...}]
Everything work if I use
management.endpoints.enabled-by-default=false
problem is solved.
Root cause:
It's not a bug of Mono:blockOptional or Spring actuator individually.
This configuration management.endpoints.enabled-by-default=true is conflict with existing actuator endpoint configurations
Solution:
Cleanup spring actuator configuration properties to avoid this conflict then Mono:blockOptional work well
I had a grails 3 app including spring security, which I recently upgraded to grails 4.
My application.yml includes the following:
environments:
test:
grails:
plugin:
springsecurity:
active: false
security:
ignored: '/**'
basic:
enabled: false
spring:
security:
enabled: false
Why doesn't this work in Grails 4? What's a good alternative solution?
Grails 4 seems to be ignoring this configuration. When I run integration tests, I am getting a 403 error with a message:
Could not verify the provided CSRF token because your session was not found.
It seems like spring security enabled, and it's using SecurityFilterAutoConfiguration, which is normally excluded for my app.
Update
I am using the following dependencies:
compile('org.grails.plugins:spring-security-core:3.2.3') {
exclude group: 'org.springframework.security'
}
compile ('org.springframework.security:spring-security-core:4.2.13.RELEASE') {
force = true
}
compile 'org.springframework.security:spring-security-web:4.2.13.RELEASE'
compile 'org.springframework.security:spring-security-config:4.2.13.RELEASE'
Update 2:
In my debugger, I found that the spring security core plugin actually is being disabled. The following code from the plugin class is executed:
SpringSecurityUtils.resetSecurityConfig()
def conf = SpringSecurityUtils.securityConfig
boolean printStatusMessages = (conf.printStatusMessages instanceof Boolean) ? conf.printStatusMessages : true
if (!conf || !conf.active) {
if (printStatusMessages) {
// <-- the code in this block is executed; active flag is false
String message = '\n\nSpring Security is disabled, not loading\n\n'
log.info message
println message
}
return
}
...however, I am still getting the CSRF filter error, so Spring Security must be configuring itself somehow regardless.
Update 3:
The CSRF filter is being set up by ManagementWebSecurityConfigurerAdapter, using the default configuration.
I tried adding the following to resources.groovy:
if (grailsApplication.config.disableSecurity == true && !Environment.isWarDeployed()) {
webSecurityConfigurerAdapter(new WebSecurityConfigurerAdapter(true) {})
}
This did not fix the issue. Although my anonymous WSCA bean is being constructed, the MWSCA default bean is still being used by spring.
Try this in
grails-app/conf/application.groovy
environments {
development {
}
test {
grails.plugin.springsecurity.active = false
}
production {
}
}
My Team migrated our Microservices from Spring Boot 1 to Version 2 and since the Actuator changed, our Health Endpoint Metrics exported via prometheus jmx exporter do not work anymore.
The usual /actuator/health is working as expected, but the prometheus-jmx-exporter won't pick it up although several things tried:
I changed the Metainformation in the exporter-config.yaml to reflect the name change in Spring Boot 2
I added the io.micrometer:micrometer-registry-prometheus to our build.gradle to see if this is the issue
I exposed web and jmx endpoints acording to the Spring Boot 2 Documentation
So now I run out of ideas and would appreciate any hints oyu might be able to give me
old prometheus-jmx-exporter exporter-config.yaml:
---
lowercaseOutputName: true
lowercaseOutputLabelNames: true
whitelistObjectNames: ["org.springframework.boot:type=Endpoint,name=healthEndpoint"]
rules:
- pattern: 'org.springframework.boot<type=Endpoint, name=healthEndpoint><(.*, )?(.*)>(.*):'
name: health_endpoint_$1$3
attrNameSnakeCase: true
new prometheus-jmx-exporter exporter-config.yaml:
---
lowercaseOutputName: true
lowercaseOutputLabelNames: true
whitelistObjectNames: ["org.springframework.boot:type=Endpoint,name=Health"]
rules:
- pattern: 'org.springframework.boot<type=Endpoint, name=Health>'
name: health_endpoint_$1$3
attrNameSnakeCase: true
current application properties about actuator endpoints:
management.endpoints.web.exposure.include=info, health, refresh, metrics, prometheus
management.endpoints.jmx.exposure.include=health, metrics, prometheus
in Spring Boot 1 with the old exporter-config.yaml I get results like this:
# HELP health_endpoint_hystrix_status Invoke the underlying endpoint (org.springframework.boot<type=Endpoint, name=healthEndpoint><hystrix, status>status)
# TYPE health_endpoint_hystrix_status untyped
health_endpoint_hystrix_status 1.0
# HELP health_endpoint_status Invoke the underlying endpoint (org.springframework.boot<type=Endpoint, name=healthEndpoint><status>status)
# TYPE health_endpoint_status untyped
health_endpoint_status 1.0
But with all the changes and in Spring Boot 2 I get nothing out of this.
You can cofigure your own health value and add it to the Prometheus Metrics endpoint:
#Configuration
public class HealthMetricsConfiguration {
#Bean
public MeterRegistryCustomizer prometheusHealthCheck(HealthEndpoint healthEndpoint) {
return registry -> registry.gauge("health", healthEndpoint, HealthMetricsConfiguration::healthToCode);
}
public static int healthToCode(HealthEndpoint ep) {
Status status = ep.health().getStatus();
return status.equals(Status.UP) ? 1 : 0;
}
}
I am using #HystrixCommand in a #Component in a Spring Boot application configured using Spring Cloud Config and the Hystrix command is not picking the configuration.
If I use the Actuator env endpoint, I see my Hystrix configuration.
"hystrix.command.default.circuitBreaker.enabled": true,
"hystrix.command.default.circuitBreaker.requestVolumeThreshold": 20,
"hystrix.command.default.circuitBreaker.sleepWindowInMilliseconds": 20000,
"hystrix.command.default.execution.isolation.strategy": "THREAD",
"hystrix.command.default.execution.isolation.thread.timeoutInMilliseconds": 10000,
"hystrix.command.default.execution.timeout.enabled": true,
"hystrix.command.default.metrics.rollingStats.numBuckets": 200,
"hystrix.command.default.metrics.rollingStats.timeInMilliseconds": 10000,
If I use the archaius endpoint I see nothing.
{}
I can see the configuration using following in my fallback. It confirms I am only getting configuration from the #HystrixCommand.
HystrixCircuitBreaker circuitBreaker = HystrixCircuitBreaker.Factory.getInstance(commandKey);
HystrixReporter.reportMetric(HystrixCommandMetrics.getInstance(commandKey), text -> logger.debug(text));
I can use the #HystrixCommand to configure the Hystric command but want dynamic run-time configuration.
Here is a snippet from my build.gradle.
springBootVersion = '1.5.4.RELEASE'
...
ext {
springCloudVersion = 'Dalston.SR1'
}
dependencies {
compile('org.springframework.boot:spring-boot-starter-actuator')
compile('org.springframework.boot:spring-boot-starter-web')
compile('org.springframework.cloud:spring-cloud-starter-config')
compile('org.springframework.cloud:spring-cloud-starter-feign')
compile('org.springframework.cloud:spring-cloud-starter-hystrix')
compile('org.springframework.integration:spring-integration-jms')
I keep seeing this comment so it should "just" work:
External Configuration: a bridge from the Spring Environment to Archaius
Here is the application.yml supplied by Spring Cloud Config server.
hystrix:
command:
default:
circuitBreaker:
enabled: true
requestVolumeThreshold: 20
sleepWindowInMilliseconds: 20000
execution:
isolation:
strategy: THREAD
thread:
timeoutInMilliseconds: 10000
timeout:
enabled: true
metrics:
rollingStats:
numBuckets: 200
timeInMilliseconds: 10000
threadpool:
default:
coreSize: 10
maximumSize: 10
maxQueueSize: -1
metrics:
polling-interval-ms: 5000