While trying to configure Reactor Netty wiretap feature on a WebFlux client to see the generated requests (and more detailed response), I've ended up with the following working setup:
In a service bean:
private WebClient client = WebClient.builder()
.clientConnector(new ReactorClientHttpConnector(
HttpClient.create().wiretap("webClientLogger", LogLevel.DEBUG, AdvancedByteBufFormat.TEXTUAL)
))
.baseUrl("https://...")
.defaultHeader(HttpHeaders.ACCEPT, MediaType.APPLICATION_NDJSON_VALUE)
.codecs(codecs -> codecs.defaultCodecs().jackson2JsonDecoder(new Jackson2JsonDecoder(Jackson2ObjectMapperBuilder.json().build(),
new MimeType("application", "json"), new MimeType("application", "x-ndjson"))))
.build();
and in application.properties:
logging.level.webClientLogger = DEBUG
My question is, why, given that both the name and the level of the logger seems to be already defined in the #wiretap method's signature, the above line in the application.property is still required (as commenting it out stops producing any log output).
Relevant Reactor Netty documentation seems to reference reactor.netty.http.client.HttpClient, however adding reactor.netty.http.client.HttpClient=DEBUG to application.properties without it having the above mentioned line doesn't make any difference and still doesn't produce the log output.
As a note, I'm using Lombok with its respective annotation #Slf4j to materialize the logging behavior.
Related
I have a spring boot application having dependency of spring-cloud-starter-sleuth-3.0.3 and spring-cloud-sleuth-zipkin-3.0.3.
I have a requirement that i need to pass trace-id to request header while calling API from webclient.
Demo webClient
#Slf4j
#Component
#RequiredArgsConstructor
public class DemoApiClient {
private final WebClient demoWebClient;
private final DemoProperties demoProperties;
private final Tracer tracer;
public Mono<DemoDetail> retrieveDemoDetail(String demo){
return demoWebClient
.get()
.uri(uriBuilder->uriBuilder
.path(demoProperties.getLookupPath())
.build(demo))
.header("trace-id", tracer.currentSpan().context().traceId())
.accept(APPLICATION_JSON)
.retrieve()
.bodyToMono(DemoDetail.class)
.doOnError(e -> log.error("Could not find demo", e));
}
}
tracer.currentSpan() is coming as null , hence NPE is thrown.
As per document, approach is given to add trace-id to header of response
https://docs.spring.io/spring-cloud-sleuth/docs/3.0.3/reference/html/howto.html#how-to-add-headers-to-the-http-server-response.
However, i need correct approach to add trace-id to request header.
WebClient is instrumented, please see the docs: WebClient integration, so tracing information should be propagated out of the box over the wire.
If you want to do this manually (I don't recommend), you need to check what you do "above" this method that prevents the tracing information to be propagated. E.g.: you are switching threads, coming from an imperative context to Reactor, etc. You can work this around by getting the tracing information before the switch and either propagate it (see Scope) or inject it into this method.
Also, you are not sending the whole trace context just the traceId so please check the docs and let Sleuth propagate the tracing information for you.
If you are creating a WebClient bean, you are not switching threads, or going back and forth between imperative and reactive, and you still don't see the tracing information in your header (propagated by Sleuth), you can try modifying the instrumentation mechanism, I recommend using DECORATE_QUEUES.
Also, Sleuth 3.1.x is out, you can try upgrading it.
The OpenApi documentation says that it supports micrometer. How does the integration works? I could not find anything except this little documentation.
I have a FeignClient in a spring boot application
#FeignClient(name = "SomeService", url = "xxx", configuration = FeignConfiguration.class)
public interface SomeService {
#GET
#Path("/something")
Something getSomething();
}
with the configuration
public class FeignConfiguration {
#Bean
public Capability capability() {
return new MicrometerCapability();
}
}
and the micrometer integration as a dependency
<dependency>
<groupId>io.github.openfeign</groupId>
<artifactId>feign-micrometer</artifactId>
<version>10.12</version>
</dependency>
The code makes a call but I could not find any new metrics via the actuator overview, expecting some general information about my HTTP requests. What part is missing?
Update
I added the support for this to spring-cloud-openfeign. After the next release (2020.0.2), if micrometer is set-up, the only thing you need to do is putting feign-micrometer onto your classpath.
Old answer
I'm not sure if you do but I recommend to use spring-cloud-openfeign which autoconfigures Feign components for you. Unfortunately, it seems it does not autoconfigure Capability (that's one reason why your solution does not work) so you need to do it manually, please see the docs how to do it.
I was able to make this work combining the examples in the OpenFeign and Spring Cloud OpenFeign docs:
#Import(FeignClientsConfiguration.class)
class FooController {
private final FooClient fooClient;
public FooController(Decoder decoder, Encoder encoder, Contract contract, MeterRegistry meterRegistry) {
this.fooClient = Feign.builder()
.encoder(encoder)
.decoder(decoder)
.contract(contract)
.addCapability(new MicrometerCapability(meterRegistry))
.target(FooClient.class, "https://PROD-SVC");
}
}
What I did:
Used spring-cloud-openfeign
Added feign-micrometer (see feign-bom)
Created the client in the way you can see above
Importing FeignClientsConfiguration and passing MeterRegistry to MicrometerCapability are vital
After these, and calling the client, I had new metrics:
feign.Client
feign.Feign
feign.codec.Decoder
feign.codec.Decoder.response_size
I have one question regarding Spring WebClient
In my application I need to do many similar API calls, sometimes I need change headers in the calls (Authentication token). So the question arises, what would be better of the two options:
To create one WebClient for all incoming requests to MyService.class, by making it private final field, like code below:
private final WebClient webClient = WebClient.builder()
.baseUrl("https://another_host.com/api/get_inf")
.defaultHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
.defaultHeader(HttpHeaders.ACCEPT, MediaType.APPLICATION_JSON_VALUE)
.build();
Here arises another question: is WebClient thread-safe? (because service is used by many threads)
To create new WebClient for each new request incoming to service class.
I want to provide maximum performance, and to use it in right way, but I don't know how WebClient works inside it, and how it expects to be used.
Thank you.
Two key things here about WebClient:
Its HTTP resources (connections, caches, etc) are managed by the underlying library, referenced by the ClientHttpConnector that you can configure on the WebClient
WebClient is immutable
With that in mind, you should try to reuse the same ClientHttpConnector across your application, because this will share the connection pool - this is arguably the most important thing for performance. This means you should try to derive all WebClient instances from the same WebClient.create() call. Spring Boot helps you with that by creating and configuring for you a WebClient.Builder bean that you can inject anywhere in your app.
Because WebClient is immutable it is thread-safe. WebClient is meant to be used in a reactive environment, where nothing is tied to a particular thread (this doesn't mean you cannot use in a traditional Servlet application).
If you'd like to change the way requests are made, there are several ways to achieve that:
configure things in the builder phase
WebClient baseClient = WebClient.create().baseUrl("https://example.org");
configure things on a per-request basis
Mono<ClientResponse> response = baseClient.get().uri("/resource")
.header("token", "secret").exchange();
create a new client instance out of an existing one
// mutate() will *copy* the builder state and create a new one out of it
WebClient authClient = baseClient.mutate()
.defaultHeaders(headers -> {headers.add("token", "secret");})
.build();
From my experience, if you are calling an external API on a server you have no control over, don't use WebClient at all, or use it with the pooling mechanism turned off. Any performance gains from connection pooling are greatly overweighed by the assumptions built into the (default reactor-netty) library that will cause random errors on one API call when another was abruptly terminated by the remote host, etc. In some cases, you don't even know where the error occurred because the calls are all made from a shared worker thread.
I made the mistake of using WebClient because the doc for RestTemplate said it would be deprecated in the future. In hindsight, I would go with regular HttpClient or Apache Commons HttpClient, but if you are like me and already implemented with WebClient, you can turn off the pooling by creating your WebClient as follows:
private WebClient createWebClient(int timeout) {
TcpClient tcpClient = TcpClient.newConnection();
HttpClient httpClient = HttpClient.from(tcpClient)
.tcpConfiguration(client -> client.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, timeout * 1000)
.doOnConnected(conn -> conn.addHandlerLast(new ReadTimeoutHandler(timeout))));
return WebClient.builder()
.clientConnector(new ReactorClientHttpConnector(httpClient))
.build();
}
*** Creating a separate WebClient does not mean that WebClient will have a separate connection pool. Just look at the code for HttpClient.create - it calls HttpResources.get() to get the global resources. You could provide the pool settings manually but considering the errors that occur even with the default setup, I don't consider it worth the risk.
I am trying to configure a Spring Batch listener to send a message to a Spring Integration Gateway for StepExecution events.
The following link explains how to configure this with XML
http://docs.spring.io/spring-batch/trunk/reference/html/springBatchIntegration.html#providing-feedback-with-informational-messages
How can this be setup using Spring Integration DSL? I've found no way to configure a gateway with a service interface using DSL.
At the moment I worked around this by implementing an actual StepExecutionListener, and have this then calling an interface which is annotated with #MessagingGateway (calling the corresponding #Gateway method) in order to get a message to a channel. And I then setup an Integration DSL flow for this channel.
Is there a simpler way using DSL, avoiding that workaround? Is there some way to connect a Batch listener direct to a gateway, like one can using XML config?
Cheers,
Menno
First of all SI DSL is just an extension of existing SI Java and Annotation configuration, so it can be used together with any other Java config. Of course an XML #Import is also posible.
There is no gateway configuration in the DSL, because its methods can't be wired with linear IntegrationFlow. There is need to provide downstream flows for each method.
So, #MessagingGateway is a right way to go ahead:
#MessagingGateway(name = "notificationExecutionsListener", defaultRequestChannel = "stepExecutionsChannel")
public interface MyStepExecutionListener extends StepExecutionListener {}
From other side #MessagingGateway parsing as well as <gateway> tag parsing ends up with GatewayProxyFactoryBean definition. So, you just can declare that bean, if you don't want to introduce a new class:
#Bean
public GatewayProxyFactoryBean notificationExecutionsListener(MessageChannel stepExecutionsChannel) {
GatewayProxyFactoryBean gateway = new GatewayProxyFactoryBean(StepExecutionListener.class);
gateway.setDefaultRequestChannel(stepExecutionsChannel);
return gateway;
}
After the latest Milestone 3 I have an idea to introduce nested flows, when we may be able to introduce Gateway support for flows. Something like this:
#Bean
public IntegrationFlow gatewayFlow() {
return IntegrationFlows
.from(MyGateway.class, g ->
g.method("save", f -> f.transform(...)
.filter(...))
.method("delete", f -> f.handle(...)))
.handle(...)
.get();
}
However I'm not sure that it will simplify the life, as far as any nested Lambda just adds more noise and might break loosely coupling principle.
my resttemplate.exchange() failed on a POST request, with the server returning a 500 error.
I tried to set the root logging level to DEBUG, but nothing was logged before the 500 error was returned. to make sure that my logging config is right, I added a line before the resttemplate call
HttpClient client = new DefaultHttpClient();
client.execute(new HttpGet("http://google.com"));
in this case indeed a lot of logging messages appeared.
so how can I make RestTemplate export the debugging data?
Thanks
Yang
From your analysis it seems that you expect RestTemplate to use Apache HttpClient.
However, by default, Spring RestTemplate does not use Apache HttpClient but uses the JDK facilities (java.net.URL#openConnection() etc.) by means of SimpleClientHttpRequestFactory.
org.springframework.http.client.support.HttpAccessor declares:
private ClientHttpRequestFactory requestFactory = new
SimpleClientHttpRequestFactory();
As far as I know, this client does not support logging requests/responses.
To change RestTemplate to use HttpClient, try this:
new RestTemplate(new HttpComponentsClientHttpRequestFactory());
Logging configuration should then enable category org.apache.http.wire at level debug for complete requests/responses to be logged.