I need to fetch the entire request body in filter and convert it into String. Below is my code but nothing is getting printed on console.
#Component
public class WebFilter01 implements WebFilter {
#Override
public Mono<Void> filter(ServerWebExchange serverWebExchange,
WebFilterChain webFilterChain) {
Flux<DataBuffer> requestBody = serverWebExchange.getRequest().getBody();
Flux<String> decodedRequest = requestBody.map(databuffer -> {
return decodeDataBuffer(databuffer);
});
decodedRequest.doOnNext(s -> System.out.print(s));
return webFilterChain.filter(serverWebExchange);
}
protected String decodeDataBuffer(DataBuffer dataBuffer) {
Charset charset = StandardCharsets.UTF_8;
CharBuffer charBuffer = charset.decode(dataBuffer.asByteBuffer());
DataBufferUtils.release(dataBuffer);
String value = charBuffer.toString();
return value;
}
}
Nothing is getting printed on console because you did not subscribe to decodedRequest ,
as we know one of the Reactive aspect:
Nothing happens until you subscribe
But if you do that you will see printed body on console but your code will not work, because the next operators cannot read the body and you will get IllegalStateException(Only one connection receive subscriber allowed.)
So, how to resolve it?
Create your own wrapper for ServerWebExchange (please read about this here: How to log request and response bodies in Spring WebFlux)
Log bodies in HttpMessageDecoder. If you see, for instance, AbstractJackson2Decoder you will found code where Spring decode you buffer to object and can log it:
try {
Object value = reader.readValue(tokenBuffer.asParser(getObjectMapper()));
if (!Hints.isLoggingSuppressed(hints)) {
LogFormatUtils.traceDebug(logger, traceOn -> {
String formatted = LogFormatUtils.formatValue(value, !traceOn);
return Hints.getLogPrefix(hints) + "Decoded [" + formatted + "]";
});
}
return value;
}
Related
In my app I have one endpoint under /my-endpoint path which supports only post method. It accepts a body that must be compatible with my MyRequest class.
#Validated
data class MyRequest(
#get:JsonProperty("age", required = true)
#field:Size(min = 3, max = 128, message = "age must be between 3 and 128")
val age: String,
#get:JsonProperty("zip_code", required = true)
#field:Pattern(regexp = "\\d{2}-\\d{3}", message = "address.zip_code is invalid. It is expected to match pattern \"\\d{2}-\\d{3}\"")
val zipCode: String
)
And my controller looks like this
#PostMapping("/my-endpoint")
fun myEndpoint(
#Valid #RequestBody request: MyRequest,
): Mono<ResponseEntity<MyResponse>> {
return myService.processRequest(request)
.map { ResponseEntity.ok().body(it) }
}
Each time I receive some request to THIS particular endpoint (I have other endpoints but them should be ignored) - I'd like to publish a message to my pubsub consisting raw request body (as a string) - no matter whether the request body was valid or not.
How to intercept the request to be able to publish the message - still having the endpoint working ?
I think you could implement your own WebFilter. Filter the API path through exchange.getRequest().getPath() using simple if block and get the body through exchange.getRequest().getBody()
#Component
#RequiredArgsConstructor
#Slf4j
public class MyFilter implements WebFilter {
private final MyPublisher myPublisher;
#Override
public Mono<Void> filter(ServerWebExchange exchange, WebFilterChain chain) {
if (pathMatches(exchange.getRequest().getPath()) {
return exchange.getRequest().getBody()
.map(dataBuffer -> {
final String requestBody = dataBuffer.toString(StandardCharsets.UTF_8));
this.myPublisher.publish(requestBody).subscribe();
return exchange;
}).then(chain.filter(exchange));
}
return chain.filter(exchange);
}
}
i'm trying to send a POST request with body data as described here: https://scrapyrt.readthedocs.io/en/stable/api.html#post.
Here's what i've tried to do but it gives me HTTP code 500
String uri = "http://localhost:3000";
WebClient webClient = WebClient.builder()
.baseUrl(uri)
.build();
LinkedMultiValueMap map = new LinkedMultiValueMap();
String q = "\"url\": \"https://blog.trendmicro.com/trendlabs-security-intelligence\",\"meta\":{\"latestDate\" : \"18-05-2020\"}}";
map.add("request", q);
map.add("spider_name", "blog");
BodyInserter<MultiValueMap<String, Object>, ClientHttpRequest> inserter2
= BodyInserters.fromMultipartData(map);
Mono<ItemsList> result = webClient.post()
.uri(uriBuilder -> uriBuilder
.path("/crawl.json")
.build())
.body(inserter2)
.retrieve()
.bodyToMono(ItemsList.class);
ItemsList tempItems = result.block();
Here's what i've tried to do but it gives me HTTP code 500
Most likely because you're sending the wrong data in a mixture of wrong formats with the wrong type:
You're using multipart form data, not JSON
You're then setting the request parameter as a JSON string (q)
The JSON string you're using in q isn't even valid (it's at least missing an opening curly brace) - and handwriting JSON is almost universally a bad idea, leverage a framework to do it for you instead.
Instead, the normal thing to do would be to create a POJO structure that maps to your request, so:
public class CrawlRequest {
private CrawlInnerRequest request;
#JsonProperty("spider_name")
private String spiderName;
//....add the getters / setters
}
public class CrawlInnerRequest {
private String url;
private String callback;
#JsonProperty("dont_filter")
private String dontFilter;
//....add the getters / setters
}
...then simply create a CrawlRequest, set the values as you wish, then in your post call use:
.body(BodyInserters.fromValue(crawlRequest))
This is a rather fundamental, basic part of using a WebClient. I'd suggest reading around more widely to give yourself a better understanding of the fundamentals, it will help tremendously in the long run.
For me following code worked:
public String wcPost(){
Map<String, String> bodyMap = new HashMap();
bodyMap.put("key1","value1");
WebClient client = WebClient.builder()
.baseUrl("domainURL")
.build();
String responseSpec = client.post()
.uri("URI")
.headers(h -> h.setBearerAuth("token if any"))
.body(BodyInserters.fromValue(bodyMap))
.exchange()
.flatMap(clientResponse -> {
if (clientResponse.statusCode().is5xxServerError()) {
clientResponse.body((clientHttpResponse, context) -> {
return clientHttpResponse.getBody();
});
return clientResponse.bodyToMono(String.class);
}
else
return clientResponse.bodyToMono(String.class);
})
.block();
return responseSpec;
}
I am trying to validate and log form data that goes through Spring Cloud Gateway. I have tried a few methods and encounter a few problems and I could not read it properly. I have tried:
#Component
public class GatewayRequestFilter {
#Bean
public GlobalFilter apply() {
return (exchange, chain) -> {
MediaType contentType = exchange.getRequest().getHeaders().getContentType();
ModifyRequestBodyGatewayFilterFactory.Config modifyRequestConfig = new ModifyRequestBodyGatewayFilterFactory.Config();
/// Method 1
if (contentType.includes(MediaType.MULTIPART_FORM_DATA)) {
modifyRequestConfig.setRewriteFunction(String.class, String.class, (exchange1, originalRequestBody) -> {
validateAndAuditLog(exchange1, originalRequestBody);
return Mono.just(originalRequestBody);
});
}
/// Method 2
if (contentType.includes(MediaType.MULTIPART_FORM_DATA)) {
return exchange.getMultipartData().flatMap(originalRequestBody -> {
validateAndAuditLog(exchange1, originalRequestBody);
return chain.filter(exchange);
});
}
/// Method 3:
/// https://github.com/spring-cloud/spring-cloud-gateway/issues/1307#issuecomment-553910834
return new ModifyRequestBodyGatewayFilterFactory().apply(modifyRequestConfig).filter(exchange, chain);
};
}
}
For the 1st and 3rd method, if I set inClass as String.class then I can see data in some kind of http format. The problem is that I don't know how to parse it into hashMap or LinkedMultiValueMap to access each of value using key. Here is the output I get:
----------------------------162653831591335516327921
Content-Disposition: form-data; name="simple-text"
text
----------------------------162653831591335516327921
Content-Disposition: form-data; name="simple-file"; filename="simple-file"
Content-Type: application/octet-stream
Simple file
----------------------------162653831591335516327921--
If I change inClass as Object.class then there is another error:
{
"timestamp": "2020-04-03T02:37:57.096+0000",
"path": "/tc/test/test",
"status": 500,
"error": "Internal Server Error",
"message": "Content type 'multipart/form-data;boundary=--------------------------537619313111072161580699' not supported for bodyType=java.lang.Object",
"requestId": "0592497a-1"
}
For the 2nd method I can get data in LinkedMultiValueMap which is good because I can read each data using key value and I can also get uploaded files name, but the problem is that, it hang for 10s before pass the request to down stream.
Anyone has any idea what should I do to read or modify form data that goes through Spring Cloud Gateway?
Rewriting the answer with example.
Basic approach is defined here, though it needs lot of refinement to work for multi-part.
https://developpaper.com/question/how-to-modify-the-request-parameters-of-multipart-form-data-format-in-spring-cloud-gateway/
For any approach to work once you read the data, you need to set a modified request object to exchange downstream to be processed again. Setting the new multi-part object downstream is bit tricky because there is not a straightforward way to convert string->multi-part->string.
Here is a sample code based on the approach. Note that this for now works only if multi-part contains form fields and not file type fields, because in later case we are dealing with a stream, which can be embedded anywhere within the entire multi-part request, and it is not possible to modify such request without blocking calls, which the netty does not allow.
private final List<HttpMessageReader<?>> messageReaders = HandlerStrategies.withDefaults().messageReaders();
public GatewayFilter apply(Config config) {
return new OrderedGatewayFilter((exchange, chain) -> {
ServerRequest serverRequest = ServerRequest.create(exchange, messageReaders);
// get modified body from original body o
Mono<MultiValueMap<String, String>> modifiedBody = serverRequest.bodyToMono(String.class).flatMap(o -> {
// create mock request to read body
SynchronossPartHttpMessageReader synchronossReader = new SynchronossPartHttpMessageReader();
MultipartHttpMessageReader reader = new MultipartHttpMessageReader(synchronossReader);
MockServerHttpRequest request = MockServerHttpRequest.post("").contentType(exchange.getRequest().getHeaders().getContentType()).body(o);
Mono<MultiValueMap<String, Part>> monoRequestParts = reader.readMono(MULTIPART_DATA_TYPE, request, Collections.emptyMap());
// modify parts
return monoRequestParts.flatMap(requestParts -> {
Map<String, List<String>> modifedBodyArray = requestParts.entrySet().stream().map(entry -> {
String key = entry.getKey();
LOGGER.info(key);
List<String> entries = entry.getValue().stream().map(part -> {
LOGGER.info("{}", part);
// read the input part
String input = ((FormFieldPart) part).value();
// return the modified input part
return new String(modifyRequest(config, exchange, key, input));
}).collect(Collectors.toList());
return new Map.Entry<String, List<String>>() {
#Override
public String getKey() {
return key;
}
#Override
public List<String> getValue() {
return entries;
}
#Override
public List<String> setValue(List<String> param1) {
return param1;
}
};
}).collect(Collectors.toMap(k -> k.getKey(), k -> k.getValue()));
return Mono.just(new LinkedMultiValueMap<String, String>(modifedBodyArray));
});
});
// insert the new modified body
BodyInserter bodyInserter = BodyInserters.fromPublisher(modifiedBody, new ParameterizedTypeReference<MultiValueMap<String, String>>() {});
HttpHeaders headers = new HttpHeaders();
headers.putAll(exchange.getRequest().getHeaders());
// the new content type will be computed by bodyInserter
// and then set in the request decorator
headers.remove(HttpHeaders.CONTENT_LENGTH);
CachedBodyOutputMessage outputMessage = new CachedBodyOutputMessage(exchange, headers);
return bodyInserter.insert(outputMessage, new BodyInserterContext())
.then(Mono.defer(() -> {
ServerHttpRequest decorator = decorate(exchange, headers, outputMessage);
return chain.filter(exchange.mutate().request(decorator).build());
}));
}, RouteToRequestUrlFilter.ROUTE_TO_URL_FILTER_ORDER + 1);
}
// some of the helper methods
private String modifyRequest(Config config, ServerWebExchange exchange, String key, String input) {
// do your thing in here !!!
return input;
}
private ServerHttpRequestDecorator decorate(ServerWebExchange exchange, HttpHeaders headers, CachedBodyOutputMessage outputMessage) {
return new ServerHttpRequestDecorator(exchange.getRequest()) {
#Override
public HttpHeaders getHeaders() {
long contentLength = headers.getContentLength();
HttpHeaders httpHeaders = new HttpHeaders();
httpHeaders.putAll(headers);
if (contentLength > 0) {
httpHeaders.setContentLength(contentLength);
} else {
// TODO: this causes a 'HTTP/1.1 411 Length Required' // on httpbin.org
httpHeaders.set(HttpHeaders.TRANSFER_ENCODING, "chunked");
}
return httpHeaders;
}
#Override
public Flux<DataBuffer> getBody() {
return outputMessage.getBody();
}
};
}
I have a method which queries a remote service. This service returns a single payload which holds many items.
How do I get those items out using a Flux and a flatMapMany?
At the moment my "fetch from service" method looks like:
public Flux<Stack> listAll() {
return this.webClient
.get()
.uri("/projects")
.accept(MediaType.APPLICATION_JSON)
.exchange()
.flatMapMany(response -> response.bodyToFlux(Stack.class));
}
a Stack is just a POJO which looks like:
public class Stack {
String id;
String name;
String title;
String created;
}
Nothing special here, but I think my deserializer is wrong:
protected Stack deserializeObject(JsonParser jsonParser, DeserializationContext deserializationContext, ObjectCodec objectCodec, JsonNode jsonNode) throws IOException {
log.info("JsonNode {}", jsonNode);
return Stack.builder()
.id(nullSafeValue(jsonNode.findValue("id"), String.class))
.name(nullSafeValue(jsonNode.findValue("name"), String.class))
.title(nullSafeValue(jsonNode.findValue("title"), String.class))
.created(nullSafeValue(jsonNode.findValue("created"), String.class))
.build();
}
What I've noticed happening is the first object is serialized correctly, but then it seems to get serialized again, rather than the next object in the payload.
The payload coming in follows standard JSON API spec and looks like:
{
"data":[
{
"type":"stacks",
"id":"1",
"attributes":{
"name":"name_1",
"title":"title_1",
"created":"2017-03-31 12:27:59",
"created_unix":1490916479
}
},
{
"type":"stacks",
"id":"2",
"attributes":{
"name":"name_2",
"title":"title_2",
"created":"2017-03-31 12:28:00",
"created_unix":1490916480
}
},
{
"type":"stacks",
"id":"3",
"attributes":{
"name":"name_3",
"title":"title_3",
"created":"2017-03-31 12:28:01",
"created_unix":1490916481
}
}
]
}
I've based this pattern on the spring-reactive-university
Any help as to where I've gone wrong would be awesome;
Cheers!
I think I solved it, still using a Flux.
public Flux<Stack> listAllStacks() {
return this.webClient
.get()
.uri("/naut/projects")
.accept(MediaType.APPLICATION_JSON)
.exchange()
.flatMap(response -> response.toEntity(String.class))
.flatMapMany(this::transformPayloadToStack);
}
Converts the incoming payload to a String where I can then parse it using a jsonapi library
private Flux<Stack> transformPayloadToStack(ResponseEntity<String> payload) {
ObjectMapper objectMapper = new ObjectMapper();
ResourceConverter resourceConverter = new ResourceConverter(objectMapper, Stack.class);
List<Stack> stackList = resourceConverter.readDocumentCollection(payload.getBody().getBytes(), Stack.class).get();
return Flux.fromIterable(stackList);
}
Which returns a Flux. Thanks to the library, I don't need to create a bunch of domains either, I can still work with my simple Stack POJO
#Data
#NoArgsConstructor
#AllArgsConstructor
#JsonIgnoreProperties(ignoreUnknown = true)
#Type("stacks")
public class Stack {
#com.github.jasminb.jsonapi.annotations.Id
String id;
String name;
String title;
String created;
}
And this in turn is called from the controller
#GetMapping("/stacks")
#ResponseBody
public Flux<Stack> findAll() {
return this.stackService.listAllStacks();
}
I've not tested if this is blocking or not yet, but seems to work okay.
You json doesn't exactly match with your model class i.e. Stack. Together with Stack create another class like this
public class Data {
List<Stack> data;
// Getters and Setters....
}
Now in your webclient you can do like this
Mono<Data> listMono = webClient
.get()
.uri("/product/projects")
.exchange()
.flatMap(clientResponse -> clientResponse.bodyToMono(Data.class));
Now if you do listMono.block() you will get Data object which will have all Stack objects.
I want to have centralised logging for requests and responses in my REST API on Spring WebFlux with Kotlin. So far I've tried this approaches
#Bean
fun apiRouter() = router {
(accept(MediaType.APPLICATION_JSON) and "/api").nest {
"/user".nest {
GET("/", userHandler::listUsers)
POST("/{userId}", userHandler::updateUser)
}
}
}.filter { request, next ->
logger.info { "Processing request $request with body ${request.bodyToMono<String>()}" }
next.handle(request).doOnSuccess { logger.info { "Handling with response $it" } }
}
Here request method and path log successfully but the body is Mono, so how should I log it? Should it be the other way around and I have to subscribe on request body Mono and log it in the callback?
Another problem is that ServerResponse interface here doesn't have access to the response body. How can I get it here?
Another approach I've tried is using WebFilter
#Bean
fun loggingFilter(): WebFilter =
WebFilter { exchange, chain ->
val request = exchange.request
logger.info { "Processing request method=${request.method} path=${request.path.pathWithinApplication()} params=[${request.queryParams}] body=[${request.body}]" }
val result = chain.filter(exchange)
logger.info { "Handling with response ${exchange.response}" }
return#WebFilter result
}
Same problem here: request body is Flux and no response body.
Is there a way to access full request and response for logging from some filters? What don't I understand?
This is more or less similar to the situation in Spring MVC.
In Spring MVC, you can use a AbstractRequestLoggingFilter filter and ContentCachingRequestWrapper and/or ContentCachingResponseWrapper. Many tradeoffs here:
if you'd like to access servlet request attributes, you need to actually read and parse the request body
logging the request body means buffering the request body, which can use a significant amount of memory
if you'd like to access the response body, you need to wrap the response and buffer the response body as it's being written, for later retrieval
ContentCaching*Wrapper classes don't exist in WebFlux but you could create similar ones. But keep in mind other points here:
buffering data in memory somehow goes against the reactive stack, since we're trying there to be very efficient with the available resources
you should not tamper with the actual flow of data and flush more/less often than expected, otherwise you'd risk breaking streaming uses cases
at that level, you only have access to DataBuffer instances, which are (roughly) memory-efficient byte arrays. Those belong to buffer pools and are recycled for other exchanges. If those aren't properly retained/released, memory leaks are created (and buffering data for later consumption certainly fits that scenario)
again at that level, it's only bytes and you don't have access to any codec to parse the HTTP body. I'd forget about buffering the content if it's not human-readable in the first place
Other answers to your question:
yes, the WebFilter is probably the best approach
no, you shouldn't subscribe to the request body otherwise you'd consume data that the handler won't be able to read; you can flatMap on the request and buffer data in doOn operators
wrapping the response should give you access to the response body as it's being written; don't forget about memory leaks, though
I didn't find a good way to log request/response bodies, but if you are just interested in meta data then you can do it like follows.
import org.springframework.http.HttpHeaders
import org.springframework.http.HttpStatus
import org.springframework.http.server.reactive.ServerHttpResponse
import org.springframework.stereotype.Component
import org.springframework.web.server.ServerWebExchange
import org.springframework.web.server.WebFilter
import org.springframework.web.server.WebFilterChain
import reactor.core.publisher.Mono
#Component
class LoggingFilter(val requestLogger: RequestLogger, val requestIdFactory: RequestIdFactory) : WebFilter {
val logger = logger()
override fun filter(exchange: ServerWebExchange, chain: WebFilterChain): Mono<Void> {
logger.info(requestLogger.getRequestMessage(exchange))
val filter = chain.filter(exchange)
exchange.response.beforeCommit {
logger.info(requestLogger.getResponseMessage(exchange))
Mono.empty()
}
return filter
}
}
#Component
class RequestLogger {
fun getRequestMessage(exchange: ServerWebExchange): String {
val request = exchange.request
val method = request.method
val path = request.uri.path
val acceptableMediaTypes = request.headers.accept
val contentType = request.headers.contentType
return ">>> $method $path ${HttpHeaders.ACCEPT}: $acceptableMediaTypes ${HttpHeaders.CONTENT_TYPE}: $contentType"
}
fun getResponseMessage(exchange: ServerWebExchange): String {
val request = exchange.request
val response = exchange.response
val method = request.method
val path = request.uri.path
val statusCode = getStatus(response)
val contentType = response.headers.contentType
return "<<< $method $path HTTP${statusCode.value()} ${statusCode.reasonPhrase} ${HttpHeaders.CONTENT_TYPE}: $contentType"
}
private fun getStatus(response: ServerHttpResponse): HttpStatus =
try {
response.statusCode
} catch (ex: Exception) {
HttpStatus.CONTINUE
}
}
This is what I came up with for java.
public class RequestResponseLoggingFilter implements WebFilter {
#Override
public Mono<Void> filter(ServerWebExchange exchange, WebFilterChain chain) {
ServerHttpRequest httpRequest = exchange.getRequest();
final String httpUrl = httpRequest.getURI().toString();
ServerHttpRequestDecorator loggingServerHttpRequestDecorator = new ServerHttpRequestDecorator(exchange.getRequest()) {
String requestBody = "";
#Override
public Flux<DataBuffer> getBody() {
return super.getBody().doOnNext(dataBuffer -> {
try (ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream()) {
Channels.newChannel(byteArrayOutputStream).write(dataBuffer.asByteBuffer().asReadOnlyBuffer());
requestBody = IOUtils.toString(byteArrayOutputStream.toByteArray(), "UTF-8");
commonLogger.info(LogMessage.builder()
.step(httpUrl)
.message("log incoming http request")
.stringPayload(requestBody)
.build());
} catch (IOException e) {
commonLogger.error(LogMessage.builder()
.step("log incoming request for " + httpUrl)
.message("fail to log incoming http request")
.errorType("IO exception")
.stringPayload(requestBody)
.build(), e);
}
});
}
};
ServerHttpResponseDecorator loggingServerHttpResponseDecorator = new ServerHttpResponseDecorator(exchange.getResponse()) {
String responseBody = "";
#Override
public Mono<Void> writeWith(Publisher<? extends DataBuffer> body) {
Mono<DataBuffer> buffer = Mono.from(body);
return super.writeWith(buffer.doOnNext(dataBuffer -> {
try (ByteArrayOutputStream byteArrayOutputStream = new ByteArrayOutputStream()) {
Channels.newChannel(byteArrayOutputStream).write(dataBuffer.asByteBuffer().asReadOnlyBuffer());
responseBody = IOUtils.toString(byteArrayOutputStream.toByteArray(), "UTF-8");
commonLogger.info(LogMessage.builder()
.step("log outgoing response for " + httpUrl)
.message("incoming http request")
.stringPayload(responseBody)
.build());
} catch (Exception e) {
commonLogger.error(LogMessage.builder()
.step("log outgoing response for " + httpUrl)
.message("fail to log http response")
.errorType("IO exception")
.stringPayload(responseBody)
.build(), e);
}
}));
}
};
return chain.filter(exchange.mutate().request(loggingServerHttpRequestDecorator).response(loggingServerHttpResponseDecorator).build());
}
}
You can actually enable DEBUG logging for Netty and Reactor-Netty related to see full picture of what's happening. You could play with the below and see what you want and don't. That was the best I could.
reactor.ipc.netty.channel.ChannelOperationsHandler: DEBUG
reactor.ipc.netty.http.server.HttpServer: DEBUG
reactor.ipc.netty.http.client: DEBUG
io.reactivex.netty.protocol.http.client: DEBUG
io.netty.handler: DEBUG
io.netty.handler.proxy.HttpProxyHandler: DEBUG
io.netty.handler.proxy.ProxyHandler: DEBUG
org.springframework.web.reactive.function.client: DEBUG
reactor.ipc.netty.channel: DEBUG
Since Spring Boot 2.2.x, Spring Webflux supports Kotlin coroutines. With coroutines, you can have the advantages of non-blocking calls without having to handle Mono and Flux wrapped objects. It adds extensions to ServerRequest and ServerResponse, adding methods like ServerRequest#awaitBody() and ServerResponse.BodyBuilder.bodyValueAndAwait(body: Any). So you could rewrite you code like this:
#Bean
fun apiRouter() = coRouter {
(accept(MediaType.APPLICATION_JSON) and "/api").nest {
"/user".nest {
/* the handler methods now use ServerRequest and ServerResponse directly
you just need to add suspend before your function declaration:
suspend fun listUsers(ServerRequest req, ServerResponse res) */
GET("/", userHandler::listUsers)
POST("/{userId}", userHandler::updateUser)
}
}
// this filter will be applied to all routes built by this coRouter
filter { request, next ->
// using non-blocking request.awayBody<T>()
logger.info("Processing $request with body ${request.awaitBody<String>()}")
val res = next(request)
logger.info("Handling with Content-Type ${res.headers().contentType} and status code ${res.rawStatusCode()}")
res
}
}
In order to create a WebFilter Bean with coRoutines, I think you can use this CoroutineWebFilter interface (I haven't tested it, I don't know if it works).
I am pretty new to Spring WebFlux, and I don't know how to do it in Kotlin, but should be the same as in Java using WebFilter:
public class PayloadLoggingWebFilter implements WebFilter {
public static final ByteArrayOutputStream EMPTY_BYTE_ARRAY_OUTPUT_STREAM = new ByteArrayOutputStream(0);
private final Logger logger;
private final boolean encodeBytes;
public PayloadLoggingWebFilter(Logger logger) {
this(logger, false);
}
public PayloadLoggingWebFilter(Logger logger, boolean encodeBytes) {
this.logger = logger;
this.encodeBytes = encodeBytes;
}
#Override
public Mono<Void> filter(ServerWebExchange exchange, WebFilterChain chain) {
if (logger.isInfoEnabled()) {
return chain.filter(decorate(exchange));
} else {
return chain.filter(exchange);
}
}
private ServerWebExchange decorate(ServerWebExchange exchange) {
final ServerHttpRequest decorated = new ServerHttpRequestDecorator(exchange.getRequest()) {
#Override
public Flux<DataBuffer> getBody() {
if (logger.isDebugEnabled()) {
final ByteArrayOutputStream baos = new ByteArrayOutputStream();
return super.getBody().map(dataBuffer -> {
try {
Channels.newChannel(baos).write(dataBuffer.asByteBuffer().asReadOnlyBuffer());
} catch (IOException e) {
logger.error("Unable to log input request due to an error", e);
}
return dataBuffer;
}).doOnComplete(() -> flushLog(baos));
} else {
return super.getBody().doOnComplete(() -> flushLog(EMPTY_BYTE_ARRAY_OUTPUT_STREAM));
}
}
};
return new ServerWebExchangeDecorator(exchange) {
#Override
public ServerHttpRequest getRequest() {
return decorated;
}
private void flushLog(ByteArrayOutputStream baos) {
ServerHttpRequest request = super.getRequest();
if (logger.isInfoEnabled()) {
StringBuffer data = new StringBuffer();
data.append('[').append(request.getMethodValue())
.append("] '").append(String.valueOf(request.getURI()))
.append("' from ")
.append(
Optional.ofNullable(request.getRemoteAddress())
.map(addr -> addr.getHostString())
.orElse("null")
);
if (logger.isDebugEnabled()) {
data.append(" with payload [\n");
if (encodeBytes) {
data.append(new HexBinaryAdapter().marshal(baos.toByteArray()));
} else {
data.append(baos.toString());
}
data.append("\n]");
logger.debug(data.toString());
} else {
logger.info(data.toString());
}
}
}
};
}
}
Here some tests on this: github
I think this is what Brian Clozel (#brian-clozel) meant.
Here is the GitHub Repo with complete implementation to log both request and response body along with http headers for webflux/java based application...
What Brian said. In addition, logging request/response bodies don't make sense for reactive streaming. If you imagine the data flowing through a pipe as a stream, you don't have the full content at any time unless you buffer it, which defeats the whole point. For small request/response, you can get away with buffering, but then why use the reactive model (other than to impress your coworkers :-) )?
The only reason for logging request/response that I could conjure up is debugging, but with the reactive programming model, debugging method has to be modified too. Project Reactor doc has an excellent section on debugging that you can refer to: http://projectreactor.io/docs/core/snapshot/reference/#debugging
Assuming we are dealing with a simple JSON or XML response, if debug level for corresponding loggers is not sufficient for some reason, one can use string representation before transforming it to object:
Mono<Response> mono = WebClient.create()
.post()
.body(Mono.just(request), Request.class)
.retrieve()
.bodyToMono(String.class)
.doOnNext(this::sideEffectWithResponseAsString)
.map(this::transformToResponse);
the following are the side-effect and transformation methods:
private void sideEffectWithResponseAsString(String response) { ... }
private Response transformToResponse(String response) { /*use Jackson or JAXB*/ }
If your using controller instead of handler best way is aop with annotating you controller class with #Log annotation.And FYI this takes plain json object as request not mono.
#Target(AnnotationTarget.FUNCTION)
#Retention(AnnotationRetention.RUNTIME)
annotation class Log
#Aspect
#Component
class LogAspect {
companion object {
val log = KLogging().logger
}
#Around("#annotation(Log)")
#Throws(Throwable::class)
fun logAround(joinPoint: ProceedingJoinPoint): Any? {
val start = System.currentTimeMillis()
val result = joinPoint.proceed()
return if (result is Mono<*>) result.doOnSuccess(getConsumer(joinPoint, start)) else result
}
fun getConsumer(joinPoint: ProceedingJoinPoint, start: Long): Consumer<Any>? {
return Consumer {
var response = ""
if (Objects.nonNull(it)) response = it.toString()
log.info(
"Enter: {}.{}() with argument[s] = {}",
joinPoint.signature.declaringTypeName, joinPoint.signature.name,
joinPoint.args
)
log.info(
"Exit: {}.{}() had arguments = {}, with result = {}, Execution time = {} ms",
joinPoint.signature.declaringTypeName, joinPoint.signature.name,
joinPoint.args[0],
response, System.currentTimeMillis() - start
)
}
}
}
I think the appropriate thing to do here is to write the contents of each request to a file in an asynchronous manner (java.nio) and set up an interval that reads those request body files asynchrolusly and writes them to the log in a memory usage aware manner (atleast one file at a time but up too 100 mb at a time) and after logging them removes the files from disk.
Ivan Lymar's answer but in Kotlin:
import org.apache.commons.io.IOUtils
import org.reactivestreams.Publisher
import org.springframework.core.io.buffer.DataBuffer
import org.springframework.http.server.reactive.ServerHttpRequestDecorator
import org.springframework.http.server.reactive.ServerHttpResponseDecorator
import org.springframework.stereotype.Component
import org.springframework.web.server.ServerWebExchange
import org.springframework.web.server.WebFilter
import org.springframework.web.server.WebFilterChain
import reactor.core.publisher.Flux
import reactor.core.publisher.Mono
import java.io.ByteArrayOutputStream
import java.io.IOException
import java.nio.channels.Channels
#Component
class LoggingWebFilter : WebFilter {
override fun filter(exchange: ServerWebExchange, chain: WebFilterChain): Mono<Void> {
val httpRequest = exchange.request
val httpUrl = httpRequest.uri.toString()
val loggingServerHttpRequestDecorator: ServerHttpRequestDecorator =
object : ServerHttpRequestDecorator(exchange.request) {
var requestBody = ""
override fun getBody(): Flux<DataBuffer> {
return super.getBody().doOnNext { dataBuffer: DataBuffer ->
try {
ByteArrayOutputStream().use { byteArrayOutputStream ->
Channels.newChannel(byteArrayOutputStream)
.write(dataBuffer.asByteBuffer().asReadOnlyBuffer())
requestBody =
IOUtils.toString(
byteArrayOutputStream.toByteArray(),
"UTF-8"
)
log.info(
"Logging Request Filter: {} {}",
httpUrl,
requestBody
)
}
} catch (e: IOException) {
log.error(
"Logging Request Filter Error: {} {}",
httpUrl,
requestBody,
e
)
}
}
}
}
val loggingServerHttpResponseDecorator: ServerHttpResponseDecorator =
object : ServerHttpResponseDecorator(exchange.response) {
var responseBody = ""
override fun writeWith(body: Publisher<out DataBuffer>): Mono<Void> {
val buffer: Mono<DataBuffer> = Mono.from(body)
return super.writeWith(
buffer.doOnNext { dataBuffer: DataBuffer ->
try {
ByteArrayOutputStream().use { byteArrayOutputStream ->
Channels.newChannel(byteArrayOutputStream)
.write(
dataBuffer
.asByteBuffer()
.asReadOnlyBuffer()
)
responseBody = IOUtils.toString(
byteArrayOutputStream.toByteArray(),
"UTF-8"
)
log.info(
"Logging Response Filter: {} {}",
httpUrl,
responseBody
)
}
} catch (e: Exception) {
log.error(
"Logging Response Filter Error: {} {}",
httpUrl,
responseBody,
e
)
}
}
)
}
}
return chain.filter(
exchange.mutate().request(loggingServerHttpRequestDecorator)
.response(loggingServerHttpResponseDecorator)
.build()
)
}
}