I am currently using Spring WebFlux to try build an async end-point, which fetches a PDF from a third-party end-point via Web Client before returning the PDF back to our API consumer. However, I am struggling with returning a Mono<ResponseEntity> with content type application/pdf due to the below exception:
Resolved [org.springframework.http.converter.HttpMessageNotWritableException: No converter for [class reactor.core.publisher.MonoMapFuseable] with preset Content-Type 'application/pdf']
Here is controller implementation. My question is:
Is my implementation in the right direction, or would I need to create some sort of converter?
Does Mono<ResponseEntity> even support returning a PDF as a response body?
#RequestMapping(value="/get-pdf", method = RequestMethod.GET)
public Mono<ResponseEntity> getPDFAsync() {
String url = "http://some-end-point";
WebClient client = WebClient.create(url);
return client.get()
.accept(MediaType.APPLICATION_PDF)
.exchangeToMono(response ->
Mono.just(ResponseEntity.ok().contentType(MediaType.APPLICATION_PDF)
.body(response.bodyToMono(ByteArrayResource.class)
.map(byteArrayResource -> byteArrayResource.getByteArray())
)));
}
To download a file reactively, you could supply the file as a Flux<DataBuffer>, where DataBuffer is org.springframework.core.io.buffer.DataBuffer, like this:
// some shared buffer factory.
private final DataBufferFactory dataBufferFactory = new NettyDataBufferFactory(ByteBufAllocator.DEFAULT);
#RequestMapping(value = "/download",
method = RequestMethod.GET,
produces = {MediaType.APPLICATION_PDF_VALUE}
)
public Mono<ResponseEntity<Flux<DataBuffer>>> downloadDocument(
...
) {
return Mono.fromCallable(() -> {
return ResponseEntity.ok(
DataBufferUtils.read(
new File("somepdf.pdf").toPath(),
dataBufferFactory,
8096
))
});
}
Or more specifically, since you seem to be using the WebFlux WebClient, you can forward the response body flux directly to your own response, without having to buffer the complete response first:
#RequestMapping(value = "/download",
method = RequestMethod.GET,
produces = {MediaType.APPLICATION_PDF_VALUE}
)
public Mono<ResponseEntity<Flux<DataBuffer>>> downloadDocument(
...
) {
String url = "http://some-end-point";
WebClient client = WebClient.create(url);
return client.get()
.accept(MediaType.APPLICATION_PDF)
.exchange()
.map(response -> response.bodyToFlux(DataBuffer.class))
.map(ResponseEntity::ok);
}
Hint: I hope you are reusing the WebClient instance and not instantiating a new one on each request.
I have found the answer! In short, returning Mono<byte[]>, and add produces = {MediaType.APPLICATION_PDF_VALUE} to #RequestMapping works. See example below.
#RequestMapping(value="/get-pdf", produces = {MediaType.APPLICATION_PDF_VALUE}, method = RequestMethod.GET)
public Mono<byte[]> getPdf() {
String url = "some-end-point";
WebClient client = WebClient.create(url);
return client.get()
.accept(MediaType.APPLICATION_PDF)
.exchangeToMono(response -> response
.bodyToMono(ByteArrayResource.class))
.map(byteArrayResource -> byteArrayResource.getByteArray());
}
i'm trying to send a POST request with body data as described here: https://scrapyrt.readthedocs.io/en/stable/api.html#post.
Here's what i've tried to do but it gives me HTTP code 500
String uri = "http://localhost:3000";
WebClient webClient = WebClient.builder()
.baseUrl(uri)
.build();
LinkedMultiValueMap map = new LinkedMultiValueMap();
String q = "\"url\": \"https://blog.trendmicro.com/trendlabs-security-intelligence\",\"meta\":{\"latestDate\" : \"18-05-2020\"}}";
map.add("request", q);
map.add("spider_name", "blog");
BodyInserter<MultiValueMap<String, Object>, ClientHttpRequest> inserter2
= BodyInserters.fromMultipartData(map);
Mono<ItemsList> result = webClient.post()
.uri(uriBuilder -> uriBuilder
.path("/crawl.json")
.build())
.body(inserter2)
.retrieve()
.bodyToMono(ItemsList.class);
ItemsList tempItems = result.block();
Here's what i've tried to do but it gives me HTTP code 500
Most likely because you're sending the wrong data in a mixture of wrong formats with the wrong type:
You're using multipart form data, not JSON
You're then setting the request parameter as a JSON string (q)
The JSON string you're using in q isn't even valid (it's at least missing an opening curly brace) - and handwriting JSON is almost universally a bad idea, leverage a framework to do it for you instead.
Instead, the normal thing to do would be to create a POJO structure that maps to your request, so:
public class CrawlRequest {
private CrawlInnerRequest request;
#JsonProperty("spider_name")
private String spiderName;
//....add the getters / setters
}
public class CrawlInnerRequest {
private String url;
private String callback;
#JsonProperty("dont_filter")
private String dontFilter;
//....add the getters / setters
}
...then simply create a CrawlRequest, set the values as you wish, then in your post call use:
.body(BodyInserters.fromValue(crawlRequest))
This is a rather fundamental, basic part of using a WebClient. I'd suggest reading around more widely to give yourself a better understanding of the fundamentals, it will help tremendously in the long run.
For me following code worked:
public String wcPost(){
Map<String, String> bodyMap = new HashMap();
bodyMap.put("key1","value1");
WebClient client = WebClient.builder()
.baseUrl("domainURL")
.build();
String responseSpec = client.post()
.uri("URI")
.headers(h -> h.setBearerAuth("token if any"))
.body(BodyInserters.fromValue(bodyMap))
.exchange()
.flatMap(clientResponse -> {
if (clientResponse.statusCode().is5xxServerError()) {
clientResponse.body((clientHttpResponse, context) -> {
return clientHttpResponse.getBody();
});
return clientResponse.bodyToMono(String.class);
}
else
return clientResponse.bodyToMono(String.class);
})
.block();
return responseSpec;
}
I am building an application to call a back-end which responds with a mime-type response.
#Override
public Mono<String> getDocument() {
return webClient.get()
.uri(path)
.retrieve()
.bodyToMono(String.class);
}
From this request, I need to preserve the response headers and pass it through as the response. This is mostly because the response headers contain the dynamic content type of the file. I need to forward these headers (all as received) to the API response. For example :
Content-Type : application/pdf
Content-Disposition: attachment; filename="test.pdf"
Following is my handler.
public Mono<ServerResponse> getDocument(ServerRequest request) {
return ServerResponse
.ok()
.contentType(MediaType.APPLICATION_PDF)
.header("Content-Disposition", "attachment; filename=\"test.pdf\"")
.body(BodyInserters.fromPublisher(documentService.getDocument(), String.class));
}
The file is coming through from the API as an attachment as expected, but I do not want to hard code the content-type header. How can I achieve this?
Update with the handler code :
public Mono<ServerResponse> getDocument(ServerRequest request) {
return ServerResponse
.ok()
.contentType(MediaType.APPLICATION_JSON)
.body(BodyInserters.fromPublisher(documentService.getDocument(), String.class));
}
I was able to resolve the problem by returning a ResponseEntity from the service instead of the body and using that to construct the ServerResponse in the handler.
Service :
public Mono<ResponseEntity<String>> getDocument() {
return webClient.get()
.uri(path)
.retrieve()
.toEntity(String.class);
}
Handler :
public Mono<ServerResponse> getDocument(ServerRequest request) {
return documentService
.getDocument()
.flatMap(r -> ServerResponse
.ok()
.headers(httpHeaders -> httpHeaders.addAll(r.getHeaders()))
.body(BodyInserters.fromValue(r.getBody()))
);
}
I am trying to understand WebFlux but having some trouble with Webclient calls. I do not see
this line System.out.println("customerId = " + customerId); executes it seems like it does not call the endpoint.
But if I subscribe to webclient with .subscribe(customer -> {}); then I can see this line System.out.println("customerId = " + customerId); works
on the endpoint side. I dont understand why I have to subscribe to Mono call, or do I have to ? Thanks
#GetMapping("/customer/{customerId}")
#ResponseStatus(HttpStatus.ACCEPTED)
public Mono<Void> getCustomer(#PathVariable("customerId") int customerId) {
WebClient webClient = WebClient.builder().baseUrl("http://localhost:8080").build();
webClient.get()
.uri("/client/customer/{customerId}",customerId)
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.bodyToMono(Customer.class);//here do I have to subscribe to actually activate to call?
return null;
}
#GET
#Path("/customer/{customerId}")
#Produces(MediaType.APPLICATION_JSON)
public Customer getCustomer(#PathParam("customerId") int customerId) throws InterruptedException {
System.out.println("customerId = " + customerId); // I do not see the call comes thru if I dont subscribe to flux call.
return new Customer(customerId,"custName");
}
If you want to return the reactive type from your WebClient, you have to return it from your controller method like:
#GetMapping("/customer/{customerId}")
#ResponseStatus(HttpStatus.ACCEPTED)
public Mono<Customer> getCustomer(#PathVariable("customerId") int customerId) {
WebClient webClient = WebClient.builder().baseUrl("http://localhost:8080").build();
return webClient.get()
.uri("/client/customer/{customerId}",customerId)
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.bodyToMono(Customer.class);
}
You can also return a Customer from your endpoint and block and wait for the result of your WebClient and leaving the reactive ecosystem like:
#GetMapping("/customer/{customerId}")
#ResponseStatus(HttpStatus.ACCEPTED)
public Customer getCustomer(#PathVariable("customerId") int customerId) {
WebClient webClient = WebClient.builder().baseUrl("http://localhost:8080").build();
return webClient.get()
.uri("/client/customer/{customerId}",customerId)
.accept(MediaType.APPLICATION_JSON)
.retrieve()
.bodyToMono(Customer.class)
.block()
}
If you are looking at a general introduction for Spring's WebClient, take a look at this tutorial
We wrote a small Spring Boot REST application, which performs a REST request on another REST endpoint.
#RequestMapping("/api/v1")
#SpringBootApplication
#RestController
#Slf4j
public class Application
{
#Autowired
private WebClient webClient;
#RequestMapping(value = "/zyx", method = POST)
#ResponseBody
XyzApiResponse zyx(#RequestBody XyzApiRequest request, #RequestHeader HttpHeaders headers)
{
webClient.post()
.uri("/api/v1/someapi")
.accept(MediaType.APPLICATION_JSON)
.contentType(MediaType.APPLICATION_JSON)
.body(BodyInserters.fromObject(request.getData()))
.exchange()
.subscribeOn(Schedulers.elastic())
.flatMap(response ->
response.bodyToMono(XyzServiceResponse.class).map(r ->
{
if (r != null)
{
r.setStatus(response.statusCode().value());
}
if (!response.statusCode().is2xxSuccessful())
{
throw new ProcessResponseException(
"Bad status response code " + response.statusCode() + "!");
}
return r;
}))
.subscribe(body ->
{
// Do various things
}, throwable ->
{
// This section handles request errors
});
return XyzApiResponse.OK;
}
}
We are new to Spring and are having trouble writing a Unit Test for this small code snippet.
Is there an elegant (reactive) way to mock the webClient itself or to start a mock server that the webClient can use as an endpoint?
We accomplished this by providing a custom ExchangeFunction that simply returns the response we want to the WebClientBuilder:
webClient = WebClient.builder()
.exchangeFunction(clientRequest ->
Mono.just(ClientResponse.create(HttpStatus.OK)
.header("content-type", "application/json")
.body("{ \"key\" : \"value\"}")
.build())
).build();
myHttpService = new MyHttpService(webClient);
Map<String, String> result = myHttpService.callService().block();
// Do assertions here
If we want to use Mokcito to verify if the call was made or reuse the WebClient accross multiple unit tests in the class, we could also mock the exchange function:
#Mock
private ExchangeFunction exchangeFunction;
#BeforeEach
void init() {
WebClient webClient = WebClient.builder()
.exchangeFunction(exchangeFunction)
.build();
myHttpService = new MyHttpService(webClient);
}
#Test
void callService() {
when(exchangeFunction.exchange(any(ClientRequest.class)))
.thenReturn(buildMockResponse());
Map<String, String> result = myHttpService.callService().block();
verify(exchangeFunction).exchange(any());
// Do assertions here
}
Note: If you get null pointer exceptions related to publishers on the when call, your IDE might have imported Mono.when instead of Mockito.when.
Sources:
WebClient
javadoc
WebClient.Builder
javadoc
ExchangeFunction
javadoc
With the following method it was possible to mock the WebClient with Mockito for calls like this:
webClient
.get()
.uri(url)
.header(headerName, headerValue)
.retrieve()
.bodyToMono(String.class);
or
webClient
.get()
.uri(url)
.headers(hs -> hs.addAll(headers));
.retrieve()
.bodyToMono(String.class);
Mock method:
private static WebClient getWebClientMock(final String resp) {
final var mock = Mockito.mock(WebClient.class);
final var uriSpecMock = Mockito.mock(WebClient.RequestHeadersUriSpec.class);
final var headersSpecMock = Mockito.mock(WebClient.RequestHeadersSpec.class);
final var responseSpecMock = Mockito.mock(WebClient.ResponseSpec.class);
when(mock.get()).thenReturn(uriSpecMock);
when(uriSpecMock.uri(ArgumentMatchers.<String>notNull())).thenReturn(headersSpecMock);
when(headersSpecMock.header(notNull(), notNull())).thenReturn(headersSpecMock);
when(headersSpecMock.headers(notNull())).thenReturn(headersSpecMock);
when(headersSpecMock.retrieve()).thenReturn(responseSpecMock);
when(responseSpecMock.bodyToMono(ArgumentMatchers.<Class<String>>notNull()))
.thenReturn(Mono.just(resp));
return mock;
}
You can use MockWebServer by the OkHttp team. Basically, the Spring team uses it for their tests too (at least how they said here). Here is an example with reference to a source:
According to Tim's blog post let's consider that we have the following service:
class ApiCaller {
private WebClient webClient;
ApiCaller(WebClient webClient) {
this.webClient = webClient;
}
Mono<SimpleResponseDto> callApi() {
return webClient.put()
.uri("/api/resource")
.contentType(MediaType.APPLICATION_JSON)
.header("Authorization", "customAuth")
.syncBody(new SimpleRequestDto())
.retrieve()
.bodyToMono(SimpleResponseDto.class);
}
}
then the test could be designed in the following way (comparing to origin I changed the way how async chains should be tested in Reactor using StepVerifier):
class ApiCallerTest {
private final MockWebServer mockWebServer = new MockWebServer();
private final ApiCaller apiCaller = new ApiCaller(WebClient.create(mockWebServer.url("/").toString()));
#AfterEach
void tearDown() throws IOException {
mockWebServer.shutdown();
}
#Test
void call() throws InterruptedException {
mockWebServer.enqueue(new MockResponse().setResponseCode(200)
.setHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
.setBody("{\"y\": \"value for y\", \"z\": 789}")
);
//Asserting response
StepVerifier.create(apiCaller.callApi())
.assertNext(res -> {
assertNotNull(res);
assertEquals("value for y", res.getY());
assertEquals("789", res.getZ());
})
.verifyComplete();
//Asserting request
RecordedRequest recordedRequest = mockWebServer.takeRequest();
//use method provided by MockWebServer to assert the request header
recordedRequest.getHeader("Authorization").equals("customAuth");
DocumentContext context = >JsonPath.parse(recordedRequest.getBody().inputStream());
//use JsonPath library to assert the request body
assertThat(context, isJson(allOf(
withJsonPath("$.a", is("value1")),
withJsonPath("$.b", is(123))
)));
}
}
I use WireMock for integration testing. I think it is much better and supports more functions than OkHttp MockeWebServer. Here is simple example:
public class WireMockTest {
WireMockServer wireMockServer;
WebClient webClient;
#BeforeEach
void setUp() throws Exception {
wireMockServer = new WireMockServer(WireMockConfiguration.wireMockConfig().dynamicPort());
wireMockServer.start();
webClient = WebClient.builder().baseUrl(wireMockServer.baseUrl()).build();
}
#Test
void testWireMock() {
wireMockServer.stubFor(get("/test")
.willReturn(ok("hello")));
String body = webClient.get()
.uri("/test")
.retrieve()
.bodyToMono(String.class)
.block();
assertEquals("hello", body);
}
#AfterEach
void tearDown() throws Exception {
wireMockServer.stop();
}
}
If you really want to mock it I recommend JMockit. There isn't necessary call when many times and you can use the same call like it is in your tested code.
#Test
void testJMockit(#Injectable WebClient webClient) {
new Expectations() {{
webClient.get()
.uri("/test")
.retrieve()
.bodyToMono(String.class);
result = Mono.just("hello");
}};
String body = webClient.get()
.uri(anyString)
.retrieve()
.bodyToMono(String.class)
.block();
assertEquals("hello", body);
}
Wire mocks is suitable for integration tests, while I believe it's not needed for unit tests. While doing unit tests, I will just be interested to know if my WebClient was called with the desired parameters. For that you need a mock of the WebClient instance. Or you could inject a WebClientBuilder instead.
Let's consider the simplified method which does a post request like below.
#Service
#Getter
#Setter
public class RestAdapter {
public static final String BASE_URI = "http://some/uri";
public static final String SUB_URI = "some/endpoint";
#Autowired
private WebClient.Builder webClientBuilder;
private WebClient webClient;
#PostConstruct
protected void initialize() {
webClient = webClientBuilder.baseUrl(BASE_URI).build();
}
public Mono<String> createSomething(String jsonDetails) {
return webClient.post()
.uri(SUB_URI)
.accept(MediaType.APPLICATION_JSON)
.body(Mono.just(jsonDetails), String.class)
.retrieve()
.bodyToMono(String.class);
}
}
The method createSomething just accepts a String, assumed as Json for simplicity of the example, does a post request on a URI and returns the output response body which is assumed as a String.
The method can be unit tested as below, with StepVerifier.
public class RestAdapterTest {
private static final String JSON_INPUT = "{\"name\": \"Test name\"}";
private static final String TEST_ID = "Test Id";
private WebClient.Builder webClientBuilder = mock(WebClient.Builder.class);
private WebClient webClient = mock(WebClient.class);
private RestAdapter adapter = new RestAdapter();
private WebClient.RequestBodyUriSpec requestBodyUriSpec = mock(WebClient.RequestBodyUriSpec.class);
private WebClient.RequestBodySpec requestBodySpec = mock(WebClient.RequestBodySpec.class);
private WebClient.RequestHeadersSpec requestHeadersSpec = mock(WebClient.RequestHeadersSpec.class);
private WebClient.ResponseSpec responseSpec = mock(WebClient.ResponseSpec.class);
#BeforeEach
void setup() {
adapter.setWebClientBuilder(webClientBuilder);
when(webClientBuilder.baseUrl(anyString())).thenReturn(webClientBuilder);
when(webClientBuilder.build()).thenReturn(webClient);
adapter.initialize();
}
#Test
#SuppressWarnings("unchecked")
void createSomething_withSuccessfulDownstreamResponse_shouldReturnCreatedObjectId() {
when(webClient.post()).thenReturn(requestBodyUriSpec);
when(requestBodyUriSpec.uri(RestAdapter.SUB_URI))
.thenReturn(requestBodySpec);
when(requestBodySpec.accept(MediaType.APPLICATION_JSON)).thenReturn(requestBodySpec);
when(requestBodySpec.body(any(Mono.class), eq(String.class)))
.thenReturn(requestHeadersSpec);
when(requestHeadersSpec.retrieve()).thenReturn(responseSpec);
when(responseSpec.bodyToMono(String.class)).thenReturn(Mono.just(TEST_ID));
ArgumentCaptor<Mono<String>> captor
= ArgumentCaptor.forClass(Mono.class);
Mono<String> result = adapter.createSomething(JSON_INPUT);
verify(requestBodySpec).body(captor.capture(), eq(String.class));
Mono<String> testBody = captor.getValue();
assertThat(testBody.block(), equalTo(JSON_INPUT));
StepVerifier
.create(result)
.expectNext(TEST_ID)
.verifyComplete();
}
}
Note that the 'when' statements test all the parameters except the request Body. Even if one of the parameters mismatches, the unit test fails, thereby asserting all these. Then, the request body is asserted in a separate verify and assert as the 'Mono' cannot be equated. The result is then verified using step verifier.
And then, we can do an integration test with wire mock, as mentioned in the other answers, to see if this class wires properly, and calls the endpoint with the desired body, etc.
I have tried all the solutions in the already given answers here.
The answer to your question is:
It depends if you want to do Unit testing or Integration testing.
For unit testing purpose, mocking the WebClient itself is too verbose and require too much code. Mocking ExchangeFunction is simpler and easier.
For this, the accepted answer must be #Renette 's solution.
For integration testing the best is to use OkHttp MockWebServer.
Its simple to use an flexible. Using a server allows you to handle some error cases you otherwise need to handle manually in a Unit testing case.
With spring-cloud-starter-contract-stub-runner you can use Wiremock to mock the API responses. Here you can find a working example I described on medium. The AutoConfigureMockMvc annotation starts a Wiremock server before your test, exposing everything you have in the classpath:/mappings location (probably src/test/resources/mappings on disk).
#SpringBootTest
#AutoConfigureMockMvc
#AutoConfigureWireMock(port = 0)
class BalanceServiceTest {
private static final Logger log = LoggerFactory.getLogger(BalanceServiceTest.class);
#Autowired
private BalanceService service;
#Test
public void test() throws Exception {
assertNotNull(service.getBalance("123")
.get());
}
}
Here is an example for what a mapping file looks like. The balance.json file contains any json content you need. You can also mimic response delays or failures in static configuration files or programatically. More info on their website.
{
"request": {
"method": "GET",
"url": "/v2/accounts/123/balance"
},
"response": {
"status": 200,
"delayDistribution": {
"type": "lognormal",
"median": 1000,
"sigma": 0.4
},
"headers": {
"Content-Type": "application/json",
"Cache-Control": "no-cache"
},
"bodyFileName": "balance.json"
}
}
I wanted to use webclient for unit testing, but mockito was too complex to setup, so i created a library which can be used to build mock webclient in unit tests. This also verifies the url, method, headers and request body before dispatching the response.
FakeWebClientBuilder fakeWebClientBuilder = FakeWebClientBuilder.useDefaultWebClientBuilder();
FakeRequestResponse fakeRequestResponse = new FakeRequestResponseBuilder()
.withRequestUrl("https://google.com/foo")
.withRequestMethod(HttpMethod.POST)
.withRequestBody(BodyInserters.fromFormData("foo", "bar"))
.replyWithResponse("test")
.replyWithResponseStatusCode(200)
.build();
WebClient client =
FakeWebClientBuilder.useDefaultWebClientBuilder()
.baseUrl("https://google.com")
.addRequestResponse(fakeRequestResponse)
.build();
// Our webclient will return `test` when called.
// This assertion would check if all our enqueued responses are dequeued by the class or method we intend to test.
Assertions.assertTrue(fakeWebClientBuilder.assertAllResponsesDispatched());
I highly recommend using Okhttp MockWebServer over mocking. The reason being MockWebServer is a much much cleaner approach.
Below is the code template you can use for unit testing WebClient.
class Test {
private ClassUnderTest classUnderTest;
public static MockWebServer mockWebServer;
#BeforeAll
static void setUp() throws IOException {
mockWebServer = new MockWebServer();
mockWebServer.start();
}
#BeforeEach
void initialize() {
var httpUrl = mockWebServer.url("/xyz");
var webClient = WebClient.create(httpUrl.toString());
classUnderTest = new ClassUnderTest(webClient);
}
#Test
void testMehod() {
var mockResp = new MockResponse();
mockResp.setResponseCode(200);
mockResp.addHeader("Content-Type", "application/json");
mockResp.setBody(
"{\"prop\":\"some value\"}");
mockWebServer.enqueue(mockResp);
// This enqueued response will be returned when webclient is invoked
...
...
classUnderTest.methodThatInvkesWebClient();
...
...
}
#AfterAll
static void tearDown() throws IOException {
mockWebServer.shutdown();
}
}
Pay special attention to the initialize method. That's the only thing tricky here.
Path /xyz is not the base url, rather your resource path.
You don't need to tell the base url to MockWebServer.
Reason being, MockWebServer will spin up a server on the local host with some random port. And if you provide your own base url, your unit test will fail.
mockWebServer.url("/xyz")
This will give you base url i.e. the host and port on which MockWebServer is listening plus the resource path, say localhost:8999/xyz. You will need to create WebClient with this url.
WebClient.create(httpUrl.toString())
This will create the WebClient that make calls to the MockWebServer for your unit tests.