Citrus httpServer & httpClient tests - spring-boot

I'm using Citrus for perform integration tests based on http requests.
The test should perform steps as follows:
Sending a POST http request to external service with a callback url as body attribute
ex:
{... "callback": "http://url-to-be-called" ...}
Waiting for 3 sequential http requests from the external service
I've used Citrus httpClient to send the Post Request, and Citrus httpServer to exspose endpoints that the service will call using the callback url.
So,
httpClient --> sends POST to ext service
<-- ext service responds OK
(after several minutes)
<-- ext service invokes httpServer API
httpClient --> responds OK
(after several minutes)
<-- ext service invokes httpServer API
httpClient --> responds OK
(after several minutes)
<-- ext service invokes httpServer API
httpClient --> responds OK
My test code is
parallel().actions(
sequential().actions(
// wait for callback
http(httpActionBuilder -> httpActionBuilder
.server(httpServer)
.receive()
.post("/api/callback")
.contentType("application/json;charset=UTF-8")
.accept("text/plain,application/json,application/*+json,*/*")
),
// callback response
http(httpActionBuilder -> httpActionBuilder
.server(httpServer)
.send()
.response(HttpStatus.OK)
.contentType("application/json")
)),
sequential().actions(
http(httpActionBuilder -> httpActionBuilder
.client(httpClient)
.send()
.post("/externalapi)
.payload("{" +
"\"ret\": \"OK\"" +
"}")
.messageType(MessageType.JSON)
.contentType(ContentType.APPLICATION_JSON.getMimeType())
),
http(httpActionBuilder -> httpActionBuilder
.client(httpClient)
.receive()
.response(HttpStatus.OK)
.messageType(MessageType.JSON)
.payload("{" +
"\"ret\": \"OK\"" +
"}")
)
)
);
My configurations
#Bean
public HttpClient httpClient() {
return CitrusEndpoints
.http()
.client()
.requestUrl("http://localhost:8282")
.build();
}
#Bean
public HttpServer httpServer() throws Exception {
return CitrusEndpoints.http()
.server()
.port(8080)
.endpointAdapter(dispatchingEndpointAdapter())
.timeout(300000)
.autoStart(true)
.build();
}
#Bean
public RequestDispatchingEndpointAdapter dispatchingEndpointAdapter() {
RequestDispatchingEndpointAdapter dispatchingEndpointAdapter = new RequestDispatchingEndpointAdapter();
dispatchingEndpointAdapter.setMappingKeyExtractor(mappingKeyExtractor());
dispatchingEndpointAdapter.setMappingStrategy(mappingStrategy());
return dispatchingEndpointAdapter;
}
#Bean
public HeaderMappingKeyExtractor mappingKeyExtractor() {
HeaderMappingKeyExtractor mappingKeyExtractor = new HeaderMappingKeyExtractor();
mappingKeyExtractor.setHeaderName(HttpMessageHeaders.HTTP_REQUEST_URI);
return mappingKeyExtractor;
}
#Bean
public SimpleMappingStrategy mappingStrategy() {
SimpleMappingStrategy mappingStrategy = new SimpleMappingStrategy();
Map<String, EndpointAdapter> mappings = new HashMap<>();
mappings.put("/api/callback", callbackResponseAdapter());
mappingStrategy.setAdapterMappings(mappings);
return mappingStrategy;
}
#Bean
public EndpointAdapter callbackResponseAdapter() {
StaticResponseEndpointAdapter endpointAdapter = new StaticResponseEndpointAdapter();
endpointAdapter.setMessagePayload("{" +
"\"ret\": \"OK\"," +
"}");
return endpointAdapter;
}
The httpClient steps works fine, but when I add the HttpServer I get this error
com.consol.citrus.exceptions.TestCaseFailedException: Unable to create endpoint for static endpoint adapter type 'class com.consol.citrus.endpoint.adapter.RequestDispatchingEndpointAdapter'

In your http-server configuration you are using both static request dispatcher and test action based dynamic response generation. This is not valid.
Please choose only one approach at a time. Either remove the request dispatcher configuration completely or do not try to receive the incoming requests in your test via receive/send action.
By the way the parallel-sequential blocks you are using may not be required. You can use fork=true option on the 1st client request instead and sequentially wait for the Http requests to arrive. This should simplify the test a lot.

Related

Pact consumer test does not successfully mock the spring webclient request using the created pact

I am new to Pact Contract testing and I am trying to create a Pact consumer test to validate a method that calls an api with get request. The api request is made using Spring Webclient.
I am not able to create the webclient object by just providing the Pact mockserver eg.
WebClient webClient = WebClient.builder().baseUrl(mockServer.getUrl()).build();
I am getting the exception java.lang.IllegalStateException: No suitable default ClientHttpConnector found. The explanation I get on the internet for that , is to include reactor-netty-http and I was able to get past this issue when i included that in the POM. But I don't think that is the right solution here because I need the mockserver to respond to the webclient request and it is not. Has anyone dealt with this issue before or am I doing this wrong?
Here is the code snippet:
public RequestResponsePact pactMethod(PactDslWithProvider builder) {
Map<String, String> headers = new HashMap<>();
headers.put(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE);
return builder.given("Consumer request")
.uponReceiving(" getResource call")
.path("/path")
.method("GET")
.willRespondWith()
.status(200)
.headers(headers)
.body(RESPONSE_JSON).toPact();
}
#Test
#PactTestFor(pactMethod = "pactMethod", port = "9999")
public void consumerTest(MockServer mockServer) {
WebClient webClient = WebClient.builder().baseUrl(mockServer.getUrl()).build();
ConsumerServiceClient consumerServiceClient = new ConsumerServiceClient(webClient);
Mono<Data> data = consumerServiceClient.getData();
StepVerifier.create(data)
.assertNext(resp -> {
try {
Value value = resp.getValue();
Assertions.assertFalse( value.isEmpty());
} catch (Exception e) {
log.error("Unable to convert response to Value", e);
Assertions.fail();
}
}).expectComplete()
.verify();
}
The webclient call:
webClient.get()
.uri("/path")
.retrieve()
.onStatus(status -> status == HttpStatus.NOT_FOUND,
res -> Mono.error(new RunTimeException()))
.bodyToMono(clazz);

Wiremock does not receive any requests

I am writing an integrationtest for a resource (Jax-rs with resteasy) which invokes a thirdparty request to the Trustpilot Api to get a review link for a product.
In my tests I wanted to mock this call by using Wiremock and invoke the request by using resteasy Dispatcher.
The code below shows my Setup (names etc are changed).
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(classes = { MyResourceTestContext.class })
public class MyResourceIntegrationTest {
#Rule
public WireMockRule wireMockRule = new WireMockRule(wireMockConfig().dynamicPort().dynamicHttpsPort());
#Autowired
private MyResource myResource;
private Dispatcher dispatcher;
private MockHttpResponse response;
#Before
public void setUp() throws Exception {
dispatcher = MockDispatcherFactory.createDispatcher();
dispatcher.getRegistry().addSingletonResource(myResource);
response = new MockHttpResponse();
}
#Test
public void testResourceShouldReturnRedirectToTrustpilotreview() throws Exception {
URI apiRequestURI = UriBuilder
.fromUri("https://api.trustpilot.com/v1/private/product-reviews/business-units"
+ "/{businessUnitId}}/invitation-links")
.build(BUSINESS_UNIT_ID);
URI expectedRedirectURI = UriBuilder.fromUri("https://products.trustpilot.com/#evaluate/{reviewLinkId}")
.build(REVIEW_LINK_ID);
stubFor(post(apiRequestURI.toString())
.willReturn(okJson("{\"reviewLinkId\":\"" + REVIEW_LINK_ID + "\","
+ "\"reviewUrl\":\"" + expectedRedirectURI.toString() + "\"}")));
MockHttpRequest request = MockHttpRequest.get("/myResource");
dispatcher.invoke(request, response);
WireMock.verify(1, postRequestedFor(urlEqualTo(apiRequestURI.toString())));
Assert.assertEquals(HttpStatus.TEMPORARY_REDIRECT.value(), response.getStatus());
Assert.assertEquals(expectedRedirectURI.toString(), response.getOutputHeaders().getFirst("Location"));
}
But what I'm getting is (with my actual bid in the url):
com.github.tomakehurst.wiremock.client.VerificationException: Expected at least one request matching: {
"url" : "https://api.trustpilot.com/v1/private/product-reviews/business-units/<bid>/invitation-links",
"method" : "POST"
}
Requests received: [ ]
Stepping into the request with the Debugger I can say this request and other requests are actually executed successfully. But they are not recognized/mocked by Wiremock.
Is there something wrong with my Wiremock setup? Any Ideas? Thank you!
There are a couple of reasons why this isn't sending anything to WireMock at the moment:
WireMock only accepts relative URLs when stubbing so passing the full URL including protocol and domain won't work. Try something equivalent to stubFor(post(urlPathEqualTo("/v1/private/product-reviews/business-units/<bid>/invitation-links") instead.
There doesn't seem to be any way for your class under test to know which host/port WireMock is running on and no forward proxying set up so the code is presumably still trying to call the real Trustpilot API.
To solve 2. you have a couple of options:
Set up your class/code under test to use localhost as the destination host and the result of wireMockRule.port() as the port number, or wireMockRule.httpsPort() if you're on HTTPS.
Use the new auto JVM proxy setup feature to proxy your app's calls through WireMock. This assumes you're on WireMock 2.31.0+ and that your app's HTTP client respects the JVM proxy system properties.

Spring Webclient not able to create connections

I'm using Spring Webclient in a Spring Boot project to call a remote API
I've observed a strange behaviour with Webclient. Whenever the Remote API timeouts increase, Webclient goes into a state with very few active connections (less than 5, even though maxConnections is set in the config as 3200), and all the incoming connections get added to the Pending Queue, due to which after a while almost all requests are rejected with a PoolAcquirePendingLimitException exception.
The expected behaviour is that Webclient should create new connections (max upto 3200) to handle the incoming traffic
Webclient Config is as follows:
#Bean
public WebClient webClient(WebClient.Builder builder)
{
TcpClient tcpClient = TcpClient.create(getConnectionProvider())
.option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 10000)
.wiretap(true)
.doOnConnected(connection ->
connection.addHandlerLast(new ReadTimeoutHandler(10000, TimeUnit.MILLISECONDS)));
ClientHttpConnector connector = new ReactorClientHttpConnector(HttpClient.from(tcpClient));
return builder.uriBuilderFactory(initUriTemplateHandler())
.clientConnector(connector)
.build();
}
private ConnectionProvider getConnectionProvider()
{
return ConnectionProvider.builder("fixed")
.maxConnections(3200)
.pendingAcquireTimeout(Duration.ofMillis(10000))
.pendingAcquireMaxCount(10000)
.maxIdleTime(Duration.ofMinutes(10))
.metrics(true)
.build();
}
private DefaultUriBuilderFactory initUriTemplateHandler()
{
DefaultUriBuilderFactory uriFactory = new DefaultUriBuilderFactory();
uriFactory.setEncodingMode(DefaultUriBuilderFactory.EncodingMode.NONE);
return uriFactory;
}
This is how I'm making the API calls
T response = webClient.get()
.uri(url)
.retrieve()
.bodyToMono(responseClass)
.timeout(Duration.ofMillis(requestTimeout)) // varies between 15-20ms
.block();
Below is a screenshot of the metrics[![enter image description here][2]][2]
This is a very high traffic application, and hence virtually it feels like the Pending Queue is stuck at 10000
Dependency Versions:
spring-boot-starter-webflux: 2.2.4.RELEASE
reactory-netty: 0.9.5.RELEASE

Apache HttpClient return cached body on conditional request

I'm using CachingHttpClient implementation of HttpClient from Apache. And having the following scenario:
I made a request for a resource that returned a response with a header:
Cache-Control:max-age=5.
So CachingHttpClient caches the response.
I following I'm making a conditional request for the same resource using If-Modified-Since. And I get a response with status code 304 No modified (btw it does not even checks the server). Without a response body. Which is fine, but I would like to access the cached body, since if its not updated I want to use that.
The question is:
Is there a convenient way to access the cached response from the first call?
(Using org.apache.httpcomponents:httpclient, org.apache.httpcomponents:httpclient-cache; version 4.5.2)
server side:
#RequestMapping("/number")
public int getNumber(HttpServletResponse response, HttpServletRequest request) {
log.info("Number gen called");
response.setHeader("Cache-Control", "max-age=" + 5);
return random.nextInt();
}
client side:
HttpGet httpget = new HttpGet("http://localhost:8080/number");
httpget.setHeader("If-Modified-Since", java.time.format.DateTimeFormatter.RFC_1123_DATE_TIME.
format(ZonedDateTime.now(ZoneId.of("GMT")).minusSeconds(1)));
HttpResponse resp = httpClient.execute(httpget);
log.info("code: " + resp.getStatusLine().getStatusCode());
// here fails because no body on 2. call
String responseString = new BasicResponseHandler().handleResponse(resp);
Http Client Init
#Bean
public HttpClient httpClient() {
return CachingHttpClients.createMemoryBound();
}
Ok so if I configure the cache explicitly, I can use that cache object directly too, so this was a typical RTFM problem, my bad.
#Bean
public HttpCacheStorage httpCacheStorage() {
CacheConfig cacheConfig = CacheConfig.custom()
.setMaxCacheEntries(1000)
.setMaxObjectSize(8192)
.build();
HttpCacheStorage cacheStorage = new BasicHttpCacheStorage(cacheConfig);
return cacheStorage;
}

RestEasy client spring integration: can not auto follow redirects

Problem: I can not get RestEasy to automatically follow redirects
I'm using the RestEasy client framework 2.3.4 to consume RESTful JSON services. I'm using the rest easy client spring integration. If I wasn't using spring RestClientProxyFactoryBean to create my services I would set the auto redirect flag on the client request factory
I have tried setting the follow redirect on my HTTP client and following the debug I can see this value is overridden to false by Rest Easy.
Looking at the source code I need to get access to the client invoker that the spring proxy factory creates but it doesn't expose this.
This is like a very common task, surely I am missing something? Cheers.
You should be able to set a custom client executor on the proxybean factory but that also didn't work e.g
#Override
public ClientRequest createRequest(String uriTemplate) {
ClientRequest clientRequest = new ClientRequest(uriTemplate, this);
clientRequest.followRedirects(true);
return clientRequest;
}
#Override
public ClientRequest createRequest(UriBuilder uriBuilder) {
ClientRequest clientRequest = super.createRequest(uriBuilder);
clientRequest.followRedirects(true);
return clientRequest;
}
}
proxyFactoryBean.setClientExecutor(new FollowRedirectsClientExecutor());
In end extending and overriding the Http client (in this case HTTP Component) was needed to make this work e.g.
public HttpUriRequest followRedirects(HttpUriRequest request) {
if (logger.isDebugEnabled()) {
logger.debug("Setting allow redirects");
}
HttpParams p = request.getParams();
HttpClientParams.setRedirecting(p, true);
request.setParams(p);
return request;
}
}
...
#Override
public <T> T execute(HttpUriRequest request, ResponseHandler<? extends T> responseHandler) throw
s IOException,
ClientProtocolException { ClientProtocolException {
request = followRedirects(request);
...

Resources