How to test a POSTing method by using an embedded Webserver in Springboot? - spring-boot

I am searching for a way to test a method, which sends a POST request to an external service. The application will NOT be itself a consumable webservice, that is why I didn't implement the shown class below as #RestController, #Controller, #Service, whatever types there may be.
But I don't know how to call the method postNumberPlate() to send a request to an embedded webserver (started in/by/at the unit test) to make some assertions on it. I want to avoid, to install an external webserver.
In other words: Can I start an embedded webserver inside a unit-test and 'tell' it to accept my POST request to inspect and assert the contents?
I already did:
a massive Webresearch (2-3 days?)
read Howto's
check the springboot docs
use an embedded Jetty Server (somehow blocking loop)
declare the Application as Webapplication and setting random port to jetty
experiment with Mockito, MockMVC
read "How to unittest a class using RestTemplate offline?" and compared it to my case, but found,
that it's very old (8y),
I don't know how to implement the parent interface, which is pretty huge
that the question and answers are too generic to deduce a solution for my case
it's not answering the embedded testing webserver problem I included.
The Class to be tested:
public class RestfulClient {
private RestTemplate restTemplate = new RestTemplate();
private HttpHeaders headers = new HttpHeaders();
#Value("${kafkaeskadapter.uri}")
private String destinationURL;
public RestfulClient() {}
public ResponseEntity<String> postNumberPlate(String key, CamImage camImage) {
LinkedMultiValueMap<String, Object> map = new LinkedMultiValueMap<>();
map.add("numplate", camImage.getIdentifier());
headers.setContentType(MediaType.MULTIPART_FORM_DATA);
HttpEntity<LinkedMultiValueMap<String, Object>> requestEntity = new HttpEntity<LinkedMultiValueMap<String,
Object>>(map, headers);
ByteArrayResource resource = new ByteArrayResource(camImage.getData()) {
/**
* IMPORTANT!!! Otherwise I receive a BAD REQUEST
* #return
*/
#Override
public String getFilename() {
return camImage.getIdentifier() + ".png";
}
};
map.add("image", resource);
ResponseEntity<String> result = restTemplate.exchange(destinationURL, HttpMethod.POST,
requestEntity, String.class);
return result;
}
}
I hope I could clarify my question a bit.

A solution is to write a simple light-weight Webservice Endpoint and include it into your Run Configuration of your IDE. I made a separate mini project and would add further methods if needed, e.g. to accept different media.
Prior to run the actual unit tests, it is possible to configure the start of the Endpoint and return a meaningful ResponseEntity. The result can be inspected et voilĂ , assertions are possible.
A word about StackOverflow user arrogance: #Raedwald, after reading and trying, the answers in the linked question are not really helpful, but involve a lot of knowlegde about the stuff, and I have no one around of my colleagues, which could ever assist at programming. So it wasn't helpful to flag my question for deletion.

Related

How do I unit test a Spring RestTemplate that takes a ResponseExtractor and RequestCallback?

I am developing in Groovy and I am trying to write a Spock unit test for the following use of Spring's RestTemplate...
Included are my request callback and response extractors, and my initialization class of the RestTemplate bean. I am using the ResponseExtractor to stream the response from GET myurl/ and copy it to a file. The RequestCallback is simply setting some headers on the request.
class RestTemplateConfig() {
#Bean(name = 'myRestTemplate')
RestTemplate getMyRestTemplate() {
RestTemplate restTemplate = new RestTemplateBuilder().build()
return restTemplate
}
}
class MyClass() {
#Autowired
#Qualifier('myRestTemplate')
RestTemplate restTemplate
File getFile() {
ResponseExtractor<Void> responseExtractor = { ClientHttpResponse response ->
// do something with the response
// in this case, the response is an input stream so we copy the input stream to a file
myFile = response.getBody() // roughly, in a psuedocode-ish way
return null
}
RequestCallback requestCallback = { ClientHttpRequest request ->
request.getHeaders().setAccept([MediaType.APPLICATION_JSON])
}
File myFile
// get my file data
restTemplate.execute('myurl/', HttpMethod.GET, requestCallback, responseExtractor)
return myFile
}
}
Spring framework docs for that particular execute(...) method: https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/web/client/RestTemplate.html#execute-java.net.URI-org.springframework.http.HttpMethod-org.springframework.web.client.RequestCallback-org.springframework.web.client.ResponseExtractor-
How do I mock out what's happening in these closures? Specifically, I'm interested in mocking out my response extractor because my current test always returns myFile as null.
when:
// do stuff
then:
1 * restTemplate.execute('myurl/, HttpMethod.GET, _, _) // how can I mock out the expected response here?
0 * _
myFile != null // this fails because myFile is null
After you updated your sample code as I requested, I can see more clearly now. You are suffering from a typical (non-)testability problem: Your method getFile does more than just getting a file. It instantiates two dependencies as local variables, making them unmockable and consequently the whole method mostly untestable.
So you want to refactor for better testability so as to be able to use one testing method I mentioned in my first comment:
If the requestCallback and responseExtractor can be injected via constructor or setter, you can inject mocks.
If they are created by some kind of factory class, you can stub that class.
In case of a factory method inside the class under test itself you can use a spy on the class and stub the factory method.
For a more general discussion of testability and how tests drive application design, see my other answer here, sections "General comments" and "Update".
If any of this is unclear, feel free to ask related(!) follow-up questions.

How do I make spring boot RestTemplate client metrics not create a new tag for query parameters

I've got a spring boot application that is defining a RestTemplate bean as follows:
#Bean
public RestTemplate restTemplate(RestTemplateBuilder builder) {
return builder.build();
}
Also, pulling in spring-boot-starter-actuator and io.micrometer:micrometer-registry-prometheus.
When I use the injected RestTemplate as follows:
#Autowired
private RestTemplate restTemplate;
private String uriTemplate = "http://my.domain.com/bookstore-api/books";
public List<Book> getBooksByAuthor(String author) {
// create URI for "http://my.domain.com/bookstore-api/books?author={authorId}"
UriComponentsBuilder uriComponentsBuilder = UriComponentsBuilder
.fromUriString(uriTemplate)
.queryParam("author", author);
// make the GET
ResponseEntity<Book[]> responseEntity = restTemplate.getForEntity(uriComponentsBuilder.toUriString(),Book[].class);
// rest ommitted for brevity
}
When getBooksByAuthor("Tolkien") is called, we can then hit /metrics/prometheus and see the following:
http_client_requests_seconds_count{clientName="my.domain.com",method="GET",status="200",uri="/bookstore-api/books?author=Tolkien",} 2.0
http_client_requests_seconds_sum{clientName="my.domain.com",method="GET",status="200",uri="/bookstore-api/books?author=Tolkien",} 0.253227898
This would be fine, except that there are lots of authors out there, and eventually I will get the "too many tags" exception.
I would prefer to have the following (similar to how path variables get templated):
http_client_requests_seconds_count{clientName="my.domain.com",method="GET",status="200",uri="/bookstore-api/books?author={author}",} 2.0
http_client_requests_seconds_sum{clientName="my.domain.com",method="GET",status="200",uri="/bookstore-api/books?author={author}",} 0.253227898
Is this possible to achieve by modifying the way I use UriComponentsBuilder? The closest thing I've found is to define my own RestTemplateExchangeTagsProvider, and override the default to do some crazy regex replacement.
Just fixed same issue in SpringBoot 2.4.5 using:
responseEntity = restTemplate.exchange(
config.getDataUrl(),
HttpMethod.GET,
httpEntity,
new ParameterizedTypeReference<String>() {},
rowId);
where getDataUrl resolves to:
https://data-service-dev.apps.cloud.net/api/hbase/getData?rowId={rowId}
metrics:
http_client_requests_seconds_count{clientName="data-service-dev.apps.cloud.net",method="GET",outcome="SUCCESS",status="200",uri="/api/hbase/getData?rowId={rowId}",} 1.0
...
I have had a same problem. Maybe this information will help you.
In my case restTemplate.setUriTemplateHandler(handler) had overwritten annonymous
MetricsClientHttpRequestInterceptor#createUriTemplateHandler.
And the original url templates had not been stored into memory for prometheus.
DefaultUriBuilderFactory builderFactory = new DefaultUriBuilderFactory();
builderFactory.setEncodingMode(DefaultUriBuilderFactory.EncodingMode.NONE);
customizer.customize(restTemplate);
restTemplate.setUriTemplateHandler(handler);
So,
I changed order of the commands:
restTemplate.setUriTemplateHandler(handler);
customizer.customize(restTemplate);
Please check that there are no setting commands for restTemplate after MetricsClientHttpRequestInterceptor.customize(restTemplate).

How to solve this "4th dimension" missing certificate problem?

I have a rest client maven module which is responsible for getting a token, and then for creating requests using this token.
Here is the "login()" method :
private static final String ROOT_URL = "https://my-url";
private static final String LOGIN_API = "/LogIn";
#Autowired
#Qualifier("myRestTemplate")
RestTemplate restTemplate;
public String login() {
LoginRequest loginRequest = new LoginRequest();
loginRequest.setAccountName("USER");
loginRequest.setPassword("P#ssw0rd");
loginRequest.setTimeout(120);
LoginResponse loginResponse = restTemplate.postForObject(ROOT_URL + LOGIN_API, loginRequest, LoginResponse.class);
String token = loginResponse.getLogInResult().getToken();
log.info(loginResponse.toString());
return token;
}
When I call this "login()" method from a JUnit inside the module, it works.
Also I have various services from other modules who depend on this rest client module and they can call it with no error.
Now I've just created a new module who needs to use this rest client service. When I call the "login()" service from this new module, it ends with :
ResourceAccessException: I/O error on POST request for "https://my-url/LogIn":
unable to find valid certification path to requested target
What can possibly could cause this error in a case and not the other, when the URL called is a constant ?
I've tried to check the "restTemplate" object is the same in both case and I think it is, but even if it was not, how could this happen ? by which mechanism ?
UPDATE : I've tried to activate spring debug on org.springframework.beans to see if a wrong restTemplate injection could be the cause of this problem, but when I do this from my parent module, I don't even see the injection of any restTemplate.
It's like some process are happening in a forked JVM, and this forked process doesnt know about the certificates (the ones in $JAVA_HOME/jre/lib/security/cacerts).
If I debug the call from the child module, I can see the restTemplate injection happening.

Cannot get Spring Boot to lazily resolve a multipart file

I have created a Spring Boot 2 demo application with the Spring Initializr and added the controller below:
#Controller
#RequestMapping("/demo")
public class UploadController {
private final static Logger LOG = LoggerFactory.getLogger(UploadController.class);
#PostMapping("/upload")
public ResponseEntity<String> uploadFile(
#RequestParam("metadata") MultipartFile metadata,
#RequestParam("payload") MultipartFile payload) throws IOException {
ObjectMapper mapper = new ObjectMapper();
Map metadataMap = mapper.readValue(metadata.getInputStream(), Map.class);
LOG.info("Received call to upload file {}", metadataMap.get("filename"));
LOG.info("File size: {}", payload.getBytes().length);
LOG.info("File {} successfully uploaded", metadataMap.get("filename"));
return ResponseEntity.ok().build();
}
}
I then added an application.yaml file containing this configuration:
spring:
servlet:
multipart:
max-file-size: 2000000MB
max-request-size: 2000000MB
resolve-lazily: true
My goal is to have the controller parse and log the metadata file before it starts reading the payload file, but the resolve-lazily setting seems to be ignored by Boot: the code inside the controller won't be executed until the whole body is read.
I use the command below to test the controller:
curl -F metadata=#metadata.json -F payload=#payload.bin http://localhost:8080/demo/upload
Is there anything wrong with my code/configuration? Am I getting the meaning of the setting right?
At present, if you want to avoid reading (and buffering) the whole body all at once, I think you will have to provide your own parser, as described in the answers here. What would be really interesting (but generally unnecessary) would be to do so in the form of a new MultipartResolver implementation.
There are two existing implementations documented for interface MultipartResolver, and both supply a function setResolveLazily(boolean) (standard), (commons). I have tried with both, and neither seem to allow for parsing or streaming multipart files or parameters independently.
Default is "false", resolving the multipart elements immediately, throwing corresponding exceptions at the time of the resolveMultipart(javax.servlet.http.HttpServletRequest) call. Switch this to "true" for lazy multipart parsing, throwing parse exceptions once the application attempts to obtain multipart files or parameters.
Despite what it says in the documentation, I have found that once you call resolveMultipart, the entire body is parsed and buffered before the call returns. I know this because I can watch the temp-files being created.
One note about "Is there anything wrong with my code"...
Answer: Yes, because by using #RequestParam you have indirectly asked Spring to resolve your parameters ahead of time, before your controller is ever called. What you should be able to do instead (if the documentation were correct) is request the parameters independently from inside your controller:
Configuration (application.properties):
spring.servlet.multipart.enabled = true
spring.servlet.multipart.resolve-lazily = true
Controller:
#PostMapping(path = "/upload", consumes = MediaType.MULTIPART_FORM_DATA_VALUE)
public ResponseEntity<Void> postUpload(HttpServletRequest rawRequest) {
multipartResolver.setResolveLazily(true); // unclear why this is exists
MultipartHttpServletRequest request = multipartResolver.resolveMultipart(rawRequest);
String p1 = request.getParameter("first-parameter");
String p2 = request.getParameter("second-parameter");
System.out.println("first-parameter="+p1+", second-parameter"+p2);
multipartResolver.cleanupMultipart(request);
return new ResponseEntity<Void>(HttpStatus.ACCEPTED);
}
One useful aspect of resolve-lazily that I have discovered is that it allows you to write your own parser for some rest controllers while using the built-in parser for others (see my answer here). In other words, you don't have to use spring.servlet.multipart.enabled = false to get your parser to work. This is a minor breakthrough relative to other advice that I had seen previously.

Spring HATEOAS and Ribbon with Eureka - host name in created links

I'm using Spring HATEOAS to build and access a REST service. The service registers itself with a eureka server, and I use Ribbon (via #LoadBalanced RestTemplate) to call it.
Per default, the requests are made for the hostname (in the Host header) for the resolved instance. This causes the LinkBuilder in Spring HATEOAS to generate links for this host. When Ribbon makes a request to follow a link, it tries to lookup the link host name in eureka again and of course gets no result.
What is the best level to address this situation? After receiving the links with the first result, I guess it would be acceptable to direct the immediate following requests to the same service instance, I still feel making all requests to the symbolic service name would be better.
Is it possible to configure Ribbon to make requests with that service name in the Host header (assuming no virtual hosting in the target service, that seems to be a valid assumption)?
Otherwise is it possible to set a canonical base URL for the link builder in HATEOAS? (defining all links as complete strings is not acceptable for me)
My solution/workaround for now is explicitly setting an X-Forwarded-Host header when requesting the root resource. This works, but is a bit verbose and cumbersome.
Traverson traverson = new Traverson(URI.create("http://photo-store/"), MediaType.APPLICATION_JSON, MediaType.APPLICATION_JSON_UTF8, MediaTypes.HAL_JSON).setRestOperations(imageService);
HttpHeaders httpHeaders = new HttpHeaders();
httpHeaders.set("X-Forwarded-Host", "photo-store");
String original = this.traverson.follow("image:original").withHeaders(httpHeaders).asTemplatedLink().expand(photoId).getHref();
(Side question: can I override the Host header like that, or is it treated special in Ribbon or RestTemplate?)
I feel that there should be a more declarative or convention or configuration based way to deal with this. Is there?
I also faced same problem. I resolved it by adding interceptor which adds X-Forwarded-Host header to every request.
#Bean
#LoadBalanced
RestTemplate restTemplate() {
RestTemplate restTemplate = new RestTemplate();
restTemplate.setInterceptors(singletonList((ClientHttpRequestInterceptor) (request, body, execution) -> {
request.getHeaders().add("X-Forwarded-Host", request.getURI().getHost());
return execution.execute(request, body);
}));
return restTemplate;
}
I am using my own linkBuilder which fetches the name of the service from the properties and uses that as a Hostname. This is pretty basic, but easy to use and implement. If you want to use the whole spring hateoas link discovery magic sauce, you need to do more. But right now I'm searching for a solution that fits better with Spring Hateoas 1.x myself, so maybe I'll find a better answer soon.
The quick and dirty way:
#Component
public class MyLinkBuilder {
#Value("${spring.application.name}")
private String servicename;
public Link getLink(String path) {
String root = "http://" + servicename;
return new Link(root + path);
}
public Link getLink(String path, String rel) {
return getLink(path, LinkRelation.of(rel));
}
public Link getLink(String path, LinkRelation rel) {
String root = "http://" + servicename;
return new Link(root + path, rel);
}
}

Resources