Spring Integration HttpRequestExecutingMessageHandler ContentType Issue - spring-boot

I am facing a problem with Spring Integration. I am trying to execute a rest call via HttpRequestExecutingMessageHandler. My rest endpoint is accepting content-type 'application/json' only.
The problem is that the HttpRequestExecutingMessageHandler is posting with content-type 'text/plain;charset=UTF-8'.
#ServiceActivator(inputChannel = "transformRequestToJsonChannel",
outputChannel = "httpRequestOutChannel")
public Message<?> transformRequest(Message<DocumentConverterRequest>
message)
{
LOG.info("transforming document converter request to json: '{}'",
ObjectToJsonTransformer transformer = new ObjectToJsonTransformer();
transformer.setContentType(MediaType.APPLICATION_JSON_UTF8_VALUE);
Object payload = transformer.transform(message).getPayload();
LOG.info("payload: '{}'", payload.toString());
return MessageBuilder.withPayload(payload).build();
}
#Bean
#ServiceActivator(inputChannel = "httpRequestOutChannel")
public HttpRequestExecutingMessageHandler outbound() {
HttpRequestExecutingMessageHandler handler = new
HttpRequestExecutingMessageHandler(documentConverterRestUrl);
handler.setHttpMethod(HttpMethod.POST);
handler.setErrorHandler(httpResponseErrorHandler);
handler.setExpectedResponseType(String.class);
handler.setCharset(Charset.defaultCharset().name());
HeaderMapper<HttpHeaders> mapper = new DefaultHttpHeaderMapper();
HttpHeaders httpHeaders = new HttpHeaders();
httpHeaders.add(HttpHeaders.CONTENT_TYPE,
MediaType.APPLICATION_JSON_VALUE);
mapper.toHeaders(httpHeaders);
handler.setHeaderMapper(mapper);
handler.setOutputChannel(httpResponseChannel());
return handler;
}
How can i override the content-type?

This piece of code does nothing:
HeaderMapper<HttpHeaders> mapper = new DefaultHttpHeaderMapper();
HttpHeaders httpHeaders = new HttpHeaders();
httpHeaders.add(HttpHeaders.CONTENT_TYPE,
MediaType.APPLICATION_JSON_VALUE);
mapper.toHeaders(httpHeaders);
That toHeaders() is called from the HttpRequestExecutingMessageHandler when we receive response. It really useless to use it explicitly in your code, especially in the bean definition phase and when you ignore a result.
You don't need to use an explicit HeaderMapper at all: a default one should be enough for you.
The ObjectToJsonTransformer really maps that setContentType() into a headers of the message it replies:
if (headers.containsKey(MessageHeaders.CONTENT_TYPE)) {
// override, unless empty
if (this.contentTypeExplicitlySet && StringUtils.hasLength(this.contentType)) {
headers.put(MessageHeaders.CONTENT_TYPE, this.contentType);
}
}
else if (StringUtils.hasLength(this.contentType)) {
headers.put(MessageHeaders.CONTENT_TYPE, this.contentType);
}
So, there is a proper content type to map. By default HttpRequestExecutingMessageHandler uses:
/**
* Factory method for creating a basic outbound mapper instance.
* This will map all standard HTTP request headers when sending an HTTP request,
* and it will map all standard HTTP response headers when receiving an HTTP response.
* #return The default outbound mapper.
*/
public static DefaultHttpHeaderMapper outboundMapper() {
With an appropriate set of headers to map to HTTP request and from HTTP response.
The new DefaultHttpHeaderMapper() brings just an empty set of headers to map.
Please, raise an issue to improve JavaDocs and Reference Manual to note that default ctor of that class doesn't bring any headers to map.

#Bean
#ServiceActivator(inputChannel = "httpRequestOutChannel")
public HttpRequestExecutingMessageHandler outbound() {
HttpRequestExecutingMessageHandler handler = Http.outboundGateway(documentConverterRestUrl)
.httpMethod(HttpMethod.POST)
.messageConverters(new MappingJackson2HttpMessageConverter())
.mappedRequestHeaders("Content-Type")
.get();
handler.setOutputChannel(httpResponseChannel());
return handler;
}
I removed my ObjectToJsonTransformer, because the messageConverters(new MappingJackson2HttpMessageConverter()) is doing the stuff.
Also i had to add the content-type to my message header: .setHeaderIfAbsent(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)

Related

RestTemplate execute() method cannot send JSON Payload

In my application, I need to take data from another request and chain into a new one
I must use the exchange() method of RestTemplate because I have issue with jacksons lib and I cannot add/change the libs.
this is my code:
final RequestCallback requestCallback = new RequestCallback() {
#Override
public void doWithRequest(final ClientHttpRequest request) throws IOException {
request.getHeaders().add(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE);
// Add basic auth header
String auth = username + ":" + password;
byte[] encodedAuth = Base64Utils.encode(auth.getBytes(StandardCharsets.US_ASCII));
String authHeader = "Basic " + new String(encodedAuth);
request.getHeaders().add("Authorization", authHeader);
// Add Headers Request
Enumeration headerNamesReq = servletRequest.getHeaderNames();
while (headerNamesReq.hasMoreElements()) {
String headerName = (String) headerNamesReq.nextElement();
if (whiteListedHeaders.contains(headerName.toLowerCase())) {
String headerValue = servletRequest.getHeader(headerName);
request.getHeaders().add(headerName, headerValue);
}
}
request.getHeaders().forEach((name, value) -> {
log.info("RestExecutorMiddleware", "HEADERS ---\t" + name + ":" + value);
});
IOUtils.copy(new ByteArrayInputStream(payload.getBytes()), request.getBody());
}
};
// Factory for restTemplate
SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory();
requestFactory.setBufferRequestBody(false);
restTemplate.setRequestFactory(requestFactory);
ClientHttpResponse responsePost = restTemplate.execute(url, method, requestCallback, new ResponseFromHeadersExtractor());
But at the end, the endpoint cannot receive my JSON (receive data, but not JSON.)
Where I wrong?
Thanks
Very inaccuracy code. Make all steps one-to-one and it will work, you make optimization later ...
Basic Auth. Don't do unnecessary actions
var headers = new HttpHeaders();
headers.setBasicAuth(username, password);
That's all, Spring will take care of everything else - to apply Base64, add Basic: and set properly a header.
Set all required headers including headers.setContentType(MediaType.APPLICATION_JSON)
Get an entity/object which you need to send (set as a body) with a request.
Serialize your object. The most popular, proper and simple way is using fasterxml json framework, you can make serialization with mapper.writeBalueAsString(<your object>). If you really cannot use external libraries, HttpEntity should make it: var request = new HttpEntity<>(<object>, headers);
Make restTemplate request. In almost all cases more convenient methods are restTemplate.postForObject(), restTemplate.getForObject(), restTemplate.postForEntity(), etc.: restTemplate.postForObject(uri, request, ResponseObject.class)

Spring Boot MVC to allow any kind of content-type in controller

I have a RestController that multiple partners use to send XML requests. However this is a legacy system that it was passed on to me and the original implementation was done in a very loose way in PHP.
This has allowed to clients, that now they refuse to change, to send different content-types (application/xml, text/xml, application/x-www-form-urlencoded) and it has left me with the need to support many MediaTypes to avoid returning 415 MediaType Not Supported Errors.
I have used the following code in a configuration class to allow many media types.
#Bean
public MarshallingHttpMessageConverter marshallingMessageConverter() {
MarshallingHttpMessageConverter converter = new MarshallingHttpMessageConverter();
converter.setMarshaller(jaxbMarshaller());
converter.setUnmarshaller(jaxbMarshaller());
converter.setSupportedMediaTypes(Arrays.asList(MediaType.APPLICATION_OCTET_STREAM, MediaType.APPLICATION_XML,
MediaType.TEXT_XML, MediaType.TEXT_PLAIN, MediaType.APPLICATION_FORM_URLENCODED, MediaType.ALL));
return converter;
}
#Bean
public Jaxb2Marshaller jaxbMarshaller() {
Jaxb2Marshaller marshaller = new Jaxb2Marshaller();
marshaller.setClassesToBeBound(CouponIssuedStatusDTO.class, CouponIssuedFailedDTO.class,
CouponIssuedSuccessDTO.class, RedemptionSuccessResultDTO.class, RedemptionResultHeaderDTO.class,
RedemptionFailResultDTO.class, RedemptionResultBodyDTO.class, RedemptionDTO.class, Param.class,
ChannelDTO.class, RedeemRequest.class);
Map<String, Object> props = new HashMap<>();
props.put(javax.xml.bind.Marshaller.JAXB_FORMATTED_OUTPUT, true);
marshaller.setMarshallerProperties(props);
return marshaller;
}
The controller method is this:
#PostMapping(value = "/request", produces = { "application/xml;charset=UTF-8" }, consumes = MediaType.ALL_VALUE)
public ResponseEntity<RedemptionResultDTO> request(
#RequestHeader(name = "Content-Type", required = false) String contentType,
#RequestBody String redeemRequest) {
return requestCustom(contentType, redeemRequest);
}
This endpoint is hit by all clients. It is only one last client giving me trouble. They are sending content-type = application/x-www-form-urlencoded; charset=65001 (UTF-8)": 65001 (UTF-8)
Due to the way the charset is sent, Spring Boot refuses to return anything but 415. Not even MediaType.ALL seems to have any effect.
Is there a way to make Spring allow this to reach me ignoring the content-type? Creating a filter and changing the content type was not feasible since the HttpServletRequest is not allowing to mutate the content-type. I am out of ideas but I really think there has to be a way to allow custom content-types.
UPDATE
If I remove the #RequestBody then I don't get the error 415 but I have no way to get the request body since the HttpServletRequest reaches the Controller action empty.
You best case is to remove the consumes argument from the RequestMapping constructor. The moment you have it added, spring will try to parse it into known type MediaType.parseMediaType(request.getContentType()) & which tries to create a new MimeType(type, subtype, parameters) and thus throws exception due to invalid charset format being passed.
However, if you remove the consumes, and you wanna validate/restrict the incoming Content-Type to certain type, you can inject HttpServletRequest in your method as parameter, and then check the value of request.getHeader(HttpHeaders.CONTENT_TYPE).
You also have to remove the #RequestBody annotation so Spring doesn't attempt to parse the content-type in attempt to unmarshall the body. If you directly attempt to read the request.getInputStream() or request.getReader() here, you will see null as the stream has already been read by Spring. So to get access to input content, use spring's ContentCachingRequestWrapper inject using Filter and then you can later repeatedly read the content as it's cached & not reading from original stream.
I am including some code snippet here for reference, however to see executable example, you can refer my github repo. Its a spring-boot project with maven, once you launch it, you can send your post request to http://localhost:3007/badmedia & it will reflect you back in response request content-type & body. Hope this helps.
#RestController
public class BadMediaController {
#PostMapping("/badmedia")
#ResponseBody
public Object reflect(HttpServletRequest request) throws IOException {
ObjectMapper mapper = new ObjectMapper();
JsonNode rootNode = mapper.createObjectNode();
((ObjectNode) rootNode).put("contentType", request.getHeader(HttpHeaders.CONTENT_TYPE));
String body = new String(((ContentCachingRequestWrapper) request).getContentAsByteArray(), StandardCharsets.UTF_8);
body = URLDecoder.decode(body, StandardCharsets.UTF_8.name());
((ObjectNode) rootNode).put("body", body);
return mapper.writerWithDefaultPrettyPrinter().writeValueAsString(rootNode);
}
}
#Component
public class CacheRequestFilter extends GenericFilterBean {
#Override
public void doFilter(ServletRequest servletRequest, ServletResponse servletResponse, FilterChain chain)
throws IOException, ServletException {
HttpServletRequest cachedRequest
= new ContentCachingRequestWrapper((HttpServletRequest) servletRequest);
//invoke caching
cachedRequest.getParameterMap();
chain.doFilter(cachedRequest, servletResponse);
}
}

Streaming upload via #Bean-provided RestTemplateBuilder buffers full file

I'm building a reverse-proxy for uploading large files (multiple gigabytes), and therefore want to use a streaming model that does not buffer entire files. Large buffers would introduce latency and, more importantly, they could result in out-of-memory errors.
My client class contains
#Autowired private RestTemplate restTemplate;
#Bean
public RestTemplate restTemplate(RestTemplateBuilder restTemplateBuilder) {
int REST_TEMPLATE_MODE = 1; // 1=streams, 2=streams, 3=buffers
return
REST_TEMPLATE_MODE == 1 ? new RestTemplate() :
REST_TEMPLATE_MODE == 2 ? (new RestTemplateBuilder()).build() :
REST_TEMPLATE_MODE == 3 ? restTemplateBuilder.build() : null;
}
and
public void upload_via_streaming(InputStream inputStream, String originalname) {
SimpleClientHttpRequestFactory requestFactory = new SimpleClientHttpRequestFactory();
requestFactory.setBufferRequestBody(false);
restTemplate.setRequestFactory(requestFactory);
InputStreamResource inputStreamResource = new InputStreamResource(inputStream) {
#Override public String getFilename() { return originalname; }
#Override public long contentLength() { return -1; }
};
MultiValueMap<String, Object> body = new LinkedMultiValueMap<String, Object>();
body.add("myfile", inputStreamResource);
HttpHeaders headers = new HttpHeaders();
headers.setContentType(MediaType.MULTIPART_FORM_DATA);
HttpEntity<MultiValueMap<String, Object>> requestEntity = new HttpEntity<>(body,headers);
String response = restTemplate.postForObject(UPLOAD_URL, requestEntity, String.class);
System.out.println("response: "+response);
}
This is working, but notice my REST_TEMPLATE_MODE value controls whether or not it meets my streaming requirement.
Question: Why does REST_TEMPLATE_MODE == 3 result in full-file buffering?
References:
How to forward large files with RestTemplate?
How to send Multipart form data with restTemplate Spring-mvc
Spring - How to stream large multipart file uploads to database without storing on local file system -- establishing the InputStream
How to autowire RestTemplate using annotations
Design notes and usage caveats, also: restTemplate does not support streaming downloads
In short, the instance of RestTemplateBuilder provided as an #Bean by Spring Boot includes an interceptor (filter) associated with actuator/metrics -- and the interceptor interface requires buffering of the request body into a simple byte[].
If you instantiate your own RestTemplateBuilder or RestTemplate from scratch, it won't include this by default.
I seem to be the only person visiting this post, but just in case it helps someone before I get around to posting a complete solution, I've found a big clue:
restTemplate.getInterceptors().forEach(item->System.out.println(item));
displays...
org.SF.boot.actuate.metrics.web.client.MetricsClientHttpRequestInterceptor
If I clear the interceptor list via setInterceptors, it solves the problem. Furthermore, I found that any interceptor, even if it only performs a NOP, will introduce full-file buffering.
public class SimpleClientHttpRequestFactory { ...
I have explicitly set bufferRequestBody = false, but apparently this code is bypassed if interceptors are used. This would have been nice to know earlier...
#Override
public ClientHttpRequest createRequest(URI uri, HttpMethod httpMethod) throws IOException {
HttpURLConnection connection = openConnection(uri.toURL(), this.proxy);
prepareConnection(connection, httpMethod.name());
if (this.bufferRequestBody) {
return new SimpleBufferingClientHttpRequest(connection, this.outputStreaming);
}
else {
return new SimpleStreamingClientHttpRequest(connection, this.chunkSize, this.outputStreaming);
}
}
public abstract class InterceptingHttpAccessor extends HttpAccessor { ...
This shows that the InterceptingClientHttpRequestFactory is used if the list of interceptors is not empty.
/**
* Overridden to expose an {#link InterceptingClientHttpRequestFactory}
* if necessary.
* #see #getInterceptors()
*/
#Override
public ClientHttpRequestFactory getRequestFactory() {
List<ClientHttpRequestInterceptor> interceptors = getInterceptors();
if (!CollectionUtils.isEmpty(interceptors)) {
ClientHttpRequestFactory factory = this.interceptingRequestFactory;
if (factory == null) {
factory = new InterceptingClientHttpRequestFactory(super.getRequestFactory(), interceptors);
this.interceptingRequestFactory = factory;
}
return factory;
}
else {
return super.getRequestFactory();
}
}
class InterceptingClientHttpRequest extends AbstractBufferingClientHttpRequest { ...
The interfaces make it clear that using InterceptingClientHttpRequest requires buffering body to a byte[]. There is not an option to use a streaming interface.
#Override
public ClientHttpResponse execute(HttpRequest request, byte[] body) throws IOException {

OAuth2RestTemplate TCP connections

I'm using a Spring OAuth2RestTemplate with ClientCredentialsResourceDetails to acquire an API authorization token. The authorization server and the API endpoints are hidden behind the same load balancers (LB). We have an issues where the first connection to the API endpoint, after acquiring the token, fails with a 404 error message but subsequent calls to the same API endpoint with the same token are successful. I believe the LB is miss-configured in some way but we've been asked if we could try using separate TCP sessions for the acquisition of the token and then the REST call. Is there a way to get the Spring RestTemplate to do this?
UPDATE
Here's how I create and configure the template:
#Bean
public OAuth2RestTemplate oauth2RestTemplate(
#Value("${token.uri}") final String tokenUri,
#Value("${token.clientId:client}") final String clientId,
#Value("${token.secret:secret}") final String clientSecret,
#Value("${token.scope:platform}") final String scope,
final MappingJackson2HttpMessageConverter customJackson2HttpMessageConverter)
{
ClientCredentialsResourceDetails rd = new
ClientCredentialsResourceDetails();
rd.setAuthenticationScheme(AuthenticationScheme.header);
rd.setAccessTokenUri(tokenUri);
rd.setClientId(clientId);
rd.setClientSecret(clientSecret);
rd.setScope(Arrays.asList(scope));
OAuth2RestTemplate rt = new OAuth2RestTemplate(rd);
List<HttpMessageConverter<?>> converters = rt.getMessageConverters();
converters.add(customJackson2HttpMessageConverter);
rt.setMessageConverters(converters);
return rt;
}
and here's the call to the api:
HttpHeaders headers = new HttpHeaders();
headers.setAccept(Arrays.asList(MediaType.APPLICATION_JSON));
headers.set("Connection", "close"); // hmm, gets replace by keep-alive on the token api request!
HttpEntity<String> entity = new HttpEntity<String>(headers);
ResponseEntity<MyObject[]> response = restTemplate.exchange(
"http://example.com/api/v1/rest/method",
HttpMethod.GET, entity, MyObject[].class);
Thanks.
Try adding the Connection request header with value as close while sending your request using resttemplate. This should force the TCP connection to be closed after each request. Not very performant though.
HttpHeaders headers = new HttpHeaders();
headers.set("Connection", "close");
This is only for the "but we've been asked if we could try using separate TCP sessions for the acquisition of the token and then the REST call." part of your question. It will not help resolve your 404 (that does seem to be an LB issue).
UPDATE: Since you're using OAuth2RestTemplate, create a ClientHttpRequestInterceptor which injects the header.
public class ConnectionCloseInterceptor implements ClientHttpRequestInterceptor {
public ClientHttpResponse intercept(HttpRequest request, byte[] body, ClientHttpRequestExecution execution) throws IOException {
HttpHeaders headers = request.getHeaders();
headers.add("Connection", "close");
return execution.execute(request, body);
}
}
Use it in your rest template (OAuth2RestTemplate extends RestTemplate so below applies to both) like so (when you create the rest template bean):
List<ClientHttpRequestInterceptor> currentInterceptors = new ArrayList<>(restTemplate.getInterceptors()); //Don't want to lose the other interceptors!
currentInterceptors.add(new ConnectionCloseInterceptor()); //Add ours
restTemplate.setInterceptors(currentInterceptors);

Apache HttpClient return cached body on conditional request

I'm using CachingHttpClient implementation of HttpClient from Apache. And having the following scenario:
I made a request for a resource that returned a response with a header:
Cache-Control:max-age=5.
So CachingHttpClient caches the response.
I following I'm making a conditional request for the same resource using If-Modified-Since. And I get a response with status code 304 No modified (btw it does not even checks the server). Without a response body. Which is fine, but I would like to access the cached body, since if its not updated I want to use that.
The question is:
Is there a convenient way to access the cached response from the first call?
(Using org.apache.httpcomponents:httpclient, org.apache.httpcomponents:httpclient-cache; version 4.5.2)
server side:
#RequestMapping("/number")
public int getNumber(HttpServletResponse response, HttpServletRequest request) {
log.info("Number gen called");
response.setHeader("Cache-Control", "max-age=" + 5);
return random.nextInt();
}
client side:
HttpGet httpget = new HttpGet("http://localhost:8080/number");
httpget.setHeader("If-Modified-Since", java.time.format.DateTimeFormatter.RFC_1123_DATE_TIME.
format(ZonedDateTime.now(ZoneId.of("GMT")).minusSeconds(1)));
HttpResponse resp = httpClient.execute(httpget);
log.info("code: " + resp.getStatusLine().getStatusCode());
// here fails because no body on 2. call
String responseString = new BasicResponseHandler().handleResponse(resp);
Http Client Init
#Bean
public HttpClient httpClient() {
return CachingHttpClients.createMemoryBound();
}
Ok so if I configure the cache explicitly, I can use that cache object directly too, so this was a typical RTFM problem, my bad.
#Bean
public HttpCacheStorage httpCacheStorage() {
CacheConfig cacheConfig = CacheConfig.custom()
.setMaxCacheEntries(1000)
.setMaxObjectSize(8192)
.build();
HttpCacheStorage cacheStorage = new BasicHttpCacheStorage(cacheConfig);
return cacheStorage;
}

Resources