Handling empty request body (protobuf3 encoded) - spring

I have a Spring Boot application running. Requests/responses are sent protobuf (Protobuf3) encoded.
My (simplified) REST controller:
#RestController
public class ServiceController {
#RequestMapping(value = "/foo/{userId}", method = RequestMethod.POST)
public void doStuff(#PathVariable int userId, #RequestBody(required = false) Stuff.Request pbRequest) {
// Do stuff
}
}
My (simplified) protobuf3 schema:
syntax = "proto3";
message Request {
int32 data = 1;
}
My configuration to have content negotiation available:
#Configuration
public class ProtobufConfig {
#Bean
ProtobufHttpMessageConverter protobufHttpMessageConverter() {
return new ProtobufHttpMessageConverter();
}
}
Everything is working like a charm as long as the request body has some bytes set. BUT protobuf does not write any bytes if just default values are sent. As soon I have a request message which contains data = 0 the generated bytes are just empty. On the app side the request body is null and won't be converted to a protobuf message (it even throws an exception if request body is set to required = true). The HTTP input message isn't processed by the ProtobufHttpMessageConverter at all. Is there a way to handle that?

I found a way of handling it. But it uses reflection which is really something I don't want to have:
#ControllerAdvice
public class RequestBodyAdviceChain implements RequestBodyAdvice {
#Override
public boolean supports(MethodParameter methodParameter, Type type,
Class< ? extends HttpMessageConverter< ? >> aClass) {
return true;
}
#Override
public Object handleEmptyBody(Object body, HttpInputMessage httpInputMessage, MethodParameter methodParameter,
Type type, Class< ? extends HttpMessageConverter< ? >> aClass) {
try {
Class<?> cls = Class.forName(type.getTypeName());
Method m = cls.getMethod("getDefaultInstance");
return m.invoke(null);
} catch (Exception e) {
e.printStackTrace();
}
return body;
}
#Override
public HttpInputMessage beforeBodyRead(HttpInputMessage httpInputMessage, MethodParameter methodParameter,
Type type, Class< ? extends HttpMessageConverter< ? >> aClass) throws IOException {
return httpInputMessage;
}
#Override
public Object afterBodyRead(Object body, HttpInputMessage httpInputMessage, MethodParameter methodParameter, Type type,
Class< ? extends HttpMessageConverter< ? >> aClass) {
return body;
}
}
So in the case of an empty body I create a default instance of the protobuf message object.

Related

Spring Boot escape characters at Request Body for XSS protection

I'm trying to secure my spring boot application using a XSSFilter like this:
public class XSSFilter implements Filter {
#Override
public void init(FilterConfig filterConfig) throws ServletException { }
#Override
public void destroy() { }
#Override
public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {
chain.doFilter(new XSSRequestWrapper((HttpServletRequest) request), response);
}
}
And the wrapper:
public class XSSRequestWrapper extends HttpServletRequestWrapper {
public XSSRequestWrapper(HttpServletRequest servletRequest) {
super(servletRequest);
}
#Override
public String[] getParameterValues(String parameter) {
String[] values = super.getParameterValues(parameter);
if (values == null) {
return null;
}
int count = values.length;
String[] encodedValues = new String[count];
for (int i = 0; i < count; i++) {
encodedValues[i] = replaceXSSCharacters((values[i]));
}
return encodedValues;
}
private String replaceXSSCharacters(String value) {
if (value == null) {
return null;
}
return value
.replace("&","&")
.replace("<", "<")
.replace(">",">")
.replace("\"",""")
.replace("'","'");
}
#Override
public String getParameter(String parameter) {
return replaceXSSCharacters(super.getParameter(parameter));
}
#Override
public String getHeader(String name) {
return replaceXSSCharacters(super.getHeader(name));
}
}
The problem is, that only secures the Request parameters and Headers, not the Request body, and sometimes my Controller receive data using #RequestBody.
So, if i submit to my controller a json like this:
{"name":"<script>alert('hello!')</script>"}
The html chars at the name property doesn't get escaped like i need. How can i escape the RequestBody?
EDIT:
This is different from the "duplicated" question. My question is very Specific. How to escape characters on Request Body.
To remove XSS characters you just override AbstractJackson2HttpMessageConverter - this converter has responsibility to read request.inputStream to RequestBody object
#Component
public class XSSRequestBodyConverter extends AbstractJackson2HttpMessageConverter {
public XSSRequestBodyConverter(ObjectMapper objectMapper) {
super(objectMapper, MediaType.APPLICATION_JSON, new MediaType("application", "*+json"));
}
#Override
public Object read(Type type, Class<?> contextClass, HttpInputMessage inputMessage)
throws IOException, HttpMessageNotReadableException {
Object requestBody = super.read(type, contextClass, inputMessage);
//Remove xss from requestBody here
String requestInStr = objectMapper.writeValueAsString(requestBody);
return objectMapper.readValue(replaceXSSCharacters(requestInStr), Object.class);
}
}
I resolved with a custom class:
#Configuration
public class AntiXSSConfig {
#Autowired()
public void configeJackson(ObjectMapper mapper) {
mapper.getFactory().setCharacterEscapes(new HTMLCharacterEscapes());
mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
}
public static class HTMLCharacterEscapes extends JsonpCharacterEscapes {
#Override
public int[] getEscapeCodesForAscii() {
int[] asciiEscapes = CharacterEscapes.standardAsciiEscapesForJSON();
// and force escaping of a few others:
asciiEscapes['<'] = CharacterEscapes.ESCAPE_CUSTOM;
asciiEscapes['>'] = CharacterEscapes.ESCAPE_CUSTOM;
asciiEscapes['&'] = CharacterEscapes.ESCAPE_CUSTOM;
asciiEscapes['"'] = CharacterEscapes.ESCAPE_CUSTOM;
asciiEscapes['\''] = CharacterEscapes.ESCAPE_CUSTOM;
return asciiEscapes;
}
#Override
public SerializableString getEscapeSequence(int ch) {
switch (ch) {
case '&' : return new SerializedString("&");
case '<' : return new SerializedString("<");
case '>' : return new SerializedString(">");
case '\"' : return new SerializedString(""");
case '\'' : return new SerializedString("'");
default : return super.getEscapeSequence(ch);
}
}
}
}
It covers all the cases.
Have a local String field in XSSRequestWrapper which holds the cleaned-up body (probably not suitable for large bodies).
Populate this field in the constructor by reading request.getInputStream() and cleaning up the body the same way as parameters.
Override getInputStream and getReader methods of HttpServletRequestWrapper, and construct an InputStream (string -> byte array -> ByteArrayInputStream) and Reader (StringReader) from the String field and return them respectively. Maybe cache the constructed InputStream and Reader objects for better performance for when the methods are called repeatedly.
You may also be interested in cleaning up JSON when it is being deserialized into Java object.

How can I validate size of Pageable?

I'm using org.springframework.data.domain.Pageable with my #RestController.
How can I validate or limit the page size?
Without any validation, when clients call with size of 10000. The actual pageSize is 2000.
This could lead wrong signal for last page, I think.
How can I validate it and notify clients about it? Say with 400?
You can write a custom annotation to validate Pageable object
#Constraint(validatedBy = PageableValidator.class)
#Target( { ElementType.METHOD, ElementType.PARAMETER })
#Retention(RetentionPolicy.RUNTIME)
public #interface PageableConstraint {
String message() default "Invalid pagination";
Class<?>[] groups() default {};
Class<? extends Payload>[] payload() default {};
int maxPerPage() default 100;;
}
and Its implementation
public class PageableValidator implements
ConstraintValidator<PageableConstraint, Pageable> {
private int maxPerPage;
#Override
public void initialize(PageableConstraint constraintAnnotation) {
maxPerPage=constraintAnnotation.maxPerPage();
}
#Override
public boolean isValid(Pageable value, ConstraintValidatorContext context) {
return value.getPageSize()<=maxPerPage;
}
}
and you can use it over your controller like any other javax validation annotations.
#RestController
#RequestMapping("/api")
#Validated
public class EmployeeRestController {
#Autowired
private EmployeeService employeeService;
#GetMapping("/employees")
public List<Employee> getAllEmployees(#PageableConstraint(message = "Invalid page size",maxPerPage = 400) Pageable pageable) {
return employeeService.getAllEmployees();
}
}
Try using Custom Annotations to validate the Pageable object. If it is not working, try the argument resolver as shown below.
You can create Argument Resolver in your Spring Configuration that is extending WebMvcConfigurerAdapter
#Override
public void addArgumentResolvers(List<HandlerMethodArgumentResolver> argumentResolvers) {
PageableHandlerMethodArgumentResolver resolver = new PageableHandlerMethodArgumentResolver() {
#Override
public Pageable resolveArgument(MethodParameter methodParameter, ModelAndViewContainer mavContainer,
NativeWebRequest webRequest, WebDataBinderFactory binderFactory) {
Pageable p = super.resolveArgument(methodParameter, mavContainer, webRequest, binderFactory);
if (webRequest.getParameter("per_page") != null) {
int pageSize = Integer.parseInt(webRequest.getParameter("per_page"));
if (pageSize < 1 || pageSize > 2000) {
message = "Invalid page size";
}
}
if (message != null) {
Set<CustomConstraintViolation> constraintViolations = new HashSet<>();
constraintViolations.add(new CustomConstraintViolationImpl(message));
throw new ConstraintViolationException(constraintViolations);
}
return new PageRequest(p.getPageNumber(), p.getPageSize());
}
};
resolver.setMaxPageSize(2000);
argumentResolvers.add(resolver);
super.addArgumentResolvers(argumentResolvers);
}
This resolver will make sure the max page size is 2000 for every request to your controller.
You need to throw a ConstraintViolationException when the size is more than 2000. For that you need to create a custom ConstraintViolation interface and implement it
public CustomConstraintViolation implements ConstraintViolation<CustomConstraintViolation> {}
public CustomConstraintViolationImpl implements CustomConstraintViolation {
...
}
You can simply use configuration
spring.data.web.pageable.max-page-size=400
This will not lead to any error if one tries to get 401 records. Only 400 will be returned.
More on stuff that can be configured via spring properties could be found here:
https://docs.spring.io/spring-boot/docs/current/reference/html/appendix-application-properties.html

calling request.setattribute in Spring validator interface

How to access request.setattribute inside spring custom validator class. i need to set these values in jsp side I am trying something like below
#Component
public class ProductSearchValidator implements Validator {
#Override
public boolean supports(Class<?> clazz) {
return Product.class.isAssignableFrom(clazz);
}
#Override
public void validate(Object target, Errors errors) {
Product product = (Product) target;
String name = product.getName();
String cod="Validated";
request.setAttribute("isVal",cod);
}
}
You can access request this way
RequestAttributes attrs = RequestContextHolder.getRequestAttributes();
if (attrs instanceof ServletRequestAttributes) {
HttpServletRequest request = ((ServletRequestAttributes)attrs).getRequest();
}

How to get the #RequestBody in an #ExceptionHandler (Spring REST)

I am using Spring Boot 1.4.1 which includes spring-web-4.3.3. I have a class annotated with #ControllerAdvice and methods annotated with #ExceptionHandler to handle exceptions thrown by the service code. When handling these exceptions, I would like to log the #RequestBody that was part of the request for PUT and POST operations so I can see the request body that caused the problem which in my case is crucial for diagnosis.
Per Spring Docs the method signature for #ExceptionHandler methods can include various things including the HttpServletRequest. The request body can normally be obtained from here via getInputStream() or getReader(), but if my controller methods parse the request body like "#RequestBody Foo fooBody" as all of mine do, the HttpServletRequest's input stream or reader is already closed by the time my exception handler method is called. Essentially the request body has already been read by Spring, similar to the issue described here. It is a common problem working with servlets that the request body can only be read once.
Unfortunately #RequestBody is not one of the options available for the exception handler method, if it were then I could use that.
I can add an InputStream to the exception handler method, but that ends up being the same thing as the HttpServletRequest's InputStream and so has the same issue.
I also tried getting the current request with ((ServletRequestAttributes) RequestContextHolder.currentRequestAttributes()).getRequest() which is another trick for getting the current request, but this ends up being the same HttpServletRequest that Spring passes into the exception handler method and so has the same problem.
I have read about a few solutions like this and this that involve inserting a custom request wrapper in the filter chain that will read the contents of the request and cache them so they can be read more than once. I don't like this solution because I don't want to interrupt the entire filter/request/response chain (and potentially introduce performance or stability problems) just to implement logging, and if I have any large requests such as uploaded documents (which I do), I don't want to cache that in memory. Besides, Spring probably has the #RequestBody cached somewhere already if I could only find it.
Incidentally many solutions recommend using the ContentCachingRequestWrapper Spring class but in my experience this does not work. Aside from not being documented, looking at its source code it looks like it only caches the parameters, but not the request body. Trying to get the request body from this class always results in an empty string.
So I am looking for any other options that I may have missed. thanks for reading.
Accepted answer creates a new POJO to pass things around, but the same behaviour can be achieved without creating additional objects by reusing the http request.
Example code for Controller mapping:
public ResponseEntity savePerson(#RequestBody Person person, WebRequest webRequest) {
webRequest.setAttribute("person", person, RequestAttributes.SCOPE_REQUEST);
And later in the ExceptionHandler class / method you can use:
#ExceptionHandler(Exception.class)
public ResponseEntity exceptionHandling(WebRequest request,Exception thrown) {
Person person = (Person) request.getAttribute("person", RequestAttributes.SCOPE_REQUEST);
You can reference the request body object to a request-scoped bean. And then inject that request-scoped bean in your exception handler to retrieve the request body (or other request-context beans that you wish to reference).
// #Component
// #Scope("request")
#ManagedBean
#RequestScope
public class RequestContext {
// fields, getters, and setters for request-scoped beans
}
#RestController
#RequestMapping("/api/v1/persons")
public class PersonController {
#Inject
private RequestContext requestContext;
#Inject
private PersonService personService;
#PostMapping
public Person savePerson(#RequestBody Person person) throws PersonServiceException {
requestContext.setRequestBody(person);
return personService.save(person);
}
}
#ControllerAdvice
public class ExceptionMapper {
#Inject
private RequestContext requestContext;
#ExceptionHandler(PersonServiceException.class)
protected ResponseEntity<?> onPersonServiceException(PersonServiceException exception) {
Object requestBody = requestContext.getRequestBody();
// ...
return responseEntity;
}
}
You should be able to get the content of the request body by using the RequestBodyAdvice interface. If you implement this on a class annotated with #ControllerAdvice it should be picked up automatically.
To get other request information like the HTTP method and query params I'm using an interceptor. I'm capturing all this request info for error reporting in a ThreadLocal variable which I clear on the afterCompletion hook in that same interceptor.
The class below implements this and can be used in your ExceptionHandler to get all request information:
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.core.MethodParameter;
import org.springframework.http.HttpInputMessage;
import org.springframework.http.converter.HttpMessageConverter;
import org.springframework.web.bind.annotation.ControllerAdvice;
import org.springframework.web.servlet.handler.HandlerInterceptorAdapter;
import org.springframework.web.servlet.mvc.method.annotation.RequestBodyAdvice;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import java.lang.reflect.Type;
import java.util.HashMap;
import java.util.Map;
#ControllerAdvice
public class RequestInfo extends HandlerInterceptorAdapter implements RequestBodyAdvice {
private static final Logger logger = LoggerFactory.getLogger(RequestInfo.class);
private static final ThreadLocal<RequestInfo> requestInfoThreadLocal = new ThreadLocal<>();
private String method;
private String body;
private String queryString;
private String ip;
private String user;
private String referrer;
private String url;
public static RequestInfo get() {
RequestInfo requestInfo = requestInfoThreadLocal.get();
if (requestInfo == null) {
requestInfo = new RequestInfo();
requestInfoThreadLocal.set(requestInfo);
}
return requestInfo;
}
public Map<String,String> asMap() {
Map<String,String> map = new HashMap<>();
map.put("method", this.method);
map.put("url", this.url);
map.put("queryParams", this.queryString);
map.put("body", this.body);
map.put("ip", this.ip);
map.put("referrer", this.referrer);
map.put("user", this.user);
return map;
}
private void setInfoFromRequest(HttpServletRequest request) {
this.method = request.getMethod();
this.queryString = request.getQueryString();
this.ip = request.getRemoteAddr();
this.referrer = request.getRemoteHost();
this.url = request.getRequestURI();
if (request.getUserPrincipal() != null) {
this.user = request.getUserPrincipal().getName();
}
}
public void setBody(String body) {
this.body = body;
}
private static void setInfoFrom(HttpServletRequest request) {
RequestInfo requestInfo = requestInfoThreadLocal.get();
if (requestInfo == null) {
requestInfo = new RequestInfo();
}
requestInfo.setInfoFromRequest(request);
requestInfoThreadLocal.set(requestInfo);
}
private static void clear() {
requestInfoThreadLocal.remove();
}
private static void setBodyInThreadLocal(String body) {
RequestInfo requestInfo = get();
requestInfo.setBody(body);
setRequestInfo(requestInfo);
}
private static void setRequestInfo(RequestInfo requestInfo) {
requestInfoThreadLocal.set(requestInfo);
}
// Implementation of HandlerInterceptorAdapter to capture the request info (except body) and be able to add it to the report in case of an error
#Override
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler) {
RequestInfo.setInfoFrom(request);
return true;
}
#Override
public void afterCompletion(HttpServletRequest request, HttpServletResponse response, Object handler, Exception exception) {
RequestInfo.clear();
}
// Implementation of RequestBodyAdvice to capture the request body and be able to add it to the report in case of an error
#Override
public boolean supports(MethodParameter methodParameter, Type targetType, Class<? extends HttpMessageConverter<?>> converterType) {
return true;
}
#Override
public HttpInputMessage beforeBodyRead(HttpInputMessage inputMessage, MethodParameter parameter, Type targetType, Class<? extends HttpMessageConverter<?>> converterType) {
return inputMessage;
}
#Override
public Object afterBodyRead(Object body, HttpInputMessage inputMessage, MethodParameter parameter, Type targetType, Class<? extends HttpMessageConverter<?>> converterType) {
RequestInfo.setBodyInThreadLocal(body.toString());
return body;
}
#Override
public Object handleEmptyBody(Object body, HttpInputMessage inputMessage, MethodParameter parameter, Type targetType, Class<? extends HttpMessageConverter<?>> converterType) {
return body;
}
}
Just an enhancement to quintencls answer
I got request body and can use it anywhere inside exception handler class.
#ControllerAdvice
public class CustomErrorHandler extends ResponseEntityExceptionHandler implements RequestBodyAdvice {
...
private Object reqBody;
...
#ExceptionHandler(NoSuchElementException.class)
public ResponseEntity<Object> handleNoSuchElementException(final NoSuchElementException ex,
final WebRequest request) {
System.out.println("===================================" + reqBody);
return handleNotFoundException(ex, request);
}
...
#Override
public boolean supports(MethodParameter methodParameter, Type targetType,
Class<? extends HttpMessageConverter<?>> converterType) {
return true;
}
#Override
public HttpInputMessage beforeBodyRead(HttpInputMessage inputMessage, MethodParameter parameter, Type targetType,
Class<? extends HttpMessageConverter<?>> converterType) throws IOException {
return inputMessage;
}
#Override
public Object afterBodyRead(Object body, HttpInputMessage inputMessage, MethodParameter parameter, Type targetType,
Class<? extends HttpMessageConverter<?>> converterType) {
// capture request body here to use in our controller advice class
this.reqBody = body;
return body;
}
#Override
public Object handleEmptyBody(Object body, HttpInputMessage inputMessage, MethodParameter parameter,
Type targetType, Class<? extends HttpMessageConverter<?>> converterType) {
return body;
}
}
Here is a solution in Kotlin syntax that I used for some fields validation control.
I needed to enhance the default handleMethodArgumentNotValid(...) method from the #RestControllerAdvice, to systematically log a field that was embedded in that same request body object.
override fun handleMethodArgumentNotValid(e: MethodArgumentNotValidException, headers: HttpHeaders, status: HttpStatus, request: WebRequest): ResponseEntity<Any> {
val error = e.bindingResult.fieldErrors.first()
val requestBody = try {
val field = error.javaClass.getDeclaredField("violation").apply { trySetAccessible() }
((field[error] as ConstraintViolationImpl<Any>).rootBean as MyRequestBodyObject)
} catch (ex : Exception) {
//do some failsafe here
}
}
See here: https://stackoverflow.com/a/61813076/1036433 - for a clean way to have access to the HttpServerRequest.

How to make json response Unified format in spring?

I want all json response is
{
"status":"ok"
"data":"..."
}
I only care #ResponseBody function return value;Don't need wrap any object to do it;
example:
#ResponseBody
public String test(){
return "Hello,World"
}
I want get
{
"status":"ok"
"data":"Hello,World"
}
ResponseBodyAdvicecan do it.you can implement the interface to handle ReqeustBody. Here is the api.
In the documents.
Allows customizing the response after the execution of an #ResponseBody or a ResponseEntity controller method but before the body is written with an HttpMessageConverter.
Implementations may be registered directly with RequestMappingHandlerAdapter and ExceptionHandlerExceptionResolver or more likely annotated with #ControllerAdvice in which case they will be auto-detected by both.
here is a simple code.
#ControllerAdvice
public customerResponseBody implements ResponseBodyAdvice{
#Override
public boolean supports (MethodParameter returnType, Class<? extends HttpMessageConverter<?>> converterType){
return true;
}
#Override
public Object beforeBodyWrite (Object body, MethodParameter returnType, MediaType selectedContentType, Class<? extends HttpMessageConverter<?>> selectedConverterType, ServerHttpRequest request, ServerHttpResponse response){
body = new ResponseTemplate<Object>("001",body);
return body;
}
}
You must return a Object instead of String for example :
public class CustomResponse {
private String status;
private String data;
// Getters & Setters
}
#ResponseBody
public CustomResponse test(){
CustomResponse response = new CustomResponse();
response.setStatus("OK");
response.setData("Hello,World");
return response;
}
I thought it is not a good solution but you can format string
#RequestMapping(value="testData")
public #ResponseBody String testData(){
String sdata="ok";
String value ="Hello,World";
return "{\"status\" :\""+sdata+"\",\"data \":\""+value+"\"}";
}
I found this solve problem.
rewrite default RequestResponseBodyMethodProcessor can wrap any object to return value

Resources