Spring Boot allow Square brackets for nested objects in Request Parameters - spring

I'm using Spring Boot 2.6.7 and org.springdoc:springdoc-openapi-ui:1.6.7 to run swagger ui with OpenApi 3 definition along with my backend.
My Setup
#GetMapping(value = "/hello", produces = MediaType.APPLICATION_JSON_VALUE)
public String getHello(HelloInput input) {
return "";
}
public class HelloInput {
public NestedInput nested;
// getter setter omitted
}
public class NestedInput {
public boolean a = true;
// getter setter omitted
}
This produces the following request when trying it out in swagger ui:
curl -X 'GET' \
'http://localhost:8080/api/hello?nested[a]=true' \
-H 'accept: application/json'
The Problem
However, when I execute this call I get the following Exception:
org.springframework.beans.InvalidPropertyException: Invalid property 'nested[a]' of bean class [com.proj.App.rest.explore.HelloInput]: Property referenced in indexed property path 'nested[a]' is neither an array nor a List nor a Map; returned value was [com.proj.App.rest.explore.NestedInput#68d1c7c4]
What do I need
The square braces notation causes this issue. The same call with dot notation does not cause exceptions:
curl -X 'GET' \
'http://localhost:8080/api/hello?nested.a.=true' \
-H 'accept: application/json'
Is there a way to either:
A: Configure Spring Boot in a way that it accepts the square brackets notation also for object properties and not just maps, lists etc.
OR
B: Configure Swagger UI in a way that it uses the dot notation when constructing the calls?
What did I already try
I already did a lot of research, but could not find other people with this problem.
I found a feature request for Spring to support squared brackets, but it was rejected: https://github.com/spring-projects/spring-framework/issues/20052
I found basically the same question, but the answers seem to be outdated, since the RelaxedDataBinder does not seem to be a part of Spring Boot 2.6.7 anymore: Customize Spring #RequestParam Deserialization for Maps and/or Nested Objects
Other than that, no one else seems to have this problem. Am I completely misunderstanding how Spring Boot handles Request Parameters and this problem occurs because I'm breaking some convention on how to handle GET Requests with many parameters?

I have the same problem and can't find any good solution. I have found only one more hack. So you can reformat request param json with dot-separated fields. It is a little bit inconvenient but works for me.
For example, by default for the controller in question, the request param input form will have json that will look like this:
{
"nested": {
"a": true
}
}
But you can use
{
"nested.a": true
}
to get what you want without any even more hacky changes(as I see this) to swagger-ui or spring components.

CAUTION: Hacky workaround for making swagger ui use dots instead of squared braces (probably breaks in a lot of scenarios)
Swagger Ui offers to define a requestInterceptor, that takes outbound requests and offers the user to change them.
When using Springdoc, swagger ui downloads a swagger-initializer.js for configuration of swagger ui.
This file is generated by an instance of a SwaggerIndexPageTransformer. The default instance already creates a requestInterceptor for CORS requests. I took that code and adapted it to change all requests with the squared braces notation to the dot notation.
For reference see AbstractSwaggerIndexTransformer#addCRSF
Workaround
First define a Configuration that offers a SwaggerIndexTransformer Bean:
#Configuration
public class OpenApiConfig {
#Bean
public SwaggerIndexTransformer swaggerIndexTransformer(
SwaggerUiConfigProperties a,
SwaggerUiOAuthProperties b,
SwaggerUiConfigParameters c,
SwaggerWelcomeCommon d) {
return new CustomSwaggerIndexTransformer(a, b, c, d);
}
}
Next implement the CustomSwaggerIndexTransformer, that now edits the swagger-initializer.js to add a requestInterceptor function, that replaces every [ with a . and removes all ].
public class CustomSwaggerIndexTransformer extends SwaggerIndexPageTransformer {
private static final String PRESETS = "presets: [";
...
#Override
protected String defaultTransformations(InputStream inputStream) throws IOException {
String html = super.defaultTransformations(inputStream);
html = addDottedRequests(html);
return html;
}
protected String addDottedRequests(String html) {
String interceptorFn = """
requestInterceptor: (request) => {
request.url = request.url.replaceAll("[", ".").replaceAll("]", "");
return request;
},
""" + PRESETS;
return html.replace(PRESETS, interceptorFn);
}
}
Limitations
This probably breaks working behavior when swagger ui uses squared braces for Maps, Arrays and Lists
This does not work when CORS is enabled for springdoc, since it also defines a requestInterceptor, in that case the javascript should somehow be incorporated into the CORS requestInterceptor function.

Related

Mirror #RequestPart behavior in WebFlux functional router definitions with different content types

Problem
We're developing a Spring Boot service to upload data to different back end databases. The idea is that, in one multipart/form-data request a user will send a "model" (basically a file) and "modelMetadata" (which is JSON that defines an object of the same name in our code).
We got the below to work in the WebFlux annotated controller syntax, when the user sends the "modelMetadata" in the multipart form with the content-type of "application/json":
#PostMapping(consumes = [MediaType.MULTIPART_FORM_DATA_VALUE], produces = [MediaType.APPLICATION_JSON_VALUE])
fun saveModel(#RequestPart("modelMetadata") monoModelMetadata: Mono<ModelMetadata>,
#RequestPart("model") monoModel: Mono<FilePart>,
#RequestHeader headers: HttpHeaders) : Mono<ResponseEntity<ModelMetadata>> {
return modelService.saveModel(monoModelMetadata, monoModel, headers)
}
But we can't seem to figure out how to do the same thing in Webflux's functional router definition. Below are the relevant code snippets we have:
#Bean
fun modelRouter() = router {
accept(MediaType.MULTIPART_FORM_DATA).nest {
POST(ROOT, handler::saveModel)
}
}
fun saveModel(r: ServerRequest): Mono<ServerResponse> {
val headers = r.headers().asHttpHeaders()
val monoModelPart = r.multipartData().map { multiValueMap ->
it["model"] // What do we do with this List<Part!> to get a Mono<FilePart>
it["modelMetadata"] // What do we do with this List<Part!> to get a Mono<ModelMetadata>
}
From everything we've read, we should be able to replicate the same functionality found in the annotation controller syntax with the router functional syntax, but this particular aspect doesn't seem to be well documented. Our goal was to move over to use the new functional router syntax since this is a new application we're developing and there are some nice forward thinking features/benefits as described here.
What we've tried
Googling to the ends of the Earth for a relevant example
this is a similar question, but hasn't gained any traction and doesn't relate to our need to create an object from one piece of the multipart request data
this may be close to what we need for uploading the file component of our multipart request data, but doesn't handle the object creation from JSON
Tried looking at the #RequestPart annotation code to see how things are done on that side, there's a nice comment that seems to hint at how they are converting the parts to objects, but we weren't able to figure out where that code lives or any relevant example of how to use an HttpMessageConverter on the ``
the content of the part is passed through an {#link HttpMessageConverter} taking into consideration the 'Content-Type' header of the request part.
Any and all help would be appreciated! Even just some links for us to better understand Part/FilePart types and there role in multipart requests would be helpful!
I was able to come up with a solution to this issue using an autowired ObjectMapper. From the below solution I could turn the modelMetadata and modelPart into Monos to mirror the #RequestPart return types, but that seems ridiculous.
I was also able to solve this by creating a MappingJackson2HttpMessageConverter and turning the metadataDataBuffer into a MappingJacksonInputMessage, but this solution seemed better for our needs.
fun saveModel(r: ServerRequest): Mono<ServerResponse> {
val headers = r.headers().asHttpHeaders()
return r.multipartData().flatMap {
// We're only expecting one Part of each to come through...assuming we understand what these Parts are
if (it.getOrDefault("modelMetadata", listOf()).size == 1 && it.getOrDefault("model", listOf()).size == 1) {
val modelMetadataPart = it["modelMetadata"]!![0]
val modelPart = it["model"]!![0] as FilePart
modelMetadataPart
.content()
.map { metadataDataBuffer ->
// TODO: Only do this if the content is JSON?
objectMapper.readValue(metadataDataBuffer.asInputStream(), ModelMetadata::class.java)
}
.next() // We're only expecting one object to be serialized from the buffer
.flatMap { modelMetadata ->
// Function was updated to work without needing the Mono's of each type
// since we're mapping here
modelService.saveModel(modelMetadata, modelPart, headers)
}
}
else {
// Send bad request response message
}
}
Although this solution works, I feel like it's not as elegant as the one alluded to in the #RequestPart annotation comments. Thus I will accept this as the solution for now, but if someone has a better solution please let us know and I will accept it!

AWS Lex receives Invalid Response from lambda function - Can not construct instance of IntentResponse

Using Java8 in eclipse AWS SDK, I've created and uploaded a lambda function that is hooked in upon fulfillment of my lex intent.
Lambda has not problem receiving JSON request and parsing.
Then, I format a simple "Close" dialogAction response and send back to lex and receive the following error from the Test Bot page in the lex console:
An error has occurred: Received invalid response from Lambda:
Can not construct instance of IntentResponse:
no String-argument constructor/factory method to deserialize
from String value
('{"dialogAction
{"type":"Close","fulfillmentState":"Fulfilled","message":
{"contentType":"PlainText","content":"Thanks I got your info"}}}')
at [Source: "{\"dialogAction\":
{\"type\":\"Close\",\"fulfillmentState\":\"Fulfilled\",\"message\":
{\"contentType\":\"PlainText\",\"content\":\"Thanks I got your
info\"}}}";line: 1, column: 1]
It seems to have a problem right away with the format (line 1, column 1), but my JSON string looks ok to me. Before returning the output string in the handleRequest java function, I am writing the it to the Cloudwatch log and it writes as follows:
{
"dialogAction": {
"type": "Close",
"fulfillmentState": "Fulfilled",
"message": {
"contentType": "PlainText",
"content": "Thanks I got your info"
}
}
}
Things I've tried:
Removing the message element as it's not required
Adding in non-required properties like sessionAttributes,
responseCard, etc
removing the double quotes
replacing double quotes with single quotes
hardcoding json from sample response format message in documentation
Is there something hidden at the http headers level or is java8 doing something to the JSON that is not visible?
Not sure if this is because I'm using Java8 or not, but a return value of "String" from the RequestHandler class handleRequest method will not work.
Yes, String is an object, but the constructors on the Lex side are expecting an "Object". I was converting my lex response POJO to a String before returning it in the handleRequest method. That was my mistake.
I fixed it by changing the return type of the handleRequest method to be "Object" instead of "String".
public Object handleRequest(Object input, Context context)
instead of
public String handleRequest(Object input, Context context)
You also have to implement the
public class LambdaFunctionHandler implements RequestHandler<Object, Object>
not
public class LambdaFunctionHandler implements RequestHandler<Object, String>
This solved my issue.
In my case I was facing exactly the same issue and was able to fix it by creating specific response POJO type and using this POJO as the return type for 'handleRequest' method. E.g. BotResponse.java as follow:
public class BotResponse implements Serializable{
private static final long serialVersionUID = 1L;
public DialogAction dialogAction = new DialogAction();
public DialogAction getDialogAction() {
return dialogAction;
}
public void setDialogAction(DialogAction dialogAction) {
this.dialogAction = dialogAction;
}
}
Note, I have also added the 'implements Serializable' just to be on safer side. Probably it is an overkill.
Not sure why but for me returning a well formatted JSON String object did not worked even after changing the return type of 'handleRequest' method to 'Object'.
I know this is an old question however thought this might help some else
#Mattbob Solution dint fix my issue, However he is in the right path. Best approach is to use a Response object, a custom response object and make the lambda return the custom response object. So i went to the Documentation and created a custom object that looks Response format here
http://docs.aws.amazon.com/lex/latest/dg/lambda-input-response-format.html
At the time of answering question i couldnt find an object in SDK that matched the response Object so i had to recreate but if some one knows please comment below
Class xxxxx implements RequestHandler<Object, AccountResponse> {
#Override
public AccountResponse handleRequest(Object input, Context context) {
}
}
Lambda will look somewhat like this and just populate and return the object to match response structure and error goes away. Hope this helps.
Whenever we are returning the object to the bot from the backend make sure we need to pass content type along with content. But here we are passing wrong. So wE need to pass as like below. It is in Node.js
let message = {
contentType: "PlainText",
content: 'Testing bot'
};

How to force URIBuilder.path(...) to encode parameters like "%AD"? This method doesn't always encode parameters with percentage, correctly

How to force URIBuilder.path(...) to encode parameters like "%AD"?
The methods path, replacePath and segment of URIBuilder do not always encode parameters with percentage, correctly.
When a parameter contains the character "%" followed by two characters that together form an URL-encoded character, the "%" is not encoded as "%25".
For example
URI uri = UriBuilder.fromUri("https://dummy.com").queryParam("param", "%AD");
String test = uri.build().toString();
"test" is "https://dummy.com?param=%AD"
But it should be "https://dummy.com?param=%25AD" (with the character "%" encoded as "%25")
The method UriBuilderImpl.queryParam(...) behaves like this when the two characters following the "%" are hexadecimal. I.e, the method "com.sun.jersey.api.uri.UriComponent.isHexCharacter(char)" returns true for the characters following the "%".
I think the behavior of UriBuilderImpl is correct because I guess it tries to not encode parameters that already are encoded. But in my scenario, I will never try to create URLs with parameters that already encoded.
What should I do?
My Web application uses Jersey and in many places I build URIs using the class UriBuilder or invoke the method getBaseUriBuilder of UriInfo objects.
I can replace "%" with "%25", every time I invoke the methods queryParam, replaceQueryParam or segment. But I am looking for a less cumbersome solution.
How can I make Jersey to return my own implementation of UriBuilder?
I thought of creating a class that extends UriBuilderImpl that overrides these methods and that perform this replacing before invoking super.queryParam(...) or whatever.
Is there any way of making Jersey to return my own UriBuilder instead of UriBuilderImpl, when invoking UriBuilder.fromURL(...), UriInfo.getBaseUriBuilder(...), etc?
Looking at the method RuntimeDelegate, I thought of extending RuntimeDelegateImpl. My implementation would override the method createUriBuilder(...), which would return my own UriBuilder, instead of UriBuilderImpl.
Then, I would add the file META-INF/services/javax.ws.rs.ext.RuntimeDelegate and in it, a the full class name of my RuntimeDelegateImpl.
The problem is that the jersey-bundle.jar already contains a META-INF/services/javax.ws.rs.ext.RuntimeDelegate that points to com.sun.jersey.server.impl.provider.RuntimeDelegateImpl, so the container loads that file instead of my javax.ws.rs.ext.RuntimeDelegate. Therefore, it does not load my RuntimeDelegateimplementation.
Is it possible to provide my own implementation of RuntimeDelegate?
Should I take a different approach?
UriBuilder
This is possible with help of UriComponent from Jersey or URLEncoder directly from Java:
UriBuilder.fromUri("https://dummy.com")
.queryParam("param",
UriComponent.encode("%AD",
UriComponent.Type.QUERY_PARAM_SPACE_ENCODED))
.build();
Which result in:
https://dummy.com/?param=%25AD
Or:
UriBuilder.fromUri("https://dummy.com")
.queryParam("param", URLEncoder.encode("%AD", "UTF-8"))
.build()
Will result in:
https://dummy.com/?param=%25AD
For a more complex examples (i.e. encoding JSON in query param) this approach is also possible. Let's assume you have a JSON like {"Entity":{"foo":"foo","bar":"bar"}}. When encoded using UriComponent the result for query param would look like:
https://dummy.com/?param=%7B%22Entity%22:%7B%22foo%22:%22foo%22,%22bar%22:%22bar%22%7D%7D
JSON like this could be even injected via #QueryParam into resource field / method param (see JSON in Query Params or How to Inject Custom Java Types via JAX-RS Parameter Annotations).
Which Jersey version do you use? In the tags you mention Jersey 2 but in the RuntimeDelegate section you're using Jersey 1 stuff.
See if the following examples help. The thread linked below has an extensive discussion on the available functions and their differing outputs.
The following:
UriBuilder.fromUri("http://localhost:8080").queryParam("name", "{value}").build("%20");
UriBuilder.fromUri("http://localhost:8080").queryParam("name", "{value}").buildFromEncoded("%20");
UriBuilder.fromUri("http://localhost:8080").replaceQuery("name={value}).build("%20");
UriBuilder.fromUri("http://localhost:8080").replaceQuery("name={value}).buildFromEncoded("%20");
Will output:
http://localhost:8080?name=%2520
http://localhost:8080?name=%20
http://localhost:8080?name=%2520
http://localhost:8080?name=%20
via http://comments.gmane.org/gmane.comp.java.jsr311.user/71
Also, based on the Class UriBuilder documentation, the following example shows how to obtain what you're after.
URI templates are allowed in most components of a URI but their value
is restricted to a particular component. E.g.
UriBuilder.fromPath("{arg1}").build("foo#bar");
would result in encoding of the '#' such that the resulting URI is
"foo%23bar". To create a URI "foo#bar" use
UriBuilder.fromPath("{arg1}").fragment("{arg2}").build("foo", "bar")
instead. URI template names and delimiters are never encoded but their
values are encoded when a URI is built. Template parameter regular
expressions are ignored when building a URI, i.e. no validation is
performed.
It is possible to overwrite the default behavior in jersey manually at start up e.g. with a static helper that calls RuntimeDelegate.setInstance(yourRuntimeDelegateImpl).
So if you want to have an UriBuilder that encodes percents even if they look like they are part of an already encoded sequence, this would look like:
[...]
import javax.ws.rs.core.UriBuilder;
import javax.ws.rs.ext.RuntimeDelegate;
import com.sun.jersey.api.uri.UriBuilderImpl;
import com.sun.ws.rs.ext.RuntimeDelegateImpl;
// or for jersey2:
// import org.glassfish.jersey.uri.internal.JerseyUriBuilder;
// import org.glassfish.jersey.internal.RuntimeDelegateImpl;
public class SomeBaseClass {
[...]
// this is the lengthier custom implementation of UriBuilder
// replace this with your own according to your needs
public static class AlwaysPercentEncodingUriBuilder extends UriBuilderImpl {
#Override
public UriBuilder queryParam(String name, Object... values) {
Object[] encValues = new Object[values.length];
for (int i=0; i<values.length; i++) {
String value = values[i].toString(); // TODO: better null check here, like in base class
encValues[i] = percentEncode(value);
}
return super.queryParam(name, encValues);
}
private String percentEncode(String value) {
StringBuilder sb = null;
for (int i=0; i < value.length(); i++) {
char c = value.charAt(i);
// if this condition is is true, the base class will not encode the percent
if (c == '%'
&& i + 2 < value.length()
&& isHexCharacter(value.charAt(i + 1))
&& isHexCharacter(value.charAt(i + 2))) {
if (sb == null) {
sb = new StringBuilder(value.substring(0, i));
}
sb.append("%25");
} else {
if (sb != null) sb.append(c);
}
}
return (sb != null) ? sb.toString() : value;
}
// in jersey2 one can call public UriComponent.isHexCharacter
// but in jersey1 we need to provide this on our own
private static boolean isHexCharacter(char c) {
return ('0' <= c && c <= '9')
|| ('A' <=c && c <= 'F')
|| ('a' <=c && c <= 'f');
}
}
// here starts the code to hook up the implementation
public static class AlwaysPercentEncodingRuntimeDelegateImpl extends RuntimeDelegateImpl {
#Override
public UriBuilder createUriBuilder() {
return new AlwaysPercentEncodingUriBuilder();
}
}
static {
RuntimeDelegate myDelegate = new AlwaysPercentEncodingRuntimeDelegateImpl();
RuntimeDelegate.setInstance(myDelegate);
}
}
Caveat: Of course that way it is not very configurable, and if you do that in some library code that might be reused by others, this might cause some irritation.
For example I had the same problem as the OP when writing a rest client in a Confluence plugin, and ended up with the "manual encode every parameter" solution instead, as the plugins are loaded via OSGi and thus are simply not able to touch the RuntimeDelegateImpl (getting java.lang.ClassNotFoundException: com.sun.ws.rs.ext.RuntimeDelegateImpl at runtime instead).
(And just for the record, in jersey2 this looks very similar; especially the code to hook the custom RuntimeDelegateImpl is the same.)

use camel case serialization only for specific actions

I've used WebAPI for a while, and generally set it to use camel case json serialization, which is now rather common and well documented everywhere.
Recently however, working on a much larger project, I came across a more specific requirement: we need to use camel case json serialization, but because of backward compatibility issues with our client scripts, I only want it to happen for specific actions, to avoid breaking other parts of the (extremely large) website.
I figure one option is to have a custom content type, but that then requires client code to specify it.
Is there any other option?
Thanks!
Try this:
public class CamelCasingFilterAttribute : ActionFilterAttribute
{
private JsonMediaTypeFormatter _camelCasingFormatter = new JsonMediaTypeFormatter();
public CamelCasingFilterAttribute()
{
_camelCasingFormatter.SerializerSettings.ContractResolver = new CamelCasePropertyNamesContractResolver();
}
public override void OnActionExecuted(HttpActionExecutedContext actionExecutedContext)
{
ObjectContent content = actionExecutedContext.Response.Content as ObjectContent;
if (content != null)
{
if (content.Formatter is JsonMediaTypeFormatter)
{
actionExecutedContext.Response.Content = new ObjectContent(content.ObjectType, content.Value, _camelCasingFormatter);
}
}
}
}
Apply this [CamelCasingFilter] attribute to any action you want to camel-case. It will take any JSON response you were about to send back and convert it to use camel casing for the property names instead.

Jersey and Odata Key Path Param format

I have a RESTful api using Jersey right now, and am converting it to be OData standard compliant. There are a few things I have not converted yet, but will get there, and is not important at this moment. One of the things I need to convert that is important is the key path params. Odata has the standard of making the key wrapped in parenthesis. So in this example myapi.com/product(1) - is the OData call to get a product whose id is 1. Currently that is possible in my system with this myapi.com/product/1
When I add the parenthesis to the path parameter I get a 404 error. My class level path is #Path("/product") and my method level path is #Path("({id})"), and use to be #Path("/{id}"). I've tried adding the parenthesis as part of the variable planning to strip them off in the method, and I've tried formatting the id with some regex #Path("{id : regex stuff}"), and neither works.
If I make my method path parameter like this #Path"/({id})") - so the call is myapi.com/product/(1), it works fine. The parenthesis is not the issue obviously. It seems the Jersey splits the uri into chunks using the forward slashes for the routing, and sense there is no forward slash between the id an root resource name, then nothing is found. It makes sense.
Is there a way to change Jerseys method of matching uri strings with some regex or something? Has anyone used Jersey with Odata? I would rather not use odata4j just for the resolution to this issue, it seems like there should be a way to get this to work.
What I did:
Based on Pavel Bucek's answer I did implement a ContainrRequestFilter independently to the filter I use for security. In my case I didn't look to see if existed, I just tried to do the replace.
try
{
String uriString = request.getRequestUri().toString();
uriString = uriString.replaceAll("(\(|\)\/?)", "/");
request.setUris(request.getBaseUri(), new URI(uriString));
} catch (final Exception e)
{
}
return request;
I think that the easiest way how to handle this "protocol" would be introducing ContainerRequestFilter, which would replace "()$" with "/$" in the incoming URI. So you will be able to serve OData and standard REST request in one app.
See http://jersey.java.net/nonav/apidocs/1.11/jersey/com/sun/jersey/spi/container/ContainerRequestFilter.html
Simple filter I used to test this case:
rc.getProperties().put(ResourceConfig.PROPERTY_CONTAINER_REQUEST_FILTERS, new ContainerRequestFilter() {
#Override
public ContainerRequest filter(ContainerRequest request) {
try {
if(request.getRequestUri().toString().endsWith("(1)")) {
request.setUris(
request.getBaseUri(),
new URI(request.getRequestUri().toString().replace("(1)", "/1")));
}
} catch (Exception e) {
}
return request;
}
});
both
curl "http://localhost:9998/helloworld(1)"
curl "http://localhost:9998/helloworld/1"
hit same Resource method now. (Obviously you'll need to improve current filter to be able to handle various values, but it should work for you).

Resources