How to set Max Pagination Size in Spring Boot? - spring

I'm developing Spring Boot + Spring Data Mongo + Spring HATEOAS example. I'm using Spring Boot V2.2.2.RELEASE.
I'm globally trying to set the Pagination PageSize limit to 200. For that I went through spring data jpa limit pagesize, how to set to maxSize and configurations like below
# DATA WEB (SpringDataWebProperties)
spring.data.web.pageable.default-page-size=20 # Default page size.
spring.data.web.pageable.max-page-size=2000 # Maximum page size to be accepted.
spring.data.web.pageable.one-indexed-parameters=false # Whether to expose and assume 1-based page number indexes.
spring.data.web.pageable.page-parameter=page # Page index parameter name.
spring.data.web.pageable.prefix= # General prefix to be prepended to the page number and page size parameters.
spring.data.web.pageable.qualifier-delimiter=_ # Delimiter to be used between the qualifier and the actual page number and size properties.
spring.data.web.pageable.size-parameter=size # Page size parameter name.
spring.data.web.sort.sort-parameter=sort # Sort parameter name.
If Client sends more than 200 PageSize, then I need to show user friendly error message.
#Configuration
public class PaginationConfig extends SpringDataWebConfiguration{
public PaginationConfig(ApplicationContext context, ObjectFactory<ConversionService> conversionService) {
super(context, conversionService);
}
#Bean
public PageableHandlerMethodArgumentResolver pageableResolver() {
PageableHandlerMethodArgumentResolver resolver = new PageableHandlerMethodArgumentResolver(sortResolver());
resolver.setMaxPageSize(1000);
return resolver;
}
}
I used above configurations, but getting below error.
2020-02-06 21:08:31.963 ERROR [Reference Data,,,] 22812 --- [ main] o.s.b.d.LoggingFailureAnalysisReporter :
***************************
APPLICATION FAILED TO START
***************************
Description:
Field pagedAssembler in com.example.EmployeeController required a bean of type 'org.springframework.data.web.PagedResourcesAssembler' that could not be found.
The injection point has the following annotations:
- #org.springframework.beans.factory.annotation.Autowired(required=true)
Action:
Consider defining a bean of type 'org.springframework.data.web.PagedResourcesAssembler' in your configuration.
Code:
#GetMapping(value = "/countries", produces = { MediaType.APPLICATION_JSON })
public ResponseEntity<PagedModel<EmployeeModel>> findEmployees(#Parameter(hidden=true) Pageable pageable) {
Page<EmployeeDto> page = EmployeeService.findAllEmployees(page_params, pageable);
PagedModel<EmployeeModel> model = pagedAssembler.toModel(page, EmployeeAssembler);
return new ResponseEntity<>(model, HttpStatus.OK);
}

I was able to solve the issue using below code.
#Configuration
public class WebConfig implements WebMvcConfigurer {
#Override
public void addArgumentResolvers(List<HandlerMethodArgumentResolver> resolvers) {
PageableHandlerMethodArgumentResolver resolver = new PageableHandlerMethodArgumentResolver() {
#Override
public Pageable resolveArgument(MethodParameter methodParameter,
#Nullable ModelAndViewContainer mavContainer, NativeWebRequest webRequest,
#Nullable WebDataBinderFactory binderFactory) {
Pageable p = super.resolveArgument(methodParameter, mavContainer, webRequest, binderFactory);
if(p.getPageSize() > Integer.parseInt(500)) {
throw new MandatoryResourceException("Error"), env.getProperty("500"));
}
return p;
}
};
resolvers.add(resolver);
WebMvcConfigurer.super.addArgumentResolvers(resolvers);
}
}

Related

Micrometer filter is ignored with CompositeMeterRegistry

I use Spring Boot 2.1.2.RELEASE, and I try to use Micrometer with CompositeMeterRegistry. My goal is to publish some selected meters to ElasticSearch. The code below shows my sample config. The problem is, that the filter is completely ignored (so all metrics are sent to ElasticSearch), although I can see in the logs that it was processed ("filter reply of meter ..." lines).
Strangely, if I define the MeterFilter as a Spring bean, then it's applied to ALL registries (however, I want it to be applied only on "elasticMeterRegistry").
Here is a sample configuration class:
#Configuration
public class AppConfiguration {
#Bean
public ElasticConfig elasticConfig() {
return new ElasticConfig() {
#Override
#Nullable
public String get(final String k) {
return null;
}
};
}
#Bean
public MeterRegistry meterRegistry(final ElasticConfig elasticConfig) {
final CompositeMeterRegistry registry = new CompositeMeterRegistry();
registry.add(new SimpleMeterRegistry());
registry.add(new JmxMeterRegistry(new JmxConfig() {
#Override
public Duration step() {
return Duration.ofSeconds(10);
}
#Override
#Nullable
public String get(String k) {
return null;
}
}, Clock.SYSTEM));
final ElasticMeterRegistry elasticMeterRegistry = new ElasticMeterRegistry(elasticConfig, Clock.SYSTEM);
elasticMeterRegistry.config().meterFilter(new MeterFilter() {
#Override
public MeterFilterReply accept(Meter.Id id) {
final MeterFilterReply reply =
id.getName().startsWith("logback")
? MeterFilterReply.NEUTRAL
: MeterFilterReply.DENY;
log.info("filter reply of meter {}: {}", id.getName(), reply);
return reply;
}
});
registry.add(elasticMeterRegistry);
return registry;
}
}
So, I expect ElasticSearch to receive only "logback" metrics, and JMX to receive all metrics.
UPDATE:
I have played with filters and found a "solution", but I don't really understand why the code above doesn't work.
This works:
elasticMeterRegistry.config().meterFilter(new MeterFilter() {
#Override
public MeterFilterReply accept(Meter.Id id) {
final MeterFilterReply reply =
id.getName().startsWith("logback")
? MeterFilterReply.ACCEPT
: MeterFilterReply.DENY;
log.info("filter reply of meter {}: {}", id.getName(), reply);
return reply;
}
});
The difference is: I return ACCEPT instead of NEUTRAL.
Strangely, the following code does not work (ES gets all metrics):
elasticMeterRegistry.config().meterFilter(
MeterFilter.accept(id -> id.getName().startsWith("logback")));
But this works:
elasticMeterRegistry.config().meterFilter(
MeterFilter.accept(id -> id.getName().startsWith("logback")));
elasticMeterRegistry.config().meterFilter(
MeterFilter.deny());
CONCLUSION:
So, it seems that instead of NEUTRAL, the filter should return ACCEPT. But for meters not starting with "logback", my original filter (with NEUTRAL) returns DENY. Then why are those metrics published to ElasticSearch registry?
Can someone explain this?
This is really a composite of questions. I'll just point out a few points.
For the MeterRegistry bean you defined, Spring Boot will auto-configure an ElasticMeterRegistry bean as there's no ElasticMeterRegistry bean. Instead of creating a CompositeMeterRegistry bean on your own, just define a custom ElasticMeterRegistry bean which is applied the MeterFilter you want and let Spring Boot create one (CompositeMeterRegistry bean) for you.
For MeterFilterReply, ACCEPT will accept the meter immediately, DENY will deny the meter immediately, and NEUTRAL will postpone the decision to next filter(s). Basically meters will be accepted unless there's any DENY.

Own HttpTraceRepository

i created my own HttpTraceRepository, for write and read httpTrace from mongodb on capped collection.
Today, i don't understand why HttpTrace is a final class without public constructor.
In my case, when i read mongodb ( findAll method ) to find the last X hours http trace and show it in spring-boot admin UI, it's impossible to instanciante HttpTrace object.
#Component
public class MongoTraceRepository implements HttpTraceRepository {
private static final String ADMIN_TRACE = "admin.trace";
private MongoOperations mongoOps;
#Value("${admin.display.trace.last.x.hours}")
private int displayTraceLastXHours;
#Autowired
public MongoTraceRepository(MongoOperations mongoOps) {
this.mongoOps = mongoOps;
}
#PostConstruct
public void initialize() {
MongoCollection<Document> collection = mongoOps.getCollection(ADMIN_TRACE);
boolean collectionExists = mongoOps.collectionExists(ADMIN_TRACE);
if (!collectionExists) {
collection.drop();
// 100 Mo max, 500 000 documents max, capped !
mongoOps.createCollection(ADMIN_TRACE, CollectionOptions.empty().size(104857600).maxDocuments(500000).capped());
}
}
#Override
public List<HttpTrace> findAll() {
Date yesterday = Date.from(LocalDateTime.now().minusHours(displayTraceLastXHours).atZone(ZoneId.systemDefault()).toInstant());
return mongoOps.find(new Query(where("timestamp").gte(yesterday)), HttpTrace.class, ADMIN_TRACE)
.stream()
.sorted((o1, o2) -> o2.getTimestamp().compareTo(o1.getTimestamp()))
.collect(Collectors.toList());
}
#Override
#Async
public void add(HttpTrace traceInfo) {
mongoOps.save(traceInfo, ADMIN_TRACE);
}
}
This is the error log :
org.springframework.data.mapping.MappingException: No property request found on entity class org.springframework.boot.actuate.trace.http.HttpTrace$Request to bind constructor parameter to!
at org.springframework.data.mapping.model.PersistentEntityParameterValueProvider.getParameterValue(PersistentEntityParameterValueProvider.java:68)
at org.springframework.data.mapping.model.SpELExpressionParameterValueProvider.getParameterValue(SpELExpressionParameterValueProvider.java:49)
at org.springframework.data.convert.ReflectionEntityInstantiator.createInstance(ReflectionEntityInstantiator.java:75)
at org.springframework.data.convert.ClassGeneratingEntityInstantiator.createInstance(ClassGeneratingEntityInstantiator.java:86)
at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:273)
at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:253)
at org.springframework.data.mongodb.core.convert.MappingMongoConverter.readValue(MappingMongoConverter.java:1387)
at org.springframework.data.mongodb.core.convert.MappingMongoConverter$MongoDbPropertyValueProvider.getPropertyValue(MappingMongoConverter.java:1334)
at org.springframework.data.mongodb.core.convert.MappingMongoConverter$MongoDbPropertyValueProvider.getPropertyValue(MappingMongoConverter.java:1276)
at org.springframework.data.mapping.model.PersistentEntityParameterValueProvider.getParameterValue(PersistentEntityParameterValueProvider.java:71)
at org.springframework.data.mapping.model.SpELExpressionParameterValueProvider.getParameterValue(SpELExpressionParameterValueProvider.java:49)
at org.springframework.data.convert.ReflectionEntityInstantiator.createInstance(ReflectionEntityInstantiator.java:75)
at org.springframework.data.convert.ClassGeneratingEntityInstantiator.createInstance(ClassGeneratingEntityInstantiator.java:86)
at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:273)
at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:253)
at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:202)
at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:198)
at org.springframework.data.mongodb.core.convert.MappingMongoConverter.read(MappingMongoConverter.java:86)
at org.springframework.data.mongodb.core.MongoTemplate$ReadDocumentCallback.doWith(MongoTemplate.java:2785)
at org.springframework.data.mongodb.core.MongoTemplate.executeFindMultiInternal(MongoTemplate.java:2448)
at org.springframework.data.mongodb.core.MongoTemplate.doFind(MongoTemplate.java:2244)
at org.springframework.data.mongodb.core.MongoTemplate.doFind(MongoTemplate.java:2227)
at org.springframework.data.mongodb.core.MongoTemplate.find(MongoTemplate.java:770)
There is a constructor especially suited to create HTTPTrace from a persistence store.
It was added since 2.1.0 though.

Spring boot 2 Undertow, not work set UndertowOptions.ALLOW_UNESCAPED_CHARACTERS_IN_URL

Springboot 2.0
When navigating through type references: localhost:9093/?foo=bar|bar
Return error: Caused by: java.net.URISyntaxException: Illegal
character in query
I'm adding an extra option:
#Component
public class MyWebServerCustomizer implements WebServerFactoryCustomizer<UndertowServletWebServerFactory> {
#Override
public void customize(UndertowServletWebServerFactory factory) {
factory.addBuilderCustomizers(builder-> builder.setServerOption(UndertowOptions.ALLOW_UNESCAPED_CHARACTERS_IN_URL, Boolean.TRUE));
}
}
Does not work, the exception is still.
What am I doing wrong?
Thanks.
#Configuration
public class HttpConfig {
#Bean
public UndertowServletWebServerFactory servletWebServerFactory(ApplicationProperties properties) {
UndertowServletWebServerFactory factory = new UndertowServletWebServerFactory();
factory.addBuilderCustomizers((UndertowBuilderCustomizer) builder ->
builder.setServerOption(UndertowOptions.ALLOW_UNESCAPED_CHARACTERS_IN_URL, Boolean.TRUE));
return factory;
}
}
You should encode the parameters
myUrl = "localhost:9093/?" + URLEncoder.encode("foo=bar|bar", "UTF-8");
please refer here for url encoding. Say suppose if you want to send bar|bar. You can send as bar%7Cbar.
Here %7C is PIPE Character.

spring mvc processing xml with relative path to dtd

My webservice receives an xml from a third-party source, which contains a !DOCTYPE declaration. Therefore I must use the second method in my controller to parse the xml document, the first one gives me this exception:
Failed to read HTTP message: org.springframework.http.converter.HttpMessageNotReadableException: Could not unmarshal to [class com.example.MeterBusXml]: null; nested exception is javax.xml.bind.UnmarshalException
- with linked exception:
[org.xml.sax.SAXParseException; lineNumber: 1; columnNumber: 48; DOCTYPE is disallowed when the feature "http://apache.org/xml/features/disallow-doctype-decl" set to true.]
I have no control over the application which posts the xml, so I must adapt my webservice to parse it with the dtd.
My question is, what is the spring framework's way of injecting the EntityResolver into every XMLReader instance?
#RestController
public class MeterBusDataController {
#RequestMapping (
consumes = APPLICATION_XML_VALUE,
method = POST,
path = "/meterbus1"
)
public void method1(#RequestBody MeterBusXml xml) {
System.out.println(xml);
}
#RequestMapping(
method = POST,
path = "/meterbus2"
)
public void method2(HttpServletRequest rq) throws IOException, ParserConfigurationException, SAXException, JAXBException {
JAXBContext jc = newInstance(MeterBusXml.class);
Unmarshaller um = jc.createUnmarshaller();
SAXParserFactory spf = SAXParserFactory.newInstance();
spf.setNamespaceAware(true);
spf.setValidating(true);
SAXParser sp = spf.newSAXParser();
XMLReader xr = sp.getXMLReader();
xr.setEntityResolver(new EntityResolver() {
#Override
public InputSource resolveEntity(String publicId, String systemId) throws SAXException, IOException {
return new InputSource(new StringReader(""));
}
});
BufferedReader reader = rq.getReader();
InputSource inputSource = new InputSource(reader);
SAXSource saxSource = new SAXSource(xr, inputSource);
MeterBusXml xml = (MeterBusXml)um.unmarshal(saxSource);
System.out.println(xml);
}
}
See the following document for an example of the mbus.xml I'm trying to unmarshal.
http://prevodniky.sk/products/product_EthMBus_common/download/Ethernet_converters_exports_v1_02_EN.pdf
I've found the root of the problem. First I tried to create and configure a Jaxb2Marshaller bean, but that did not work out. Then I realized, I need a HttpMessageConverter, so I had to override the extendMessageConverters method in the WebMvcConfigurerAdapter class, and set the required properties on Jaxb2RootElementHttpMessageConverter. This message converter does not use a Jaxb2Marshaller, but it's internal workings are very similar.
setSupportDtd(true) is required, to force the parser to accept the !DOCTYPE declaration.
setProcessExternalEntities(false) is required, because if this property is false, then the converter uses a blank EntityResolver, just as I did in method2.
#Configuration
public class WebConfiguration extends WebMvcConfigurerAdapter {
#Override
public void extendMessageConverters(List<HttpMessageConverter<Jaxb2RootElementHttpMessageConverter?>> converters) {
for (final Iterator<HttpMessageConverter<?>> iterator = converters.iterator(); iterator.hasNext();) {
HttpMessageConverter<?> next = iterator.next();
if (next instanceof Jaxb2RootElementHttpMessageConverter) {
Jaxb2RootElementHttpMessageConverter jaxbConverter = (Jaxb2RootElementHttpMessageConverter) next;
jaxbConverter.setProcessExternalEntities(false);
jaxbConverter.setSupportDtd(true);
}
}
}
}

#MessageMapping with placeholders

I am working with Spring-websocket and I have the following problem:
I am trying to put a placeholder inside a #MessageMapping annotation in order to get the url from properties. It works with #RequestMapping but not with #MessageMapping.
If I use this placeholder, the URL is null. Any idea or suggestion?
Example:
#RequestMapping(value= "${myProperty}")
#MessageMapping("${myProperty}")
Rossen Stoyanchev added placeholder support for #MessageMapping and #SubscribeMapping methods.
See Jira issue: https://jira.spring.io/browse/SPR-13271
Spring allows you to use property placeholders in #RequestMapping, but not in #MessageMapping. This is 'cause the MessageHandler. So, we need to override the default MessageHandler to do this.
WebSocketAnnotationMethodMessageHandler does not support placeholders and you need add this support yourself.
For simplicity I just created another WebSocketAnnotationMethodMessageHandler class in my project at the same package of the original, org.springframework.web.socket.messaging, and override getMappingForMethod method from SimpAnnotationMethodMessageHandler with same content, changing only how SimpMessageMappingInfo is contructed using this with this methods (private in WebSocketAnnotationMethodMessageHandler):
private SimpMessageMappingInfo createMessageMappingCondition(final MessageMapping annotation) {
return new SimpMessageMappingInfo(SimpMessageTypeMessageCondition.MESSAGE, new DestinationPatternsMessageCondition(
this.resolveAnnotationValues(annotation.value()), this.getPathMatcher()));
}
private SimpMessageMappingInfo createSubscribeCondition(final SubscribeMapping annotation) {
final SimpMessageTypeMessageCondition messageTypeMessageCondition = SimpMessageTypeMessageCondition.SUBSCRIBE;
return new SimpMessageMappingInfo(messageTypeMessageCondition, new DestinationPatternsMessageCondition(
this.resolveAnnotationValues(annotation.value()), this.getPathMatcher()));
}
These methods now will resolve value considering properties (calling resolveAnnotationValues method), so we need use something like this:
private String[] resolveAnnotationValues(final String[] destinationNames) {
final int length = destinationNames.length;
final String[] result = new String[length];
for (int i = 0; i < length; i++) {
result[i] = this.resolveAnnotationValue(destinationNames[i]);
}
return result;
}
private String resolveAnnotationValue(final String name) {
if (!(this.getApplicationContext() instanceof ConfigurableApplicationContext)) {
return name;
}
final ConfigurableApplicationContext applicationContext = (ConfigurableApplicationContext) this.getApplicationContext();
final ConfigurableBeanFactory configurableBeanFactory = applicationContext.getBeanFactory();
final String placeholdersResolved = configurableBeanFactory.resolveEmbeddedValue(name);
final BeanExpressionResolver exprResolver = configurableBeanFactory.getBeanExpressionResolver();
if (exprResolver == null) {
return name;
}
final Object result = exprResolver.evaluate(placeholdersResolved, new BeanExpressionContext(configurableBeanFactory, null));
return result != null ? result.toString() : name;
}
You still need to define a PropertySourcesPlaceholderConfigurer bean in your configuration.
If you are using XML based configuration, include something like this:
<context:property-placeholder location="classpath:/META-INF/spring/url-mapping-config.properties" />
If you are using Java based configuration, you can try in this way:
#Configuration
#PropertySources(value = #PropertySource("classpath:/META-INF/spring/url-mapping-config.properties"))
public class URLMappingConfig {
#Bean
public static PropertySourcesPlaceholderConfigurer propertySourcesPlaceholderConfigurer() {
return new PropertySourcesPlaceholderConfigurer();
}
}
Obs.: in this case, url-mapping-config.properties file are in a gradle/maven project in src\main\resources\META-INF\spring folder and content look like this:
myPropertyWS=urlvaluews
This is my sample controller:
#Controller
public class WebSocketController {
#SendTo("/topic/test")
#MessageMapping("${myPropertyWS}")
public String test() throws Exception {
Thread.sleep(4000); // simulated delay
return "OK";
}
}
With default MessageHandler startup log will print something like this:
INFO: Mapped "{[/${myPropertyWS}],messageType=[MESSAGE]}" onto public java.lang.String com.brunocesar.controller.WebSocketController.test() throws java.lang.Exception
And with our MessageHandler now print this:
INFO: Mapped "{[/urlvaluews],messageType=[MESSAGE]}" onto public java.lang.String com.brunocesar.controller.WebSocketController.test() throws java.lang.Exception
See in this gist the full WebSocketAnnotationMethodMessageHandler implementation.
EDIT: this solution resolves the problem for versions before 4.2 GA. For more information, see this jira.
Update :
Now I understood what you mean, but I think that is not possible(yet).
Documentation does not mention anything related to Path mapping URIs.
Old answer
Use
#MessageMapping("/handler/{myProperty}")
instead of
#MessageMapping("/handler/${myProperty}")
And use it like this:
#MessageMapping("/myHandler/{username}")
public void handleTextMessage(#DestinationVariable String username,Message message) {
//do something
}
#MessageMapping("/chat/{roomId}")
public Message handleMessages(#DestinationVariable("roomId") String roomId, #Payload Message message, Traveler traveler) throws Exception {
System.out.println("Message received for room: " + roomId);
System.out.println("User: " + traveler.toString());
// store message in database
message.setAuthor(traveler);
message.setChatRoomId(Integer.parseInt(roomId));
int id = MessageRepository.getInstance().save(message);
message.setId(id);
return message;
}

Resources