Is there a way to handle case when Tomcat aborts file upload due to defined limits, in order to return decent response (for example 413 Payload Too Large)?
I'm using Spring Boot 2.4.0 with following configuration:
spring.servlet.multipart.enabled=true
spring.servlet.multipart.max-file-size=20MB
spring.servlet.multipart.max-request-size=21MB
server.tomcat.max-swallow-size=21MB
server.tomcat.max-http-form-post-size=20MB
When trying to upload files larger than 21MB (or 26MB for some reason) using REST API, I receive java.net.SocketException: Connection reset by peer because Tomcat aborts the upload (closes the socket), due to defined maxSwallowSize. There's no exception in the logs by Spring. When running tests, I can see that RestTemplate retries couple of times, but eventually throws exception.
You can find MWE on GitHub.
Related
I recently upgraded CAS to 6.4.6.x and noticed that the liveness/readiness probes will intermittently throw 403 error codes. It appears to be a threading issue in the Spring Security Filter Chain. I have validated with the barebone CAS images that this does not happen in the 6.3.x version but can repeat it rather easily with the 6.4.x version. My configuration has not changed after the upgrade and I'm following the documentation.
Endpoint Configuration:
# allow all by default
cas.monitor.endpoints.endpoint.defaults.access[0]=PERMIT
# enable the health endpoint
management.endpoints.enabled-by-default=true
management.endpoints.web.base-path=/actuator
management.endpoints.web.exposure.include=health
management.endpoint.health.enabled=true
Running load tests against the instance if I send 1 request at a time I get 200 responses. If I bump up the concurrency to 2 or more I'm able to reproduce the threading issue and some of the responses return with a 403 after getting picked up by the Spring Default Error Controller.
Setting a breakpoint on the Error Controller, I'm able to see the same thread in the logs essentially jump to two different points in the code path.
I've gone through the Pull Requests from 6.3.x to 6.4.x and nothing jumped out to me that might be causing this issue. I haven't seen any issues raised up in Spring Boot around the Actuator Health Points failing. I've bumped up Spring and Tomcat to the latest patch versions. Any thoughts on what could be causing this or other things I could try to determine how to fix it?
Our application is a spring batch application running on EC2.
We have a RestTemplate call to an external application and it works fine.
But after 2 days, it started to fail.
we checked with the external application and logs shows the rest service response was successful from their end.
But our application log shows the below exception.
We tried redeploying our application and this behaviour still persists(works for 2 days and fails after that).
We are using spring boot 1.5.9 version.
2019-07-04 19:50:38.078 [10.174.81.249:SimpleAsyncTaskExecutor-2380]
Exception when getting data : org.springframework.web.client.UnknownHttpStatusCodeException: Unknown status code [494] null
at org.springframework.web.client.DefaultResponseErrorHandler.getHttpStatusCode(DefaultResponseErrorHandler.java:88)
at org.springframework.web.client.DefaultResponseErrorHandler.hasError(DefaultResponseErrorHandler.java:48)
at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:688)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:653)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:613)
at org.springframework.web.client.RestTemplate.exchange(RestTemplate.java:531)
I found status code 494 Request header too large is used by nginx and spring has not covered this status as it is not from the standard.
Please look enum org.springframework.http.HttpStatus
Also org.springframework.web.client.DefaultResponseErrorHandler
Solution: You have to check your configuration to verify which header is too large.
During the work with Spring Batch for a project, I ran into the following problem. The project consists of:
a MySQL database (mysqld 10.2.13-MariaDB)
Spring Boot (2.1.1.RELEASE)
Spring Batch (4.1.0.RELEASE)
Spring Batch is configured to use the same datasource as the business logic for the JobRepository
The hikari connection pool size for the datasource is configured to have a size of 4 (which is the default when pusing the app to our CloudFoundry instance and injected during the auto-reconfiguration of Spring)
As a reference you can take a look into the sample project where I created a minimal project to reproduce the problem:
https://github.com/FlorianSW/spring-batch-connection-issue
The problem:
Given you've a controller, which handles one RequestMapping in which at least two things happen: The controller does an arbitrary action agains the business model database schema (e.g. saving or requesting an entity from the database) and afterwards starting a Spring Batch job through a call to JobLauncher#run. Spring Batch is configured to run the tasks asynchronous with a ThreadPoolTaskExecutor with the pool size of 1.
This works pretty fine, if the request mapping is queries only few times per second (round about 1-3 times). However, if the mapping is queried more than that, let's say 4+ (during testing I used 4 and up to 20 requests) times in a synchronized way (if testing locally), then the requests run into a deadlock, where some of them will be aborted with the following exception:
Servlet.service() for servlet [dispatcherServlet] in context with path [] threw exception [Request processing failed; nested exception is org.springframework.dao.DataAccessResourceFailureException: Could not obtain last_insert_id(); nested exception is java.sql.SQLTransientConnectionException: HikariPool-1 - Connection is not available, request timed out after 5001ms.]
This problem can be mitigated when increasing the size of the connection pool to at least 7 (I don't know where this number could come from). After restarting the application and executing my test JavaScript code[1] to dispatch a number of requests I can easily increase the number of fetch requests to 150 and the application will handle these requests without any problems (as expected). Where, before increasing the pool size, numerous requests run into an timeout.
I'm not sure, if this qualifies as a bug or as an Improvement or whatever, and I'm not sure if the component is the documentation or something like that, however, my intention of this issue is:
Finding out, if Spring Batch, together with Spring Boot, requires a minimum number of available connections in the DB connection pool? If so, should this be mentioned in the documentation?
Or, if this is a bug as the JDBC connections used during the request processing are not returned to the pool in a reasonable short amount of time?
[1]
for (i = 0; i <= 150; i++) {fetch('http://localhost:8080/api/jobs/' + i, {method: 'PUT'})}
There are many REST services running which are Spring boot projects and are proxied via Apache Web Server. Now I am facing below issues:- 1. Randomly I get Response body does not contain any data but HTTP status Code is proper. So, why there is No Body even though the server is setting up the Body. In apache I just do ProxyPass /test http://localhost:7000/test
2. The response headers sometimes have Connection close and sometime keep Alive. Why there is a difference in Request header?Do I need to modify any default settings of ApacheServer? Can this happen due to Load on Apache Server? Can someone please give some pointers as to why this is happening.
I am also facing similar issue. Suddenly my spring boot application is not responding. Day before it was fine. I am developing JSF appliation - using Spring boot and primefaces. I remember, I was updating xhtml page and during deploying below stack-trace was thrown -
java.lang.IllegalStateException: The resources may not be accessed if they are not currently started
at org.apache.catalina.webresources.StandardRoot.validate(StandardRoot.java:245)
at org.apache.catalina.webresources.StandardRoot.getResource(StandardRoot.java:212)
at org.apache.catalina.webresources.StandardRoot.getResource(StandardRoot.java:206)
at org.apache.catalina.core.ApplicationContext.getResource(ApplicationContext.java:554)
at org.apache.catalina.core.ApplicationContextFacade.getResource(ApplicationContextFacade.java:199)
at org.springframework.web.context.support.ServletContextResource.exists(ServletContextResource.java:102)
at org.springframework.boot.autoconfigure.web.WebMvcAutoConfiguration$WebMvcAutoConfigurationAdapter.addStaticIndexHtmlViewControllers(WebMvcAutoConfiguration.java:276)
Note that I was using Spring boot 1.2.6.RELEASE, so I updated pom.xml to use latest version 1.3.3.RELEASE. Now deploy was successful, but Chrome web browser gets no response and blank page displayed. However, Firefox give below error in the page -
XML Parsing Error: no element found Location: http://127.0.0.1:8080/index.xhtml Line Number 1, Column 1:
Changed from embedded Tomcat to Jetty, but Jetty provides below stack-track during deploying -
javax.servlet.ServletException: Faces Servlet#87ee159b==javax.faces.webapp.FacesServlet,1,false
at org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:637)
at org.eclipse.jetty.servlet.ServletHolder.initialize(ServletHolder.java:396)
at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:871)
at org.springframework.boot.context.embedded.jetty.JettyEmbeddedWebAppContext$JettyEmbeddedServletHandler.deferredInitialize(JettyEmbeddedWebAppContext.java:46)
at org.springframework.boot.context.embedded.jetty.JettyEmbeddedWebAppContext.deferredInitialize(JettyEmbeddedWebAppContext.java:36)
at org.springframework.boot.context.embedded.jetty.JettyEmbeddedServletContainer.handleDeferredInitialize(JettyEmbeddedServletContainer.java:167)
at org.springframework.boot.context.embedded.jetty.JettyEmbeddedServletContainer.start(JettyEmbeddedServletContainer.java:114)
... 10 common frames omitted
Caused by: java.lang.IllegalStateException: Could not find backup for factory javax.faces.context.FacesContextFactory.
at javax.faces.FactoryFinderInstance.getFactory(FactoryFinderInstance.java:555)
at javax.faces.FactoryFinder.getFactory(FactoryFinder.java:283)
at javax.faces.webapp.FacesServlet.init(FacesServlet.java:358)
at org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:616)
Any light into this is highly appreciated.
I use spring cloud and the FeignRibbonClient to access remote services. The problem is, that this client ignores the retry configuration given by the properties:
example-client.ribbon.MaxAutoRetries=5
example-client.ribbon.MaxAutoRetriesNextServer=5
example-client.ribbon.OkToRetryOnAllOperations=true.
The retryHandlers are created without any configuration. What I want to get is to retry the next server after ConnectException. What I get is a RetryableException caused by a ConnectException.
Does anybody knows how to get the client call to the next server in case of a ConnectException?
Thanx
Lutz