How to resolve 502 and 503 errors with tomcat and springboot - spring-boot

We are constantly getting 502 and 503 errors during load test. 10% errors during the test and all of them are 502 and 503. We are using apache webserver and tomcat 9 as application server with springboot as framework. Any specific configuration to be added to avoid these errors? Please let me know. PFB the tomcat connector configurations:
port="8080" protocol="HTTP/1.1"
connectionTimeout="20000"
maxHttpHeaderSize="65536"
maxPostSize="6291456"
maxConnections="1500"
maxThreads="1500"
redirectPort="8443"

Related

Spring application and external tomcat server HTTP2

When I deploy a spring boot application on an external tomcat server, do I need to configure HTTP2 on the tomcat server as well my spring boot application? I am a bit confused here about how the communication occurs when I call my spring application deployed inside of Tomcat via the browser or any other client say, Postman.
I have enabled HTTP2 on Tomcat and verified the same:
0:0:0:0:0:0:0:1 - - [08/Jan/2023:18:28:44 +0530] "GET / HTTP/2.0" 200 11408
0:0:0:0:0:0:0:1 - - [08/Jan/2023:18:28:44 +0530] "GET /tomcat.svg HTTP/2.0" 200 68761
0:0:0:0:0:0:0:1 - - [08/Jan/2023:18:28:44 +0530] "GET /tomcat.css HTTP/2.0" 200 5895
and I have configured my spring boot application to use HTTP2 as well using the steps mentioned in this link.
When I am calling my RestController deployed in the external tomcat, I am still getting HTTP 1.1
127.0.0.1 - - [08/Jan/2023:23:41:26 +0530] "HEAD /demo-0.0.1-SNAPSHOT/ HTTP/1.1" 200 -
127.0.0.1 - - [08/Jan/2023:23:41:51 +0530] "HEAD /demo-0.0.1-SNAPSHOT/getMessage HTTP/1.1" 200 -
where demo-0.0.1-SNAPSHOT is the deployed artifact.
I verified the same using curl
HTTP/1.1 200
Content-Type: text/plain;charset=UTF-8
Content-Length: 41
Date: Sun, 08 Jan 2023 18:11:51 GMT
What am I missing here?
My application.properties file looks like:
# configuring SSL
server.port=8443
server.ssl.key-store-type=pkcs12
server.ssl.key-store=classpath:springboot.p12
server.ssl.key-store-password=password
server.ssl.key-alias=springboot
# enabling http2
server.http2.enabled=true
and I have created the Keystore correctly using the following command:
keytool -genkeypair -alias springboot -keyalg RSA -keysize 4096 -storetype PKCS12 -keystore springboot.p12 -validity 3650 -storepass password
EDIT 1:
Spring Boot version - 2.7.2
Tomcat server - 9.0.70
In my server.xml, I have commented and edited the following Connector:
<Connector port="8443" protocol="org.apache.coyote.http11.Http11NioProtocol"
maxThreads="200" SSLEnabled="true">
<UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" />
<SSLHostConfig>
<Certificate certificateKeystoreFile="ssl/tomcat.jks"
certificateKeyAlias="tomcat"
certificateKeystorePassword="password"
type="RSA" />
</SSLHostConfig>
</Connector>
which has helped me enable TLS as well as HTTP2 both on Tomcat. The problem is when I am deploying a spring boot application and making rest calls to that application's RestController, I am getting HTTP1.1 response. Making calls to Tomcat webservices is giving correct HTTP2 response.
Tomcat as standalone sever has by default in it's sever.xml disabled the connector for http 2.
Spring Boot 2, up to Tomcat 9
If this is a spring boot 2 deployed as war application then this could be using up to Tomcat 9.
You must find your Tomcat 9 (Server.xml) connector for http2 under the deployed server conf directory and uncomment this connector, while also providing the necessary certificate files.
You need to uncomment the
<Connector port="8443" protocol="org.apache.coyote.http11.Http11AprProtocol"
maxThreads="150" SSLEnabled="true" >
<UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" />
<SSLHostConfig>
<Certificate certificateKeyFile="conf/localhost-rsa-key.pem"
certificateFile="conf/localhost-rsa-cert.pem"
certificateChainFile="conf/localhost-rsa-chain.pem"
type="RSA" />
</SSLHostConfig>
</Connector>
Spring Boot 3, Tomcat 10 or later
In case you have a spring boot 3 deployed as war application then this should be using Tomcat 10 or later.
You must find your Tomcat 10 (Server.xml) connector for http2 under the deployed server conf directory and uncomment while also providing the necessary certificate file.
<Connector port="8443" protocol="org.apache.coyote.http11.Http11NioProtocol"
maxThreads="150" SSLEnabled="true">
<UpgradeProtocol className="org.apache.coyote.http2.Http2Protocol" />
<SSLHostConfig>
<Certificate certificateKeystoreFile="conf/localhost-rsa.jks"
type="RSA" />
</SSLHostConfig>
</Connector>
I found the solution. While deploying a spring boot application on external tomcat there is no need to specify Server SSL and HTTP2 parameters in the spring properties file as Tomcat will handle all the HTTPS requests. I was trying to mix 2 things.
Configuring the Spring Boot application to support HTTP2 will work if we are deploying using an embedded Tomcat server and I was able to verify that with Chrome browser.
As of writing this post, Postman still doesn't support HTTP2 requests, hence, I was seeing HTTP 1.1 protocol in the Tomcat access log. The same thing I believe is the issue with curl request. As my certificate is self-signed, I am using options -k and -sI. Maybe that's the reason, I am getting the response :
HTTP/1.1 200
Content-Type: text/html;charset=UTF-8
Transfer-Encoding: chunked
Date: Tue, 10 Jan 2023 18:09:29 GMT
when running curl -k -sI https://localhost:8443. I will figure out why I am getting HTTP 1.1 and update it here.
Sharing this answer to help anyone coming across this question next time.

Setup port 8443 on Tomcat errors while using tomcat monitor

I try to setup port 8443 for https on Tomcat 9.0.54.
For this I followed the following instructions :
I first created a key store:
"%JAVA_HOME%\bin\keytool" -genkey -alias tomcat -keyalg RSA
Then I updated my server.xml file
<Connector SSLEnabled="true"
acceptCount="100"
clientAuth="false"
disableUploadTimeout="true"
enableLookups="false"
maxThreads="25"
port="8443"
keystoreFile="C:/Users/Me/.keystore"
keystorePass="changeit"
protocol="org.apache.coyote.http11.Http11NioProtocol"
scheme="https"
secure="true"
sslProtocol="TLS"/>
So far so good. I launch startup.bat and I do have my server running startup.bat command. My issue is that my server must be launched at server startup if this one must be restarted. For this I set a service that launch automatically tomcat at startup.
At this stage, my servers starts and http://localhost:8080/ redirects to https://localhost:8443/.
I can reach my manager page on 8080 (http://localhost:8080/manager/html) and fails on 8443(can't open the page on https://localhost:8443/manager/html)

SSL Handshake Debugging not working any more with Apache Tomcat/9.0.0.M22

I have to debug a SSL Handshake on Tomcat (OS: MS Windows), so I followed the instructions found in the web an enabled it with the following line in setenv.bat:
set "JAVA_OPTS=%JAVA_OPTS% -Djavax.net.debug=ssl"
With Apache Tomcat/9.0.0.M21 everything works fine, when I opened the page with my browser I can see the handshake on the commandline.
With Apache Tomcat/9.0.0.M22 I only can see which certificates are beeing loaded on startup but after then when I call the server via browser no more output is generated on the commandline.
The only difference I could find out is that the Protocolhandler is different between those versions:
Apache Tomcat/9.0.0.M21 uses ProtocolHandler ["https-jsse-nio-8083"]
Apache Tomcat/9.0.0.M22 uses ProtocolHandler ["https-openssl-nio-8083"]
Is there anything I have to do additional to enable ssl handshake-debugging?
This is my Connector configured in server.xml:
<Connector port="8083" protocol="org.apache.coyote.http11.Http11NioProtocol"
maxThreads="150" SSLEnabled="true" scheme="https" secure="true"
clientAuth="false" sslProtocol="TLS"
keystoreFile="<Path to Keystore>"
keystorePass="<KeystorePW>" />
Try forcing JSSE use by adding sslImplementationName="org.apache.tomcat.util.net.jsse.JSSEImplementation" in your Connector . For any reason it's detecting APR and trying to use OpenSSL which is not working. You might fill a bug or ask the Tomcat users mailing-list, but 9.0.0 was a development release.

Is it Possible to Configure Tomcat 7 SSL to access certificates via SunMSCAPI?

Is it Possible to Configure Tomcat 7 SSL to access certificates via SunMSCAPI?
We are currently deploying Tomcat 7 to an Azure PAAS Cloud Service, which creates all instances as Window Servers.
We have configured SSL in Tomcat such that it accesses the certificate from a keystore file using JSSE (not APR):
<Connector port="443"
protocol="org.apache.coyote.http11.Http11NioProtocol"
maxThreads="150"
scheme="https"
secure="true"
keystoreFile="xxxxx" keystorePass="xxxxx" keystoreType="PKCS12"
clientAuth="false"
SSLEnabled="true"
sslProtocol="TLS" />
We would like to configure Tomcat to access the Windows Server Cert Manager, because MS Azure tooling provides for easy ways to deploy certificates to the instances via Powershell/REST APIs. I know this can be done with jetty using the Java SunMSCAPI.
Can this also be done in Tomcat 7?
Any help would be appreciated.
Not yet tested: Tell Tomcat to use the SubMSCAPI keystoreprovider:
--JvmOptions="-Djavax.net.ssl.keyStoreProvider=SunMSCAPI -Djavax.net.ssl.trustStoreProvider=SunMSCAPI -Djavax.net.ssl.trustStoreType=Windows-Root"
Configure the connector as
<Connector ...
keyAlias="my.www.server" keyStoreFile="" keyStoreType="Windows-My"
The value of keyAlias is the common name of the certificate, see
keytool -list -providername SunMSCAPI -storetype Windows-My

JBoss 6 Can Not Find an Installed Certifiate

I have created a keystore with a self signed certificate, exported the certificate, and re-imported the certificate with an alias of the IP address.
I want to make a call from inside this server to another service on this server at address]/.....
I can not get the client side of the call to trust itself. How do I specify the truststore?
Here is the key part of the server.xml:
<Connector protocol="HTTP/1.1" SSLEnabled="true"
port="8443" address="${jboss.bind.address}"
scheme="https" secure="true" clientAuth="false"
keystoreFile="${jboss.server.home.dir}/conf/mykey"
keystorePass="changeit"
truststoreFile="${jboss.server.home.dir}/conf/mykey"
truststorePass="changeit"
sslProtocol = "TLS" />
I get this error on attempting to use https to call a service from this server, to itself.
PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
Thanks
Answer: jBoss 6 is using the Java runtime for the client truststore, not its own file. Putting the certificate in the Java environment worked.
I would like to learn if I get set the truststore in jBoss's configuration for the connector as I have above...

Resources