I'm currently trying to get a reverse proxy up and running using Spring Boot and Zuul. The HTTP part is working already. However if I'm trying to request a site via HTTPS e.g. curl https://twitter.com then the request doesn't even reach my implemented logic.
The processing stops in the middle of normalizing the URL, because it looks like this: twitter.com:443. The comment of the relevant code snippet is not that helpful.
// The URL must start with '/'
if (b[start] != (byte) '/') {
return false;
}
I'm using everything without much configuration, just server.port: 9999. Am I missing something here, or why does Tomcat not accept the request?
The relevant Stacktrace:
org.apache.catalina.connector.CoyoteAdapter.normalize(CoyoteAdapter.java:1172)
org.apache.catalina.connector.CoyoteAdapter.postParseRequest(CoyoteAdapter.java:623)
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336)
org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:799)
org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:868)
org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1455)
Update
In the mean time I figured out the difference. I want to use Zuul as a proxy for both HTTP and HTTPS. However the request that hits the server is not the same as with HTTP but the following:
CONNECT twitter.com:443 HTTP/1.1
Host: twitter.com:443
User-Agent: curl/7.54.1
Proxy-Connection: Keep-Alive
So is it even possible to build a HTTPS proxy using Spring Boot and Zuul?
Related
I don't know much about Webflux / Reactor / Netty. I'm using Spring's WebClient to do all the heavy lifting. But it appears not to work correctly when a server responds back early with an error.
My understanding is when you are POSTing data to a server, the server can respond at any time with an HTTP 4XX error. The client is supposed to stop sending the HTTP body and read that error.
I have a very simply WebClient that POSTs data to a server. It looks like this:
FileResponse resp = client.post().uri(uri)
.contentType(MediaType.APPLICATION_OCTET_STREAM)
.header("Authorization", "Bearer " + authorizationToken)
.accept(MediaType.APPLICATION_JSON)
.bodyValue(data)
.retrieve()
.bodyToMono(FileResponse.class)
.block();
The body can contain a large amount of data (100+KB). Apparently the server looks at the header, validates the authorization token, and only if it's valid, reads the body. If the authorization token is not valid (expired, etc) it immediately responds with an "HTTP 401 Unauthorized" with the response body "{"message": "Invalid user/password"}" while the client is still sending the body. The server then closes the socket which results in the WebClient throwing this:
2022-08-10 15:56:03,474 WARN [reactor.netty.http.client.HttpClientConnect] (reactor-http-nio-1) [id: 0xa7b48bb8, L:/5.6.7.8:51122 - R:dubcleoa030/1.2.3.4:5443] The connection observed an error: java.io.IOException: An existing connection was forcibly closed by the remote host
at java.base/sun.nio.ch.SocketDispatcher.read0(Native Method)
at java.base/sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:43)
at java.base/sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:276)
at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:233)
at java.base/sun.nio.ch.IOUtil.read(IOUtil.java:223)
at java.base/sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:358)
at deployment.bp-global.war//io.netty.buffer.PooledByteBuf.setBytes(PooledByteBuf.java:253)
at deployment.bp-global.war//io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1133)
at deployment.bp-global.war//io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:350)
at deployment.bp-global.war//io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:148)
at deployment.bp-global.war//io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714)
at deployment.bp-global.war//io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650)
at deployment.bp-global.war//io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576)
at deployment.bp-global.war//io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493)
at deployment.bp-global.war//io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at deployment.bp-global.war//io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at deployment.bp-global.war//io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:834)
I've made the same request with curl, and it's handled properly. Curl sees the server's early response, stops sending the body and processes the response from the server. I've chopped out a lot of fluff from the curl output but here is the important stuff...
Trying 1.2.3.4...
* TCP_NODELAY set
* Connected to 1.2.3.4 port 5443 (#0)
> POST /api/folders/file/?path=/out HTTP/1.1
> Host: 1.2.3.4:5443
> User-Agent: curl/7.61.1
> accept-encoding: gzip
> Content-Type: application/octet-stream
> Authorization: Bearer youshallnotpass
> accept: application/json
> Content-Length: 298190
>
< HTTP/1.1 401 Unauthorized
< Server: Cleo Harmony/5.7.0.3 (Linux)
< Date: Wed, 10 Aug 2022 21:59:23 GMT
< Content-Length: 36
< Content-Language: en
< Content-Type: application/json
< Connection: keep-alive
* HTTP error before end of send, stop sending
* Closing connection 0
{"message": "Invalid user/password"}
I'm not sure if the issue is with Spring's WebClient or the underlying reactor-netty stuff. But am I crazy or does it just look broken if the server responds early? If I am correct that it's broken, any thoughts on a work-around?
Thank you!
Todd
I setup a small stand-alone command line Spring Boot program so I could test various aspects of this issue. What I found is that if I send the body from memory (byte[]) the issue occurs. If I send the body from a file resource as shown below, everything works correctly.
Our current very large product is using Spring Boot 2.3.3. My stand-alone test program gave me the ability to quickly upgrade Spring Boot to 2.7.2. Everything works correctly in Spring Boot 2.7.2. So it was definitely a bug that was fixed at some point.
Unfortunately our large project cannot be upgraded to Spring Boot 2.7.2 overnight. It will be a large effort which will require extensive testing from our QA department. So for now, as a work-around, I'll write the body payload to a temporary file so I can get WebClient to read it as a file resource which works in 2.3.3 as shown below. Just in case any other poor developer runs into this and needs a work-around, try this...
InputStreamResource resource = new InputStreamResource(new FileInputStream(tempFilename));
String resp = client.post().uri(uri)
.contentType(MediaType.APPLICATION_OCTET_STREAM)
.header("Authorization", "Bearer youshallnotpass")
.accept(MediaType.APPLICATION_JSON)
.body(BodyInserters.fromResource(resource))
.retrieve()
.bodyToMono(String.class)
.block();
EDIT: I started going through the releases to determine when this was fixed. This issues appears to be fixed in Spring Boot 2.3.4 (Netty 4.1.52). I cannot find any reference to this bug in the release notes of Spring Boot or Netty. So I'm not sure where the issue was. Perhaps it was in a deeper dependency. But it was definitely fixed with Spring Boot 2.3.4.
It may happened because there is a conflict in dependencies (may create conflict with azure dependency for same package):
update your spring boot version I ahve updted to latest and works for me.
I am in the process of upgrading our jetty from 9.2.24 to 9.4.10, for an app that works extensively with websockets.
I have an existing test (junit) that sets embedded jetty, registers to it rest resource and websocket servlet and then tests to see if they can be accessed.
The test works perfectly well when jetty is at version 9.2.24. An attempt to move to version 9.4.10 with the very same code fails with
java.io.IOException: Connect failure
at org.eclipse.jetty.websocket.jsr356.ClientContainer.connect(ClientContainer.java:232)
at org.eclipse.jetty.websocket.jsr356.ClientContainer.connectToServer(ClientContainer.java:255)
...
Caused by: org.eclipse.jetty.websocket.api.UpgradeException: 400 Bad Request
at org.eclipse.jetty.websocket.client.WebSocketUpgradeRequest.onComplete(WebSocketUpgradeRequest.java:522)
at org.eclipse.jetty.client.ResponseNotifier.notifyComplete(ResponseNotifier.java:193)
The websocket definition on server side is based o JSR356 spec (i.e. extends EndPoint). The websocket client used to access the websocket is also based on the javax.websocket (i.e. ContainerProvider.getWebSocketContainer().connectToServer(Endpoint instance...)) - where the websocket container is effectively a jetty one...
The server sets up perfectly. The problem is only when trying to access the websocket. I have debugged and could not find any difference in the way the client initiates and sends the websocket request. In particular the request has a the 'upgrade' header set to 'websocket' as expected.
So I could only assume that the problem is in the way the websocket resource is registered in the embedded jetty. I have debugged the working flow (with 9.2.24) and found the most early place where the connection is accepted in jetty (one of the selector threads at AbstractConnection). but from some reason I am not getting to that point for the websocket when working with 9.4.10
I have read several resources and SO question (e.g. this question) and could not found anything that will help me with this problem.
I am in a dead end.
Here is the key elements in the relevant code of the server registration (I also have another rest resource along with the websocket one):
// web socket
ServletContextHandler wsContext = new ServletContextHandler(ServletContextHandler.SESSIONS);
wsContext.setContextPath("/ws_api");
ServerContainer container = WebSocketServerContainerInitializer.configureContext(servletContextHandler);
container.addEndpoint(new BasicServerEndpointConfig(container.getClient(), endpointClassObject, path)
// rest handler
ServletContextHandler restContext = new ServletContextHandler(ServletContextHandler.SESSIONS);
restContext.setContextPath("/rest_api");
...
ServletHolder sh = new ServletHolder(...);
restContext.addServlet(sh, "/*");
final HandlerList handlers = new HandlerList();
handlers.setHandlers(new Handler[]{wsContext, restContext, new DefaultHandler()});
server.setHandler(handlers);
Help...
Update (additional information per Joakim Erdfelt request):
I am at class HTTPConnection class, in BP at onComplete() method, fetching the request headers from the _channel._fields object I get:
The response object's status is 200 (and not 101 as expected):
My endpoint object is part of a large inheritance chain. It is full of boilerplate business logic code that I need to remove before I can upload it, but in the root stands the javax.websocket.Endpont class, where we implemented only the onOpen(Session session, EndpointConfig config) method. I am not getting to that method when debugging, seems to fail long before...
Your request headers looks like this ...
Accept: application/json, application/*+json
Accept-Encoding: gzip
Cache-Control: no-cache
Connection: keep-alive
Content-Type: application/json
Host: 127.0.0.1:8080
Pragma: no-cache
Sec-WebSocket-Key: sMQPm6Cf00itLII3QBb4w==
Sec-WebSocket-Version: 13
Upgrade: websocket
User-Agent: Java/1.8.0_144
That is an invalid WebSocket Upgrade Request.
The most glaring omission is
Connection: upgrade
But there's also other fields that a compliant WebSocket Client would never set.
Content-Type: application/json
Accept: application/json, application/*+json
Accept-Encoding: gzip
I am developing an OpenCMIS client using BindingType.Browser. Creating a session passing the required parameters for USER, PASSWORD, BROWSER_URL, BINDING_TYPE and REPOSITORY_ID works as expected. The session is created and further steps can be executed.
Now I want to make my client run on machines using HTTP proxies to access the internet. To specify the proxy to access I set the system properties http.proxyUrl and http.proxyPort. This works too as long as the proxy does not require authentication.
Well, that's the point where I am struggeling right now. I activated the authentication in my test proxy and in the client code I added the parameters PROXY_USER and PROXY_PASSWORD to create the session. But this doesn't seem to work. I already debugged the used StandardAuthenticationProvider to verify what happens. The HTTP-Header "Proxy-Authenticate" is created by the authentication provider but it seems that the chemistry framework is sending the request without that header. In the log of my proxy the received requests do not contain any security headers.
CONNECT my.server.org:443 HTTP/1.1
User-Agent: Java/1.8.0_111
Host: my.server.org
Accept: text/html, image/gif, image/jpeg, *; q=.2, */*; q=.2
Proxy-Connection: keep-alive
Any suggestions?
I found the solution in the following thread: Java Web Start: Unable to tunnel through proxy since Java 8 Update 111.
Since Java 8 Update 111 the Basic authentication scheme has been deactivated, by default, in the Oracle Java Runtime, by adding "Basic" to the jdk.http.auth.tunneling.disabledSchemes networking property when proxying HTTPS. For more information refer to 8u111 Update Release Notes.
To re-activate the proxying you need to set jdk.http.auth.tunneling.disabledSchemes="". This cann be done via VM Argument when starting the application
-Djdk.http.auth.tunneling.disabledSchemes=""
or at runtime by setting the property in the main method of the application
System.setProperty("jdk.http.auth.tunneling.disabledSchemes", "");
I tried to send a Javascript XHR request that includes a
'Authorization: Basic base64encodedCredentials'
header through a Zuul proxy. Behind the proxy is my Spring Security Server that exposes an endpoint which is secured by Basic HTTP authentication. My server reacts with a 401 ("unauthorized"). If I send the same request via curl, all is well.
Using a network monitoring tool, I observed that the header Zuul is sending looks like this:
'authorization: Basic base64encodedCredentials' (notice the lowercase 'a' in 'authorization')
After I had a look at the source code of BasicAuthenticationFilter in Spring Security, I suspect that the lowercase 'a' is the reason why my XHR request is not authenticated by my server.
Is this a bug in Zuul ? What do you think ?
Edit: http header names are actually case insensitive.
So Zuul is not to blame. What else has then caused the authentication to fail ?
I was looking at the wrong place.
For everyone stumbling over this, I'll depict what happened:
My Zuul route was set to
http://<host>/<securityContext>/auth/token
while it should have been set to
http://<host>/<securityContext>/oauth/token (notice the extra 'o')
The wicked reaction of the Spring Security Server was not to respond with a '404' ("not found") to the wrong route, but with a '401' ("unauthorized"). This lead me to the assumption that there is something wrong with the authentication header (or some other header for that matter).
As of now, I don't know why the Security Server responded that way. It seems to me that there is actually an endpoint exposed under /auth/token.
I have one micro service "user-service". It is secured with spring-cloud-oauth2. It has one REST endpoint "/data/v1" which returns some JSON response.
When I send GET request to this endpoint it gets redirected to /oauth/login for authentication and after successful authentication it returns the token and then I am able to get the JSON response.
Later I added zuul proxy service ("micro-proxy-service") to route all external requests to internal back end services like "user-service".
It has single route "/resource" which then forwards the request to "/data/v1"
But now if I send GET requests to "/resource/data/v1" even after successful oauth2 authentication I get HTTP 302 response code with redirection to /login page.
If I try to access the "/data/v1" without zuul proxy then I am able to get response after oauth2. But when I request through zuul then I get 302.
If I remove the oauth2 from "user-service" then I can access "/data/v1" in both ways: directly or through zuul.
Please let me know If I am missing anything in zuul configuration.
Thanks.
Zuul strips the authentication headers by default, to enable them use the following configuration
zuul:
sensitiveHeaders: Cookie,Set-Cookie