Jetty upgrade (9.2.24 -> 9.4.10) fails with websocket (UpgradeException) - websocket

Question

I am in the process of upgrading our jetty from 9.2.24 to 9.4.10, for an app that works extensively with websockets.
I have an existing test (junit) that sets embedded jetty, registers to it rest resource and websocket servlet and then tests to see if they can be accessed.
The test works perfectly well when jetty is at version 9.2.24. An attempt to move to version 9.4.10 with the very same code fails with
java.io.IOException: Connect failure
at org.eclipse.jetty.websocket.jsr356.ClientContainer.connect(ClientContainer.java:232)
at org.eclipse.jetty.websocket.jsr356.ClientContainer.connectToServer(ClientContainer.java:255)
...
Caused by: org.eclipse.jetty.websocket.api.UpgradeException: 400 Bad Request
at org.eclipse.jetty.websocket.client.WebSocketUpgradeRequest.onComplete(WebSocketUpgradeRequest.java:522)
at org.eclipse.jetty.client.ResponseNotifier.notifyComplete(ResponseNotifier.java:193)
The websocket definition on server side is based o JSR356 spec (i.e. extends EndPoint). The websocket client used to access the websocket is also based on the javax.websocket (i.e. ContainerProvider.getWebSocketContainer().connectToServer(Endpoint instance...)) - where the websocket container is effectively a jetty one...
The server sets up perfectly. The problem is only when trying to access the websocket. I have debugged and could not find any difference in the way the client initiates and sends the websocket request. In particular the request has a the 'upgrade' header set to 'websocket' as expected.
So I could only assume that the problem is in the way the websocket resource is registered in the embedded jetty. I have debugged the working flow (with 9.2.24) and found the most early place where the connection is accepted in jetty (one of the selector threads at AbstractConnection). but from some reason I am not getting to that point for the websocket when working with 9.4.10
I have read several resources and SO question (e.g. this question) and could not found anything that will help me with this problem.
I am in a dead end.
Here is the key elements in the relevant code of the server registration (I also have another rest resource along with the websocket one):
// web socket
ServletContextHandler wsContext = new ServletContextHandler(ServletContextHandler.SESSIONS);
wsContext.setContextPath("/ws_api");
ServerContainer container = WebSocketServerContainerInitializer.configureContext(servletContextHandler);
container.addEndpoint(new BasicServerEndpointConfig(container.getClient(), endpointClassObject, path)
// rest handler
ServletContextHandler restContext = new ServletContextHandler(ServletContextHandler.SESSIONS);
restContext.setContextPath("/rest_api");
...
ServletHolder sh = new ServletHolder(...);
restContext.addServlet(sh, "/*");
final HandlerList handlers = new HandlerList();
handlers.setHandlers(new Handler[]{wsContext, restContext, new DefaultHandler()});
server.setHandler(handlers);
Help...
Update (additional information per Joakim Erdfelt request):
I am at class HTTPConnection class, in BP at onComplete() method, fetching the request headers from the _channel._fields object I get:
The response object's status is 200 (and not 101 as expected):
My endpoint object is part of a large inheritance chain. It is full of boilerplate business logic code that I need to remove before I can upload it, but in the root stands the javax.websocket.Endpont class, where we implemented only the onOpen(Session session, EndpointConfig config) method. I am not getting to that method when debugging, seems to fail long before...

Answer 1

Your request headers looks like this ...
Accept: application/json, application/*+json
Accept-Encoding: gzip
Cache-Control: no-cache
Connection: keep-alive
Content-Type: application/json
Host: 127.0.0.1:8080
Pragma: no-cache
Sec-WebSocket-Key: sMQPm6Cf00itLII3QBb4w==
Sec-WebSocket-Version: 13
Upgrade: websocket
User-Agent: Java/1.8.0_144
That is an invalid WebSocket Upgrade Request.
The most glaring omission is
Connection: upgrade
But there's also other fields that a compliant WebSocket Client would never set.
Content-Type: application/json
Accept: application/json, application/*+json
Accept-Encoding: gzip