I'm trying to use Websocket with my spring boot app powered by GCE.
I'm using the same websocket code than => https://github.com/spring-projects/spring-boot/tree/master/spring-boot-samples/spring-boot-sample-websocket-jetty
I'm using HTTPS load balancing and it's seems the load balancer does not like the https because it work on HTTP but not on HTTPS.
You can try my snake test game here (In simple HTTP) => http://www.writecontrol.fr/websocket/snake.html
There is an error if you test the same URL with HTTPS => https://www.writecontrol.fr/websocket/snake.html
I'm using ingress load balancer extensions/v1beta1
The SSL is based on a "secret tls" => https://cloud.google.com/endpoints/docs/openapi/enabling-ssl
So there is SSL between client and HTTPS load balancer, but not between load balancer and application instances
Logs into the application instance seems correct (No Exception) :
2017-12-20 20:07:04.986 INFO 5 --- [tp1875108260-24] monitoring : remoteAddr = 10.240.0.4, forwardedFor = 176.170.4.209, 35.190.37.177, request = /websocket/snake/info GET: 3 ms, 0 Ko
This is the log in GCE load balancing logging
httpRequest: {
remoteIp: "176.170.4.209"
requestMethod: "GET"
requestSize: "1940"
requestUrl: "https://www.writecontrol.fr/websocket/snake/370/5ho3g2ed/websocket"
responseSize: "414"
serverIp: "10.240.0.2"
status: 501
userAgent: "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.84 Safari/537.36"
}
insertId: "148f3zg42bp37f"
jsonPayload: {
#type: "type.googleapis.com/google.cloud.loadbalancing.type.LoadBalancerLogEntry"
statusDetails: "websocket_handshake_failed"
}
logName: "projects/writecontrol-1055/logs/requests"
receiveTimestamp: "2017-12-20T18:47:23.815984409Z"
resource: {
labels: {
backend_service_name: ""
forwarding_rule_name: "k8s-fws-default-gateway-preprod-ingress--69a82c51321bbe98"
project_id: "writecontrol-1055"
target_proxy_name: "k8s-tps-default-gateway-preprod-ingress--69a82c51321bbe98"
url_map_name: "k8s-um-default-gateway-preprod-ingress--69a82c51321bbe98"
zone: "global"
}
type: "http_load_balancer"
}
severity: "WARNING"
timestamp: "2017-12-20T18:47:22.935633707Z"
trace: "projects/writecontrol-1055/traces/79e8cc41423a19b5a52f6c432257a7d6"
Do you have any suggestions ?
Related
I'm sending hits to GA Measurement Protocol, and some of them do not make it to the GA. I've noticed that all of them have one thing in common: the user-agent is Firefox, only varying version and device. Some examples:
Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:103.0) Gecko/20100101 Firefox/103.0
Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Firefox/91.0
Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:103.0) Gecko/20100101 Firefox/103.0
Mozilla/5.0 (Android 10; Mobile; rv:103.0) Gecko/103.0 Firefox/103.0
GA validator is OK with those examples when checking them through the debug mode like this:
https://www.google-analytics.com/debug/collect?v=1&tid=UA-XXXXXXXX-1&t=event&ec=Ecommerce&ea=purchase&pa=purchase&cid=1234567890.1234567890&ni=1&ti=184242&tr=1060&uip=X.X.X.X&ua=Mozilla%2F5.0+%28Windows+NT+10.0%3B+Win64%3B+x64%3B+rv%3A103.0%29+Gecko%2F20100101+Firefox%2F103.0&pr1id=test_1&pr1pr=530&pr1qt=1&pr1ps=1
I get this response:
{
"hitParsingResult": [ {
"valid": true,
"parserMessage": [ ],
"hit": "/debug/collect?v=1..."
} ],
"parserMessage": [ {
"messageType": "INFO",
"description": "Found 1 hit in the request."
} ]
}
BUT in the production settings GA responses with 400 bad request error to the same requests without providing any details: "Your client has issued a malformed or illegal request. That’s all we know.".
So what might be wrong with Firefox UA?
UPD: I've managed to make this work by unsetting the 'User-Agent' header in case it contains 'Firefox' - and the corresponding 'ua' parameter in the payload gets accepted then.
if (strpos($requestHeaders['User-Agent'], 'Firefox') !== false) {
unset($requestHeaders['User-Agent']);
}
But it's still unclear what was wrong with such headers in the first place.
When using the SniffingConnectionPool it seems that Elasticsearch.net switches to port 9200 after the initial http.settings request?
I'm setting up the ConnectionPool with an IEnumerable as follows:
var nodes = cfg.Nodes.Select(x => x.Uri);
var pool = new SniffingConnectionPool(nodes);
The uris passed uses port 92. When debugging the requests, I can see that the first request is correctly made and we get 200 OK. However, the following HEAD request uses port 9200?
11 200 HTTP X.X:X.X:92 /_nodes/http,settings?flat_settings&timeout=500ms 5 121 application/json; charset=UTF-8
12 502 HTTP X.X.X.X:9200 / 512 no-cache, must-revalidate text/html; charset=UTF-8
Do I miss something? Worth to notice is that our cluster is reversed proxied by Nginx, and uses 9200/9300 to communicate internally.
Edit: The http property of http.settings looks like the following:
"http" : {
"bound_address" : [
"[::]:9200"
],
"publish_address" : "X.X.X.X:9200",
"max_content_length_in_bytes" : 104857600
}
Maybe the SniffingConnectionPool parses that content and starts using 9200?
We perform PUT request to our party using CXF JAX-RS client. Request body is empty.
A simple request invocation leads to server response with code 411.
Response-Code: 411
"Content-Length is missing"
Our party's REST-server requires Content-Length HTTP-header to be set.
We switched chunking off according to note about chunking but this did not solve the problem. The REST-server still answers with 411 error.
Here is our conduit configuration from cxf.xml file
<http-conf:conduit name="{http://myhost.com/ChangePassword}WebClient.http-conduit">
<http-conf:client AllowChunking="false"/>
</http-conf:conduit>
Line in the log confirms that execution of our request bound to our conduit configuration:
DEBUG o.a.cxf.transport.http.HTTPConduit - Conduit '{http://myhost.com/ChangePassword}WebClient.http-conduit' has been configured for plain http.
Adding Content-Length header explicitly also did not help.
Invocation.Builder builder = ...
builder = builder.header(HttpHeaders.CONTENT_LENGTH, 0);
A CXF Client's log entry confirms header setting, however when we sniffed packets, we have surprisingly found that header setting has been completely ignored by CXF client. Content-Length header was not sent.
Here is the log. Content-Length header is present:
INFO o.a.c.i.LoggingOutInterceptor - Outbound Message
---------------------------
ID: 1
Address: http://myhost.com/ChangePassword?username=abc%40gmail.com&oldPassword=qwerty123&newPassword=321ytrewq
Http-Method: PUT
Content-Type: application/x-www-form-urlencoded
Headers: {Accept=[application/json], client_id=[abcdefg1234567890abcdefg12345678], Content-Length=[0], Content-Type=[application/x-www-form-urlencoded], Cache-Control=[no-cache], Connection=[Keep-Alive]}
--------------------------------------
DEBUG o.apache.cxf.transport.http.Headers - Accept: application/json
DEBUG o.apache.cxf.transport.http.Headers - client_id: abcdefg1234567890abcdefg12345678
DEBUG o.apache.cxf.transport.http.Headers - Content-Length: 0
DEBUG o.apache.cxf.transport.http.Headers - Content-Type: application/x-www-form-urlencoded
DEBUG o.apache.cxf.transport.http.Headers - Cache-Control: no-cache
DEBUG o.apache.cxf.transport.http.Headers - Connection: Keep-Alive
And here is an output of the packet sniffer. Content-Length header is not present:
PUT http://myhost.com/ChangePassword?username=abc%40gmail.com&oldPassword=qwerty123&newPassword=321ytrewq HTTP/1.1
Content-Type: application/x-www-form-urlencoded
Accept: application/json
client_id: abcdefg1234567890abcdefg12345678
Cache-Control: no-cache
User-Agent: Apache-CXF/3.1.8
Pragma: no-cache
Host: myhost.com
Proxy-Connection: keep-alive
Does anyone know how actually disable chunking?
Here is our code:
public static void main(String[] args)
{
String clientId = "abcdefg1234567890abcdefg12345678";
String uri = "http://myhost.com";
String user = "abc#gmail.com";
Client client = ClientBuilder.newBuilder().newClient();
WebTarget target = client.target(uri);
target = target.path("ChangePassword").queryParam("username", user).queryParam("oldPassword", "qwerty123").queryParam("newPassword", "321ytrewq");
Invocation.Builder builder = target.request("application/json").header("client_id", clientId).header(HttpHeaders.CONTENT_LENGTH, 0);
Response response = builder.put(Entity.form(new Form()));
String body = response.readEntity(String.class);
System.out.println(body);
}
Versions:
OS: Windows 7 Enterprise SP1
Arch: x86_64
Java: 1.7.0_80
CXF: 3.1.8
I had a very similar issue that I was not able to solve as you did by trying to turn off chunking.
What I ended up doing was setting the Content-Length to 1 and adding some white space " " as the body. For me it seemed that the proxy servers before the server application was rejected the request and by doing that got me past the proxy servers and the server was able to process the request as it was only operating based on the URL.
I'm trying to initiate a websocket connection between chrome browser client and server.
Overview of my implementation :
There are set of different up and running projects. The main project is the hub to all other projects and it handle all http requests, routes and proxy to other sub projects. These all projects use load balancers. My attempt is to create a websocket connection from chrom browser to one sub project.
caddy version : 0.9.3
websocket library : github.com/gorilla/websocket
The main project's caddy configs :
https://{$DOMAIN_NAME}/analytics/ {
tls ../resources/security/server.pem ../resources/security/server.key
proxy / https://localhost:8107/analytics {
websocket
insecure_skip_verify
}
}
The sub project's caddy configs :
localhost:{$ANALYTICS_CADDY_PORT}/analytics {
root webapps/analytics
gzip
ext .html
tls {$ANALYTICS_CERTIFICATE_FILE} {$ANALYTICS_KEY_FILE}
proxy /api https://localhost:{$ANALYTICS_HTTPS_PORT} {
websocket
insecure_skip_verify
}
}
Inside the analytics sub project, " /api/ws " would trigger CreateSocketConnection() method.
//Starting the API server
router := routes.NewRouter()
http.Handle("/", router)
http.HandleFunc("/api/ws", api.CreateSocketConnection)
CreateSocketConnection implementation :
func CreateSocketConnection(w http.ResponseWriter, r *http.Request) {
var upgrader = websocket.Upgrader{
ReadBufferSize: 1024,
WriteBufferSize: 1024,
}
_, err = upgrader.Upgrade(w, r, nil)
if err != nil {
log.Fatal("upgrader failed :", err.Error())
}
//controllers.HandleSocket(ws)
}
Client side implementation :
conn = new WebSocket("wss://xxxx.com/analytics/api/ws");
Issue is I'm not getting any error log in backend, but the socket connection fails on browser.
WebSocket connection to 'wss://xxxx.com/analytics/api/ws' failed: Error during WebSocket handshake: Unexpected response code: 502
Request header :
Accept-Encoding:gzip, deflate, sdch, br
Accept-Language:en-US,en;q=0.8
Cache-Control:no-cache
Connection:Upgrade
Cookie:username=admin; tenantid=1; tenantdomain=super.com;
DNT:1
Host:xxxx.com
Origin:https://xxxx.com
Pragma:no-cache
Sec-WebSocket-Extensions:permessage-deflate; client_max_window_bits
Sec-WebSocket-Key:O/DS1lRHzXptoWz5WR131A==
Sec-WebSocket-Version:13
Upgrade:websocket
User-Agent:Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.59 Safari/537.36
But the response header is as follows :
Content-Encoding:gzip
Content-Length:40
Content-Type:text/plain; charset=utf-8
Date:Sat, 29 Oct 2016 03:13:23 GMT
Server:Caddy
Vary:Accept-Encoding
X-Content-Type-Options:nosniff
Please note that I'm getting the request header inside the CreateSocketConnection method as follows :
map[
Connection:[Upgrade]
X-Forwarded-For:[127.0.0.1, 127.0.0.1] Dnt:[1]
Origin:[https://xxxx.com]
Pragma:[no-cache]
Sec-Websocket-Extensions:[permessage-deflate; client_max_window_bits]
Sec-Websocket-Version:[13]
Accept-Encoding:[gzip]
User-Agent:[Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.59 Safari/537.36]
Cache-Control:[no-cache]
Sec-Websocket-Key:[O/DS1lRHzXptoWz5WR131A==]
Upgrade:[websocket]
Cookie:[username=admin; tenantid=1; tenantdomain=super.com; ]
Accept-Language:[en-US,en;q=0.8]]
Am I missing something in my implementation?
Thanks in advance
I had a similar issue, what I was missing was the transparent tag.
Ex.
https://{$DOMAIN_NAME}/analytics/ {
tls ../resources/security/server.pem ../resources/security/server.key
proxy / https://localhost:8107/analytics {
transparent
websocket
insecure_skip_verify
}
}
transparent specifies that all the headers should be sent with it, so this matters if you have authentication.
transparent:
Passes thru host information from the original request as
most backend apps would expect. Shorthand for:
header_upstream Host {host}
header_upstream X-Real-IP {remote} header_upstream X-Forwarded-For {remote}
header_upstream X-Forwarded-Port {server_port}
header_upstream X-Forwarded-Proto {scheme}
Source: https://caddyserver.com/docs/proxy
I want to run tests that change the user-agent in the http request sent from the browser (like the FF add-on, user agent switcher does). I saw you can do it by playing with the FF profile (http://seleniumhq.org/docs/09_webdriver.html).
Is there a way to do it within a test? Something like the function addCustomRequestHeader() that sets a header rather than adding it
You could insert a function like this to change the user agent on the fly before you make your http request:
function changeuserAgent() {
var altuserAgentGetter = function () {
return "Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2) Gecko/20100115 <choose your string>";
};
if (Object.defineProperty) {
Object.defineProperty(navigator, "userAgent", {
get: altuserAgentGetter
});
}
else if (Object.prototype.__defineGetter__) {
navigator.__defineGetter__("userAgent", altuserAgentGetter);
}
}
If you're using the Selenium 2 Web Driver in Java, you can create a Firefox profile and set the agent string as a preference in the profile. Then use the profile to create the WebDriver object:
FirefoxProfile profile = new FirefoxProfile();
profile.setPreference("general.useragent.override", "Mozilla/5.0 (iPad; U; CPU OS 4_3 like Mac OS X; de-de) AppleWebKit/533.17.9 (KHTML, like Gecko) Version/5.0.2 Mobile/8F191 Safari/6533.18.5");
WebDriver driver = new FirefoxDriver(profile);
For slightly more information and source code examples, see the Selenium Web Driver documentation for Firefox Driver at http://seleniumhq.org/docs/03_webdriver.html#firefox-driver.