I am using ballerina in order to make a proxy. I am trying to receive a message that exceeds the default websocket buffer limit of 65536.
I tried to find the parameter of the websocket configuration that allows me to set this size, but I can't find it in the official documentation. Right now the websocket is configured as follows:
ttp:WebSocketClient wsClientEp = new(
EXTRACTOR,
{callbackService: ClientService,
readyOnConnect: false
});
I get the following error on response:
2019-10-14 09:45:18,230 ERROR [] - Unexpected error hense closing the connection : error {ballerina/http}WsGenericError message=Max frame length of 65536 has been exceeded.
You can use maxFrameSize for this. Here's the documentation link
http:WebSocketClient wsClient = new(
EXTRACTOR,
{
callbackService: ClientService,
readyOnConnect: false,
maxFrameSize: <YOUR BUFFER SIZE>
}
)
Related
I have a use case where I need to send 2 requests to the server. The output of first request is used in second request so the calls have to be synchronous. I am using ktor (OkHttp)client websocket for this. I am failing at first attempt to even connect to the server with this error
Exception in thread "main" java.net.UnknownHostException: https: nodename nor servname provided, or not known
I suspect I haven't split my url properly and thats why its not able to connect to host.
Couple of qns
Is there any benefit to using websocket instead of using 2 separate Http requests?
Is there a way I can just pass URL to the websocket request?
Best and easiest way to get response and send another request?
I have been able to find very limited documentation on ktor client websocket.
const val HOST = "https://sample.com"
const val PATH1 = "/path/to/config?val1=<val1>&val2=<val2>"
const val PATH2 = "/path/to/config?val=<response_from_first_req>"
fun useSocket() {
val client = HttpClient() {
install(WebSockets)
}
runBlocking {
client.webSocket(method = HttpMethod.Get, host = HOST, path = PATH1) {
val othersMessage = incoming.receive() as? Frame.Text
println(othersMessage?.readText())
println("Testing")
}
}
client.close()
}
Thanks in advance.
I'm using serverless stack, now attempting to add a Lambda Custom Authenticator to validate authorization tokens with Auth0 and add custom data to my request context when the authentication passes.
Everything works mostly fine at this point, except for when I cache the Authenticator response for the same token.
I'm using a 5-second cache for development. The first request with a valid token goes through as it should. The next requests in the 5-second window fail with a mysterious 500 error without ever reaching my code.
Authorizer configuration
// MyStack.ts
const authorizer = new sst.Function(this, "AuthorizerFunction", {
handler: "src/services/Auth/handler.handler",
});
const api = new sst.Api(this, "MarketplaceApi", {
defaultAuthorizationType: sst.ApiAuthorizationType.CUSTOM,
defaultAuthorizer: new HttpLambdaAuthorizer("Authorizer", authorizer, {
authorizerName: "LambdaAuthorizer",
resultsCacheTtl: Duration.seconds(5), // <-- this is the cache config
}),
routes: {
"ANY /{proxy+}": "APIGateway.handler",
},
});
Authorizer handler
const handler = async (event: APIGatewayAuthorizerEvent): Promise<APIGatewayAuthorizerResult> => {
// Authenticates with Auth0 and serializes context data I wanna
// forward to the underlying service
const authentication = await authenticate(event);
const context = packAuthorizerContext(authentication.value);
const result: APIGatewayAuthorizerResult = {
principalId: authentication.value?.id || "unknown",
policyDocument: buildPolicy(authentication.isSuccess ? "Allow" : "Deny", event.methodArn),
context, // context has the following shape:
// {
// info: {
// id: string,
// marketplaceId: string,
// roles: string,
// permissions: string
// }
// }
};
return result;
};
CloudWatch logs
☝️ Every uncached request succeeds, with status code 200, an integration ID and everything, as it's supposed to. Every other request during the 5-second cache fails with 500 error code and no integration ID, meaning it doesn't reach my code.
Any tips?
Update
I just found this in an api-gateway.d.ts #types file (attention to the comments, please):
// Poorly documented, but API Gateway will just fail internally if
// the context type does not match this.
// Note that although non-string types will be accepted, they will be
// coerced to strings on the other side.
export interface APIGatewayAuthorizerResultContext {
[name: string]: string | number | boolean | null | undefined;
}
And I did have this problem before I could get the Authorizer to work in the first place. I had my roles and permissions properties as string arrays, and I had to transform them to plain strings. Then it worked.
Lo and behold, I just ran a test right now, removing the context information I was returning for successfully validated tokens and now the cache is working 😔 every request succeeds, but I do need my context information...
Maybe there's a max length for the context object? Please let me know of any restrictions on the context object. As the #types file states, that thing is poorly documented. This is the docs I know about.
The issue is that none of the context object values may contain "special" characters.
Your context object must be something like:
"context": {
"someString": "value",
"someNumber": 1,
"someBool": true
},
You cannot set a JSON object or array as a valid value of any key in the context map. The only valid value types are string, number and boolean.
In my case, though, I needed to send a string array.
I tried to get around the type restriction by JSON-serializing the array, which produced "[\"valueA\",\"valueB\"]" and, for some reason, AWS didn't like it.
TL;DR
What solved my problem was using myArray.join(",") instead of JSON.stringify(myArray)
I have a simple downstream service for file upload. Sample code
#RestController
#RequestMapping("/file")
public class FileController {
#PostMapping("/upload")
public ResponseEntity<?> uploadFile(#RequestParam("file") MultipartFile file,
#RequestParam(value = "delay", required = false, defaultValue = "0") int delay) throws Exception {
System.out.println(String.join(System.getProperty("line.separator"),
"File Name => " + file.getOriginalFilename(),
"File Size => " + file.getSize() + "bytes",
"File Content Type => " + file.getContentType()));
TimeUnit.MILLISECONDS.sleep(delay);
return ResponseEntity.ok(file.getName() + " uploaded");
}
}
and a CustomExceptionHandler that returns BAD_REQUEST if there is a MultipartException:
#Configuration
#ControllerAdvice
public class CustomExceptionHandler {
#ExceptionHandler(MultipartException.class)
public ResponseEntity<String> handleMultipartException(MultipartException ex) {
return ResponseEntity.status(HttpStatus.BAD_REQUEST).body(ex.getMessage());
}
}
The size limit is 10MB in application.yml:
spring:
servlet:
multipart:
max-file-size: 10MB
max-request-size: 10MB
If I upload a large file, it gives me a a 400 status as expected
When I try to hit the same via spring cloud gateway I get the following result:
and the logs shows following:
2019-11-08 00:36:10.797 ERROR 21904 --- [ctor-http-nio-2] a.w.r.e.AbstractErrorWebExceptionHandler : [86e57f7e] 500 Server Error for HTTP POST "/product-service/file/upload"
reactor.netty.http.client.PrematureCloseException: Connection has been closed BEFORE response, while sending request body
Note that the gateway is configured to take in large file size with RequestSize filter set globally to take way more than 10MB.
How can I get the same response code as given by the downstream service?
Also, I check with traditional Zuul, and i get a 500 error too.
For the gateway, for this particular case I know we can use the RequestSize filter and now the gateway will return the error code, but then we have to identify all the routes that expect this beforehand.
Also, other validation in the API, like authorization, etc will have the same the same issue. The response code produced because of these validations will not propagate up.
Sample code spring-cloud-gateway/product-service/eureka - https://github.com/dhananjay12/spring-cloud/tree/master/spring-routing
can you try to go through a non limitation of the volume of the file directly to without going through the getway? try the value -1 for the properties :
properties file of the MS where you want to upload the file
spring.servlet.multipart.max-file-size =-1
spring.servlet.multipart.max-request-size =-1
if it good, it may give a problem with the zuul proxy's ribbon socket size, there are properties informed for this type of situation, the following:
Properties file of the getway :
ribbon.eager-load.enabled=true
hystrix.command.default.execution.timeout.enabled=false
hystrix.command.default.execution.isolation.strategy=THREAD
hystrix.command.default.execution.isolation.thread.timeoutInMilliseconds=3999996
ribbon.ConnectTimeout=999999
ribbon.ReadTimeout=999999
ribbon.SocketTimeout=999999
zuul.host.socket-timeout-millis=999999
zuul.host.connect-timeout-millis=999999
zuul.sensitiveHeaders=Cookie,Set-Cookie
I am working with an API server implements a "server-push" feature by using an infinite chunked response. Each chunk in the response represents an message server pushed to client. Each chunk is actually a complete json object. Here is the code I am using as a client receiving the messages server pushed to.
Flux<JSONObject> jsonObjectFlux = client
.post(uriBuilder.expand("/data/long_poll").toString(), request -> {
String pollingRequest = createPollingRequest();
return request
.failOnClientError(false)
.failOnServerError(false)
.addHeader("Authorization", host.getToken())
.addHeader(HttpHeaders.CONTENT_TYPE, MediaType.APPLICATION_JSON_VALUE)
.addHeader(HttpHeaders.CONTENT_LENGTH,
String.valueOf(ByteBufUtil.utf8Bytes(pollingRequest)))
.sendString(Mono.just(pollingRequest));
}).flatMapMany(response -> response.receiveContent().map(httpContent -> {
ByteBuf byteBuf = httpContent.content();
String source = new String(ByteBufUtil.getBytes(byteBuf), Charsets.UTF_8);
return new JSONObject(source);
}));
jsonObjectFlux.subscribe(jsonObject -> {
logger.debug("JSON: {}", jsonObject);
});
However I got exception like:
reactor.core.Exceptions$ErrorCallbackNotImplemented: org.json.JSONException: Unterminated string at 846 [character 847 line 1]
Caused by: org.json.JSONException: Unterminated string at 846 [character 847 line 1]
at org.json.JSONTokener.syntaxError(JSONTokener.java:433)
at org.json.JSONTokener.nextString(JSONTokener.java:260)
at org.json.JSONTokener.nextValue(JSONTokener.java:360)
at org.json.JSONObject.<init>(JSONObject.java:214)
at org.json.JSONTokener.nextValue(JSONTokener.java:363)
at org.json.JSONObject.<init>(JSONObject.java:214)
Obviously, I am not getting a whole json data. I am wondering if using response.receiveContent() is the right way to deal with one chunk data.
I am trying to use the status to see if there is an error in C++ XPCOM component in my observer class on Mac.
OnStateChange(
nsIWebProgress *aWebProgress,
nsIRequest *aRequest,
PRUint32 aStateFlags,
nsresult aStatus)
{
}
In proxy environment the aStatus parameter is always true though the browser fails to load the page.
In non-proxy environments it gives the proper value (error) in status.
You can see it if you try accessing http://10.1.3.3/ (some random IP). With a proxy the status is zero (success) and without a proxy you get an error value.
Should some parameters be set to get the proper error value?
This is the expected behavior if you use an HTTP proxy. A non-zero aStatus means "didn't get any response because of some error". On the other hand, a zero aStatus value means "there was some response, check nsIHttpChannel.responseStatus to see whether the request was successful". That's what you get if the server responds with "404 Not Found" for example - aStatus will be zero (you got a response back) but nsIHttpChannel.responseStatus will be 404.
It's the same with an HTTP proxy because the proxy will always send a response back, probably "502 Bad Gateway" if it couldn't connect to the server. That's what the browser gets so aStatus will be zero and nsIHttpChannel.responseStatus will be 502. So in you code you should do something like this:
OnStateChange(
nsIWebProgress *aWebProgress,
nsIRequest *aRequest,
PRUint32 aStateFlags,
nsresult aStatus)
{
if (FAILED(aStatus))
{
// Got no response
}
else
{
nsCOMPtr<nsIHttpChannel> channel = do_QueryInterface(aRequest);
PRUint32 status = 0;
if (channel)
channel->GetResponseStatus(&status);
if (status >= 400)
{
// Got an HTTP error
}
else
{
// Success!
}
}
}