I build a REST service generated by a proto file with rpc.
I succeed receiving a single event as follows:
rpc PostEvent(Event) returns (google.protobuf.Empty) {
option (google.api.http) = {
post: "/myEP"
body: "*"
};
}
and it works - converts a json object to Event{} struct.
My question is, how to do the same thing when I want to receive an array of Event{}s.
This could work:
message EventsWrapper {
repeated Event events = 1;
}
rpc PostEvents(EventsWrapper) returns (google.protobuf.Empty) {
option (google.api.http) = {
post: "/myEP"
body: "*"
};
}
But then it will expect a json like:
{"events":[{},..,{}]}
While I receive only:
[{},..,{}]
I don't control the way I receive the call. Any ideas how I can tweak my code to handle such an array call?
If you need to accept a JSON array, and the gRPC transcoding implementation you use supports it, you can use the body attribute to specify a repeated field that is mapped to the request body, instead of using *.
Quoting from the Google HttpRule API documentation:
If an API needs to use a JSON array for request or response body, it can map the request or response body to a repeated field. However, some gRPC Transcoding implementations may not support this feature.
body — The name of the request field whose value is mapped to the HTTP request body, or * for mapping all request fields not captured by the path pattern to the HTTP body, or omitted for not having any HTTP request body.
In other words, you would define your service as:
message EventsWrapper {
repeated Event events = 1;
}
// ...
rpc PostEvents(EventsWrapper) returns (google.protobuf.Empty) {
option (google.api.http) = {
post: "/myEP"
body: "events"
};
}
I believe this is supported by at least gRPC-Gateway (since PR #712) and likely also Envoy Proxy's gRPC-JSON transcoder. The latter also supports translating between JSON arrays and streaming gRPC methods, so if you define an rpc PostEvent(stream Event) returns (google.protobuf.Empty) method, Envoy will expect an array of Event objects in the request (and possibly even stream the translated messages as they arrive).
Related
I have a controller proxy api endpoint where it receives different request payloads which are intended to different services. This controller validates payload and adds few headers based on certain rules. In this current context, i do not want to parse the received response from upstream services. proxy method should simply stream response to downstream clients so that it can scale well without going into any memory issues when dealing with large response payloads.
I have implemented method like this:
suspend fun proxyRequest(
url: String,
request: ServerHttpRequest,
customHeaders: HttpHeaders = HttpHeaders.EMPTY,
): ResponseEntity<String>? {
val modifiedReqHeaders = getHeadersWithoutOrigin(request, customHeaders)
val uri = URI.create(url)
val webClient = proxyClient.method(request.method!!)
.uri(uri)
.body(request.body)
modifiedReqHeaders.forEach {
val list = it.value.iterator().asSequence().toList()
val ar: Array<String> = list.toTypedArray()
#Suppress("SpreadOperator")
webClient.header(it.key, *ar)
}
return webClient.exchangeToMono { res ->
res.bodyToMono(String::class.java).map { b -> ResponseEntity.status(res.statusCode()).body(b) }
}.awaitFirstOrNull()
}
But this doesn't seems to be streaming. When i try to download large file, it is complaining failed to hold large data buffer.
Can someone help me in writing reactive streamed approach?
This is what i have done finally.
suspend fun proxyRequest(
url: String,
request: ServerHttpRequest,
response: ServerHttpResponse,
customHeaders: HttpHeaders = HttpHeaders.EMPTY,
): Void? {
val modifiedReqHeaders = getHeadersWithoutOrigin(request, customHeaders)
val uri = URI.create(url)
val webClient = proxyClient.method(request.method!!)
.uri(uri)
.body(request.body)
modifiedReqHeaders.forEach {
val list = it.value.iterator().asSequence().toList()
val ar: Array<String> = list.toTypedArray()
#Suppress("SpreadOperator")
webClient.header(it.key, *ar)
}
val respEntity = webClient
.retrieve()
.toEntityFlux<DataBuffer>()
.awaitSingle()
response.apply {
headers.putAll(respEntity.headers)
statusCode = respEntity.statusCode
}
return response.writeWith(respEntity.body ?: Flux.empty()).awaitFirstOrNull()
}
Let me know if this is truly sending data downstream and flushing?
Your first code snippet fails with memory issues because it is buffering in memory the whole response body as a String and forwards it after. If the response is quite large, you might fill the entire available memory.
The second approach also fails because instead of returning the entire Flux<DataBuffer> (so the entire response as buffers), you're only returning the first one. This fails because the response is incomplete.
Even if you manage to fix this particular issue, there are many other things to pay attention to:
it seems you're not returning the original response headers, effectively changing the response content type
you should not forward all the incoming response headers, as some of them are really up to the server (like transfer encoding)
what happens with security-related request/response headers?
how are you handling tracing and metrics?
You could take a look at the Spring Cloud Gateway project, which handles a lot of those subtleties and let you manipulate requests/responses.
I am trying to get access token from MYOB. The POST call i make returns a "400 Bad Request error"
i'm using "axios" to make the POST call
i already got the Access code which i use in the data i'm sending in the POST call
here is my code
const config= { headers:{'Content-Type':"application/x-www-form-urlencoded"}}
const data={
client_id:"xxxxxxxxxxxxxxxxxxxxxxx",
client_secret:"xxxxxxxxxxxxxxxxxxxxx",
scope:"CompanyFile",
code: code,
redirect_uri:"http%3A%2F%2Flocalhost%3A30002Fcallback",
grant_type : "authorization_code"
}
axios.post("https://secure.myob.com/oauth2/v1/authorize", data, config)
.then((res) =>{
console.log ("response ...............", res
}
)
.catch((error) => {
console.error("Error here is ........",error)
}
)
Axios will, by default, attempt to POST your data fields as JSON which is not correct.
Instead, you want to url encode them and post the url-encoded string in the HTTP body. See the 'example call' in the docs.
There's a good example of how to url encode w/ axios here.
I also note that your redirect_uri field is already url encoded, so attempting to simply encode it a second time means you'll end up with something like http%253A%252F%252Flocalhost which is not correct. Double check your URL encoding against the example call to make sure you're not accidentally encoding certain fields twice. From memory the access code is already encoded appropriately so you might need to fiddle with decoding it before re-encoding it to get it working.
I'm using Spring Boot to create an API that needs to be consumed in Angular 4. Spring and Angular are on different ports.
The problem is that Spring's ResponseEntity raises an error in Angular.
#RequestMapping(value = "/{id}", method = RequestMethod.GET)
public ResponseEntity getFlow(#PathVariable int id) {
Flow flow = flowService.findById(id);
return new ResponseEntity(flow, HttpStatus.FOUND);
}
Now, I can perfectly use Postman to test the API and it works.
But when I make a request from Angular, it returns an error:
Strangely, it returns an error alongside the requested object.
Now, the cause of the problem is that the Spring Boot application returns a ResponseEntity and not a normal object (like String), and Angular doesn't know how to interpret it. If the controller returns just a Flow object, it works.
How can it be solved using ResponseEntity? Or, how else can I send the object alongside the HTTP status code?
Also, in #RequestMapping put produces = "application/json", and in get request in angular, add http options :
const httpOptions = {
headers: new HttpHeaders({
'Accept': 'application/json',
'Content-Type': 'application/json'
})
};
So your get request looks like this:
this.http.get(url, httpOptions)
As per the document mentioned here
https://docs.angularjs.org/api/ng/service/$http
A response status code between 200 and 299 is considered a success status and will result in the success callback being called. Any response status code outside of that range is considered an error status and will result in the error callback being called. Also, status codes less than -1 are normalized to zero. -1 usually means the request was aborted, e.g. using a config.timeout. Note that if the response is a redirect, XMLHttpRequest will transparently follow it, meaning that the outcome (success or error) will be determined by the final response status code.
As you are sending an instance of ResponseEntity(HttpStatus.Found) whose Http status code is 302 which doesnt fall under the success range thats why error callback is called.
Try returning the content like this
return new ResponseEntity(flow, HttpStatus.OK);
I am attempting to use a node based lambda function to return jpeg images from s3, using API Gateway.
My Lambda function reads as:
s3.getObject(params).promise().then((result) => {
let resp = {
statusCode: 200,
headers: {
'Content-Type': 'image/jpeg'
},
body: result.Body.toString('base64'),
isBase64Encoded: true
};
callback(null, resp);
});
I have also modified the integration response in API gateway to "Convert to binary (if needed)". When I try testing this function I receive the error "Execution failed due to configuration error: Unable to base64 decode the body.".
Is there a step I am missing to allow me to retrieve base64 encoded files?
I'm not sure about it, but have you tried to use this instead of the toString called directly on your object?
Buffer.from(result.Body).toString('base64')
Sounds like you're using AWS integration type of API Gateway instead of LAMBDA integration and in that case API Gateway would expect entire message to be base64 encoded, not just the body. For your use case you probably should use LAMBDA integration and return json with statusCode, body, headers, and Content-Type as you currently do.
How can you set the body of a POST request using the Ruby Mechanize gem. I know you can do
mechanize.post(url, query, headers)
but I want to set the body of the POST request with a JSON string. Is that possible? So, similar to something like this with jQuery:
$.ajax({
type: 'POST',
url: 'myurl',
data: "{'key1':'value1','key2':'value2'}",
...
});
I don't really like the answer you linked to in your comment because it employs to_json() which is a rails method, and the tags for your question do not indicate that your question pertains to rails. In any case, I think the answer needs some discussion.
Here is the mechanize method:
Mechanize#post(url, query, headers)
...and your stated goal is:
I want to set the body of the POST request
Mechanize#post() allows you to set the body of the request to anything you want, but you also have to consider the question:
What is the server side expecting?
You gave an example of a jquery ajax() request for what you want to do. jquery uses the following default Content-Type header when sending an ajax() request:
application/x-www-form-urlencoded; charset=UTF-8
That tells the server that the body of the post request is going to be written in a specific secret code. Well, it's not much of a secret; it looks like this:
name1=val1&name2=val2
That secret code's name is x-www-form-urlencoded. Because the server is given the name of the secret code in the Content-Type header, the server knows how to read the body of the post request.
In the Mechanize#post() method, the second parameter is 'query', and the mechanize docs say this about the query argument:
The query is specified by either a string, or
a list of key-value pairs represented by a hash, or
an array of arrays.
http://rubydoc.info/gems/mechanize/Mechanize#post-instance_method
If you want to use the secret code named x-www-form-urlencoded in the body of your Mechanize#post() request, then you can provide a Hash with name/value pairs, e.g.
my_hash = {
'data' => '{"key1":"value1","key2":"value2"}'
}
Then you call Mechanize#post() like this:
my_agent.post(
'http://target_site.com',
my_hash,
{'Content-Type' => 'application/x-www-form-urlencoded; charset=UTF-8'},
)
Then Mechanize will convert the 'query' Hash into a String using the secret code named x-www-form-urlencoded and insert the string into the body of the post request. On the server side, the application that receives the post request can retrieve the json string doing something like this:
json_str = post_variables['data']
You should be aware that there are other secret codes that can be used for the body of a post request. One of them is called json, which is a string formatted using javascript syntax, for example:
'{
"id": 1,
"name": "A green door",
"price": 12.50,
"tags": ["home", "green"]
}'
Note how there are no '=' signs or '&' symbols in the json format--as there are with the x-www-form-urlencoded format, so the json secret code is much different from the x-www-form-urlencoded secret code.
If you want to use the json secret code in the body of your post request, you need to change two things when you call Mechanize#post(url, query, headers):
Provide a String for the 'query' argument.
Tell the server that the body of the post request uses the json secret code.
Like this:
json_str = '{"key1":"value1","key2":"value2"}'
my_agent.post(
'http://target_site.com',
json_str,
{'Content-Type' => 'application/json'},
)
When you pass a String argument for the query parameter, Mechanize doesn't do any processing of the String before inserting the String into the body of the post request. On the server side, the application that receives the post request can retrieve the json string by doing something like this:
json_str = request.body.read
#Then probably:
hash = JSON.parse(json_str)
The one hitch is that the server can ignore the Content-Type header and try to read the body of the post request using a secret code that it has already decided upon. If the body of your post request is not written in the secret code that the server expects, then you will get an error.
Note that the 'data' string you posted isn't valid json because it uses single quotes around the properties and values.