I am writing an application in Go that implements a gRPC API.
Here is the ProtoBuff specification that describes my endpoint's response message:
rpc MyEndpoint(RequestMessage) returns (ResponseMessage) {
option (google.api.http) = {
post: "/MyEndpoint"
body: "*"
};
}
message ResponseMessage {
message SubMessage {
int32 a = 1;
map<string, int32> b = 2;
}
string c = 1;
map<string, SubMessage> d = 2;
}
I am writing a test case to verify the response from the endpoint is what I expected it to be.
Here is my code to do that:
expectedResponse := ResponseMessage{
C: "string1",
D: map[string]*ResponseMessage_SubMessage{
"string2": &ResponseMessage_SubMessage{
A: 1,
B: map[string]int32{
"string3": 2,
},
},
},
}
fmt.Println("responseFromEndpoint = ", responseFromEndpoint)
fmt.Println("expectedResponse = ", expectedResponse)
assert.Equal(t, expectedResponse, responseFromEndpoint) // This is line #230 in file my_test_file.go
When I run this test-case, I get this output:
responseFromEndpoint = map[c:string1 d:map[string2:map[a:1 b:map[string3:2]]]]
expectedResponse = {{{} [] [] <nil>} 0 [] string1 map[string2:a:1 b:{key:"string3" value:2}]}
And the test case fails like this:
MyTestClass/MyTestCase: my_test_file.go:230:
Error Trace: my_test_file.go:230
Error: Not equal:
expected: ResponseMessage(ResponseMessage{state:impl.MessageState{NoUnkeyedLiterals:pragma.NoUnkeyedLiterals{}, DoNotCompare:pragma.DoNotCompare{}, DoNotCopy:pragma.DoNotCopy{}, atomicMessageInfo:(*impl.MessageInfo)(nil)}, sizeCache:0, unknownFields:[]uint8(nil), Name:"string1", SubMessages:map[string]*ResponseMessage_SubMessage{"string2":(*ResponseMessage_SubMessage)(0xc000098180)}})
actual : map[string]interface {}(map[string]interface {}{"c":"string1", "d":map[string]interface {}{"string2":map[string]interface {}{"a":1, "b":map[string]interface {}{"string3":2}}}})
How can I modify my test case so that I can ensure the endpoint responded with the data that I expected it to?
Related
I have proto file like this:
message StartAssignmentRequest {
string additional_comment = 3;
repeated RideSlip slips = 4;
}
message RideSlip{
string slip_name = 2;
string slip_ext = 3;
string slip_link = 4;
}
Now I want to create its request and I am doing something like this:
req := &api.StartAssignmentRequest{
AdditionalComment:"AdditionalComment",
Slips: &api.RideSlip[],
}
but not having idea how I can send RideSlip data properly.
Protobuffer (both 2 and 3) repeated fields are compiled to slices in Go.
Just append to it:
req := &api.StartAssignmentRequest{
AdditionalComment: "AdditionalComment",
}
req.Slips = append(req.Slips, &api.RideSlip{
SlipName: "foo",
SlipExt: "bar",
SlipLink: "https://stackoverflow.com",
})
Or assign to it a literal value:
req := &api.StartAssignmentRequest{
AdditionalComment: "AdditionalComment",
Slips: []*api.RideSlip{
{
SlipName: "foo",
SlipExt: "bar",
SlipLink: "https://stackoverflow.com",
},
},
}
I have the following protobuf definition:
service MyService {
rpc ServiceMethod (ServiceMethodRequest) returns (ServiceMethodResponse) {}
}
message ServiceMethodRequest{
string requestParam = 1;
}
message ServiceMethodResponse{
Error error = 1;
SomeObjectList data = 2;
}
message Error{
string code = 1;
string errorMessage = 2;
}
message SomeObject {
string myobject = 1;
}
message SomeObjectList {
repeated SomeObject myobjects = 1;
}
As you can see I want to return formatted response so my API has some standardized way of responding. I formatted my response like this (this is in JSON format because of readability):
{"error":{"code":"-1","errorMessage":""},"data":{"myobjects":[{"myobject":"some string"},{"myobject":"another string"}]}}
But on the client side I am constantly getting:
{ Error: 2 UNKNOWN: Unknown Error
at Object.exports.createStatusError (.../node_modules/grpc/src/common.js:91:15)
at Object.onReceiveStatus (.../node_modules/grpc/src/client_interceptors.js:1204:28)
at InterceptingListener._callNext (.../node_modules/grpc/src/client_interceptors.js:568:42)
at InterceptingListener.onReceiveStatus (.../node_modules/grpc/src/client_interceptors.js:618:8)
at callback (.../node_modules/grpc/src/client_interceptors.js:845:24)
code: 2,
metadata: Metadata { _internal_repr: {}, flags: 0 },
details: 'Unknown Error' }
when I try to console.log response object.
What am I doing wrong?
Also, is there any good book on grpc with some examples (C, C#, Java, Python, JavaScript)?
Thanks in advance!
server.js
callback(null, message);
I am using Elastic search 6.1 version
My data is appending correctly and I am adding '\n' at the end of the request.
My code is as follows:
def insert_in_bulk(self, filee, rtype):
U = urljoin(self.args.host, "/_bulk")
body = []
f = open(filee)
for line in f:
action = {
'index' :{
'_index' : self.args.index,
'_type' : rtype,
}
}
item = {
'word' : line.strip()
}
body.append(json.dumps(action))
body.append(json.dumps(item))
f.close()
body = '\n'.join(body)+'\n'
success = False
try:
r = requests.post(U, data=body)
self.log.info("after request")
if r.status_code == 200:
success = True
r = r.json()
self.log.info("inserted %s items of type = %s", self.args.index , rtype)
except (SystemExit, KeyboardInterrupt): raise
except:
self.log.exception("during bulk index")
if not success:
self.log.error("failed to index records of type = %s", rtype)
I am using the python to connect to elastic search.
I got the answer from this link
Bulk index document from JSON file into ElasticSearch
I have to pass the header to the request as application/x-ndjson.
Though it is quite some time question is asked, but i want to give a solution that has worked for me in most case,
def insert_in_bulk(self, filee, rtype):
U = urljoin(self.args.host, "/_bulk")
body = []
f = open(filee)
for line in f:
action = {
'index' :{
'_index' : self.args.index,
'_type' : rtype,
}
}
item = {
'word' : line.strip()
}
body.append(json.dumps(action))
body.append(json.dumps(item))
f.close()
payload = ""
for l in body:
payload = payload + f"{l} \n"
data = payload.encode('utf-8')
r = requests.post(U, data=data, headers={"Content-Type": "application/x-ndjson"})
print(r.text)
I have a string
var str = "1 2 3 4"
and I want to convert it into [Int]. It can be done as follow
let intArray = str.characters.split {$0 == " "}.map(String.init).map { Int($0)!}
Now what if my string is
var invalid = " 1 a 4"
Then, the program will crash with
fatal error: unexpectedly found nil while unwrapping an Optional value
I need to be able to check number and throw number format error in map.
You can use throws, try - throw, do - try - catch and guard (if) for that. Here is the code
var invalid = " 1 a 4"
let intArray: [Int]
do {
intArray = try getIntArray(invalid, delimiter: " ")
}catch let error as NSError {
print(error)
intArray = []
}
func getIntArray(input:String, delimiter:Character ) throws -> [Int] {
let strArray = input.characters.split {$0 == delimiter}.map(String.init)
let intArray = try strArray.map {
(int:String)->Int in
guard Int(int) != nil else {
throw NSError.init(domain: " \(int) is not digit", code: -99, userInfo: nil)
}
return Int(int)!
}
return intArray
}
In getIntArray function, We first convert the input string to string array.
Then when we are converting string array to int array, we are expanding the map closure function parameter to include number format checking and throwing error using "guard".
"guard" can be replaced with "if" too if it is not available
if Int(int) == nil {
throw NSError.init(domain: " \(int) is not digit", code: -99, userInfo: nil)
}
Rather than throwing NSError types, you can create your own Swift native enum conforming to ErrorType where your enumeration contains the error case you would like to explicitly handle. E.g.
enum MyErrors : ErrorType {
case NumberFormatError(String)
}
/* throwing function attempting to initialize an
integer given a string (if failure: throw error) */
func strToInt(str: String) throws -> Int {
guard let myInt = Int(str) else { throw MyErrors.NumberFormatError(str) }
return myInt
}
Example usage within a do-try-catch construct:
func strAsIntArr(str: String) -> [Int]? {
var intArr: [Int] = []
do {
intArr = try str.characters.split {$0 == " "}
.map(String.init)
.map { try strToInt($0) }
} catch MyErrors.NumberFormatError(let faultyString) {
print("Format error: '\(faultyString)' is not number convertible.")
// naturally you could rethrow here to propagate the error
} catch {
print("Unknown error.")
}
return intArr
}
/* successful example */
let myStringA = "1 2 3 4"
let intArrA = strAsIntArr(myStringA)
/*[1, 2, 3, 4] */
/* error throwing example */
let myStringB = "1 b 3 4"
let intArrB = strAsIntArr(myStringB)
/* [], Format error: 'b' is not number convertible. */
I think I'm saving the image to Postgres correctly, but get unexpected results trying to load the image. I don't really know if the error is in save or load.
Here is my Anorm code for saving the image:
def storeBadgeImage(badgeHandle: String, imgFile: File) = {
val cmd = """
|update badge
|set img={imgBytes}
|where handle = {badgeHandle}
"""
var fis = new FileInputStream(imgFile)
var imgBytes: Array[Byte] = Resource.fromInputStream(fis).byteArray
// at this point I see the image in my browser if I return the imgBytes in the HTTP response, so I'm good so far.
DB.withConnection { implicit c =>
{
try {
SQL(cmd stripMargin).on("badgeHandle" -> badgeHandle, "imgBytes" -> imgBytes).executeUpdate() match {
case 0 => "update failed for badge " + badgeHandle + ", image " + imgFile.getCanonicalPath
case _ => "Update Successful"
}
} catch {
case e: SQLException => e.toString()
}
}
}
}
...I get "update succesful", so I presume the save is working (I could be wrong). Here is my code for loading the image:
def fetchBadgeImage(badgeHandle: String) = {
val cmd = """
|select img from badge
|where handle = {badgeHandle}
"""
DB.withConnection { implicit c =>
SQL(cmd stripMargin).on("badgeHandle" -> badgeHandle)().map {
case Row(image: Array[Byte]) => {
"image = " + image
}
case Row(Some(unknown: Any)) => {
println(unknown + " unknown type is " + unknown.getClass.getName) //[B#11be1c6 unknown type is [B
"unknown"
}
}
}
}
...rather than going into the case "Row(image: Array[Byte])" as hoped, it goes into the "Row(Some(unknown: Any))" case. My println outputs "[B#11be1c6 unknown type is [B"
I don't know what type [B is or where I may have gone wrong...
It's an array of byte in Java(byte[]). > "I don't know what type [B".
And You can write match { case Row(Some(image: Array[Byte])) => } too in this case and that might be better.
Or you might be able to do that as follows.
val results: Stream[Array[Byte]] = SQL(cmd stripMargin)
.on("badgeHandle" -> "name")().map { row => row[Array[Byte]]("img") }
...Oops, got the following compile error.
<console>:43: error: could not find implicit value for parameter c: anorm.Column[Array[Byte]]
val res: Stream[Array[Byte]] = SQL(cmd stripMargin).on("badgeHandle" -> "name")().map { row => row[Array[Byte]]("img") }
Unfortunately, scala.Array is not supported by default. If you imitate the way of other types, It works.
implicit def rowToByteArray: Column[Array[Byte]] = {
Column.nonNull[Array[Byte]] { (value, meta) =>
val MetaDataItem(qualified, nullable, clazz) = meta
value match {
case bytes: Array[Byte] => Right(bytes)
case _ => Left(TypeDoesNotMatch("..."))
}
}
}
val results: Stream[Array[Byte]] = SQL(cmd stripMargin)
.on("badgeHandle" -> "name")().map { row => row[Array[Byte]]("img") }
https://github.com/playframework/Play20/blob/master/framework/src/anorm/src/main/scala/anorm/Anorm.scala