I have some methods registered to a type in server program for RPC in Golang. I want to use those methods from clients as well as the server itself. e.g. I have an addition method registered as RPC method which I want to invoke from clients. But I want to call the same method in the same server program for some other function too. Is there a way to do this?
Difficult to answer, if we don't know how you build your code with some exemple it will be easier.
Two choices.
Create client with goroutine, set your connection to the server and call the method. This method is bad, wtf.
Second :
type CalcServ struct {
LastAnswer
}
type CalcReq struct {
X int
Y int
}
type CalcResp struct {
Z int
}
func (serv *CalcServ) Add(req *CalcReq, resp *CalcResp) error {
serv.LastAnswer = req.X + req.Y
serv.Show(req, resp)
return nil
}
func (serv *CalcServ) Show(req *CalcReq, resp *CalcResp) error {
resp.Z = serv.LastAnswer
return nil
}
My exemple is not a good implementation, but it's just to show the idea to you.
There are Two RPC method, Add and Show, and Add call Show
I had to do this myself. This is a contrived example showing what I did in my case:
This is the RPC function...
func (server *Server) RemoteGoroutineCount(request GoroutineCountsRequest, response *GoroutineCountsResponse) error {
response.Count = runtime.NumGoroutine()
return nil
}
Now, if I want to call that locally, I can just do
req := GoroutineCountsRequest{}
res := &GorouteCountsResponse{}
s := &Server{}
err := s.RemoteGoroutineCount(req, res)
// handle error
// Now res has your filled in response
res.NumGoroutines
Related
I have the following gRPC interceptor running serverside, which wraps a serverstream and passes it on to the next handler:
// HarmonyContext contains a custom context for passing data from middleware to handlers
type HarmonyContext struct {
context.Context
Request interface{}
UserID uint64
Limiter *rate.Limiter
}
type IHarmonyWrappedServerStream interface {
GetWrappedContext() HarmonyContext
}
type HarmonyWrappedServerStream struct {
grpc.ServerStream
WrappedContext HarmonyContext
}
func (ss HarmonyWrappedServerStream) GetWrappedContext() HarmonyContext {
return ss.WrappedContext
}
func (m Middlewares) HarmonyContextInterceptorStream(srv interface{}, ss grpc.ServerStream, info *grpc.StreamServerInfo, handler grpc.StreamHandler) error {
wrapped := WrapServerStream(ss)
return handler(srv, wrapped)
}
func WrapServerStream(stream grpc.ServerStream) HarmonyWrappedServerStream {
if existing, ok := stream.(HarmonyWrappedServerStream); ok {
return existing
}
return HarmonyWrappedServerStream{ServerStream: stream, WrappedContext: HarmonyContext{
Context: stream.Context(),
}}
}
and in the handler itself, I have the following code:
func (v1 *V1) StreamGuildEvents(r *corev1.StreamGuildEventsRequest, s corev1.CoreService_StreamGuildEventsServer) error {
wrappedStream := s.(middleware.IHarmonyWrappedServerStream)
println(wrappedStream)
return nil
}
However, I get the following runtime error when sending a streaming request:
interface conversion: *corev1.coreServiceStreamGuildEventsServer is not middleware.IHarmonyWrappedServerStream: missing method GetWrappedContext
In fact, the ServerStream in the handler is completely different from the one in the interceptors. Is there any way to make the interceptor pass the custom ServerStream properly?
We've opened up one of our repos to demonstrate this if it helps anybody.
https://github.com/drud/api-common/blob/main/interceptors/state.go#L207
Very similar to #Bluskript impl however I do not have the same issue encountered above. I believe this might be from the return type grpc.StreamServerInterceptor and where I am using the context getter.
I am trying to build an abstraction for an AWS service ( ECR ). Here is the code:
type ECR struct {
Client ecriface.ECRAPI
Ctx context.Context
}
// ECRCreate establishes aws session and creates a repo with provided input
func (e *ECR) ECRCreate(ecrInput *ecr.CreateRepositoryInput) {
result, err := e.Client.CreateRepositoryWithContext(e.Ctx, ecrInput)
if err != nil {
if aerr, ok := err.(awserr.Error); ok {
switch aerr.Code() {
case ecr.ErrCodeServerException:
log.Errorln(ecr.ErrCodeServerException, aerr.Error())
case ecr.ErrCodeInvalidParameterException:
log.Errorln(ecr.ErrCodeInvalidParameterException, aerr.Error())
case ecr.ErrCodeInvalidTagParameterException:
log.Errorln(ecr.ErrCodeInvalidTagParameterException, aerr.Error())
case ecr.ErrCodeTooManyTagsException:
log.Errorln(ecr.ErrCodeTooManyTagsException, aerr.Error())
case ecr.ErrCodeRepositoryAlreadyExistsException:
log.Errorln(ecr.ErrCodeRepositoryAlreadyExistsException, aerr.Error())
case ecr.ErrCodeLimitExceededException:
log.Errorln(ecr.ErrCodeLimitExceededException, aerr.Error())
case ecr.ErrCodeKmsException:
log.Errorln(ecr.ErrCodeKmsException, aerr.Error())
default:
log.Errorln(aerr.Error())
}
} else {
// Print the error, cast err to awserr.Error to get the Code and
// Message from an error.
log.Errorln(err.Error())
}
return
}
log.Infof("Result: %v", result)
}
To mock aws sdk create repository call:
type mockECRClient struct {
ecriface.ECRAPI
}
func (m *mockECRClient) CreateRepositoryWithContext(ctx aws.Context, input *ecr.CreateRepositoryInput, opts ...request.Option) (*ecr.CreateRepositoryOutput, error) {
createdAt := time.Now()
encryptionType := "AES256"
//awsMockAccount := "974589621236"
encryptConfig := ecr.EncryptionConfiguration{EncryptionType: &encryptionType}
imageScanConfig := input.ImageScanningConfiguration
mockRepo := ecr.Repository{
CreatedAt: &createdAt,
EncryptionConfiguration: &encryptConfig,
ImageScanningConfiguration: imageScanConfig,
}
mockRepoOuput := ecr.CreateRepositoryOutput{Repository: &mockRepo}
return &mockRepoOuput, nil
}
func TestECR_ECRCreate(t *testing.T) {
ctx := context.TODO()
mockSvc := &mockECRClient{}
scan := true
name := "Test1"
inputTest1 := ecr.CreateRepositoryInput{
RepositoryName: &name,
ImageScanningConfiguration: &ecr.ImageScanningConfiguration{ScanOnPush: &scan},
}
ecrTest := ECR{
mockSvc,
ctx,
}
ecrTest.ECRCreate(&inputTest1)
}
And this works. However, I am a bit confused around the usage of interface & composition here. ECRAPI is defined by ecriface package and I implement one of the signatures of the interface and still able to use mocked client mockSvc.
Questions:
How does this work? Is this idiomatic way of mocking interfaces?
What about the other methods of the ECRAPI interface? How are those taken care of ?
Is my understanding then correct, that we can define an interface with arbitrary number of method signatures, embed that interface into a struct and then pass the struct around where ever interface is expected. But then this means, I skip implementing other signatures of my interface?
I am thinking I am missing some important concept here, please advise!
The short answer is: this works because the function tested only calls the method you explicitly defined. It'll fail if it calls anything else.
This is how it happens:
The mockECRClient struct embeds the interface, so it has all the methods of that interface. However, to call any of those methods, you have to set that interface to an implementation:
x:=mockECRClient{}
x.ECRAPI=<real implementation of ECRAPI>
A call to x.Func() where Func is defined for ECRAPI will actually call x.ECRAPI.Func(). Since you didn't set ECRAPI, x.ECRAPI above is nil, and any method you call that uses the receiver will panic.
However, you defined a method for mockECRClient: CreateRepositoryWithContext. When you call x.CreateRepositoryWithContext, the method belongs to x and not to x.ECRAPI, and the receiver will be x, so it works.
type Request struct{
A string
B string
C string
D string
//...
}
func validator(req *Request)error{
if req.A == "" && req.B != ""{
return errors.New("Error 1 !!")
}
//...
}
I have some existing code like above which is already being used so I can not change the function signature.
I am writing a caller function which has to throttle some types of errors.
All existing errors are created using either errors.New("some string") or fmt.Errorf("some string").
What I can do is
if err.Error() == "Error 1 !!" {
return nil
}
But this is not ideal. If on the server side, the message changes, client side breaks.
I thought about naming all the errors on the server side like:
const ErrorType1 = "Error 1 !!"
But it's difficult to name each error.
Any better solutions?
error is an interface, so you can dynamically check - using type assertions - some specific types and act accordingly.
Here's a code snippet that may be useful (playground link):
package main
import (
"errors"
"fmt"
)
type MyError struct {
msg string
id int
}
func (me MyError) Error() string {
return me.msg
}
func checkErr(e error) {
if me, ok := e.(MyError); ok {
fmt.Println("found MyError", me)
} else {
fmt.Println("found error", e)
}
}
func main() {
checkErr(errors.New("something bad"))
checkErr(MyError{msg: "MyError bad"})
}
The first line in checkErr is the ticket here - it checks if e is of some special underlying type.
While the use of typed errors surely has its place and can be used, there are different approaches.
As for the validation part, I prefer not to reinvent the wheel and use [govalidator][gh:govalidator]. Custom validations can be easily added and if your validation needs are not rather complicated, it may already give you what you need.
However, as per the second part your question consists of, the elegant handling of errors, there is an alternative to an implementation of the Error interface: predefined variables which you can directly compare, as shown in the switch statement in the example program below.
package main
import (
"errors"
"log"
"github.com/asaskevich/govalidator"
)
// Error Examples
var (
NoUrlError = errors.New("Not an URL")
NotLowerCaseError = errors.New("Not all lowercase")
)
func init() {
govalidator.SetFieldsRequiredByDefault(true)
}
// Request is your rather abstract domain model
type Request struct {
A string `valid:"-"`
B string `valid:"uppercase"`
// Note the custom error message
C string `valid:"url,lowercase~ALL lowercase!!!"`
D string `valid:"length(3|10),lowercase"`
E string `valid:"email,optional"`
}
// Validate takes the whole request and validates it against the struct tags.
func (r Request) Validate() (bool, error) {
return govalidator.ValidateStruct(r)
}
// ValidC does a custom validation of field C.
func (r Request) ValidC() (bool, error) {
if !govalidator.IsURL(r.C) {
return false, NoUrlError
}
if !govalidator.IsLowerCase(r.C) {
return false, NotLowerCaseError
}
return true, nil
}
func main() {
// Setup some Requests
r1 := Request{C: "asdf"}
r2 := Request{C: "http://www.example.com"}
r3 := Request{C: "http://WWW.EXAMPLE.com"}
r4 := Request{B: "HELLO", C: "http://world.com", D: "foobarbaz", E: "you#example.com"}
for i, r := range []Request{r1, r2, r3, r4} {
log.Printf("=== Request %d ===", i+1)
log.Printf("\tValidating struct:")
// Validate the whole struct...
if isValid, err := r.Validate(); !isValid {
log.Printf("\tRequest %d is invalid:", i+1)
// ... and iterate over the validation errors
for k, v := range govalidator.ErrorsByField(err) {
log.Printf("\t\tField %s: %s", k, v)
}
} else {
log.Printf("\t\tValid!")
}
log.Println("\tValidating URL")
valid, e := r.ValidC()
if !valid {
switch e {
// Here you got your comparison against a predefined error
case NoUrlError:
log.Printf("\t\tRequest %d: No valid URL!", i)
case NotLowerCaseError:
log.Printf("\t\tRequest %d: URL must be all lowercase!", i)
}
} else {
log.Printf("\t\tValid!")
}
}
}
Imho, a custom implementation only makes sense if you want to add behavior. But then, this would first call for a custom interface and an according implementation as a secondary necessity:
package main
import (
"errors"
"log"
)
type CustomReporter interface {
error
LogSomeCoolStuff()
}
type PrefixError struct {
error
Prefix string
}
func (p PrefixError) LogSomeCoolStuff() {
log.Printf("I do cool stuff with a prefix: %s %s", p.Prefix, p.Error())
}
func report(r CustomReporter) {
r.LogSomeCoolStuff()
}
func isCustomReporter(e error) {
if _, ok := e.(CustomReporter); ok {
log.Println("Error is a CustomReporter")
}
}
func main() {
p := PrefixError{error: errors.New("AN ERROR!"), Prefix: "Attention -"}
report(p)
isCustomReporter(p)
}
Run on playground
So, in short: If you want to make sure that the user can identify the kind of error use a variable, say yourpackage.ErrSomethingWentWrong. Only if you want to ensure a behavior implement a custom interface. Creating the type just for positively identifying a semantic value is not the way to do it. Again, imho.
I do it like:
normal cases:
request is well formated and server side handle it well.
status :200, body:{"message":"success"}
client bad request:
client sent a bad request , maybe lack of args.It should be fixed by your client mate, and avoid appearing when online.
status:400, body: {"message":"error reasson"}
client normal request but not success:
maybe users use api to get award times more than default value.The request is normal but should be limit.
status:200, body: {"message":"success", "tip":"Dear,you've finish it", "tip_id":1}
server inner error:
some bugs or unnavoided error happened on server side.
status：500 body: {"message":"error_stack_trace", "error_id":"XKZS-KAKZ-SKD-ZKAQ"}
Above all, client should divide response.status into three possible values(200,400,500) and has different handle ways.
On 200 case, show anythings client want or tip.
On 400 case, show message.
On 500 case, show error_id.
I am using JSON-RPC over Websocket. And, in an RPC method (say, Multiply in the example below), I need to know which connection called this method. The part below that says "// Need Websocket connection information here". How do I do so?
package main
import (
"code.google.com/p/go.net/websocket"
"net/http"
"net/rpc"
"net/rpc/jsonrpc"
)
type Args struct {
A int
B int
}
type Arith int
func (t *Arith) Multiply(args *Args, reply *int) error {
*reply = args.A * args.B
// Need Websocket connection information here
return nil
}
func main() {
rpc.Register(new(Arith))
http.Handle("/conn", websocket.Handler(serve))
http.ListenAndServe("localhost:7000", nil)
}
func serve(ws *websocket.Conn) {
jsonrpc.ServeConn(ws)
}
This will be challenging because it violates the abstraction that RPC provides. Here's a strategy suggestion:
Google uses a context object for lots of their APIs: https://blog.golang.org/context. Part of that interface is a Value method for arbitrary data:
Value(key interface{}) interface{}
That will give you something like thread-local storage, which is often used for this purpose in other programming languages.
So how do you add a context object to the request? One way would be to create a new, custom ServerCodec:
type ServerCodec interface {
ReadRequestHeader(*Request) error
ReadRequestBody(interface{}) error
// WriteResponse must be safe for concurrent use by multiple goroutines.
WriteResponse(*Response, interface{}) error
Close() error
}
Your implementation can mostly mirror jsonrpc's:
var params [1]interface{}
params[0] = x
return json.Unmarshal(*c.req.Params, ¶ms)
But before returning, you can use a bit of reflection and look for a field in params with name/type Context and then fill it. Something like:
ctx := createContextSomehow()
v := reflect.ValueOf(x)
if v.Kind() == reflect.Ptr {
v = v.Elem()
if v.Kind() == reflect.Struct {
ctxv := v.FieldByName("Context")
ctxv.Set(ctx)
}
}
Then change your request:
type Args struct {
A int
B int
}
Change it to:
type Args struct {
A int
B int
Context context.Context
}
Kinda clumsy but I think you could make that work.
I'm currently writing against an xml service in which batched results are returned. I currently have the following:
type QueryEnvelope struct {
XMLName xml.Name `xml:"http://schemas.xmlsoap.org/soap/envelope/ Envelope"`
Body *QueryBody `xml:"http://schemas.xmlsoap.org/soap/envelope/ Body"`
}
type QueryBody struct {
QueryResult *QueryResult `xml:"queryResponse>result"`
}
type QueryResult struct {
QueryLocator QueryLocator `xml:"queryLocator"`
Done bool `xml:"done"`
Size int `xml:"size"`
Records interface{} `xml:"records"`
}
type QueryLocator string
func (r *Resource) Query(sql string, r interface{}) error {
// builds xml request
// sends xml request through Resource which contains the net client
// decodes the result into r (type for records in QueryResult)
}
func (r *Resource) QueryMore(q QueryLocator, r interface{}) error {
// same as above except takes queryLocator and makes call to different endpoint in order to continue retrieving results, when QueryResult.Done == true, done fetching results
}
Obviously this needs refactored since the Client needs to see whether Done == true so they can continue to fetch. I was thinking of adding the following the following, and moving Query and QueryMore to be a method of Querier:
type Querier struct {
r *Resource
Done bool
QueryLocator QueryLocator
}
func New(r *Resource) *Querier {}
Client side would then behave as such:
err := q.Query("sql statement", r)
if err != nil {
// handle
}
// do stuff with r
for q.Done == false {
q.QueryMore(r)
// do stuff with r
}
I'm wondering what the idiomatic approach would be here in order to best "stream" the results.
One option is to use the pattern the stdlib sql package uses to iterate over rows. This way you can unify the initial query with subsequent calls.
There's a Next() method, which populates the internal structure with the query results, and returns true if there's a result pending. In your case, you would still need an initial constructor, like Query, but that would just setup the data-structure. Calling Next() does the real work, and calling Scan(r) (or whatever you'd like to call reading the result) just presents to result. Once iteration is done, you have an Err() method to check for any operational errors.
Altering the example from the sql package a bit:
// setup the query, but delay any action until we start iterating.
query, err := NewQuery(queryParameters)
// check err of course
defer query.Close()
for query.Next() {
err = query.Scan(&r)
...
}
err = query.Err() // get any error encountered during iteration
You can also check out other drivers like mgo for variations on this pattern.