On refactoring singleton pattern with once.Do and Cloud Functions, I got caught in a case where service creation fails due to config error, which can be rectified by reuploading valid config, but when used with once.Do you would have to also redeploy Cloud Functions.
I wonder if there is a way to avoid CF redeployment or a different way of implementing the singleton pattern with once.Do.
Original singleton implementation:
var srv Service
//Singleton returns service
func Singleton(ctx context.Context) (Service, error) {
if srv != nil {
return srv, nil
}
config, err := NewConfig(ctx, shared.URL)
if err != nil {
return nil, err
}
srv, err = New(ctx, config)
return srv, err
}
refactored with run once
var once sync.Once
//Singleton returns service
func Singleton(ctx context.Context) (Service, error) {
var err error
onceBody := func() {
srv, err = newService(ctx)
}
once.Do(onceBody)
return srv, err
}
func newService(ctx context.Context) (Service, error) {
var config *Config
if config, err = NewConfig(ctx, shared.URL); err != nil {
return nil, err
}
return New(ctx, config)
}
Related
I'm new when it comes to Google PubSub(and pubsub applications in general). I'm also relatively new when it comes to Go.
I'm working on a pretty heavy backend service application that already has too many responsibilities. The service needs to fire off one message for each incoming request to a Google PubSub topic. It only needs to "fire and forget". If something goes wrong with the publishing, nothing will happen. The messages are not crucial(only used for analytics), but there will be many of them. We estimate between 50 and 100 messages per second for most of the day.
Now to the code:
func(p *publisher) Publish(message Message, log zerolog.Logger) error {
ctx := context.Background()
client, err := pubsub.NewClient(ctx, p.project)
defer client.Close()
if err != nil {
log.Error().Msgf("Error creating client: %v", err)
return err
}
marshalled, _ := json.Marshal(message)
topic := client.Topic(p.topic)
result := topic.Publish(ctx, &pubsub.Message{
Data: marshalled,
})
_, err = result.Get(ctx)
if err != nil {
log.Error().Msgf("Failed to publish message: %v", err)
return err
}
return nil
}
Disclaimer: p *publisher only contains configuration.
I wonder if this is the best way? Will this lead to the service creating and closing a client 100 times per second? If so, then I guess I should create the client once and pass it as an argument to the Publish()-function instead?
This is how the Publish()-function gets called:
defer func(publisher publish.Publisher, message Message, log zerolog.Logger) {
err := publisher.Publish(log, Message)
if err != nil {
log.Error().Msgf("Failed to publish message: %v", err)
}
}(publisher, message, logger,)
Maybe the way to go is to hold pubsubClient & pubsubTopic inside struct?
type myStruct struct {
pubsubClient *pubsub.Client
pubsubTopic *pubsub.Topic
logger *yourLogger.Logger
}
func newMyStruct(projectID string) (*myStruct, error) {
ctx := context.Background()
pubsubClient, err := pubusb.NewClient(ctx, projectID)
if err != nil {...}
pubsubTopic := pubsubClient.Topic(topicName)
return &myStruct{
pubsubClient: pubsubClient,
pubsubTopic: pubsubTopic,
logger: Logger,
// and whetever you want :D
}
}
And then for that struct create a method, which will take responsibility of marshalling the msg and sends it to Pub/sub
func (s *myStruct) request(ctx context.Context data yorData) {
marshalled, err := json.Marshal(message)
if err != nil {..}
res := s.pubsubTopic.Publish(ctx, &pubsub.Message{
Data: marshalled,
})
if _, err := res.Get(ctx); err !=nil {..}
return nil
}
How to handle error of singleton only once?
I have a singleton service which could generate error only at first call and then it returns already created instance.
Service looks like below:
package data
import (
"sync"
"gorm.io/driver/postgres"
"gorm.io/gorm"
)
var (
databaseSingleton *gorm.DB
once sync.Once
)
func NewDatabase() (*gorm.DB, error) {
once.Do(func() {
// ...
databaseSingleton, err = gorm.Open(postgres.Open(connectionString), config)
if err != nil {
return nil, err
}
})
return databaseSingleton, nil
}
The problem is multiple services which uses databaseSingleton above handle error which can occurs only once.
Services which uses databaseSingleton looks like below:
func NewServiceOne() (ServiceOne, error) {
database, err := NewDatabase()
// want omit this error handling
if err != nil {
return nil, err
}
return &serviceOne{database}, nil
}
func NewServiceTwo() (ServiceTwo, error) {
database, err := NewDatabase()
// want omit this error handling
if err != nil {
return nil, err
}
return &serviceTwo{database}, nil
}
func NewServiceThree() (ServiceThree, error) {
database, err := NewDatabase()
// want omit this error handling
if err != nil {
return nil, err
}
return &serviceThree{database}, nil
}
If there any way to omit this error handling because err could be generated only once?
If the error occurs (only once), your databaseSingleton will not be setup. You should return the error in all cases.
Although this isn't something you can do anything about (since the attempt to initialize databaseSingleton will not be repeated due to the use of sync.Once), you could as well halt the app.
In fact, there is no point deferring this initialization, you could just do it during package init, and terminate if it fails. And if it succeeds, you could use databaseSingleton without having to check error of the initialization.
So simply do it like this:
var databaseSingleton *gorm.DB
func init() {
var err error
databaseSingleton, err = gorm.Open(postgres.Open(connectionString), config)
if err != nil {
log.Fatalf("Failed to connect to DB: %v", err)
}
}
NewServiceOne() could look like this:
func NewServiceOne() ServiceOne {
return &serviceOne{databaseSingleton}
}
I have below snippet to server the graphql server over http listener. Could you help me how I can do similar implementation with GRPC server? I mean serving graphQL server over grpc server?
func Run() {
var cfg AppConfig
cfg.ProductURL = "0.0.0.0:8001"
err := envconfig.Process("", &cfg)
if err != nil {
log.Fatal(err)
}
s, err := NewGraphQLServer(cfg.ProductURL)
if err != nil {
log.Fatal(err)
}
http.Handle("/graphql", handler.GraphQL(gql.NewExecutableSchema(gql.Config{
Resolvers: s,
})))
http.Handle("/playground", handler.Playground("test", "/graphql"))
log.Fatal(http.ListenAndServe(":8080", nil))
}
// NewGraphQLServer is to create get the connections to other services
func NewGraphQLServer(productURL string) (*resolvers.Resolver, error) {
// Connect to Product Service
productClient, err := service.NewProductClient(productURL)
if err != nil {
return nil, err
}
return &resolvers.Resolver{
ProductClient: productClient,
}, nil
}```
I'm sure all the services are working properly.
I have the code below:
This snippet is used for registering two endpoints.
func RegisterEndpoints(ctx context.Context, c *utils.AppConfig, r resolver.Builder) (http.Handler, error) {
var err error
mux := runtime.NewServeMux()
dialOpts := []grpc.DialOption{grpc.WithBalancerName("round_robin"), grpc.WithInsecure()}
err = protos.RegisterUserCenterHandlerFromEndpoint(ctx, mux, r.Scheme()+"://author/user-center", dialOpts)
if err != nil {
return nil, err
}
err = protos.RegisterSsoHandlerFromEndpoint(ctx, mux, r.Scheme()+"://author/sso", dialOpts)
if err != nil {
return nil, err
}
return mux, nil
}
And in my main.go，I build a resolver to resolve name to address, then register the two endpoints and listen on port 8080.
func run() error {
c := utils.GetAppConfig()
ctx := context.Background()
ctx, cancel := context.WithCancel(ctx)
defer cancel()
r := localresolver.NewResolver(fmt.Sprintf("%s:%d", c.Registry.Host, c.Registry.Port))
resolver.Register(r)
mux := http.NewServeMux()
// Register endpoints here
gw, err := routes.RegisterEndpoints(ctx, c, r)
if err != nil {
return err
}
mux.Handle("/", gw)
fmt.Println("Listening localhost:8080...")
return http.ListenAndServe(fmt.Sprintf("%s:%d", c.Gateway.Host, c.Gateway.Port), mux)
}
func main() {
defer glog.Flush()
if err := run(); err != nil {
glog.Fatal(err)
}
}
But after I ran go run main.go, I found that only the last service I registered can be accessed, that is sso service (the err = protos.RegisterSsoHandlerFromEndpoint(ctx, mux, r.Scheme()+"://author/sso", dialOpts) line).
Can anyone show me an example of the correct way to register multiple endpoints via grpc-gateway? (make all the services registered with grpc-gateway can successfully be visited)
[2020-01-31] Need more help, now my code is like below:
Other code are same as before.
Additional, this is the result which name resolver shows:
There is no need to pass the ServeMux (gw) to mux var as handler, you can just ListenAndServe to the returned gw variable.
// Register endpoints here
gw, err := routes.RegisterEndpoints(ctx, c, r)
if err != nil {
return err
}
fmt.Println("Listening localhost:8080...")
return http.ListenAndServe(fmt.Sprintf("%s:%d", c.Gateway.Host, c.Gateway.Port), gw)
and in RegisterEndpoints function, the endpoint parameter should be your host:port, the api endpoint should be provided in the google api annotation in the proto file.
err = protos.RegisterUserCenterHandlerFromEndpoint(ctx, mux, fmt.Sprintf("%s:%d", c.Gateway.Host, c.Gateway.Port), dialOpts)
if err != nil {
return nil, err
}
err = protos.RegisterSsoHandlerFromEndpoint(ctx, mux, fmt.Sprintf("%s:%d", c.Gateway.Host, c.Gateway.Port), dialOpts)
if err != nil {
return nil, err
}
I appended grpc.WithBlock() to grpc.DialOption, then all services can be accessed via grpc-gateway now.
Like below:
dialOpts := []grpc.DialOption{grpc.WithBalancerName("round_robin"), grpc.WithInsecure(), grpc.WithBlock()}
I'm trying to abstract the start of a GRPC server
The original main function is the following:
func main() {
lis, err := net.Listen("tcp", port)
if err != nil {
log.Fatalf("failed to listen: %v", err)
}
s := grpc.NewServer()
pb.RegisterCollectionServer(s, &server.Server{})
// Register reflection service on gRPC server.
reflection.Register(s)
if err := s.Serve(lis); err != nil {
log.Fatalf("failed to serve: %v", err)
}
}
My goal is to have something like this:
func startService(sr func(*grpc.Server, interface{}), srv interface{}) error {
lis, err := net.Listen("tcp", port)
if err != nil {
return err
}
s := grpc.NewServer()
sr(s, srv)
reflection.Register(s)
return s.Serve(lis)
}
func main() {
err := startService(pb.RegisterCollectionServer, &server.Server{})
if err != nil {
log.Fatalf("failed to start Service: %v", err)
}
}
But this gives me the following error:
cannot use collection_api.RegisterCollectionServer (type func(*grpc.Server, collection_api.CollectionServer)) as type func(*grpc.Server, interface {}) in argument to startServicego
It seems that collection_api.CollectionServer is not a valid interface{} type.
Any idea how to make this work?
In your setup the function signature of collector_api.RegisterCollectionServer must match exactly func(*grpc.Server, interface{}), there is no "is-kind-of" in Go as there is in other languages.
If you want to keep the startService function independent of the collection_api types, you can use a anonymous function in main(). If you know that you what ever you are registering is always dependent on the implementation of the anonymous function (in this case &server.Server{}), then you can leave this detail out of the function signature of startService and put it into the anonymous function:
func startService(sr func(*grpc.Server)) error {
lis, err := net.Listen("tcp", port)
if err != nil {
return err
}
s := grpc.NewServer()
sr(s)
reflection.Register(s)
return s.Serve(lis)
}
func main() {
err := startService(func(grpcServer *grpc.Server) {
pb.RegisterCollectionServer(grpcServer, &server.Server{})
}, &server.Server{})
if err != nil {
log.Fatalf("failed to start Service: %v", err)
}
}