I have an existing TCP connection (net.Conn) on which I am trying to use (develop) an RPC client-server application.
So far, I explored gRPC and net/rpc in the standard lib; unless I am missing something, both seem to need full control over the connection(s).
I could just go for a custom implementation with gob to serialize requests/replies but since I also require a decent error handling (at least for errors I am generating on the server side) I am trying to see if there is some existing framework I could use.
Any pointers/ideas?
After some more investigation, I realized I was wrong about net/rpc. Apologies for that.
You can call rpc.ServeConn on server side and rpc.NewClient on client-side passing an existing connection, as long as it implements io.ReadWriteCloser
Example server:
package main
import (
"net/rpc"
"os"
"myapp/pkg"
)
func main() {
fsops := new(pkg.FSOps)
s := rpc.NewServer()
s.Register(fsops)
s.ServeConn(pkg.RWCloser{os.Stdin, os.Stdout})
}
Example client:
package main
import (
"net/rpc"
"os"
"myapp/pkg"
)
func main() {
client := rpc.NewClient(pkg.RWCloser{os.Stdin, os.Stdout})
var reply pkg.XXXReply
err := client.Call("FSOps.XXX", "args", &reply)
}
Related
I'm playing with fasthttp and it's router, I have no issues with basic things, I have a working server and a router, that is the easy part.
The issue is with the logger, I would like to be able to customize that one, but it does not seem possible with the ctx.Logger() as it only takes a Printf argument and the format is not what I'm looking for.
Does anyone knows where in the documentation I can find a working example of what I want to do?
Example of code I currently have:
package server
import (
"github.com/fasthttp/router"
"github.com/valyala/fasthttp"
)
// Router will manage the routes of our API server
func Router() *router.Router {
r := router.New()
r.GET("/", index)
return r
}
func index(ctx *fasthttp.RequestCtx) {
ctx.Logger().Printf("/")
ctx.WriteString("Welcome!")
}
As I'm still trying my hand with the web servers and I still don't understand some things with it and Go also. So An example would be welcome.
For example I would like to be able to do something like that using a logger define in the main package:
package server
import (
"github.com/fasthttp/router"
"github.com/valyala/fasthttp"
"go.uber.org/zap"
)
// Router will manage the routes of our API server
func Router(loger *zap.Logger) *router.Router {
r := router.New()
r.GET("/", index)
return r
}
func index(ctx *fasthttp.RequestCtx) {
ctx.Logger().Printf("/") // Here should print in the zap format of my choice.
ctx.WriteString("Welcome!")
}
If you look at the source code, it's apparent that all you have is the ability to write standard Go-formatted strings:
func (cl *ctxLogger) Printf(format string, args ...interface{}) {
msg := fmt.Sprintf(format, args...)
ctxLoggerLock.Lock()
cl.logger.Printf("%.3f %s - %s", time.Since(cl.ctx.ConnTime()).Seconds(), cl.ctx.String(), msg)
ctxLoggerLock.Unlock()
}
The logger simply adds some additional information from the context. So further cutomisation beyond the standard Go formatting does not seem possible. I'm not sure what "zap format of my choice" is, so I can't say if there's a workaround or even if standard Go formatting options will serve for you here.
I'm trying to create a context object, so that I can connect to firestore from cloud run, however, all the examples I find on the net basically say I need a context objects, examples I find online usually look like this:
ctx := context.Background()
client, err := firestore.NewClient(ctx, "projectID")
if err != nil {
fail(w, err.Error())
// TODO: Handle error.
}
You can find these examples in places like this one:
https://godoc.org/cloud.google.com/go/firestore#example-NewClient
Nowhere in this example is there an explanation of where to find the context object.
so I just get this error:
undefined: context
I think the documentation is just too confusing.
You should use r.Context() of http.Request object in the request handler instead of initializing a new detached context like context.Background().
The main purpose of contexts is to propagate them, and in Cloud Run you always process requests, so if you pass the request’s context, it’s the right thing to do.
I think in your case, “context” package is not imported. Make sure to use go 1.11+ in your Dockerfile and say:
import “context”
In Go, you need to import packages. For this statement ctx := context.Background() add to the top of your source file import "context" or merge with your existing import set.
Like most languages, the more experience you have the more the language makes sense and you just know what to do. Most languages are the same. In C/C++ you have the include statement, C# the using statement, in Python the import statement, etc.
Google has a large package of examples for using Go and Google Cloud Platform:
Google Cloud Examples in Go
I wrote an article that documents my 30-day journey to learn Go and Google Cloud Platform.
Google Cloud and Go – My Journey to Learn a new Language in 30 days
Given the plethora of 3 lines of code examples that are hard for beginners, and the lack of complete working examples online, like myself, here is a full working example, which is kinda what I needed when I started this task, I hope this helps anybody in the future.
package main
import (
"cloud.google.com/go/firestore" // https://godoc.org/cloud.google.com/go/firestore"
"context" // https://blog.golang.org/context
firebase "firebase.google.com/go"
"fmt"
"log"
"net/http"
"os"
)
func fail(w http.ResponseWriter, msg string) {
fmt.Fprintln(w, "fail:"+msg)
log.Println("fail:" + msg)
}
// State example code
type State struct {
Capital string `firestore:"capital"`
Population float64 `firestore:"pop"` // in millions
}
func main() {
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
override := make(map[string]interface{})
ctx := context.Background()
client, err := firestore.NewClient(ctx, "YOURPID")// set GOOGLE_APPLICATION_CREDENTIALS env var
if err != nil {
fail(w, err.Error())
return
}
states := client.Collection("States")
ny := states.Doc("NewYork")
wr, err := ny.Create(ctx, State{
Capital: "Albany",
Population: 19.8,
})
fmt.Println(wr)
})
log.Fatal(http.ListenAndServe("0.0.0.0:8082", nil))
}
I am trying to configure a RabbitMQ broker using the go-micro framework. I have noticed that the broker interface in go-micro has a broker.SubscriberOptions struct which allows configuring the parameters I am looking for (AutoAck, Queue name and so on) however I am unable to figure out how to pass this when starting a broker.
This is how a simple rabbit go-micro setup would look like
package main
import (
"log"
"github.com/micro/go-micro/server"
"github.com/micro/go-plugins/broker/rabbitmq"
micro "github.com/micro/go-micro"
)
func main() {
// Create a new service. Optionally include some options here.
service := micro.NewService(
micro.Name("go-micro-rabbit"),
micro.Broker(rabbitmq.NewBroker()),
)
// Init will parse the command line flags.
service.Init()
// Register handler
proto.RegisterGreeterHandler(service.Server(), new(Greeter))
micro.RegisterSubscriber("micro-exchange", service.Server(), myFunc, server.SubscriberQueue("my-queue"))
// Run the server
if err := service.Run(); err != nil {
log.Fatal(err)
}
}
The micro.RegisterSubscriber method takes in a list of server.SubscriberOption but does not allow me to set the broker.SubscriberOptions and the rabbitmq.NewBroker allows setting broker.Options but once again, not broker.SubscriberOptions
I have dug in the code of go-micro but have been unable to figure out how the broker.Subscribe method (Which exposes the correct struct) is called or by who.
Is this possible at all? Is it maybe something not yet fully fleshed out in the API?
I have a init() function defined in "config/config.go"
config.go
package config
import(
log "github.com/sirupsen/logrus"
)
func init() {
log.SetReportCaller(true)
}
I have another go file called auth.go in auth package
package auth
import(
log "github.com/sirupsen/logrus"
)
func auth(username string, pwd string) {
//some auth code
log.Info("Auth success")
}
When log.Info() is called in auth.go the log prints as below
2018-11-09T16:38:27+05:30 auth/auth.go:36 level=info msg="Auth success"
What I am confused here is that, how "log" in auth.go is aware of the settings done in config.go. log.SetReportCaller() is in config.go but even when auth.go is logged it takes settings of log.SetReportCaller() done in config.go
Since log.SetReportCaller() is not set in auth.go expected log should be as below without showing line number of caller method.
2018-11-09T16:38:27+05:30 level=info msg="Auth success"
main.go
package main
import (
"path/to/auth"
log "github.com/sirupsen/logrus"
"net/http"
)
func main() {
r := server.Route()
log.Info("Listening on 8080")
http.Handle("/", r)
log.Fatal(http.ListenAndServe(":8080", nil))
}
auth/router.go
package auth
import (
"github.com/gorilla/mux"
"github.com/rs/cors"
"net/http"
)
func Route() http.Handler {
r := mux.NewRouter()
// UI handlers
r.HandleFunc("/", IndexPageHandler)
r.HandleFunc("/login", LoginHandler).Methods("POST")
handler := cors.Default().Handler(r)
return
}
auth/login.go
package auth
import (
"fmt"
"path/to/config"
log "github.com/sirupsen/logrus"
"net/http"
)
func LoginHandler(w http.ResponseWriter, r *http.Request) {
username := r.FormValue("username")
password := r.FormValue("password")
status, userName, mail, _ := auth(username, password)
//some code goes here
}
Kindly explain how this is happening
Check this diagram to understand how init() work: diagram
Initialization order are as follows,
If a package imports other packages, the imported packages are initialised first.
Package level variables are initialised then.
init function of current package is called next. A package can have multiple init functions (either in a single file or distributed across multiple files) and they are called in the order in which they are presented to the compiler.
You will found an example explaining this here.
I am suspecting in dependency tree, there is a common ancestor file that import both config package and auth package.
Here is official doc on initialization: Package initialization
UPD: As you have added respective codes, let's visualize what is happening here. Look at the picture bellow,
What happening:
Your main package start initialing. But it is has imported auth package. So to complete initialization, auth package must be initialized.
auth package start initializing. But it has imported config package. So to complete initialization, config package must be initialized.
config package complete initialization(log.SetReportCaller(true) is called).
auth package complete initialization.
main package complete initialization.
main() function starts executing...
The behavior you're asking about actually has nothing to do with how init() functions work. SetReportCaller sets a global variable in github.com/sirupsen/logrus, not in your config or auth packages. So it doesn't matter where you call that function or where you call log.Info et al; the setting affects all calls to logrus regardless of call origin.
I'm an experienced programmer, but new to go. Apologies in advance if this is an obvious question or well tread path. I'm still getting my bearings in the language and its semantics.
I'm trying to create a web server in go that
Examines the HTTP request
Based on the results of the request, serves a specific static folder
i.e., something where the simplified pseudo code looks like
import (
"io"
"net/http"
"fmt"
"strings"
"encoding/base64"
)
func examineRequest(request *http.Request) {
//looks at request header
if(request headers have one thing){
return "foo"
}
return "bar"
}
func processRequest(responseWriter http.ResponseWriter, request *http.Request) {
folderToServe = examineRequest(request);
if folderToServe == "bar" {
//serve static files from the ./static/bar folder
//go freaks out if I try these
//http.Handle("/", http.FileServer(http.Dir("./static/bar")))
//http.FileServer(http.Dir("./static/bar")()
}
else if folderToServer == "foo" {
//serve static files from the ./static/foo folder
//go freaks out if I try these
//http.Handle("/", http.FileServer(http.Dir("./static/foo")))
//http.FileServer(http.Dir("./static/foo")()
}
}
func main(){
http.HandleFunc("/", processRequest)
//http.Handle("/", http.FileServer(http.Dir("./static")))
}
Experienced go programmers may have already spotted the problem. I'm performing my examination in processRequest, and because of that, its too late to to call Handle -- however, you can't register more than one handle for the same path in go, and nested handle calls freak go out.
I though the handler might be similar to an anonymous function in other languages and tried call it -- but go did like that either.
So -- is there a way to manually invoke the handler returned from the call to http.FileServer(http.Dir("./static"))?
Is that even the right question to be asking here?
What exactly is a handler in the context of the http module?
Use http.FileServer(http.Dir("./static/foo")).ServeHTTP(w, req).
//edit
http.FileServer returns an http.Handler which in turn provides the ServerHTTP method.