The service I'm writing receive code snippets and process them, the snippets could either be complement program or a fragment, if it is a fragment, I need to add the enclosing main function. For example, the snippet:
var v int
v = 3
fmt.Println(v)
should be classified as a fragment, and add main to it:
func main() {
var v int
v = 3
fmt.Println(v)
}
If the snippet is:
package main
import "fmt"
func main() {
fmt.Println("hello")
}
then no modification should be done.
The way I'm doing now is run the go parser against the snippet:
var fset *token.FileSet
file, err := parser.ParseFile(fset, "stdin", code, 0)
if err != nil {
// add function
code = fmt.Sprintf("func main() {\n%s\n}", code)
// ...
}
This works for the 1st snippet above, however it fails if the fragment has a main function after some other declarations, e.g.
type S struct {
a int
}
func main() {
fmt.Println("foo")
}
I also try to look into the file returned by ParseFile, check the Decls, but it looks it will stop parsing after the 1st error, so Decls is nil in this case. So my question is is there a robust way to handle this?
PS. The inclusion of package clause and the required imports are not relevant because I'm feeding the processed code into golang.org/x/tools/imports anyway.
The dumbest thing that might work is after reading in the file (to a buffer most likely), is to do a string search for func main(){.
If it isn't formatted already, you might need to change that to a regex with whitespaces, but it should be pretty straight-forward.
Related
This code:
package main
import (
"fmt"
)
func main() {
fmt.Println("Hello, playground")
var a bool
var b interface{}
b = true
if a, ok := b.(bool); !ok {
fmt.Println("Problem!")
}
}
Yields this error in the golang playground:
tmp/sandbox791966413/main.go:11:10: a declared and not used
tmp/sandbox791966413/main.go:14:21: a declared and not used
This is confusing because of what we read in the short variable declaration golang docs:
Unlike regular variable declarations, a short variable declaration may
redeclare variables provided they were originally declared earlier in
the same block (or the parameter lists if the block is the function
body) with the same type, and at least one of the non-blank variables
is new. As a consequence, redeclaration can only appear in a
multi-variable short declaration. Redeclaration does not introduce a
new variable; it just assigns a new value to the original.
So, my questions:
Why can't I redeclare the variable in the code snippet above?
Supposing I really can't, what I'd really like to do is find
a way to populate variables with the output of functions while
checking the error in a more concise way. So is there any way
to improve on the following form for getting a value out of an
error-able function?
var A RealType
if newA, err := SomeFunc(); err != nil {
return err
} else {
A = newA
}
This is happening because you are using the short variable declaration in an if initialization statement, which introduces a new scope.
Rather than this, where a is a new variable that shadows the existing one:
if a, ok := b.(bool); !ok {
fmt.Println("Problem!")
}
you could do this, where a is redeclared:
a, ok := b.(bool)
if !ok {
fmt.Println("Problem!")
}
This works because ok is a new variable, so you can redeclare a and the passage you quoted is in effect.
Similarly, your second code snippet could be written as:
A, err := SomeFunc()
if err != nil {
return err
}
You did redeclare the variable, that is the problem. When you redeclare a variable, it is a different variable. This is known as "shadowing".
package main
import (
"fmt"
)
func main() {
fmt.Println("Hello, playground")
// original a.
var a bool
var b interface{}
b = true
// redeclared a here. This is a a'
if a, ok := b.(bool); !ok {
// a' will last until the end of this block
fmt.Println("Problem!")
}
// a' is gone. So using a here would get the original a.
}
As for your second question. That code looks great. I would probably switch it to if err == nil (and swap the if and else blocks). But that is a style thing.
The error raised by the compiler is saying that both declarations of variable a are not being used.
You're actually declaring a twice, when using short declaration in the if condition you're creating a new scope that shadows the previous declaration of the variable.
The actual problem you're facing is that you're never using the variable value which, in Go, is considered a compile time error.
Regarding your second question, I think that the shortest way to get the value and check the error is to do something similar to this:
func main() {
a, ok := someFunction()
if !ok {
fmt.Println("Problem!")
}
}
My project is a login-register web server that consists of multiple files and uses another package in which the Manager struct is defined.
Overview of my files:
my-package/
main.go
handlers.go
...
I have a variable: var M *Manager declared in main.go before definition of main() and it is assigned inside main():
var M *Manager
func main() {
...
M = InitManager(...)
...
}
handleLogin(...) and handleRegister(...) are functions defined in handlers.go that use the M variable:
func handleRegister(...){
...
fmt.Println("M:", M)
M.Log1("logging informations...")
...
}
func handleLogin(...) {
...
fmt.Println("M:", M)
M.GetAccount(login)
...
}
When I go to /loginor /register and the appropriate handle function is triggered it displays: M: <nil>
To find out something more I modified main() as shown below:
var M *Manager
func main() {
...
go func() { // for debugging
for {
fmt.Println("main() goloop1: M:", M)
time.Sleep(time.Second / 2)
}
}()
M = InitManager(...)
go func() { // for debugging
for {
fmt.Println("main() goloop2: M:", M)
time.Sleep(time.Second / 2)
}
}()
...
}
and the output:
main() goloop2: M: &{...data as expected...}
main() goloop1: M: <nil>
main() goloop2: M: &{...data as expected...}
main() goloop1: M: <nil>
...
My question are:
How do pointers work then if one pointer gives out two values?
How to fix my issue and properly plan code (if that was the cause) to avoid this in future?
Per the Go Memory Model, the provided code writes and reads M without proper synchronization, which is a data race and leads to undefined behavior (see icza's comment).
The compiler "assumes" that the code is properly synchronized (this is the responsibility of the developer) and so it is "allowed" to assume that M is never modified inside the infinite loops, so it may use a copy in a given register or stack memory location over and over, leading to the surprising output.
You may use a sync.Mutex to protect every access to the global *Manager variable M, as in this modified code.
Also beware of variable shadowing! It is possible to write M := f() instead of M = f(), resulting in an unrelated local variable, not affecting the global variable.
My solution:
I have added init.go:
my-package/
main.go
handlers.go
...
init.go
I moved global variables like M and CONFIG_MAP into the init.go file:
package main
import ...
var CONFIG_MAP = LoadConfig()
var M *asrv.Manager = InitManager()
func LoadConfig() map[string]string {
// return map from 'conf.json' file
}
func InitManager() *asrv.Manager {
// return Manager configured with CONFIG_MAP
// other functions also use CONFIG_MAP that is the reason why it is global
}
func init() {
LoadTemplatesFiles() // load templates and assign value
// to a variable declared in templates.go
}
This way handler functions (handleLoginGet etc.) in handlers.go could properly read M.
This fix just made my program work and I still don't know what is the proper way of handling this type of situation, that is why I added more info under EDIT in my question.
I have recently looked into Go plugins instead of manually loading .so files myself.
Basically, I have a game server application, and I want to be able to load plugins (using plugins package) when the server starts. Then, in the plugin itself, I want to be able to call exported functions that are a part of the server.
Say I have this plugin, which is compiled to example_plugin.so using go build -buildmode=plugin:
package main
import "fmt"
func init() {
fmt.Println("Hello from plugin!")
}
Then say I have this server application, which loads the plugin (and ultimately calls the "init" function under the hood):
package main
import (
"fmt"
"plugin"
)
func main() {
fmt.Println("Server started")
if _, err := plugin.Open("example_plugin.so"); err != nil {
panic(err)
}
}
// some API function that loaded plugins can call
func GetPlayers() {}
The output is:
Server started
Hello from plugin!
This works as expected, however I want to be able to call that GetPlayers function (and any other exported functions in the server application, ideally) from the plugin (and any other plugins.) I was thinking about making some sort of library consisting of interfaces containing API functions that the server implements, however I have no idea where to start. I know I will probably need to use a .a file or something similar.
For clarification, I am developing this application for use on Linux, so I am fine with a solution that only works on Linux.
Apologies if this is poorly worded, first time posting on SO.
As mentioned in the comments, there is a Lookup function. In the documentation for the module they have the following example:
// A Symbol is a pointer to a variable or function.
// For example, a plugin defined as
//
// var V int
//
// func F() { fmt.Printf("Hello, number %d\n", V) }
//
// may be loaded with the Open function and then the exported package
// symbols V and F can be accessed
package main
import (
"fmt"
"plugin"
)
func main() {
p, err := plugin.Open("plugin_name.so")
if err != nil {
panic(err)
}
v, err := p.Lookup("V")
if err != nil {
panic(err)
}
f, err := p.Lookup("F")
if err != nil {
panic(err)
}
*v.(*int) = 7
f.(func())() // prints "Hello, number 7"
}
I think the most confusing lines here are
*v.(*int) = 7
f.(func())() // prints "Hello, number 7"
The first one of them performs a type assertion to *int to assert that v is indeed a pointer to int. That is needed since Lookup returns an interface{} and in order to do anything useful with a value, you should clarify its type.
The second line performs another type assertion, this time making sure that f is a function with no arguments and no return values, after which, immediately calls it. Since function F from the original module was referencing V (which we've replaced with 7), this call will display Hello, number 7.
I'm using github.com/sirupsen/logrus and github.com/pkg/errors. When I hand an error wrapped or created from pkg/errors, all I see in the log out is the error message. I want to see the stack trace.
From this issue, https://github.com/sirupsen/logrus/issues/506, I infer that logrus has some native method for working with pkg/errors.
How can I do this?
The comment on your Logrus issue is incorrect (and incidentally, appears to come from someone with no affiliation with Logrus, and who has made no contributions to Logrus, so not actually from "the Logrus team").
It is easy to extract the stack trace in a pkg/errors error, as documented:
type stackTracer interface {
StackTrace() errors.StackTrace
}
This means that the easiest way to log the stack trace with logrus would be simply:
if stackErr, ok := err.(stackTracer); ok {
log.WithField("stacktrace", fmt.Sprintf("%+v", stackErr.StackTrace()))
}
As of today, when my a pull request of mine was merged with pkg/errors, this is now even easier, if you're using JSON logging:
if stackErr, ok := err.(stackTracer); ok {
log.WithField("stacktrace", stackErr.StackTrace())
}
This will produce a log format similar to "%+v", but without newlines or tabs, with one log entry per string, for easy marshaling into a JSON array.
Of course, both of these options force you to use the format defined by pkg/errors, which isn't always ideal. So instead, you can iterate through the stack trace, and produce your own formatting, possibly producing a format easily marshalable to JSON.
if err, ok := err.(stackTracer); ok {
for _, f := range err.StackTrace() {
fmt.Printf("%+s:%d\n", f, f) // Or your own formatting
}
}
Rather than printing each frame, you can coerce it into any format you like.
The inference is wrong. Logrus does not actually know how to handle the error.
Update the Logrus team official said that this is NOT a supported feature, https://github.com/sirupsen/logrus/issues/895#issuecomment-457656556.
A Java-ish Response
In order to universally work with error handlers in this way, I composed a new version of Entry, which is from Logrus. As the example shows, create a new Entry with what ever common fields you want (below the example is a logger set in a handler that keeps track of the caller id. Pass PgkError through your layers as you work the Entry. When you need to log specific errors, like call variables experiencing the error, start with the PkgError.WithError(...) then add your details.
This is a starting point. If you want to use this generally, implement all of the Entity interface on PkgErrorEntry. Continue to delegate to the internal entry, but return a new PkgErrorEntry. Such a change would make the value true drop in replacement for Entry.
package main
import (
"fmt"
"github.com/sirupsen/logrus"
"strings"
unwrappedErrors "errors"
"github.com/pkg/errors"
)
// PkgErrorEntry enables stack frame extraction directly into the log fields.
type PkgErrorEntry struct {
*logrus.Entry
// Depth defines how much of the stacktrace you want.
Depth int
}
// This is dirty pkg/errors.
type stackTracer interface {
StackTrace() errors.StackTrace
}
func (e *PkgErrorEntry) WithError(err error) *logrus.Entry {
out := e.Entry
common := func(pError stackTracer) {
st := pError.StackTrace()
depth := 3
if e.Depth != 0 {
depth = e.Depth
}
valued := fmt.Sprintf("%+v", st[0:depth])
valued = strings.Replace(valued, "\t", "", -1)
stack := strings.Split(valued, "\n")
out = out.WithField("stack", stack[2:])
}
if err2, ok := err.(stackTracer); ok {
common(err2)
}
if err2, ok := errors.Cause(err).(stackTracer); ok {
common(err2)
}
return out.WithError(err)
}
func someWhereElse() error {
return unwrappedErrors.New("Ouch")
}
func level1() error {
return level2()
}
func level2() error {
return errors.WithStack(unwrappedErrors.New("All wrapped up"))
}
func main() {
baseLog := logrus.New()
baseLog.SetFormatter(&logrus.JSONFormatter{})
errorHandling := PkgErrorEntry{Entry: baseLog.WithField("callerid", "1000")}
errorHandling.Info("Hello")
err := errors.New("Hi")
errorHandling.WithError(err).Error("That should have a stack.")
err = someWhereElse()
errorHandling.WithError(err).Info("Less painful error")
err = level1()
errorHandling.WithError(err).Warn("Should have multiple layers of stack")
}
A Gopher-ish way
See https://www.reddit.com/r/golang/comments/ajby88/how_to_get_stack_traces_in_logrus/ for more detail.
Ben Johnson wrote about making errors part of your domain. An abbreviated version is that you should put tracer attributes onto a custom error. When code directly under your control errors or when an error from a 3rd party library occurs, the code immediately dealing with the error should put a unique value into the custom error. This value will print as part of the custom error's Error() string implementation.
When developers get the log file, they will be able to grep the code base for that unique value. Ben says "Finally, we need to be able to provide all this information plus a logical stack trace to our operator so they can debug issues. Go already provides a simple method, error.Error(), to print error information so we can utilize that."
Here's Ben's example
// attachRole inserts a role record for a user in the database
func (s *UserService) attachRole(ctx context.Context, id int, role string) error {
const op = "attachRole"
if _, err := s.db.Exec(`INSERT roles...`); err != nil {
return &myapp.Error{Op: op, Err: err}
}
return nil
}
An issue I have with the grep-able code is that it's easy for the value to diverge from the original context. For example, say the name of the function was changed from attachRole to something else and the function was longer. It possible that the op value can diverge from the function name. Regardless, this appears to satisfy the general need of tracing to a problem, while treating errors a first class citizens.
Go2 might throw a curve at this into more the Java-ish response. Stay tuned.
https://go.googlesource.com/proposal/+/refs/changes/97/159497/3/design/XXXXX-error-values.md
Use custom hook to extract stacktrace
import (
"fmt"
"github.com/pkg/errors"
"github.com/sirupsen/logrus"
)
type StacktraceHook struct {
}
func (h *StacktraceHook) Levels() []logrus.Level {
return logrus.AllLevels
}
func (h *StacktraceHook) Fire(e *logrus.Entry) error {
if v, found := e.Data[logrus.ErrorKey]; found {
if err, iserr := v.(error); iserr {
type stackTracer interface {
StackTrace() errors.StackTrace
}
if st, isst := err.(stackTracer); isst {
stack := fmt.Sprintf("%+v", st.StackTrace())
e.Data["stacktrace"] = stack
}
}
}
return nil
}
func main() {
logrus.SetFormatter(&logrus.TextFormatter{DisableQuote: true})
logrus.AddHook(&StacktraceHook{})
logrus.WithError(errors.New("Foo")).Error("Wrong")
}
Output
time=2009-11-10T23:00:00Z level=error msg=Wrong error=Foo stacktrace=
main.main
/tmp/sandbox1710078453/prog.go:36
runtime.main
/usr/local/go-faketime/src/runtime/proc.go:250
runtime.goexit
/usr/local/go-faketime/src/runtime/asm_amd64.s:1594
I am a beginner in Golang.
I have a problem with variable type assigning from user input.
When the user enters data like "2012BV352" I need to be able to ignore the BV and pass 2012352 to my next function.
There has a package name gopkg.in/validator.v2 in doc
But what it returns is whether or not the variable is safe or not.
I need to cut off the unusual things.
Any idea on how to achieve this?
You could write your own sanitizing methods and if it becomes something you'll be using more often, I'd package it out and add other methods to cover more use cases.
I provide two different ways to achieve the same result. One is commented out.
I haven't run any benchmarks so i couldn't tell you for certain which is more performant, but you could write your own tests if you wanted to figure it out. It would also expose another important aspect of Go and in my opinion one of it's more powerful tools... testing.
package main
import (
"fmt"
"log"
"regexp"
"strconv"
"strings"
)
// using a regex here which simply targets all digits and ignores everything else. I make it a global var and use MustCompile because the
// regex doesn't need to be created every time.
var extractInts = regexp.MustCompile(`\d+`)
func SanitizeStringToInt(input string) (int, error) {
m := extractInts.FindAllString(input, -1)
s := strings.Join(m, "")
return strconv.Atoi(s)
}
/*
// if you didn't want to use regex you could use a for loop
func SanitizeStringToInt(input string) (int, error) {
var s string
for _, r := range input {
if !unicode.IsLetter(r) {
s += string(r)
}
}
return strconv.Atoi(s)
}
*/
func main() {
a := "2012BV352"
n, err := SanitizeStringToInt(a)
if err != nil {
log.Fatal(err)
}
fmt.Println(n)
}