Type assertion syntax using go/ast - go

I have read through the piece of code bellow and I don't know what exactly the syntax of
d.()
in
if f, ok := d.(*ast.FuncDecl); mean.
Can anybody explain it for me?
package main
import (
"go/ast"
"go/parser"
"go/token"
"regexp"
"github.com/posener/complete"
)
func functionsInFile(path string, regexp *regexp.Regexp) (tests []string) {
fset := token.NewFileSet()
f, err := parser.ParseFile(fset, path, nil, 0)
if err != nil {
complete.Log("Failed parsing %s: %s", path, err)
return nil
}
for _, d := range f.Decls {
if f, ok := d.(*ast.FuncDecl); ok {
name := f.Name.String()
if regexp == nil || regexp.MatchString(name) {
tests = append(tests, name)
}
}
}
return
}

As the comments say, this is a type assertion. Type assertions are described in the Go spec: https://golang.org/ref/spec#Type_assertions
Applied to your specific code sample, I think it's taken from the AST module documentation, which you should read if you want to work with Go ASTs.
ParseFile returns an ast.File, which is:
type File struct {
Doc *CommentGroup // associated documentation; or nil
Package token.Pos // position of "package" keyword
Name *Ident // package name
Decls []Decl // top-level declarations; or nil
Scope *Scope // package scope (this file only)
Imports []*ImportSpec // imports in this file
Unresolved []*Ident // unresolved identifiers in this file
Comments []*CommentGroup // list of all comments in the source file
}
So Decls is a slice of Decl, which is an interface. In brief, this means that at compile-time you don't know what the actual underlying type is (though you do know it satisfies the interface), so you do a run-time type assertion to verify it's something you expect.
if f, ok := d.(*ast.FuncDecl); ok {
name := f.Name.String()
if regexp == nil || regexp.MatchString(name) {
tests = append(tests, name)
}
}
This means "if d is actually a FuncDecl, do this thing".

Related

Is this the correct behavior of ast parsing

I am working on learning how to use and how golang's ast library works. I am parsing https://github.com/modern-go/concurrent, avoiding the test files and the go_below_19.go since it causes errors.
My problem is with the parsing of these lines in the file unbounded_executor.go,
var HandlePanic = func(recovered interface{}, funcName string) {
ErrorLogger.Println(fmt.Sprintf("%s panic: %v", funcName, recovered))
ErrorLogger.Println(string(debug.Stack()))
}
The ast.Ident for ErrorLogger in both instances have a nil obj.
But, I believe that it should not be nil and should reference these lines from log.go,
// ErrorLogger is used to print out error, can be set to writer other than stderr
var ErrorLogger = log.New(os.Stderr, "", 0)
Am I wrong, or is there a problem with the parser? I've followed several references on parsing files and reuse a *token.FileSet across each of the files and use ParseComments as the mode.
edit:
There is a large code base surrounding this, so the code demonstrating this will include snippets.
This is performed with the same fset across all non-test go files, without build restrictions that would stop the code from being used with 1.16
parsedFile, parseErr := parser.ParseFile(fset, filePath, nil, parser.ParseComments)
Call ast.NewPackage to resolve identifiers in the AST:
fset := token.NewFileSet()
files := make(map[string]*ast.File)
for _, name := range []string{"unbounded_executor.go", "log.go"} {
f, err := parser.ParseFile(fset, name, nil, parser.ParseComments)
if err != nil {
log.Fatal(err)
}
files[name] = f
}
ast.NewPackage(fset, files, nil, nil)
ast.Inspect(files["unbounded_executor.go"], func(n ast.Node) bool {
if n, ok := n.(*ast.Ident); ok && n.Name == "ErrorLogger" {
fmt.Println(n.Obj)
}
return true
})
Because a proper importer is not provided and the list of files does not include all files in the package, NewPackage returns unresolved symbol errors.

How to parse Prometheus data

I have been able to obtain the metrices by sending an HTTP GET as follows:
# TYPE net_conntrack_dialer_conn_attempted_total untyped net_conntrack_dialer_conn_attempted_total{dialer_name="federate",instance="localhost:9090",job="prometheus"} 1 1608520832877
Now I need to parse this data and obtain control over every piece of data so that I can convert tand format like json.
I have been looking into the ebnf package in Go:
ebnf package
Can somebody point me the right direction to parse the above data?
There's a nice package already available to do that and it's by the Prometheus's Authors itself.
They have written a bunch of Go libraries that are shared across Prometheus components and libraries. They are considered internal to Prometheus but you can use them.
Refer: github.com/prometheus/common doc. There's a package called expfmt that can decode and encode the Prometheus's Exposition Format (Link). Yes, it follows the EBNF syntax so ebnf package could also be used but you're getting expfmt right out of the box.
Package used: expfmt
Sample Input:
# HELP net_conntrack_dialer_conn_attempted_total
# TYPE net_conntrack_dialer_conn_attempted_total untyped
net_conntrack_dialer_conn_attempted_total{dialer_name="federate",instance="localhost:9090",job="prometheus"} 1 1608520832877
Sample Program:
package main
import (
"flag"
"fmt"
"log"
"os"
dto "github.com/prometheus/client_model/go"
"github.com/prometheus/common/expfmt"
)
func fatal(err error) {
if err != nil {
log.Fatalln(err)
}
}
func parseMF(path string) (map[string]*dto.MetricFamily, error) {
reader, err := os.Open(path)
if err != nil {
return nil, err
}
var parser expfmt.TextParser
mf, err := parser.TextToMetricFamilies(reader)
if err != nil {
return nil, err
}
return mf, nil
}
func main() {
f := flag.String("f", "", "set filepath")
flag.Parse()
mf, err := parseMF(*f)
fatal(err)
for k, v := range mf {
fmt.Println("KEY: ", k)
fmt.Println("VAL: ", v)
}
}
Sample Output:
KEY: net_conntrack_dialer_conn_attempted_total
VAL: name:"net_conntrack_dialer_conn_attempted_total" type:UNTYPED metric:<label:<name:"dialer_name" value:"federate" > label:<name:"instance" value:"localhost:9090" > label:<name:"job" value:"prometheus" > untyped:<value:1 > timestamp_ms:1608520832877 >
So, expfmt is a good choice for your use-case.
Update: Formatting problem in OP's posted input:
Refer:
https://github.com/prometheus/pushgateway/issues/147#issuecomment-368215305
https://github.com/prometheus/pushgateway#command-line
Note that in the text protocol, each line has to end with a line-feed
character (aka 'LF' or '\n'). Ending a line in other ways, e.g. with
'CR' aka '\r', 'CRLF' aka '\r\n', or just the end of the packet, will
result in a protocol error.
But from the error message, I could see \r char is present in in the put which is not acceptable by design. So use \n for line endings.

How to find the datatype of a value stored as a string in Go?

I'm in a situation where all my variables are stored as strings. E.g. var boolVar = "false". Is there some package in Go for returning the datatype of the value in boolVar?
I can think of a bit cumbersome way of using strconv and then testing the returned error using if-else, but I wonder if some package already provides this functionality.
var boolVar = "false"
var type = "string"
if _, err := strconv.ParseBool(boolVar); err == nil {
type = "bool"
}
There isn't any silver-bullet solution for your problem, you should check all types one by one.
Firstly go is not built for these requirements,
but there is a package for you requirement which does this: go eval
remember it can eval expr only and not code like eval in python can.
package main
import (
"fmt"
"github.com/apaxa-go/eval"
)
func main() {
str := "int64(10*(1.1+2))"
exp, err := eval.ParseString(str, "")
if err != nil {
fmt.Println(err)
}
val, err := exp.EvalToInterface(nil)
if err != nil {
fmt.Println(err)
}
fmt.Printf("%v %T", val, val)
}

Finding imports and dependencies of a go program

The go list -json command run from the command line will tell you the imports and dependencies of a go program ( in json format). Is there a way to get this information from within a go program I.e at runtime, either by running the 'go list' command somehow or another way?
The following code uses the go/build to get the imports for the application in the current working directory.
p, err := build.Default.Import(".", ".", 0)
if err != nil {
// handle error
}
for _, i := range p.Imports {
fmt.Println(i)
}
You can build a list of all dependencies using a simple recursive function.
To get the imports for a specific path, use:
p, err := build.Default.Import(path, ".", 0)
if err != nil {
// handle error
}
for _, i := range p.Imports {
fmt.Println(i)
}
I don't think you can do it without using the go binary since go needs to analyze your source code.
It's pretty easy to do but it must have access to go and your source code at run time. Heres a quick example:
package main
import (
"encoding/json"
"fmt"
"os/exec"
)
func main() {
cmd := exec.Command("go", "list", "-json")
stdout, err := cmd.Output()
if err != nil {
println(err.Error())
return
}
var list GoList
err = json.Unmarshal(stdout, &list)
for _, d := range list.Deps {
fmt.Printf(" - %s\n", d)
}
}
type GoList struct {
Dir string
ImportPath string
Name string
Target string
Stale bool
Root string
GoFiles []string
Imports []string
Deps []string
}
No, I don't think this is possible without the source in a reliable way. Go binaries on different platforms, compiled with different compilers may or may not have (or may not have in the future) these informations compiled in.
But as Go programs are compiled anyway: Why not record this information while you do have access to the source code?

Get a simple string representation of a struct field’s type

Using Go’s ast package, I am looping over a struct’s field list like so:
type Thing struct {
Field1 string
Field2 []int
Field3 map[byte]float64
}
// typ is a *ast.StructType representing the above
for _, fld := range typ.Fields.List {
// get fld.Type as string
}
…and would like to get a simple string representation of fld.Type, as it appears in the source code, e.g. []int or map[byte]float64.
The ast package field type Type property is an Expr, so I’ve found myself getting off into the weeds using type switches and handling every type specifically – when my only goal is to get out the plain string to the right of each field name, which seems like it should be simpler.
Is there a simple way?
There are two things you could be getting at here, one is the type of an expression as would ultimately be resolved during compilation and the other is the code which would determine that type.
Digging through the docs, I don't believe the first is at all available. You can get at the later, however, by using End() and Pos() on Node.
Quick example program:
package main
import (
"fmt"
"go/ast"
"go/parser"
"go/token"
)
func main() {
src := `
package foo
type Thing struct {
Field1 string
Field2 []int
Field3 map[byte]float64
}`
fset := token.NewFileSet()
f, err := parser.ParseFile(fset, "", src, 0)
if err != nil {
panic(err)
}
// hard coding looking these up
typeDecl := f.Decls[0].(*ast.GenDecl)
structDecl := typeDecl.Specs[0].(*ast.TypeSpec).Type.(*ast.StructType)
fields := structDecl.Fields.List
for _, field := range fields {
typeExpr := field.Type
start := typeExpr.Pos() - 1
end := typeExpr.End() - 1
// grab it in source
typeInSource := src[start:end]
fmt.Println(typeInSource)
}
}
This prints:
string
[]int
map[byte]float64
I through this together in the golang playground, if you want to mess with it.
You can use go/types ExprString
This works with complicated types like []string, []map[string]string, etc.
import (
...
"go/types"
...
)
...
// typ is a *ast.StructType representing the above
for _, fld := range typ.Fields.List {
...
typeExpr := fld.Type
typeString := types.ExprString(typeExpr)
...
}
https://golang.org/src/go/types/exprstring.go
This is exactly what Fprint in the go/printer package is for. It takes any AST node as an argument and writes its string representation to a io.Writer.
You can use it in your example as follows:
package main
import (
"bytes"
"fmt"
"go/ast"
"go/parser"
"go/printer"
"go/token"
"log"
)
func main() {
src := `
package foo
type Thing struct {
Field1 string
Field2 []int
Field3 map[byte]float64
}`
fset := token.NewFileSet()
f, err := parser.ParseFile(fset, "", src, 0)
if err != nil {
panic(err)
}
typeDecl := f.Decls[0].(*ast.GenDecl)
structDecl := typeDecl.Specs[0].(*ast.TypeSpec).Type.(*ast.StructType)
for i, fld := range structDecl.Fields.List {
// get fld.Type as string
var typeNameBuf bytes.Buffer
err := printer.Fprint(&typeNameBuf, fset, fld.Type)
if err != nil {
log.Fatalf("failed printing %s", err)
}
fmt.Printf("field %d has type %q\n", i, typeNameBuf.String())
}
}
Output:
field 0 has type "string"
field 1 has type "[]int"
field 2 has type "map[byte]float64"
Try it in playground: https://play.golang.org/p/cyrCLt_JEzQ
I found a way to do this without using the original source code as a reference for simple members (not slices, arrays or structs):
for _, field := range fields {
switch field.Type.(type) {
case *ast.Ident:
stype := field.Type.(*ast.Ident).Name // The type as a string
tag = ""
if field.Tag != nil {
tag = field.Tag.Value //the tag as a string
}
name := field.Names[0].Name //name as a string
...
For the non-simple members you just need another case statement (IE: case *ast.ArrayType:).

Resources