I have written a small constant number expression evaluator. I’m running go-fuzz on it, but it can only detect crashes.
I would like to test that expression considered invalid by my program are really invalid, and valid expression are really valid and yield a correct result.
How could I do that in Go ?
I looked into existing packages like this one, but it allows much more operations and types than I currently support. It thus can’t use it for validation.
The values that I handle are int and float64, and the operations are | & ^ ~(inverse) + - * / %. Also I support the same type of number literals as Go.
I did manual checks but this doesn’t bring me very far. I would need to check the result against another expression evaluator.
You can use go/types and co.:
Constant folding computes the exact constant value (constant.Value)
for every expression (ast.Expr) that is a compile-time constant. Use
Info.Types[expr].Value for the results of constant folding.
I'll use Info.Defs instead of Info.Types as that's simpler for this case, IMO at least.
expr := "float64(20)/6.8"
// parse the expression as part of a file so that it can be type checked
fset := token.NewFileSet()
f, err := parser.ParseFile(fset, "x.go", "package x\nconst K = "+expr, 0)
if err != nil {
panic(err)
}
// type check the file so that the constant value gets computed
info := types.Info{Defs: map[*ast.Ident]types.Object{}}
conf := types.Config{}
if _, err := conf.Check("x", fset, []*ast.File{f}, &info); err != nil {
panic(err)
}
// get the value (as string)
for id, obj := range info.Defs {
if id.Name == "K" {
v := obj.(*types.Const).Val()
fmt.Println(v.String())
}
}
https://play.golang.org/p/h2e0No2F1CY
Related
Context: I'm trying to resolve this issue.
In other words, there's a NormalizeJsonString() for JSON strings (see this for more context:
// Takes a value containing JSON string and passes it through
// the JSON parser to normalize it, returns either a parsing
// error or normalized JSON string.
func NormalizeJsonString(jsonString interface{}) (string, error) {
that allows to have the following code:
return structure.NormalizeJsonString(old) == structure.NormalizeJsonString(new)
but it doesn't work for strings that are proto files (all proto files are guaranteed to have just one message definition). For example, I could see:
syntax = "proto3";
- package bar.proto;
+ package bar.proto;
option java_outer_classname = "FooProto";
message Foo {
...
- int64 xyz = 3;
+ int64 xyz = 3;
Is there NormalizeProtoString available in some Go SDKs? I found MessageDifferencer but it's in C++ only. Another option I considered was to replace all new lines / group of whitespaces with a single whitespace but it's a little bit hacky.
To do this in a semantic fashion, the proto definitions should really be parsed. Naively stripping and/or replacing whitespace may get you somewhere, but likely will have gotchas.
As far as I'm aware the latest official Go protobuf package don't have anything to handle parsing protobuf definitions - the protoc compiler handles that side of affairs, and this is written in C++
There would be options to execute the protoc compiler to get hold of the descriptor set output (e.g. protoc --descriptor_set_out=...), however I'm guessing this would also be slightly haphazard considering it requires one to have protoc available - and version differences could potentially cause problems too.
Assuming that is no go, one further option is to use a 3rd party parser written in Go - github.com/yoheimuta/go-protoparser seems to handle things quite well. One slight issue when making comparisons is that the parser records meta information about source line + column positions for each type; however it is relatively easy to make a comparison and ignore these, by using github.com/google/go-cmp
For example:
package main
import (
"fmt"
"log"
"os"
"github.com/google/go-cmp/cmp"
"github.com/google/go-cmp/cmp/cmpopts"
"github.com/yoheimuta/go-protoparser/v4"
"github.com/yoheimuta/go-protoparser/v4/parser"
"github.com/yoheimuta/go-protoparser/v4/parser/meta"
)
func main() {
if err := run(); err != nil {
log.Fatal(err)
}
}
func run() error {
proto1, err := parseFile("example1.proto")
if err != nil {
return err
}
proto2, err := parseFile("example2.proto")
if err != nil {
return err
}
equal := cmp.Equal(proto1, proto2, cmpopts.IgnoreTypes(meta.Meta{}))
fmt.Printf("equal: %t", equal)
return nil
}
func parseFile(path string) (*parser.Proto, error) {
f, err := os.Open(path)
if err != nil {
return nil, err
}
defer f.Close()
return protoparser.Parse(f)
}
outputs:
equal: true
for the example you provided.
I have a function that simply reads a file with ioutil.ReadFile(). The type returned is []byte, although the value itself can be represented as a float.
I am converting the []byte in this manner (where value is the []byte being returned from a function that reads a file):
var floatValue float64
fmt.Fscanf(bytes.NewReader(value), "%f", &floatValue)
Is this really the only way to extract/parse a valid float value from a []byte? There's a similar discussion but looks like it didn't really go anywhere.
You can easily use strconv.ParseFloat for this, just converting your []byte to a string first. This would surely have less overhead than creating a reader and scanning with a scanf-like function.
sb := []byte("3.1415")
s := string(sb)
f, err := strconv.ParseFloat(s, 64)
if err != nil {
panic("whoops!")
}
fmt.Printf("%f\n", f)
Output:
3.141500
My aim is to create a logging function that lists the name of a function and the list of passed parameters.
An example would be the following:
func MyFunc(a string, b int){
... some code ...
if err != nil{
errorDescription := myLoggingFunction(err)
fmt.Println(errorDescription)
}
}
func main(){
MyFunc("hello", 42)
}
// where MyLoggingFunction should return something like:
// "MyFunc: a = hello, b = 42, receivedError = "dummy error description"
So far it seems that in Go there is no way to get the name of the parameters of a function at runtime, as answered in this question, but I could give up this feature.
I've managed to get the function name and the memory address of the passed parameters by analysing the stack trace, but I'm hitting a wall when it comes to print somehow the parameters starting from their address (I understand that it might not be trivial depending on the type of the parameters, but even something very simple will do for now)
This is an implementation of the logging function I'm building (you can test it on this playground), is there away to print the parameter values?
func MyLoggingFunction(err error) string {
callersPCs := make([]uintptr, 10)
n := runtime.Callers(2, callersPCs) //skip first 2 entries, (Callers, GetStackTrace)
callersPCs = callersPCs[:n]
b := make([]byte, 1000)
runtime.Stack(b, false)
stackString := string(b)
frames := runtime.CallersFrames(callersPCs)
frame, _ := frames.Next()
trimmedString := strings.Split(strings.Split(stackString, "(")[2], ")")[0]
trimmedString = strings.Replace(trimmedString, " ", "", -1)
parametersPointers := strings.Split(trimmedString, ",")
return fmt.Sprintf("Name: %s \nParameters: %s \nReceived Error: %s", frame.Function, parametersPointers, err.Error())
}
If there are other ideas for building such logging function without analysing the stack trace, except the one that consists in passing a map[string]interface{} containing all the passed parameter names as keys and their values as values (that is my current implementation and is tedious since I'd like to log errors very often), I'd be glad to read them.
With reference to this:
https://play.golang.org/p/0kYRHO5f7kE
If I have 20+ different fields, if one of the fields in the Struct is empty, don't update it. Only update the ones with values in them.
What's the best way forward? I've seen passing as variadic input to another function but how best can I do this elegantly?
you can use this library to convert your struct fields into map of interfaces (can be done by yourself using reflect from stdlib) then loop over it
pipe := redisClient.TxPipeline()
m := structs.Map(server)
for k, v := range m {
pipe.HMSet(username, k, v)
}
cmder, err := pipe.Exec()
if err != nil {
return nil, err
}
the driver for redis used is go-redis
I'm trying to scan some folders and sort them to find the highest version number. {"10.1","9.6","7.2"} and then build a path. However, What I get has [] brackets in the path and I need to get rid of those.
Here's what I'm getting:
C:\Program Files\PostgreSQL\[10.1]\bin\psql.exe
root := "C:/Program Files/PostgreSQL"
files, err := ioutil.ReadDir(root)
if err != nil {
return "", err
}
folders := []float64{}
for _, f := range files {
if f.IsDir() {
if converted, err := strconv.ParseFloat(f.Name(),64); err == nil {
folders = append(folders, converted)
}
}
}
sort.Float64s(folders)
log.Println(folders[len(folders)-1:])
highestVersion := fmt.Sprintf("%v",folders[len(folders)-1:])
execPath = filepath.Join(root, highestVersion, "bin", "psql.exe")
log.Println(execPath)
The issues are on this line:
highestVersion := fmt.Sprintf("%v",folders[len(folders)-1:])
The %v format specifier, as some people have mentioned, is shorthand for "value". Now let's look at your value:
folders[len(folders)-1:]
What you are saying here is, "take a slice from folders starting at len(folders-1)". Your variable is a slice that only contains the last item in folders.
Slices are printed by using brackets around the values, and since you have one value, it prints the value surrounded by square brackets.
If you want to print just the float contained in that location, you should remove the colon as specified in a comment. I would recommend printing using the fmt verb %f or %g, depending on your use case.
More information can be found in the pkg/fmt docs about what verbs are available to printf and other related functions.
One possible approach would be to use a regular expression to ensure that each path has the expected format and to extract the version number as a float via a submatch (matching group), then sort the path strings by their floating point version number value and return the highest one.
For example:
func main() {
paths := []string{
`C:\Program Files\PostgreSQL\[1.2]\bin\psql.exe`,
`C:\Program Files\PostgreSQL\[9.6]\bin\psql.exe`,
`C:\Program Files\PostgreSQL\[10.1]\bin\psql.exe`,
`C:\Program Files\PostgreSQL\[7.2]\bin\psql.exe`,
}
sort.Slice(paths, func(i, j int) bool {
return parseVersion(paths[i]) >= parseVersion(paths[j])
})
fmt.Printf("OK: highest version path = %s\n", paths[0])
// OK: highest version path = C:\Program Files\PostgreSQL\[10.1]\bin\psql.exe
}
var re = regexp.MustCompile(`C:\\Program Files\\PostgreSQL\\\[(\d+\.\d+)\]\\bin\\psql.exe`)
func parseVersion(s string) float32 {
match := re.FindStringSubmatch(s)
if match == nil {
panic(fmt.Errorf("invalid path %q", s))
}
version, err := strconv.ParseFloat(match[1], 32)
if err != nil {
panic(err)
}
return float32(version)
}
Of course, you could modify the path regular expression to match different location patterns if that matters for your use case.