How to update default values from http.Request elegantly? - go

I'm doing book gopl's exercise 1.12, Basically, the code need to update several default values from http.Request if it is present in URL parameters.
Say here is the code I'm working on:
var ( // Need to update those values if corresponding parameter present in URL
cycles = 5
res = 0.001
size = 100
)
I can do the updating one by one:
if c := r.FormValue("cycles"); c != "" { // r is a *http.Request
i, err := strconv.ParseInt(c); err != nil {
cycles = i
}
}
if r := r.FormValue("res"); r != "" {
if f, err := strconv.ParseFloat(r); err != nil {
res = f
}
}
// ...
But I'm not satisfied by this solution:
If I have dozens of params, this is very cumbersome
How to handle the conversion errors?
The repeating pattern seems requires a function, like this (I don't know how to implement it yet, just showing my thought)
func setParam(p interface{}, name string, r *http.Request) error {
if f := r.FormValue(name); f != "" {
switch p.(type) {
case int:
// strconv.ParseInt
case float64:
// strconv.PraseFloat
// ...
}
}
This looks better, but still cumbersome. I don't know if this is the best solution. Or I overlooked some feature in Go that should be used in this situation.
So, what's the idiomatic way to do this?

Write functions that get a form value as a specific type or return default when value is missing. Example:
func intValue(r *http.Request, name string, def int) (int, error) {
if _, ok := r.Form[name]; !ok {
return def, nil
}
return strconv.Atoi(r.FormValue(name))
}
Call these functions from your handler. This is repetitive like the code in the question, but combines variable declaration, default value and fetching value in a single line of code.
cycles, err := intValue(r, "cycles", 5)
if err != nil {
// TODO; handle bad value
}

Related

Recursive data structure unmarshalling gives error "cannot parse invalid wire-format data" in Go Lang Protobuf

OS and protobuf version
go1.18.1 linux/amd64, github.com/golang/protobuf v1.5.2
Introduction
I am trying to use recursive proto definitions.
.proto file
message AsyncConsensus {
int32 sender = 1;
int32 receiver = 2;
string unique_id = 3; // to specify the fall back block id to which the vote asyn is for
int32 type = 4; // 1-propose, 2-vote, 3-timeout, 4-propose-async, 5-vote-async, 6-timeout-internal, 7-consensus-external-request, 8-consensus-external-response, 9-fallback-complete
string note = 5;
int32 v = 6 ; // view number
int32 r = 7;// round number
message Block {
string id = 1;
int32 v = 2 ; // view number
int32 r = 3;// round number
Block parent = 4;
repeated int32 commands = 5;
int32 level = 6; // for the fallback mode
}
Block blockHigh = 8;
Block blockNew = 9;
Block blockCommit = 10;
}
The following is how I Marshal and Un-Marshal
func (t *AsyncConsensus) Marshal(wire io.Writer) error {
data, err := proto.Marshal(t)
if err != nil {
return err
}
lengthWritten := len(data)
var b [8]byte
bs := b[:8]
binary.LittleEndian.PutUint64(bs, uint64(lengthWritten))
_, err = wire.Write(bs)
if err != nil {
return err
}
_, err = wire.Write(data)
if err != nil {
return err
}
return nil
}
func (t *AsyncConsensus) Unmarshal(wire io.Reader) error {
var b [8]byte
bs := b[:8]
_, err := io.ReadFull(wire, bs)
if err != nil {
return err
}
numBytes := binary.LittleEndian.Uint64(bs)
data := make([]byte, numBytes)
length, err := io.ReadFull(wire, data)
if err != nil {
return err
}
err = proto.Unmarshal(data[:length], t)
if err != nil {
return err
}
return nil
}
func (t *AsyncConsensus) New() Serializable {
return new(AsyncConsensus)
}
My expected outcome
When marshaled and sent to the same process via TCP, it should correctly unmarshal and produce correct data structures.
Resulting error
error "cannot parse invalid wire-format data"
Additional information
I tried with non-recursive .proto definitions, and never had this issue before.
The stupidest error I can think about is that the wire.Write(bs) don’t write as many bytes as the io.ReadFull(wire, bs) read - so I’d just make sure that their return value is actually 8 in both cases.
Then I don’t know the golang/protobuf very well, but I guess it should be able to do this. Shouldn’t you create the go-code and then call out to it? I’m not sure how to call it.
If you think that it’s actually a problem in the protobuf implementation, there are some online protobuf-decoders, which can help. But they sometimes interpret the stream incorrectly, which could be the case here with a recursive pattern, so you have to be careful. But at least they helped me to debug the dedis/protobuf package more than once.
As a last resort you can make a minimal example with recursive data, check if it works, and then slowly add fields until it breaks…
This is not a bug with Protobuf, but its a mater of how you marshal and unmarshal protobuf structs.
As a concrete guideline, never concurrently marshal and unmarshal protobuf structs as it my lead to race conditions.
In the specific example you have provided, I see recursive data structs, so even if you use a separate struct for each invocation of marshal and unmarshal, it's likely that the pointers in the parent can lead to shared pointers.
Use a deep copy technique to remove any dependency so that you do not run in to race conditions.
func CloneMyStruct(orig *proto.AsyncConsensus_Block) (*proto.AsyncConsensus_Block, error) {
origJSON, err := json.Marshal(orig)
if err != nil {
return nil, err
}
clone := proto.AsyncConsensus_Block{}
if err = json.Unmarshal(origJSON, &clone); err != nil {
return nil, err
}
return &clone, nil
}

Golang parse array

I am trying to figure out why my code is not working. I wish to take a slice of numbers and strings, and separate it into three slices. For each element in the slice, if it is a string, append it to the strings slice, and if it is a positive number, append it to the positive numbers, and likewise with negative. Yet, here is the output
Names:
EvTremblay
45.39934611083154
-75.71148292845268
[Crestview -75.73795670904249
BellevueManor -75.73886856878032
Dutchie'sHole -75.66809864107668 ...
Positives:[45.344387632924054 45.37223315413918 ... ]
Negatives: []
Here is my code. Can someone tell me what is causing the Negatives array to not have any values?
func main() {
fmt.Printf("%q\n", strings.Split("a,b,c", ","))
var names []string
var positives, negatives []float64
bs, err := ioutil.ReadFile("poolss.txt")
if err != nil {
return
}
str := string(bs)
fmt.Println(str)
tokens := strings.Split(str, ",")
for _, token := range tokens {
if num, err := strconv.ParseFloat(token, 64); err == nil {
if num > 0 {
positives = append(positives, num)
} else {
negatives = append(negatives, num)
}
} else {
names = append(names, token)
}
fmt.Println(token)
}
fmt.Println(fmt.Sprintf("Strings: %v",names))
fmt.Println(fmt.Sprintf("Positives: %v", positives))
fmt.Println(fmt.Sprintf("Negatives: %v",negatives))
for i := range names{
fmt.Println(names[i])
fmt.Println(positives[i])
fmt.Println(negatives[i])
}
}
Your code has strings as a variable name:
var strings []string
and strings as a package name:
tokens := strings.Split(str, ",")
Don't do that!
strings.Split undefined (type []string has no field or method Split)
Playground: https://play.golang.org/p/HfZGj0jOT-P
Your problem above I think lies with the extra \n attached to each float probably - you get no negative entries if you end in a linefeed or you would get one if you have no linefeed at the end. So insert a printf so that you can see the errors you're getting from strconv.ParseFloat and all will become clear.
Some small points which may help:
Check errors, and don't depend on an error to be of only one type (this is what is confusing you here) - always print the error if it arrives, particularly when debugging
Don't use the name of a package for a variable (strings), it won't end well
Use a datastructure which reflects your data
Use the CSV package to read CSV data
So for example for storing the data you might want:
type Place struct {
Name string
Latitude int64
Longitude int64
}
Then read the data into that, depending on the fact that cols are in a given order, and store it in a []Place.
Here's what I tried, it works now! Thanks for the help, everyone!
func main() {
findRoute("poolss.csv", 5)
}
func findRoute( filename string, num int) []Edge {
var route []Edge
csvFile, err := os.Open(filename)
if err != nil {
return route
}
reader := csv.NewReader(bufio.NewReader(csvFile))
var pools []Pool
for {
line, error := reader.Read()
if error == io.EOF {
break
} else if error != nil {
log.Fatal(error)
}
lat, err := strconv.ParseFloat(line[1], 64)
long, err := strconv.ParseFloat(line[2], 64)
if err == nil {
pools = append(pools, Pool{
name: line[0],
latitude: lat,
longitude: long,
})
}
}
return route
}

Golang: Can you type a returned interface{} in one statement?

Let's say I have this:
type Donut string
type Muffin string
func getPastry () (interface{}, error) {
// some logic - this is contrived
var d Donut
d = "Bavarian"
return d, nil
}
Is it possible to reduce this to one line:
p, err := getPastry()
thisPastry := p.(Donut)
In other words, something like this, which does not compile:
thisPastry, err := getPastry().(Donut, error)
Not that having two lines of code to get the "generic" and type it is a big deal, but it just feels wasteful and un-simple to me, and that usually means I'm missing something obvious :-)
You can't. Best you can do is write a helper function (and do the type assertion in that):
func getDonut(p interface{}, err error) (Donut, error) {
return p.(Donut), err
}
And then it becomes a one-line:
d, err := getDonut(getPastry())
Or you may even "incorporate" the getPastry() call in the helper function:
func getDonutPastry() (Donut, error) {
p, err := getPastry()
return p.(Donut), err
}
And then calling it (an even shorter one-liner):
d, err := getDonutPastry()
Note:
Of course if the value returned by getPastry() is not of dynamic type Donut, this will be a runtime panic. To prevent that, you may use the special form of the type assertion:
v, ok := x.(T)
Which yields an additional untyped boolean value. The value of ok is true if the assertion holds. Otherwise it is false and the value of v is the zero value for type T. No run-time panic occurs in this case.
Safe versions of the helper functions using the special form could look like this (they return an error rather than panic):
func getDonut2(p interface{}, err error) (Donut, error) {
if d, ok := p.(Donut); ok {
return d, err
} else {
return "", errors.New("Not a Donut!")
}
}
func getDonutPastry2() (Donut, error) {
p, err := getPastry()
if d, ok := p.(Donut); ok {
return d, err
} else {
return "", errors.New("Not a Donut!")
}
}
See related questions:
Return map like 'ok' in Golang on normal functions
Go: multiple value in single-value context
Short answer: It's not possible.
Long answer:
It will be more readable and understandable by more people as there are no long lines chaining together many things.
In your example, I'm assuming you will have to test for an error and then check for a Muffin etc.
To make it super clear which types you expect and what to do with them, you can do a type switch:
thisPastry, err := getPastry()
if err != nil { ... }
switch v := thisPastry.(type) {
case Donut:
fmt.Println(v)
case Muffin:
fmt.Println(v, "mmm!")
default:
// some kind of error?
}

Return if there is no error Golang

In Golang, is it ok to run the function
err, value := function()
if err == nil {
return value
}
instead of doing this:
err, value := function()
if err != nil {
panic(err)
}
return err
If so, is there any time advantages / bonuses?
This is not a non-fatal error. I am trying to convert something to different types, and i'm not sure which I should use.
A panic is similar to an exception, but doesn't get passed to the caller (aka when you call panic, it happens then and there; you don't get to wait). You should go with the first sample of your code, where you can attempt an action, fail, and continue.
func main() {
s1 := rand.NewSource(time.Now().UnixNano())
r1 := rand.New(s1)
// Generate some random numbers, and call into add()
for i := 0; i < 10; i++ {
s, err := add(r1.Intn(100), r1.Intn(100))
if err != nil {
fmt.Println(err)
continue
}
fmt.Println(s)
}
}
// Error if we get a sum over 100
func add(a int, b int) (int, error) {
s := a + b
if s > 100 {
return s, errors.New("Hey doofus, error!")
}
return s, nil
}
If you were to panic in this example, you'd be done (try it-- instead of returning an error do panic("Some error"). But instead, we determine there's an error, and we can try to generate another random number.
Like others have said, if you have a use case where you just can't recover (say you were trying to read from a file, but the file isn't there), you might decide it's better to panic. But if you have a long running process (like an API), you'll want to keep churning, despite any errors.
GoPlay here: http://play.golang.org/p/ThXTxVfM6R
OP has update his post with a use case-- he's trying to convert to a type. If you were to panic in this function, you would be dead in the water. Instead, we want to return an error, and let the caller decide what to do with the error. Take this as an example:
func interfaceToString(i interface{}) (string, error) {
if i == nil {
return "", errors.New("nil interface")
}
switch i.(type) {
case string:
return i.(string), nil
case float64:
return strconv.Itoa(int(i.(float64))), nil
case int:
return strconv.Itoa(i.(int)), nil
}
return "", errors.New(fmt.Sprintf("Unable to convert %v", i))
}
GoPlay here: http://play.golang.org/p/7y7v151EH4

Reading bytes into structs using reflection

I'm trying to write functions that will allow me to marshal/unmarshal simple structs into byte arrays. I've succeeded in writing Marshal, with help from the kind folks at #go-nuts, but I'm running into trouble writing Unmarshal.
// Unmarshal unpacks the binary data and stores it in the packet using
// reflection.
func Unmarshal(b []byte, t reflect.Type) (pkt interface{}, err error) {
buf := bytes.NewBuffer(b)
p := reflect.New(t)
v := reflect.ValueOf(p)
for i := 0; i < t.NumField(); i++ {
f := v.Field(i)
switch f.Kind() {
case reflect.String:
// length of string
var l int16
var e error
e = binary.Read(buf, binary.BigEndian, &l)
if e != nil {
err = e
return
}
// read length-of-string bytes from the buffer
raw := make([]byte, l)
_, e = buf.Read(raw)
if e != nil {
err = e
return
}
// convert the bytes to a string
f.SetString(bytes.NewBuffer(raw).String())
default:
e := binary.Read(buf, binary.BigEndian, f.Addr())
if e != nil {
err = e
return
}
}
}
pkt = p
return
}
The problem with the code above is that the call to f.Addr() near the end is apparently trying to get the address of an unaddressable value.
If there is an alternative solution, I would appreciate that as well. Either way, any help would be much appreciated.
Thanks!
I think you should use
v := p.Elem() // Get the value that 'p' points to
instead of
v := reflect.ValueOf(p)
Working example with lots of assumptions and a trivial data format:
package main
import (
"fmt"
"reflect"
"strconv"
)
// example marshalled format. lets say that marshalled data will have
// four bytes of a formatted floating point number followed by two more
// printable bytes.
type m42 []byte
// example struct we'd like to unmarshal into.
type packet struct {
S string // exported fields required for reflection
F float64
}
// example usage
func main() {
var p packet
if err := Unmarshal(m42("3.14Pi"), &p); err == nil {
fmt.Println(p)
} else {
fmt.Println(err)
}
}
func Unmarshal(data m42, structPtr interface{}) error {
vp := reflect.ValueOf(structPtr)
ve := vp.Elem() // settable struct Value
vt := ve.Type() // type info for struct
nStructFields := ve.NumField()
for i := 0; i < nStructFields; i++ {
fv := ve.Field(i) // settable field Value
sf := vt.Field(i) // StructField type information
// struct field name indicates which m42 field to unmarshal.
switch sf.Name {
case "S":
fv.SetString(string(data[4:6]))
case "F":
s := string(data[0:4])
if n, err := strconv.ParseFloat(s, 64); err == nil {
fv.SetFloat(n)
} else {
return err
}
}
}
return nil
}
Appropriate alternative solutions would depend heavily on the real data you need to support.
I'm going to bet that the reason f.Addr() has the problem because it actually isn't addressable.
the reflect package Type object has a method that will tell you if the type is addressable called CanAddr(). Assuming the field is addressable if it's not a string is not always true. If the struct is not passed in as a pointer to a struct then it's fields won't be addressable. For more details about what is and isn't addressable see: http://weekly.golang.org/pkg/reflect/#Value.CanAddr which outlines the correct rules.
Essentially for your code to work I think you need to ensure you always call it with a pointer to a struct.

Resources