I have a struct that looks like this:
type authEnum int
const (
never authEnum = iota
sometimes
always
)
type Attrs struct {
Secret string `redis:"secret"`
RequireSecret authEnum `redis:"requireSecret"`
UserID string `redis:"userId"`
}
func (e *authEnum) RedisScan(src interface{}) error {
// This never gets called!
if e == nil {
return fmt.Errorf("nil pointer")
}
switch src := src.(type) {
case string:
if src == "false" || src == "never" {
*e = never
} else if src == "sometimes" {
*e = sometimes
} else { // "always" or "true"
*e = always
}
default:
return fmt.Errorf("cannot convert authEnum from %T to %T", src, e)
}
return nil
}
func getAttributes(ctx *AppContext, hash string) (*Attrs, error) {
rc := ctx.RedisPool.Get()
values, err := redis.Values(rc.Do("HGETALL", "redishash"))
rc.Close()
if err != nil {
return nil, err
}
attrs := Attrs{}
redis.ScanStruct(values, &attrs)
return &attrs, nil
}
How do I implement the Scanner interface on the RequireSecret attribute to parse an authEnum type out of "never", "sometimes" or "always" redis hash values?
How do I calculate the value and assign it to the authEnum?
In my code example RedisScan never gets called.
You don't implement interfaces on fields, but rather on types.
You can make your authEnum type satisfy the interface, simply by creating a method with the signature RedisScan(src interface{}) error on that type.
To assign to the receiver, you need to receive a pointer. Then you can assign to it as so:
func (e *authEnum) RedisScan(src interface{}) error {
var value authEnum
// Logic here to convert src to value
*e = value
}
Implement the method on a pointer receiver. Redis bulk strings are represented as []byte, not string:
func (e *authEnum) RedisScan(src interface{}) error {
b, ok := src.([]byte)
if !ok {
return fmt.Errorf("cannot convert authEnum from %T to %T", src, b)
}
switch string(b) {
case "false", "never":
*e = never
case "sometimes":
*e = sometimes
default:
*e = always
}
return nil
}
Always check and handle errors. The error returned from ScanStruct reports the type problem.
There's no need to check for nil pointer to the struct member. If the argument to ScanStruct is nil, then Redigo will panic well before the RedisScan method is called.
Related
I have the concept of Context which is a map that can hold any structure. Basically, I want to create a generic getter that adddressably 'populates' the destination interface (similarly to how json decoding works).
Here's an example of how I want this to work:
type Context map[string]interface{}
// Random struct that will be saved in the context
type Step struct {
Name string
}
func main() {
stepA := &Step{Name: "Cool Name"}
c := Context{}
c["stepA"] = stepA
var stepB *Step
err := c.Get("stepA", stepB)
if err != nil {
panic(err)
}
fmt.Println(stepB.Name) // Cool Name
stepB.Name = "CoolName2"
fmt.Println(stepA.Name) // I want to say: CoolName2
}
func (c Context) Get(stepId string, dest interface{}) error {
context, ok := c[stepId]
if !ok {
return nil
}
destinationValue := reflect.ValueOf(dest)
contextValue := reflect.ValueOf(context)
destinationValue.Set(contextValue) // Errors here
return nil
}
I leaned towards using reflect, but maybe I don't need it? - so opened to other suggestions (except for generics as that complicates other matters) I'm getting the following error with the above:
panic: reflect: reflect.Value.Set using unaddressable value
You can test it here.
The argument passed to Get must be a pointer type whose element type is identical to the type in the context map. So if the value in the context map is of type *Step, then the argument's type must be **Step. Also the passed in argument cannot be nil, it can be a pointer to nil, but it itself cannot be nil.
So in your case you should do:
var stepB *Step
err := c.Get("stepA", &stepB) // pass pointer-to-pointer
if err != nil {
panic(err)
}
And the Get method, fixed up a bit:
func (c Context) Get(stepId string, dest interface{}) error {
context, ok := c[stepId]
if !ok {
return nil
}
dv := reflect.ValueOf(dest)
if dv.Kind() != reflect.Ptr || dv.IsNil() {
return errors.New("dest must be non-nil pointer")
}
dv = dv.Elem()
cv := reflect.ValueOf(context)
if dv.Type() != cv.Type() {
return errors.New("dest type does not match context value type")
}
dv.Set(cv)
return nil
}
https://go.dev/play/p/OECttqp1aVg
In other words, how do I implement type-specific solutions for different types in a union type set?
Given the following code...
type FieldType interface {
string | int
}
type Field[T FieldType] struct {
name string
defaultValue T
}
func NewField[T FieldType](name string, defaultValue T) *Field[T] {
return &Field[T]{
name: name,
defaultValue: defaultValue,
}
}
func (f *Field[T]) Name() string {
return f.name
}
func (f *Field[T]) Get() (T, error) {
value, ok := os.LookupEnv(f.name)
if !ok {
return f.defaultValue, nil
}
return value, nil
}
the compiler shows the error:
field.go:37:9: cannot use value (variable of type string) as type T in return statement
Is there a way to provide implementations for all possible FieldTypes?
Like...
func (f *Field[string]) Get() (string, error) {
value, ok := os.LookupEnv(f.name)
if !ok {
return f.defaultValue, nil
}
return value, nil
}
func (f *Field[int]) Get() (int, error) {
raw, ok := os.LookupEnv(f.name)
if !ok {
return f.defaultValue, nil
}
value, err := strconv.ParseInt(raw, 10, 64)
if err != nil {
return *new(T), err
}
return int(value), nil
}
Any hint would be welcome.
The error occurs because operations that involve a type parameter (including assignments and returns) must be valid for all types in its type set.
In case of string | int, there isn't a common operation to initialize their value from a string.
However you still have a couple options:
Type-switch on T
You use the field with the generic type T in a type-switch, and temporarily set the values with concrete types into an interface{}/any. Then type-assert the interface back to T in order to return it. Beware that this assertion is unchecked, so it may panic if for some reason ret holds something that isn't in the type set of T. Of course you can check it with comma-ok but it's still a run-time assertion:
func (f *Field[T]) Get() (T, error) {
value, ok := os.LookupEnv(f.name)
if !ok {
return f.defaultValue, nil
}
var ret any
switch any(f.defaultValue).(type) {
case string:
ret = value
case int:
// don't actually ignore errors
i, _ := strconv.ParseInt(value, 10, 64)
ret = int(i)
}
return ret.(T), nil
}
Type-switch on *T
You can further simplify the code above and get rid of the empty interface. In this case you take the address of the T-type variable and switch on the pointer types. This is fully type-checked at compile time:
func (f *Field[T]) Get() (T, error) {
value, ok := env[f.name]
if !ok {
return f.defaultValue, nil
}
var ret T
switch p := any(&ret).(type) {
case *string:
*p = value
case *int:
i, _ := strconv.ParseInt(value, 10, 64)
*p = int(i)
}
// ret has the zero value if no case matches
return ret, nil
}
Note that in both cases you must convert the T value to an interface{}/any in order to use it in a type switch. You can't type-switch directly on T.
Playground with map to simulate os.LookupEnv: https://go.dev/play/p/JVBEZwCXRMW
Ok, the type switch works if reflections are used.
func (f *Field[T]) Get() (T, error) {
raw, ok := os.LookupEnv(f.name)
if !ok {
return f.defaultValue, nil
}
v := reflect.ValueOf(new(T))
switch v.Type().Elem().Kind() {
case reflect.String:
v.Elem().Set(reflect.ValueOf(raw))
case reflect.Int:
value, err := strconv.ParseInt(raw, 10, 64)
if err != nil {
return f.defaultValue, err
}
v.Elem().Set(reflect.ValueOf(int(value)))
}
return v.Elem().Interface().(T), nil
}
But better solutions are very welcome ;-)
I have JSON like this that I need to parse into a golang type:
{
name: "something"
rules: [
{
"itemTypeBasedConditions": [["containsAny", ["first_match", "second_match"]]],
"validity": "INVALID"
}]
}
The problem is that each array of the array in itemTypeBasedConditions contains a mix of strings (always first element) and another array (second element), and I am not sure how to parse all of that into an object that I could then manipulate.
I got to:
type RulesFile struct {
Name string
Rules []RulesItem
}
type RulesItem struct {
itemTypeBasedConditions [][]interface{}
validity bool
}
And then I guess I have to convert elements one by one from interface{} to either string (containsAny) or an array of strings ("first_match", "second_match")
Is there a better way of approaching this JSON parsing?
I would do something like this, you can probably alter this to your needs.
package main
import (
"encoding/json"
"fmt"
"os"
"reflect"
)
type RulesFile struct {
Name string `json:"name"`
Rules []RulesItem `json:"rules"`
}
type RulesItem struct {
ItemTypeBasedConditions [][]Condition `json:"itemTypeBasedConditions"`
Validity bool `json:"validity"`
}
type Condition struct {
Value *string
Array *[]string
}
func (c Condition) String() string {
if c.Value != nil {
return *c.Value
}
return fmt.Sprintf("%v", *c.Array)
}
func (c *Condition) UnmarshalJSON(data []byte) error {
var y interface{}
err := json.Unmarshal(data, &y)
if err != nil {
return err
}
switch reflect.TypeOf(y).String() {
case "string":
val := fmt.Sprintf("%v", y)
c.Value = &val
return nil
case "[]interface {}":
temp := y.([]interface{})
a := make([]string, len(temp))
for i, v := range temp {
a[i] = fmt.Sprint(v)
}
c.Array = &a
return nil
}
return fmt.Errorf("cannot unmarshall into string or []string: %v", y)
}
var input string = `
{
"name": "something",
"rules": [
{
"itemTypeBasedConditions": [["containsAny",["first_match", "second_match"]]],
"validity": false
}
]
}`
func main() {
var ruleFile RulesFile
err := json.Unmarshal([]byte(input), &ruleFile)
if err != nil {
fmt.Println(err)
os.Exit(1)
}
fmt.Printf("%+v\n", ruleFile)
}
You can implement the json.Unmarshaler interface. Have that implementation first unmarshal the json into a slice of json.RawMessage, then, once you've done that, you can unmarshal the individual elements to their corresponding types.
type Cond struct {
Name string
Args []string
}
func (c *Cond) UnmarshalJSON(data []byte) error {
// unmarshal into a slice of raw json
var raw []json.RawMessage
if err := json.Unmarshal(data, &raw); err != nil {
return err
} else if len(raw) != 2 {
return errors.New("unsupported number of elements in condition")
}
// unmarshal the first raw json element into a string
if err := json.Unmarshal(raw[0], &c.Name); err != nil {
return err
}
// unmarshal the second raw json element into a slice of string
return json.Unmarshal(raw[1], &c.Args)
}
https://go.dev/play/p/-tbr73TvX0d
I am a experienced python programmer but I am still new to Golang so my apologies if this is an obvious or silly question. But I am trying to create my own type that I want to act exactly like the base type with the exception of several methods being overridden. The reason for this is because several libraries I am using are checking the type against time.Time and I want it to match.
type PythonTime struct {
time.Time
}
var pythonTimeFormatStr = "2006-01-02 15:04:05-0700"
func (self *PythonTime) UnmarshalJSON(b []byte) (err error) {
// removes prepending/trailing " in the string
if b[0] == '"' && b[len(b)-1] == '"' {
b = b[1 : len(b)-1]
}
self.Time, err = time.Parse(pythonTimeFormatStr, string(b))
return
}
func (self *PythonTime) MarshalJSON() ([]byte, error) {
return []byte(self.Time.Format(pythonTimeFormatStr)), nil
}
type OtherType struct {
Uuid string `json:"uuid`
Second PythonTime `json:"second"`
Location string `json:"location"`
Action string `json:"action"`
Duration int `json:"duration"`
Value string `json:"value"`
}
So the the above works fine for marshalling and unmarshalling JSON. However, for my library that I am using (gocql and cqlr) they are checking if the type is a time.Time type so they can make some other modifications before putting it in C*. How do I get my PythonTime type to equate to either show as time.Time or override the default marshalling/unmarshalling for a time.Time object just for the use of my OtherType objects?
My temporary solution has been to modify their code and add a special case for the PythonTime type that does the same thing as the time.Time type. However, this is causing me circular imports and is not the best solution. Here is their code with my modifications.
func marshalTimestamp(info TypeInfo, value interface{}) ([]byte, error) {
switch v := value.(type) {
case Marshaler:
return v.MarshalCQL(info)
case int64:
return encBigInt(v), nil
case time.Time:
if v.IsZero() {
return []byte{}, nil
}
x := int64(v.UTC().Unix()*1e3) + int64(v.UTC().Nanosecond()/1e6)
return encBigInt(x), nil
case models.PythonTime:
x := int64(v.UTC().Unix()*1e3) + int64(v.UTC().Nanosecond()/1e6)
return encBigInt(x), nil
}
if value == nil {
return nil, nil
}
rv := reflect.ValueOf(value)
switch rv.Type().Kind() {
case reflect.Int64:
return encBigInt(rv.Int()), nil
}
return nil, marshalErrorf("can not marshal %T into %s", value, info)
}
Don't do this. You're checking for a time.Time object when you should be checking that it satisfies an interface.
type TimeLike interface {
Day() int
Format(string) string
... // whatever makes a "time" object to your code!
// looks like in this case it's
UTC() time.Time
IsZero() bool
}
then any code that expects a time.Time that can be substituted with a PythonTime, expect a TimeLike instead.
function Foo(value interface{}) int {
switch v := value.(type) {
case TimeLike:
return v.Day() // works for either time.Time or models.PythonTime
}
return 0
}
Just like you have done with the json.Marshaler and json.Unamrshaler, you can also implement the gocql.Marshaler gocql.Unamrshaler interfaces.
func (t *PythonTime) MarshalCQL(info gocql.TypeInfo) ([]byte, error) {
b := make([]byte, 8)
x := t.UnixNano() / int64(time.Millisecond)
binary.BigEndian.PutUint64(b, uint64(x))
return b, nil
}
func (t *PythonTime) UnmarshalCQL(info gocql.TypeInfo, data []byte) error {
x := int64(binary.BigEndian.Uint64(data)) * int64(time.Millisecond)
t.Time = time.Unix(0, x)
return nil
}
(note, untested in the context of CQL, but this does round-trip with itself)
Unfortunately, that will not work in Go. Your best option would be to create some import and export methods, so that you can cast your PythonTime to a time.Time and vice versa. That will give you flexibility you desire along with compatibility with other libraries.
package main
import (
"fmt"
"reflect"
"time"
)
func main() {
p, e := NewFromTime(time.Now())
if e != nil {
panic(e)
}
v, e := p.MarshalJSON()
if e != nil {
panic(e)
}
fmt.Println(string(v), reflect.TypeOf(p))
t, e := p.GetTime()
if e != nil {
panic(e)
}
fmt.Println(t.String(), reflect.TypeOf(t))
}
type PythonTime struct {
time.Time
}
var pythonTimeFormatStr = "2006-01-02 15:04:05-0700"
func NewFromTime(t time.Time) (*PythonTime, error) {
b, e := t.GobEncode()
if e != nil {
return nil, e
}
p := new(PythonTime)
e = p.GobDecode(b)
if e != nil {
return nil, e
}
return p, nil
}
func (self *PythonTime) GetTime() (time.Time, error) {
return time.Parse(pythonTimeFormatStr, self.Format(pythonTimeFormatStr))
}
func (self *PythonTime) UnmarshalJSON(b []byte) (err error) {
// removes prepending/trailing " in the string
if b[0] == '"' && b[len(b)-1] == '"' {
b = b[1 : len(b)-1]
}
self.Time, err = time.Parse(pythonTimeFormatStr, string(b))
return
}
func (self *PythonTime) MarshalJSON() ([]byte, error) {
return []byte(self.Time.Format(pythonTimeFormatStr)), nil
}
That should give output like this:
2016-02-04 14:32:17-0700 *main.PythonTime
2016-02-04 14:32:17 -0700 MST time.Time
How is one supposed to not render the null sql.NullString if it is not valid? - http://play.golang.org/p/pzSWS9vE0J
It doesn't seem to be working with omitempty struct tag and I can't quite figure out what to return from the MarshalJSON such that omitempty will be honored
type NS struct {
sql.NullString
}
func (ns *NS) MarshalJSON() ([]byte, error) {
fmt.Println("Marshaling json for NS")
if ns.String == "" && !ns.Valid {
return []byte("null"), nil
}
return json.Marshal(ns.String)
}
type A struct {
RStr string `json:rstr,omitempty"`
Str NS `json:"str,omitempty"`
}
func main() {
a := A{RStr: "rstr"}
s, _ := json.Marshal(&a)
fmt.Println(string(s))
}
As far as I know, there's no way to do that, however you can always use a pointer instead : http://play.golang.org/p/b4Q7YgpUa-
func main() {
a := A{"rstr", &sql.NullString{"Test", true}}
s, err := json.Marshal(&a)
fmt.Println(string(s), err)
a = A{}
fmt.Println(json.Unmarshal(s, &a))
fmt.Printf("%v %v\n", a.RStr, a.Str.String)
a = A{RStr: "rstr"}
s, err = json.Marshal(&a)
fmt.Println(string(s), err)
}
The reason why a pointer makes a difference from http://tip.golang.org/pkg/encoding/json/#Marshal:
-the field is empty and its tag specifies the "omitempty" option.
The empty values are false, 0, any nil pointer or interface value, and any array, slice, map, or string of length zero.