How is one supposed to not render the null sql.NullString if it is not valid? - http://play.golang.org/p/pzSWS9vE0J
It doesn't seem to be working with omitempty struct tag and I can't quite figure out what to return from the MarshalJSON such that omitempty will be honored
type NS struct {
sql.NullString
}
func (ns *NS) MarshalJSON() ([]byte, error) {
fmt.Println("Marshaling json for NS")
if ns.String == "" && !ns.Valid {
return []byte("null"), nil
}
return json.Marshal(ns.String)
}
type A struct {
RStr string `json:rstr,omitempty"`
Str NS `json:"str,omitempty"`
}
func main() {
a := A{RStr: "rstr"}
s, _ := json.Marshal(&a)
fmt.Println(string(s))
}
As far as I know, there's no way to do that, however you can always use a pointer instead : http://play.golang.org/p/b4Q7YgpUa-
func main() {
a := A{"rstr", &sql.NullString{"Test", true}}
s, err := json.Marshal(&a)
fmt.Println(string(s), err)
a = A{}
fmt.Println(json.Unmarshal(s, &a))
fmt.Printf("%v %v\n", a.RStr, a.Str.String)
a = A{RStr: "rstr"}
s, err = json.Marshal(&a)
fmt.Println(string(s), err)
}
The reason why a pointer makes a difference from http://tip.golang.org/pkg/encoding/json/#Marshal:
-the field is empty and its tag specifies the "omitempty" option.
The empty values are false, 0, any nil pointer or interface value, and any array, slice, map, or string of length zero.
Related
In other words, how do I implement type-specific solutions for different types in a union type set?
Given the following code...
type FieldType interface {
string | int
}
type Field[T FieldType] struct {
name string
defaultValue T
}
func NewField[T FieldType](name string, defaultValue T) *Field[T] {
return &Field[T]{
name: name,
defaultValue: defaultValue,
}
}
func (f *Field[T]) Name() string {
return f.name
}
func (f *Field[T]) Get() (T, error) {
value, ok := os.LookupEnv(f.name)
if !ok {
return f.defaultValue, nil
}
return value, nil
}
the compiler shows the error:
field.go:37:9: cannot use value (variable of type string) as type T in return statement
Is there a way to provide implementations for all possible FieldTypes?
Like...
func (f *Field[string]) Get() (string, error) {
value, ok := os.LookupEnv(f.name)
if !ok {
return f.defaultValue, nil
}
return value, nil
}
func (f *Field[int]) Get() (int, error) {
raw, ok := os.LookupEnv(f.name)
if !ok {
return f.defaultValue, nil
}
value, err := strconv.ParseInt(raw, 10, 64)
if err != nil {
return *new(T), err
}
return int(value), nil
}
Any hint would be welcome.
The error occurs because operations that involve a type parameter (including assignments and returns) must be valid for all types in its type set.
In case of string | int, there isn't a common operation to initialize their value from a string.
However you still have a couple options:
Type-switch on T
You use the field with the generic type T in a type-switch, and temporarily set the values with concrete types into an interface{}/any. Then type-assert the interface back to T in order to return it. Beware that this assertion is unchecked, so it may panic if for some reason ret holds something that isn't in the type set of T. Of course you can check it with comma-ok but it's still a run-time assertion:
func (f *Field[T]) Get() (T, error) {
value, ok := os.LookupEnv(f.name)
if !ok {
return f.defaultValue, nil
}
var ret any
switch any(f.defaultValue).(type) {
case string:
ret = value
case int:
// don't actually ignore errors
i, _ := strconv.ParseInt(value, 10, 64)
ret = int(i)
}
return ret.(T), nil
}
Type-switch on *T
You can further simplify the code above and get rid of the empty interface. In this case you take the address of the T-type variable and switch on the pointer types. This is fully type-checked at compile time:
func (f *Field[T]) Get() (T, error) {
value, ok := env[f.name]
if !ok {
return f.defaultValue, nil
}
var ret T
switch p := any(&ret).(type) {
case *string:
*p = value
case *int:
i, _ := strconv.ParseInt(value, 10, 64)
*p = int(i)
}
// ret has the zero value if no case matches
return ret, nil
}
Note that in both cases you must convert the T value to an interface{}/any in order to use it in a type switch. You can't type-switch directly on T.
Playground with map to simulate os.LookupEnv: https://go.dev/play/p/JVBEZwCXRMW
Ok, the type switch works if reflections are used.
func (f *Field[T]) Get() (T, error) {
raw, ok := os.LookupEnv(f.name)
if !ok {
return f.defaultValue, nil
}
v := reflect.ValueOf(new(T))
switch v.Type().Elem().Kind() {
case reflect.String:
v.Elem().Set(reflect.ValueOf(raw))
case reflect.Int:
value, err := strconv.ParseInt(raw, 10, 64)
if err != nil {
return f.defaultValue, err
}
v.Elem().Set(reflect.ValueOf(int(value)))
}
return v.Elem().Interface().(T), nil
}
But better solutions are very welcome ;-)
I have JSON like this that I need to parse into a golang type:
{
name: "something"
rules: [
{
"itemTypeBasedConditions": [["containsAny", ["first_match", "second_match"]]],
"validity": "INVALID"
}]
}
The problem is that each array of the array in itemTypeBasedConditions contains a mix of strings (always first element) and another array (second element), and I am not sure how to parse all of that into an object that I could then manipulate.
I got to:
type RulesFile struct {
Name string
Rules []RulesItem
}
type RulesItem struct {
itemTypeBasedConditions [][]interface{}
validity bool
}
And then I guess I have to convert elements one by one from interface{} to either string (containsAny) or an array of strings ("first_match", "second_match")
Is there a better way of approaching this JSON parsing?
I would do something like this, you can probably alter this to your needs.
package main
import (
"encoding/json"
"fmt"
"os"
"reflect"
)
type RulesFile struct {
Name string `json:"name"`
Rules []RulesItem `json:"rules"`
}
type RulesItem struct {
ItemTypeBasedConditions [][]Condition `json:"itemTypeBasedConditions"`
Validity bool `json:"validity"`
}
type Condition struct {
Value *string
Array *[]string
}
func (c Condition) String() string {
if c.Value != nil {
return *c.Value
}
return fmt.Sprintf("%v", *c.Array)
}
func (c *Condition) UnmarshalJSON(data []byte) error {
var y interface{}
err := json.Unmarshal(data, &y)
if err != nil {
return err
}
switch reflect.TypeOf(y).String() {
case "string":
val := fmt.Sprintf("%v", y)
c.Value = &val
return nil
case "[]interface {}":
temp := y.([]interface{})
a := make([]string, len(temp))
for i, v := range temp {
a[i] = fmt.Sprint(v)
}
c.Array = &a
return nil
}
return fmt.Errorf("cannot unmarshall into string or []string: %v", y)
}
var input string = `
{
"name": "something",
"rules": [
{
"itemTypeBasedConditions": [["containsAny",["first_match", "second_match"]]],
"validity": false
}
]
}`
func main() {
var ruleFile RulesFile
err := json.Unmarshal([]byte(input), &ruleFile)
if err != nil {
fmt.Println(err)
os.Exit(1)
}
fmt.Printf("%+v\n", ruleFile)
}
You can implement the json.Unmarshaler interface. Have that implementation first unmarshal the json into a slice of json.RawMessage, then, once you've done that, you can unmarshal the individual elements to their corresponding types.
type Cond struct {
Name string
Args []string
}
func (c *Cond) UnmarshalJSON(data []byte) error {
// unmarshal into a slice of raw json
var raw []json.RawMessage
if err := json.Unmarshal(data, &raw); err != nil {
return err
} else if len(raw) != 2 {
return errors.New("unsupported number of elements in condition")
}
// unmarshal the first raw json element into a string
if err := json.Unmarshal(raw[0], &c.Name); err != nil {
return err
}
// unmarshal the second raw json element into a slice of string
return json.Unmarshal(raw[1], &c.Args)
}
https://go.dev/play/p/-tbr73TvX0d
How do I iterate over the map of interfaces below to get the values of the interface returned in the map?
I have read through the extensive list of questions regarding iteration in Go but they didn't help me.
// https://api.kraken.com/0/public/AssetPairs
pairsResult, err := api.Query("AssetPairs", map[string]string{
})
if err != nil {
log.Fatal(err)
}
ks := reflect.ValueOf(pairsResult).MapKeys()
fmt.Printf(" %+v ", pairsResult) // result A below
fmt.Printf(" %+v ", ks) // result B below
// the value here is the value of MapKeys, which is the key
for kix, key := range ks {
fmt.Printf(" %+v %+v \n ", kix, key) // result C below
}
Result A
map[AAVEETH:map[aclass_base:currency aclass_quote:currency altname:AAVEETH base:AAVE fee_volume_currency:ZUSD fees:[[0 0.26] [50000 0.24] [100000 0.22] [250000 0.2] [500000 0.18]...
Result B
[KEEPXBT LINKUSD LINKXBT NANOEUR ...]
Result C
0 KEEPXBT 1 LINKUSD 2 LINKXBT 3 NANOEUR 4 USDTAUD ...
This is the source of the API wrapper function that is being called above
// Query sends a query to Kraken api for given method and parameters
func (api *KrakenAPI) Query(method string, data map[string]string) (interface{}, error) {
values := url.Values{}
for key, value := range data {
values.Set(key, value)
}
// Check if method is public or private
if isStringInSlice(method, publicMethods) {
return api.queryPublic(method, values, nil)
} else if isStringInSlice(method, privateMethods) {
return api.queryPrivate(method, values, nil)
}
return nil, fmt.Errorf("Method '%s' is not valid", method)
}
This happens when I try to iterate over the value:
This happens when I try to iterate over the initial result:
Assuming you're using this and from looking a bit into its source it seems to me that the concrete type of the result will be map[string]interface{}, if that is the case then you can do this.
res, err := api.Query("AssetPairs", map[string]string{})
if err != nil {
log.Fatal(err)
}
pairs, ok := res.(map[string]interface{})
if !ok {
log.Fatal("unsupported type")
}
for k, v := range pairs {
fmt.Printf("key=%s value=%+v\n ", k, v)
}
As the previous response mentions, we see that the interface returned becomes a map[string]interface{}, the following code would do the trick to retrieve the types:
for _, v := range d.(map[string]interface{}) {
switch v.(type) {
case map[string]interface{}:
fmt.Println("Its another map of string interface")
case interface{}:
fmt.Println("its an interface")
case string:
fmt.Println("its a string")
case []string:
fmt.Println("its a string array")
case float32:
fmt.Println("its a float32")
case float64:
fmt.Println("its a float64")
default:
fmt.Printf("Different thing, %T\n", v)
}
}
Code here:
https://play.golang.org/p/81LLYSvJVf8
However, I would recommend use explicit type, that would make your life much easier:
// Generated by https://quicktype.io
type KrakenTypes struct {
Error []interface{} `json:"error"`
Result map[string]Result `json:"result"`
}
type Result struct {
Altname string `json:"altname"`
Wsname *string `json:"wsname,omitempty"`
AclassBase Aclass `json:"aclass_base"`
Base string `json:"base"`
AclassQuote Aclass `json:"aclass_quote"`
Quote FeeVolumeCurrency `json:"quote"`
Lot Lot `json:"lot"`
PairDecimals int64 `json:"pair_decimals"`
LotDecimals int64 `json:"lot_decimals"`
LotMultiplier int64 `json:"lot_multiplier"`
LeverageBuy []int64 `json:"leverage_buy"`
LeverageSell []int64 `json:"leverage_sell"`
Fees [][]float64 `json:"fees"`
FeesMaker [][]float64 `json:"fees_maker"`
FeeVolumeCurrency FeeVolumeCurrency `json:"fee_volume_currency"`
MarginCall int64 `json:"margin_call"`
MarginStop int64 `json:"margin_stop"`
Ordermin *string `json:"ordermin,omitempty"`
}
Here we can use json decoding once the response was read, so the iterations to find out the types of each level could be avoided and we can just access to each member directly.
Complete code here: https://play.golang.org/p/v3tlroyx1mW
I am a experienced python programmer but I am still new to Golang so my apologies if this is an obvious or silly question. But I am trying to create my own type that I want to act exactly like the base type with the exception of several methods being overridden. The reason for this is because several libraries I am using are checking the type against time.Time and I want it to match.
type PythonTime struct {
time.Time
}
var pythonTimeFormatStr = "2006-01-02 15:04:05-0700"
func (self *PythonTime) UnmarshalJSON(b []byte) (err error) {
// removes prepending/trailing " in the string
if b[0] == '"' && b[len(b)-1] == '"' {
b = b[1 : len(b)-1]
}
self.Time, err = time.Parse(pythonTimeFormatStr, string(b))
return
}
func (self *PythonTime) MarshalJSON() ([]byte, error) {
return []byte(self.Time.Format(pythonTimeFormatStr)), nil
}
type OtherType struct {
Uuid string `json:"uuid`
Second PythonTime `json:"second"`
Location string `json:"location"`
Action string `json:"action"`
Duration int `json:"duration"`
Value string `json:"value"`
}
So the the above works fine for marshalling and unmarshalling JSON. However, for my library that I am using (gocql and cqlr) they are checking if the type is a time.Time type so they can make some other modifications before putting it in C*. How do I get my PythonTime type to equate to either show as time.Time or override the default marshalling/unmarshalling for a time.Time object just for the use of my OtherType objects?
My temporary solution has been to modify their code and add a special case for the PythonTime type that does the same thing as the time.Time type. However, this is causing me circular imports and is not the best solution. Here is their code with my modifications.
func marshalTimestamp(info TypeInfo, value interface{}) ([]byte, error) {
switch v := value.(type) {
case Marshaler:
return v.MarshalCQL(info)
case int64:
return encBigInt(v), nil
case time.Time:
if v.IsZero() {
return []byte{}, nil
}
x := int64(v.UTC().Unix()*1e3) + int64(v.UTC().Nanosecond()/1e6)
return encBigInt(x), nil
case models.PythonTime:
x := int64(v.UTC().Unix()*1e3) + int64(v.UTC().Nanosecond()/1e6)
return encBigInt(x), nil
}
if value == nil {
return nil, nil
}
rv := reflect.ValueOf(value)
switch rv.Type().Kind() {
case reflect.Int64:
return encBigInt(rv.Int()), nil
}
return nil, marshalErrorf("can not marshal %T into %s", value, info)
}
Don't do this. You're checking for a time.Time object when you should be checking that it satisfies an interface.
type TimeLike interface {
Day() int
Format(string) string
... // whatever makes a "time" object to your code!
// looks like in this case it's
UTC() time.Time
IsZero() bool
}
then any code that expects a time.Time that can be substituted with a PythonTime, expect a TimeLike instead.
function Foo(value interface{}) int {
switch v := value.(type) {
case TimeLike:
return v.Day() // works for either time.Time or models.PythonTime
}
return 0
}
Just like you have done with the json.Marshaler and json.Unamrshaler, you can also implement the gocql.Marshaler gocql.Unamrshaler interfaces.
func (t *PythonTime) MarshalCQL(info gocql.TypeInfo) ([]byte, error) {
b := make([]byte, 8)
x := t.UnixNano() / int64(time.Millisecond)
binary.BigEndian.PutUint64(b, uint64(x))
return b, nil
}
func (t *PythonTime) UnmarshalCQL(info gocql.TypeInfo, data []byte) error {
x := int64(binary.BigEndian.Uint64(data)) * int64(time.Millisecond)
t.Time = time.Unix(0, x)
return nil
}
(note, untested in the context of CQL, but this does round-trip with itself)
Unfortunately, that will not work in Go. Your best option would be to create some import and export methods, so that you can cast your PythonTime to a time.Time and vice versa. That will give you flexibility you desire along with compatibility with other libraries.
package main
import (
"fmt"
"reflect"
"time"
)
func main() {
p, e := NewFromTime(time.Now())
if e != nil {
panic(e)
}
v, e := p.MarshalJSON()
if e != nil {
panic(e)
}
fmt.Println(string(v), reflect.TypeOf(p))
t, e := p.GetTime()
if e != nil {
panic(e)
}
fmt.Println(t.String(), reflect.TypeOf(t))
}
type PythonTime struct {
time.Time
}
var pythonTimeFormatStr = "2006-01-02 15:04:05-0700"
func NewFromTime(t time.Time) (*PythonTime, error) {
b, e := t.GobEncode()
if e != nil {
return nil, e
}
p := new(PythonTime)
e = p.GobDecode(b)
if e != nil {
return nil, e
}
return p, nil
}
func (self *PythonTime) GetTime() (time.Time, error) {
return time.Parse(pythonTimeFormatStr, self.Format(pythonTimeFormatStr))
}
func (self *PythonTime) UnmarshalJSON(b []byte) (err error) {
// removes prepending/trailing " in the string
if b[0] == '"' && b[len(b)-1] == '"' {
b = b[1 : len(b)-1]
}
self.Time, err = time.Parse(pythonTimeFormatStr, string(b))
return
}
func (self *PythonTime) MarshalJSON() ([]byte, error) {
return []byte(self.Time.Format(pythonTimeFormatStr)), nil
}
That should give output like this:
2016-02-04 14:32:17-0700 *main.PythonTime
2016-02-04 14:32:17 -0700 MST time.Time
Trying to get this approach to timestamps working in my application: https://gist.github.com/bsphere/8369aca6dde3e7b4392c#file-timestamp-go
Here it is:
package timestamp
import (
"fmt"
"labix.org/v2/mgo/bson"
"strconv"
"time"
)
type Timestamp time.Time
func (t *Timestamp) MarshalJSON() ([]byte, error) {
ts := time.Time(*t).Unix()
stamp := fmt.Sprint(ts)
return []byte(stamp), nil
}
func (t *Timestamp) UnmarshalJSON(b []byte) error {
ts, err := strconv.Atoi(string(b))
if err != nil {
return err
}
*t = Timestamp(time.Unix(int64(ts), 0))
return nil
}
func (t Timestamp) GetBSON() (interface{}, error) {
if time.Time(*t).IsZero() {
return nil, nil
}
return time.Time(*t), nil
}
func (t *Timestamp) SetBSON(raw bson.Raw) error {
var tm time.Time
if err := raw.Unmarshal(&tm); err != nil {
return err
}
*t = Timestamp(tm)
return nil
}
func (t *Timestamp) String() string {
return time.Time(*t).String()
}
and the article that goes with it: https://medium.com/coding-and-deploying-in-the-cloud/time-stamps-in-golang-abcaf581b72f
However, I'm getting the following error:
core/timestamp/timestamp.go:31: invalid indirect of t (type Timestamp)
core/timestamp/timestamp.go:35: invalid indirect of t (type Timestamp)
My relevant code looks like this:
import (
"github.com/path/to/timestamp"
)
type User struct {
Name string
Created_at *timestamp.Timestamp `bson:"created_at,omitempty" json:"created_at,omitempty"`
}
Can anyone see what I'm doing wrong?
Related question
I can't see how to implement this package either. Do I create a new User model something like this?
u := User{Name: "Joe Bloggs", Created_at: timestamp.Timestamp(time.Now())}
Your code has a typo. You can't dereference a non-pointer, so you need to make GetBSON a pointer receiver (or you could remove the indirects to t, since the value of t isn't changed by the method).
func (t *Timestamp) GetBSON() (interface{}, error) {
To set a *Timestamp value inline, you need to have a *time.Time to convert.
now := time.Now()
u := User{
Name: "Bob",
CreatedAt: (*Timestamp)(&now),
}
Constructor and a helper functions like New() and Now() may come in handy for this as well.
You cannot refer to an indirection of something that is not a pointer variable.
var a int = 3 // a = 3
var A *int = &a // A = 0x10436184
fmt.Println(*A == a) // true, both equals 3
fmt.Println(*&a == a) // true, both equals 3
fmt.Println(*a) // invalid indirect of a (type int)
Thus, you can not reference the address of a with *a.
Looking at where the error happens:
func (t Timestamp) GetBSON() (interface{}, error) {
// t is a variable type Timestamp, not type *Timestamp (pointer)
// so this is not possible at all, unless t is a pointer variable
// and you're trying to dereference it to get the Timestamp value
if time.Time(*t).IsZero() {
return nil, nil
}
// so is this
return time.Time(*t), nil
}