I have a custom time format that is the result of some custom unmarshalling:
type customTime struct {
time.Time
}
I have implemented the Scanner and Valuer interface on this customTime like so:
func (ct *customTime) Scan(value interface{}) error {
ct.Time = value.(time.Time)
return nil
}
func (ct *customTime) Value() (driver.Value, error) {
return ct.Time, nil
}
But it still errs when I try to do the insert:
sql: converting Exec argument $3 type: unsupported type main.customTime, a struct
What am I missing?
Found the solution, Scanner and Valuer should be implemented on the actual value and not a pointer to the customTime
func (ct customTime) Scan(value interface{}) error {
ct.Time = value.(time.Time)
return nil
}
func (ct customTime) Value() (driver.Value, error) {
return ct.Time, nil
}
Related
gorm unsupported data type interface.
How do I save different structures to one field in json (gorm)? Type checking needed, not just serialization in Json.
Example code with error: unsupported data type: models.Fields
// table questions (GORM)
type Questions struct {
ID
Fields `json:"fields"`
// .....
}
type Fields interface{
// ...
}
// radio ----------
type Radio struct {
Text string
Img string
// ...
}
func (d Radio) Value() (driver.Value, error) {
return json.Marshal(d)
}
func (d *Radio) Scan(value interface{}) error {
b, ok := value.([]byte)
if !ok {
return errors.New("type assertion to []byte failed")
}
return json.Unmarshal(b, &d)
}
// Checkbox ----------
type Checkbox struct {
Text string
MaxCheckbox uint16
// ...
}
func (d Checkbox) Value() (driver.Value, error) {
return json.Marshal(d)
}
func (d *Checkbox) Scan(value interface{}) error {
b, ok := value.([]byte)
if !ok {
return errors.New("type assertion to []byte failed")
}
return json.Unmarshal(b, &d)
}
You might have already solved this, but you need to use polymorphism.
I am trying to write a simple custom marshaler and failing. Notice I have an interface that has three functions. Both Happy and Sad structs implement this interface by embedding the emotion struct which implements all the three required functions.
The problem is UnmarshalJSON does not get invoked when I call json.Unmarshal() on the pointer to either Happy or Sad and I can't understand why. You can reproduce the exact codebase in Go Playground or just look below. You will notice that while MarshalJSON is correctly called, UnmarshalJSON isn't.
type Emotion interface {
String() string
MarshalJSON() ([]byte, error)
UnmarshalJSON(data []byte) error
}
type emotion struct {
status string
}
func (s emotion) String() string {
return s.status
}
func (s emotion) MarshalJSON() ([]byte, error) {
fmt.Println("MarshalJSON is overriden: I am called fine")
x := struct {
Status string
}{
Status: s.String(),
}
return json.Marshal(x)
}
func (s *emotion) UnmarshalJSON(data []byte) error {
fmt.Println("MarshalJSON is overriden: I am never called")
y := struct {
Status string
}{
Status: "",
}
err := json.Unmarshal(data, &y)
if err != nil {
return err
}
s.status = y.Status
return nil
}
type Happy struct {
*emotion
}
// Job is not in any detention
type Sad struct {
*emotion
}
func main() {
x := Happy{&emotion{status: "happy"}}
jsonX, _ := json.Marshal(x)
var y Emotion
err := json.Unmarshal(jsonX, &y)
fmt.Printf("%v", err)
}
You cannot unmarshal into an abstract interface type.
An interface value is just a pointer to a type (associating that types methods) - it has no storage behind it - because an abstract type cannot know the exact size of any concrete value it may have in the future.
Using a concrete value type (that also implements that interface) will work:
y2 := emotion{}
err = json.Unmarshal(jsonX, &y2)
Playground: https://play.golang.org/p/8aCEjLgfKVQ
MarshalJSON is overriden: I am called fine
EXPECTED ERROR, Can't unmarshal into a non-concrete value: json: cannot unmarshal object into Go value of type main.Emotion
MarshalJSON is overriden: I am (fixed) and now called
SHOULD NOT ERROR: <nil>
VALUE: happy
In my current go project (~5K LOC), I am using sqlite3 as my underlying database layer, and I am using gorm as my ORM engine. One of the models is a Platform with a field of PlatformType enum type. Here's a code snippet to demonstrate my problem.
package main
import (
_ "github.com/jinzhu/gorm/dialects/sqlite"
"github.com/jinzhu/gorm"
"database/sql/driver"
"fmt"
)
/****************************\
Object Layer
\****************************/
// Platform ID
type PlatformID string
func (u *PlatformID) Scan(value interface{}) error { *u = PlatformID(value.([]byte)); return nil }
func (u PlatformID) Value() (driver.Value, error) { return string(u), nil }
// Platform Type enumeration
type PlatformType int
const (
PLATFORM_TYPE_NOT_A_VALUE PlatformType = iota
PLATFORM_TYPE_TYPE1
PLATFORM_TYPE_TYPE2
)
var types = [...]string {
"Not a type",
"Type1",
"Type2",
}
func (platform_type PlatformType) String() string {
return types[platform_type]
}
func (u *PlatformType) Scan(value interface{}) error { *u = PlatformType(value.(int)); return nil }
func (u PlatformType) Value() (driver.Value, error) { return int(u), nil }
// Platform descriptor.
type Platform struct {
ID PlatformID `json:"ID" gorm:"type:varchar(100);unique;not null"` // Assigned by LCBO.
Type PlatformType `json:"Type" gorm:"type:integer"`
}
type PlatformStore interface {
Init() error
Save(platform *Platform) error
}
/****************************\
Persist Layer
\****************************/
func NewSqlite3Store(dbname string) *gorm.DB {
db, err := gorm.Open("sqlite3", dbname)
if err != nil {
panic("failed to connect database")
}
return db
}
type DBPlatformStore struct {
db *gorm.DB
}
func NewDBPlatformStore(db *gorm.DB) PlatformStore {
return &DBPlatformStore{
db: db,
}
}
func (store *DBPlatformStore) Init() error {
err := store.db.AutoMigrate(&Platform{}).Error
if err != nil {
panic(err)
}
return err
}
func (store *DBPlatformStore) Save(platform *Platform) error {
err := store.db.Create(platform).Error
if err != nil {
panic(err)
}
return err
}
/****************************\
Application Layer
\****************************/
func main() {
db := NewSqlite3Store("enum_test.db")
platformStore := NewDBPlatformStore(db)
fmt.Println("Initialize Database")
err := platformStore.Init()
if err != nil {
panic(err)
}
platform := new(Platform)
platform.ID = "12345"
platform.Type = PLATFORM_TYPE_TYPE1
platformStore.Save(platform)
}
After running the code above, I got a runtime error "sql: converting Exec argument #1's type: non-Value type int returned from Value"
]# go run enumtest.go
Initialize Database
panic: sql: converting Exec argument #1's type: non-Value type int returned from Value
goroutine 1 [running]:
panic(0x66d380, 0xc8203ae350)
/*/panic.go:481 +0x3e6
main.(*DBPlatformStore).Save(0xc820020b20, 0xc820304500, 0x0, 0x0)
/*/enumtest.go:84 +0x9f
main.main()
/*/enumtest.go:106 +0x247
exit status 2
And I checked my database, the platforms table has been created successfully.
]# sqlite3 enum_test.db
sqlite> .schema platforms
CREATE TABLE "platforms" ("id" varchar(100) NOT NULL UNIQUE,"type" integer , PRIMARY KEY ("id"));
The (not-so) trivial question is how do I modify my code so that I can correctly save the entry to database.
My bigger question is: How to save a customized GO enum type to a sql database?(with a ORM engine hopefully)
According to current database/sql docs, the sql has four builtin functions that returns driver.Value, and the underlying types are int64, float64, string and bool. So I guess that's the only four types supported.
I just changed the underlying type of my enum from int to int64 and things are working.
The problematic section is updated to the following snippet:
// Platform Type enumeration
type PlatformType int64
const (
PLATFORM_TYPE_NOT_A_VALUE PlatformType = iota
PLATFORM_TYPE_TYPE1
PLATFORM_TYPE_TYPE2
)
var types = [...]string {
"Not a type",
"Type1",
"Type2",
}
func (platform_type PlatformType) String() string {
return types[platform_type]
}
func (u *PlatformType) Scan(value interface{}) error { *u = PlatformType(value.(int64)); return nil }
func (u PlatformType) Value() (driver.Value, error) { return int64(u), nil }
I am a experienced python programmer but I am still new to Golang so my apologies if this is an obvious or silly question. But I am trying to create my own type that I want to act exactly like the base type with the exception of several methods being overridden. The reason for this is because several libraries I am using are checking the type against time.Time and I want it to match.
type PythonTime struct {
time.Time
}
var pythonTimeFormatStr = "2006-01-02 15:04:05-0700"
func (self *PythonTime) UnmarshalJSON(b []byte) (err error) {
// removes prepending/trailing " in the string
if b[0] == '"' && b[len(b)-1] == '"' {
b = b[1 : len(b)-1]
}
self.Time, err = time.Parse(pythonTimeFormatStr, string(b))
return
}
func (self *PythonTime) MarshalJSON() ([]byte, error) {
return []byte(self.Time.Format(pythonTimeFormatStr)), nil
}
type OtherType struct {
Uuid string `json:"uuid`
Second PythonTime `json:"second"`
Location string `json:"location"`
Action string `json:"action"`
Duration int `json:"duration"`
Value string `json:"value"`
}
So the the above works fine for marshalling and unmarshalling JSON. However, for my library that I am using (gocql and cqlr) they are checking if the type is a time.Time type so they can make some other modifications before putting it in C*. How do I get my PythonTime type to equate to either show as time.Time or override the default marshalling/unmarshalling for a time.Time object just for the use of my OtherType objects?
My temporary solution has been to modify their code and add a special case for the PythonTime type that does the same thing as the time.Time type. However, this is causing me circular imports and is not the best solution. Here is their code with my modifications.
func marshalTimestamp(info TypeInfo, value interface{}) ([]byte, error) {
switch v := value.(type) {
case Marshaler:
return v.MarshalCQL(info)
case int64:
return encBigInt(v), nil
case time.Time:
if v.IsZero() {
return []byte{}, nil
}
x := int64(v.UTC().Unix()*1e3) + int64(v.UTC().Nanosecond()/1e6)
return encBigInt(x), nil
case models.PythonTime:
x := int64(v.UTC().Unix()*1e3) + int64(v.UTC().Nanosecond()/1e6)
return encBigInt(x), nil
}
if value == nil {
return nil, nil
}
rv := reflect.ValueOf(value)
switch rv.Type().Kind() {
case reflect.Int64:
return encBigInt(rv.Int()), nil
}
return nil, marshalErrorf("can not marshal %T into %s", value, info)
}
Don't do this. You're checking for a time.Time object when you should be checking that it satisfies an interface.
type TimeLike interface {
Day() int
Format(string) string
... // whatever makes a "time" object to your code!
// looks like in this case it's
UTC() time.Time
IsZero() bool
}
then any code that expects a time.Time that can be substituted with a PythonTime, expect a TimeLike instead.
function Foo(value interface{}) int {
switch v := value.(type) {
case TimeLike:
return v.Day() // works for either time.Time or models.PythonTime
}
return 0
}
Just like you have done with the json.Marshaler and json.Unamrshaler, you can also implement the gocql.Marshaler gocql.Unamrshaler interfaces.
func (t *PythonTime) MarshalCQL(info gocql.TypeInfo) ([]byte, error) {
b := make([]byte, 8)
x := t.UnixNano() / int64(time.Millisecond)
binary.BigEndian.PutUint64(b, uint64(x))
return b, nil
}
func (t *PythonTime) UnmarshalCQL(info gocql.TypeInfo, data []byte) error {
x := int64(binary.BigEndian.Uint64(data)) * int64(time.Millisecond)
t.Time = time.Unix(0, x)
return nil
}
(note, untested in the context of CQL, but this does round-trip with itself)
Unfortunately, that will not work in Go. Your best option would be to create some import and export methods, so that you can cast your PythonTime to a time.Time and vice versa. That will give you flexibility you desire along with compatibility with other libraries.
package main
import (
"fmt"
"reflect"
"time"
)
func main() {
p, e := NewFromTime(time.Now())
if e != nil {
panic(e)
}
v, e := p.MarshalJSON()
if e != nil {
panic(e)
}
fmt.Println(string(v), reflect.TypeOf(p))
t, e := p.GetTime()
if e != nil {
panic(e)
}
fmt.Println(t.String(), reflect.TypeOf(t))
}
type PythonTime struct {
time.Time
}
var pythonTimeFormatStr = "2006-01-02 15:04:05-0700"
func NewFromTime(t time.Time) (*PythonTime, error) {
b, e := t.GobEncode()
if e != nil {
return nil, e
}
p := new(PythonTime)
e = p.GobDecode(b)
if e != nil {
return nil, e
}
return p, nil
}
func (self *PythonTime) GetTime() (time.Time, error) {
return time.Parse(pythonTimeFormatStr, self.Format(pythonTimeFormatStr))
}
func (self *PythonTime) UnmarshalJSON(b []byte) (err error) {
// removes prepending/trailing " in the string
if b[0] == '"' && b[len(b)-1] == '"' {
b = b[1 : len(b)-1]
}
self.Time, err = time.Parse(pythonTimeFormatStr, string(b))
return
}
func (self *PythonTime) MarshalJSON() ([]byte, error) {
return []byte(self.Time.Format(pythonTimeFormatStr)), nil
}
That should give output like this:
2016-02-04 14:32:17-0700 *main.PythonTime
2016-02-04 14:32:17 -0700 MST time.Time
Trying to get this approach to timestamps working in my application: https://gist.github.com/bsphere/8369aca6dde3e7b4392c#file-timestamp-go
Here it is:
package timestamp
import (
"fmt"
"labix.org/v2/mgo/bson"
"strconv"
"time"
)
type Timestamp time.Time
func (t *Timestamp) MarshalJSON() ([]byte, error) {
ts := time.Time(*t).Unix()
stamp := fmt.Sprint(ts)
return []byte(stamp), nil
}
func (t *Timestamp) UnmarshalJSON(b []byte) error {
ts, err := strconv.Atoi(string(b))
if err != nil {
return err
}
*t = Timestamp(time.Unix(int64(ts), 0))
return nil
}
func (t Timestamp) GetBSON() (interface{}, error) {
if time.Time(*t).IsZero() {
return nil, nil
}
return time.Time(*t), nil
}
func (t *Timestamp) SetBSON(raw bson.Raw) error {
var tm time.Time
if err := raw.Unmarshal(&tm); err != nil {
return err
}
*t = Timestamp(tm)
return nil
}
func (t *Timestamp) String() string {
return time.Time(*t).String()
}
and the article that goes with it: https://medium.com/coding-and-deploying-in-the-cloud/time-stamps-in-golang-abcaf581b72f
However, I'm getting the following error:
core/timestamp/timestamp.go:31: invalid indirect of t (type Timestamp)
core/timestamp/timestamp.go:35: invalid indirect of t (type Timestamp)
My relevant code looks like this:
import (
"github.com/path/to/timestamp"
)
type User struct {
Name string
Created_at *timestamp.Timestamp `bson:"created_at,omitempty" json:"created_at,omitempty"`
}
Can anyone see what I'm doing wrong?
Related question
I can't see how to implement this package either. Do I create a new User model something like this?
u := User{Name: "Joe Bloggs", Created_at: timestamp.Timestamp(time.Now())}
Your code has a typo. You can't dereference a non-pointer, so you need to make GetBSON a pointer receiver (or you could remove the indirects to t, since the value of t isn't changed by the method).
func (t *Timestamp) GetBSON() (interface{}, error) {
To set a *Timestamp value inline, you need to have a *time.Time to convert.
now := time.Now()
u := User{
Name: "Bob",
CreatedAt: (*Timestamp)(&now),
}
Constructor and a helper functions like New() and Now() may come in handy for this as well.
You cannot refer to an indirection of something that is not a pointer variable.
var a int = 3 // a = 3
var A *int = &a // A = 0x10436184
fmt.Println(*A == a) // true, both equals 3
fmt.Println(*&a == a) // true, both equals 3
fmt.Println(*a) // invalid indirect of a (type int)
Thus, you can not reference the address of a with *a.
Looking at where the error happens:
func (t Timestamp) GetBSON() (interface{}, error) {
// t is a variable type Timestamp, not type *Timestamp (pointer)
// so this is not possible at all, unless t is a pointer variable
// and you're trying to dereference it to get the Timestamp value
if time.Time(*t).IsZero() {
return nil, nil
}
// so is this
return time.Time(*t), nil
}