I am new in Golang and need some help! I have several questions.
In PostgreSQL database I have table called surveys.
CREATE TABLE SURVEYS(
SURVEY_ID UUID PRIMARY KEY NOT NULL DEFAULT uuid_generate_v4(),
SURVEY_NAME VARCHAR NOT NULL,
SURVEY_DESCRIPTION TEXT,
START_PERIOD TIMESTAMP,
END_PERIOD TIMESTAMP
);
As you can see SURVEY_ID column is PRIMARY KEY and it's type is UUID4.
In Golang application I create such struct to this table:
type Survey struct {
ID string `json:"survey_id"`
Name string `json:"survey_name"`
Description utils.NullString `json:"survey_description"`
StartPeriod utils.NullTime `json:"start_period"`
EndPeriod utils.NullTime `json:"end_period"`
}
As you can see type of ID field is string. Is it correct? I am not sure that it's best practice.
My second question about strange result which I have when make GET request to specific survey by it's ID.
For example when I make such request:
http://localhost:8000/api/survey/0cf1cf18-d5fd-474e-a8be-754fbdc89720
As response I have this:
{
"survey_id": "0cf1cf18-d5fd-474e-a8be-754fbdc89720",
"survey_name": "NAME",
"survey_description": {
"String": "DESCRIPTION",
"Valid": true
},
"start_period": {
"Time": "2019-01-01T00:00:00Z",
"Valid": false
},
"end_period": {
"Time": "0001-01-01T00:00:00Z",
"Valid": false
}
}
As you can see something wrong with last 3 field: survey_description, start_period and end_period. I want to see key and value in one line. For example as here:
{
"survey_id": "0cf1cf18-d5fd-474e-a8be-754fbdc89720",
"survey_name": "NAME",
"survey_description": "DESCRIPTION",
"start_period": "2019-01-01 00:00:00",
"end_period": null
}
Where exactly I make mistake in my code?
utils.go:
package utils
import (
"database/sql"
"encoding/json"
"fmt"
"github.com/lib/pq"
"time"
)
// NullTime is an alias for pq.NullTime data type.
type NullTime struct {
pq.NullTime
}
// MarshalJSON for NullTime.
func (nt *NullTime) MarshalJSON() ([]byte, error) {
if !nt.Valid {
return []byte("null"), nil
}
val := fmt.Sprintf("\"%s\"", nt.Time.Format(time.RFC3339))
return []byte(val), nil
}
// UnmarshalJSON for NullTime.
func (nt *NullTime) UnmarshalJSON(b []byte) error {
err := json.Unmarshal(b, &nt.Time)
nt.Valid = err == nil
return err
}
// NullInt64 is an alias for sql.NullInt64 data type.
type NullInt64 struct {
sql.NullInt64
}
// MarshalJSON for NullInt64.
func (ni *NullInt64) MarshalJSON() ([]byte, error) {
if !ni.Valid {
return []byte("null"), nil
}
return json.Marshal(ni.Int64)
}
// UnmarshalJSON for NullInt64.
func (ni *NullInt64) UnmarshalJSON(b []byte) error {
err := json.Unmarshal(b, &ni.Int64)
ni.Valid = err == nil
return err
}
// NullString is an alias for sql.NullString data type.
type NullString struct {
sql.NullString
}
// MarshalJSON for NullString.
func (ns *NullString) MarshalJSON() ([]byte, error) {
if !ns.Valid {
return []byte("null"), nil
}
return json.Marshal(ns.String)
}
// UnmarshalJSON for NullString.
func (ns *NullString) UnmarshalJSON(b []byte) error {
err := json.Unmarshal(b, &ns.String)
ns.Valid = err == nil
return err
}
routes.go:
router.HandleFunc("/api/survey/{survey_id:[a-fA-F0-9]{8}-[a-fA-F0-9]{4}-4[a-fA-F0-9]{3}-[8|9|aA|bB][a-fA-F0-9]{3}-[a-fA-F0-9]{12}}", controllers.GetSurvey).Methods("GET")
controllers/survey.go:
var GetSurvey = func(responseWriter http.ResponseWriter, request *http.Request) {
// Initialize variables.
survey := models.Survey{}
var err error
vars := mux.Vars(request)
// Execute SQL statement.
err = database.DB.QueryRow("SELECT * FROM surveys WHERE survey_id = $1;", vars["survey_id"]).Scan(&survey.ID, &survey.Name, &survey.Description, &survey.StartPeriod, &survey.EndPeriod)
// Shape the response depending on the result of the previous command.
if err != nil {
log.Println(err)
switch err {
case sql.ErrNoRows:
utils.ResponseWithError(responseWriter, http.StatusNotFound, "The entry not found.")
default:
utils.ResponseWithError(responseWriter, http.StatusInternalServerError, err.Error())
}
return
}
utils.Response(responseWriter, http.StatusOK, survey)
}
Well, finally I found the result.
I changed the struct for the table:
type Survey struct {
ID string `json:"survey_id"`
Name string `json:"survey_name"`
Description *string `json:"survey_description", sql:"index"`
StartPeriod *time.Time `json:"start_period", sql:"index"`
EndPeriod *time.Time `json:"end_period", sql:"index"`
}
I don't see any issue with using a string for a UUID.
As for the MarshalJSON not working, I think I know what's going on. Your null types don't implement MarshalJSON, only the pointers to them. The fix would be to either change the function to use a non-pointer receiver, or make the fields pointers in your struct.
func (ns *NullString) MarshalJSON() ([]byte, error)
If you did make them pointers, then you could just keep them like that, since they're nullable.
Related
trying to create auth service that is abstract by using generic type. I wanted to create package Auth that will have New() method for initialization but currently facing the issue that I can resolve. Anyone have idea if this is even possible and if this is even go way of solving the issue?
type Register[T any] interface {
SignUp(T) (*AuthEntity, error)
}
type AuthEntity struct {
ID string
UpdatedAt time.Time
CreatedAt time.Time
}
type AuthService[T any] struct {
Register[T]
}
type PasswordStrategy struct{}
type SignUpViaPassword struct {
Email string
Password string
}
func (s PasswordStrategy) SignUp(signUpData SignUpViaPassword) (*AuthEntity, error) {
return &AuthEntity{ID: "email#gmail.com"}, nil
}
func New[T any](register Register[T]) AuthService[T] {
return AuthService[T]{
Register: register,
}
}
type GoogleStrategy struct{}
type SignUpViaGoogle struct {
Token string
}
func (s GoogleStrategy) SignUp(signUpData SignUpViaGoogle) (*AuthEntity, error) {
return &AuthEntity{}, nil
}
func main() {
// This works as expected
authService := AuthService[SignUpViaPassword]{
Register: PasswordStrategy{},
}
entity, err := authService.SignUp(SignUpViaPassword{Password: "Kako ide", Email: "email#gmail.com"})
if err != nil {
log.Println(err)
}
log.Println(entity)
// This also works as expected
googleAuthService := AuthService[SignUpViaGoogle]{
Register: GoogleStrategy{},
}
entity2, err := authService.SignUp(SignUpViaPassword{Password: "Kako ide", Email: "email#gmail.com"})
if err != nil {
log.Println(err)
}
log.Println(entity2)
// this doesn't work and returns error type PasswordStrategy of PasswordStrategy{} does not match Register[T] (cannot infer T)
New(PasswordStrategy{})
New(GoogleStrategy{})
}
It cannot infer what T should be, so you can explicitly specify it:
New[SignUpViaPassword](PasswordStrategy{})
I want to create a generic Redis interface for storing and getting values.
I am beginner to Golang and redis.
If there are any changes to be done to the code I would request you to help me.
package main
import (
"fmt"
"github.com/go-redis/redis"
)
func main() {
student := map[string]string{
"id": "st01",
"name": "namme1",
}
set("key1", student, 0)
get("key1")
}
func set(key string, value map[string]string, ttl int) bool {
client := redis.NewClient(&redis.Options{
Addr: "localhost:6379",
Password: "",
DB: 0,
})
err := client.Set(key, value, 0).Err()
if err != nil {
fmt.Println(err)
return false
}
return true
}
func get(key string) bool {
client := redis.NewClient(&redis.Options{
Addr: "localhost:6379",
Password: "",
DB: 0,
})
val, err := client.Get(key).Result()
if err != nil {
fmt.Println(err)
return false
}
fmt.Println(val)
return true
}
When i run this code i receive an error of "redis: can't marshal map[string]string (implement encoding.BinaryMarshaler)".
I have tried using marshal but there was no use.
I would request you to help me with this.
The non-scalar type of go cannot be directly converted to the storage structure of redis, so the structure needs to be converted before storage
If you want to implement a general method, then the method should receive a type that can be stored directly, and the caller is responsible for converting the complex structure into a usable type, for example:
// ...
student := map[string]string{
"id": "st01",
"name": "namme1",
}
// Errors should be handled here
bs, _ := json.Marshal(student)
set("key1", bs, 0)
// ...
func set(key string, value interface{}, ttl int) bool {
// ...
}
A specific method can structure a specific structure, but the structure should implement the serializers encoding.MarshalBinary and encoding.UnmarshalBinary, for example:
type Student map[string]string
func (s Student) MarshalBinary() ([]byte, error) {
return json.Marshal(s)
}
// make sure the Student interface here accepts a pointer
func (s *Student) UnmarshalBinary(data []byte) error {
return json.Unmarshal(data, s)
}
// ...
student := Student{
"id": "st01",
"name": "namme1",
}
set("key1", student, 0)
// ...
func set(key string, value Student, ttl int) bool {
// ...
}
In PostgreSQL, I have table called surveys.
CREATE TABLE SURVEYS(
SURVEY_ID UUID PRIMARY KEY NOT NULL DEFAULT uuid_generate_v4(),
SURVEY_NAME VARCHAR NOT NULL,
SURVEY_DESCRIPTION TEXT,
START_PERIOD TIMESTAMP,
END_PERIOD TIMESTAMP
);
As you can see only SURVEY_ID and SURVEY_NAME columns are NOT NULL.
In Go, I want to create new entry in that table by POST request. I send JSON object like this:
{
"survey_name": "NAME",
"survey_description": "DESCRIPTION",
"start_period": "2019-01-01 00:00:00",
"end_period": "2019-02-28 23:59:59"
}
Unfortunatly it raise strange ERROR:
parsing time ""2019-01-01 00:00:00"" as ""2006-01-02T15:04:05Z07:00"": cannot parse " 00:00:00"" as "T"
Where I make mistake and how to fix my problem?
models/surveys.go:
import (
"database/sql"
"time"
)
type NullTime struct {
time.Time
Valid bool
}
type Survey struct {
ID int `json:"survey_id"`
Name string `json:"survey_name"`
Description sql.NullString `json:"survey_description"`
StartPeriod NullTime `json:"start_period"`
EndPeriod NullTime `json:"end_period"`
}
controllers/surveys.go:
var CreateSurvey = func(responseWriter http.ResponseWriter, request *http.Request) {
// Initialize variables.
survey := models.Survey{}
var err error
// The decoder introduces its own buffering and may read data from argument beyond the JSON values requested.
err = json.NewDecoder(request.Body).Decode(&survey)
if err != nil {
log.Println(err)
utils.ResponseWithError(responseWriter, http.StatusInternalServerError, err.Error())
return
}
defer request.Body.Close()
// Execute INSERT SQL statement.
_, err = database.DB.Exec("INSERT INTO surveys (survey_name, survey_description, start_period, end_period) VALUES ($1, $2, $3, $4);", survey.Name, survey.Description, survey.StartPeriod, survey.EndPeriod)
// Shape the response depending on the result of the previous command.
if err != nil {
log.Println(err)
utils.ResponseWithError(responseWriter, http.StatusInternalServerError, err.Error())
return
}
utils.ResponseWithSuccess(responseWriter, http.StatusCreated, "The new entry successfully created.")
}
The error already says what is wrong:
parsing time ""2019-01-01 00:00:00"" as ""2006-01-02T15:04:05Z07:00"": cannot parse " 00:00:00"" as "T"
You are passing "2019-01-01 00:00:00" while it expects a different time format, namely RFC3339 (UnmarshalJSON's default).
To solve this, you either want to pass the time in the expected format "2019-01-01T00:00:00Z00:00" or define your own type CustomTime like this:
const timeFormat = "2006-01-02 15:04:05"
type CustomTime time.Time
func (ct *CustomTime) UnmarshalJSON(data []byte) error {
newTime, err := time.Parse(timeFormat, strings.Trim(string(data), "\""))
if err != nil {
return err
}
*ct = CustomTime(newTime)
return nil
}
func (ct *CustomTime) MarshalJSON() ([]byte, error) {
return []byte(fmt.Sprintf("%q", time.Time(*ct).Format(timeFormat))), nil
}
Careful, you might also need to implement the Valuer and the Scanner interfaces for the time to be parsed in and out of the database, something like the following:
func (ct CustomTime) Value() (driver.Value, error) {
return time.Time(ct), nil
}
func (ct *CustomTime) Scan(src interface{}) error {
if val, ok := src.(time.Time); ok {
*ct = CustomTime(val)
} else {
return errors.New("time Scanner passed a non-time object")
}
return nil
}
Go Playground example.
In my current go project (~5K LOC), I am using sqlite3 as my underlying database layer, and I am using gorm as my ORM engine. One of the models is a Platform with a field of PlatformType enum type. Here's a code snippet to demonstrate my problem.
package main
import (
_ "github.com/jinzhu/gorm/dialects/sqlite"
"github.com/jinzhu/gorm"
"database/sql/driver"
"fmt"
)
/****************************\
Object Layer
\****************************/
// Platform ID
type PlatformID string
func (u *PlatformID) Scan(value interface{}) error { *u = PlatformID(value.([]byte)); return nil }
func (u PlatformID) Value() (driver.Value, error) { return string(u), nil }
// Platform Type enumeration
type PlatformType int
const (
PLATFORM_TYPE_NOT_A_VALUE PlatformType = iota
PLATFORM_TYPE_TYPE1
PLATFORM_TYPE_TYPE2
)
var types = [...]string {
"Not a type",
"Type1",
"Type2",
}
func (platform_type PlatformType) String() string {
return types[platform_type]
}
func (u *PlatformType) Scan(value interface{}) error { *u = PlatformType(value.(int)); return nil }
func (u PlatformType) Value() (driver.Value, error) { return int(u), nil }
// Platform descriptor.
type Platform struct {
ID PlatformID `json:"ID" gorm:"type:varchar(100);unique;not null"` // Assigned by LCBO.
Type PlatformType `json:"Type" gorm:"type:integer"`
}
type PlatformStore interface {
Init() error
Save(platform *Platform) error
}
/****************************\
Persist Layer
\****************************/
func NewSqlite3Store(dbname string) *gorm.DB {
db, err := gorm.Open("sqlite3", dbname)
if err != nil {
panic("failed to connect database")
}
return db
}
type DBPlatformStore struct {
db *gorm.DB
}
func NewDBPlatformStore(db *gorm.DB) PlatformStore {
return &DBPlatformStore{
db: db,
}
}
func (store *DBPlatformStore) Init() error {
err := store.db.AutoMigrate(&Platform{}).Error
if err != nil {
panic(err)
}
return err
}
func (store *DBPlatformStore) Save(platform *Platform) error {
err := store.db.Create(platform).Error
if err != nil {
panic(err)
}
return err
}
/****************************\
Application Layer
\****************************/
func main() {
db := NewSqlite3Store("enum_test.db")
platformStore := NewDBPlatformStore(db)
fmt.Println("Initialize Database")
err := platformStore.Init()
if err != nil {
panic(err)
}
platform := new(Platform)
platform.ID = "12345"
platform.Type = PLATFORM_TYPE_TYPE1
platformStore.Save(platform)
}
After running the code above, I got a runtime error "sql: converting Exec argument #1's type: non-Value type int returned from Value"
]# go run enumtest.go
Initialize Database
panic: sql: converting Exec argument #1's type: non-Value type int returned from Value
goroutine 1 [running]:
panic(0x66d380, 0xc8203ae350)
/*/panic.go:481 +0x3e6
main.(*DBPlatformStore).Save(0xc820020b20, 0xc820304500, 0x0, 0x0)
/*/enumtest.go:84 +0x9f
main.main()
/*/enumtest.go:106 +0x247
exit status 2
And I checked my database, the platforms table has been created successfully.
]# sqlite3 enum_test.db
sqlite> .schema platforms
CREATE TABLE "platforms" ("id" varchar(100) NOT NULL UNIQUE,"type" integer , PRIMARY KEY ("id"));
The (not-so) trivial question is how do I modify my code so that I can correctly save the entry to database.
My bigger question is: How to save a customized GO enum type to a sql database?(with a ORM engine hopefully)
According to current database/sql docs, the sql has four builtin functions that returns driver.Value, and the underlying types are int64, float64, string and bool. So I guess that's the only four types supported.
I just changed the underlying type of my enum from int to int64 and things are working.
The problematic section is updated to the following snippet:
// Platform Type enumeration
type PlatformType int64
const (
PLATFORM_TYPE_NOT_A_VALUE PlatformType = iota
PLATFORM_TYPE_TYPE1
PLATFORM_TYPE_TYPE2
)
var types = [...]string {
"Not a type",
"Type1",
"Type2",
}
func (platform_type PlatformType) String() string {
return types[platform_type]
}
func (u *PlatformType) Scan(value interface{}) error { *u = PlatformType(value.(int64)); return nil }
func (u PlatformType) Value() (driver.Value, error) { return int64(u), nil }
Trying to get this approach to timestamps working in my application: https://gist.github.com/bsphere/8369aca6dde3e7b4392c#file-timestamp-go
Here it is:
package timestamp
import (
"fmt"
"labix.org/v2/mgo/bson"
"strconv"
"time"
)
type Timestamp time.Time
func (t *Timestamp) MarshalJSON() ([]byte, error) {
ts := time.Time(*t).Unix()
stamp := fmt.Sprint(ts)
return []byte(stamp), nil
}
func (t *Timestamp) UnmarshalJSON(b []byte) error {
ts, err := strconv.Atoi(string(b))
if err != nil {
return err
}
*t = Timestamp(time.Unix(int64(ts), 0))
return nil
}
func (t Timestamp) GetBSON() (interface{}, error) {
if time.Time(*t).IsZero() {
return nil, nil
}
return time.Time(*t), nil
}
func (t *Timestamp) SetBSON(raw bson.Raw) error {
var tm time.Time
if err := raw.Unmarshal(&tm); err != nil {
return err
}
*t = Timestamp(tm)
return nil
}
func (t *Timestamp) String() string {
return time.Time(*t).String()
}
and the article that goes with it: https://medium.com/coding-and-deploying-in-the-cloud/time-stamps-in-golang-abcaf581b72f
However, I'm getting the following error:
core/timestamp/timestamp.go:31: invalid indirect of t (type Timestamp)
core/timestamp/timestamp.go:35: invalid indirect of t (type Timestamp)
My relevant code looks like this:
import (
"github.com/path/to/timestamp"
)
type User struct {
Name string
Created_at *timestamp.Timestamp `bson:"created_at,omitempty" json:"created_at,omitempty"`
}
Can anyone see what I'm doing wrong?
Related question
I can't see how to implement this package either. Do I create a new User model something like this?
u := User{Name: "Joe Bloggs", Created_at: timestamp.Timestamp(time.Now())}
Your code has a typo. You can't dereference a non-pointer, so you need to make GetBSON a pointer receiver (or you could remove the indirects to t, since the value of t isn't changed by the method).
func (t *Timestamp) GetBSON() (interface{}, error) {
To set a *Timestamp value inline, you need to have a *time.Time to convert.
now := time.Now()
u := User{
Name: "Bob",
CreatedAt: (*Timestamp)(&now),
}
Constructor and a helper functions like New() and Now() may come in handy for this as well.
You cannot refer to an indirection of something that is not a pointer variable.
var a int = 3 // a = 3
var A *int = &a // A = 0x10436184
fmt.Println(*A == a) // true, both equals 3
fmt.Println(*&a == a) // true, both equals 3
fmt.Println(*a) // invalid indirect of a (type int)
Thus, you can not reference the address of a with *a.
Looking at where the error happens:
func (t Timestamp) GetBSON() (interface{}, error) {
// t is a variable type Timestamp, not type *Timestamp (pointer)
// so this is not possible at all, unless t is a pointer variable
// and you're trying to dereference it to get the Timestamp value
if time.Time(*t).IsZero() {
return nil, nil
}
// so is this
return time.Time(*t), nil
}