struct Time property doesn't load from Go sqlx library - go

I have a struct with a property time:
type Basket struct {
...
Created_at time.Time `db:"created_at"`
}
with the time saved as:
basket.Created_at = time.Now().UTC()
If I save it using Insert sql statement, it saves the time nicely in the SQLite3 but when I select the desired record using:
ret_basket := Basket{}
err := database.DB.Get(&ret_basket, "SELECT id, ..., created_at FROM baskets WHERE user_id = ?", some_user_id)
It returns the record with other properties loaded properly except the time property which is ret_basket.Created_at as 0001-01-01 00:00:00 +0000 UTC
Any suggestions?

There is no official package for Sqlite so I assume you are using https://github.com/mattn/go-sqlite3 Probably your issue is result of wrong created_at field declaration in database which should be DATETIME because the next code works perfectly on my machine (I've removed all error checks):
package main
import (
"github.com/jmoiron/sqlx"
_ "github.com/mattn/go-sqlite3"
"log"
"time"
)
type Post struct {
Id int64 `db:"post_id"`
Created time.Time `db:"created"`
}
func main() {
db, _ := sqlx.Connect("sqlite3", "post_db.db")
db.MustExec("DROP TABLE IF EXISTS posts; CREATE TABLE posts (post_id INT, created DATETIME);")
p1 := Post{Id: 1, Created: time.Now().UTC()}
p2 := Post{}
tx := db.MustBegin()
tx.NamedExec("INSERT INTO posts (post_id, created) VALUES (:post_id, :created)", &p1)
tx.Commit()
db.Get(&p2, "SELECT post_id, created FROM posts WHERE post_id = $1", p1.Id)
log.Println(p2.Created.Format("2006-01-02"))
}

Related

Go: how to use sqlx to query two columns and map the data to key value pairs

Imagine a query that returns the following:
+------------------------------------+---------+
|id |user_type|
+------------------------------------+---------+
|0435b164-55ef-4b1a-8090-f75c3594332f|4 |
|0e1c17d5-d9df-4b5a-9310-431cfc8a6b36|4 |
|170202b6-7eb4-4242-9a3e-a4fc16daaccc|4 |
|19ff1812-5dd2-4707-a9de-125edf926ce5|4 |
|1a090e81-47ff-4d5c-b625-97c84043ff3a|4 |
+------------------------------------+---------+
What I want from this query is a map[uuid.UUID]int that has all of the id column values as keys, and the user type ints as the values.
Is this at all possible without a bunch of extra code? MapScan does not work and passing in the map value as the dest struct also fails.
Ideally the following or something similar works
var roles map[uuid.UUID]int
sqlx.Select(&roles, query)
package main
import (
"fmt"
"github.com/google/uuid"
_ "github.com/mattn/go-sqlite3"
"github.com/jmoiron/sqlx"
)
func main() {
var db *sqlx.DB
db, _ = sqlx.Open("sqlite3", ":memory:")
schema := `CREATE TABLE tmp (
id integer,
uuid text
);`
// execute a query on the server
db.MustExec(schema)
qry := `INSERT INTO tmp (id, uuid) VALUES (?, ?)`
db.MustExec(qry, 1, uuid.NewString())
db.MustExec(qry, 2, uuid.NewString())
db.MustExec(qry, 3, uuid.NewString())
rows, _ := db.Query("SELECT id, uuid FROM tmp")
var tmp = make(map[string]int)
// iterate over each row
for rows.Next() {
var id int
var uuid string
_ = rows.Scan(&id, &uuid)
tmp[uuid] = id
// fmt.Println(id, uuid)
// rows.MapScan(&tmp)
}
fmt.Println(tmp)
}
Note: There is MapScan in sqlx which works with map but it can process 1 row at a time.

db.First() not using primary key name

I am writing some sample code to understand gorm but appear to be having an issue with setting up the primary_key value. As a result, the resulting SQL query is broken.
Please see the sample code:
package main
import (
"os"
"fmt"
"time"
"gorm.io/driver/postgres"
"gorm.io/gorm"
)
type Post struct {
id int `json:"id" gorm:"primary_key:id"`
url string `gorm:"url"`
content string `gorm:"content"`
created_at time.Time `gorm:"created_at"`
normalized string `gorm:"normalized"`
account_id int `gorm:"account_id"`
posthash []byte `gorm:"posthash"`
received_at time.Time `gorm:"received_at"`
}
func main() {
dsn := "user=postgres dbname=localtest"
db, err := gorm.Open(postgres.Open(dsn), &gorm.Config{})
if err != nil {
fmt.Println("An error occurred", err)
os.Exit(1)
}
db.AutoMigrate(&Post{})
var post Post
db.First(&post, 1)
fmt.Println(post)
}
When I run this code, I receive the following error:
$ go run gorm_test.go
2020/12/01 00:34:06 /home/farhan/gorm_test.go:32 ERROR: syntax error at or near "=" (SQLSTATE 42601)
[0.128ms] [rows:0] SELECT * FROM "posts" WHERE "posts". = 1 ORDER BY "posts". LIMIT 1
{0 {0 0 <nil>} 0 [] {0 0 <nil>}}
The nature of this error suggests to me that the primary_key value is not set. I tried primaryKey and primarykey, but none appeared to work. Any ideas?
#Shubham Srivastava - Your structure declaration is bad; you need to export fields so gorm can use them Declaring Models
❗️Use exported ID field
In Go, when a field name starts with a lowercase letter that means the field is private (called unexported in Go parlance). Gorm, being a third party package, cannot see inside your struct to know there is an id field, or any other unexported field for that matter.
The solution is to make sure all the fields that need to come from the DB are exported:
type Post struct {
ID uint `json:"id" gorm:"primaryKey"`
...
}

What's the format of timestamp to write parquet file in go

I am trying to write a Go struct in a Parquet file and upload to S3. What format and type do I specify for timestamp parameter in the struct so that athena displays correct timestamp when reading from the parquet file.
type example struct {
ID int64 `parquet:"name=id, type=INT64"`
CreatedAt int64 `parquet:"name=created_at,type=TIMESTAMP_MILLIS"`
}
ex := example{}
ex.ID = int64(10)
ex.CreatedAt = time.Now().Unix()
fw, err := ParquetFile.NewLocalFileWriter("new.parquet")
pw, err := ParquetWriter.NewParquetWriter(fw, new(example), 1)
pw.Write(ex)
Upload the file new.parquet to S3
Reference - https://github.com/xitongsys/parquet-go. I created a table in Athena with int and timestamp field for the same and trying querying the table. The date is showing something like - 1970-01-18 21:54:23.751.
which no where matches the current timestamp.
For example,
package main
import (
"fmt"
"time"
)
func main() {
type example struct {
CreatedAt int64 `parquet:"name=created_at,type=TIMESTAMP_MILLIS"`
}
ex := example{}
ex.CreatedAt = time.Now().UnixNano() / int64(time.Millisecond)
fmt.Println(ex.CreatedAt)
}
Playground: https://play.golang.org/p/ePOlUKiT6fD
Output:
1257894000000

Golang's GORM not adding associations to "has many" relationship

I just started using GORM and tried to build a "has many relationship". I'm trying to add an association to Previous.Holdings (I think I followed the docs correctly) but when I try to do a select * from previous I don't see anything showing up in the database. Any idea on what I'm missing.
import (
orm "github.com/jinzhu/gorm"
_ "github.com/jinzhu/gorm/dialects/sqlite"
)
type Previous struct {
orm.Model
Holdings []Holding `gorm:"foreignkey:ID"`
}
type Holding struct {
ID uint `gorm:"primary_key"`
Symbol string
PurchaseDate time.Time
SellDate time.Time
}
func main() {
t1, _ := time.Parse("2006-01-02", "2017-06-16")
h := Holding{
Symbol: "abc",
PurchaseDate: t1,
}
db.Model(&Previous{}).Association("Holdings").Append(h)
}
First of all you should create your previous table. You can make that by making migrations. You probably should make that migrations after db connection initialization. e.g.
db.AutoMigrate(&Previous{})
So when u using db.Model(&Previous{}) you not saving any entity and if u wanna make asscociation with Holdings entity you need as first step to Save or Find existing Previous record by doing e.g.
previous := &Previous{}
db.Save(previous)
After that you can append your holding record to Model like you do in your code but changing referenced Previous. So it will look like this
h := Holding{
Symbol: "abc",
PurchaseDate: t1,
}
db.Model(previous).Association("Holdings").Append(h)
I don't know if its for testing but when you modeling entities you can make referenced id whithout specifing foreign key also you are using your Holding ID as ForeignKey so ID of Previous will be your ID of Holding.
For me your model declaration should look like this (PreviousID will be automaticaly signed as foreign key for Previous)
type Previous struct {
orm.Model
Holdings []Holding
}
type Holding struct {
ID uint `gorm:"primary_key"`
PreviousID uint
Symbol string
PurchaseDate time.Time
SellDate time.Time
}

GET blob from sql database Golang

import (
"database/sql"
"encoding/json"
"fmt"
_ "github.com/go-sql-driver/mysql"
)
type User struct {
Name string `json:name`
Picture []uint8 `json:picture`
}
func main(){
//straight to the query
rows, err := 'SELECT name, picture FROM ms_users' // picture is longblob type in database
checkErr(err)
var usr User
for rows.Next(){
err = rows.Scan(&usr.Name, &usr.Picture)
checkErr(err)
}
jsn, err := json.Marshal(usr)
fmt.Printf("%v, "string(jsn))
}
With above code, I only get name value but the picture is empty.
How do I store blob value from databse to struct ?
Any answer will be appreciated! thank you!
I'm relatively new to GO I encountered this question while searching a solution for a similar problem I was able to find a solution.
When you get BLOB data from the database you get it as type []byte your struct can look like this below
type User struct {
Name string `json:name`
Picture []byte`json:picture`
}
I guess you can process the byte array according to you need later. In my case I needed a JSON object so I unmarshalled it to a type interface{} variable.

Resources