This question already has answers here:
How do I batch sql statements with package database/sql
(12 answers)
Golang Route for insert - PrepareContext error
(1 answer)
Need to insert struct directly in a PostgreSQL DB
(2 answers)
Closed 9 months ago.
The community reviewed whether to reopen this question 9 months ago and left it closed:
Original close reason(s) were not resolved
My question is not related to that one. My question was mostly on preparex and ctx. I have already done with the implementation using the db.NamedExec and my code is working for those. What I am trying to do here is to understand context.context and preparex. Implement using these two.
CODE SNIPPET:
model.go
type User struct {
ID int `db:"id" json:"id"`
CreatedAt time.Time `db:"created_at" json:"created_at"`
UpdatedAt time.Time `db:"updated_at" json:"updated_at"`
DeletedAt time.Time `db:"deleted_at" json:"deleted_at"`
Username string `db:"username" json:"username"`
Password string `db:"password" json:"password"`
FirstName string `db:"first_name" json:"first_name"`
LastName string `db:"last_name" json:"last_name"`
Phone string `db:"phone" json:"phone"`
Status bool `db:"status" json:"status"`
Addrs []UserAddress
}
Query:
queryInsertUserData = `
INSERT INTO users (id, created_at, updated_at, deleted_at, username, password, first_name, last_name, phone, status) VALUES($1, now(), now(), $2, $3, $4, $5, $6, $7, $8)
`
type queries struct {
insertUserData,
insertAddrData *sqlx.Stmt
}
//prepareStatement is a method for preparing sqlx statement
func (queries *queries) prepareStatement(db *sqlx.DB, query string) (*sqlx.Stmt, error) {
stmt, err := db.Preparex(query) //https://go.dev/doc/database/prepared-statements
if err != nil {
return nil, err
}
return stmt, err
}
constructUserData, err := queries.prepareStatement(db, queryInsertUserData)
queries.insertUserData = constructUserData
Implementation:
// Insert User data
func (postgres *Postgres) InsertUserData(ctx context.Context) (*entity.User, error) {
c := entity.User{}
err := postgres.queries.insertUserData.SelectContext(ctx, &c) //<---here
if err != nil {
return nil, err
}
return &c, nil
}
my ctx is :
ctx = context.WithValue(ctx, "ID", 1)
ctx = context.WithValue(ctx, "Username", "John")
ctx = context.WithValue(ctx, "Password", "pass")
when I am passing to postgres.queries.insertUserData.SelectContext(ctx, &c)
I am getting: sql: expected 8 arguments, got 0
why it is saying got 0? Can anyone help me with this? How to pass ctx and provide the insert query values?
I didn't get the structure of your code, but from the main context (inserting query) you should do something like this:
package main
import (
"context"
"database/sql"
"log"
)
var db *sql.DB
type User struct {
ID int `db:"id" json:"id"`
CreatedAt time.Time `db:"created_at" json:"created_at"`
UpdatedAt time.Time `db:"updated_at" json:"updated_at"`
DeletedAt time.Time `db:"deleted_at" json:"deleted_at"`
Username string `db:"username" json:"username"`
Password string `db:"password" json:"password"`
FirstName string `db:"first_name" json:"first_name"`
LastName string `db:"last_name" json:"last_name"`
Phone string `db:"phone" json:"phone"`
Status bool `db:"status" json:"status"`
Addrs []UserAddress
}
func main() {
users := []User {
{...User 1 data...},
{...User 2 data...},
}
stmt, err := db.Prepare("INSERT INTO users (id, created_at, updated_at, deleted_at, username, password, first_name, last_name, phone, status) VALUES($1, now(), now(), $2, $3, $4, $5, $6, $7, $8)")
if err != nil {
log.Fatal(err)
}
defer stmt.Close() // Prepared statements take up server resources and should be closed after use.
for id, user := range users {
if _, err := stmt.Exec(id+1, user.Username, user.Password,... other data); err != nil {
log.Fatal(err)
}
}
}
also, you can use the Gorm powerful package for using all relational databases.
I use this helper for inserts.
Works with query syntax as follows:
INSERT INTO checks (
status) VALUES (:status)
returning id;
Sample struct
type Row struct {
Status string `db:"status"`
}
// Insert inserts row into table using query SQL command
// table used only for loging, actual table name defined in query
// function expects Query with named parameters
func Insert(ctx context.Context, row interface{}, query string, table string, tx *sqlx.Tx) (int64, error) {
// convert named query to native parameters format
query, args, err := tx.BindNamed(query, row)
if err != nil {
return 0, fmt.Errorf("cannot bind parameters for insert into %q: %w", table, err)
}
var id struct {
Val int64 `db:"id"`
}
err = sqlx.GetContext(ctx, tx, &id, query, args...)
if err != nil {
return 0, fmt.Errorf("cannot insert into %q: %w", table, err)
}
return id.Val, nil
}
Related
type Book struct {
tableName struct{} `pg:"book" json:"-"`
Id int `pg:"id,pk" json:"id"`
Author int `pg:"author_id,notnull" json:"-"`
Author *Author `pg:"fk:author_id" json:"author,omitempty"`
}
I want select book and author in one query.
If I try this:
var r []model.Book
_, err := dao.FusedDb.Query(&r, `SELECT * FROM book b INNER JOIN author a on a.id = b.author_id`)
I get an error
pg: can't find column=name in model=Book (try discard_unknown_columns)
I wrote down a piece of code that I always use when I've to deal with this scenario. First, let me show the code and then I'll comment on the relevant parts:
package main
import (
"database/sql"
"fmt"
_ "github.com/lib/pq"
)
type Book struct {
gorm.Model
Title string
Description string
AuthorID uint
Author Author
}
type Author struct {
gorm.Model
FirstName string
LastName string
Books []Book
}
type Result struct {
BookId int
AuthorId int
Title string
FirstName string
LastName string
}
func main() {
conn, err := sql.Open("postgres", "host=localhost user=postgres password=postgres dbname=postgres port=5432 sslmode=disable")
if err != nil {
panic(err)
}
defer conn.Close()
// query
var result []Result
rows, err := conn.Query("select b.id, a.id, b.title, a.first_name, a.last_name from authors a inner join books b on a.id = b.author_id")
if err != nil {
panic(err)
}
defer rows.Close()
for rows.Next() {
var record Result
if err := rows.Scan(&record.BookId, &record.AuthorId, &record.Title, &record.FirstName, &record.LastName); err != nil {
panic(err)
}
result = append(result, record)
}
if err := rows.Err(); err != nil {
panic(err)
}
fmt.Printf("%v", result)
}
Structs definition
The Book and Author structs represent the tables defined in my database. Result is used to hold the fetched records through the query specified below.
The query
The query is pretty straightforward. We only used the method Query on the SQL client opened at the beginning of the main function. Then, we've to defer a call to the method Close on the rows variable to clean up.
Scanning
The for makes sure that we scan all of the rows retrieved with the Query method. To understand if there are other rows to fetch we use the method Next that returns a bool value indicating whether or not there are other rows to scan.
In the body of the for we declare a loop-scoped variable to hold the current record. Thanks to the Scan method we'll be able to assign each column to the relative field of the struct.
Lastly, we've to check for any error by invoking the method Err on the rows variable and handle it.
Let me know if this clarifies your question, thanks!
Have this struct of User and Post and I try to make Name from User to be included within Post Struct when a user create a new post.
type User struct {
ID int
Name string
Created time.Time
}
type Post struct {
ID int
PostTitle string
PostDesc string
Created time.Time
}
How can I create something connected between this two struct such as Author of the Post ?
The goal is try to get the name of the post author which from User struct with the code below:
post, err := app.Models.Posts.GetPost(id)
GetPost() just run SELECT query and scan row
This approach is without any ORM.
It's a simple query that can return multiple rows. You've to scan the whole resultset and map each column on the struct's fields.
Keep in mind to always check for errors.
Below you can find the solution:
package main
import (
"database/sql"
"fmt"
"time"
_ "github.com/lib/pq"
)
type Post struct {
ID int
PostTitle string
PostDesc string
Created time.Time
UserID int
User User
}
type User struct {
ID int
Name string
Created time.Time
}
func main() {
conn, err := sql.Open("postgres", "host=localhost user=postgres password=postgres dbname=postgres port=5432 sslmode=disable")
if err != nil {
panic(err)
}
defer conn.Close()
// get MAX id
var id int
conn.QueryRow(`SELECT MAX(id) FROM posts`).Scan(&id)
// insert
sqlInsertStmt := `INSERT INTO posts (id, post_title, post_desc, created, user_id) VALUES ($1,$2,$3,$4,$5)`
if _, err = conn.Exec(sqlInsertStmt, id+1, "TDD", "Introduction to Test-Driven-Development", time.Now(), 1); err != nil {
panic(err)
}
// read
rows, err := conn.Query(`SELECT posts.id, post_title, post_desc, posts.created, users.id, users.name, users.created FROM posts INNER JOIN users ON posts.user_id=users.id`)
if err != nil {
panic(err)
}
var posts []Post
for rows.Next() {
var post Post
if err = rows.Scan(&post.ID, &post.PostTitle, &post.PostDesc, &post.Created, &post.User.ID, &post.User.Name, &post.User.Created); err != nil {
panic(err)
}
posts = append(posts, post)
}
if err = rows.Err(); err != nil {
panic(err)
}
for _, v := range posts {
fmt.Printf("author name: %q\n", v.User.Name)
}
}
Let me know if this helps.
Edit
I've also included an example of INSERT in Postgres. To achieve it, we've to use the db.Exec() function, and provide the parameters.
Pay attention to how you construct the query as you can get a SQL-Injection vulnerability.
Lastly, in a real-world scenario, you shouldn't lookup for the MAX id in the posts table but should be automatically generated.
Give it a try to this solution, maybe it resolves your issue.
First, I defined the structs in this way:
// "Post" belongs to "User", "UserID" is the foreign key
type Post struct {
gorm.Model
ID int
PostTitle string
PostDesc string
Created time.Time
UserID int
User User
}
type User struct {
ID int
Name string
Created time.Time
}
In this way, you can say that Post belongs to User and access the User's information within the Post struct.
To query the records, you've to use Preload("User") to be sure to eager load the User records from the separate table.
Keep attention to the name you pass in as the argument in Preload, as it can be tricky.
Lastly, you'll be able to access data in the embedded struct (with the dot notation).
Below, you can find a complete working example (implemented with the use of Docker):
package main
import (
"fmt"
"time"
"gorm.io/driver/postgres"
"gorm.io/gorm"
)
// "Post" belongs to "User", "UserID" is the foreign key
type Post struct {
gorm.Model
ID int
PostTitle string
PostDesc string
Created time.Time
UserID int
User User
}
type User struct {
ID int
Name string
Created time.Time
}
func main() {
dsn := "host=localhost user=postgres password=postgres dbname=postgres port=5432 sslmode=disable"
db, err := gorm.Open(postgres.Open(dsn), &gorm.Config{})
if err != nil {
panic(err)
}
db.AutoMigrate(&Post{})
newPost := &Post{ID: 1, PostTitle: "Golang", PostDesc: "Introduction to Golang", Created: time.Now(), UserID: 1, User: User{ID: 1, Name: "John Doe", Created: time.Now()}}
db.Create(newPost)
var post Post
db.Preload("User").Find(&post, 1)
fmt.Printf("author name: %q\n", post.User.Name)
}
Let me know if I answered your question!
I am trying to wrote this test bellow, other tests works fine, however I am having problems with the UPDATE query
func TestDeleteWorkspace(t *testing.T) {
conn, mock, repository, err := setup()
defer conn.Close()
assert.NoError(t, err)
uid := uuid.New()
// mock.ExpectBegin()
mock.ExpectQuery(regexp.QuoteMeta(`UPDATE "workspaces" SET`)).WithArgs(sqlmock.AnyArg(), uid)
// mock.ExpectCommit()
var e bool
e, err = repository.Delete(uid)
assert.NoError(t, err)
assert.True(t, e)
err = mock.ExpectationsWereMet()
assert.NoError(t, err)
}
repository.Delete does this query
func (r *WorkspaceRepository) Delete(id any) (bool, error) {
if err := r.db.Delete(&model.Workspace{}, "id = ?", id).Error; err != nil {
return false, nil
}
return true, nil
}
Which runs this query
UPDATE "workspaces" SET "deleted_at"='2022-07-04 09:09:20.778' WHERE id = 'c4610193-b43a-4ed7-9ed6-9d67b3f97502' AND "workspaces"."deleted_at" IS NULL
I am using Soft-Delete, that is why it is an UPDATE and not a DELETE query
However, I get the following error
workspace_test.go:169:
Error Trace: workspace_test.go:169
Error: Received unexpected error:
there is a remaining expectation which was not matched: ExpectedQuery => expecting Query, QueryContext or QueryRow which:
- matches sql: 'UPDATE "workspaces" SET'
- is with arguments:
0 - 28e7aa46-7a22-4dc7-b3ce-6cf02af525ca
1 - {}
What I am doing wrong?
EDIT: It is a soft-delete operation, that why is a UPDATE and not a DELETE
My model
type Workspace struct {
ID uuid.UUID `gorm:"type:uuid;default:uuid_generate_v4()" json:"id"`
Name string `gorm:"not null,type:text" json:"name"`
CreatedAt time.Time `gorm:"autoCreateTime" json:"create_time"`
UpdatedAt time.Time `gorm:"autoUpdateTime" json:"update_time"`
DeletedAt gorm.DeletedAt `gorm:"index,->" json:"-"`
}
Error message is quite self-explanatory.
This is your query:
'UPDATE "workspaces" SET "deleted_at"=$1 WHERE id = $2 AND "workspaces"."deleted_at" IS NULL'
it includes 2 arguments:
"deleted_at"=$1 WHERE id = $2
You set only 1 in your SQL mock:
.WithArgs(uid)
You need to send both arguments in mock.
It is not reliable to use Time.Now() in test because that value occasionally is going to be a few nanoseconds different from the value you set in code and test will fail.
The quick and dirty fix is to use sqlmock.AnyArg():
.WithArgs(sqlmock.AnyArg(), uid)
A more sophisticated alternative is to write custom Argument that checks type and compares value with time.Now(). Difference should be less than a few seconds.
See an example: https://github.com/DATA-DOG/go-sqlmock#matching-arguments-like-timetime
Hello I am using pgx to use my postgres, and I have doubts as to how I can transform a row in the database into an aggregate
I am using entities and value objects
without value object it seems easy using marshal, but using value object I think it is not a good idea to have fields exported and then my question comes in, how can I convert my line into a struct of my aggregate
my aggregrate :
type Email struct {
address string
}
type Password struct {
value string
}
type Name struct {
firstName string
lastName string
}
type Person struct {
Id string
Name valueObject.Name
Email valueObject.Email
Password valueObject.Password
Created time.Time
Updated time.Time
}
func NewPerson(name valueObject.Name, email valueObject.Email, password valueObject.Password) *Person {
id := uuid.New()
return &Person{
Id: id.String(),
Name: name,
Email: email,
Password: password,
Created: time.Now(),
Updated: time.Now(),
}
}
all my value objects have a method to get the private value through a function simulating a get, I didn’t put the rest of the code of my value objects so it wouldn’t get big
func to get all rows from table:
func (r *personRepository) GetAll() (persons []*entities.Person, err error) {
qry := `select id, first_name, last_name, email, password created_at, updated_at from persons`
rows, err := r.conn.Query(context.Background(), qry)
return nil, fmt.Errorf("err")
}
If someone can give me a glimpse of how I can pass this line from the bank to a struct of my aggregate using this value object
You can use something like this (not yet tested and need optimization):
func (r *personRepository) GetAll() (persons []*entities.Person, err error) {
qry := `select id, first_name, last_name, email, password, created_at, updated_at from persons`
rows, err := r.conn.Query(context.Background(), qry)
var items []*entities.Person
if err != nil {
// No result found with the query.
if err == pgx.ErrNoRows {
return items, nil
}
// Error happened
log.Printf("can't get list person: %v\n", err)
return items, err
}
defer rows.Close()
for rows.Next() {
// Build item Person for earch row.
// must be the same with the query column position.
var id, firstName, lastName, email, password string
var createdAt, updatedAt time.Time
err = rows.Scan(&id, &firstName, &lastName, &email,
&createdAt, updatedAt)
if err != nil {
log.Printf("Failed to build item: %v\n", err)
return items, err
}
item := &entities.Person{
Id: id,
FirstName: firstName,
// fill other value
}
// Add item to the list.
items = append(items, item)
}
return items, nil
}
Don't forget to add the comma after text password in your query.
I am using entities and value objects without value object it seems easy using marshal,
Sorry, I don't know about the value object in your question.
In PostgreSQL, I have table called surveys.
CREATE TABLE SURVEYS(
SURVEY_ID UUID PRIMARY KEY NOT NULL DEFAULT uuid_generate_v4(),
SURVEY_NAME VARCHAR NOT NULL,
SURVEY_DESCRIPTION TEXT,
START_PERIOD TIMESTAMP,
END_PERIOD TIMESTAMP
);
As you can see only SURVEY_ID and SURVEY_NAME columns are NOT NULL.
In Go, I want to create new entry in that table by POST request. I send JSON object like this:
{
"survey_name": "NAME",
"survey_description": "DESCRIPTION",
"start_period": "2019-01-01 00:00:00",
"end_period": "2019-02-28 23:59:59"
}
Unfortunatly it raise strange ERROR:
parsing time ""2019-01-01 00:00:00"" as ""2006-01-02T15:04:05Z07:00"": cannot parse " 00:00:00"" as "T"
Where I make mistake and how to fix my problem?
models/surveys.go:
import (
"database/sql"
"time"
)
type NullTime struct {
time.Time
Valid bool
}
type Survey struct {
ID int `json:"survey_id"`
Name string `json:"survey_name"`
Description sql.NullString `json:"survey_description"`
StartPeriod NullTime `json:"start_period"`
EndPeriod NullTime `json:"end_period"`
}
controllers/surveys.go:
var CreateSurvey = func(responseWriter http.ResponseWriter, request *http.Request) {
// Initialize variables.
survey := models.Survey{}
var err error
// The decoder introduces its own buffering and may read data from argument beyond the JSON values requested.
err = json.NewDecoder(request.Body).Decode(&survey)
if err != nil {
log.Println(err)
utils.ResponseWithError(responseWriter, http.StatusInternalServerError, err.Error())
return
}
defer request.Body.Close()
// Execute INSERT SQL statement.
_, err = database.DB.Exec("INSERT INTO surveys (survey_name, survey_description, start_period, end_period) VALUES ($1, $2, $3, $4);", survey.Name, survey.Description, survey.StartPeriod, survey.EndPeriod)
// Shape the response depending on the result of the previous command.
if err != nil {
log.Println(err)
utils.ResponseWithError(responseWriter, http.StatusInternalServerError, err.Error())
return
}
utils.ResponseWithSuccess(responseWriter, http.StatusCreated, "The new entry successfully created.")
}
The error already says what is wrong:
parsing time ""2019-01-01 00:00:00"" as ""2006-01-02T15:04:05Z07:00"": cannot parse " 00:00:00"" as "T"
You are passing "2019-01-01 00:00:00" while it expects a different time format, namely RFC3339 (UnmarshalJSON's default).
To solve this, you either want to pass the time in the expected format "2019-01-01T00:00:00Z00:00" or define your own type CustomTime like this:
const timeFormat = "2006-01-02 15:04:05"
type CustomTime time.Time
func (ct *CustomTime) UnmarshalJSON(data []byte) error {
newTime, err := time.Parse(timeFormat, strings.Trim(string(data), "\""))
if err != nil {
return err
}
*ct = CustomTime(newTime)
return nil
}
func (ct *CustomTime) MarshalJSON() ([]byte, error) {
return []byte(fmt.Sprintf("%q", time.Time(*ct).Format(timeFormat))), nil
}
Careful, you might also need to implement the Valuer and the Scanner interfaces for the time to be parsed in and out of the database, something like the following:
func (ct CustomTime) Value() (driver.Value, error) {
return time.Time(ct), nil
}
func (ct *CustomTime) Scan(src interface{}) error {
if val, ok := src.(time.Time); ok {
*ct = CustomTime(val)
} else {
return errors.New("time Scanner passed a non-time object")
}
return nil
}
Go Playground example.