I have been trying to reuse a the prepared query in concurrent calls.
type product struct {
query *gocql.Query
}
type resp struct {
Pk string
Product Product
Err error
}
func (p *product) A() {
...
respFromDB := make(chan resp, len(pks))
for _, pk := range pks {
go p.getAll(pk, resp)
}
}
func (p *product) getAll(pk string, product chan resp) {
var (
err error
prodResp resp
)
prod := Product{}
prodResp = resp{
Pk: pk,
}
err = p.preparedStatement.Bind(gtin13).Scan(&prod.Pk,...)
if err != nil {
prodResp.Err = err
product <- prodResp
return
}
prodResp.Product = prod
product <- prodResp
}
The pks are being passed to getAll function but the Bind function seems to not be bind every pk to the corresponding call.
Instead the some pk is being repeatedly binded.
Is there something wrong in this approach? Not sure why the pks are being binded to the corresponding calls.
The Query is created once in the main and injected for all further calls.
Related
I think if i keep using the method below, i'll have to write too much code.
I declared structures for all the tables. and i used the go validate package for validation.
[types.go]
type TableA struct {
Field1 string `json:"field1" validate:"required, max=10"`
Field2 int `json:"field2" validate:"number"`
}
type TableB struct {
...
}
And i initialized the router for each method and connected the handlers.
[tableA.go]
router.Get("/table-a", r.Get_tableA_Handler),
router.Post("/table-a", r.Post_tableA_Handler),
router.Patch("/table-a", r.Patch_tableA_Handler),
router.Delete("/table-a", r.Delete_tableA_Handler)
...
Each handler parses the json in the request body, validates the data and call the db function.
[tableA_router.go]
func (rt *tableARouter) Post_tableA_Handler(w http.ResponseWriter, r *http.Request) error {
//Json to Struct
req := new(types.tableA)
if err := httputils.DecodeJsonBody(r, req); err != nil {
return err
}
// Validation
if err := validCheck(req); err != nil {
return err
}
// DB function
err := rt.insert_tableA_DB(r.Context(), req)
if err != nil {
return err
}
return rt.rd.JSON(w, http.StatusCreated, "Create Success")
}
...
func validCheck(data interface{}) error {
validate := validator.New()
err := validate.Struct(data)
return err
}
This is a DB function called from the handler function above (using Gorm)
[tableA_db.go]
func (rt *tableARouter) insert_tableA_DB(ctx context.Context, data *types.TableA) error {
// DB Connect
db, err := db.Open(rt.dbcfg)
if err != nil {
return err
}
defer db.Close()
tx := db.Begin()
defer tx.Rollback()
// == INSERT ==
query := `INSERT INTO table_a
(field1, field2, ...)
VALUES (?, ?, ...)`
result := tx.WithContext(ctx).Exec(query,
data.Field1, data.Field2, ...)
//Result
if result.Error != nil {
...
}
There are too many tables now... If there are 100 tables i have to write 100 handlers and 100 DB functions.
Is there any way to use something like /tables/{tableName}?
Please give me any advice.... Thank you.
You can use an ORM package, like GORM to make easier your work.
Or you can make an universal handler and with the reflect package, analyze your defined structs and make every SQL query dinamically. But it's not the best solution if any of your struct has inner slices, other embedded structs, or if you need to use joined tables you also have to deal with it manually. I have servers where we have more than 200 endpoints with more than 3-400 methods with 200+ SQL tables and the whole server was written by hand. But I can say, it's very rare when a handler and the DB func can be reused without modifying.
Maybe you can wrap the error handling, rollback/commit, json parse and response parts in a func then use it to call the DB methods.
I have database store function:
func (p *ProductsRep) FindAll(PageNumber int, PaginationSize int, Query string) []*postgresmodels.Product {
Also I have SQL query look like this:
SELECT * FROM table_name.
Then I want to concat conditional action like WHERE some_value=3 if some value (in this case Query) exists then I want to get SELECT * FROM table_name WHERE some_value=3.
I tried to use fmt.Sprintf to concat, or strings.Join, or bytes.Buffer.WriteString. But everytime I getting this error:
I replace real value for understanding:
pq: column "Some value" does not exist.
How can I do "adaptive" queries, which depends on inputed function values.
I believe you are trying to query rows in the database by using parameters.
You need to make sure you don't pass this data in as RAW values, due to the potential risk of SQL injection. You can make queries by using store procedures
You can use the function Query to pass in your query with your parameters. In the example case this is $1. If you wanted to you could add $2, $3... etc depending on how many parameters you wanted to query
Here is two examples
Postgres
using "github.com/jackc/pgx/v4" driver
ctx := context.Background()
type Bar struct {
ID int64
SomeValue string
}
rows, err := conn.Query(ctx, `SELECT * FROM main WHERE some_value=$1`, "foo")
if err != nil {
fmt.Println("ERRO")
panic(err) // handle error
}
defer rows.Close()
var items []Bar
for rows.Next() {
var someValue string
var id int64
if err := rows.Scan(&id, &someValue); err != nil {
log.Fatal(err) // handle error
}
item := Bar{
ID: id,
SomeValue: someValue,
}
items = append(items, item)
}
fmt.Println(items)
MySQL Driver
https://golang.org/pkg/database/sql/#DB.QueryRow
type Bar struct {
ID int64
SomeValue string
}
rows, err := conn.Query(`SELECT * FROM main WHERE some_value=$1`, "foo")
if err != nil {
fmt.Println("ERRO")
panic(err) // handle error
}
defer rows.Close()
var items []Bar
for rows.Next() {
var someValue string
var id int64
if err := rows.Scan(&id, &someValue); err != nil {
log.Fatal(err) // handle error
}
item := Bar{
ID: id,
SomeValue: someValue,
}
items = append(items, item)
}
fmt.Println(items)
I have the following function in the BaseModel that I can use anywhere.
func (d *Dummy) Save() (int64, error) {
o := orm.NewOrm()
var err error
var count int64
if d.Id == 0 {
count, err = o.Insert(d)
} else {
count, err = o.Update(d)
}
return count, err
}
I am using like this
d := models.Dummy{Id: 10}
d.SomeValue = "x"
d.Save()
The problem is I have "d.OtherValue" is already in DB with value.
After executing this function it's getting updated to 0.
As it is a common model function effective for all models, How can I solve this? Basically, I wanted to do this in a single query just like update/Save Django ORM
You need to load the record first. You are missing the Read(&struct) ORM method:
o := orm.NewOrm()
d := models.Dummy{Id: 10}
readErr:= o.Read(&d)
// check if the record with Id of 10 exists and update...
if readErr!= o.ErrNoRows {
if rowsAffected, err := o.Update(&d); err == nil {
// record updated (rowsAffected indicates the number of affected rows)
}
} else {
// record does not exist, create a new one
id, insertErr:= o.Insert(&d)
if insertErr== nil {
// success
}
}
Then you should check if a record is found by the ORM
For more details you can refer to the Read and Update methods.
Go here. Trying to get the chi renderer to return a list of Order struct instances and getting a compiler error that I don't understand:
package myapp
import (
"net/http"
"github.com/go-chi/render"
)
type Order struct {
OrderId string
Status string
}
func (*Order) Bind(r *http.Request) error {
return nil
}
func GetAllOrderByCustomerId(dbClient DbClient, customerId string) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
// fetch all customer orders from the DB
orders,err := dbClient.FetchAllOrdersByCustomerId(customerId)
if err != nil {
log.Error("unable to fetch orders for customer", err)
render.Render(w, r, NewInternalServerError(err))
return
}
render.Bind(r, &orders)
return
}
}
When I go to compile this code I get:
fizz/buzz/myapp/order_fetcher.go:136:20: cannot use &orders (type *[]Order) as type render.Binder in argument to render.Bind:
*[]Order does not implement render.Binder (missing Bind method)
So even though I've defined a Bind for Order, it doesn't seem to automatically apply that Bind to a collection/list of Orders.
Can anyone see what I'm missing? Some endpoints will only be returning a single Order, whereas others (like this one) need to be able to return a collection/list of Orders.
As in the example in chi repository, you have to create a helper method to render the list of something, in your case, list of orders.
First, you have to implement the render.Renderer method then create a helper method to build a list of render.Renderer.
I've adapted your code from the example here:
type Order struct {
OrderId string
Status string
}
// Render implement render.Renderer
func (*Order) Render(w http.ResponseWriter, r *http.Request) error {
// do something
return nil
}
// newOrderList is a helper method to make list of render.Renderer
func newOrderList(orders []*Order) []render.Renderer {
list := []render.Renderer{}
for _, order := range orders {
list = append(list, order)
}
return list
}
func GetAllOrderByCustomerId(dbClient DbClient, customerId string) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
// fetch all customer orders from the DB
orders, err := dbClient.FetchAllOrdersByCustomerId(customerId)
if err != nil {
log.Error("unable to fetch orders for customer", err)
render.Render(w, r, NewInternalServerError(err))
return
}
// render list of orders
render.RenderList(w, r, newOrderList(orders))
return
}
}
I have created an object mapping in Go that is not relational, it is very simple.
I have several structs that looks like this:
type Message struct {
Id int64
Message string
ReplyTo sql.NullInt64 `db:"reply_to"`
FromId int64 `db:"from_id"`
ToId int64 `db:"to_id"`
IsActive bool `db:"is_active"`
SentTime int64 `db:"sent_time"`
IsViewed bool `db:"is_viewed"`
Method string `db:"-"`
AppendTo int64 `db:"-"`
}
To create a new message I just run this function:
func New() *Message {
return &Message{
IsActive: true,
SentTime: time.Now().Unix(),
Method: "new",
}
}
And then I have a message_crud.go file for this struct that looks like this:
To find a message by a unique column (for example by id) I run this function:
func ByUnique(column string, value interface{}) (*Message, error) {
query := fmt.Sprintf(`
SELECT *
FROM message
WHERE %s = ?
LIMIT 1;
`, column)
message := &Message{}
err := sql.DB.QueryRowx(query, value).StructScan(message)
if err != nil {
return nil, err
}
return message, nil
}
And to save a message (insert or update in the database) I run this method:
func (this *Message) save() error {
s := ""
if this.Id == 0 {
s = "INSERT INTO message SET %s;"
} else {
s = "UPDATE message SET %s WHERE id=:id;"
}
query := fmt.Sprintf(s, sql.PlaceholderPairs(this))
nstmt, err := sql.DB.PrepareNamed(query)
if err != nil {
return err
}
res, err := nstmt.Exec(*this)
if err != nil {
return err
}
if this.Id == 0 {
lastId, err := res.LastInsertId()
if err != nil {
return err
}
this.Id = lastId
}
return nil
}
The sql.PlaceholderPairs() function looks like this:
func PlaceholderPairs(i interface{}) string {
s := ""
val := reflect.ValueOf(i).Elem()
count := val.NumField()
for i := 0; i < count; i++ {
typeField := val.Type().Field(i)
tag := typeField.Tag
fname := strings.ToLower(typeField.Name)
if fname == "id" {
continue
}
if t := tag.Get("db"); t == "-" {
continue
} else if t != "" {
s += t + "=:" + t
} else {
s += fname + "=:" + fname
}
s += ", "
}
s = s[:len(s)-2]
return s
}
But every time I create a new struct, for example a User struct I have to copy paste the "crud section" above and create a user_crud.go file and replace the words "Message" with "User", and the words "message" with "user". I repeat alot of code and it is not very dry. Is there something I could do to not repeat this code for things I would reuse? I always have a save() method, and always have a function ByUnique() where I can return a struct and search by a unique column.
In PHP this was easy because PHP is not statically typed.
Is this possible to do in Go?
Your ByUnique is almost generic already. Just pull out the piece that varies (the table and destination):
func ByUnique(table string, column string, value interface{}, dest interface{}) error {
query := fmt.Sprintf(`
SELECT *
FROM %s
WHERE %s = ?
LIMIT 1;
`, table, column)
return sql.DB.QueryRowx(query, value).StructScan(dest)
}
func ByUniqueMessage(column string, value interface{}) (*Message, error) {
message := &Message{}
if err := ByUnique("message", column, value, &message); err != nil {
return nil, err
}
return message, error
}
Your save is very similar. You just need to make a generic save function along the lines of:
func Save(table string, identifier int64, source interface{}) { ... }
Then inside of (*Message)save, you'd just call the general Save() function. Looks pretty straightforward.
Side notes: do not use this as the name of the object inside a method. See the link from #OneOfOne for more on that. And do not get obsessed about DRY. It is not a goal in itself. Go focuses on code being simple, clear, and reliable. Do not create something complicated and fragile just to avoid typing a simple line of error handling. This doesn't mean that you shouldn't extract duplicated code. It just means that in Go it is usually better to repeat simple code a little bit rather than create complicated code to avoid it.
EDIT: If you want to implement Save using an interface, that's no problem. Just create an Identifier interface.
type Ider interface {
Id() int64
SetId(newId int64)
}
func (msg *Message) Id() int64 {
return msg.Id
}
func (msg *Message) SetId(newId int64) {
msg.Id = newId
}
func Save(table string, source Ider) error {
s := ""
if source.Id() == 0 {
s = fmt.Sprintf("INSERT INTO %s SET %%s;", table)
} else {
s = fmt.Sprintf("UPDATE %s SET %%s WHERE id=:id;", table)
}
query := fmt.Sprintf(s, sql.PlaceholderPairs(source))
nstmt, err := sql.DB.PrepareNamed(query)
if err != nil {
return err
}
res, err := nstmt.Exec(source)
if err != nil {
return err
}
if source.Id() == 0 {
lastId, err := res.LastInsertId()
if err != nil {
return err
}
source.SetId(lastId)
}
return nil
}
func (msg *Message) save() error {
return Save("message", msg)
}
The one piece that might blow up with this is the call to Exec. I don't know what package you're using, and it's possible that Exec won't work correctly if you pass it an interface rather than the actual struct, but it probably will work. That said, I'd probably just pass the identifier rather than adding this overhead.
You probably want to use an ORM.
They eliminate a lot of the boilerplate code you're describing.
See this question for "What is an ORM?"
Here is a list of ORMs for go: https://github.com/avelino/awesome-go#orm
I have never used one myself, so I can't recommend any. The main reason is that an ORM takes a lot of control from the developer and introduces a non-negligible performance overhead. You need to see for yourself if they fit your use-case and/or if you are comfortable with the "magic" that's going on in those libraries.
I don't recommend doing this, i personally would prefer being explicit about scanning into structs and creating queries.
But if you really want to stick to reflection you could do:
func ByUnique(obj interface{}, column string, value interface{}) error {
// ...
return sql.DB.QueryRowx(query, value).StructScan(obj)
}
// Call with
message := &Message{}
ByUnique(message, ...)
And for your save:
type Identifiable interface {
Id() int64
}
// Implement Identifiable for message, etc.
func Save(obj Identifiable) error {
// ...
}
// Call with
Save(message)
The approach i use and would recommend to you:
type Redirect struct {
ID string
URL string
CreatedAt time.Time
}
func FindByID(db *sql.DB, id string) (*Redirect, error) {
var redirect Redirect
err := db.QueryRow(
`SELECT "id", "url", "created_at" FROM "redirect" WHERE "id" = $1`, id).
Scan(&redirect.ID, &redirect.URL, &redirect.CreatedAt)
switch {
case err == sql.ErrNoRows:
return nil, nil
case err != nil:
return nil, err
}
return &redirect, nil
}
func Save(db *sql.DB, redirect *Redirect) error {
redirect.CreatedAt = time.Now()
_, err := db.Exec(
`INSERT INTO "redirect" ("id", "url", "created_at") VALUES ($1, $2, $3)`,
redirect.ID, redirect.URL, redirect.CreatedAt)
return err
}
This has the advantage of using the type system and mapping only things it should actually map.