(GoLANG) *sql.DB Scan rows into string array pointer - go

I'm fairly new to handling sql databases through GO. I have an instance where I am scanning rows from my DB connection, into an slices of a nested struct in an instantiated slice. But can't seem to properly preform it. Is there some looping techniques or references that would be of some use in Golang. I have provided example code and can provide anything else. The connection pool is established and only when I go to scan the rows is where my program craps out. So my question is, if there are multiple rows (4 rows & 2 columns) that I want to insert into the (tiger & lion) objects (columns) how would i loop over and do that with the rows.Scan??
rows, err := db.Query(`Query`)
if err != nil {
//error
return
} else {
// logging
}
}
for rows.Next() {
ref := &structurre{}
err := rows.Scan(&ref.number, &ref.animal[0].tiger, &ref.animal[o].lion)
if err != nil {
logEntry.Info(err)
return details, err
}
details = append(details, ref)
}
type structure struct {
number string
animal []*zoo
}
type zoo struct {
tiger string
lion string
}

Maybe you're looking for something like this:
type zoo struct {
tiger string
lion string
}
type structure struct {
number string
animal []*zoo
}
var ref []structure
rows, err := db.QueryContext(ctx, `query`, `args...`)
if err != nil {
//error
return err
}
// logging
for rows.Next() {
var scans structure
err = rows.Scan(&scans.number, &scans.animal[0].tiger, &scans.animal[0].lion)
if err != nil {
fmt.Println(err)
if err == sql.ErrNoRows {
fmt.Println("No Rows found")
}
return err
}
ref = append(ref, scans)
}

Related

How to create a new instance of a struct that is read from file

I know this may seem like bad design (and I wish I didn't need to do this), but I need to read a struct that is generated automatically at run time and create a new instance of it.
The struct that I create I only need limited metadata of so that it can be passed to the Gorm Gen FieldRelateModel method (from here) where 'relModel' is the new instance I am hoping to make (see below):
FieldRelateModel = func(relationship field.RelationshipType, fieldName string, relModel interface{}, config *field.RelateConfig) model.CreateFieldOpt {
st := reflect.TypeOf(relModel)
if st.Kind() == reflect.Ptr {
st = st.Elem()
}
fieldType := st.String()
if config == nil {
config = &field.RelateConfig{}
}
if config.JSONTag == "" {
config.JSONTag = ns.ColumnName("", fieldName)
}
return func(*model.Field) *model.Field {
return &model.Field{
Name: fieldName,
Type: config.RelateFieldPrefix(relationship) + fieldType,
JSONTag: config.JSONTag,
GORMTag: config.GORMTag,
NewTag: config.NewTag,
OverwriteTag: config.OverwriteTag,
Relation: field.NewRelationWithModel(relationship, fieldName, fieldType, relModel),
}
}
}
I am able to parse the struct and get its metadata using ast and reflect (see below), but I don't know how I can do the last step so that the 'structInstance' I return is valid when passed as the 'relModel' to the 'FieldRelateModel' function/option.
func StructExtract(filePath string) (any, error) {
file, err := os.Open(filePath)
if err != nil {
fmt.Println(err)
return nil, err
}
defer file.Close()
fset := token.NewFileSet()
f, err := parser.ParseFile(fset, filePath, file, parser.AllErrors)
if err != nil {
fmt.Println(err)
return nil, err
}
var structType *ast.StructType
// Iterate over the top-level declarations in the file
for _, decl := range f.Decls {
// Check if the declaration is a GenDecl (which can contain structs)
genDecl, ok := decl.(*ast.GenDecl)
if !ok {
continue
}
// Iterate over the GenDecl's specs
for _, spec := range genDecl.Specs {
// Check if the spec is a TypeSpec (which can contain structs)
typeSpec, ok := spec.(*ast.TypeSpec)
if !ok {
continue
}
// Check if the TypeSpec's type is a struct
structType, ok = typeSpec.Type.(*ast.StructType)
if !ok {
continue
}
break
}
}
if structType == nil {
fmt.Println("No struct found in file.")
return nil, err
}
structInstance := reflect.New(reflect.TypeOf(structType)).Elem().Interface()
return structInstance, nil
}
Here is a go playground that I thought might make it easier if someone were to help, however, due to the nature of the problem it won't be able to run within the playground (for various reasons).

Map response to a struct using Golang

I am attempting to map a response from an API to a struct using Golang.
The JSON that comes back when I view the link in the browser is below:
{
"GBP": 657.54
}
And I just want to map it to a simple struct like so:
type Price struct {
Name string
Value float64
}
Here is my current code.
func FetchCoinPrice(fsym string, tsyms string) Price {
url := fmt.Sprintf("https://min-api.cryptocompare.com/data/price?fsym=" + fsym + "&tsyms=" + tsyms)
fmt.Println("Requesting data from " + url)
price := Price{}
// getting the data using http
request, err := http.Get(url)
if err != nil {
log.Fatal(err.Error())
}
// Read the response body using ioutil
body, err := ioutil.ReadAll(request.Body)
if err != nil {
log.Fatal(err.Error())
}
defer request.Body.Close()
if request.StatusCode == http.StatusOK {
json.Unmarshal(body, &price)
}
return price
}
At the moment all I receive is an empty struct, I know the link is bringing back the correct data and I've tested it in my browser.
The mapping doesn't work that way. Instead, you should use a map:
data := []byte(`{
"GBP": 657.54
}`)
priceMap := map[string]float64{}
err := json.Unmarshal(data, &priceMap)
// Check your errors!
if err != nil {
log.Fatal(err.Error())
}
fmt.Println(priceMap)
This will print:
map[GBP:657.54]
You can then iterate over the map and build the struct you mentioned above, or just access the entry directly if you know the currency. eg: priceMap["GBP"]
You should really check your errors, especially if you're not getting the output you expect from Unmarshal.
The problem is that the unmarshaler cannot guess that keys in a JSON object should correspond to some value in a struct. Golang JSON mapping simply doesn't work that way.
However, you can make your "Price" type implement json.Unmarshaler to deserialize a message into a map of floats (map[string]float64) then ensure the shape is right and populate the struct accordingly:
func (p *Price) UnmarshalJSON(bs []byte) error {
kvs := map[string]float64{}
err := json.Unmarshal(bs, &kvs)
if err != nil {
return err
}
if len(kvs) != 1 {
return fmt.Errorf("expected 1 key, got %d", len(kvs))
}
for name, value := range kvs {
p.Name, p.Value = name, value
}
return nil
}
func main() {
jsonstr := `[{"GBP":657.54},{"USD":123.45}]`
ps := []Price{}
err := json.Unmarshal([]byte(jsonstr), &ps)
if err != nil {
panic(err)
}
// ps=[]main.Price{
// main.Price{Name:"GBP", Value:657.54},
// main.Price{Name:"USD", Value:123.45}
// }
}

Golang gRPC server-stream

I have some issue with a gRPC server-stream on golang.
I had no problem with one row, I just used the simple gRPC response for it.
But now I need to send a number of rows from my database and I can't finish my server-stream application.
I just learn Golang and gRPC and this task a little bit difficult to me to solve this task now. And I will be very grateful if someone could help with this because not too many examples of this material on the web.
Or maybe you now where I can find an example, how to stream data from database using gRPC + golang. Thank you
I have this code:
....
type record struct {
id int
lastname string
}
type server struct {
}
func (s *server) QueryData(city *pb.City, stream pb.GetDataStream_QueryDataServer) error {
db, err := sql.Open("postgres", "postgres://adresspass")
if err != nil {
log.Fatal(err)
}
defer db.Close()
rows, err := db.Query(`SELECT id, last_name FROM person WHERE city=$1`, city.City)
if err != nil {
log.Fatal(err)
}
var rs = make([]record, 0)
var rec record
for rows.Next() {
err = rows.Scan(&rec.id, &rec.lastname)
if err != nil {
return err
}
rs = append(rs, rec)
}
for _, onedata := range rs {
if err := stream.Send(onedata); err != nil {
return err
}
}
return nil
}
...
You should scan your values from database as protocol buffer types and not native types.
You should have defined a message format in proto file
message DbVal{
uint32 id = 1;
message string = 2;
}
And pass that struct to the stream.

GO: Return map from SQL query

I am querying a mysql database in a GO function and want to return key value pairs in a map but can't quite figure out how to accomplish this. So far I have this function:
func GetData(callIds []string) map[string]Records {
//db insert
db, err := sql.Open("mysql", mySql)
if err != nil {
fmt.Printf(err.Error())
}
defer db.Close()
//db query
var foo string
err = db.QueryRow("select foo from bardata where callId = %v", 1).Scan(&foo)
if err != nil {
fmt.Printf(err.Error())
}
fmt.Println(foo)
return nil
I want to return a map with the key being callId and value being foo for each row returned from the query.
First, you need to build up your query. As it is, you're not even using your function input. Since we have a variable number of arguments, we need to do a little work to construct the right number of placeholders:
query := `select callid, foo from bardata where callid in (` +
strings.Repeat(`?,`, len(callIds) - 1) + `?)`
then, execute with the values passed in:
rows, err := db.Query(query, callIds...)
if err != nil {
// handle it
}
defer rows.Close()
then collect the results:
ret := map[string]string{}
for rows.Next() {
var callid, foo string
err = rows.Scan(&callid, &foo)
if err != nil {
// handle it
}
ret[callid] = foo
}
return ret
Caveats:
This will cause a placeholder mismatch error if callIds is an empty slice. If that's possible, then you need to detect it and handle it separately (maybe by returning an error or an empty map — querying the DB shouldn't be necessary).
This returns a map[string]string where the values are whatever "foo" is. In your question you have the function returning a map[string]Records but there's no information about what a Records might be or how to fetch one.
You might want to handle sql.ErrNoRows differently from other errors.
package main
import (
"fmt"
"github.com/bobby96333/goSqlHelper"
)
func main(){
fmt.Println("hello")
conn,err :=goSqlHelper.MysqlOpen("user:password#tcp(127.0.0.1:3306)/dbname")
checkErr(err)
row,err := conn.QueryRow("select * from table where col1 = ? and col2 = ?","123","abc")
checkErr(err)
if *row==nil {
fmt.Println("no found row")
}else{
fmt.Printf("%+v",row)
}
}
func checkErr(err error){
if err!=nil {
panic(err)
}
}
output:
&map[col1:abc col2:123]

How to optimize performance with clear struct value in Go?

My API server accept post request and the request body is JSON,so I create two struct object to accept JSON string and persist it into database.But everytime I accept a request I need create the struct object over and over again,I try to clear the struct instead of recreate it.And the demo code like this following:
//The two struct
type Card struct {
Number string
Type string
}
type Person struct {
Name string
Cards []Card
}
var p Person
//parse JSON to the struct object
func init() {
str := `{"name":"aaa","cards":[{"number":"1","type":"visa"},{"number":"2","type":"mastercard"}]}`
json.Unmarshal([]byte(str), &p)
}
func PersistToDatabase() {
var err error
tx, err := db.Begin()
if err != nil {
return
}
defer func() {
if err != nil && tx != nil {
if err := tx.Rollback(); err != nil {
return
}
}
}
finish := make(chan bool)
stmt1, err := tx.Prepare(`insert into tb1(name) values(?)`)
if err != nil {
panic(err.Error())
}
defer stmt1.Close()
stmt2, err := tx.Prepare(`insert into tb2(name, num, type) values(?, ?, ?)`)
if err != nil {
panic(err.Error())
}
defer stmt2.Close()
go func() {
defer func() { finish <- true }()
if _, err = stmt1.Exec(p.name); err != nil {
log.Println("stmt1.Exec: ", err.Error())
return
}
for _, x := range p.Cards {
if _, err = stmt2.Exec(p.name, x.Number, x.Type); err != nil {
log.Println("stmt2.Exec: ", err.Error())
return
}
}
}
<-finish
//clear struct object
p.Name = ""
p.Cards = nil //have anything do this better?
}
In the code bottom I clear the Name element but let the Cards element to be nil,how can I do better? And do my databases operation have any problem?
A struct is not an object[1]. While fundamentally they're both mishmashes of ordered data in some way or another, a struct is effectively just a list of variables. No constructor is called. No memory beyond that of its member elements is allocated, and it's not allocated in any special place. (Granted this probably isn't too expensive even in most OO languages, optimizers are pretty cool sometimes) If I have
type A struct {
I int
S string
}
var MyA A
It's not really substantially different from
var (
I int
S string
)
Writing the zero struct of a type MyA = A{} is effectively equivalent to doing MyA.I = 0; MyA.S = "" which is, in turn, effectively equivalent to I = 0; S = "". There should be no (significant) performance overhead doing any of the above.
This doesn't mean structs aren't useful. They're exceptionally useful, conceptually, for defining methods on, for filling in data such as in cases like JSON. But what they're not is significantly different. Finally, and this is the most important thing, is there a significant performance bottleneck you have profiled in this block of code? If not (and I suspect not), don't worry about it, it's probably fine. :)
[1] Java or C#-style object, which are theoretically (even if not in practice) more expensive. Not C++ objects which are different (and more like Go structs).

Resources