How to optimize performance with clear struct value in Go? - performance

My API server accept post request and the request body is JSON,so I create two struct object to accept JSON string and persist it into database.But everytime I accept a request I need create the struct object over and over again,I try to clear the struct instead of recreate it.And the demo code like this following:
//The two struct
type Card struct {
Number string
Type string
}
type Person struct {
Name string
Cards []Card
}
var p Person
//parse JSON to the struct object
func init() {
str := `{"name":"aaa","cards":[{"number":"1","type":"visa"},{"number":"2","type":"mastercard"}]}`
json.Unmarshal([]byte(str), &p)
}
func PersistToDatabase() {
var err error
tx, err := db.Begin()
if err != nil {
return
}
defer func() {
if err != nil && tx != nil {
if err := tx.Rollback(); err != nil {
return
}
}
}
finish := make(chan bool)
stmt1, err := tx.Prepare(`insert into tb1(name) values(?)`)
if err != nil {
panic(err.Error())
}
defer stmt1.Close()
stmt2, err := tx.Prepare(`insert into tb2(name, num, type) values(?, ?, ?)`)
if err != nil {
panic(err.Error())
}
defer stmt2.Close()
go func() {
defer func() { finish <- true }()
if _, err = stmt1.Exec(p.name); err != nil {
log.Println("stmt1.Exec: ", err.Error())
return
}
for _, x := range p.Cards {
if _, err = stmt2.Exec(p.name, x.Number, x.Type); err != nil {
log.Println("stmt2.Exec: ", err.Error())
return
}
}
}
<-finish
//clear struct object
p.Name = ""
p.Cards = nil //have anything do this better?
}
In the code bottom I clear the Name element but let the Cards element to be nil,how can I do better? And do my databases operation have any problem?

A struct is not an object[1]. While fundamentally they're both mishmashes of ordered data in some way or another, a struct is effectively just a list of variables. No constructor is called. No memory beyond that of its member elements is allocated, and it's not allocated in any special place. (Granted this probably isn't too expensive even in most OO languages, optimizers are pretty cool sometimes) If I have
type A struct {
I int
S string
}
var MyA A
It's not really substantially different from
var (
I int
S string
)
Writing the zero struct of a type MyA = A{} is effectively equivalent to doing MyA.I = 0; MyA.S = "" which is, in turn, effectively equivalent to I = 0; S = "". There should be no (significant) performance overhead doing any of the above.
This doesn't mean structs aren't useful. They're exceptionally useful, conceptually, for defining methods on, for filling in data such as in cases like JSON. But what they're not is significantly different. Finally, and this is the most important thing, is there a significant performance bottleneck you have profiled in this block of code? If not (and I suspect not), don't worry about it, it's probably fine. :)
[1] Java or C#-style object, which are theoretically (even if not in practice) more expensive. Not C++ objects which are different (and more like Go structs).

Related

How to create a new instance of a struct that is read from file

I know this may seem like bad design (and I wish I didn't need to do this), but I need to read a struct that is generated automatically at run time and create a new instance of it.
The struct that I create I only need limited metadata of so that it can be passed to the Gorm Gen FieldRelateModel method (from here) where 'relModel' is the new instance I am hoping to make (see below):
FieldRelateModel = func(relationship field.RelationshipType, fieldName string, relModel interface{}, config *field.RelateConfig) model.CreateFieldOpt {
st := reflect.TypeOf(relModel)
if st.Kind() == reflect.Ptr {
st = st.Elem()
}
fieldType := st.String()
if config == nil {
config = &field.RelateConfig{}
}
if config.JSONTag == "" {
config.JSONTag = ns.ColumnName("", fieldName)
}
return func(*model.Field) *model.Field {
return &model.Field{
Name: fieldName,
Type: config.RelateFieldPrefix(relationship) + fieldType,
JSONTag: config.JSONTag,
GORMTag: config.GORMTag,
NewTag: config.NewTag,
OverwriteTag: config.OverwriteTag,
Relation: field.NewRelationWithModel(relationship, fieldName, fieldType, relModel),
}
}
}
I am able to parse the struct and get its metadata using ast and reflect (see below), but I don't know how I can do the last step so that the 'structInstance' I return is valid when passed as the 'relModel' to the 'FieldRelateModel' function/option.
func StructExtract(filePath string) (any, error) {
file, err := os.Open(filePath)
if err != nil {
fmt.Println(err)
return nil, err
}
defer file.Close()
fset := token.NewFileSet()
f, err := parser.ParseFile(fset, filePath, file, parser.AllErrors)
if err != nil {
fmt.Println(err)
return nil, err
}
var structType *ast.StructType
// Iterate over the top-level declarations in the file
for _, decl := range f.Decls {
// Check if the declaration is a GenDecl (which can contain structs)
genDecl, ok := decl.(*ast.GenDecl)
if !ok {
continue
}
// Iterate over the GenDecl's specs
for _, spec := range genDecl.Specs {
// Check if the spec is a TypeSpec (which can contain structs)
typeSpec, ok := spec.(*ast.TypeSpec)
if !ok {
continue
}
// Check if the TypeSpec's type is a struct
structType, ok = typeSpec.Type.(*ast.StructType)
if !ok {
continue
}
break
}
}
if structType == nil {
fmt.Println("No struct found in file.")
return nil, err
}
structInstance := reflect.New(reflect.TypeOf(structType)).Elem().Interface()
return structInstance, nil
}
Here is a go playground that I thought might make it easier if someone were to help, however, due to the nature of the problem it won't be able to run within the playground (for various reasons).

(GoLANG) *sql.DB Scan rows into string array pointer

I'm fairly new to handling sql databases through GO. I have an instance where I am scanning rows from my DB connection, into an slices of a nested struct in an instantiated slice. But can't seem to properly preform it. Is there some looping techniques or references that would be of some use in Golang. I have provided example code and can provide anything else. The connection pool is established and only when I go to scan the rows is where my program craps out. So my question is, if there are multiple rows (4 rows & 2 columns) that I want to insert into the (tiger & lion) objects (columns) how would i loop over and do that with the rows.Scan??
rows, err := db.Query(`Query`)
if err != nil {
//error
return
} else {
// logging
}
}
for rows.Next() {
ref := &structurre{}
err := rows.Scan(&ref.number, &ref.animal[0].tiger, &ref.animal[o].lion)
if err != nil {
logEntry.Info(err)
return details, err
}
details = append(details, ref)
}
type structure struct {
number string
animal []*zoo
}
type zoo struct {
tiger string
lion string
}
Maybe you're looking for something like this:
type zoo struct {
tiger string
lion string
}
type structure struct {
number string
animal []*zoo
}
var ref []structure
rows, err := db.QueryContext(ctx, `query`, `args...`)
if err != nil {
//error
return err
}
// logging
for rows.Next() {
var scans structure
err = rows.Scan(&scans.number, &scans.animal[0].tiger, &scans.animal[0].lion)
if err != nil {
fmt.Println(err)
if err == sql.ErrNoRows {
fmt.Println("No Rows found")
}
return err
}
ref = append(ref, scans)
}

Get value of struct with interfaces

I'm trying to parse this petition (https://www.binance.com/api/v1/depth?symbol=MDABTC&limit=500)
I was having tons of problems to create an struct for it, so I used an automated tool, this is what my struct looks like:
type orderBook struct {
Bids [][]interface{} `json:"Bids"`
Asks [][]interface{} `json:"Asks"`
}
I recover and parse the petition by doing:
url := "https://www.binance.com/api/v1/depth?symbol=MDABTC&limit=500"
resp, err := http.Get(url)
if err != nil {
panic(err)
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
panic(err)
}else{
book := orderBook{}
if err := json.Unmarshal(body, &book); err != nil {
panic(err)
}
But whenever I try to make an operation with the struct, like:
v := book.Asks[i][0] * book.Asks[i][1]
I get an error:
invalid operation: book.Asks[i][0] * book.Asks[i][1] (operator * not
defined on interface)
How do I define it? Do I need to create an struct for bids/asks, if so, how would that look like?
Sorry if this seems basic, I just started learning go.
In Golang Spec
For an expression x of interface type and a type T, the primary
expression
x.(T)
asserts that x is not nil and that the value stored in x is of type T.
The notation x.(T) is called a type assertion.
Fetching an underlying value of string type you need to type assert to string from interface.
books.Asks[0][0].(string)
For performing an arithmetic operation on same you needs to convert string into float64 to take into account decimal values
v := strconv.ParseFloat(books.Asks[0][0].(string), 64) * strconv.ParseFloat(books.Asks[0][1].(string), 64)
Checkout code on Go playground
Note that you could define proper structs that unmarshal from the JSON document by implementing the json.Unmarshaler interface.
For example (on the Go Playground):
type OrderBook struct {
Asks, Bids []Order
LastUpdateId int
}
type Order struct {
Price, Volume float64
}
func (o *Order) UnmarshalJSON(bs []byte) error {
values := make([]interface{}, 0, 3)
err := json.Unmarshal(bs, &values)
if err != nil {
return err
}
// TODO: more error checking re: len(values), and their types.
price, err := strconv.ParseFloat(values[0].(string), 10)
if err != nil {
return err
}
volume, err := strconv.ParseFloat(values[1].(string), 10)
if err != nil {
return err
}
*o = Order{price, volume}
return nil
}
As such, unmarshaling those documents looks idiomatic:
func main() {
book := OrderBook{}
err := json.Unmarshal([]byte(jsonstr), &book)
if err != nil {
panic(err)
}
fmt.Printf("Asks: %#v\n", book.Asks)
fmt.Printf("Bids: %#v\n", book.Bids)
fmt.Printf("Update: %#v\n", book.LastUpdateId)
// Asks: []main.Order{main.Order{Price:0.00013186, Volume:167}, main.Order{Price:0.00013187, Volume:128}, ...
// Bids: []main.Order{main.Order{Price:0.00013181, Volume:110}, main.Order{Price:0.00013127, Volume:502}, ...
// Update: 14069188
}

Map response to a struct using Golang

I am attempting to map a response from an API to a struct using Golang.
The JSON that comes back when I view the link in the browser is below:
{
"GBP": 657.54
}
And I just want to map it to a simple struct like so:
type Price struct {
Name string
Value float64
}
Here is my current code.
func FetchCoinPrice(fsym string, tsyms string) Price {
url := fmt.Sprintf("https://min-api.cryptocompare.com/data/price?fsym=" + fsym + "&tsyms=" + tsyms)
fmt.Println("Requesting data from " + url)
price := Price{}
// getting the data using http
request, err := http.Get(url)
if err != nil {
log.Fatal(err.Error())
}
// Read the response body using ioutil
body, err := ioutil.ReadAll(request.Body)
if err != nil {
log.Fatal(err.Error())
}
defer request.Body.Close()
if request.StatusCode == http.StatusOK {
json.Unmarshal(body, &price)
}
return price
}
At the moment all I receive is an empty struct, I know the link is bringing back the correct data and I've tested it in my browser.
The mapping doesn't work that way. Instead, you should use a map:
data := []byte(`{
"GBP": 657.54
}`)
priceMap := map[string]float64{}
err := json.Unmarshal(data, &priceMap)
// Check your errors!
if err != nil {
log.Fatal(err.Error())
}
fmt.Println(priceMap)
This will print:
map[GBP:657.54]
You can then iterate over the map and build the struct you mentioned above, or just access the entry directly if you know the currency. eg: priceMap["GBP"]
You should really check your errors, especially if you're not getting the output you expect from Unmarshal.
The problem is that the unmarshaler cannot guess that keys in a JSON object should correspond to some value in a struct. Golang JSON mapping simply doesn't work that way.
However, you can make your "Price" type implement json.Unmarshaler to deserialize a message into a map of floats (map[string]float64) then ensure the shape is right and populate the struct accordingly:
func (p *Price) UnmarshalJSON(bs []byte) error {
kvs := map[string]float64{}
err := json.Unmarshal(bs, &kvs)
if err != nil {
return err
}
if len(kvs) != 1 {
return fmt.Errorf("expected 1 key, got %d", len(kvs))
}
for name, value := range kvs {
p.Name, p.Value = name, value
}
return nil
}
func main() {
jsonstr := `[{"GBP":657.54},{"USD":123.45}]`
ps := []Price{}
err := json.Unmarshal([]byte(jsonstr), &ps)
if err != nil {
panic(err)
}
// ps=[]main.Price{
// main.Price{Name:"GBP", Value:657.54},
// main.Price{Name:"USD", Value:123.45}
// }
}

Convert map to struct

Ok, the title is a little bit misleading. What I'm after is as follows:
type MyStruct struct {
id int
name string
age int
}
func CreateFromMap(m map[string]interface{}) (MyStruct, error) {
var (
id int
name string
age int
ok bool
)
err := errors.New("Error!")
id, ok = m["id"].(int)
if !ok {
return nil, err
}
name, ok = m["name"].(string)
if !ok {
return nil, err
}
age, ok = m["age"].(int)
if !ok {
return nil, err
}
return MyStruct{id, name, age}, nil
}
Don't ask: Why I'm not using CreateFromMap(int, string, int). That object comes from somewhere else, out of my control.
It's already boring to map each key, value pair in the map to struct properties. But checking if everything is ok or not after each conversion is chaotic.
Is there an easier way of doing this other than reflection?
Let's say I assume you don't want to use reflection because you don't want to do it yourself. In this case, what about using an external package that does it for you ?
package main
import "fmt"
import "github.com/mitchellh/mapstructure"
type MyStruct struct {
Id int
Name string
Age int
}
func main() {
var m = make(map[string]interface{})
m["Id"] = 17
m["Name"] = "foo"
m["Age"] = 42
fmt.Printf("%+v\n", m)
res, err := CreateFromMap(m)
if err != nil {
fmt.Println(err)
return
}
fmt.Printf("%+v\n", res)
}
func CreateFromMap(m map[string]interface{}) (MyStruct, error) {
var result MyStruct
err := mapstructure.Decode(m, &result)
return result, err
}
Output:
map[Age:42 Name:foo Id:17]
{Id:17 Name:foo Age:42}
It has the advantage to work whatever your structure looks like, but it uses reflection internally. Actually, I don't see any "nice" way to do what you want without using reflection and/or repetitive code for each attribute of your structure. The downside, however, is that you will have to use capitalized attributes so that they would be exported to external packages.
Edit (to answer your question in the comments):
On my opinion, if you want to specify additional rules when "creating" the structure, it should be after the decoding operation.
For instance:
func CreateFromMap(m map[string]interface{}) (MyStruct, error) {
var result MyStruct
err := mapstructure.Decode(m, &result)
if err != nil {
return result, err
}
if result.Age <= 0 {
result.Age = 0
}
if result.Name == "" {
return result, errors.New("empty name is not allowed")
}
return result, err
}
This way, you will clearly separate the "conversion" part from your structure's specific rules processing.
You can just Marshal/Unmarshal, but property names should match
func CreateFromMap(m map[string]interface{}) (MyStruct, error) {
data, _ := json.Marshal(m)
var result MyStruct
err := json.Unmarshal(data, &result)
return result, err
}

Resources