How to convert sqlx query results to an array of structs? - go

I am trying to query all the results from a postgres table without where condition and map it with array of structs with the help of sqlx db Query by passing the args ...interface {}.
But the code pasted below never works, Instead of iterating and scanning the result one by one , is it possible to get the following code work ??
Inputs are much appreciated . Thank you
type CustomData struct {
ID string `db:"id" json:",omitempty"`
Name string `db:"name" json:",omitempty"`
Description string `db:"description" json:",omitempty"`
SourceID string `db:"sourceid" json:",omitempty"`
StatusID string `db:"statusid" json:",omitempty"`
StatusReason string `db:"statusreason" json:",omitempty"`
CreateTime string `db:"createtime" json:",omitempty"`
UpdateTime string `db:"updatetime" json:",omitempty"`
}
var myData []CustomData
*sqlx.DB.Query("SELECT id as ID, name as Name, description as Description, sourceid as SourceID, statusid as StatusID, statusreason as StatusReason, createtime as CreateTime, updatetime as UpdateTime FROM myschema.my_table", &myData)
// tried with following statement but din't work either
// *sqlx.DB.Query("SELECT * FROM myschema.my_table", &myData)
for _, data := range myData {
fmt.Println("--", data)
}
Expected results:
--- CustomData{1,x,x,x,x}
--- CustomData{2,x,x,x,x}
Actual:
Nothing..

You don't need to rename the fields in the query, since you're defining the actual DB fields in the struct tags.
If you want to scan directly to the slice of CustomData and if you are using SQLX, you should use the SQLX specific Select method, rather than the generic SQL Query. Slightly modified relevant example from the illustrated guide to SQLX (https://jmoiron.github.io/sqlx/#getAndSelect):
pp := []Place{}
err = db.Select(&pp, "SELECT * FROM place")
So in your case:
myData := []CustomData
err = db.Select(&myData, "SELECT * FROM myschema.my_table")

you can use the following:
for rows.Next() {
s := CustomData{}
if err := rows.Scan(&s); err != nil {
return err
}
fmt.Println(s)
}
and you can always use ORM library as gorm if you like code first approach or sqlboiler if you like DB first approach

Related

Bleve Search: How do I setup the index on for my struct?

I have this struct in order to index an rdf graph.
type Statement struct {
StatementHash string `json:"statementHash"`
SubjectType string `json:"subjType"`
Subject string `json:"subject"`
Predicate string `json:"predicate"`
Object string `json:"object"`
ObjectType string `json:"objectType"`
ShortType string `json:"shortType"`
ShortPredicate string `json:"shortPredicate"`
DocType string `json:"docType"`
}
DocType is initialised to "statement".
My objective is to be able to form a query string like
"shortType:Tweet covid"
in order to find Tweets in the index that have covid anywhere in the index.
I have set up the index as follows:
func buildIndexMapping() (mapping.IndexMapping, error) {
// a generic reusable mapping for english text
englishTextFieldMapping := bleve.NewTextFieldMapping()
englishTextFieldMapping.Analyzer = en.AnalyzerName
// a generic reusable mapping for keyword text
keywordFieldMapping := bleve.NewTextFieldMapping()
keywordFieldMapping.Analyzer = keyword.Name
statementMapping := bleve.NewDocumentMapping()
// mapping is based on short names only and object literal
// shortType
statementMapping.AddFieldMappingsAt("shortType", keywordFieldMapping)
// others here
indexMapping := bleve.NewIndexMapping()
indexMapping.AddDocumentMapping("statement", statementMapping)
indexMapping.TypeField = "docType"
indexMapping.DefaultAnalyzer = "en"
return indexMapping, nil
}
At this stage, I can get a search on covid to work, but when the term shortType:Tweet is added there are no results.
This is the searching snippet
type IndexHandler struct {
sync.Mutex
Index bleve.Index
}
...
// h is of type IndexHandler
queryS := "shortType:Tweet covid"
query := bleve.NewQueryStringQuery(queryS)
searchRequest := bleve.NewSearchRequestOptions(query, size, from, false)
searchRequest.Fields = []string{"*"}
searchRequest.Highlight = bleve.NewHighlight()
searchResult, _ := h.Index.Search(searchRequest)
docCount, _ := h.Index.DocCount()
fmt.Printf("GET %s\n", queryS)
fmt.Printf("%d(%d) of %d\n", len(searchResult.Hits), searchResult.Total, docCount)
Will this set-up be able to achieve the kind of query that I indicated? If so, what could explain the failure to retrieve any results?

Is there a way to directly map to struct the result of Couchbase N1QL query in Go

When I implement a query by selecting all the elements using * operator, the query is not mapped to the struct.
query := gocb.NewN1qlQuery("SELECT * FROM `item-bucket` WHERE itemBarcode=$1")
queryParams = append(queryParams, itemBarcode)
rows, err := itemBucket.ExecuteN1qlQuery(query, queryParams)
var row ItemEntity
for rows.Next(&row) {
fmt.Printf("Results: %+v\n", row)
}
However, when I add each of the fields into the query, it is directly mapped.
query := gocb.NewN1qlQuery("SELECT itemBarcode, color, price FROM `item-bucket` WHERE itemBarcode=$1")
queryParams = append(queryParams, itemBarcode)
rows, err := itemBucket.ExecuteN1qlQuery(query, queryParams)
var row ItemEntity
for rows.Next(&row) {
fmt.Printf("Results: %+v\n", row)
}
Is there a way to map directly to struct using the * operator?
The Go version of the project is 1.6
Have a look into sqlx, where their README contains examples using queries like SELECT * FROM x. If your database column names don't match your struct field names directly, you can specify them as struct tags like:
type ItemBucket struct {
ItemBarcode string `db:"itemBarcode"`
Color string `db:"last_name"`
...
}

Go - Save JSON string into a DB (most efficient?)

I am getting a JSON response from a server and want to save that into a db, where one of the columns is a JSON column. The response looks similar to the following:
msgJson = [{"id": 1, "type": "dog", "attributes": {"weight":20, "sound":"bark"}}]
So I am currently making a struct and trying to update each element in the DB.
type Animal struct {
Id int `json:"id"`
Type string `json:"type"`
Attributes string `json:"attributes"`
}
var animals []Animal
json.Unmarshal([]byte(msgJson), &animals)
sqlStatement := `
UPDATE animals
SET type = $2, attributes = $3
WHERE id = $1;`
_, err := db.Exec(
sqlStatement,
animals[0].Id,
animals[0].Type,
animals[0].Attributes)
Of course, this doesn't work, because the attributes field is supposed to be JSON.
I believe I could Unmarshal the JSON into nested structs, and then Marshal it when updating the DB, but since this will have many fields, is there a way to take the string and immediately represent it as JSON when adding to the DB? I hope that question makes sense.
Thank you
Unmarshal the attributes field to a json.RawMessage. Save the raw message to the database.
type Animal struct {
Id int `json:"id"`
Type string `json:"type"`
Attributes json.RawMessage `json:"attributes"`
}
⋮
_, err := db.Exec(
sqlStatement,
animals[0].Id,
animals[0].Type,
animals[0].Attributes)

Gorm query returning only a single row

We're trying to use Gorm with mysql 8 to much frustration.
I have the following tables (simplified for brevity here)
type StoragePool struct {
gorm.Model
PoolId string `json:"id" gorm:"column:poolid;size:40;unique;not null"`
Volumes []Volume `json:"volumes" gorm:"foreignkey:StorageId;association_foreignkey:PoolId"`
}
type Volume struct {
gorm.Model
StorageId string `json:"storageid" gorm:"column:storageid;size:40"`
}
Data insertions seem to work fine. Both tables get populated and no constraints are violated.
A query that expects a single record seems to work fine:
poolRecord := &StoragePool{}
if err := tx.Where("poolid = ?", pool.PoolId).First(&StoragePool{}).Scan(poolRecord).Error; err != nil {
return err
}
This query only returns a single row. When I perform this exact query as raw SQL outside of go, it returns all 31 records I expect.
var poolVolumes []Volume
if err := tx.Where("storageid = ?", pool.PoolId).Find(&Volume{}).Scan(&poolVolumes).Error; err != nil {
return err
}
log.Debugf("found %d volumes belonging to %q [%s]", len(poolVolumes), pool.Name, pool.PoolId)
According to the docs, that second sql statement is the equivalent of "SELECT * FROM VOLUMES WHERE STORAGEID = 'poolid'". That is not the behavior I am getting.
Anyone have any ideas what I'm doing wrong here?
I rarely use an ORM while coding with go, but following the doc from gorm, it seems like you are doing it the wrong way.
Scan is used for scanning result into another struct, like this:
type Result struct {
Name string
Age int
}
var result Result
db.Table("users").Select("name, age").Where("name = ?", 3).Scan(&result)
The correct way to get query results into a slice of structs should be:
var poolVolumes []Volume
if err := tx.Where("storageid = ?", pool.PoolId).Find(&poolVolumes).Error; err != nil {
return err
}

Dynamically generate struct fields from SQL QueryRow result

I want to retrieve all fields of the row and than render them to html. I know how to do it and here is a code for a row with 3 fields:
type View struct {
Id int
Name_and_requisits string
Reg_Date string
}
func getViewById(id int) (*View, error){
var vie View
row := db.QueryRow("select id, name_and_requisits, reg_date from book where id = ?;", id)
err := row.Scan(&vie.Id, &vie.Name_and_requisites, &vie.Reg_Date)
if err != nil {
return nil, err
}
return &vie, nil
}
But in my table one row includes about 20 columns and i need all of them with their names but i dont want to create a nasted hardcoded struct. I have an idea like to generate struct fields dynamically, from names of columns, and than use row.Scan on it. Any ideas? Maybe map is better for this situation?
Thanks!
generate struct fields dynamically
https://golang.org/pkg/reflect/#StructOf
But please: Don't do it.

Resources