I'm stuck with an obvious operation: retrieving multiple rows using gorm.Find() method.
(resolver.go)
package resolver
type Root struct {
DB *gorm.DB
}
func (r *Root) Users(ctx context.Context) (*[]*UserResolver, error) {
var userRxs []*UserResolver
var users []model.User
// debug-start
// This is to prove r.DB is allocated and working
// It will print {2 alice#mail.com} in the console
var user model.User
r.DB.Find(&user)
log.Println(user)
// debug-end
if err := r.DB.Find(&users); err != nil { // <-- not working
log.Fatal(err)
}
for _, user := range users {
userRxs = append(userRxs, &UserResolver{user})
log.Println(user)
}
return &userRxs, nil
}
(model.go)
package model
type User struct {
ID graphql.ID `gorm:"primary_key"`
Email string `gorm:"unique;not null"`
}
The mysql table is filled with 2 values. Here is the content in json style:
{
{ Email: bob#mail.com },
{ Email: alice#mail.com },
}
This is the result when I run the program:
2020/05/13 12:23:17 Listening for requests on :8000
2020/05/13 12:23:22 {2 alice#mail.com}
2020/05/13 12:23:22 &{{{0 0} 0 0 0 0} 0xc0004cee40 <nil> 2 0xc00031e3c0 false 0 {0xc00035bea0} 0xc0004b3080 {{0 0} {<nil>} map[] 0} 0xc000408340 <nil> 0xc0004cee60 false <nil>}
What is wrong with my code? It seems from all the tuto/so/etc.. sources that I'm correctly defining a slice var and passing it to the Find() function..
if err := r.DB.Find(&users); err != nil { // <-- not working
log.Fatal(err)
}
Probably you forgot to mention Error property and returned object in this case is not nil for sure (please mention that Find returns not error interface in this case)
Please try something like that
if err := r.DB.Find(&users).Error; err != nil {
log.Fatal(err)
}
Hope it helps
You need to use a slice of pointers:
users := make([]*model.User, 0, 2)
if err := r.DB.Find(&users).Error; err != nil {
log.Fatal(err)
}
Related
I'm trying to seed my Postgres database as functionally. In my case, SeedSchema() function can take any type struct. So I define a interface and create functions to my structs which will seed. I tried with generics and without.
When I unmarshall any json array from file as byte array, json.Unmarshall method change my tempMember member of struct. Exp, models.Term to map[string]interface{}. I've used unmarshall before this function and I've not seen like this situation.
Here is my SeedSchema() function:
func (db *Database) SeedSchema(models ...globals.Seeder[any]) error {
var (
subjects []globals.Seeder[any]
fileByte []byte
err error
// tempMember any
)
if len(models) == 0 {
subjects = seederModelList
} else {
subjects = models
}
for _, model := range subjects {
fileName, tempMember := model.Seed()
fmt.Printf("%+v\n", reflect.TypeOf(tempMember)) //1
if fileByte, err = os.ReadFile("db/seeds/" + fileName); err != nil {
fmt.Println(err)
return err
}
if err = json.Unmarshal(fileByte, &tempMember); err != nil {
fmt.Println(err)
return err
}
fmt.Printf("%+v\n", reflect.TypeOf(tempMember)) //2
}
return nil
}
First print returns []models.AirportCodes and the second []interface {}.
Here is my interface and model:
func (AirportCodes) Seed() (string, any) {
return "airport_codes.json", []AirportCodes{}
}
type Seeder[T any] interface {
Seed() (string, T)
// Seed(*gorm.DB) error
TableName() string
}
seederModelList = []globals.Seeder[any]{
m.AirportCodes{},
m.Term{},
}
After a few weeks, I have looking for solve this problem and look unmarshaler interfaces and examples. Then Like what icza said, I started to look over the my code that convention between types and I solved like this. If you guys have better answer than mine, please add answer.
Data:
[
{
"id":1,
"name":"Term 1",
"content": [
"a1",
"a2",
"a3"
]
}
]
Result:
[{ID:1 Name:Term 1 Content:[a1 a2 a3]}]
UnmarshalJSON Function:
func (term *Term) UnmarshalJSON(data []byte) error {
tempMap := map[string]interface{}{}
if err := json.Unmarshal(data, &tempMap); err != nil {
return err
}
*term = Term{
Name: tempMap["name"].(string),
}
if tempMap["content"] != nil {
for _, v := range tempMap["content"].([]interface{}) {
(*term).Content = append((term).Content, v.(string))
}
}
return nil
}
Thank you for comments.
I have a function that retrieve the mongodb admin users using .command
In the below function, I have the DbUsers struct, and I am running the command to retrieve the users from 2 different database.
My question is, how can I concat the 2 results (adminUsers & externalUsers) and return after merged? They are of the same struct.
type DbUsers struct {
...lots of stuff about the server
Users []Users
}
type Users struct {
User string
...lots of stuff
}
func getUsers() Users {
admin := CNX.Database("admin")
external := CNX.Database("$external")
command := bson.D{primitive.E{Key: "usersInfo", Value: 1}}
var adminUsers DbUsers
var externalUsers DbUsers
err := admin.RunCommand(context.TODO(), command).Decode(&adminUsers)
if err != nil {
panic(err)
}
err2 := external.RunCommand(context.TODO(), command).Decode(&externalUsers)
if err2 != nil {
panic(err2)
}
//New to Golang, not sure what I am doing but this doesn't work
return []Users{adminUsers.Users, externalUsers.Users}
}
You can do
return append(adminUsers.Users, externalUsers.Users...)
I am trying to build a generic CrudRepository struct using Gorm for my api.
I know generics are coming to GoLang in version 2 but I try to build this lib using reflection or any other lib.
In my CrudRepository:
func (repository *BaseRepository) find(result interface{}, pageSize int, page int) error {
if page < 1 {
return errors.ExceedsMinimumInt("page", "", 0, true, nil)
}
offset := (page - 1) * pageSize
ent := reflect.Zero(reflect.TypeOf(result))
repository.db = repository.db.Limit(pageSize).Offset(offset)
err := repository.db.Find(&ent).Error
result = ent
if err != nil {
return err
}
return nil
}
And calling this method sth like:
func List(){
var entityList []MyEntity
find(entityList, 1, 10)
}
I think, I cannot pass any interface reference into Gorm.db.Find() method
Is there any other way to succeed?
Use a pointer of a slice as input argument of custom find method.
func (repository *BaseRepository) find(result interface{}, pageSize int, page int) error {
if page < 1 {
return errors.ExceedsMinimumInt("page", "", 0, true, nil)
}
if reflect.TypeOf(result).Kind() != reflect.Slice { 👈 check ❗️
return errors.New("`result` is not a slice")
}
offset := (page - 1) * pageSize
db = db.Limit(pageSize).Offset(offset)
if err := db.Find(result).Error; err != nil {
return err
}
return nil
}
usage 👇🏻
var entityList []MyEntity
err := find(&entityList, 10, 1)
Also you have to check input argument (result), because db.Find isn't fit to find single strut 👇🏻 (Retrieving a single object)
If you want to avoid the ErrRecordNotFound error, you could use Find
like db.Limit(1).Find(&user), the Find method accepts both struct and
slice data
For example (Book table is empty):
b := Book{}
rowsAffectedQuantity := db.Find(&b).RowsAffected // 👈 0
err = db.Find(&b).Error // 👈 nil
I have this json that I convert to:
var leerCHAT []interface{}
but I am going through crazy hoops to get to any point on that map inside map and inside map crazyness, specially because some results are different content.
this is the Json
[
null,
null,
"hub:zWXroom",
"presence_diff",
{
"joins":{
"f718a187-6e96-4d62-9c2d-67aedea00000":{
"metas":[
{
"context":{},
"permissions":{},
"phx_ref":"zNDwmfsome=",
"phx_ref_prev":"zDMbRTmsome=",
"presence":"lobby",
"profile":{},
"roles":{}
}
]
}
},
"leaves":{}
}
]
I need to get to profile then inside there is a "DisplayName" field.
so I been doing crazy hacks.. and even like this I got stuck half way...
First is an array so I can just do something[elementnumber]
then is when the tricky mapping starts...
SORRY about all the prints etc is to debug and see the number of elements I am getting back.
if leerCHAT[3] == "presence_diff" {
var id string
presence := leerCHAT[4].(map[string]interface{})
log.Printf("algo: %v", len(presence))
log.Printf("algo: %s", presence["joins"])
vamos := presence["joins"].(map[string]interface{})
for i := range vamos {
log.Println(i)
id = i
}
log.Println(len(vamos))
vamonos := vamos[id].(map[string]interface{})
log.Println(vamonos)
log.Println(len(vamonos))
metas := vamonos["profile"].(map[string]interface{}) \\\ I get error here..
log.Println(len(metas))
}
so far I can see all the way to the meta:{...} but can't continue with my hacky code into what I need.
NOTICE: that since the id after Joins: and before metas: is dynamic I have to get it somehow since is always just one element I did the for range loop to grab it.
The array element at index 3 describes the type of the variant JSON at index 4.
Here's how to decode the JSON to Go values. First, declare Go types for each of the variant parts of the JSON:
type PrescenceDiff struct {
Joins map[string]*Presence // declaration of Presence type to be supplied
Leaves map[string]*Presence
}
type Message struct {
Body string
}
Declare a map associating the type string to the Go type:
var messageTypes = map[string]reflect.Type{
"presence_diff": reflect.TypeOf(&PresenceDiff{}),
"message": reflect.TypeOf(&Message{}),
// add more types here as needed
}
Decode the variant part to a raw message. Use use the name in the element at index 3 to create a value of the appropriate Go type and decode to that value:
func decode(data []byte) (interface{}, error) {
var messageType string
var raw json.RawMessage
v := []interface{}{nil, nil, nil, &messageType, &raw}
err := json.Unmarshal(data, &v)
if err != nil {
return nil, err
}
if len(raw) == 0 {
return nil, errors.New("no message")
}
t := messageTypes[messageType]
if t == nil {
return nil, fmt.Errorf("unknown message type: %q", messageType)
}
result := reflect.New(t.Elem()).Interface()
err = json.Unmarshal(raw, result)
return result, err
}
Use type switches to access the variant part of the message:
defer ws.Close()
for {
_, data, err := ws.ReadMessage()
if err != nil {
log.Printf("Read error: %v", err)
break
}
v, err := decode(data)
if err != nil {
log.Printf("Decode error: %v", err)
continue
}
switch v := v.(type) {
case *PresenceDiff:
fmt.Println(v.Joins, v.Leaves)
case *Message:
fmt.Println(v.Body)
default:
fmt.Printf("type %T not handled\n", v)
}
}
Run it on the playground.
This question already has answers here:
Unmarshal 2 different structs in a slice
(3 answers)
Closed 3 years ago.
i'm struggling to create a data structure for unmarshal the following json:
{
"asks": [
["2.049720", "183.556", 1576323009],
["2.049750", "555.125", 1576323009],
["2.049760", "393.580", 1576323008],
["2.049980", "206.514", 1576322995]
],
"bids": [
["2.043800", "20.691", 1576322350],
["2.039080", "755.396", 1576323007],
["2.036960", "214.621", 1576323006],
["2.036930", "700.792", 1576322987]
]
}
If I use the following struct with interfaces, there is no problem:
type OrderBook struct {
Asks [][]interface{} `json:"asks"`
Bids [][]interface{} `json:"bids"`
}
But i need a more strict typing, so i've tried with:
type BitfinexOrderBook struct {
Pair string `json:"pair"`
Asks [][]BitfinexOrder `json:"asks"`
Bids [][]BitfinexOrder `json:"bids"`
}
type BitfinexOrder struct {
Price string
Volume string
Timestamp time.Time
}
But unfortunately i had not success.
This is the code that I have used for parse the Kraken API for retrieve the order book:
// loadKrakenOrderBook is delegated to load the data related to pairs info
func loadKrakenOrderBook(data []byte) (datastructure.BitfinexOrderBook, error) {
var err error
// Creating the maps for the JSON data
m := map[string]interface{}{}
var orderbook datastructure.BitfinexOrderBook
// Parsing/Unmarshalling JSON
err = json.Unmarshal(data, &m)
if err != nil {
zap.S().Debugw("Error unmarshalling data: " + err.Error())
return orderbook, err
}
a := reflect.ValueOf(m["result"])
if a.Kind() == reflect.Map {
key := a.MapKeys()[0]
log.Println("KEY: ", key)
strct := a.MapIndex(key)
log.Println("MAP: ", strct)
m, _ := strct.Interface().(map[string]interface{})
log.Println("M: ", m)
data, err := json.Marshal(m)
if err != nil {
zap.S().Warnw("Panic on key: ", key.String(), " ERR: "+err.Error())
return orderbook, err
}
log.Println("DATA: ", string(data))
err = json.Unmarshal(data, &orderbook)
if err != nil {
zap.S().Warnw("Panic on key: ", key.String(), " during unmarshal. ERR: "+err.Error())
return orderbook, err
}
return orderbook, nil
}
return orderbook, errors.New("UNABLE_PARSE_VALUE")
}
The data that i use for test are the following:
{
"error": [],
"result": {
"LINKUSD": {
"asks": [
["2.049720", "183.556", 1576323009],
["2.049750", "555.125", 1576323009],
["2.049760", "393.580", 1576323008],
["2.049980", "206.514", 1576322995]
],
"bids": [
["2.043800", "20.691", 1576322350],
["2.039080", "755.396", 1576323007],
["2.036960", "214.621", 1576323006],
["2.036930", "700.792", 1576322987]
]
}
}
}
EDIT
NOTE: the data that i receive in input is the latest json that i've post, not the array of bids and asks.
I've tried to integrate the solution proposed by #chmike. Unfortunately there is a few preprocessing to be made, cause the data is the latest json that i've post.
So i've changed to code as following in order to extract the json data related to asks and bids.
func order(data []byte) (datastructure.BitfinexOrderBook, error) {
var err error
// Creating the maps for the JSON data
m := map[string]interface{}{}
var orderbook datastructure.BitfinexOrderBook
// var asks datastructure.BitfinexOrder
// var bids datastructure.BitfinexOrder
// Parsing/Unmarshalling JSON
err = json.Unmarshal(data, &m)
if err != nil {
zap.S().Warn("Error unmarshalling data: " + err.Error())
return orderbook, err
}
// Extract the "result" json
a := reflect.ValueOf(m["result"])
if a.Kind() == reflect.Map {
key := a.MapKeys()[0]
log.Println("KEY: ", key)
log.Println()
strct := a.MapIndex(key)
log.Println("MAP: ", strct)
m, _ := strct.Interface().(map[string]interface{})
log.Println("M: ", m)
log.Println("Asks: ", m["asks"])
log.Println("Bids: ", m["bids"])
// Here i retrieve the asks array
asks_data, err := json.Marshal(m["asks"])
log.Println("OK: ", err)
log.Println("ASKS: ", string(asks_data))
var asks datastructure.BitfinexOrder
// here i try to unmarshal the data into the struct
asks, err = UnmarshalJSON(asks_data)
log.Println(err)
log.Println(asks)
}
return orderbook, errors.New("UNABLE_PARSE_VALUE")
}
Unfortunately, i receive the following error:
json: cannot unmarshal array into Go value of type json.Number
As suggested by #Flimzy, you need a custom Unmarshaler. Here it is.
Note that the BitfinexOrderBook definition is slightly different from yours. There was an error in it.
// BitfinexOrderBook is a book of orders.
type BitfinexOrderBook struct {
Asks []BitfinexOrder `json:"asks"`
Bids []BitfinexOrder `json:"bids"`
}
// BitfinexOrder is a bitfinex order.
type BitfinexOrder struct {
Price string
Volume string
Timestamp time.Time
}
// UnmarshalJSON decode a BifinexOrder.
func (b *BitfinexOrder) UnmarshalJSON(data []byte) error {
var packedData []json.Number
err := json.Unmarshal(data, &packedData)
if err != nil {
return err
}
b.Price = packedData[0].String()
b.Volume = packedData[1].String()
t, err := packedData[2].Int64()
if err != nil {
return err
}
b.Timestamp = time.Unix(t, 0)
return nil
}
Note also that this custom unmarshaler function allows you to convert the price or volume to a float, which is probably what you want.
While you can hack your way by using reflex, or maybe even write your own parser, the most efficient way is to implement a json.Unmarshaler.
There are a few problem remaining, though.
You are transforming a json array to the struct, not just interface{} elements in it, so it should be: Asks []BitfinexOrder and Bids []BitfinexOrder.
You need to wrap the struct BitfinexOrderBook to get it work with its data. It is trivial and much simpler than using reflex.
By default, json.Unmarshal unmarshals a json number into a float64, which is not a good thing when parsing timestamp. You can use json.NewDecoder to get a decoder and then use Decoder.UseNumber to force use a string.
For example,
func (bo *BitfinexOrder) UnmarshalJSON(data []byte) error {
dec := json.NewDecoder(bytes.NewReader(data))
dec.UseNumber()
var x []interface{}
err := dec.Decode(&x)
if err != nil {
return errParse(err.Error())
}
if len(x) != 3 {
return errParse("length is not 3")
}
price, ok := x[0].(string)
if !ok {
return errParse("price is not string")
}
volume, ok := x[1].(string)
if !ok {
return errParse("volume is not string")
}
number, ok := x[2].(json.Number)
if !ok {
return errParse("timestamp is not number")
}
tint64, err := strconv.ParseInt(string(number), 10, 64)
if err != nil {
return errParse(fmt.Sprintf("parsing timestamp: %s", err))
}
*bo = BitfinexOrder{
Price: price,
Volume: volume,
Timestamp: time.Unix(tint64, 0),
}
return nil
}
and main func (wrapping the struct):
func main() {
x := struct {
Result struct{ LINKUSD BitfinexOrderBook }
}{}
err := json.Unmarshal(data, &x)
if err != nil {
log.Fatalln(err)
}
bob := x.Result.LINKUSD
fmt.Println(bob)
}
Playground link: https://play.golang.org/p/pC124F-3M_S .
Note: the playground link use a helper function to create errors. Some might argue it is best to name the helper function NewErrInvalidBitfinexOrder or rename the error. That is not the scope of this question and I think for the sake of typing, I will keep the short name for now.