I use Go to build my game server. I am going to compare a time from time.Now() and one from a Firestore field. How can I do that in Go?
playerDataSnap, err := Instace.FireStoreClient.Collection("PlayerData").Doc(playerUID).Get(Instace.Context)
if err != nil {
log.Printf("EventModeFee Get PlayerData Fail: %v", err)
return
}
playerData := playerDataSnap.Data()
if value, exist := playerData["EventPlayTimes"]; exist {
eventPlayTimes = value.(int64)
}
if discount_Subscribe > 0 {
if value, exist := playerData["SubscriptionExpiredDate"]; exist { //Get Subscribe expired time
var expireTimeStamp = value //<-------------timestamp from firestore field
if time.Now().Before(expireTimeStamp) {//<---------------expireTimeStamp is not a valid type, but How can I convert it to a valid type
isSubscribed = true
}
}
}
expireTimeStamp is not a valid type, but how can I convert it to a valid one?
It turns out that I can directly assert the timestamp from Firestore field to time.Time with the following code:
if time.Now().Before(expireTimeStamp.(time.Time)) {
isSubscribed = true
}
Related
I'm trying to add string "Employee" to my existing JSON response. Also, we need to be able to generate this version of json based on an user condition. Only if the user condition is met, I need to generate second version of json with string "Employee" added. If not the first version without string "Employee" should be generated. How can I achieve it with out updating the existing struct and how can I check this with if clause to check for the condition and then generate json based on it?
Below is my existing json response in go
[
{
"EmpId":{
"String":"ABCD",
"Valid":true
},
"Department":{
"Float64":0,
"Valid":true
}
}
]
How can I get my json response like below with out changing existing struct based on input parameter?
{
"Employee":[
{
"EmpId":{
"String":"ABCD",
"Valid":true
},
"Department":{
"Float64":0,
"Valid":true
}
}
]
}
Below is my code:
Step 1: model folder
type EmployeeWithRoot struct {
Employee []Employee
}
type Employee struct {
EmpNbr sql.NullString `json:"EmpNbr"`
DateofJoin sql.NullString `json:"DateofJoin"`
DeptId sql.NullString `json:"DeptId"`
DeptName sql.NullString `json:"DeptName"`
}
Step 2: code folder
func GetEmpDetails(logRequestId string, logNestedLevel int, EmpNbr string, DateofJoin string) ([]model.EmployeeWithRoot, error) {
logFunctionFunctionName := "code.GetEmpDetails"
logStartTime := time.Now()
logNestedLevel++
defer configurations.TimeTrack(logFunctionFunctionName, logRequestId, logStartTime, logNestedLevel)
rows, err := db.Query(utils.SELECT_OF_EMP_AGGR, EmpNbr, DateofJoin, DeptId, DeptName)
if err != nil {
return nil, err
}
defer rows.Close()
var e []model.EmployeeWithRoot
for rows.Next() {
var x model.EmployeeWithRoot
err := rows.Scan(&x.Employee.EmpNbr, &x.Employee.DateofJoin, &x.Employee.DeptId,&x.Employee.DeptName)
if err != nil {
return nil, err
}
e = append(e, x)
}
err = rows.Err()
if err != nil {
return nil, err
}
return e, nil
}
STEP 3: API folder
Employee, err := code.GetEmpDetails(logRequestId, logNestedLevel, EmpNbr, DateofJoin)
if err != nil {
log.Panic(err)
}
marshalDataForRequestContentType(logRequestId, logNestedLevel, w, r, Employee)
I'm getting the below error.
x.Employee.EmpNbr undefined (type []model.Employee has no field or method EmpNbr)
x.Employee.DateofJoin undefined (type []model.Employee has no field or method DateofJoin)enter code here
x.Employee.DeptId undefined (type []model.Employee has no field or method DeptId)
x.Employee.DeptName undefined (type []model.Employee has no field or method DeptName)
Considering you're just wrapping it it an outer object, I don't see any reason you'd need to change the existing struct, just wrap it in a new one. I'll have to make some guesses/assumptions here since you've only shown the JSON and not the Go code that produces it, but assuming your existing JSON is produced by marshaling something like var response []Employee, the desired JSON could be produced in your condition by marshaling instead:
json.Marshal(struct{Employee []Employee}{response})
Working example: https://go.dev/play/p/vwDvxnQ96G_2
Use string concatenation:
func addRoot(json string) string {
return `{ "Employee":` + json + `}`
}
Run an example on the GoLang playground.
Here's the code if you are working with []byte instead of string:
func addRoot(json []byte) []byte {
const prefix = `{ "Employee":`
const suffix = `}`
result := make([]byte, 0, len(prefix)+len(json)+len(suffix))
return append(append(append(result, prefix...), json...), suffix...)
}
Run this example on the GoLang playground.
If you have some JSON in a byte slice ([]byte) then you can just add the outer element directly - e.g. (playground):
existingJSON := []byte(`[
{
"EmpId":{
"String":"ABCD",
"Valid":true
},
"Department":{
"Float64":0,
"Valid":true
}
}
]`)
b := bytes.NewBufferString(`{"Employee":`)
b.Write(existingJSON)
b.WriteString(`}`)
fmt.Println(string(b.Bytes()))
If this is not what you are looking for please add further details to your question (ideally your attempt as a minimal, reproducible, example)
I'm making an http request in golang to an external api. It gives a general response of {"error":[]string, "result":changing interface{}}. depending on the function that is making the request, the Result field changes. Since I know the structure of the Result field for each function I run, I want to be able to change the value of Result before unmarshalling to json. I've tried to do this with the following code:
func GetAssets(output *Resp, resultType interface{}) error {
return publicRequest("/Assets", output, resultType)
}
func publicRequest(endPoint string, output *Resp, resultType interface{}) error {
url := Rest_url + Pub_rest_url + endPoint //"https://api.kraken.com/0/public/Assets in this case
resp, err := http.Get(url)
if err != nil {
return err
}
defer resp.Body.Close()
output.Result = resultType
return json.NewDecoder(resp.Body).Decode(&output)
}
Here is how it's being ran in main
type Resp struct {
Error []string `json:"error"`
Result interface{} `json:"result"`
}
type AssetInfo struct {
Aclass string `json:"aclass"`
Altname string `json:"altname"`
Decimals int `json:"decimals"`
Display int `json:"display_decimals"`
}
func main() {
var result map[string]AssetInfo
jsonData := Resp{}
rest_api_client.GetAssets(&jsonData, result)
fmt.Println(jsonData)
}
The issue is that it doesn't unmarshal correctly. A map is created for each asset, but the data contained inside of each asset is also being stored inside of a map. I'm not sure if I explained this well, but here is the current response after unmarshalling to understand what I mean.
Here is the data type of Resp.Result: map[string]interface {}
{[] map[1INCH:map[aclass:currency altname:1INCH decimals:10 display_decimals:5] AAVE:map[aclass:currency altname:AAVE decimals:10 display_decimals:5] ACA:map[aclass:currency altname:ACA decimals:10 display_decimals:5] ADA:map[aclass:currency altname:ADA decimals:8 display_decimals:6]...}
The response type I'm looking for is map[string]AssetInfo. Hopefully it could be unmarshalled like this:
{[] map[1INCH:{currency 1INCH 10 5} AAVE:{currency AAVE 10 5} ACA:{currency ACA 10 5} ADA:{currency ADA 8 6} ADA.S:{currency ADA.S 8 6}...}
Any help? I'd rather keep the Resp struct as generic as possible and just change the value of the Result field (if this is even possible to do correctly) since I plan to have multiple functions that call different endpoints of the api, and they'll all have the same underlying response type of the Resp struct with different Result types
You can view a working example in the following repo:
https://github.com/alessiosavi/GoArbitrage/blob/e107af466852b1ed30c2413eb4401595f7412b4f/markets/kraken/kraken.go
Basically, I've defined the following structure:
type Tickers struct {
Error []interface{} `json:"error"`
Result map[string]Ticker `json:"result"`
}
type Ticker struct {
Aclass string `json:"aclass"`
Altname string `json:"altname"`
Decimals int `json:"decimals"`
DisplayDecimals int `json:"display_decimals"`
}
Than I execute the request in the following way:
const KRAKEN_TICKERS_URL string = `https://api.kraken.com/0/public/Assets`
type Kraken struct {
PairsNames []string `json:"pairs_name"`
Pairs map[string]datastructure.KrakenPair `json:"pairs"`
OrderBook map[string]datastructure.KrakenOrderBook `json:"orderbook"`
MakerFee float64 `json:"maker_fee"`
TakerFees float64 `json:"taker_fee"`
// FeePercent is delegated to save if the fee is in percent or in coin
FeePercent bool `json:"fee_percent"`
Tickers []string
}
// Init is delegated to initialize the maps for the kraken
func (k *Kraken) Init() {
k.Pairs = make(map[string]datastructure.KrakenPair)
k.OrderBook = make(map[string]datastructure.KrakenOrderBook)
k.SetFees()
}
// SetFees is delegated to initialize the fee type/amount for the given market
func (k *Kraken) SetFees() {
k.MakerFee = 0.16
k.TakerFees = 0.26
k.FeePercent = true
}
func (k *Kraken) GetTickers() error {
res := datastructure.Tickers{}
var err error
var request req.Request
var data []byte
var tickers []string
resp := request.SendRequest(KRAKEN_TICKERS_URL, "GET", nil, nil, false, 10*time.Second)
if resp.Error != nil {
zap.S().Debugw("Error during http request. Err: " + resp.Error.Error())
return resp.Error
}
if resp.StatusCode != 200 {
zap.S().Warnw("Received a non 200 status code: " + strconv.Itoa(resp.StatusCode))
return errors.New("NON_200_STATUS_CODE")
}
data = resp.Body
if err = json.Unmarshal(data, &res); err != nil {
zap.S().Warn("ERROR! :" + err.Error())
return err
}
zap.S().Infof("Data: %v", res.Result)
tickers = make([]string, len(res.Result))
i := 0
for key := range res.Result {
tickers[i] = res.Result[key].Altname
i++
}
k.Tickers = tickers
return nil
}
I have this json that I convert to:
var leerCHAT []interface{}
but I am going through crazy hoops to get to any point on that map inside map and inside map crazyness, specially because some results are different content.
this is the Json
[
null,
null,
"hub:zWXroom",
"presence_diff",
{
"joins":{
"f718a187-6e96-4d62-9c2d-67aedea00000":{
"metas":[
{
"context":{},
"permissions":{},
"phx_ref":"zNDwmfsome=",
"phx_ref_prev":"zDMbRTmsome=",
"presence":"lobby",
"profile":{},
"roles":{}
}
]
}
},
"leaves":{}
}
]
I need to get to profile then inside there is a "DisplayName" field.
so I been doing crazy hacks.. and even like this I got stuck half way...
First is an array so I can just do something[elementnumber]
then is when the tricky mapping starts...
SORRY about all the prints etc is to debug and see the number of elements I am getting back.
if leerCHAT[3] == "presence_diff" {
var id string
presence := leerCHAT[4].(map[string]interface{})
log.Printf("algo: %v", len(presence))
log.Printf("algo: %s", presence["joins"])
vamos := presence["joins"].(map[string]interface{})
for i := range vamos {
log.Println(i)
id = i
}
log.Println(len(vamos))
vamonos := vamos[id].(map[string]interface{})
log.Println(vamonos)
log.Println(len(vamonos))
metas := vamonos["profile"].(map[string]interface{}) \\\ I get error here..
log.Println(len(metas))
}
so far I can see all the way to the meta:{...} but can't continue with my hacky code into what I need.
NOTICE: that since the id after Joins: and before metas: is dynamic I have to get it somehow since is always just one element I did the for range loop to grab it.
The array element at index 3 describes the type of the variant JSON at index 4.
Here's how to decode the JSON to Go values. First, declare Go types for each of the variant parts of the JSON:
type PrescenceDiff struct {
Joins map[string]*Presence // declaration of Presence type to be supplied
Leaves map[string]*Presence
}
type Message struct {
Body string
}
Declare a map associating the type string to the Go type:
var messageTypes = map[string]reflect.Type{
"presence_diff": reflect.TypeOf(&PresenceDiff{}),
"message": reflect.TypeOf(&Message{}),
// add more types here as needed
}
Decode the variant part to a raw message. Use use the name in the element at index 3 to create a value of the appropriate Go type and decode to that value:
func decode(data []byte) (interface{}, error) {
var messageType string
var raw json.RawMessage
v := []interface{}{nil, nil, nil, &messageType, &raw}
err := json.Unmarshal(data, &v)
if err != nil {
return nil, err
}
if len(raw) == 0 {
return nil, errors.New("no message")
}
t := messageTypes[messageType]
if t == nil {
return nil, fmt.Errorf("unknown message type: %q", messageType)
}
result := reflect.New(t.Elem()).Interface()
err = json.Unmarshal(raw, result)
return result, err
}
Use type switches to access the variant part of the message:
defer ws.Close()
for {
_, data, err := ws.ReadMessage()
if err != nil {
log.Printf("Read error: %v", err)
break
}
v, err := decode(data)
if err != nil {
log.Printf("Decode error: %v", err)
continue
}
switch v := v.(type) {
case *PresenceDiff:
fmt.Println(v.Joins, v.Leaves)
case *Message:
fmt.Println(v.Body)
default:
fmt.Printf("type %T not handled\n", v)
}
}
Run it on the playground.
I'm trying to upload data to a PSQL database and receive the following message:
error:
null value in column \"alarm\" violates not-null constraint
I've tracked it down to being that the alarm value is 0, not null, but 0... However Golang does not recognize this... Is there a way to get around this quirk?
type HsDatum struct {
ID int64
UUID uuid.UUID
HsAssociationID int64
Timestamp time.Time `sql:",notnull"`
Value int `sql:",notnull"`
Alarm int `sql:",notnull"`
CreatedAt time.Time
UpdatedAt time.Time
}
hsDatum := hsdata.HsDatum{
HsAssociationID: systemAssoc.ID,
Timestamp: startTime,
Value: 1,
Alarm: int(alarm), // 0 in this example
}
Added more information about the database logic:
func (h *HsDatum) BeforeInsert(db orm.DB) error {
timeNow := timeutil.TimeNow()
h.CreatedAt = timeNow
h.UpdatedAt = timeNow
// Manage distinct alarm table by checking for new value
// Note: HS data needs to arrive in chronological order by timestamp
needsUpdate := false
lastAlarm, err := hsdistinctalarms.GetHsDistinctAlarmByMostRecent(h.HsAssociationID)
var timeGap time.Duration
if database.IsEmptyTable(err) {
// No values so mark it
needsUpdate = true
} else if err == nil {
if lastAlarm.Alarm != h.Alarm {
// New value so mark it
needsUpdate = true
}
timeGap = h.Timestamp.Sub(lastAlarm.LastUpdated)
} else {
logrus.Errorf("Failed to get most recent distinct alarm: %s", err)
}
if needsUpdate {
// Create new alarm
newAlarm := hsdistinctalarms.HsDistinctAlarm{
HsAssociationID: h.HsAssociationID,
Timestamp: h.Timestamp,
Alarm: h.Alarm,
LastUpdated: h.Timestamp,
}
err = hsdistinctalarms.CreateHsDistinctAlarm(&newAlarm)
if err != nil {
logrus.Errorf("Failed to create hs distinct alarm: %s", err)
}
} else {
// Update timestamp
lastAlarm.LastUpdated = h.Timestamp
err = hsdistinctalarms.UpdateHsDistinctAlarm(&lastAlarm)
if err != nil {
logrus.Errorf("Failed to update hs distinct alarm: %s", err)
}
}
return nil
}
// CreateHsDatum creates the resource.
// It modifies the parameter in place and returns error information.
func CreateHsDatum(h *HsDatum) error {
return database.GetDB().Insert(h)
}
Option 1:
use pointers
e.g.
Alarm *int `sql:",notnull"`
...
Alarm: new(int) // init with zero value
or
Alarm: &alarm // set a pointer to the alarm variable (should be int)
Ugly? Yes.
Option 2:
use sql.NullInt32
e.g.
Alarm sql.NullInt32 `sql:",notnull"`
...
var a sql.NullInt32
if err := a.Scan(alarm); err != nil {
// handle error
}
...
Alarm: a
...
I'm looking to iterate over the string fields of a struct so I can do some clean-up/validation (with strings.TrimSpace, strings.Trim, etc).
Right now I have a messy switch-case that's not really scalable, and as this isn't in a hot spot of my application (a web form) it seems leveraging reflect is a good choice here.
I'm at a bit of a roadblock for how to implement this however, and the reflect docs are a little confusing to me (I've been digging through some other validation packages, but they're way too heavyweight + I'm using gorilla/schema for the unmarshalling part already):
Iterate over the struct
For each field of type string, apply whatever I need to from the strings package i.e. field = strings.TrimSpace(field)
If there exists a field.Tag.Get("max"), we'll use that value (strconv.Atoi, then unicode.RuneCountInString)
Provide an error slice that's also compatible with the error interface type
type FormError []string
type Listing struct {
Title string `max:"50"`
Location string `max:"100"`
Description string `max:"10000"`
ExpiryDate time.Time
RenderedDesc template.HTML
Contact string `max:"255"`
}
// Iterate over our struct, fix whitespace/formatting where possible
// and return errors encountered
func (l *Listing) Validate() error {
typ := l.Elem().Type()
var invalid FormError
for i = 0; i < typ.NumField(); i++ {
// Iterate over fields
// For StructFields of type string, field = strings.TrimSpace(field)
// if field.Tag.Get("max") != "" {
// check max length/convert to int/utf8.RuneCountInString
if max length exceeded, invalid = append(invalid, "errormsg")
}
if len(invalid) > 0 {
return invalid
}
return nil
}
func (f FormError) Error() string {
var fullError string
for _, v := range f {
fullError =+ v + "\n"
}
return "Errors were encountered during form processing: " + fullError
}
Thanks in advance.
What you want is primarily the methods on reflect.Value called NumFields() int and Field(int). The only thing you're really missing is the string check and SetString method.
package main
import "fmt"
import "reflect"
import "strings"
type MyStruct struct {
A,B,C string
I int
D string
J int
}
func main() {
ms := MyStruct{"Green ", " Eggs", " and ", 2, " Ham ", 15}
// Print it out now so we can see the difference
fmt.Printf("%s%s%s%d%s%d\n", ms.A, ms.B, ms.C, ms.I, ms.D, ms.J)
// We need a pointer so that we can set the value via reflection
msValuePtr := reflect.ValueOf(&ms)
msValue := msValuePtr.Elem()
for i := 0; i < msValue.NumField(); i++ {
field := msValue.Field(i)
// Ignore fields that don't have the same type as a string
if field.Type() != reflect.TypeOf("") {
continue
}
str := field.Interface().(string)
str = strings.TrimSpace(str)
field.SetString(str)
}
fmt.Printf("%s%s%s%d%s%d\n", ms.A, ms.B, ms.C, ms.I, ms.D, ms.J)
}
(Playground link)
There are two caveats here:
You need a pointer to what you're going to change. If you have a value, you'll need to return the modified result.
Attempts to modify unexported fields generally will cause reflect to panic. If you plan on modifying unexported fields, make sure to do this trick inside the package.
This code is rather flexible, you can use switch statements or type switches (on the value returned by field.Interface()) if you need differing behavior depending on the type.
Edit: As for the tag behavior, you seem to already have that figured out. Once you have field and have checked that it's a string, you can just use field.Tag.Get("max") and parse it from there.
Edit2: I made a small error on the tag. Tags are part of the reflect.Type of a struct, so to get them you can use (this is a bit long-winded) msValue.Type().Field(i).Tag.Get("max")
(Playground version of the code you posted in the comments with a working Tag get).
I got beat to the punch, but since I went to the work, here's a solution:
type FormError []*string
type Listing struct {
Title string `max:"50"`
Location string `max:"100"`
Description string `max:"10000"`
ExpiryDate time.Time
RenderedDesc template.HTML
Contact string `max:"255"`
}
// Iterate over our struct, fix whitespace/formatting where possible
// and return errors encountered
func (l *Listing) Validate() error {
listingType := reflect.TypeOf(*l)
listingValue := reflect.ValueOf(l)
listingElem := listingValue.Elem()
var invalid FormError = []*string{}
// Iterate over fields
for i := 0; i < listingElem.NumField(); i++ {
fieldValue := listingElem.Field(i)
// For StructFields of type string, field = strings.TrimSpace(field)
if fieldValue.Type().Name() == "string" {
newFieldValue := strings.TrimSpace(fieldValue.Interface().(string))
fieldValue.SetString(newFieldValue)
fieldType := listingType.Field(i)
maxLengthStr := fieldType.Tag.Get("max")
if maxLengthStr != "" {
maxLength, err := strconv.Atoi(maxLengthStr)
if err != nil {
panic("Field 'max' must be an integer")
}
// check max length/convert to int/utf8.RuneCountInString
if utf8.RuneCountInString(newFieldValue) > maxLength {
// if max length exceeded, invalid = append(invalid, "errormsg")
invalidMessage := `"`+fieldType.Name+`" is too long (max allowed: `+maxLengthStr+`)`
invalid = append(invalid, &invalidMessage)
}
}
}
}
if len(invalid) > 0 {
return invalid
}
return nil
}
func (f FormError) Error() string {
var fullError string
for _, v := range f {
fullError = *v + "\n"
}
return "Errors were encountered during form processing: " + fullError
}
I see you asked about how to do the tags. Reflection has two components: a type and a value. The tag is associated with the type, so you have to get it separately than the field: listingType := reflect.TypeOf(*l). Then you can get the indexed field and the tag from that.
I don't know if it's a good way, but I use it like this.
https://play.golang.org/p/aQ_hG2BYmMD
You can send the address of a struct to this function.
Sorry for My English is not very good.
trimStruct(&someStruct)
func trimStruct(v interface{}) {
bytes, err := json.Marshal(v)
if err != nil {
fmt.Println("[trimStruct] Marshal Error :", err)
}
var mapSI map[string]interface{}
if err := json.Unmarshal(bytes, &mapSI); err != nil {
fmt.Println("[trimStruct] Unmarshal to byte Error :", err)
}
mapSI = trimMapStringInterface(mapSI).(map[string]interface{})
bytes2, err := json.Marshal(mapSI)
if err != nil {
fmt.Println("[trimStruct] Marshal Error :", err)
}
if err := json.Unmarshal(bytes2, v); err != nil {
fmt.Println("[trimStruct] Unmarshal to b Error :", err)
}
}
func trimMapStringInterface(data interface{}) interface{} {
if values, valid := data.([]interface{}); valid {
for i := range values {
data.([]interface{})[i] = trimMapStringInterface(values[i])
}
} else if values, valid := data.(map[string]interface{}); valid {
for k, v := range values {
data.(map[string]interface{})[k] = trimMapStringInterface(v)
}
} else if value, valid := data.(string); valid {
data = strings.TrimSpace(value)
}
return data
}