Golang dynamically creating member of struct - go

I know there is struct in Go, but for all I know, you have to define struct
type Circle struct{
x,y,r float64
}
I am wondering how you can declare a new variable that doesn't exist in the struct
circle := new(Circle)
circle.color = "black"

You will need to use a map (of type map[string]interface{}) to work with dynamic JSON. Here is an example of creating a new map:
// Initial declaration
m := map[string]interface{}{
"key": "value",
}
// Dynamically add a sub-map
m["sub"] = map[string]interface{}{
"deepKey": "deepValue",
}
Unmarshalling JSON into a map looks like:
var f interface{}
err := json.Unmarshal(b, &f)
The code above would leave you with a map in f, with a structure resembling:
f = map[string]interface{}{
"Name": "Wednesday",
"Age": 6,
"Parents": []interface{}{
"Gomez",
"Morticia",
},
}
You will need to use a type assertion to access it, otherwise Go won't know it's a map:
m := f.(map[string]interface{})
You will also need to use assertions or type switches on each item you pull out of the map. Dealing with unstructured JSON is a hassle.
More information:
https://blog.golang.org/json-and-go
https://godoc.org/encoding/json#Unmarshal

I've started to work on this small repository https://github.com/Ompluscator/dynamic-struct
It's possible at this point to extend existing struct in runtime, by passing a instance of struct and modifying fields (adding, removing, changing types and tags).
Still in progress, so don't expect something huge :)
EDIT: At this point, work on library is done, and it looks stable for last a couple of months :)

You can do it using reflect package, check StructOf method it allows you to create a new struct from []reflect.StructField. Example:
func main() {
typ := reflect.StructOf([]reflect.StructField{
{
Name: "Height",
Type: reflect.TypeOf(float64(0)),
Tag: `json:"height"`,
},
{
Name: "Age",
Type: reflect.TypeOf(int(0)),
Tag: `json:"age"`,
},
})
v := reflect.New(typ).Elem()
v.Field(0).SetFloat(0.4)
v.Field(1).SetInt(2)
s := v.Addr().Interface()
w := new(bytes.Buffer)
if err := json.NewEncoder(w).Encode(s); err != nil {
panic(err)
}
fmt.Printf("value: %+v\n", s)
fmt.Printf("json: %s", w.Bytes())
r := bytes.NewReader([]byte(`{"height":1.5,"age":10}`))
if err := json.NewDecoder(r).Decode(s); err != nil {
panic(err)
}
fmt.Printf("value: %+v\n", s)
}

You can't. Go is statically typed, and does not allow such constructs.
Structs have a layout in memory that directly related to the definition, and there's no where to store such additional fields.
You can use a map instead. Moreover, you can use &circle as a key or part of a key, to associate map elements with arbitrary structs.
type key struct {
target interface{}
field string
}
x := make(map[key]string)
x[key{ target: circle, field: "color" }] = "black"

Related

Type assertion for TypeList in terraform provider

I'm writing a Terraform provider and I'm trying to figure out the best way to do type assertion when I have a TypeList containing elements of TypeString.
The resouce is defined as follows:
return &schema.Resource{
Create: resourceConfigObjectCreate,
Read: resourceConfigObjectRead,
Update: resourceConfigObjectUpdate,
Delete: resourceConfigObjectDelete,
Schema: map[string]*schema.Schema{
"name": &schema.Schema{
Type: schema.TypeString,
Required: true,
},
"notification_options": &schema.Schema{
Type: schema.TypeList,
Optional: true,
Elem: schema.Schema{
Type: schema.TypeString,
},
},
},
}
}
And i would like to load those values to a custom type defined like that:
type ConfigObject struct {
Name string `json:"name,omitempty"`
NotificationOptions []string `json:"notification_options,omitempty"`
}
Since schema.ResourceData.Get returns an interface{} a type assertion is needed.
item := thruk.ConfigObject{
Name: schema.ResourceData.Get("name").(string),
NotificationOptions: extractSliceOfStrings(schema.ResourceData.Get("notification_options")),
}
I've done it easily for string but the slice of strings was more complex and I created the following function:
func extractSliceOfStrings(i interface{}) (slice []string) {
s := reflect.ValueOf(i)
if !s.IsValid() {
return
}
for i := 0; i < s.Len(); i++ {
slice = append(slice, s.Index(i).String())
}
return
}
Is this the correct approach?
When working with the ResourceData API in a Terraform provider, it's helpful to know which Go type corresponds to each of the schema types. You've already inferred that schema.TypeString corresponds to string. Here's a complete list:
TypeBool ⇒ bool
TypeString ⇒ string
TypeInt ⇒ int
TypeList ⇒ []interface{}
TypeMap ⇒ map[string]interface{}
TypeSet ⇒ *schema.Set
Element type when Elem is set to a *schema.Resource: map[string]interface{}
The translations above are documented on the Schema Types documentation page for the SDK, as "Data structure:" under each of the headings.
Whenever you are dealing with a collection, the element type from Go's perspective is always interface{} to reflect the fact that the element type isn't decided until runtime. However, the same mapping rules defined above apply to those element values too, and so to convert a TypeList whose Elem is a TypeString you'd first assert the slice type, and then assert each element in turn:
itemsRaw := d.Get("example").([]interface{})
items := make([]string, len(itemsRaw))
for i, raw := range itemsRaw {
items[i] = raw.(string)
}
Unfortunately there is no way to go directly from []interface{} to []string in a single step, due to the design of Go interfaces and type assertions.
You can take a similar approach for TypeMap, if you ultimately need map[string]string:
itemsRaw := d.Get("example").(map[string]interface{})
items := make(map[string]string, len(itemsRaw))
for k, raw := range itemsRaw {
items[k] = raw.(string)
}
TypeSet is a little more complicated due to the custom *schema.Set container, but you can call the List method of the set to obtain a []interface{} which you can then treat the same as with TypeList above:
itemsRaw := d.Get("example").(*schema.Set).List()
items := make([]string, len(itemsRaw))
for i, raw := range itemsRaw {
items[i] = raw.(string)
}

Creating structs programmatically at runtime - possible?

Is it possible in Go to create a struct type programmatically (i.e. not in the compiled source code)?
We have a particular use case where a type will be created via user-defined metadata (so the schema/types are not known in advance)
and will vary for every customer. We would then need to auto-generate REST services for those and persist them in a NoSQL backend.
We would also need to define different dynamic validators per field (e.g. mandatory, regex, max/min size, max/min value, a reference to another type instance, etc.)
I was wondering if something similar is possible in the Go?
Edit 1:
For example
From frontend in JSON
For customer 1:
{
"id":1,
"patientid":1,
"name":"test1",
"height":"160",
"weight":"70",
"temp":"98"
}
For customer 2:
{
"id":2,
"patientid":1,
"name":"test1",
"height":"160",
"weight":"70"
}
For customer 3
may be different new fields will add
Backend
// For One customer need to have these fields
type Vitalsigns struct {
ID int64 `datastore:"-"`
PatientID int64 `json:"patientid,omitempty"`
Name string `json:"name,omitempty"`
Height string `json:"height,omitempty"`
Weight string `json:"weight,omitempty"`
Temp string `json:"temp,omitempty"`
}
// Another need to have these fields
type Vitalsigns struct {
ID int64 `datastore:"-"`
PatientID int64 `json:"patientid,omitempty"`
Name string `json:"name,omitempty"`
Height string `json:"height,omitempty"`
Weight string `json:"weight,omitempty"`
}
//CreateVitalsignsHandler is to create vitals for a patient
func CreateVitalsignsHandler(w http.ResponseWriter, r *http.Request, ps httprouter.Params) {
//Creating the Vitalsigns
kinVitalsigns := &Vitalsigns{}
ctx := appengine.NewContext(r)
if err := json.NewDecoder(r.Body).Decode(kinVitalsigns); err != nil {
RespondErr(w, r, http.StatusInternalServerError, err.Error())
return
}
//Getting namespace
namespace := ps.ByName("namespace")
ctx, err := appengine.Namespace(ctx, namespace)
if err != nil {
log.Infof(ctx, "Namespace error from CreateVitalsignsHandler")
RespondErr(w, r, http.StatusInternalServerError, err.Error())
return
}
//Geting the patientID
pID, err := strconv.Atoi(ps.ByName("id"))
if err != nil {
log.Infof(ctx, "Srting to Int64 conversion error from CreateVitalsignsHandler")
RespondErr(w, r, http.StatusInternalServerError, err.Error())
return
}
patientID := int64(pID)
kinVitalsigns.PatientID = patientID
//Generating the key
vitalsignsKey := datastore.NewIncompleteKey(ctx, "Vitalsigns", nil)
//Inserting the data to the datastore
vk, err := datastore.Put(ctx, vitalsignsKey, kinVitalsigns)
if err != nil {
log.Infof(ctx, "Entity creation was failed from CreateVitalsignsHandler")
RespondErr(w, r, http.StatusInternalServerError, err.Error())
return
}
kinVitalsigns.ID = vk.IntID()
message := "Vitalsigns created successfully!! "
Respond(w, r, http.StatusOK, SuccessResponse{kinVitalsigns.ID, 0, "", message})
return
}
Edit: Your edit reveals you want to handle dynamic objects to put / retrieve from Google Datastore. For this it is completely unnecessary to create struct types at runtime, you may just use a dynamic map presented in this answer: How can I have dynamic properties in go on the google app engine datastore.
Original answer follows.
Note that if the types are known at compile time, best / most efficient is to create the types and compile them, so everything will be "static". You may create the types manually, or you may use go generate to automate the process.
Also note that you may not necessarily need struct types to model dynamic objects, many times maps may be sufficient.
If types are not known at compile time, and struct types are a must, read on.
Yes, it's possible to create "dynamic" struct types at runtime using Go's reflection, specifically with the reflect.StructOf() function.
Let's see a simple example, creating a struct type at runtime that has a Name string and an Age int field:
t := reflect.StructOf([]reflect.StructField{
{
Name: "Name",
Type: reflect.TypeOf(""), // string
},
{
Name: "Age",
Type: reflect.TypeOf(0), // int
},
})
fmt.Println(t)
v := reflect.New(t)
fmt.Printf("%+v\n", v)
v.Elem().FieldByName("Name").Set(reflect.ValueOf("Bob"))
v.Elem().FieldByName("Age").Set(reflect.ValueOf(12))
fmt.Printf("%+v\n", v)
This outputs (try it on the Go Playground):
struct { Name string; Age int }
&{Name: Age:0}
&{Name:Bob Age:12}
If you want to define validation rules, you may use a 3rd party lib for that, for example github.com/go-validator/validator. This package uses struct tags to specify validation rules, struct tags which you may also specify using reflection.
For example, if you want to specify that the Name must be at least 3 characters and 40 at most, and it may only contain letters of the English alphabet, and valid range for Age is 6..100 (both inclusive), this is how it would look like:
t := reflect.StructOf([]reflect.StructField{
{
Name: "Name",
Type: reflect.TypeOf(""), // string
Tag: reflect.StructTag(`validate:"min=3,max=40,regexp=^[a-zA-Z]*$"`),
},
{
Name: "Age",
Type: reflect.TypeOf(0), // int
Tag: reflect.StructTag(`validate:"min=6,max=100"`),
},
})
Printing this type would output (wrapped by me) (try it on the Go Playground):
struct { Name string "validate:\"min=3,max=40,regexp=^[a-zA-Z]*$\"";
Age int "validate:\"min=6,max=100\"" }
Once you create an instance of this struct, you can validate it using the validator.Validate() function, e.g.:
v := reflect.New(t)
v.Elem().FieldByName("Name").Set(reflect.ValueOf("Bob"))
v.Elem().FieldByName("Age").Set(reflect.ValueOf(12))
if errs := validator.Validate(v.Elem().Interface()); errs != nil {
// values not valid, deal with errors here
}

Update object in memory with values from req.Body

I am creating a simple in-memory server before doing things with a database. I have this update method:
type Nearby struct {
ID int `json:"id,omitempty"`
Me int `json:"me,omitempty"`
You int `json:"you,omitempty"`
ContactTime int64 `json:"contactTime,omitempty"`
}
func (h NearbyHandler) updateById(v NearbyInjection) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
params := mux.Vars(r)
decoder := json.NewDecoder(r.Body)
var t Nearby
err := decoder.Decode(&t)
if err != nil {
panic(err)
}
mtx.Lock()
item, ok := v.Nearby[params["id"]]
mtx.Unlock()
if !ok {
panic(errors.New("No item to update"))
}
// !! Ok, how can I overwrite the properties from t onto item
if ok {
json.NewEncoder(w).Encode(item)
} else {
io.WriteString(w, "null")
}
}
}
I am looking to take the key/values from t, and write them onto the item object. t gets decoded into a struct value (I assume), and item is a struct value found in a map. Both item and t have the same type (Nearby)
In JavaScript, all I am looking to do would be:
Object.assign(item, t);
Just trying to accomplish something similar with Go.
With Golang, I can do this:
item.ContactTime = t.ContactTime
but I only want to overwrite item.ContactTime if t.ContactTime is defined.
Just overwrite the item in your map:
v.Nearby[params["id"]] = t
I'd suggest to use https://github.com/imdario/mergo function Merge. I don't think there is any reason to reinvent the wheel in this one and go's lack of generics does not help with such operations. example:
src := Foo{
A: "one",
B: 2,
}
dest := Foo{
A: "two",
}
mergo.Merge(&dest, src)
fmt.Println(dest)
// Will print
// {two 2}
I also think you should make all Nearby's properties pointers so that you can always compare them against nil to make sure they were not set.

How to pass multiple data to Go template?

I want to pass two data objects to Go Template. One is a MongoDB query result and other is an integer array.
MongoDB Query:-
var results []User
sess, db := GetDatabase()
defer sess.Close()
c := db.C("user")
err := c.Find(nil).All(&results)
I want to sent 'result' and an int array through following code
GetTemplate("list").Execute(w,???????)
If there is only db result, we could use it as
GetTemplate("list").Execute(w,results)
and in template we could access it {{.Name}} etc. (where Name is a struct field of []User)
Please tell me how to pass these data and how to access them in template.
You may only pass a single value, but that value may be a composed value of multiple values, e.g. a struct, map or slice. So simply wrap your multiple data intended for the template in a struct or in a map.
Example with a struct:
type Data struct {
Results []User // Must be exported!
Other []int // Must be exported!
}
data := &Data{results, []int{1, 2, 3}}
if err := GetTemplate("list").Execute(w, data); err != nil {
// Handle error
}
Also note that a new, named type is not required, you could also use an anonymous struct literal, which could look like this:
data := struct {
Results []User // Must be exported!
Other []int // Must be exported!
}{results, []int{1, 2, 3}}
Example with a map:
m := map[string]interface{}{
"Results": results,
"Other": []int{1, 2, 3},
}
if err := GetTemplate("list").Execute(w, m); err != nil {
// Handle error
}
Note that using a map, it is not required to use capitalized strings as keys, e.g. you could've used "results" and "other" too (but in my opinion it's better to use keys with capital starting letters, should you move to struct sometimes in the future, you would have less corrections to make).
In both cases you may refer to the []User results with {{.Results}} and to the additional int slice with {{.Other}}.
So for example to range over the users:
{{range .Results}}
User name:{{.Name}}
{{end}}
Example with a slice:
s := []interface{}{
results,
[]int{1, 2, 3},
}
if err := GetTemplate("list").Execute(w, s); err != nil {
// Handle error
}
This is less readable, but a viable solution. In the template you have to index the template data to get the "individual" values, e.g.:
{{range index . 0}}
User name:{{.Name}}
{{end}}
Other: {{index . 1}}
Try it on the Go Playground.
Other ways...
There are other "theoretical" ways too, but I wouldn't use them just because it works.
For example, you could also pass in a channel from which receives would provide the values.
Yet another solution could be to register custom functions which when called would return the values.
You should define a struct populated with the database results query, then assign that struct to the Execute method.
tmpl.Execute require a Writer interface and a struct
type Inventory struct {
Material string
Count uint
}
items := Inventory{"trouser", 1}
if err := GetTemplate("list").Execute(w, items); err != nil {
// ... do your work
}
You can put your structs inside another struct using the range keyword. e.g
type Data struct {
Users []User
Developers []Developer
}
var data = Data{
Users: ...,
Developers: ...,
}
err := t.Execute(w, &data)
if err != nil {
panic(err)
}
{{range .Users}}{{.UserName}}
{{end}}
{{range .Developers}}{{.DeveloperName}}
{{end}}

Should I be able to type assert a slice of string maps?

I am receiving a message using the Go NSQ library where a field is a slice of map[string]string's. I feel like I should be able to type assert this field as value.([]map[string]string) but it's failing and I can't tell if this is expected or not.
This snippet replicates the behavior https://play.golang.org/p/qcZM880Nal
Why does this type assertion fail?
This is covered briefly here in the FAQ.
The types []interface{} and []map[string]string have two different representation in memory. There is no direct way to convert between them.
Also, even when a conversion is allowed, you should note that you can't successfully assert to a different basic type at all (http://play.golang.org/p/zMp1qebIZZ). You can only assert to the original type, or another type of interface,
// panics
var i interface{} = int32(42)
_ = i.(int64)
The conversion referred to doesn't work as described in Jim's answer. However, if you actually have the type you claim, and the interface you state it implements is just interface{} then the type assertion works fine. I don't want to speculate on the details of why the other doesn't work but I believe it's because you would have to unbox it in two phases as the map[string]string's inside the slice are actually being viewed as some interface{} themselves. Here's an example;
package main
import "fmt"
func main() {
var value interface{}
value = []map[string]string{{"address": string("this is interface literal")}}
// value = []map[string]string{{"address": "this is map literal"}}
AssertIt(value)
}
func AssertIt(value interface{}) {
if str, ok := value.([]map[string]string); ok && len(str) > 0 {
fmt.Println(str[0]["address"])
} else {
fmt.Println("nope")
}
}
https://play.golang.org/p/hJfoh_havC
you can do it by reflect in 2022
res := `[
{
"name": "jhon",
"age": 35
},
{
"name": "wang",
"age": 30
}
]`
// res := `{
// "name": "jhon",
// "age": 35
// }`
var rst any
err := json.Unmarshal([]byte(res), &rst)
if err != nil {
panic(err)
}
t := reflect.TypeOf(rst).Kind()
fmt.Println(t)

Resources