Nil pointer evaluation in template...why? Is there a better strategy? - go

I am attempting to wrap html/template so I am guaranteed to have certain data in my templates (session data for example) in addition to the data I want to render. However, my current approach is...flawed. Here's a simplified example below:
package main
import "fmt"
import "os"
import "html/template"
func main() {
// Passing nil directly to Execute doesn't render anything for missing struct fields
fmt.Print("Directly rendering nil\n")
tmpl, err := template.New("master").Parse("Foo is: {{.Data.Foo}}")
if err != nil {
fmt.Printf(err.Error())
return
}
err = tmpl.Execute(os.Stdout, nil)
if err != nil {
fmt.Printf(err.Error())
return
}
// Wrapping templates works as long as I supply data...
fmt.Print("\nRendering Foo\n")
render(struct {
Foo string
}{
"foo",
})
// ...but this breaks.
fmt.Print("\nRendering nil\n")
render(nil)
}
func render(data interface{}) {
allData := struct {
Session string
Data interface{}
}{
"sessionData",
data,
}
// Hardcoded template for the example - this could be any arbitrary template
tmpl, err := template.New("master").Parse("Foo is: {{.Data.Foo}}")
if err != nil {
fmt.Printf(err.Error())
return
}
err = tmpl.Execute(os.Stdout, allData)
if err != nil {
fmt.Printf(err.Error())
return
}
}
I get the following output:
Directly rendering nil
Foo is:
Rendering Foo
Foo is: foo
Rendering nil
Foo is: template: master:1:15: executing "master" at <.Data.Foo>: nil pointer evaluating interface {}.Foo
So I'm not quite sure what's going on in the first place - why is it that html/template is capable of handling being passed nil, but can't figure out what to do with a nil pointer?
Secondly, is there a better way of approaching this problem?

Your best bet is to always make Data a map or struct, either by making the type a map or struct, or by not using a nil with interface{}:
package main
import "fmt"
import "os"
import "text/template"
func main() {
tmpl, err := template.New("master").Parse("{{if .Data.Foo}}Foo is: {{.Data.Foo}}{{else}}Foo is empty{{end}}")
if err != nil {
fmt.Printf(err.Error())
return
}
err = tmpl.Execute(os.Stdout, nil)
if err != nil {
fmt.Printf(err.Error())
return
}
fmt.Println("")
err = tmpl.Execute(os.Stdout, struct {
Session string
Data map[string]string
}{
"sessionData",
nil,
})
if err != nil {
fmt.Printf(err.Error())
return
}
fmt.Println("")
err = tmpl.Execute(os.Stdout, struct {
Session string
Data interface{}
}{
"sessionData",
map[string]string{},
})
if err != nil {
fmt.Printf(err.Error())
return
}
fmt.Println("")
}
Play: http://play.golang.org/p/9GkAp6ysvD
As for why it works like this, it's a bit complicated, you have to look at the code: https://golang.org/src/text/template/exec.go?s=4647:4717#L521
When execute is called with nil, reflect.ValueOf(nil) returns an invalid Value, so evalField returns the zero value, and you end up with an empty string.
However, when execute is called with a valid struct, that first reflect.ValueOf returns a valid value. The .Data command calls evalField on the whole struct you passed to Execute, and evalField calls FieldByIndex/FieldByName to get the "Data" field. This doesn't return an invalid Value.
Next, when .Foo is evaluated, if Data is an interface or a pointer, the indirect function follows it to the end, and if it finds that it's nil, it fails with this error.
When Data is a map, the indirect function doesn't do anything, and it doesn't fail.
This might be a bug in the text/template package.

Related

How to test nested input in Go

I have function for getting user input using os.Stdin
func (i input) GetInput(stdin io.Reader) (string, error) {
reader := bufio.NewReader(stdin)
data, err := reader.ReadString('\n')
if err != nil {
return "", fmt.Errorf("get input error: %w", err)
}
return strings.ReplaceAll(data, "\n", ""), nil
}
In my programm I need to have 2 inputs:
First for getting base info for example user name
Second for getting aditional info that depends on first input
name,err := GetInput(os.Stdin)
if err != nil {
// error handling.....
}
switch name {
case "test":
//do something...
age, err := GetInput(os.Stdin)
if err != nil {
// error handling.....
}
fmt.Println(age)
case "another":
// Here another input
}
It it possible to write unit tests for that case?
For testing one user input I use this snippet and it works:
var stdin bytes.Buffer
stdin.Write([]byte(fmt.Sprintf("%s\n", tt.input)))
GetInput(stdin)
But it didn't work with 2 nested inputs
Maybe consider having a function that returns a specific type as a result and put it into a separate package.
Since I see name and age mentioned, perhaps we can assume a concrete type like Person for illustration.
It is important to note that we want to include the actual reader as a parameter and not have a hard coded reference to os.Stdin. This makes the mocking of nested inputs possible in the first place.
With this, the signature of the method could look something like the following:
func NestedInput(input io.Reader) (*Person, error)
The corresponding type could be:
type Person struct {
Name string
Age int
}
If one now combines your code snippets to a complete GO file with the name input.go in a separate directory, it might look something like this:
package input
import (
"bufio"
"fmt"
"io"
"strconv"
"strings"
)
func getInput(reader *bufio.Reader) (string, error) {
data, err := reader.ReadString('\n')
if err != nil {
return "", fmt.Errorf("get input error: %w", err)
}
return strings.ReplaceAll(data, "\n", ""), nil
}
type Person struct {
Name string
Age int
}
func NestedInput(input io.Reader) (*Person, error) {
reader := bufio.NewReader(input)
name, err := getInput(reader)
if err != nil {
return nil, err
}
switch name {
case "q":
return nil, nil
default:
ageStr, err := getInput(reader)
if err != nil {
return nil, err
}
age, err := strconv.Atoi(ageStr)
if err != nil {
return nil, err
}
return &Person{Name: name, Age: age}, nil
}
}
An input of q returns nil, nil and could be used to terminate the input, e.g. if the query was made in a loop.
Unit Test
The unit test
func Test_nestedInput(t *testing.T)
in a file named input_test.go should now provide the input data.
Since the NestedInput function now expects an io.Reader as a parameter, we can simply generate the desired input with, for example,
input := strings.NewReader("George\n26\n")
So the test could look something like this:
package input
import (
"strings"
"testing"
)
func Test_nestedInput(t *testing.T) {
input := strings.NewReader("George\n26\n")
person, err := NestedInput(input)
if err != nil {
t.Error("nested input failed")
}
if person == nil {
t.Errorf("expected person, but got nil")
return
}
if person.Name != "George" {
t.Errorf("wrong name %s, expected 'George'", person.Name)
}
if person.Age != 26 {
t.Errorf("wrong age %d, expected 26", person.Age)
}
}
Of course, the tests can be extended with further details. But this, as you can see, mocks a nested input.

How can I return two different concrete types from a single method in Go 1.18?

Let say that I have this code:
type Type1 struct {
Name string `json:"name,omitempty"`
Path string `json:"path"`
File string `json:"file"`
Tag int `json:"tag"`
Num int `json:"num"`
}
func LoadConfiguration(data []byte) (*Type1, error) {
config, err := loadConf1(data)
if err != nil {
return nil, err
}
confOther, err := loadConfOther1()
if err != nil {
return nil, err
}
// do something with confOther
fmt.Println("confOther", confOther)
if confOther.Tag == 0 {
config.Num = 5
}
// do something with config attributes of type1
if config.Tag == 0 {
config.Tag = 5
}
if config.Num == 0 {
config.Num = 4
}
return config, nil
}
func loadConf1(bytes []byte) (*Type1, error) {
config := &Type1{}
if err := json.Unmarshal(bytes, config); err != nil {
return nil, fmt.Errorf("cannot load config: %v", err)
}
return config, nil
}
func loadConfOther1() (*Type1, error) {
// return value of this specific type
flatconfig := &Type1{}
// read a file as []byte
// written as a fixed array to simplify this example
fileContent := []byte{10, 22, 33, 44, 55}
if err := json.Unmarshal(fileContent, flatconfig); err != nil {
return nil, fmt.Errorf("cannot read config %v", err)
}
return flatconfig, nil
}
The only public function is LoadConfiguration.
It's based on a real code and It's used to read a json data as a specific struct. If something seems useless, it's because I simplified the original code.
The code above is ok, but now I want to create another struct type called "Type2" and re-use the same methods to read data into Type2 without copying and pasting everything.
type Type2 struct {
Name string `json:"name,omitempty"`
Path string `json:"path"`
Map *map[string]interface{} `json:"map"`
Other string `json:"other"`
}
Basically, I want to be able to call LoadConfiguration to get also Type2. I can accept to call a specific method like LoadConfiguration2, but I don't want to copy and paste also loadConf1 and loadConfOther1.
Is there a way to do that in an idiomatic way in Go 1.18?
Actually the code shown in your question doesn't do anything more than passing a type into json.Unmarshal and format an error so you can rewrite your function to behave just like it:
func LoadConfiguration(data []byte) (*Type1, error) {
config := &Type1{}
if err := loadConf(data, config); err != nil {
return nil, err
}
// ...
}
// "magically" accepts any type
// you could actually get rid of the intermediate function altogether
func loadConf(bytes []byte, config any) error {
if err := json.Unmarshal(bytes, config); err != nil {
return fmt.Errorf("cannot load config: %v", err)
}
return nil
}
In case the code actually does something more than just passing a pointer into json.Unmarshal, it can benefit from type parameters.
type Configurations interface {
Type1 | Type2
}
func loadConf[T Configurations](bytes []byte) (*T, error) {
config := new(T)
if err := json.Unmarshal(bytes, config); err != nil {
return nil, fmt.Errorf("cannot load config: %v", err)
}
return config, nil
}
func loadConfOther[T Configurations]() (*T, error) {
flatconfig := new(T)
// ... code
return flatconfig, nil
}
In these cases you can create a new pointer of either type with new(T) and then json.Unmarshal will take care of deserializing the content of the byte slice or file into it — provided the JSON can be actually unmarshalled into either struct.
The type-specific code in the top-level function should still be different, especially because you want to instantiate the generic functions with an explicit concrete type. So I advise to keep LoadConfiguration1 and LoadConfiguration2.
func LoadConfiguration1(data []byte) (*Type1, error) {
config, err := loadConf[Type1](data)
if err != nil {
return nil, err
}
confOther, err := loadConfOther[Type1]()
if err != nil {
return nil, err
}
// ... type specific code
return config, nil
}
However if the type-specific code is a small part of it, you can probably get away with a type-switch for the specific part, though it doesn't seem a viable option in your case. I would look like:
func LoadConfiguration[T Configuration](data []byte) (*T, error) {
config, err := loadConf[T](data)
if err != nil {
return nil, err
}
// let's pretend there's only one value of type parameter type
// type-specific code
switch t := config.(type) {
case *Type1:
// ... some *Type1 specific code
case *Type2:
// ... some *Type2 specific code
default:
// can't really happen because T is restricted to Configuration but helps catch errors if you extend the union and forget to add a corresponding case
panic("invalid type")
}
return config, nil
}
Minimal example playground: https://go.dev/play/p/-rhIgoxINTZ

Golang pongo2 filter returns incorrect value

I created a pongo2 filter which should return Absolute value of the parameter passed in to the filter. However, the filter only returns correct answer if I pass the value as a parameter. But if I pass the value directly it returns incorrect value.
My question is why does it not return correct result if I pass the value directly instead of as a parameter?
package main
import (
"fmt"
"math"
"github.com/flosch/pongo2"
"log"
)
func init() {
pongo2.RegisterFilter("abs", func(in *pongo2.Value, param *pongo2.Value) (*pongo2.Value, *pongo2.Error) {
if in.IsString() {
return pongo2.AsValue(math.Abs(in.Float())), nil
}
return pongo2.AsValue(math.Abs(in.Float())), nil
})
}
func correctWay() {
tpl, err := pongo2.FromString("Hello {{ val|abs }}!")
if err != nil {
log.Fatal(err)
}
// Now you can render the template with the given
// pongo2.Context how often you want to.
out, err := tpl.Execute(pongo2.Context{"val": -5})
if err != nil {
log.Fatal(err)
}
fmt.Println(out)
}
func incorrectWay() {
tpl, err := pongo2.FromString("Hello {{ -5|abs }}!")
if err != nil {
log.Fatal(err)
}
// Now you can render the template with the given
// pongo2.Context how often you want to.
out, err := tpl.Execute(pongo2.Context{})
if err != nil {
log.Fatal(err)
}
fmt.Println(out)
}
func main() {
correctWay()
incorrectWay()
}
I think the easiest option is to pass it as a string.
// This works:
tpl, err := pongo2.FromString("Hello {{ '-5'|abs }}!")
// prints Hello 5.000000!
I did a bit of experimenting, originally I thought it's because of how Go handles the minus sign in templates, but I don't think this is the case. I can't work out what pongo is doing, I am fairly certain if you pass in -5 only the 5 is passed to the function you register (add fmt.Println(in.Float()) to the closure), which then is negated when it's outputted by Go. This could be a bug 🤷‍♂️. Either way, it works with a string.

Receiving an empty map with no errors when trying to unmarshal a string read from a file (Answer: unmarshal into the data structure itself)

I have a file, 'test.txt', containing the following data. This file was created from the same structures from the code below using marshaling.
{"VLETXGJM":{"attrib1":"test1","attrib2":"test2"}}
I am trying to read it back from the file and unmarshal it into a map using the same structures. I can successfully read the data from the file. I receive no errors when I try to unmarshal it into the map. However, my map is empty.
The mutex is used to protect the map since my real implementation (this is an extracted test) needs to use a protected map for concurrency. I have tried this same code removing the sync library and received the same negative result.
The test code:
package main
import (
"encoding/json"
"fmt"
"sync"
"os"
)
type TestObject struct {
Attrib1 string `json:"attrib1"`
Attrib2 string `json:"attrib2"`
}
type TestMap map[string]TestObject
type TestList struct {
sync.RWMutex
list TestMap
}
func main() {
tl := TestList{ list: make(TestMap) }
// Read the list back out of the file
fi, err := os.Open("test.txt")
if os.IsNotExist(err) {
fmt.Println("data file does not exist")
panic(nil)
}
if err != nil {
panic(err)
}
defer func() {
if err := fi.Close(); err != nil {
panic(err)
}
}()
data := make([]byte, 1024 * 1024)
count, err := fi.Read(data)
if err != nil {
panic(err)
}
fmt.Printf("read from file: \"%s\"\n",data[:count])
tl.Lock()
err = json.Unmarshal(data[:count], &tl)
if err != nil {
panic(err)
}
tl.Unlock()
// List it out
tl.Lock()
if len(tl.list) == 0 {
fmt.Println("Empty list")
} else {
for key, _ := range tl.list {
fmt.Printf("%s: %s\n", tl.list[key].Attrib1, tl.list[key].Attrib2)
}
}
tl.Unlock()
}
The output of the run is:
read from file: "{"VLETXGJM":{"attrib1":"test1","attrib2":"test2"}}"
Empty list
Thank you for your help. I have searched for similar issues and not yet found an exact duplicate of this scenario.
I think you want to unmarshal into tl.list instead of tl:
err = json.Unmarshal(data[:count], &tl.list)
tl has no exported fields, so Unmarshal into tl won't do anything. tl.list (i.e., type TestMap) matches your data.

Converting struct with embedded interface into JSON

I have a struct that I want to marshal to JSON. It has a defined field called Foo (exported as foo) and a data interface field to which I want to pass a dynamic struct with additional JSON fields.
However when the data field is an interface instead of the specific struct it never gets exported as JSON. How can I make this work?
package main
import (
"encoding/json"
"fmt"
)
type data interface{}
type foo struct {
Foo string `json:"foo,omitempty"`
data
}
type bar struct {
Bar string `json:"bar,omitempty"`
}
func main() {
b := bar{"bar"}
f := foo{"foo", b}
byt, err := json.Marshal(f)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(string(byt))
}
I need to output to look like this (it needs to be flat, not nested):
{"foo": "foo", "bar": "bar"}
You could do this with a custom json.Marshaler implementation and a little bit of byte slicing.
func (f foo) MarshalJSON() ([]byte, error) {
type goo foo
g := goo(f)
b1, err := json.Marshal(g)
if err != nil {
return nil, err
}
b2, err := json.Marshal(g.data)
if err != nil {
return nil, err
}
s1 := string(b1[:len(b1)-1])
s2 := string(b2[1:])
return []byte(s1 + ", " + s2), nil
}
https://play.golang.org/p/NYTNWIL-xu
Please note that this is not checking whether the bytes can actually be sliced and it does also not consider the possible case of the data field being a slice or an array, which i'm unsure how you would want that flattened anyway.
I would write a custom marshaller, like so:
func (f foo) MarshalJSON() ([]byte, error) {
type tmp foo
g := tmp(f)
first, err := json.Marshal(g)
if err != nil {
return nil, err
}
second, err := json.Marshal(f.data)
if err != nil {
return nil, err
}
data := make(map[string]interface{})
json.Unmarshal(first, &data)
json.Unmarshal(second, &data)
return json.Marshal(data)
//{"bar":"bar","foo":"foo"}
}
https://play.golang.org/p/TENiCe9nR0
2 options:
Set to it type json.RawMessage so it won’t be decoded automatically and left as an interface. https://golang.org/pkg/encoding/json/#RawMessage
Write custom unmarshaler on the structure.

Resources