Simplify substituting variables in string with Golang [duplicate] - go

In Python, you can do this:
"File {file} had error {error}".format(file=myfile, error=err)
or this:
"File %(file)s had error %(error)s" % {"file": myfile, "error": err}
In Go, the simplest option is:
fmt.Sprintf("File %s had error %s", myfile, err)
which doesn't let you swap the order of the parameters in the format string, which you need to do for I18N. Go does have the template package, which would require something like:
package main
import (
"bytes"
"text/template"
"os"
)
func main() {
type Params struct {
File string
Error string
}
var msg bytes.Buffer
params := &Params{
File: "abc",
Error: "def",
}
tmpl, _ := template.New("errmsg").Parse("File {{.File}} has error {{.Error}}")
tmpl.Execute(&msg, params)
msg.WriteTo(os.Stdout)
}
which seems like a long way to go for an error message. Is there a more reasonable option that allows me to give string parameters independent of order?

With strings.Replacer
Using strings.Replacer, implementing a formatter of your desire is very easy and compact.
func main() {
file, err := "/data/test.txt", "file not found"
log("File {file} had error {error}", "{file}", file, "{error}", err)
}
func log(format string, args ...string) {
r := strings.NewReplacer(args...)
fmt.Println(r.Replace(format))
}
Output (try it on the Go Playground):
File /data/test.txt had error file not found
We can make it more pleasant to use by adding the brackets to the parameter names automatically in the log() function:
func main() {
file, err := "/data/test.txt", "file not found"
log2("File {file} had error {error}", "file", file, "error", err)
}
func log2(format string, args ...string) {
for i, v := range args {
if i%2 == 0 {
args[i] = "{" + v + "}"
}
}
r := strings.NewReplacer(args...)
fmt.Println(r.Replace(format))
}
Output (try it on the Go Playground):
File /data/test.txt had error file not found
Yes, you could say that this only accepts string parameter values. This is true. With a little more improvement, this won't be true:
func main() {
file, err := "/data/test.txt", 666
log3("File {file} had error {error}", "file", file, "error", err)
}
func log3(format string, args ...interface{}) {
args2 := make([]string, len(args))
for i, v := range args {
if i%2 == 0 {
args2[i] = fmt.Sprintf("{%v}", v)
} else {
args2[i] = fmt.Sprint(v)
}
}
r := strings.NewReplacer(args2...)
fmt.Println(r.Replace(format))
}
Output (try it on the Go Playground):
File /data/test.txt had error 666
A variant of this to accept params as a map[string]interface{} and return the result as a string:
type P map[string]interface{}
func main() {
file, err := "/data/test.txt", 666
s := log33("File {file} had error {error}", P{"file": file, "error": err})
fmt.Println(s)
}
func log33(format string, p P) string {
args, i := make([]string, len(p)*2), 0
for k, v := range p {
args[i] = "{" + k + "}"
args[i+1] = fmt.Sprint(v)
i += 2
}
return strings.NewReplacer(args...).Replace(format)
}
Try it on the Go Playground.
With text/template
Your template solution or proposal is also way too verbose. It can be written as compact as this (error checks omitted):
type P map[string]interface{}
func main() {
file, err := "/data/test.txt", 666
log4("File {{.file}} has error {{.error}}", P{"file": file, "error": err})
}
func log4(format string, p P) {
t := template.Must(template.New("").Parse(format))
t.Execute(os.Stdout, p)
}
Output (try it on the Go Playground):
File /data/test.txt has error 666
If you want to return the string (instead of printing it to the standard output), you may do it like this (try it on the Go Playground):
func log5(format string, p P) string {
b := &bytes.Buffer{}
template.Must(template.New("").Parse(format)).Execute(b, p)
return b.String()
}
Using explicit argument indices
This was already mentioned in another answer, but to complete it, know that the same explicit argument index may be used arbitrary number of times and thus resulting in the same parameter substituted in multiple times. Read more about this in this question: Replace all variables in Sprintf with same variable

I don't know of any easy way of naming the parameters, but you can easily change the order of the arguments, using explicit argument indexes:
From docs:
In Printf, Sprintf, and Fprintf, the default behavior is for each formatting verb to format successive arguments passed in the call. However, the notation [n] immediately before the verb indicates that the nth one-indexed argument is to be formatted instead. The same notation before a '*' for a width or precision selects the argument index holding the value. After processing a bracketed expression [n], subsequent verbs will use arguments n+1, n+2, etc. unless otherwise directed.
Then you can, ie:
fmt.Printf("File %[2]s had error %[1]s", err, myfile)

The parameter can also be a map, so the following function would work if you don't mind parsing every error format every time you use it:
package main
import (
"bytes"
"text/template"
"fmt"
)
func msg(fmt string, args map[string]interface{}) (str string) {
var msg bytes.Buffer
tmpl, err := template.New("errmsg").Parse(fmt)
if err != nil {
return fmt
}
tmpl.Execute(&msg, args)
return msg.String()
}
func main() {
fmt.Printf(msg("File {{.File}} has error {{.Error}}\n", map[string]interface{} {
"File": "abc",
"Error": "def",
}))
}
It's still a little wordier than I would have liked, but it's better than some other options, I suppose. You could turn map[string]interface{} into a local type and reduce it further to:
type P map[string]interface{}
fmt.Printf(msg("File {{.File}} has error {{.Error}}\n", P{
"File": "abc",
"Error": "def",
}))

Alas, there's no built-in function in Go for string interpolation with named parameters (yet). But you are not the only one suffering out there :) Some packages should exist, for example: https://github.com/imkira/go-interpol . Or, if feeling adventurous, you could write such a helper yourself, as the concept is actually quite simple.
Cheers,
Dennis

You can try the Go Formatter library that implements replacement fields surrounded by curly braces {} format strings similar to Python format.
Working code example Go Playground:
package main
import (
"fmt"
"gitlab.com/tymonx/go-formatter/formatter"
)
func main() {
formatted, err := formatter.Format("Named placeholders {file}:{line}:{function}():", formatter.Named{
"line": 3,
"function": "func1",
"file": "dir/file",
})
if err != nil {
panic(err)
}
fmt.Println(formatted)
}
Output:
Named placeholders dir/file:3:func1():

Instead of using template.New, where you have to provide a template name, you
can just instantiate a template pointer:
package main
import (
"strings"
"text/template"
)
func format(s string, v interface{}) string {
t, b := new(template.Template), new(strings.Builder)
template.Must(t.Parse(s)).Execute(b, v)
return b.String()
}
func main() {
params := struct{File, Error string}{"abc", "def"}
println(format("File {{.File}} has error {{.Error}}", params))
}

Use os.Expand to replace fields in a format string. Expand replaces ${var} or $var in the string using a func(string) string mapping function.
Here are a couple of ways to wrap os.Expand in convenient to use functions:
func expandMap(s string, m map[string]string) string {
return os.Expand(s, func(k string) string { return m[k] })
}
func expandArgs(s string, kvs ...string) string {
return os.Expand(s, func(k string) string {
for i := 1; i < len(kvs); i++ {
if kvs[i-1] == k {
return kvs[i]
}
}
return ""
})
}
Example use:
s = expandMap("File ${file} had error ${error}",
map[string]string{"file": "myfile.txt", "error": "Not found"})
s = expandArgs("File ${file} had error ${error}",
"file", "myfile.txt", "error", "Not found"))
Run the code on the playground.

You can get quite close to that sweet python formatting experience:
message := FormatString("File {file} had error {error}", Items{"file"=myfile, "error"=err})
Declare the following somewhere in your code:
type Items map[string]interface{}
func FormatString(template string, items Items) string {
for key, value := range items {
template = strings.ReplaceAll(template, fmt.Sprintf("{%v}", key), fmt.Sprintf("%v", value))
}
return template
}
💡 note that my implementation is very naive and inefficient for high-performance needs
sudo make me a package
Seeing the development experience potential with having a simple signature like this, I've got tempted and uploaded a go package called format.
package main
import (
"fmt"
"github.com/jossef/format"
)
func main() {
formattedString := format.String(`hello "{name}". is lizard? {isLizard}`, format.Items{"name": "Mr Dude", "isLizard": false})
fmt.Println(formattedString)
}
https://repl.it/#jossef/format

text/template is interesting. I Provide some example below
Usage
func TestFString(t *testing.T) {
// Example 1
fs := &FString{}
fs.MustCompile(`Name: {{.Name}} Msg: {{.Msg}}`, nil)
fs.MustRender(map[string]interface{}{
"Name": "Carson",
"Msg": 123,
})
assert.Equal(t, "Name: Carson Msg: 123", fs.Data)
fs.Clear()
// Example 2 (with FuncMap)
funcMap := template.FuncMap{
"largest": func(slice []float32) float32 {
if len(slice) == 0 {
panic(errors.New("empty slice"))
}
max := slice[0]
for _, val := range slice[1:] {
if val > max {
max = val
}
}
return max
},
"sayHello": func() string {
return "Hello"
},
}
fs.MustCompile("{{- if gt .Age 80 -}} Old {{else}} Young {{- end -}}"+ // "-" is for remove empty space
"{{ sayHello }} {{largest .Numbers}}", // Use the function which you created.
funcMap)
fs.MustRender(Context{
"Age": 90,
"Numbers": []float32{3, 9, 13.9, 2.1, 7},
})
assert.Equal(t, "Old Hello 13.9", fs.Data)
}
Lib
package utils
import (
"text/template"
)
type Context map[string]interface{}
type FString struct {
Data string
template *template.Template
}
func (fs *FString) MustCompile(expr string, funcMap template.FuncMap) {
fs.template = template.Must(template.New("f-string").
Funcs(funcMap).
Parse(expr))
}
func (fs *FString) Write(b []byte) (n int, err error) {
fs.Data += string(b)
return len(b), nil
}
func (fs *FString) Render(context map[string]interface{}) error {
if err := fs.template.Execute(fs, context); err != nil {
return err
}
return nil
}
func (fs *FString) MustRender(context Context) {
if err := fs.Render(context); err != nil {
panic(err)
}
}
func (fs *FString) Clear() string {
// return the data and clear it
out := fs.Data
fs.Data = ""
return out
}
important document
https://golang.org/pkg/text/template/#hdr-Actions

Here is a function I wrote which replaces fields with strings in a map, similar to what you can do with Python. It takes a string which should have fields that look like ${field} and replaces them with any such keys in the given map like map['field']='value':
func replaceMap(s string,m *map[string]string) string {
r := regexp.MustCompile("\\${[^}]*}")
for x,i := range *m {
s = strings.Replace(s,"${"+x+"}",i,-1)
}
// Remove missing parameters
s = r.ReplaceAllString(s,"")
return s
}
Playground example:
https://go.dev/play/p/S5rF5KLooWq

Related

How to print a function that is passed as parameter to a function

I know how to do in javascript, it will be something similar to:
The Go function will receive a function as parameter, I wanna get function as string to build a map that then I'll save in some database.
package main
import (
"fmt"
)
func describe(i interface{}) string {
return fmt.Sprintf("%v", i)
}
func dummyReducer(int) int {
return 1
}
func Accumulator(reducer func(int, int) int, init int) func(int) int {
input := map[string]interface{}{
"func_name": "accumulatorLoger",
"func_data": map[string]interface{}{
"reducer": string(describe(reducer)),
"init": init,
},
}
// {
// func_data: { init: 10, reducer: '0x64b880' },
// func_name: 'accumulatorLoger'
// }
// TODO: next: save the input data in the database
fmt.Println(input)
return dummyReducer
}
If you want the body you need the source. That means that your program will need access to the Go file in which the function you want the body of was declared.
To get a function's file, you can use (*runtime.Func).FileLine. And you'll also need the name of the function later, so use the runtime information to get that too:
func getFuncInfo(f interface{}) (name, file string) {
pc := reflect.ValueOf(f).Pointer()
fn := runtime.FuncForPC(pc)
file, _ = fn.FileLine(pc)
return fn.Name(), file
}
The name of the function may be package-qualified, if so you should clean it up:
if i := strings.LastIndexByte(name, '.'); i >= 0 {
name = name[i+1:]
}
https://play.golang.org/p/63zwvOh1qzE
Once you have the file, and your program has access to it, you can parse it with go/parser.ParseFile, retrieve the function's AST, and then print the body with go/printer.Fprint:
func getFuncAST(funcname, filename string) (*ast.FuncDecl, *token.FileSet) {
fs := token.NewFileSet()
file, err := parser.ParseFile(fs, filename, "", 0)
if err != nil {
panic(err)
}
for _, d := range file.Decls {
if f, ok := d.(*ast.FuncDecl); ok && f.Name.Name == funcname {
return f, fs
}
}
panic("function not found")
}
func getFuncBodyString(f *ast.FuncDecl, fs *token.FileSet) string {
var buf bytes.Buffer
if err := printer.Fprint(&buf, fs, f.Body); err != nil {
panic(err)
}
return buf.String()
}
https://play.golang.org/p/QDMSMhwrf39

How to omit empty json fields using json.decoder

I try to understand why both functions return the same output.
As far as I understood, the point of omit empty is to not add that key to the result struct.
I wrote this example, I was expecting the first output not to have the "Empty" key, but for some reason its value still shows as 0.
package main
import (
"encoding/json"
"fmt"
"strings"
)
type agentOmitEmpty struct {
Alias string `json:"Alias,omitempty"`
Skilled bool `json:"Skilled,omitempty"`
FinID int32 `json:"FinId,omitempty"`
Empty int `json:"Empty,omitempty"`
}
type agent struct {
Alias string `json:"Alias"`
Skilled bool `json:"Skilled"`
FinID int32 `json:"FinId"`
Empty int `json:"Empty"`
}
func main() {
jsonString := `{
"Alias":"Robert",
"Skilled":true,
"FinId":12345
}`
fmt.Printf("output with omit emtpy: %v\n", withEmpty(strings.NewReader(jsonString)))
// output with omit emtpy: {Robert true 12345 0}
fmt.Printf("output regular: %v\n", withoutEmpty(strings.NewReader(jsonString)))
// output without omit: {Robert true 12345 0}
}
func withEmpty(r *strings.Reader) agentOmitEmpty {
dec := json.NewDecoder(r)
body := agentOmitEmpty{}
err := dec.Decode(&body)
if err != nil {
panic(err)
}
return body
}
func withoutEmpty(r *strings.Reader) agent {
dec := json.NewDecoder(r)
body := agent{}
err := dec.Decode(&body)
if err != nil {
panic(err)
}
return body
}
You need to define Empty as *int so it will be replaced with nil when there is no value. Then it will not be saved in the database.

Can a variable be used as a placeholder for a function call?

I am writing a program which has several structs and functions to handle these structs differently. I am having a generic function which calls the required function based on the inputs. Is there a generic way to use the returned value from getStruct()?
package main
var X func(s []string) A
var Y func(s []string) B
type A struct {
Name string
Place string
}
type B struct {
Name string
Place string
Value string
}
func newA(s []string) A {
a := A{
Name: s[0],
Place: s[1],
}
return a
}
func newB(s []string) B {
a := B{
Name: s[0],
Place: s[1],
Value: s[2],
}
return a
}
func getStruct(t string) interface{} {
switch {
case t == "A":
return X
case t == "B":
return Y
default:
return //someStruct
}
}
func main() {
buildNewStruct := getStruct("A") //Lets assume "A" here is got as an argument
var strSlice = []string{"Bob", "US"}
buildNewStruct(strSlice) //How to do this operation?
//I am hoping to use buildNewStruct(strSlice) to dynamically call
//either of newA(strSlice) or newB(strSlice) function
}
I have tried looking at this and this the later is not exactly the same as my question.
Since I am new to go, I am not sure if something like this is possible.
you can use the reflect package to set the struct properties to the equivalent index positioned value from an []interface{} slice.
package main
import (
"fmt"
"log"
"reflect"
)
func main() {
var a A
err := decode(&a, []interface{}{"Name", "Place"})
log.Println(err)
log.Println(a)
}
func decode(dst interface{}, values []interface{}) error {
rvptr := reflect.ValueOf(dst)
if rvptr.Kind() != reflect.Ptr {
return fmt.Errorf("value must be ptr")
}
rv := rvptr.Elem()
if rv.NumField() < len(values) {
return fmt.Errorf("too many values")
}
if rv.NumField() > len(values) {
return fmt.Errorf("too few values")
}
rvalues := reflect.ValueOf(values)
for i := range values {
f := rv.FieldByIndex([]int{i})
f.Set(rvalues.Index(i).Elem())
}
return nil
}
type A struct {
Name string
Place string
}
type B struct {
Name string
Place string
Value string
}
prints
$ go run main.go
2019/11/21 17:00:17 <nil>
2019/11/21 17:00:17 {Name Place}
The problem is the return type for the function.
func newA(in []string) interface{} {...}
func newB(in []string) interface{} {...}
func getStruct(name string) func([]string) interface{} {
switch name {
case "A": return newA
case "B": return newB
}
return nil
}
func main() {
buildNewStruct := getStruct("A")
var strSlice = []string{"Bob", "US"}
str:=buildNewStruct(strSlice)
if a, ok:=str.(A); ok {
...
}
}
With this approach, even though you saved some code by calling a unified buildNewStruct(), you have to use type assertions to figure out what is returned from that function, so this may not make a lot of sense. It depends on your exact use case though.

go: var declared but not used error - how to work around it?

In this function I get "s declared and not used" which I don't understand - do I need to somehow tag it as 'really I used it' or something?
func getString(data map[string]interface{}, name string) (string, error) {
s := data[name]
if reflect.TypeOf(s).Kind() != reflect.String {
return s.(string), nil
}
return "", &apiError{1, "it's not a string"}
}
Oddly, I don't get the error from this function:
func getInt(data map[string]interface{}, name string) (int, error) {
t := data[name]
if reflect.TypeOf(t).Kind() == reflect.Int {
return t.(int), nil
}
return 0, &apiError{1, "it's not an int"}
}
Also, any thoughts on the right way to factor these into a single function would be welcomed!
Your error comes from (declaring and not) using the same identifier elsewhere because this compiles and runs fine on golang.org:
package main
import "reflect"
func main() {
m := make(map[string]interface{})
m["foo"] = "25"
getString(m, "foo")
}
func getString(data map[string]interface{}, name string) (string, error) {
s := data[name]
if reflect.TypeOf(s).Kind() != reflect.String {
return s.(string), nil
}
return "", nil
}
Your code looks correct, error isn't reproducible.
Sure you can refactor these into a single function, but you may not like it depending of tastes.
type VType int
const (
VInteger VType = iota
VString
VUnknown
)
func getValue(data map[string]interface{}, name string) (VType, int, string) {
switch v := data[name].(type) {
case int:
return VInteger, v, ""
case string:
return VString, 0, v
default:
return VUnknown, 0, ""
}
}
func main() {
m := make(map[string]interface{})
m["foo"] = "25"
switch t, i, s := getValue(m, "foo"); t {
case VInteger:
fmt.Println("int ", i) //do something with int
case VString:
fmt.Println("string ", s) //do something with string
case VUnknown:
err := &apiError{1, "it's not an int"} //do something with err
}
}

Variable modification in golang [Mutability]

The below code opens up a .txt file and counts the word frequencies. I am following a book and I got confused:
My question is here:
filename := os.Args[1]
frequencyForWord := map[string]int{}
updateFrequencies(filename, frequencyForWord)
fmt.Println(frequencyForWord)
I create a variable called frequencyForWord and pass it into a function that does not return anything called func updateFrequencies
This function modifies the variable and that's why when I do fmt.Println(frequencyForWord) it shows me a map that has words as keys and their counts as values.
My question is:
why don't I have to do something like this
frequencyForWord = updateFrequencies(filename, frequencyForWord)
fmt.Println(frequencyForWord)
// And then change func updateFrequencies to something to returns a map
I thought in order for a function to modify a variable I need to pass in the variable as a reference like this updateFrequencies(filename, &frequencyForWord)
Original Code:
package main
import(
"fmt"
"path/filepath"
"os"
"log"
"bufio"
"strings"
"unicode"
)
func main() {
if len(os.Args) == 1 || os.Args[1] == "-h" {
fmt.Printf("usage: %s <file>\n", filepath.Base(os.Args[0]))
os.Exit(1)
}
filename := os.Args[1]
frequencyForWord := map[string]int{}
updateFrequencies(filename, frequencyForWord)
fmt.Println(frequencyForWord)
}
func updateFrequencies(filename string, frequencyForWord map[string]int) string {
file, err := os.Open(filename)
if err != nil {
log.Printf("Failed to open the file: %s.", filename)
}
defer file.Close()
readAndUpdateFrequencies(bufio.NewScanner(file), frequencyForWord)
}
func readAndUpdateFrequencies(scanner *bufio.Scanner, frequencyForWord map[string]int) {
for scanner.Scan() {
for _, word := range SplitOnNonLetter(strings.TrimSpace(scanner.Text())) {
frequencyForWord[strings.ToLower(word)] += 1
}
}
if err := scanner.Err(); err != nil {
log.Fatal(err)
}
}
func SplitOnNonLetter(line string) []string {
nonLetter := func(char rune) bool { return !unicode.IsLetter(char) }
return strings.FieldsFunc(line, nonLetter)
}
Because the map structure doesn't contain the values itself but points to the structures holding the values.
As written in the documentation :
Like slices, maps hold references to an underlying data structure. If
you pass a map to a function that changes the contents of the map, the
changes will be visible in the caller.
That's just like when you pass a pointer to a function : it lets the function change your value.
Here's another example of the same phenomenon :
type A struct {
b *B
}
type B struct {
c int
}
func incr(a A) {
a.b.c++
}
func main() {
a := A{}
a.b = new(B)
fmt.Println(a.b.c) // prints 0
incr(a)
fmt.Println(a.b.c) // prints 1
}
The function is not modifying the variable, but the value bound to the variable. That's possible because a map is a mutable data structure and passing it to a function does not copy the structure. (A map is implicitly a reference to a hash table and the reference is passed around.)

Resources