How to change empty *strings to nil using mold package? - go

During validation of user input I wanna set to nil some empty *strings in some structs.
E.g. I have this struct:
type User {
Username *string
}
If Username == "" I wanna set Username to nil before proceed.
But I wanna do this using tags, e.g.:
type User {
Username *string `mod:"nil_if_empty"`
}
because I'm already using this package: https://github.com/go-playground/mold in my project for sanitize.
So I'm creating a custom function to set empty strings to nil, but I'm stuck.
Reproduction: https://play.golang.org/p/_DlluqE2Y3k
package main
import (
"context"
"fmt"
"log"
"reflect"
"github.com/go-playground/mold/v3"
)
var tform *mold.Transformer
func main() {
tform = mold.New()
tform.Register("nilEmptyString", nilEmptyString)
type Test struct {
String *string `mold:"nilEmptyString"`
}
myEmptyString := ""
tt := Test{String: &myEmptyString}
err := tform.Struct(context.Background(), &tt)
if err != nil {
log.Fatal(err)
}
fmt.Printf("%+v\n", tt)
}
func nilEmptyString(_ context.Context, _ *mold.Transformer, v reflect.Value, _ string) error {
s, ok := v.Interface().(*string)
if !ok { // ok is always false, why?
return nil
}
if s != nil && *s != "" {
return nil
}
if s != nil && *s == "" {
// v.SetPointer(nil) // should I use this?
}
// v.Set(nil) // how to do this?
return nil
}
Why ok in s, ok := v.Interface().(*string) is always false?
How to set v to nil? Why is v.Set(nil) wrong?
I hope this is a good question to ask on StackOverflow. If not tell me how to improve.
Thanks.

Related

How does Golang accept uncertain values

Back-end return values are not fixed, sometimes:
{"application": {"instance": [{"instanceId": "v1"}, {"instanceId": "v2"}]}}
or sometimes:
{"application": {"instance": {"instanceId": "v"}}}
how should I take out the corresponding instanceId value?
package main
import (
"encoding/json"
"fmt"
)
type Application struct {
Application struct {
Instance json.RawMessage `json:"instance"`
} `json:"application"`
}
func main() {
a := `{"application": {"instance": {"instanceId": "v"}}}`
//a := `{"application": {"instance": [{"instanceId": "v1"}, {"instanceId": "v2"}]}} `
var p Application
errJson := json.Unmarshal([]byte(a), &p)
if errJson != nil {
fmt.Printf("errJson")
}
fmt.Printf("type:%T", p.Application.Instance)
}
Since the 2 value types clash (one a struct another a slice of structs) it gets messy to encapsulate this into a single type even using catch-all solutions like interface{}.
The simplest solution is present two distinct types and marshal into either to see which "works":
func unmarsh(body []byte) (*type1, *type2, error) {
var (
t1 type1
t2 type2
)
err := json.Unmarshal(body, &t1)
if err == nil {
return &t1, nil, nil
}
err = json.Unmarshal(body, &t2)
if err == nil {
return nil, &t2, nil
}
return nil, nil, err
}
and in your example the two types would be:
type type1 struct {
Application struct {
Instance []struct {
InstanceID string `json:"instanceId"`
} `json:"instance"`
} `json:"application"`
}
type type2 struct {
Application struct {
Instance struct {
InstanceID string `json:"instanceId"`
} `json:"instance"`
} `json:"application"`
}
Working example:
https://play.golang.org/p/Kma32gWfghb
A cleaner solution would be a custom unmarshaler:
type Instances []Instance
func (i *Instances) UnmarshalJSON(in []byte) error {
if len(in)>0 && in[0]=='[' {
var a []Instance
if err:=json.Unmarshal(in,&a); err!=nil {
return err
}
*i=a
return nil
}
var s Instance
if err:=json.Unmarshal(in,&s) ; err!=nil {
return err
}
*i=[]Instance{s}
return nil
}
This would unmarshal an object into a slice of 1.
A more compact solution is provided by #mkopriva:
func (i *Instances) UnmarshalJSON(in []byte) error {
if len(in) > 0 && in[0] == '[' {
return json.Unmarshal(in, (*[]Instance)(i))
}
*i = Instances{{}}
return json.Unmarshal(in, &(*i)[0])
}

How to check reflect.Value is nil or not?

I have this code:
package main
import (
"fmt"
"reflect"
)
type cmd struct{
Echo func(string) (string,error)
}
func main() {
cmd := cmd{
Echo : func(arg string) (string, error) {
return arg, nil
},
}
result := reflect.ValueOf(cmd).FieldByName("Echo").Call([]reflect.Value{reflect.ValueOf("test")})
if result[1] == nil{
fmt.Println("ok")
}
}
I want to check if my error is nil, but in my code, it doesn't work cuz it has different types. I try to make like this :
reflect[1] == reflect.Value(reflect.ValueOf(nil))
So it has the same type but the value of reflect.Value(reflect.ValueOf(nil)) isn't nil, it is <invalid reflect.Value>.
Use .IsNil() to check whether the value that's stored in reflect.Value is nil.
if result[1].IsNil() {
fmt.Println("ok")
}
Or you can use .Interface() to get the actual value that's stored in reflect.Value and check whether this is nil.
if result[1].Interface() == nil {
fmt.Println("ok")
}

How to omit empty json fields using json.decoder

I try to understand why both functions return the same output.
As far as I understood, the point of omit empty is to not add that key to the result struct.
I wrote this example, I was expecting the first output not to have the "Empty" key, but for some reason its value still shows as 0.
package main
import (
"encoding/json"
"fmt"
"strings"
)
type agentOmitEmpty struct {
Alias string `json:"Alias,omitempty"`
Skilled bool `json:"Skilled,omitempty"`
FinID int32 `json:"FinId,omitempty"`
Empty int `json:"Empty,omitempty"`
}
type agent struct {
Alias string `json:"Alias"`
Skilled bool `json:"Skilled"`
FinID int32 `json:"FinId"`
Empty int `json:"Empty"`
}
func main() {
jsonString := `{
"Alias":"Robert",
"Skilled":true,
"FinId":12345
}`
fmt.Printf("output with omit emtpy: %v\n", withEmpty(strings.NewReader(jsonString)))
// output with omit emtpy: {Robert true 12345 0}
fmt.Printf("output regular: %v\n", withoutEmpty(strings.NewReader(jsonString)))
// output without omit: {Robert true 12345 0}
}
func withEmpty(r *strings.Reader) agentOmitEmpty {
dec := json.NewDecoder(r)
body := agentOmitEmpty{}
err := dec.Decode(&body)
if err != nil {
panic(err)
}
return body
}
func withoutEmpty(r *strings.Reader) agent {
dec := json.NewDecoder(r)
body := agent{}
err := dec.Decode(&body)
if err != nil {
panic(err)
}
return body
}
You need to define Empty as *int so it will be replaced with nil when there is no value. Then it will not be saved in the database.

How to handle Response JSON have custom field with out key?

Query Api and response a custom JSON, how to Unmarshal it. the sample JSON:
{"14AcKEr19gHJvgwQhK7sfFm6YJGmoZZoqu": {
"final_balance": 61914248289,
"n_tx": 3472,
"total_received": 3479994002972
}}
The key is a hex string. So how to handle it with golang convention, anyone can help me?
Below is my try test code:
c.OnResponse(func(r *colly.Response) {
jsonData := r.Body
fmt.Println(string(jsonData))
fmt.Println("==================")
//parse bitcoin json
jsonMap := make(map[string]interface{})
err := json.Unmarshal([]byte(jsonData), &jsonMap)
if err != nil {
panic(err)
}
fmt.Println(jsonMap)
dumpMap("", jsonMap)
})
func dumpMap(space string, m map[string]interface{}) {
for k, v := range m {
if mv, ok := v.(map[string]interface{}); ok {
fmt.Printf("{ \"%v\": \n", k)
dumpMap(space+"\t", mv)
fmt.Printf("}\n")
} else {
fmt.Printf("%v %v : %v\n", space, k, v)
}
}
}
and go run cmd/main.go, the console is print here:
{"14AcKEr19gHJvgwQhK7sfFm6YJGmoZZoqu": {
"final_balance": 75494521080,
"n_tx": 3493,
"total_received": 3493574275763
}}
==================
map[14AcKEr19gHJvgwQhK7sfFm6YJGmoZZoqu:map[n_tx:3493 total_received:3.493574275763e+12 final_balance:7.549452108e+10]]
{ "14AcKEr19gHJvgwQhK7sfFm6YJGmoZZoqu":
final_balance : 7.549452108e+10
n_tx : 3493
total_received : 3.493574275763e+12
}
Do I need customised unmarshal func to get string key? If I use 14AcKEr19gHJvgwQhK7sfFm6YJGmoZZoqu as key I can't easily to access. I just want to know how handle it.
you can unmarshal it into map, so you can get generated key as a key of map
https://play.golang.org/p/IfEjjvKakpu
package main
import (
"encoding/json"
"fmt"
"log"
)
var input = `{"14AcKEr19gHJvgwQhK7sfFm6YJGmoZZoqu": {
"final_balance": 61914248289,
"n_tx": 3472,
"total_received": 3479994002972
}}`
type object struct {
FinalBalance uint64 `json:"final_balance"`
NTX uint64 `json:"n_tx"`
TotalReceived uint64 `json:"total_received"`
}
func main() {
var result map[string]object;
err := json.Unmarshal([]byte(input), &result);
if err != nil {
log.Fatal(err)
}
fmt.Printf("result: %+v", result)
// result: map[14AcKEr19gHJvgwQhK7sfFm6YJGmoZZoqu:{FinalBalance:61914248289 NTX:3472 TotalReceived:3479994002972}]
}

Go url parameters mapping

Is there a native way for inplace url parameters in native Go?
For Example, if I have a URL: http://localhost:8080/blob/123/test I want to use this URL as /blob/{id}/test.
This is not a question about finding go libraries. I am starting with the basic question, does go itself provide a basic facility to do this natively.
There is no built in simple way to do this, however, it is not hard to do.
This is how I do it, without adding a particular library. It is placed in a function so that you can invoke a simple getCode() function within your request handler.
Basically you just split the r.URL.Path into parts, and then analyse the parts.
// Extract a code from a URL. Return the default code if code
// is missing or code is not a valid number.
func getCode(r *http.Request, defaultCode int) (int, string) {
p := strings.Split(r.URL.Path, "/")
if len(p) == 1 {
return defaultCode, p[0]
} else if len(p) > 1 {
code, err := strconv.Atoi(p[0])
if err == nil {
return code, p[1]
} else {
return defaultCode, p[1]
}
} else {
return defaultCode, ""
}
}
Well, without external libraries you can't, but may I recommend two excellent ones:
httprouter - https://github.com/julienschmidt/httprouter - is extremely fast and very lightweight. It's faster than the standard library's router, and it creates 0 allocations per call, which is great in a GCed language.
Gorilla Mux - http://www.gorillatoolkit.org/pkg/mux -
Very popular, nice interface, nice community.
Example usage of httprouter:
func Hello(w http.ResponseWriter, r *http.Request, ps httprouter.Params) {
fmt.Fprintf(w, "hello, %s!\n", ps.ByName("name"))
}
func main() {
router := httprouter.New()
router.GET("/hello/:name", Hello)
log.Fatal(http.ListenAndServe(":8080", router))
}
What about trying using regex, and find a named group in your url, like playground:
package main
import (
"fmt"
"net/url"
"regexp"
)
var myExp = regexp.MustCompile(`/blob/(?P<id>\d+)/test`) // use (?P<id>[a-zA-Z]+) if the id is alphapatic
func main() {
s := "http://localhost:8080/blob/123/test"
u, err := url.Parse(s)
if err != nil {
panic(err)
}
fmt.Println(u.Path)
match := myExp.FindStringSubmatch(s) // or match := myExp.FindStringSubmatch(u.Path)
result := make(map[string]string)
for i, name := range myExp.SubexpNames() {
if i != 0 && name != "" {
result[name] = match[i]
}
}
fmt.Printf("id: %s\n", result["id"])
}
output
/blob/123/test
id: 123
Below full code to use it with url, that is receiving http://localhost:8000/hello/John/58 and returning http://localhost:8000/hello/John/58:
package main
import (
"fmt"
"net/http"
"regexp"
"strconv"
)
var helloExp = regexp.MustCompile(`/hello/(?P<name>[a-zA-Z]+)/(?P<age>\d+)`)
func hello(w http.ResponseWriter, req *http.Request) {
match := helloExp.FindStringSubmatch(req.URL.Path)
if len(match) > 0 {
result := make(map[string]string)
for i, name := range helloExp.SubexpNames() {
if i != 0 && name != "" {
result[name] = match[i]
}
}
if _, err := strconv.Atoi(result["age"]); err == nil {
fmt.Fprintf(w, "Hello, %v year old named %s!", result["age"], result["name"])
} else {
fmt.Fprintf(w, "Sorry, not accepted age!")
}
} else {
fmt.Fprintf(w, "Wrong url\n")
}
}
func main() {
http.HandleFunc("/hello/", hello)
http.ListenAndServe(":8090", nil)
}
How about writing your own url generator (extend net/url a little bit) as below.
// --- This is how does it work like --- //
url, _ := rest.NewURLGen("http", "stack.over.flow", "1234").
Pattern(foo/:foo_id/bar/:bar_id).
ParamQuery("foo_id", "abc").
ParamQuery("bar_id", "xyz").
ParamQuery("page", "1").
ParamQuery("offset", "5").
Do()
log.Printf("url: %s", url)
// url: http://stack.over.flow:1234/foo/abc/bar/xyz?page=1&offset=5
// --- Your own url generator would be like below --- //
package rest
import (
"log"
"net/url"
"strings"
"straas.io/base/errors"
"github.com/jinzhu/copier"
)
// URLGen generates request URL
type URLGen struct {
url.URL
pattern string
paramPath map[string]string
paramQuery map[string]string
}
// NewURLGen new a URLGen
func NewURLGen(scheme, host, port string) *URLGen {
h := host
if port != "" {
h += ":" + port
}
ug := URLGen{}
ug.Scheme = scheme
ug.Host = h
ug.paramPath = make(map[string]string)
ug.paramQuery = make(map[string]string)
return &ug
}
// Clone return copied self
func (u *URLGen) Clone() *URLGen {
cloned := &URLGen{}
cloned.paramPath = make(map[string]string)
cloned.paramQuery = make(map[string]string)
err := copier.Copy(cloned, u)
if err != nil {
log.Panic(err)
}
return cloned
}
// Pattern sets path pattern with placeholder (format `:<holder_name>`)
func (u *URLGen) Pattern(pattern string) *URLGen {
u.pattern = pattern
return u
}
// ParamPath builds path part of URL
func (u *URLGen) ParamPath(key, value string) *URLGen {
u.paramPath[key] = value
return u
}
// ParamQuery builds query part of URL
func (u *URLGen) ParamQuery(key, value string) *URLGen {
u.paramQuery[key] = value
return u
}
// Do returns final URL result.
// The result URL string is possible not escaped correctly.
// This is input for `gorequest`, `gorequest` will handle URL escape.
func (u *URLGen) Do() (string, error) {
err := u.buildPath()
if err != nil {
return "", err
}
u.buildQuery()
return u.String(), nil
}
func (u *URLGen) buildPath() error {
r := []string{}
p := strings.Split(u.pattern, "/")
for i := range p {
part := p[i]
if strings.Contains(part, ":") {
key := strings.TrimPrefix(p[i], ":")
if val, ok := u.paramPath[key]; ok {
r = append(r, val)
} else {
if i != len(p)-1 {
// if placeholder at the end of pattern, it could be not provided
return errors.Errorf("placeholder[%s] not provided", key)
}
}
continue
}
r = append(r, part)
}
u.Path = strings.Join(r, "/")
return nil
}
func (u *URLGen) buildQuery() {
q := u.URL.Query()
for k, v := range u.paramQuery {
q.Set(k, v)
}
u.RawQuery = q.Encode()
}
With net/http the following would trigger when calling localhost:8080/blob/123/test
http.HandleFunc("/blob/", yourHandlerFunction)
Then inside yourHandlerFunction, manually parse r.URL.Path to find 123.
Note that if you don't add a trailing / it won't work. The following would only trigger when calling localhost:8080/blob:
http.HandleFunc("/blob", yourHandlerFunction)
As of 19-Sep-22, with go version 1.19, instance of http.request URL has a method called Query, which will return a map, which is a parsed query string.
func helloHandler(res http.ResponseWriter, req *http.Request) {
// when request URL is `http://localhost:3000/?first=hello&second=world`
fmt.Println(req.URL.Query()) // outputs , map[second:[world] first:[hello]]
res.Write([]byte("Hello World Web"))
}
No way without standard library. Why you don't want to try some library? I think its not so hard to use it, just go get bla bla bla
I use Beego. Its MVC style.
how about a simple utility function ?
func withURLParams(u url.URL, param, val string) url.URL{
u.Path = strings.ReplaceAll(u.Path, param, val)
return u
}
you can use it like this:
u, err := url.Parse("http://localhost:8080/blob/:id/test")
if err != nil {
return nil, err
}
u := withURLParams(u, ":id","123")
// now u.String() is http://localhost:8080/blob/123/test
If you need a framework and you think it will be slow because it's 'bigger' than a router or net/http, then you 're wrong.
Iris is the fastest go web framework that you will ever find, so far according to all benchmarks.
Install by
go get gopkg.in/kataras/iris.v6
Django templates goes easy with iris:
import (
"gopkg.in/kataras/iris.v6"
"gopkg.in/kataras/iris.v6/adaptors/httprouter"
"gopkg.in/kataras/iris.v6/adaptors/view" // <-----
)
func main() {
app := iris.New()
app.Adapt(iris.DevLogger())
app.Adapt(httprouter.New()) // you can choose gorillamux too
app.Adapt(view.Django("./templates", ".html")) // <-----
// RESOURCE: http://127.0.0.1:8080/hi
// METHOD: "GET"
app.Get("/hi", hi)
app.Listen(":8080")
}
func hi(ctx *iris.Context){
ctx.Render("hi.html", iris.Map{"Name": "iris"})
}

Resources