Using validator to check if value is boolean - validation

I'm new to Go, so this might be very easy, but I can't find it. I have an entity Page with these two properties:
type Page struct {
Title string `form:"title" binding:"required"`
Active bool
}
Now, if you don't send a title we get a (not very pretty, but acceptable*):
Key: 'Page.Title' Error:Field validation for 'Title' failed on the 'required' tag.
Now, if I send this to the endpoint:
{
"title": "I'm a valid title",
"active": "I'm not a boolean at all!"
}
We get this:
json: cannot unmarshal string into Go struct field Page.Active of type bool
Which, IMO, is giving way too much info. What is the standard practice in Go to validate user input?
I was first making a page-validor.go with some checks, then I found this, but I'm not sure what is good practice in Go.
How do I validate this properly? Should I find check what is provided and then try to move it into the struct and validate the actual contents?
I am using GinGonic
* I've found a way to unwrap the errors and make it nicer

Write custom JSON Unmarshaller method for the type Page and inside UnmarshalJSON(bytes []byte) method, you can unmarshal the JSON bytes to map[string]interface{} and then validate the types you need with the JSON field keys.
An example of the JSON Unmarshaller looks like below.
type Page struct {
Title string `form:"title" binding:"required"`
Active bool
}
func (p *Page) UnmarshalJSON(bytes []byte) error {
var data map[string]interface{}
err := json.Unmarshal(bytes, &data)
if err != nil {
return err
}
actv, _ := data["active"]
if reflect.TypeOf(actv).Kind() != reflect.Bool {
return errors.New("active field should be a boolean")
}
p.Active = actv.(bool)
return nil
}
See the full example here in Playground.

After some more research, I've implemented Go-map-schema.
var page Page
src := make(map[string]interface{})
json.Unmarshal(jsonData, &src)
results, _ := schema.CompareMapToStruct(page, src, nil)
fmt.Println(results.MissingFields)
fmt.Println(results.MismatchedFields)
This works simple with the standard notations for an struct:
type Page struct {
Title string `json:"title" validator:"required"`
Active bool `json:"metaRobotsFollow" validate:"required,bool"`
}

You should use validator v10 available with Go
validator documentation go
For your use case you can use boolean validator
https://pkg.go.dev/github.com/go-playground/validator/v10#hdr-Boolean
type Page struct {
Title string `json:"title" binding:"required"`
Active bool `json:"active" binding:"required,boolean"`
}
Below is sample Test case with one positive and negative
func TestPage(t *testing.T) {
tests := []struct {
testName string
input string
wantErr bool
}{
{
testName: "positive test",
input: `{
"title": "first book title",
"active": false
}`,
wantErr: false,
}, {
testName: "wrong boolean",
input: `{
"title": "second book title",
"active": falsee
}`,
wantErr: false,
},
}
for _, tt := range tests {
t.Run(tt.input, func(t *testing.T) {
var p Page
b := []byte(tt.input)
err := json.Unmarshal(b, &p)
assert.Nil(t, err, "got error %v", err)
})
}

Related

Add root element to existing Json in Go lang

I'm trying to add string "Employee" to my existing JSON response. Also, we need to be able to generate this version of json based on an user condition. Only if the user condition is met, I need to generate second version of json with string "Employee" added. If not the first version without string "Employee" should be generated. How can I achieve it with out updating the existing struct and how can I check this with if clause to check for the condition and then generate json based on it?
Below is my existing json response in go
[
{
"EmpId":{
"String":"ABCD",
"Valid":true
},
"Department":{
"Float64":0,
"Valid":true
}
}
]
How can I get my json response like below with out changing existing struct based on input parameter?
{
"Employee":[
{
"EmpId":{
"String":"ABCD",
"Valid":true
},
"Department":{
"Float64":0,
"Valid":true
}
}
]
}
Below is my code:
Step 1: model folder
type EmployeeWithRoot struct {
Employee []Employee
}
type Employee struct {
EmpNbr sql.NullString `json:"EmpNbr"`
DateofJoin sql.NullString `json:"DateofJoin"`
DeptId sql.NullString `json:"DeptId"`
DeptName sql.NullString `json:"DeptName"`
}
Step 2: code folder
func GetEmpDetails(logRequestId string, logNestedLevel int, EmpNbr string, DateofJoin string) ([]model.EmployeeWithRoot, error) {
logFunctionFunctionName := "code.GetEmpDetails"
logStartTime := time.Now()
logNestedLevel++
defer configurations.TimeTrack(logFunctionFunctionName, logRequestId, logStartTime, logNestedLevel)
rows, err := db.Query(utils.SELECT_OF_EMP_AGGR, EmpNbr, DateofJoin, DeptId, DeptName)
if err != nil {
return nil, err
}
defer rows.Close()
var e []model.EmployeeWithRoot
for rows.Next() {
var x model.EmployeeWithRoot
err := rows.Scan(&x.Employee.EmpNbr, &x.Employee.DateofJoin, &x.Employee.DeptId,&x.Employee.DeptName)
if err != nil {
return nil, err
}
e = append(e, x)
}
err = rows.Err()
if err != nil {
return nil, err
}
return e, nil
}
STEP 3: API folder
Employee, err := code.GetEmpDetails(logRequestId, logNestedLevel, EmpNbr, DateofJoin)
if err != nil {
log.Panic(err)
}
marshalDataForRequestContentType(logRequestId, logNestedLevel, w, r, Employee)
I'm getting the below error.
x.Employee.EmpNbr undefined (type []model.Employee has no field or method EmpNbr)
x.Employee.DateofJoin undefined (type []model.Employee has no field or method DateofJoin)enter code here
x.Employee.DeptId undefined (type []model.Employee has no field or method DeptId)
x.Employee.DeptName undefined (type []model.Employee has no field or method DeptName)
Considering you're just wrapping it it an outer object, I don't see any reason you'd need to change the existing struct, just wrap it in a new one. I'll have to make some guesses/assumptions here since you've only shown the JSON and not the Go code that produces it, but assuming your existing JSON is produced by marshaling something like var response []Employee, the desired JSON could be produced in your condition by marshaling instead:
json.Marshal(struct{Employee []Employee}{response})
Working example: https://go.dev/play/p/vwDvxnQ96G_2
Use string concatenation:
func addRoot(json string) string {
return `{ "Employee":` + json + `}`
}
Run an example on the GoLang playground.
Here's the code if you are working with []byte instead of string:
func addRoot(json []byte) []byte {
const prefix = `{ "Employee":`
const suffix = `}`
result := make([]byte, 0, len(prefix)+len(json)+len(suffix))
return append(append(append(result, prefix...), json...), suffix...)
}
Run this example on the GoLang playground.
If you have some JSON in a byte slice ([]byte) then you can just add the outer element directly - e.g. (playground):
existingJSON := []byte(`[
{
"EmpId":{
"String":"ABCD",
"Valid":true
},
"Department":{
"Float64":0,
"Valid":true
}
}
]`)
b := bytes.NewBufferString(`{"Employee":`)
b.Write(existingJSON)
b.WriteString(`}`)
fmt.Println(string(b.Bytes()))
If this is not what you are looking for please add further details to your question (ideally your attempt as a minimal, reproducible, example)

Can I copy the value of pointer of interface with tags in golang with unknown type

Can I copy the value of pointer of interface with tags in golang with unknown type
I have a function called "CensorPayload" it will receive a pointer and Unmarshal and censor some field that have tag called "censored" but I don't want to update value in argument called "target" that mean I have to copy the value of target to new interface{}. I try many way such as reflect.New but the tag does not pass through Unmarshal or Do function.
type HelloWorld struct {
Hello string `json:"hello"`
Password string `json:"password" censored:"*****"`
}
a := HelloWorld{}
CensorPayload(&a, `{"hello": "world", "password": "12345"}`)
func CensorPayload(target interface{}, payload string) (string, error) {
err := json.Unmarshal([]byte(payload), target)
if err != nil {
return "", err
}
err = Do(target)
if err != nil {
return "", err
}
censoredPayload, err := json.Marshal(target)
if err != nil {
return "", err
}
return string(censoredPayload), nil
}
There are quite a few ways to improve and accomplish this. I'll start with some improvements, and work my way up to possible implementations of what you're looking for.
Improvements
First up, I'd avoid using target interface{} as an argument. There is absolutely nothing there to guarantee that the caller will pass in a pointer. At the very least, I'd consider using a trick that protobuf uses to ensure arguments of the interface proto.Message are pointer values. This is done by adding an empty method with pointer receiver to the types. In this case, that would be:
type JSONPtr interface {
JSONPtr()
}
// implemented:
func (*HelloWorld) JSONPtr() {} // empty
func CensorPayload(target JSONPtr, payload string) (string, error) {
// pretty much the same as you have here
}
Another option would be to use generics, depending on how many types like this you actually need to handle, this could be a valid use-case for them:
type CensorTypes interface {
*HelloWorld | *AnotherType | *YetAnother
}
func CensorPayload[T CensorTypes](target T, payload string) (string, error) {
// same as what you have now
}
Reducing duplication with embedding
If you have a number of types with a Password field, all of which need to be capable of being censored, you could just split out the Password field into its own type and embed it where needed:
type Password struct {
Password string `json:"password"`
}
type HelloWorld struct {
// regular fields here
Password // embed the password field
}
In case you need to use this password censoring a lot, I'd definitely take this approach. Towards the end, I'll include an example where I use embedded types to accomplish what you're trying to do with minimal code duplication...
Possible implementations
All this does, however, is ensure that a pointer is passed, which the interface approach already does anyway. You want to be able to censor the password, and have a specific call that does this in your code, so why not implement a method on the relevant types, and add that to an interface. Adding generics, we can get a compile-time check to ensure that the relevant method(s) were implemented correctly (ie with a pointer receiver):
type Censored interface {
*HelloWorld | *AnotherType // and so on
CensorPass()
}
func (h *HelloWorld) CensorPass() { // pointer receiver
h.Password = "******"
}
// in case `Password` is an embedded type, no need to implement it on `HelloWorld`
func (p *Password) CensorPass() {
p.Password = "******"
}
func CenssorPayload[T Censored](target T, payload string) (string, error) {
// same as before until:
// Do() is replaced with:
target.CensorPass()
// the rest is the same
}
Now, if your HelloWorld type implements CensorPass with a value receiver (func (h HelloWorld) CensorPass), or the caller fails to pass in a pointer value as the target argument, you'll get an error.
Option 2: custom marshallers
Another option here is to create a wrapper type that embeds all of the types you need to handle, and implement a custom marshaller interface on said type:
type Wrapper struct {
*HelloWorld
*AnotherType
*YestAnother
}
func (w Wrapper) MarshalJSON() ([]byte, error) {
if w.HelloWorld != nil {
w.HelloWorld.Password = "*****"
return json.Marshal(w.HelloWorld)
}
if w.AnotherType != nil {
w.AnotherType.Pass = "*****"
w.AnotherType.AnotherSensitiveField = "*******"
return json.Marshal(w.AnotherType)
}
func w.YetAnother != nil {
w.YetAnother.User = "<censored>"
return json.Marshal(w.YetAnother)
}
}
Populating this Wrapper type can all be handled by a single function and a type-switch. I'm using any (short for interface{} below, but I'd recommend you use an interface or generics for the reasons given above if you want to call this method from other packages, if you're calling it from CensorPayload alone, and you've already changed it to use interfaces/generics, the code below is fine:
func wrap(target any) (*Wrapper, error) {
switch tt := target.(type) {
case *HelloWorld:
return &Wrapper{
HelloWorld: tt,
}, nil
case *AnotherType:
return &Wrapper{
AnotherType: tt,
}, nil
case *YetAnother:
return &Wrapper{
YetAnother: tt,
}, nil
}
// if we reach this point, the type passed to this function is unknown/can't be censored
return nil, errors.New("type not supported by wrapper")
}
With this in place, you can simply change your CensorPayload function to something like this:
// CensorPayload using interface to ensure pointer argument
func CensorPayload(target JSONPtr, payload string) (string, error) {
if err := json.Unmarshal(target, []byte(payload)); err != nil {
return "", err
}
wrapped, err := wrap(target)
if err != nil {
// type not handled by wrapper, either return an error
return "", err // target was not supported
// or in case that means this data should not be censored, just return the marhsalled raw data:
raw, err := json.Marshal(target)
if err != nil {
return "", err
}
return string(raw), nil
}
raw, err := json.Marshal(wrapped) // this will call Wrapper.MarshalJSON and censor what needs to be censored
if err != nil {
return "", err
}
return string(raw), nil
}
What I'd go for
I'd probably use the Censored type in the CensorPayload function. It has the benefit of not needing the wrapper at all, shifts the responsibility of knowing what fields/values to censor on to the data-types, and ensures that both the CensorPass method is implemented correctly (pointer receiver), and the actual value being passed is a pointer. To reduce the total LoC to a minimum, I'd leave it to the caller to cast the []byte return value to a string, too. Same goes for the payload argument. Raw JSON data is represented as a byte slice, so the payload argument, IMO, should reflect this:
func CensorPayload[T Censored](target T, payload []byte) ([]byte, error) {
if err := json.Unmarshal(target, payload); err != nil {
return nil, err
}
target.CensorPass() // type knows what fields to censor
return json.Marshal(target) // return the data censored & marshalled
}
As mentioned above, I'd combine this approach with commonly censored fields being embedded types:
type Password struct {
Password `json:"password"`
}
type AnotherSensitiveField struct {
AnotherSensitiveField string `json:"example_db_ip"`
}
// Password implements CensorPass, is embedded so we don't need to do anything here
type HelloWorld struct {
// fields
Password
}
// this one has 2 fields that implement CensorPass
// we need to ensure both are called
type AnotherType struct {
// fields
Password
AnotherSensitiveField
}
// this is a one-off, so we need to implement CensorPass on the type
type YetAnother struct {
// all fields - no password
User string `json:"user"` // this one needs to be censored
}
func (a *AnotherType) CensorPass() {
a.Password.CensorPass()
a.AnotherSensitiveField.CensorPass()
}
func (y *YetAnother) CensorPass() {
y.User = "<censored>"
}
func (p *Password) CensorPass() {
p.Password = "******"
}
func (a *AnotherSensitiveField) CensorPass() {
a.AnotherSensitiveField = "******"
}
Any type that embeds one type like Password will automatically have a CensorPass method. The types are embedded, and no other embedded types have a name collision, so the top level type can resolve HelloWorld.CensorPass to HelloWorld.Password.CensorPass(). This doesn't work for AnotherType, as it has 2 embedded types that provide this method. To get around this, we simply implement the method at the top level, and pass the call on to the embedded types. The third example is one where we want to censor a specific field that isn't used anywhere else (yet). Creating a separate type to embed isn't required, so we can just implement the method directly on the YetAnother type. Should we have data where user credentials are present, and we wish to censor multiple fields in different places, we can easily create a type for this:
type Credentials struct {
User string `json:"user"`
Pass string `json:"pass"`
}
type FullName struct {
Name string `json:"name"`
First string `json:"first_name"`
}
func (c *Credentials) CensorPass() {
c.User = "<redacted>"
c.Pass = "******"
}
func (f *FullName) CensorPass() {
if len(f.First) > 0 {
f.First = string([]rune(f.First)[0])
}
if len(f.Last) == 0 {
return
}
last := make([]string, 0, 2)
for _, v := range strings.Split(f.Last) {
if len(v) > 0 {
last = append(last, string([]rune(v)[0]))
}
}
last = append(last, "") // for the final .
f.Last = strings.Join(last, ". ") // initials only
}
And all we have to do is embed this type wherever we need it. Once that's done, our main function still looks like this:
func CensorPayload[T Censored](target T, payload []byte) ([]byte, error) {
if err := json.Unmarshal(target, payload); err != nil {
return nil, err
}
target.CensorPass() // type knows what fields to censor
return json.Marshal(target) // return the data censored & marshalled
}
But now, we can add new types to this Censored constraint very easily:
type Login struct {
Credentials // username pass
FullName
}
type Person struct {
FullName
Status string `json:"status"`
}
func (n *NewType) CensorPass() {
n.Credentials.CensorPass()
n.FullName.CensorPass()
}
With these couple of lines of code, we now have 2 more types that can be passed in to the CensorPayload function (by updating the constraint, of course). Given a Login and Person payload like:
// login
{
"user": "myLogin",
"pass": "weakpass",
"name": "Pothering Smythe",
"first_name": "Emanual",
}
// person
{
"name": "Doe",
"first_name": "John",
"status": "missing",
}
We should get the output:
{
"user": "<redacted>",
"pass": "******",
"name": "P. S. "
"first_name": "E",
}
{
"name": "D. ",
"first_name": "J",
"status": "missing",
}
Note:
I've written all of the code above without testing it out first. There could by typo's and mistakes here and there, but it should contain enough useful information and examples to get you started. If I have some spare time, I'll probably give this a whirl and update the snippets where needed

Gin Gonic Custom Error Message Fails When Invalid Data Sent

Validator struct
type RegisterValidator struct {
Name string `form:"name" json:"name" binding:"required,min=4,max=50"`
Email string `form:"email" json:"email" binding:"required,email,min=4,max=50"`
Password string `form:"password" json:"password" binding:"required,min=8,max=50"`
MobileCountryCode int `form:"mobile_country_code" json:"mobile_country_code" binding:"required,gte=2,lt=5"`
Mobile int `form:"mobile" json:"mobile" binding:"required,gte=5,lt=15"`
UserModel users.User `json:"-"`
}
Formatting a custom error as below:
type CustomError struct {
Errors map[string]interface{} `json:"errors"`
}
func NewValidatorError(err error) CustomError {
res := CustomError{}
res.Errors = make(map[string]interface{})
errs := err.(validator.ValidationErrors)
for _, v := range errs {
param := v.Param()
field := v.Field()
tag := v.Tag()
if param != "" {
res.Errors[field] = fmt.Sprintf("{%v: %v}", tag, param)
} else {
res.Errors[field] = fmt.Sprintf("{key: %v}", tag)
}
}
return res
}
works when data sent is
{
"email": "me#example.com",
"name": "John Doe",
"mobile_country_code": 1,
"mobile": 1234567
}
but sending an invalid type
{
"email": "me#example.com",
"name": "John Doe",
"mobile_country_code": "1",
"mobile": 1234567
}
throws the error interface conversion: error is *json.UnmarshalTypeError, not validator.ValidationErrors
This question is related to this one: How to assert error type json.UnmarshalTypeError when caught by gin c.BindJSON
however the answers do not make sense.
As the exception suggests, the following line failed type conversion
errs := err.(validator.ValidationErrors)
A different type of error must have got passed into the function that is not validator.ValidationErrors.
So either make sure other errors won't be passed into NewValidatorError. Or do a safer type check like:
errs, ok := err.(validator.ValidationErrors)
if !ok {
// handles other err type
}
More info: A Tour of Go - type assertions, A Tour of Go - type switches
I added a check for the UnmarshalTypeError as below:
if reflect.TypeOf(err).Elem().String() == "json.UnmarshalTypeError" {
errs := err.(*json.UnmarshalTypeError)
res.Errors[errs.Field] = fmt.Sprintf("{key: %v}", errs.Error())
return res
}
errs := err.(validator.ValidationErrors)
I guess Golang is strict when json is type-hinted. It has to be the exact type otherwise it will throw an UnmarshalTypeError error.

Go-Gin binding data with one-to-many relationship

I'm new to Golang and Gin framework, I have created two models
type Product struct {
gorm.Model
Name string
Media []Media
}
type Media struct {
gorm.Model
URI string
ProductID uint
}
and I send a POST request to save a new product, the body was:
{
"name": "Product1",
"media": [
"https://server.com/image1",
"https://server.com/image2",
"https://server.com/image3",
"https://server.com/video1",
"https://server.com/video2"
]
}
And I save a new product using this code
product := Product{}
if err := context.ShouldBindJSON(product); err != nil { // <-- here the error
context.String(http.StatusBadRequest, fmt.Sprintf("err: %s", err.Error()))
return
}
tx := DB.Create(&product)
if tx.Error != nil {
context.String(http.StatusBadRequest, fmt.Sprintf("err: %s", tx.Error))
return
}
the return error message is
err: json: cannot unmarshal string into Go struct field Product.Media of type models.Media
I know that ShouldBindJSON can't convert media-string to media-object, but what is the best practice to do this?
Your payload doesn't match the model. In the JSON body, media is an array of strings, whereas in the model it's a struct with two fields and the embedded gorm model.
If you can't change anything of your current setup, implement UnmarshalJSON on Media and set the URI field from the raw bytes. In the same method you may also initialize ProductID to something (if needed).
func (m *Media) UnmarshalJSON(b []byte) error {
m.URI = string(b)
return nil
}
Then the binding will work as expected:
product := Product{}
// pass a pointer to product
if err := context.ShouldBindJSON(&product); err != nil {
// handle err ...
return
}
fmt.Println(product) // {Product1 [{"https://server.com/image1" 0} ... }

combining GO GIN-GONIC GORM and VALIDATOR.V2

I am quite new to Go and I would like to startup by setting a GIN-GONIC API. I found this tutorial and I am very happy with that skeleton. But now I am stuck with the validating process which I added: "gopkg.in/validator.v2" and
type Todo struct {
gorm.Model
Title string `json:"title"`
Completed int `json:"completed"`
}
became
type Todo struct {
gorm.Model
Title string `json:"title" **validate:"size:2"**`
Completed int `json:"completed"`
}
and then in the CreateTodo function which I added :
if errs := validator.Validate(todo); errs!=nil {
c.JSON(500, gin.H{"Error": errs.Error()})
}
but then a POST call send :
"Error": "Type: unknown tag"
after some research I found that :
Using a non-existing validation func in a field tag will always return false and with error validate.ErrUnknownTag.
so the **validate:"size:2"** must be wrong ...
I don't get how to set the validation and also how to display the correct error within the "catch":
c.JSON(500, gin.H{"Error": errs.Error()})
Looks like you haven't defined size validation function. Also you can do it.
Custom validation functions:
func size(v interface{}, param int) error {
st := reflect.ValueOf(v)
if st.Kind() != reflect.String {
return validate.ErrUnsupported
}
if utf8.RuneCountInString(st.String()) != param {
return errors.New("Wrong size")
}
return nil
}
validate.SetValidationFunc("size", size)

Resources