Validator struct
type RegisterValidator struct {
Name string `form:"name" json:"name" binding:"required,min=4,max=50"`
Email string `form:"email" json:"email" binding:"required,email,min=4,max=50"`
Password string `form:"password" json:"password" binding:"required,min=8,max=50"`
MobileCountryCode int `form:"mobile_country_code" json:"mobile_country_code" binding:"required,gte=2,lt=5"`
Mobile int `form:"mobile" json:"mobile" binding:"required,gte=5,lt=15"`
UserModel users.User `json:"-"`
}
Formatting a custom error as below:
type CustomError struct {
Errors map[string]interface{} `json:"errors"`
}
func NewValidatorError(err error) CustomError {
res := CustomError{}
res.Errors = make(map[string]interface{})
errs := err.(validator.ValidationErrors)
for _, v := range errs {
param := v.Param()
field := v.Field()
tag := v.Tag()
if param != "" {
res.Errors[field] = fmt.Sprintf("{%v: %v}", tag, param)
} else {
res.Errors[field] = fmt.Sprintf("{key: %v}", tag)
}
}
return res
}
works when data sent is
{
"email": "me#example.com",
"name": "John Doe",
"mobile_country_code": 1,
"mobile": 1234567
}
but sending an invalid type
{
"email": "me#example.com",
"name": "John Doe",
"mobile_country_code": "1",
"mobile": 1234567
}
throws the error interface conversion: error is *json.UnmarshalTypeError, not validator.ValidationErrors
This question is related to this one: How to assert error type json.UnmarshalTypeError when caught by gin c.BindJSON
however the answers do not make sense.
As the exception suggests, the following line failed type conversion
errs := err.(validator.ValidationErrors)
A different type of error must have got passed into the function that is not validator.ValidationErrors.
So either make sure other errors won't be passed into NewValidatorError. Or do a safer type check like:
errs, ok := err.(validator.ValidationErrors)
if !ok {
// handles other err type
}
More info: A Tour of Go - type assertions, A Tour of Go - type switches
I added a check for the UnmarshalTypeError as below:
if reflect.TypeOf(err).Elem().String() == "json.UnmarshalTypeError" {
errs := err.(*json.UnmarshalTypeError)
res.Errors[errs.Field] = fmt.Sprintf("{key: %v}", errs.Error())
return res
}
errs := err.(validator.ValidationErrors)
I guess Golang is strict when json is type-hinted. It has to be the exact type otherwise it will throw an UnmarshalTypeError error.
Related
I'm trying to add string "Employee" to my existing JSON response. Also, we need to be able to generate this version of json based on an user condition. Only if the user condition is met, I need to generate second version of json with string "Employee" added. If not the first version without string "Employee" should be generated. How can I achieve it with out updating the existing struct and how can I check this with if clause to check for the condition and then generate json based on it?
Below is my existing json response in go
[
{
"EmpId":{
"String":"ABCD",
"Valid":true
},
"Department":{
"Float64":0,
"Valid":true
}
}
]
How can I get my json response like below with out changing existing struct based on input parameter?
{
"Employee":[
{
"EmpId":{
"String":"ABCD",
"Valid":true
},
"Department":{
"Float64":0,
"Valid":true
}
}
]
}
Below is my code:
Step 1: model folder
type EmployeeWithRoot struct {
Employee []Employee
}
type Employee struct {
EmpNbr sql.NullString `json:"EmpNbr"`
DateofJoin sql.NullString `json:"DateofJoin"`
DeptId sql.NullString `json:"DeptId"`
DeptName sql.NullString `json:"DeptName"`
}
Step 2: code folder
func GetEmpDetails(logRequestId string, logNestedLevel int, EmpNbr string, DateofJoin string) ([]model.EmployeeWithRoot, error) {
logFunctionFunctionName := "code.GetEmpDetails"
logStartTime := time.Now()
logNestedLevel++
defer configurations.TimeTrack(logFunctionFunctionName, logRequestId, logStartTime, logNestedLevel)
rows, err := db.Query(utils.SELECT_OF_EMP_AGGR, EmpNbr, DateofJoin, DeptId, DeptName)
if err != nil {
return nil, err
}
defer rows.Close()
var e []model.EmployeeWithRoot
for rows.Next() {
var x model.EmployeeWithRoot
err := rows.Scan(&x.Employee.EmpNbr, &x.Employee.DateofJoin, &x.Employee.DeptId,&x.Employee.DeptName)
if err != nil {
return nil, err
}
e = append(e, x)
}
err = rows.Err()
if err != nil {
return nil, err
}
return e, nil
}
STEP 3: API folder
Employee, err := code.GetEmpDetails(logRequestId, logNestedLevel, EmpNbr, DateofJoin)
if err != nil {
log.Panic(err)
}
marshalDataForRequestContentType(logRequestId, logNestedLevel, w, r, Employee)
I'm getting the below error.
x.Employee.EmpNbr undefined (type []model.Employee has no field or method EmpNbr)
x.Employee.DateofJoin undefined (type []model.Employee has no field or method DateofJoin)enter code here
x.Employee.DeptId undefined (type []model.Employee has no field or method DeptId)
x.Employee.DeptName undefined (type []model.Employee has no field or method DeptName)
Considering you're just wrapping it it an outer object, I don't see any reason you'd need to change the existing struct, just wrap it in a new one. I'll have to make some guesses/assumptions here since you've only shown the JSON and not the Go code that produces it, but assuming your existing JSON is produced by marshaling something like var response []Employee, the desired JSON could be produced in your condition by marshaling instead:
json.Marshal(struct{Employee []Employee}{response})
Working example: https://go.dev/play/p/vwDvxnQ96G_2
Use string concatenation:
func addRoot(json string) string {
return `{ "Employee":` + json + `}`
}
Run an example on the GoLang playground.
Here's the code if you are working with []byte instead of string:
func addRoot(json []byte) []byte {
const prefix = `{ "Employee":`
const suffix = `}`
result := make([]byte, 0, len(prefix)+len(json)+len(suffix))
return append(append(append(result, prefix...), json...), suffix...)
}
Run this example on the GoLang playground.
If you have some JSON in a byte slice ([]byte) then you can just add the outer element directly - e.g. (playground):
existingJSON := []byte(`[
{
"EmpId":{
"String":"ABCD",
"Valid":true
},
"Department":{
"Float64":0,
"Valid":true
}
}
]`)
b := bytes.NewBufferString(`{"Employee":`)
b.Write(existingJSON)
b.WriteString(`}`)
fmt.Println(string(b.Bytes()))
If this is not what you are looking for please add further details to your question (ideally your attempt as a minimal, reproducible, example)
I'm new to Go, so this might be very easy, but I can't find it. I have an entity Page with these two properties:
type Page struct {
Title string `form:"title" binding:"required"`
Active bool
}
Now, if you don't send a title we get a (not very pretty, but acceptable*):
Key: 'Page.Title' Error:Field validation for 'Title' failed on the 'required' tag.
Now, if I send this to the endpoint:
{
"title": "I'm a valid title",
"active": "I'm not a boolean at all!"
}
We get this:
json: cannot unmarshal string into Go struct field Page.Active of type bool
Which, IMO, is giving way too much info. What is the standard practice in Go to validate user input?
I was first making a page-validor.go with some checks, then I found this, but I'm not sure what is good practice in Go.
How do I validate this properly? Should I find check what is provided and then try to move it into the struct and validate the actual contents?
I am using GinGonic
* I've found a way to unwrap the errors and make it nicer
Write custom JSON Unmarshaller method for the type Page and inside UnmarshalJSON(bytes []byte) method, you can unmarshal the JSON bytes to map[string]interface{} and then validate the types you need with the JSON field keys.
An example of the JSON Unmarshaller looks like below.
type Page struct {
Title string `form:"title" binding:"required"`
Active bool
}
func (p *Page) UnmarshalJSON(bytes []byte) error {
var data map[string]interface{}
err := json.Unmarshal(bytes, &data)
if err != nil {
return err
}
actv, _ := data["active"]
if reflect.TypeOf(actv).Kind() != reflect.Bool {
return errors.New("active field should be a boolean")
}
p.Active = actv.(bool)
return nil
}
See the full example here in Playground.
After some more research, I've implemented Go-map-schema.
var page Page
src := make(map[string]interface{})
json.Unmarshal(jsonData, &src)
results, _ := schema.CompareMapToStruct(page, src, nil)
fmt.Println(results.MissingFields)
fmt.Println(results.MismatchedFields)
This works simple with the standard notations for an struct:
type Page struct {
Title string `json:"title" validator:"required"`
Active bool `json:"metaRobotsFollow" validate:"required,bool"`
}
You should use validator v10 available with Go
validator documentation go
For your use case you can use boolean validator
https://pkg.go.dev/github.com/go-playground/validator/v10#hdr-Boolean
type Page struct {
Title string `json:"title" binding:"required"`
Active bool `json:"active" binding:"required,boolean"`
}
Below is sample Test case with one positive and negative
func TestPage(t *testing.T) {
tests := []struct {
testName string
input string
wantErr bool
}{
{
testName: "positive test",
input: `{
"title": "first book title",
"active": false
}`,
wantErr: false,
}, {
testName: "wrong boolean",
input: `{
"title": "second book title",
"active": falsee
}`,
wantErr: false,
},
}
for _, tt := range tests {
t.Run(tt.input, func(t *testing.T) {
var p Page
b := []byte(tt.input)
err := json.Unmarshal(b, &p)
assert.Nil(t, err, "got error %v", err)
})
}
I am trying to create a CRUD and validating my request body using a Go library called [validator][1]
Validation Code sample:
func (v *Validation) Validate(i interface{}) ValidationErrors {
errs := v.validate.Struct(i).(validator.ValidationErrors) // panic originates here
if len(errs) == 0 {
return nil
}
var returnErrs []ValidationError
for _, err := range errs {
// cast the FieldError into our ValidationError and append to the slice
ve := ValidationError{err.(validator.FieldError)}
returnErrs = append(returnErrs, ve)
}
return returnErrs
}
The above validator works for invalid request body such as invalid ID.
But for a valid body, it initiates a panic
Stack Trace:
products-api 2020/07/12 15:15:11 http: panic serving 127.0.0.1:33288: interface conversion: error is nil, not validator.ValidationErrors
goroutine 21 [running]:
net/http.(*conn).serve.func1(0xc0003b6140)
/usr/local/go/src/net/http/server.go:1772 +0x139
panic(0x93d6c0, 0xc0003845d0)
/usr/local/go/src/runtime/panic.go:973 +0x3e3
github.com/AymanArif/golang-microservices/data.(*Validation).Validate(0xc0005a8108, 0x8f3480, 0xc0005aa2c0, 0x0, 0x0, 0x203000)
/home/ayman/Desktop/golang-microservices/data/validation.go:70 +0x211
interface conversion: interface conversion: error is nil, not validator.ValidationErrors
Create REST Logic:
func (p *Products) Create(rw http.ResponseWriter, r *http.Request) {
prod := r.Context().Value(KeyProduct{}).(data.Product) // Panic originate here. Check below for struct definiton
p.l.Printf("[DEBUG] Inserting product: %#v\n", prod)
data.AddProduct(prod)
}
// data.Product
type Product struct {
ID int `json:"id"` // Unique identifier for the product
Name string `json:"name" validate:"required"`
Description string `json:"description"`
SKU string `json:"sku" validate:"sku"`
}
How can I do error handling for correct requests?
[1]: https://github.com/go-playground/validator
The error you got is eloquent:
interface conversion: interface {} is *data.Product, not data.Product
The line r.Context().Value(KeyProduct{}) returns an interface type interface{}, which, the error tells you, is holding a value of concrete type *data.Product (pointer to data.Product)
Instead, you are attempting to convert it to a data.Product, without checking whether the conversion is valid.
Replace the line with:
prod := r.Context().Value(KeyProduct{}).(*data.Product)
You might want to read the Go Tour about type assertions.
After your update, the error you have now is still the same kind of issue:
interface conversion: error is nil, not validator.ValidationErrors
With the expression err.(validator.FieldError) you are attempting to convert err to a validator.FieldError when err is in fact nil.
The previous check len(errs) == 0 is only verifying that errs length is not zero, but the slice might be of non-zero length and contain nil values.
You can instead test the type assertion:
if fe, ok := err.(validator.FieldError); ok {
ve := ValidationError{fe}
returnErrs = append(returnErrs, ve)
}
If you re write the Validate function to something like this it should work. Type assertion needs to be checked for errs & err
func (v *Validation) Validate(i interface{}) ValidationErrors {
var returnErrs []ValidationError
if errs, ok := v.validate.Struct(i).(validator.ValidationErrors); ok {
if errs != nil {
for _, err := range errs {
if fe, ok := err.(validator.FieldError); ok {
ve := ValidationError{fe}
returnErrs = append(returnErrs, ve)
}
}
}
}
return returnErrs
}
Requirements
Two services:
Server - for writing blog posts to MongoDB
Client - sends request to the first service
The blog post has title of type string, and content which is a dynamic type - can be any JSON value.
Protobuf
syntax = "proto3";
package blog;
option go_package = "blogpb";
import "google/protobuf/struct.proto";
message Blog {
string id = 1;
string title = 2;
google.protobuf.Value content = 3;
}
message CreateBlogRequest {
Blog blog = 1;
}
message CreateBlogResponse {
Blog blog = 1;
}
service BlogService {
rpc CreateBlog (CreateBlogRequest) returns (CreateBlogResponse);
}
Let's start with protobuf message, which meats requirements - string for title and any JSON value for content.
Client
package main
import (...)
func main() {
cc, _ := grpc.Dial("localhost:50051", grpc.WithInsecure())
defer cc.Close()
c := blogpb.NewBlogServiceClient(cc)
var blog blogpb.Blog
json.Unmarshal([]byte(`{"title": "First example", "content": "string"}`), &blog)
c.CreateBlog(context.Background(), &blogpb.CreateBlogRequest{Blog: &blog})
json.Unmarshal([]byte(`{"title": "Second example", "content": {"foo": "bar"}}`), &blog)
c.CreateBlog(context.Background(), &blogpb.CreateBlogRequest{Blog: &blog})
}
The client sends two requests to the server - one with content having string type, and other with the object. No errors here.
Server
package main
import (...)
var collection *mongo.Collection
type server struct {
}
type blogItem struct {
ID primitive.ObjectID `bson:"_id,omitempty"`
Title string `bson:"title"`
Content *_struct.Value `bson:"content"`
}
func (*server) CreateBlog(ctx context.Context, req *blogpb.CreateBlogRequest) (*blogpb.CreateBlogResponse, error) {
blog := req.GetBlog()
data := blogItem{
Title: blog.GetTitle(),
Content: blog.GetContent(),
}
// TODO: convert "data" or "data.Content" to something that could be BSON encoded..
res, err := collection.InsertOne(context.Background(), data)
if err != nil {
log.Fatal(err)
}
oid, _ := res.InsertedID.(primitive.ObjectID)
return &blogpb.CreateBlogResponse{
Blog: &blogpb.Blog{
Id: oid.Hex(),
Title: blog.GetTitle(),
Content: blog.GetContent(),
},
}, nil
}
func main() {
client, _ := mongo.NewClient(options.Client().ApplyURI("mongodb://localhost:27017"))
client.Connect(context.TODO())
collection = client.Database("mydb").Collection("blog")
lis, _ := net.Listen("tcp", "0.0.0.0:50051")
s := grpc.NewServer([]grpc.ServerOption{}...)
blogpb.RegisterBlogServiceServer(s, &server{})
reflection.Register(s)
go func() { s.Serve(lis) }()
ch := make(chan os.Signal, 1)
signal.Notify(ch, os.Interrupt)
<-ch
client.Disconnect(context.TODO())
lis.Close()
s.Stop()
}
And here I get:
cannot transform type main.blogItem to a BSON Document: no encoder
found for structpb.isValue_Kind
What do I expect? To see the exact value of content in MongoDB, something like this:
{ "_id" : ObjectId("5e5f6f75d2679d058eb9ef79"), "title" : "Second example", "content": "string" }
{ "_id" : ObjectId("5e5f6f75d2679d058eb9ef78"), "title" : "First example", "content": { "foo": "bar" } }
I guess I need to transform data.Content in the line where I added TODO:...
I can create github repo with this example if that could help.
So as suggested by #nguyenhoai890 in the comment I managed to fix it using jsonpb lib - first MarshalToString to covert from structpb to string(json) and then json.Unmarshal to convert from string(json) to interface{} which is supported by BSON. Also I had to fix a Client to correctly unmarshal from string to protobuf.
Client
func main() {
cc, _ := grpc.Dial("localhost:50051", grpc.WithInsecure())
defer cc.Close()
c := blogpb.NewBlogServiceClient(cc)
var blog blogpb.Blog
jsonpb.UnmarshalString(`{"title": "Second example", "content": {"foo": "bar"}}`, &blog)
c.CreateBlog(context.Background(), &blogpb.CreateBlogRequest{Blog: &blog})
jsonpb.UnmarshalString(`{"title": "Second example", "content": "stirngas"}`, &blog)
c.CreateBlog(context.Background(), &blogpb.CreateBlogRequest{Blog: &blog})
}
Server
type blogItem struct {
ID primitive.ObjectID `bson:"_id,omitempty"`
Title string `bson:"title"`
Content interface{} `bson:"content"`
}
func (*server) CreateBlog(ctx context.Context, req *blogpb.CreateBlogRequest) (*blogpb.CreateBlogResponse, error) {
blog := req.GetBlog()
contentString, err := new(jsonpb.Marshaler).MarshalToString(blog.GetContent())
if err != nil {
log.Fatal(err)
}
var contentInterface interface{}
json.Unmarshal([]byte(contentString), &contentInterface)
data := blogItem{
Title: blog.GetTitle(),
Content: contentInterface,
}
res, err := collection.InsertOne(context.Background(), data)
if err != nil {
log.Fatal(err)
}
oid, _ := res.InsertedID.(primitive.ObjectID)
return &blogpb.CreateBlogResponse{
Blog: &blogpb.Blog{
Id: oid.Hex(),
Title: blog.GetTitle(),
Content: blog.GetContent(),
},
}, nil
}
This question already has an answer here:
Return custom error message from struct tag validation
(1 answer)
Closed 1 year ago.
For example I've got the following struct
type Address struct {
City string `json:"city" binding:"required"`
AddressLine string `json:"address_line" binding:"required"`
}
and I've got the following function to handle request from users
func AddressCreate(c *gin.Context) {
var address Address
if err := c.BindJSON(&address); err == nil {
// if everything is good save to database
// and return success message
db.Create(&address)
c.JSON(http.StatusOK, gin.H {"status":"success"})
} else {
c.JSON(http.StatusBadRequest, err)
}
}
Expected behavior is to return JSON, formatted this way
[
{
"city":"required"
}
{
"address_line":"required"
}
]
But I'm getting an error formatted like this
"Address.City": {
"FieldNamespace": "Address.City",
"NameNamespace": "City",
"Field": "City",
"Name": "City",
"Tag": "required",
"ActualTag": "required",
"Kind": 24,
"Type": {},
"Param": "",
"Value": ""
},
"Address.AddressLine": {
"FieldNamespace": "AddressLine",
"NameNamespace": "AddressLine",
"Field": "AddressLine",
"Name": "AddressLine",
"Tag": "required",
"ActualTag": "required",
"Kind": 24,
"Type": {},
"Param": "",
"Value": ""
}
What I tried:
I created function which casts error to ValidationErrors and iterates through all FieldError's in it
func ListOfErrors(e error) []map[string]string {
ve := e.(validator.ValidationErrors)
InvalidFields := make([]map[string]string, 0)
for _, e := range ve {
errors := map[string]string{}
// field := reflect.TypeOf(e.NameNamespace)
errors[e.Name] = e.Tag
InvalidFields = append(InvalidFields, errors)
}
return InvalidFields
}
The output look's much better
[
{
"City":"required"
},
{
"AddressLine":"required"
}
]
But I cannot solve the problem with the name of the fields. I cannot swap Name into name which I noted in structs tag json:"city". So my question is did I choose correct way to solve the problem if the answer is yes how to get structs JSON tag for field?
If you want it to be same as defined in your json tag, then you should use reflection to pull that tag from your data type.
I don't have your libraries, so can't compile and check it. But I believe what you are after should go along those lines:
func ListOfErrors(address *Address, e error) []map[string]string {
ve := e.(validator.ValidationErrors)
InvalidFields := make([]map[string]string, 0)
for _, e := range ve {
errors := map[string]string{}
// field := reflect.TypeOf(e.NameNamespace)
field, _ := reflect.TypeOf(address).Elem().FieldByName(e.Name)
jsonTag := string(field.Tag.Get("json"))
errors[jsonTag] = e.Tag
InvalidFields = append(InvalidFields, errors)
}
return InvalidFields
}
Note that it is a bit contrived as type of address parameter is essentially known. So, not strictly required as a function parameter. But you can change address *Address to address interface{} and use it for other types too.
Disclaimer: I skipped error checking for brevity, but you certainly should check for errors in your code (e.g. no such field error or no json tag on that field).
You can use ToSnake to snake case the names:
import (
"unicode"
)
// ToSnake convert the given string to snake case following the Golang format:
// acronyms are converted to lower-case and preceded by an underscore.
func ToSnake(in string) string {
runes := []rune(in)
length := len(runes)
var out []rune
for i := 0; i < length; i++ {
if i > 0 && unicode.IsUpper(runes[i]) && ((i+1 < length && unicode.IsLower(runes[i+1])) || unicode.IsLower(runes[i-1])) {
out = append(out, '_')
}
out = append(out, unicode.ToLower(runes[i]))
}
return string(out)
}
func ListOfErrors(e error) []map[string]string {
ve := e.(validator.ValidationErrors)
invalidFields := make([]map[string]string, 0)
for _, e := range ve {
errors := map[string]string{}
errors[ToSnake(e.Name)] = e.Tag
invalidFields = append(InvalidFields, errors)
}
return invalidFields
}