GIN problems with posted data validation - go

I'm currently learning GO and I wanted to replicate one of the services we are running in production using GO for training. I'm working with the framework GIN and I need to validate a simple struct like this. It is posted to the search endpoint
type SearchData struct {
Field1 string `json:"field1,omitempty"`
Field2 string `json:"field2,omitempty"`
Field3 string `json:"field,omitempty" binding:"alphanum"`
}
first problem: I can't check if Field3 != nil because is a string, one solution I found by reading online is that I can cast it as a pointer, the problem is that then I get an error from the binding alphanum because nil fail the validation.
second problem: I can remove the omitempty check but if I do so the alphanum validator fail and the fields are set to their default value "". I can't check if the user want to submit "" or is the bindingJSON method that is setting the default value.
By reading online I see that there isn't a proper solution to this problem (or maybe yes), as I'm new to GO I will be happy to see how other people handle this in published / production APIs.
EDIT: This is how I parse the json
func Search(c *gin.Context) {
db := c.MustGet("db").(*mgo.Database)
var json SearchData
if err := c.ShouldBindJSON(&json); err != nil {
fmt.Println(err)
utils.ReturnError400(c, "Invalid")
return
}
// the json is used here
}

Gin return an empty string if the parameter was not provided.
You can use the following method in order to verify if the string is provided:
// IsBlank is delegated to verify that the does not contains only empty char
func IsBlank(str string) bool {
// Check length
if len(str) > 0 {
// Iterate string
for i := range str {
// Check about char different from whitespace
if str[i] > 32 {
return false
}
}
}
return true
}
You can read some useful utils method in this repository in order to validate string and execute common task on string:
https://github.com/alessiosavi/GoGPUtils/blob/master/string/stringutils.go
For the second problem, you can use the DefaultQuery in order to set a predefined value if no one are provided, here an example
func get() gin.HandlerFunc {
fn := func(c *gin.Context) {
// firstname will be "NOT_PRESENT" if the user does not send the parameter
firstname := c.DefaultQuery("test", "NOT_PRESENT")
lastname := c.Query("test1")
fmt.Println(firstname, lastname)
}
return gin.HandlerFunc(fn)
}
Here the documentation about DefaultQuery:
https://godoc.org/github.com/gin-gonic/gin#Context.DefaultQuery

Related

Why do I get "Composite Literal Uses Unkeyed" error?

I'm relatively new to Go and am working on building out a request decoder. The request comes in JSON format and we decode that to a map[string]interface{}. We then pass that object data in to be decoded to our own ProcessRequest struct. As I said I'm new so I am reusing some logic in similar parts of the code wrote by previous developers. Essentially we are checking the map for the necessary pieces and then setting and returning those. Can someone explain to me why I am getting the titled error? Would I have to set the items all the way down to base structs that no longer have any nested? Is there a better way to accomplish what I want? Here is the code and the related structs. It is highlighting the error on the return of the model.ProcessRequest. TYIA
type ProcessRequest struct {
RequestID string
Message *message.Message
Rule *Rule
Options *ProcessOptions
//TODO: Context EvaluatorContext
//TODO: Links
}
type Message struct {
ID int
Key string
Source string
Headers *HeaderSet
Categories *CategorySet
Properties *PropertySet
Streams *StreamSet
}
type RuleAction struct {
Name string
Expression map[string]interface{}
}
type RuleLink struct {
LinkID int
Condition map[string]interface{}
TargetRuleID int
}
type Rule struct {
Name string
Code string
Actions []RuleAction
Links []RuleLink
}
type object = map[string]interface{}
func DecodeProcessRequest(dataObject map[string]interface{}) (*model.ProcessRequest, error) {
var (
requestID string
message *message.Message
rule *model.Rule
options *model.ProcessOptions
err error
)
if reqIDSrc, ok := dataObject["requestId"]; ok {
if requestID, err = converter.ToString(reqIDSrc); err != nil {
return nil, errors.Wrapf(err, "Error reading property 'requestID'")
}
if requestID == "" {
return nil, errors.Errorf("Property 'requestID' is an empty string")
}
} else {
return nil, errors.Errorf("Missing required property 'requestID'")
}
if messageSrc, ok := dataObject["message"]; ok {
messageData, ok := messageSrc.(object)
if !ok {
return nil, errors.Errorf("Error reading property 'message': Value is not an object")
}
if message, err = DecodeMessage(messageData); err != nil {
return nil, errors.Wrapf(err, "Error reading property 'message'")
}
} else {
return nil, errors.Errorf("Missing required property 'message'")
}
if ruleSrc, ok := dataObject["rule"]; ok {
ruleObj, ok := ruleSrc.(object)
if !ok {
return nil, errors.Errorf("Error reading property 'rule': Value is not an object")
}
if rule, err = DecodeRule(ruleObj); err != nil {
return nil, errors.Wrapf(err, "Error reading 'rule' during decoding")
}
} else {
return nil, errors.Errorf("Missing required property 'requestID'")
}
// Parse plain object to a Message struct
return &model.ProcessRequest{
requestID,
message,
rule,
options,
}, nil
}
super said in this comment:
In general, the warning says that you should prefer to use the syntax ProcessRequest{ RequestID: requestID, ... }. Naming the keys instead of unkeyed values.
That worked for me. Also the explanation by kostix in this comment really helped.
Basically the idea is that if you use "unkeyed" way of defining struct literals, the meaning of your definitions depends on the way the fields of the underlying type are layed out. Now consider that your type has three fields of type string in a certain order. Now a couple of iterations down the road some programmer moves the second field to the 1st position—your literal will still compile but will end up defining a completely different value at runtime.

Bind Query inside middleware

I'm trying to write a "Binder" middleware that will validate any request query using a struct type with gin bindings/validators
So for example, let's say I have an endpoint group called /api/subject which requires the query string to have a subject code and an ID that will be validated using the following struct (called entity.Subject):
type Subject struct {
Code string `binding:"required,alphanum"`
ID string `binding:"required,alphanum,len=4"`
}
That's just one example, but I'd like to be able to pass any struct type to this middleware, because I'd like to access the query data on future handlers without worrying about query validation.
So I tried something like this:
func Binder(t reflect.Type) gin.HandlerFunc {
return func(c *gin.Context) {
obj := reflect.New(t).Elem().Interface()
if err := c.BindQuery(&obj); err != nil {
c.AbortWithStatus(http.StatusBadRequest)
return
}
c.Set(t.Name(), obj)
}
}
And added this middleware like so:
apiGroup := router.Group("/api")
{
// other subgroups/endpoints
// ...
subjectGroup := apiGroup.Group("/subject", middleware.Binder(reflect.TypeOf(entity.Subject{})))
}
And later on, in another handler function, let's say GetSubject, I want to access the subject data passed by doing c.MustGet("Subject").(entity.Subject)
But this isn't working =(, when I print obj, it's just an empty interface, how would I do this?
I managed to do something similar!
I created the following middleware
var allowedTypes = []binding.Binding{
binding.Query,
binding.Form,
binding.FormPost,
binding.FormMultipart,
}
func Bind(name string, data interface{}, bindingType binding.Binding) gin.HandlerFunc {
return func(ctx *gin.Context) {
ok := false
for _, b := range allowedTypes {
if b == bindingType {
ok = true
}
}
if !ok {
ctx.AbortWithError(
http.StatusInternalServerError,
fmt.Errorf("Bind function only allows %v\n", allowedTypes),
)
}
_ = ctx.MustBindWith(data, bindingType)
ctx.Set(name, data)
}
}
Remember to pass a pointer to your desired type in the call, like so:
router.GET("/something", Bind("Object", &myObject, binding.Query))
I restricted only to a few binding types because they allow ShouldBind to be called multiple times, whereas JSON, XML and others consume the Request body.
This way you can pass multiple Bind middlewares and if the validation fails it automatically aborts with http.StatusBadRequest

Chaincode GetState returns an empty response

func (t *ballot) initBallot(stub shim.ChaincodeStubInterface, args []string) peer.Response {
if len(args) != 2 {
return shim.Error("Incorrect number of arguments. Expecting 2")
}
// ==== Input sanitation ====
fmt.Println("- start init ballot")
if len(args[0]) == 0 {
return shim.Error("1st argument must be a non-empty string")
}
if len(args[1]) == 0 {
return shim.Error("2nd argument must be a non-empty string")
}
personFirstName := args[0]
personLastName := args[1]
hash := sha256.New()
hash.Write([]byte(personFirstName + personLastName)) // ballotID is created based on the person's name
ballotID := hex.EncodeToString(hash.Sum(nil))
voteInit := "VOTE INIT"
// ==== Create ballot object and marshal to JSON ====
Ballot := ballot{personFirstName, personLastName, ballotID, voteInit}
ballotJSONByte, err := json.Marshal(Ballot)
if err != nil {
return shim.Error(err.Error())
}
err = stub.PutState(string(ballotID), ballotJSONByte)
//FIXME:0-------------------------------------------------
ballotAsByte, err := stub.GetState(string(ballotID))
if err != nil {
return shim.Error(err.Error())
}
BBBallot := ballot{}
//umarshal the data to a new ballot struct
json.Unmarshal(ballotAsByte, &BBBallot)
//
fmt.Println(BBBallot)
fmt.Println(BBBallot.personFirstName)
return shim.Success([]byte(ballotID))
}
Above is the code and this is the test script i am running it against
func Test_Invoke_initBallot(t *testing.T) {
scc := new(ballot)
stub := shim.NewMockStub("voting", scc)
res := stub.MockInvoke("1", [][]byte{[]byte("initBallot"), []byte("John"), []byte("C")})
if res.Status != shim.OK {
t.Log("bad status received, expected: 200; received:" + strconv.FormatInt(int64(res.Status), 10))
t.Log("response: " + string(res.Message))
t.FailNow()
}
if res.Payload == nil {
t.Log("initBallot failed to create a ballot")
t.FailNow()
}
}
I am trying to read from the ledger after putting the transaction in. However, I have been getting empty responses from both of the Println statements.
// PutState puts the specified `key` and `value` into the transaction's
// writeset as a data-write proposal. PutState doesn't effect the ledger
// until the transaction is validated and successfully committed.
// Simple keys must not be an empty string and must not start with null
// character (0x00), in order to avoid range query collisions with
// composite keys, which internally get prefixed with 0x00 as composite
// key namespace.
PutState(key string, value []byte) error
It does say on the documentation that putState does not commit transactions to the ledger until its validated, but I am just trying to test my chaincode using the MockStub without setting up the fabric network. What is the fix to this problem?
P.S the problem has been solved, here is the right way to set up a struct
type ballot struct {
PersonFirstName string
PersonLastName string
BallotID string
VoteInit string
}
You haven't provided the code for the ballot struct yet. But from what you provided, I have a guess what might be going on. I think you probably haven't exported the fields and your struct looks like this:
type ballot struct {
personFirstName string
personLastName string
ballotID string
voteInit string
}
But when you tried to convert this object to JSON using json.Marshal(Ballot), none of the fields are added to the JSON object because they were not exported. All that you have to do in this case is exporting the necessary fields (using Uppercase letter at the beginning of field names). Your updated struct should look something like the following:
type ballot struct {
PersonFirstName string
PersonLastName string
BallotID string
VoteInit string
}
This is a very common mistake many newcomers make. Wish you all the best in your journey forward!!!
P.S. Please edit your question and add the code of you ballot struct here even if this solution solves your problem as that might help others in the future. Also, please add proper indentation to the code and add the last } symbol in the code block.

create array of struct from by decoding a JSON file in go

All I looking to do is to create an array of struct Response from a json encoded file.
the file that contains json data looks like this.
cat init.txt
{"events": [{"action":"cpr","id":69,"sha1":"abc","cpr":"cpr_data0"},{"action":"cpr","id":85,"sha1":"def","cpr":"cpr_data1"}]}
The way I have gone about approaching this is
I created a response of type map[string][]Response
.. decoded the JSON from the file
.. created a responseStruct of type []Response
But somehow when I inspect the Value they all look 0 or empty
map[events:[{ 0 } { 0 }]
What is wrong with the approach mentioned above.
type Response struct {
action string `json:"action"`
id int64 `json:"id"`
sha1 string `json:"sha1"`
cpr string `json:"cpr"`
}
func main() {
file, err := os.Open("init.txt")
if err != nil {
fmt.Println(err)
os.Exit(1)
}
var response map[string][]Response
err = json.NewDecoder(file).Decode(&response)
if err != nil {
fmt.Println(err)
os.Exit(1)
}
var responseArray []Response
responseArray = response["events"]
for _, responseStruct := range responseArray {
fmt.Println("id =", responseStruct.id)
fmt.Println("action =", responseStruct.action)
fmt.Println("sha1 = ", responseStruct.sha1)
fmt.Println("cpr =", responseStruct.cpr)
fmt.Println("==========")
}
fmt.Println(response)
}
Well If I modify the struct to look like this it works
type Response struct {
Action string `json:"action"`
ID int64 `json:"id"`
Sha1 string `json:"sha1"`
Cpr string `json:"cpr"`
}
So my question is this how the stuff would work, can't I get the above code to work in the way it is?
Go has the notion of public and private fields in objects, and the only distinction is whether they start with an initial capital letter or not. This applies to not just code modules but also the reflect package and things that use it. So in your initial version
type Response struct {
action string `json:"action"`
...
}
nothing outside your source package, not even the "encoding/json" module, can see the private fields, so it can't fill them in. Changing these to public fields by capitalizing Action and the other field names makes them visible to the JSON decoder.
Lowercase struct elements in golang are private, Hence json decoder (which is an external package) can't access them.
Its able to create the struct object but not able to set values. They appear zero because they 0 is default value.

How to use Go's type alias to make own models work with protobufs?

I've got some REST API with my models defined as Go structs.
type User struct {
FirstName string
LastName string
}
Then I've got my database methods for getting data.
GetUserByID(id int) (*User, error)
Now I'd like to replace my REST API with https://github.com/twitchtv/twirp .
Therefore I started defining my models inside .proto files.
message User {
string first_name = 2;
string last_name = 3;
}
Now I've got two User types. Let's call them the native and the proto type.
I've also got a service defined in my .proto file which returns a user to the frontend.
service Users {
rpc GetUser(Id) returns (User);
}
This generates an interface that I have to fill in.
func (s *Server) GetUser(context.Context, id) (*User, error) {
// i'd like to reuse my existing database methods
u, err := db.GetUserByID(id)
// handle error
// do more stuff
return u, nil
}
Unfortunately this does not work. My database returns a native User but the interface requires a proto user.
Is there an easy way to make it work? Maybe using type aliases?
Thanks a lot!
One way you can solve your problem is by doing the conversion manually.
type User struct {
FirstName string
LastName string
}
type protoUser struct {
firstName string
lastName string
}
func main() {
u := db() // Retrieve a user from a mocked db
fmt.Println("Before:")
fmt.Printf("%#v\n", *u) // What db returns (*protoUser)
fmt.Println("After:")
fmt.Printf("%#v\n", u.AsUser()) // What conversion returns (User)
}
// Mocked db that returns pointer to protoUser
func db() *protoUser {
pu := protoUser{"John", "Dough"}
return &pu
}
// Conversion method (converts protoUser into a User)
func (pu *protoUser) AsUser() User {
return User{pu.firstName, pu.lastName}
}
The key part is the AsUser method on the protoUser struct.
There we simply write our custom logic for converting a protoUser into a User type we want to be working with.
Working Example
As #Peter mentioned in the comment section.
I've seen a project which made it with a custom Convert function. It converts the Protobuf to local struct via json.Unmarshal, not sure how's the performance but it's a way to go.
Preview Code PLAYGROUND
// Convert converts the in struct to out struct via `json.Unmarshal`
func Convert(in interface{}, out interface{}) error {
j, err := json.Marshal(in)
if err != nil {
return err
}
err = json.Unmarshal(j, &out)
if err != nil {
return err
}
return nil
}
func main() {
// Converts the protobuf struct to local struct via json.Unmarshal
var localUser User
if err := convert(protoUser, &localUser); err != nil {
panic(err)
}
}
Output
Before:
main.ProtoUser{FirstName:"John", LastName:"Dough"}
After:
main.User{FirstName:"John", LastName:"Dough"}
Program exited.

Resources