httptest.NewRequest issues with query parameters - go

I'm using the GIN framework with a Postgres DB and GORM as an ORM.
One of the routes accepts query parameters. When I search for this in my browser, I get the expected results: an array of json objects. Here is the route:
/artworks/?limit=10&last_id=1
However, when I try to test the handler used by that route, I get the following error:
routes_test.go:184: [ERROR] Unable to unmarshal data to artworks: json: cannot unmarshal object into Go value of type []models.Artwork
The query that the ORM is trying to run in the test function is the folowing:
SELECT * FROM "artwork_migrate_artwork" WHERE id = ''
So when I run the request in the browser, it properly pulls the query parameters and then the ORM runs the proper sql query. But when using httptest.NewRequest it seems like the query parameters are not used.
Here is my test function:
func TestGetArtworks(t *testing.T) {
dsn := fmt.Sprintf("host=%s user=%s password=%s dbname=%s port=%s sslmode=%s TimeZone=%s", env_var.Host, env_var.User, env_var.Password, env_var.DBname, env_var.Port, env_var.SSLMODE, env_var.TimeZone)
db, err := gorm.Open(postgres.Open(dsn), &gorm.Config{})
if err != nil {
panic("failed to connect to db")
}
route := "/artworks/"
handler := handlers.GetArtwork(db)
router := setupGetRouter(handler, route)
writer := httptest.NewRecorder()
req := httptest.NewRequest(http.MethodGet, "/artworks/?limit=10&last_id=1", nil)
fmt.Println(req)
router.ServeHTTP(writer, req)
assert.Equal(t, 200, writer.Code)
data, err := ioutil.ReadAll(writer.Body)
if err != nil {
t.Errorf("\u001b[31m[Error] Unable to read writer.Body: %s", err)
}
// no body can be unmarshalled
fmt.Println("Here is the body:", writer.Body.String())
var artworks []models.Artwork
if err := json.Unmarshal(data, &artworks); err != nil {
t.Errorf("\u001b[31m[ERROR] Unable to unmarshal data to artworks: %s", err)
}
assert.Equal(t, 10, len(artworks))
}
Here is my route handler:
func GetArtworks(db *gorm.DB) gin.HandlerFunc {
return func(c *gin.Context) {
limit, err := strconv.Atoi(c.Query("limit"))
if err != nil {
panic(err)
}
last_id := c.Query("last_id")
var artworks []models.Artwork
db.Where("id > ?", last_id).Limit(limit).Find(&artworks)
c.JSON(http.StatusOK, artworks)
}
}
router.GET("/artworks", han.GetArtworks(db))
Here is the model struct:
type Artwork struct {
ID int `json:"id"`
Title string `json:"title"`
Nationality string `json:"nationality"`
Artist_Bio string `json:"artist_bio"`
Desc string `json:"desc"`
Culture string `json:"culture"`
Gender string `json:"gender"`
Nation string `json:"nation"`
Medium string `json:"medium"`
Date_of_Release string `json:"date_of_release"`
Image string `json:"image"`
Image_Small string `json:"image_small"`
Last_Modified time.Time `json:"last_modified"`
Artist_ID int `json:"artist_id"`
Source_ID int `json:"source_id"`
}

#Brits is correct: It was due to a misspelled handler
handler := handlers.GetArtwork(db)
should have been handler := handlers.GetArtworks(db)

Related

Bigquery Entity Tag

When I loaded my data from Cloud Storage into Bigquery, the click_url field turned into Record type when get created even though it's a string (maybe because of the noindex not sure tho). When I try to insert the data into BigQuery using Inserter. I got this error message:
Cannot convert std::string to a record field:optional .Msg_0_CLOUD_QUERY_TABLE.Msg_1_CLOUD_QUERY_TABLE_click_url click_url = 1
Table in bigquery:
Schema:
Here's the code:
type Product struct {
Name string `datastore:"name" bigquery:"name"`
ClickUrl string `datastore:"click_url,noindex" bigquery:"click_url"`
DateAdded time.Time `datastore:"date_added" bigquery:"date_added"`
}
func insertRows(data interface{}) error {
projectID := "my-project-id"
datasetID := "mydataset"
tableID := "mytable"
ctx := context.Background()
client, err := bigquery.NewClient(ctx, projectID)
if err != nil {
return fmt.Errorf("bigquery.NewClient: %v", err)
}
defer client.Close()
inserter := client.Dataset(datasetID).Table(tableID).Inserter()
if err := inserter.Put(ctx, data); err != nil {
return err
}
return nil
}
func main() {
product := Product{"product_name", "click_url", "date_added_value"} // Example data from datastore
if err := insertRows(product); err != nil {
fmt.Println(err)
}
}
What should I put on the entity tag "bigquery:click_url" to make this work?
Because the ClickUrl in BigQuery is a STRUCT type, which has key-value pairs.
Maybe try this?
type Product struct {
Name string `datastore:"name" bigquery:"name"`
ClickUrl struct {
String string
Text string
Provided string
} `datastore:"click_url,noindex" bigquery:"click_url"`
DateAdded time.Time `datastore:"date_added" bigquery:"date_added"`
}

use same struct for both create and update operations in go lang using gin and gorm

Disclaimer: go lang noob here
I've my user struct as
type User struct {
ID uint32 `json:"id"`
FirstName string `json:"firstName" binding:"required"`
LastName string `json:"lastName"`
Email string `json:"email" binding:"required,email,uniqueModelValue=users email"`
Active bool `json:"active"`
Password string `json:"password,omitempty" binding:"required,gte=8"`
UserType string `json:"userType" binding:"oneof=admin backoffice principal staff parent student"`
CreatedAt time.Time `json:"createdAt"`
UpdatedAt time.Time `json:"updatedAt"`
}
/POST /users handler
func Create(ctx *gin.Context) {
user := models.User{}
//validate
if err := ctx.ShouldBind(&user); err != nil {
response.Error(ctx, err)
return
}
db, _ := database.GetDB()
db.Create(&user)
// removing password from the response
user.Password = ""
response.Success(ctx, user)
}
I want to create an update handler using the same struct, is there any way I can perform using the same struct ?
Mind you, struct has required bindings on many fields firstName,email etc.
while updating, I might not pass these fields
I came up with something like
/PUT /users/ handler
func Update(ctx *gin.Context) {
userID := ctx.Param("userId")
user := models.User{}
db, _ := database.GetDB()
if err := db.First(&user, userID).Error; err != nil {
response.Error(ctx, err)
return
}
updateUser := models.User{}
if err := ctx.BindJSON(&updateUser); err != nil {
response.Error(ctx, err)
}
//fmt.Printf("%v", updateUser)
db.Model(&user).Updates(updateUser)
response.Success(ctx, user)
}
this is failing obviously due to required validations missing, if I try to update, let's say, just the lastName
You could try for this case to bind to the User struct you just pulled out of the database, like so:
func Update(ctx *gin.Context) {
userID := ctx.Param("userId")
user := models.User{}
db, _ := database.GetDB()
if err := db.First(&user, userID).Error; err != nil {
response.Error(ctx, err)
return
}
if err := ctx.BindJSON(&user); err != nil {
response.Error(ctx, err)
}
db.Model(&user).Updates(user)
response.Success(ctx, user)
}
That might work because the old values + the changes (written into the struct by BindJSON) might be able to pass validation.
Generally speaking, this pattern isn't going to help you for long in Go.
Using the same struct for two different purposes: representation of the entity model and representation of the messages in your API that pertain to the model, will end up giving you trouble sooner or later. These tend to be slightly different, as for example you might eventually have fields that are only exposed internally or, as you have encountered, you have validation that doesn't make sense for all use cases.
For this problem what would help you is to create a new struct to represent the update user message:
package messages
type UpdateUser struct {
FirstName string `json:"firstName"`
LastName string `json:"lastName"`
... fields that are updatable with appropriate validation tags
}
func (u *UpdateUser) ToModel() *model.User {
return &model.User{
FirstName: u.FirstName,
LastName: u.LastName,
...
}
}
Then use that as to validate your request model, then turn it into a model.User for the update:
func Update(ctx *gin.Context) {
userID := ctx.Param("userId")
user := models.User{}
db, _ := database.GetDB()
if err := db.First(&user, userID).Error; err != nil {
response.Error(ctx, err)
return
}
updateUser := messages.UpdateUser{}
if err := ctx.BindJSON(&updateUser); err != nil {
response.Error(ctx, err)
}
//fmt.Printf("%v", updateUser)
db.Model(&user).Updates(updateUser.ToModel())
response.Success(ctx, user)
}
If you use an ORM like Gorm in your project, it is recommended to use viewmodel structs for your requests and responses. Because the structure of your database tables are mostly different then your rest api models. Data binding and validation are easier with viewmodel structs.

Read flattened entity from cloud datastore in golang

func (db *dataStore) AddAcceptance(ctx context.Context, req *acceptance.PolicyAcceptance) (uint64, error) {
accpKey := datastore.IncompleteKey("Acceptance", nil)
key, err := db.Put(context.Background(), accpKey, req);
if err != nil {
log.Fatalf("Failed to save Acceptance: %v", err)
}
accpKey = key
val := uint64(accpKey.ID)
return val, err
}
type PolicyAcceptance struct {
Id string `protobuf:"bytes,1,opt,name=id" json:"id,omitempty"`
PolicyNumber int64 `protobuf:"varint,2,opt,name=policyNumber" json:"policyNumber,omitempty"`
Version string `protobuf:"bytes,3,opt,name=version" json:"version,omitempty"`
SignerData *SignerData `protobuf:"bytes,4,opt,name=signerData" json:"signerData,omitempty" datastore:",flatten"`
GroupID int64 `protobuf:"varint,5,opt,name=groupID" json:"groupID,omitempty"`
LocationID int64 `protobuf:"varint,6,opt,name=locationID" json:"locationID,omitempty"`
BusinessId int64 `protobuf:"varint,7,opt,name=businessId" json:"businessId,omitempty"`
AcceptedDate *google_protobuf.Timestamp `protobuf:"bytes,8,opt,name=acceptedDate" json:"acceptedDate,omitempty" datastore:",flatten"`
IssuerName string `protobuf:"bytes,9,opt,name=issuerName" json:"issuerName,omitempty"`
Place string `protobuf:"bytes,10,opt,name=place" json:"place,omitempty"`
}
type SignerData struct {
Name string `protobuf:"bytes,1,opt,name=name" json:"name,omitempty"`
Email string `protobuf:"bytes,2,opt,name=email" json:"email,omitempty"`
Type string `protobuf:"bytes,3,opt,name=type" json:"type,omitempty"`
Id int64 `protobuf:"varint,4,opt,name=id" json:"id,omitempty"`
}
datastore:",flatten" saves data as flattened in data store. The property names becomes flattened with . like SignerData.Id as property name but when it's read from data store, how can I map it back to struct? It fails throwing an error like:
SignerData.Id could not be found as a key in struct. Error: No such
struct field.
func (db *dataStore) GetAcceptanceBySignerData(ctx context.Context, req *acceptance.SignerData) (*acceptance.ListOfPolicyAcceptance, error) {
query := datastore.NewQuery("Acceptance").Filter("SignerData.Id =", req.Id)
var accpArr acceptance.ListOfPolicyAcceptance
var err error
it := db.Run(ctx, query)
for {
var accept acceptance.PolicyAcceptance
_, err := it.Next(&accept)
if err == iterator.Done {
break
}
if err != nil {
log.Fatalf("Error fetching : %v", err)
}
accpArr.AcceptanceList = append(accpArr.AcceptanceList, &accept)
}
return &accpArr, err
}

Calling SOAP with Golang

I am new to golang and trying make a soap call with gowsdl .
I have generated the wsdl code and installed it as a package. I am however struggling to understand the syntax for call the method from it.
When I examine the package, this is what I want in the soap body:
type AccountUser struct {
XMLName xml.Name `xml:"http://exacttarget.com/wsdl/partnerAPI AccountUser"`
*APIObject
AccountUserID int32 `xml:"AccountUserID,omitempty"`
UserID string `xml:"UserID,omitempty"`
Password string `xml:"Password,omitempty"`
Name string `xml:"Name,omitempty"`
Email string `xml:"Email,omitempty"`
MustChangePassword bool `xml:"MustChangePassword,omitempty"`
ActiveFlag bool `xml:"ActiveFlag,omitempty"`
ChallengePhrase string `xml:"ChallengePhrase,omitempty"`
ChallengeAnswer string `xml:"ChallengeAnswer,omitempty"`
UserPermissions []*UserAccess `xml:"UserPermissions,omitempty"`
Delete int32 `xml:"Delete,omitempty"`
LastSuccessfulLogin time.Time `xml:"LastSuccessfulLogin,omitempty"`
IsAPIUser bool `xml:"IsAPIUser,omitempty"`
NotificationEmailAddress string `xml:"NotificationEmailAddress,omitempty"`
IsLocked bool `xml:"IsLocked,omitempty"`
Unlock bool `xml:"Unlock,omitempty"`
BusinessUnit int32 `xml:"BusinessUnit,omitempty"`
DefaultBusinessUnit int32 `xml:"DefaultBusinessUnit,omitempty"`
DefaultApplication string `xml:"DefaultApplication,omitempty"`
Locale *Locale `xml:"Locale,omitempty"`
TimeZone *TimeZone `xml:"TimeZone,omitempty"`
DefaultBusinessUnitObject *BusinessUnit `xml:"DefaultBusinessUnitObject,omitempty"`
AssociatedBusinessUnits struct {
BusinessUnit []*BusinessUnit `xml:"BusinessUnit,omitempty"`
} `xml:"AssociatedBusinessUnits,omitempty"`
Roles struct {
Role []*Role `xml:"Role,omitempty"`
} `xml:"Roles,omitempty"`
LanguageLocale *Locale `xml:"LanguageLocale,omitempty"`
SsoIdentities struct {
SsoIdentity []*SsoIdentity `xml:"SsoIdentity,omitempty"`
} `xml:"SsoIdentities,omitempty"`
}
And the method to call the SOAP is :
func (s *SOAPClient) Call(soapAction string, request, response interface{}) error {
envelope := SOAPEnvelope{
//Header: SoapHeader{},
}
envelope.Body.Content = request
buffer := new(bytes.Buffer)
encoder := xml.NewEncoder(buffer)
//encoder.Indent(" ", " ")
if err := encoder.Encode(envelope); err != nil {
return err
}
if err := encoder.Flush(); err != nil {
return err
}
log.Println(buffer.String())
req, err := http.NewRequest("POST", s.url, buffer)
if err != nil {
return err
}
if s.auth != nil {
req.SetBasicAuth(s.auth.Login, s.auth.Password)
}
req.Header.Add("Content-Type", "text/xml; charset=\"utf-8\"")
if soapAction != "" {
req.Header.Add("SOAPAction", soapAction)
}
req.Header.Set("User-Agent", "gowsdl/0.1")
req.Close = true
tr := &http.Transport{
TLSClientConfig: &tls.Config{
InsecureSkipVerify: s.tls,
},
Dial: dialTimeout,
}
client := &http.Client{Transport: tr}
res, err := client.Do(req)
if err != nil {
return err
}
defer res.Body.Close()
rawbody, err := ioutil.ReadAll(res.Body)
if err != nil {
return err
}
if len(rawbody) == 0 {
log.Println("empty response")
return nil
}
log.Println(string(rawbody))
respEnvelope := new(SOAPEnvelope)
respEnvelope.Body = SOAPBody{Content: response}
err = xml.Unmarshal(rawbody, respEnvelope)
if err != nil {
return err
}
fault := respEnvelope.Body.Fault
if fault != nil {
return fault
}
return nil
}
I have imported the package into my go file , and would love pointers on how to call this.
To use the generated code you obviously will have to first initialize the soap client with one of the generated "constructor" functions NewSOAPClient or NewSOAPClientWithTLSConfig.
After that you'll need to prepare two values that you can use as the request and response arguments to the Call method, they represent the body content of the soap request/response payloads.
The types of those two values will depend on what kind of call you want to make, for example the hypothetical calls create_account, update_account, and delete_account would usually require different types. Basically, the type of the request value should be marshalable into an xml that matches the xml expected by the soap service for the specified action, and the type of the response should be unmarshalable from an xml that matches the soap service's documented response for the specified action.
Consider this contrived example:
There is a SOAP service that allows you to create users. For you to be able to create a user with the service it requires you to send an email and a password, and if everyting's ok it will return an id. In such a case your two request/response types would look like this:
type CreateUserRequest struct {
Email string `xml:"Email,omitempty"`
Password string `xml:"Password,omitempty"`
}
type CreateUserResponse struct {
ID string `xml:"ID"`
}
Then the client code would look like this:
client := NewSOAPClient("https://soap.example.com/call", true, nil)
req := &CreateUserRequest{
Email: "jdoe#example.com",
Password: "1234567890",
}
res := &CreateUserResponse{}
if err := client.Call("create_user", req, res); err != nil {
panic(err)
}
// if everything went well res.ID should have its
// value set with the one returned by the service.
fmt.Println(res.ID)

Read multipart-form data as []byte in GO

I'm using GIN as GO framework, I'm having an issue when uploading file and directly convert image as byte so I can store it in my BLOB field inside db table, so I have my piece of code like this:
func (a *AppHandler) Upload(ctx *gin.Context) {
form := &struct {
Name string `form:"name" validate:"required"`
Token string `form:"token" validate:"required"`
AppCode string `form:"app_code" validate:"required"`
}{}
ctx.Bind(form)
if validationErrors := a.ValidationService.ValidateForm(form); validationErrors != nil {
httpValidationErrorResponse(ctx, validationErrors)
return
}
file, header, err := ctx.Request.FormFile("file")
and I'm trying to store it in db like this
app.SetFile(file)
a.AppStore.Save(app)
and it returns this kind of error:
cannot use file (type multipart.File) as type []byte
*multipart.File implements io.Reader interface so you could copy its content into a bytes.Buffer like this:
file, header, err := ctx.Request.FormFile("file")
defer file.Close()
if err != nil {
return nil, err
}
buf := bytes.NewBuffer(nil)
if _, err := io.Copy(buf, file); err != nil {
return nil, err
}
and then add to your app
app.SetFile(buf.Bytes())

Resources