Doing an update to a database with partial data in Go - go

So I have the following function in an API:
type Product struct {
Id int32 `json:"id" db:"id"`
CompanyId int32 `json:"companyID" db:"company_id"`
Name *string `json:"name" db:"name"`
Description *string `json:"description" db:"description"`
ExternalId *string `json:"externalID" db:"external_id"`
Tags types.StringArray `json:"tags" db:"tags"`
Enabled bool `json:"enabled" db:"enabled"`
CreatedAt time.Time `json:"createdAt"`
UpdatedAt time.Time `json:"updatedAt"`
}
func updateProduct(event UpdateProductEvent) (Response, error) {
// TODO - determine how we want to log. fmt jsut does prints. The log package?
fmt.Println("This is updateProduct")
var id int64
err := db.QueryRow(`
UPDATE
v2.products
SET
company_id = $2,
name = $3,
description = $4,
external_id = $5,
enabled = $6
WHERE id = $1 RETURNING id`,
Product.Id,
Product.CompanyId,
Product.Name,
Product.Description,
Product.ExternalId,
Product.Enabled,
).Scan(&id)
if err != nil {
fmt.Println("Error updating product: " + err.Error())
fmt.Println("Product: ")
return Response{}, errors.New("Error inserting product")
}
return Response{
Id: 8,
}, nil
}
So a Product gets passed in and then I need to update the entity in the database. The above works fine in the case where I get a full product passed in. But for this API that might not be the case, it could only be a partial one.
So I might only have two fields in the JSON passed to the API.
So what I'm thinking is passing in an interface{}, instead of the Product so I dont get zero values for fields not set. Then looping through interface{} to get the fields. From those fields I would check if that field exists on the Product object. If it does get the tag from the field, this would be the appropriate column in the database, and then create the query string for the QueryRow call and add all the fields to an interface{} array and let that be the second param. Something like:
func updateProduct(event interface{}) (Response, error) {
var values []interface{}
// Loop through the event and get the field and value
for field, i := range event.fields {
if Product.Has(field.name) {
//add db tag of Product field to query
values = append(values, field.value)
}
}
err := db.QueryRow(`
UPDATE
v2.products
SET
//fields from the loop above as needed
WHERE id = $1 RETURNING id`,
values...,
).Scan(&id)
...
}
I think i can get this, or something like it, to work but it seems like a very round about way to do something very common. I just cant think of any other way to do it.So is there a more common/simpler approach that I'm missing. I cant find anything Googling either.
EDIT: In my haste and attempt to simplfy I left out an important part. This is for an AWS Lambda. So the object passed into updateProduct depends on what comes from the API Gateway. So if i make it the UpdateProductEvent it will parse the json into the object, in this case the Product. If I just make it an interface{} it will basically just make a map.
So if my request is
{
"product": {
"name": "Test17",
"externalID": "test17",
"description": "",
"companyID": 309
}
}
The first method, setting an explicit object, it will create a Product and fields not in the json will have the 0 value for their field.
If i make it an interface it will be an object with those properties and not the zero values for the rest of the fields in Product

Related

Go - Save JSON string into a DB (most efficient?)

I am getting a JSON response from a server and want to save that into a db, where one of the columns is a JSON column. The response looks similar to the following:
msgJson = [{"id": 1, "type": "dog", "attributes": {"weight":20, "sound":"bark"}}]
So I am currently making a struct and trying to update each element in the DB.
type Animal struct {
Id int `json:"id"`
Type string `json:"type"`
Attributes string `json:"attributes"`
}
var animals []Animal
json.Unmarshal([]byte(msgJson), &animals)
sqlStatement := `
UPDATE animals
SET type = $2, attributes = $3
WHERE id = $1;`
_, err := db.Exec(
sqlStatement,
animals[0].Id,
animals[0].Type,
animals[0].Attributes)
Of course, this doesn't work, because the attributes field is supposed to be JSON.
I believe I could Unmarshal the JSON into nested structs, and then Marshal it when updating the DB, but since this will have many fields, is there a way to take the string and immediately represent it as JSON when adding to the DB? I hope that question makes sense.
Thank you
Unmarshal the attributes field to a json.RawMessage. Save the raw message to the database.
type Animal struct {
Id int `json:"id"`
Type string `json:"type"`
Attributes json.RawMessage `json:"attributes"`
}
⋮
_, err := db.Exec(
sqlStatement,
animals[0].Id,
animals[0].Type,
animals[0].Attributes)

Golang Pointer Understanding

I am learning go and i need to understand something. I am getting few errors. I have created a Product struct and attached a func with it. I also got a product lists as a slice. Actually I am following one example. I was just trying add different endpoints to it.
I have added question in comment in code. Please explain. I need to return the json single object as a response to user. Please guide me.
package data
type Product struct {
ID int `json:"id"`
Name string `json:"name"`
Description string `json:"description"`
Price float32 `json:"price"`
SKU string `json:"sku"`
CreatedOn string `json:"-"`
UpdatedOn string `json:"-"`
DeletedOn string `json:"-"`
}
type Products []*Product
func (p *Products) ToJSON(w io.Writer) error {
e := json.NewEncoder(w)
return e.Encode(p)
}
func (p *Product) FromJSON(r io.Reader) error {
d := json.NewDecoder(r)
return d.Decode(p)
}
var ErrProductNotFound = fmt.Errorf("Product not found")
func GetProduct(id int) (*Product, error) { // this is returning *Product & err. When I use this in GetProduct in handler func it is giving error
for _, p := range productList {
if p.ID == id {
fmt.Println(p)
return p, nil
}
}
return nil, ErrProductNotFound
}
var productList = []*Product{ **// Why in example the teacher doing it like this.** []*Product{&Product{}, &Product{}} **what it the reason? Please explain.
&Product{ // this gives warning : redundant type from array, slice, or map composite literal. need to understand why**
ID: 1,
Name: "Latte",
Description: "chai",
Price: 2.45,
SKU: "abc123",
CreatedOn: time.Now().UTC().String(),
UpdatedOn: time.Now().UTC().String(),
},
&Product{
ID: 2,
Name: "Tea",
Description: "chai",
Price: 1.45,
SKU: "abc1234",
CreatedOn: time.Now().UTC().String(),
UpdatedOn: time.Now().UTC().String(),
},
}
package handlers
func (p *Product) GetProduct(rw http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
id, _ := strconv.Atoi(vars["id"])
p, errr := data.GetProduct(id) **// cannot use data.GetProduct(id) (value of type *data.Product) as *Product value in assignment**
errr = p.ToJSON(rw) // **p.ToJSON undefined (type *Product has no field or method ToJSON)**
if errr != nil {
http.Error(rw, "could not locate the product", http.StatusBadGateway)
}
}
cannot use data.GetProduct(id) (value of type *data.Product) as *Product value in assignment
p.ToJSON undefined (type *Product has no field or method ToJSON)
The problem here is that inside the GetProduct handler the variable p already has a type (*handlers.Product) that is different from the one returned by the data.GetProduct function (*data.Product). So what you can do is to use a different name for the variable that will store the result of data.GetProduct.
Why in example the teacher doing it like this. []*Product{&Product{}, &Product{}} what it the reason? Please explain.
In general because that's one of the available methods for how to initialize a slice of structs, as per the language spec. Why the teacher used this method specifically? Unless the teacher confided the reason to someone, then no one would know, I certainly don't.
this gives warning : redundant type from array, slice, or map composite literal. need to understand why
Because it's true, it is redundant, as per the language spec, in a composite literal expression you can elide the types of the elements and keys.
For example a non-redundant version of the following composite literal:
[]*Product{&Product{}, &Product{}}
would look like this:
[]*Product{{}, {}}
and the result of these two expressions would be the same.

Is it possible to delete a field of a struct value at runtime?

I have the following struct:
type Record struct {
Id string `json:"id"`
ApiKey string `json:"apiKey"`
Body []string `json:"body"`
Type string `json:"type"`
}
Which is a Blueprint of a dynamoDB table. And I need somehow, delete the ApiKey, after using to check if the user has access to the giving record. Explaining:
I have and endpoint in my API where the user can send a id to get a item, but he needs to have access to the ID and the ApiKey (I'm using Id (uuid) + ApiKey) to create unique items.
How I'm doing:
func getExtraction(id string, apiKey string) (Record, error) {
svc := dynamodb.New(cfg)
req := svc.GetItemRequest(&dynamodb.GetItemInput{
TableName: aws.String(awsEnv.Dynamo_Table),
Key: map[string]dynamodb.AttributeValue{
"id": {
S: aws.String(id),
},
},
})
result, err := req.Send()
if err != nil {
return Record{}, err
}
record := Record{}
err = dynamodbattribute.UnmarshalMap(result.Item, &record)
if err != nil {
return Record{}, err
}
if record.ApiKey != apiKey {
return Record{}, fmt.Errorf("item %d not found", id)
}
// Delete ApiKey from record
return record, nil
}
After checking if the ApiKey is equal to the provided apiKey, I want to delete the ApiKey from record, but unfortunately that's not possible using delete.
Thank you.
There is no way to actually edit a golang type (such as a struct) at runtime. Unfortunately you haven't really explained what you are hoping to achieve by "deleting" the APIKey field.
General approaches would be:
set the APIKey field to an empty string after inspection, if you dont want to display this field when empty set the json struct tag to omitempty (e.g `json:"apiKey,omitempty"`)
set the APIKey field to never Marshal into JSON ( e.g ApiKey string `json:"-"`) and you will still be able to inspect it just wont display in JSON, you could take this further by adding a custom marshal / unmarshal function to handle this in one direction or in a context dependent way
copy the data to a new struct, e.g type RecordNoAPI struct without the APIKey field and return that after you have inspected the original Record
Created RecordShort structure without "ApiKey"
Marshalin Record
Unmarsaling Record to ShortRecord
type RecordShot struct {
Id string `json:"id"`
Body []string `json:"body"`
Type string `json:"type"`
}
record,_:=json.Marshal(Record)
json.Unmarshal([]byte(record), &RecordShot)

Designing a Datastore schema for a page that users can edit (e.g. a wikipedia/stackoverflow page)

The idea is to design a table/entity that contains some basic info, as well as a Markdown-Content field that would allow users to easily create tables and such.
I'm thinking something like this:
type Tournament struct {
ID in64 `datastore:"-"`
MDContent []byte `datastore:",noindex"`
Name string
URL string
DateCreated int64
CreatedBy string
DateUpdated int64
UpdatedBy string
ApprovalStatus int64 // 0=to be decided, 1=approved, 2=rejected, 3=discontinued
ApprovalBy string
}
My problem is figuring out how to update it. The ID field will also be used as the URL path, e.g. if an entity has ID 7 then it will be displayed on example.com/tournament/7.
I believe this eliminates the possibility of simply creating a new entity with updated data, and then set the ApprovalStatus=3 on the previous entity, because if you do as such then the example.com/tournament/7 URL will no longer request the correct ID.
I also don't like the idea of creating my own unique ID because I think it would be great to simply take advantage of the Datastore ID generation (which also makes it easy to get the correct entity based on URL); I considered creating a new entity/table that would keep track of revisions but I'm not sure how efficient all of this is, so I was hoping some expert might be able to give some advice.
Update related to #mkopriva solution:
If you do it this way, then it's necessary to include a TournamentID field inside the TournamentEdit entity struct I think?
type TournamentEdit struct {
ID in64 `datastore:"-"`
TournamentID int64
MDContent []byte `datastore:",noindex"`
DateCreated int64
CreatedBy string
ApprovalStatus int64 // 0=to be decided, 1=approved, 2=rejected, 3=discontinued
ApprovalBy string
}
And then the retrieve function could look like this:
func (db *datastoreDB) GetTournamentByKeyID(ctx context.Context, keyID int64) (*Tournament, error) {
key := datastore.IDKey("Tournament", keyID, nil)
var tournamnet Tournament
err := db.client.Get(ctx, key, &tournament)
// err checking
tournament.ID = key.ID
var edits TournamentEdits
query := datastore.NewQuery("TournamentEdit")
query = query.Filter("TournamentID =", tournament.ID)
query = query.Filter("ApprovalStatus =", 1)
if _, err := db.client.GetAll(ctx, query, &edits); err != nil {
//err checking
}
tournament.Edits = edits // I guess this is wrong way to do it?
return &tournament, nil
}
Would this work?
One thing you could do is to simply create a new entity that would represent the edit of a tournament. By the way, I'm not a datastore user so I'm not sure if this is how you would model the entities but the general idea is the same for most, if not all, databases:
type Tournament struct {
ID in64 `datastore:"-"`
MDContent []byte `datastore:",noindex"`
Name string
URL string
DateCreated int64
CreatedBy string
DateUpdated int64
UpdatedBy string
Edits []TournamentEdit
}
type TournamentEdit struct {
ID in64 `datastore:"-"`
MDContent []byte `datastore:",noindex"`
DateCreated int64
CreatedBy string
ApprovalStatus int64 // 0=to be decided, 1=approved, 2=rejected, 3=discontinued
ApprovalBy string
}
This should allow you to have multiple edits from different users in the queue, CRUD a specific edit, and or filter edits by their status.

gorp PreUpdate method update involuntary columns

see below.
type User struct {
Id int64 `db:"id" json:"id"`
Name string `db:"name" json:"name"`
DateCreate int64 `db:"date_create"`
DateUpdate int64 `db:"date_update"`
}
func (u *User) PreInsert(s gorp.SqlExecutor) error {
u.DateCreate = time.Now().UnixNano()
u.DateUpdate = u.DateCreate
return nil
}
func (u *User) PreUpdate(s gorp.SqlExecutor) error {
u.DateUpdate = time.Now().UnixNano()
return nil
}
I executed INSERT.
user := model.User{
Name: "John",
}
err := dbMap.Insert(&user)
Result of INSERT. no problem
1,John,1444918337049394761,1444918337049394761
continue, I executed UPDATE.
user := model.User{
Id: 1,
Name: "John",
}
_, err := dbMap.Update(&user)
Result of UPDATE
1,John,0,1444918337049394900
Column DateCreate updated.
Of course, my expectation values are
1,John,1444918337049394761,1444918337049394900
That's because when initializing your user struct, you did not explicitely set user.DateCreate, which effectively sets it to 0.
gorp cannot guess which fields you mean to update or not, so it updates them all.
To do what you want, you have to choose between trade-offs
Get the struct from the ORM, with the field already set to the good value. That's one additional select query that will be executed.
Execute a custom query, losing the convenience of the ORM
I'm guessing you could have another struct type without this field. That seems messy, I do not recommend that.

Resources