I am getting a JSON response from a server and want to save that into a db, where one of the columns is a JSON column. The response looks similar to the following:
msgJson = [{"id": 1, "type": "dog", "attributes": {"weight":20, "sound":"bark"}}]
So I am currently making a struct and trying to update each element in the DB.
type Animal struct {
Id int `json:"id"`
Type string `json:"type"`
Attributes string `json:"attributes"`
}
var animals []Animal
json.Unmarshal([]byte(msgJson), &animals)
sqlStatement := `
UPDATE animals
SET type = $2, attributes = $3
WHERE id = $1;`
_, err := db.Exec(
sqlStatement,
animals[0].Id,
animals[0].Type,
animals[0].Attributes)
Of course, this doesn't work, because the attributes field is supposed to be JSON.
I believe I could Unmarshal the JSON into nested structs, and then Marshal it when updating the DB, but since this will have many fields, is there a way to take the string and immediately represent it as JSON when adding to the DB? I hope that question makes sense.
Thank you
Unmarshal the attributes field to a json.RawMessage. Save the raw message to the database.
type Animal struct {
Id int `json:"id"`
Type string `json:"type"`
Attributes json.RawMessage `json:"attributes"`
}
⋮
_, err := db.Exec(
sqlStatement,
animals[0].Id,
animals[0].Type,
animals[0].Attributes)
Related
I am trying to convert a struct to map using following method
func ConvertStructToMap(in interface{}) map[string]interface{} {
fmt.Println(in)
var inInterface map[string]interface{}
inrec, _ := json.Marshal(in)
json.Unmarshal(inrec, &inInterface)
return inInterface
}
The problem is when I am trying to convert a struct
type Data struct{
Thing string `json:"thing,omitempty"`
Age uint8 `json:"age,omitempty"`
}
With this data
c:=Data{
Thing :"i",
Age:0,
}
it just gives me the following output map[things:i] instead it should give the output
map[things:i,age:0]
And when I don't supply age
like below
c:=Data{
Thing :"i",
}
Then it should give this output map[things:i] .I am running an update query and the user may or may not supply the fields ,any way to solve it .I have looked over the internet but couldn't get my head on place
Edit -
I am sending json from frontend which gets converted to struct with go fiber Body parser
if err := c.BodyParser(&payload); err != nil {
}
Now If I send following payload from frontend
{
thing:"ii"
}
the bodyParser converts it into
Data{
Thing :"ii",
Age :0
}
that's why I have used omitempty such that bodyParser can ignore the field which are not present
omitempty ignore when convert from struct to json
type Response struct {
Age int `json:"age,omitempty"`
Name string `json:"name,omitempty"`
}
resp:=Response {
Name: "john",
}
data,_:=json.Marshal(&resp)
fmt.Println(string(data)) => {"name": "john"}
In you case, you convert from json to struct, if attribute not present, it will be fill with default value of data type, in this case is 0 with uint8
So I have the following function in an API:
type Product struct {
Id int32 `json:"id" db:"id"`
CompanyId int32 `json:"companyID" db:"company_id"`
Name *string `json:"name" db:"name"`
Description *string `json:"description" db:"description"`
ExternalId *string `json:"externalID" db:"external_id"`
Tags types.StringArray `json:"tags" db:"tags"`
Enabled bool `json:"enabled" db:"enabled"`
CreatedAt time.Time `json:"createdAt"`
UpdatedAt time.Time `json:"updatedAt"`
}
func updateProduct(event UpdateProductEvent) (Response, error) {
// TODO - determine how we want to log. fmt jsut does prints. The log package?
fmt.Println("This is updateProduct")
var id int64
err := db.QueryRow(`
UPDATE
v2.products
SET
company_id = $2,
name = $3,
description = $4,
external_id = $5,
enabled = $6
WHERE id = $1 RETURNING id`,
Product.Id,
Product.CompanyId,
Product.Name,
Product.Description,
Product.ExternalId,
Product.Enabled,
).Scan(&id)
if err != nil {
fmt.Println("Error updating product: " + err.Error())
fmt.Println("Product: ")
return Response{}, errors.New("Error inserting product")
}
return Response{
Id: 8,
}, nil
}
So a Product gets passed in and then I need to update the entity in the database. The above works fine in the case where I get a full product passed in. But for this API that might not be the case, it could only be a partial one.
So I might only have two fields in the JSON passed to the API.
So what I'm thinking is passing in an interface{}, instead of the Product so I dont get zero values for fields not set. Then looping through interface{} to get the fields. From those fields I would check if that field exists on the Product object. If it does get the tag from the field, this would be the appropriate column in the database, and then create the query string for the QueryRow call and add all the fields to an interface{} array and let that be the second param. Something like:
func updateProduct(event interface{}) (Response, error) {
var values []interface{}
// Loop through the event and get the field and value
for field, i := range event.fields {
if Product.Has(field.name) {
//add db tag of Product field to query
values = append(values, field.value)
}
}
err := db.QueryRow(`
UPDATE
v2.products
SET
//fields from the loop above as needed
WHERE id = $1 RETURNING id`,
values...,
).Scan(&id)
...
}
I think i can get this, or something like it, to work but it seems like a very round about way to do something very common. I just cant think of any other way to do it.So is there a more common/simpler approach that I'm missing. I cant find anything Googling either.
EDIT: In my haste and attempt to simplfy I left out an important part. This is for an AWS Lambda. So the object passed into updateProduct depends on what comes from the API Gateway. So if i make it the UpdateProductEvent it will parse the json into the object, in this case the Product. If I just make it an interface{} it will basically just make a map.
So if my request is
{
"product": {
"name": "Test17",
"externalID": "test17",
"description": "",
"companyID": 309
}
}
The first method, setting an explicit object, it will create a Product and fields not in the json will have the 0 value for their field.
If i make it an interface it will be an object with those properties and not the zero values for the rest of the fields in Product
I am trying to query all the results from a postgres table without where condition and map it with array of structs with the help of sqlx db Query by passing the args ...interface {}.
But the code pasted below never works, Instead of iterating and scanning the result one by one , is it possible to get the following code work ??
Inputs are much appreciated . Thank you
type CustomData struct {
ID string `db:"id" json:",omitempty"`
Name string `db:"name" json:",omitempty"`
Description string `db:"description" json:",omitempty"`
SourceID string `db:"sourceid" json:",omitempty"`
StatusID string `db:"statusid" json:",omitempty"`
StatusReason string `db:"statusreason" json:",omitempty"`
CreateTime string `db:"createtime" json:",omitempty"`
UpdateTime string `db:"updatetime" json:",omitempty"`
}
var myData []CustomData
*sqlx.DB.Query("SELECT id as ID, name as Name, description as Description, sourceid as SourceID, statusid as StatusID, statusreason as StatusReason, createtime as CreateTime, updatetime as UpdateTime FROM myschema.my_table", &myData)
// tried with following statement but din't work either
// *sqlx.DB.Query("SELECT * FROM myschema.my_table", &myData)
for _, data := range myData {
fmt.Println("--", data)
}
Expected results:
--- CustomData{1,x,x,x,x}
--- CustomData{2,x,x,x,x}
Actual:
Nothing..
You don't need to rename the fields in the query, since you're defining the actual DB fields in the struct tags.
If you want to scan directly to the slice of CustomData and if you are using SQLX, you should use the SQLX specific Select method, rather than the generic SQL Query. Slightly modified relevant example from the illustrated guide to SQLX (https://jmoiron.github.io/sqlx/#getAndSelect):
pp := []Place{}
err = db.Select(&pp, "SELECT * FROM place")
So in your case:
myData := []CustomData
err = db.Select(&myData, "SELECT * FROM myschema.my_table")
you can use the following:
for rows.Next() {
s := CustomData{}
if err := rows.Scan(&s); err != nil {
return err
}
fmt.Println(s)
}
and you can always use ORM library as gorm if you like code first approach or sqlboiler if you like DB first approach
I am trying to unmarshall multidimensional JSON. My JSON is contains dynamic key therefore I can't do it.
JSON
{
"id":"3",
"datetime":"2019-06-08",
"metadata":[{"a":"A"},{"b":"B"}]
}
Go file
type Chats struct {
Id string json:"id"
Datetime string json:"date"
Metadata string json:"metadata"
}
chat := models.Chats{}
err := c.BindJSON(&chat)
if err != nil {
c.Error(err)
return
}
fmt.Println(chat)
If metadata is dynamic then treat is as an interface{}. If you know it's always going to be a JSON container then you could do map[string]interface{} for convenience. There is also json.RawMessage if you don't necessarily want to use type assertions to see what's inside of it, but just want to preserve the JSON (I'm guessing this is what you were hoping to do by setting it to string).
type Chats struct {
Id string `json:"id"`
Datetime string `json:"date"`
Metadata interface{} `json:"metadata"`
}
type Chats struct {
Id string `json:"id"`
Datetime string `json:"date"`
Metadata map[string]interface{} `json:"metadata"`
}
type Chats struct {
Id string `json:"id"`
Datetime string `json:"date"`
Metadata json.RawMessage `json:"metadata"`
}
I have the following struct:
type Record struct {
Id string `json:"id"`
ApiKey string `json:"apiKey"`
Body []string `json:"body"`
Type string `json:"type"`
}
Which is a Blueprint of a dynamoDB table. And I need somehow, delete the ApiKey, after using to check if the user has access to the giving record. Explaining:
I have and endpoint in my API where the user can send a id to get a item, but he needs to have access to the ID and the ApiKey (I'm using Id (uuid) + ApiKey) to create unique items.
How I'm doing:
func getExtraction(id string, apiKey string) (Record, error) {
svc := dynamodb.New(cfg)
req := svc.GetItemRequest(&dynamodb.GetItemInput{
TableName: aws.String(awsEnv.Dynamo_Table),
Key: map[string]dynamodb.AttributeValue{
"id": {
S: aws.String(id),
},
},
})
result, err := req.Send()
if err != nil {
return Record{}, err
}
record := Record{}
err = dynamodbattribute.UnmarshalMap(result.Item, &record)
if err != nil {
return Record{}, err
}
if record.ApiKey != apiKey {
return Record{}, fmt.Errorf("item %d not found", id)
}
// Delete ApiKey from record
return record, nil
}
After checking if the ApiKey is equal to the provided apiKey, I want to delete the ApiKey from record, but unfortunately that's not possible using delete.
Thank you.
There is no way to actually edit a golang type (such as a struct) at runtime. Unfortunately you haven't really explained what you are hoping to achieve by "deleting" the APIKey field.
General approaches would be:
set the APIKey field to an empty string after inspection, if you dont want to display this field when empty set the json struct tag to omitempty (e.g `json:"apiKey,omitempty"`)
set the APIKey field to never Marshal into JSON ( e.g ApiKey string `json:"-"`) and you will still be able to inspect it just wont display in JSON, you could take this further by adding a custom marshal / unmarshal function to handle this in one direction or in a context dependent way
copy the data to a new struct, e.g type RecordNoAPI struct without the APIKey field and return that after you have inspected the original Record
Created RecordShort structure without "ApiKey"
Marshalin Record
Unmarsaling Record to ShortRecord
type RecordShot struct {
Id string `json:"id"`
Body []string `json:"body"`
Type string `json:"type"`
}
record,_:=json.Marshal(Record)
json.Unmarshal([]byte(record), &RecordShot)