Original Question
When using the Update method in GORM the new data does not get saved. i.e. I want to set a bool from true to false, but it stays true even after the Update method.
In the description of the method there is a warning: "WARNING when update with struct, GORM will not update fields that with zero value"
Since I am using a struct to update and false is the zero value of bool, this seems expected behaviour, but I don't see any reason why to do so and how to overcome this.
func UpdateData(c *fiber.Ctx) error {
db := database.DBConn
data := new([]entities.Data)
if err := c.BodyParser(&data); err != nil {
return err
}
db.Update(&data)
return c.JSON(data)
}
Solution Summary
First, as suggested I left out the new keyword when instantiating the structs. Then, I used a helper function (from here) for converting a struct to map while keeping the json alias as keys:
// StructToMap Converts a struct to a map while maintaining the json alias as keys
func StructToMap(obj interface{}) (newMap map[string]interface{}, err error) {
data, err := json.Marshal(obj)
if err != nil {
return
}
err = json.Unmarshal(data, &newMap) // Convert to a map
return
}
Then I loop over each element in the data slice in order to convert it and update it one by one:
func UpdateData(c *fiber.Ctx) error {
db := database.DBConn
data := []entities.Dard{}
if err := c.BodyParser(&data); err != nil {
return err
}
for _, record := range data {
mappedData, _ := StructToMap(record)
db.Model(&entities.Data{}).Update(mappedData)
}
return c.JSON(data)
}
*Error handling is obviously reduced in this example.
From official doc
NOTE When update with struct, GORM will only update non-zero
fields, you might want to use map to update attributes or use Select
to specify fields to update
So use map[string]interface{} to update non-zero fields also. Example:
db.Model(&user).Updates(map[string]interface{}{"name": "hello", "age": 18, "actived": false})
As you have struct already you can convert struct into map[string]interface{} (See details about conversion) then update. Another way is change the type of field as pointer.
Related
I'm using range to loop through an array of structs to extract data which will be used as a URL parameter for my API calls. Within this loop, I'm trying to push response data from one struct to another.
I'm able to get everything working, except for moving data from one struct to another, but not entirely sure how to solve for the errors I keep getting. I've tried multiple methods and seem to be stuck in the mud here for something I don't consider to be too hard, until now... In my code I'm using the append method but I'm not so sure that might be the correct way to proceed.
Presenting my code:
models.go
//Here is my existing struct, with populated data that I get from a CSV
type TravelItenaries struct {
Origin string
Destination string
Flight_num string
Origin_latitude string
Origin_longitude string
Destination_latitude string
Destination_longitude string
Origin_weather string
Destination_weather string
Coordinates_ori string
Coordinates_dest string
Temp_c_ori string
Temp_f_ori string
Temp_c_dest string
Temp_f_dest string
}
//Here is the response data that I'm expected to get from my API calls.
//I'm trying to "push" Temp_c_dest and Temp_f_dest data into TravelItenaries.Temp_f_dest and TravelItenaries.Temp_c_dest
//While also changing the data types to fit above.
type Response struct {
Current struct {
LastUpdatedEpoch int `json:"last_updated_epoch"`
LastUpdated string `json:"last_updated"`
Temp_c_dest float64 `json:"temp_c"`
Temp_c_dest float64 `json:"temp_f"`
IsDay int `json:"is_day"`
} `json:"current"
}
weather.go
func (s *Server) getWeather(w http.ResponseWriter, r *http.Request) {
// open file
f, err := os.Open("challenge_dataset.csv")
if err != nil {
responses.ERROR(w, http.StatusBadRequest, fmt.Errorf("helpful error"))
return
}
// remember to close the file at the end of the program
defer f.Close()
// read csv values using csv.Reader
csvReader := csv.NewReader(f)
data, err := csvReader.ReadAll()
if err != nil {
responses.ERROR(w, http.StatusBadRequest, fmt.Errorf("helpful error"))
return
}
// convert records to array of structs
travelItenaries := createTravelItenaries(data)
// remove duplicate flight records
cleanTravelItenaries:= remDupKeys(travelItenaries)
// set up params for API get request
params := url.Values{
"key": []string{"xxx"},
"q": []string{""},
}
// Construct URL for API request
u := &url.URL{
Scheme: "https",
Host: "api.weatherapi.com",
Path: "/v1/current.json",
RawQuery: params.Encode(),
}
client := &http.Client{}
// Will need this to populate the params using a range over a struct
values := u.Query()
// loop through cleaned data set
for _, service := range cleanTravelItenaries {
// dynamically acquire data from struct to pass as parameter
values.Set("q", service.Coordinates_dest)
u.RawQuery = values.Encode()
req, err := http.NewRequest("GET", u.String(), nil)
if err != nil {
responses.ERROR(w, http.StatusBadRequest, fmt.Errorf("helpful error"))
return
}
resp, err := client.Do(req)
if err != nil {
responses.ERROR(w, http.StatusBadRequest, fmt.Errorf("helpful error"))
return
}
body, err := ioutil.ReadAll(resp.Body)
// create empty struct to parse response data with using Inmarshal
var responseData models.Response
json.Unmarshal(body, &responseData)
// Here is the issue, I don't think append might be the correct procedure here?
// I simply just need to pass this response data to my already existing struct
service.Temp_c_dest = append(responseData.Current.Temp_c_dest , cleanTravelItenaries )
service.Temp_f_dest = append(responseData.Current.Temp_f_dest , cleanTravelItenaries )
}
}
The errors I get are related to both append statements at the end of the range function.
first argument to append must be slice; have float64
first argument to append must be slice; have float64
for both append methods.
Also, note how type TravelItenaries struct uses string type for:
Temp_c_dest string
Temp_f_dest string
Hence why I also need to do some field type conversion from Float64 to string.
How can I extract the fields Temp_c_dest and Temp_f_dest from API response struct to TravelItenaries struct fields while changing datatypes?
EDIT:
I've managed to get this somewhat working, but only inside the for loop. The data is not being saved outside the function.
service.Temp_f_dest = strconv.FormatFloat(responseData.Current.Temp_f_dest, 'g', -1, 64)
service.Temp_c_dest = strconv.FormatFloat(responseData.Current.Temp_c_dest, 'g', -1, 64)
I want to enable update functionality for my User object in my fiber/gorm backend. It works fine when I update all fields together using the Save function. However, when I do not have all fields present in the update request (for example only the Birthday field but not the Phone field) it overwrites the rest of the fields with their respective null values.
func UserUpdateByID(c *fiber.Ctx) error {
db := database.DBConn
// Parse the body to fit user entity
user := entities.User{}
if err := c.BodyParser(&user); err != nil {
return c.Status(500).SendString(err.Error())
}
// Update record
record := db.Save(&user)
if record.Error != nil {
return c.Status(500).SendString(record.Error.Error())
}
return c.JSON(record.Value)
When I change the line with record := db.Save(&user) to
mappedData, _ := StructToMap(user)
record := db.Model(&entities.User{}).Update(mappedData)
I receive the error that Update can not handle map of interfaces: sql: converting argument $10 type: unsupported type map[string]interface {}, a map
Update 1:
The mentioned StructToMap function looks like this:
func StructToMap(obj interface{}) (newMap map[string]interface{}, err error) {
data, err := json.Marshal(obj)
if err != nil {
return
}
err = json.Unmarshal(data, &newMap) // Convert to a map
return
}
Update 2:
The User object looks like:
type User struct {
gorm.Model
Identity string
Birthday time.Time
Phone string
City string
...
ActivityData []Activity
}
Looking, on gorm doc(https://gorm.io/docs/update.html), you can do something like this :
Use the Updates instead of Update.
db.Model(&user).Updates(User{Name: "hello", Age: 18, Active: false})
You can also use a db.Debug, to show the final query that gorm made, and see if matches with what are you expecting.
I'm trying to build a generic function which will parse input (in JSON) into a specified structure. The structure may vary at run-time, based on parameters which are passed to the function. I'm currently trying to achieve this by passing an object of the right type and using reflect.New() to create a new output object of the same type.
I'm then parsing the JSON into this object, and scanning the fields.
If I create the object and specify the type in code, everything works. If I pass an object and try to create a replica, I get an "invalid indirect" error a few steps down (see code).
import (
"fmt"
"reflect"
"encoding/json"
"strings"
)
type Test struct {
FirstName *string `json:"FirstName"`
LastName *string `json:"LastName"`
}
func genericParser(incomingData *strings.Reader, inputStructure interface{}) (interface{}, error) {
//******* Use the line below and things work *******
//parsedInput := new(Test)
//******* Use vvv the line below and things don't work *******
parsedInput := reflect.New(reflect.TypeOf(inputStructure))
decoder := json.NewDecoder(incomingData)
err := decoder.Decode(&parsedInput)
if err != nil {
//parsing error
return nil, err
}
//******* This is the line that generates the error "invalid indirect of parsedInput (type reflect.Value)" *******
contentValues := reflect.ValueOf(*parsedInput)
for i := 0; i < contentValues.NumField(); i++ {
//do stuff with each field
fmt.Printf("Field name was: %s\n", reflect.TypeOf(parsedInput).Elem().Field(i).Name)
}
return parsedInput, nil
}
func main() {
inputData := strings.NewReader("{\"FirstName\":\"John\", \"LastName\":\"Smith\"}")
exampleObject := new(Test)
processedData, err := genericParser(inputData, exampleObject)
if err != nil {
fmt.Println("Parsing error")
} else {
fmt.Printf("Success: %v", processedData)
}
}
If I can't create a replica of the object, then a way of updating / returning the one supplied would be feasible. The key thing is that this function must be completely agnostic to the different structures available.
reflect.New isn't a direct analog to new, as it can't return a specific type, it only can return a reflect.Value. This means that you are attempting to unmarshal into a *reflect.Value, which obviously isn't going to work (even if it did, your code would have passed in **Type, which isn't what you want either).
Use parsedInput.Interface() to get the underlying value after creating the new value to unmarshal into. You then don't need to reflect on the same value a second time, as that would be a reflect.Value of a reflect.Value, which again isn't going to do anything useful.
Finally, you need to use parsedInput.Interface() before you return, otherwise you are returning the reflect.Value rather than the value of the input type.
For example:
func genericParser(incomingData io.Reader, inputStructure interface{}) (interface{}, error) {
parsedInput := reflect.New(reflect.TypeOf(inputStructure).Elem())
decoder := json.NewDecoder(incomingData)
err := decoder.Decode(parsedInput.Interface())
if err != nil {
return nil, err
}
for i := 0; i < parsedInput.Elem().NumField(); i++ {
fmt.Printf("Field name was: %s\n", parsedInput.Type().Elem().Field(i).Name)
}
return parsedInput.Interface(), nil
}
https://play.golang.org/p/CzDrj6sgQNt
while writing a golang webserver I had to use some sort of cache so i chose redis.
I had the need for some sort of function that takes any structure and saves it as is to redis as a value.
Is there any way to do this without using the interface{} as a receiving parameter or repeating myself too much and still staying type safe?
Encode the struct value to a []byte using the gob, json or similar encoding package. Store the []byte in Redis. Reverse the process when fetching the data.
Assuming a Redis client with methods for Set and Get, the code using the JSON package will look something like this:
func set(c *RedisClient, key string, value interface{}) error {
p, err := json.Marshal(value)
if err != nil {
return err
}
return c.Set(key, p)
}
func get(c *RedisClient, key string, dest interface{}) error {
p, err := c.Get(key)
if err != nil {
return err
}
return json.Unmarshal(p, dest)
}
Use it like this to save a value:
var v someType
if err := set(c, key, v); err != nil {
// handle error
}
and like this to retrieve a value. Note that a pointer to the value is passed to get.
var v someType
if err := get(c, key, &v); err != nil {
// handle error
}
The details will need to adjusted depending on the Redis client that you are using.
This approach avoids repetition and is type safe as long as the application sets and gets values for a given key using the same type.
https://github.com/olivere/elastic
Version 5.x
The wiki documentation isn't really clear on how client.Update() works. It's needed to completely change a field and to modify arrays. i.e. in the example in the wiki documentation, how would one go about appending and removing tags to a tweet or changing a tweet's content? Also if a tweet was represented in go as a struct and I added a nested struct called "echo" which contains a foo of type int, content of type string and another type string array, how would one go about changing any of these fields using client.Update() if it's even possible?
In my personal example I have this function:
func UpdateEntryContent(eclient *elastic.Client, entryID string, newContent []rune) error{
ctx:=context.Background()
exists, err := eclient.IndexExists(ENTRY_INDEX).Do(ctx)
if err != nil {return err}
if !exists {return errors.New("Index does not exist")}
_, err = eclient.Update().Index(ENTRY_INDEX).Type(ENTRY_TYPE).Id(entryID).
Script("ctx._source.Content = newCont").
ScriptParams(map[string]interface{}{"newCont": newContent}).
Do(ctx)
if err != nil {return err}
return nil
}
But I get this following error when I try to compile:
cannot use "ctx._source.Content = newCont" (type string) as type *elastic.Script in argument to eclient.Update().Index(ENTRY_INDEX).Type(ENTRY_TYPE).Id(entryID).Script
eclient.Update().Index(ENTRY_INDEX).Type(ENTRY_TYPE).Id(entryID).Script("ctx._source.Content = newCont").ScriptParams undefined (type *elastic.UpdateService has no field or method ScriptParams)
The Script method accepts a *elastic.Script, not a string. The ScriptParams method is also found on *elastic.Script as Params instead of being on *elastic.UpdateService.
func UpdateEntryContent(eclient *elastic.Client, entryID string, newContent []rune) error{
ctx:=context.Background()
exists, err := eclient.IndexExists(ENTRY_INDEX).Do(ctx)
if err != nil {return err}
if !exists {return errors.New("Index does not exist")}
script := elastic.NewScript("ctx._source.Content = newCont").Params(map[string]interface{}{"newCont": newContent})
_, err = eclient.Update().Index(ENTRY_INDEX).Type(ENTRY_TYPE).Id(entryID).
Script(script).
Do(ctx)
if err != nil {return err}
return nil
}
You can see more information about the package with GoDoc or by looking through the source code.
The following code should resolve the issue
_, err = eclient.Update().Index(INDEX).
Type(TYPE).
Id(ID).
Doc(map[string]interface{}{field: message}).
Do(ctx)
Credit where it's due Gavin's answer put me on the right track. This is for another .Index but the full function that acts as a generic single field update is as follows:
func UpdateUser(eclient *elastic.Client, userID string, field string, newContent interface{})error {
//CHANGES A SINGLE FIELD OF ES USER DOCUMENT(requires an elastic client pointer,
// the user DocID, the feild you wish to modify as a string,
// and what you want to change that field to as any type necessary)
//RETURN AN error IF SUCESSFUL error = nil
ctx := context.Background()
exists, err := eclient.IndexExists(USER_INDEX).Do(ctx)
if err != nil {return err}
if !exists {return errors.New("Index does not exist")}
_, err = eclient.Update().
Index(USER_INDEX).
Type(USER_TYPE).
Id(userID).
Doc(map[string]interface{}{field: newContent}).
Do(ctx)
return nil
}
You can change the .Index, .Type, and .Id and it works with all fields and types as far as I can tell