Dealing with NULL values in app and sending to DB - go

I have nullable variables I'm trying to use in my app, and also send to a database which has columns that are default null.
This is a sample struct:
// Location type
type Location struct {
ID int `schema:"id"`
Title *string `schema:"title"`
}
Title is defined as *string, as it could be null (e.g. no user input or client app sends it as null).
Here’s my function receiving form data:
// JSONLocationCreate func
func (a *App) JSONLocationCreate(w http.ResponseWriter, r *http.Request) {
r.ParseForm()
var e Location
err := decoder.Decode(&e, r.PostForm)
if err != nil {
respondWithError(w, http.StatusBadRequest, "Invalid request payload")
return
}
// --- SUCCESS ---
// If e.Title has data, the following line works.
// --- FAIL ---
// If e.Title doesn’t have data (for whatever reason), it’s null, and crashes the app:
log.Println(*e.Title)
// Ultimately the variable would be sent off to a database.
// Below I’m removing other functions and such, just including my statement line.
// --- SUCCESS ---
// If e.Title has data, the following line works.
// --- FAIL ---
// If e.Title is null (e.g. no user input), this crashes the app.
statement := fmt.Sprintf("INSERT INTO locations(title) VALUES('%s')", *e.Title)
// In either case, the crash error is similar to this:
// panic serving [::1]:52459: runtime error: invalid memory address or nil pointer dereference
}
CONCERN 1: How can I make use of nullable variables (like e.Title), throughout the app, without throwing panic errors when the variable is null? Is the best practice to wrap it in a function that converts null to “” strings? How can such a function be applied transparently, so I don’t have to have something like “nullCheck(*e.Title)” on every instance of the variable?
CONCERN 2: In the case of my DB queries, I can’t be sending “” string values into the database in place of nulls. Up to now my queries are manually built. I suppose I need a function to generate the SQL queries automatically excluding columns and variables when the variables are null.
Am I on the right track? Any examples?
I haven’t understood all the threads/tutorials after hours of searching.

CONCERN 1 is easily remedied by adding a getter method for your fields
func (l Location) GetTitle() string {
if l.Title == nil {
return ""
}
return *l.Title
}
CONCERN 2:
It depends on what sql queries do you want to make, I would suggest that you look into some ORM libraries, which automate a lot of DB-specific code for you. Gorm is a good example of such library: https://github.com/jinzhu/gorm

Related

Go Scanner still returns <nil>

first of all: I am totally newbie in golang, so I may not understand well.
I am tasked to write a Go data extractor from one database using "gorp". The problem is with one table, that has custom field "TimeRange".
It is defined as:
type TimeRange struct {
From string
To string
}
Sadly when I try to fetch row I am getting scanner error, so I realised I need a custom scanner.
// Scan - Implement the database/sql scanner interface
func (tr *TimeRange) Scan(value interface{}) error {
tr.From = "mis"
tr.To = "lala"
fmt.Printf("%v\n", *tr)
return nil
}
So I expect to see fixed '{mis lala}' in returned string.
Why I am getting:
var q2 []models.Dashboard
result, err := dbmap.Select(&q2, "select * from dashboard where id=3")
fmt.Printf("q2=%v\n", q2)
prints:
p2=[{{<nil> Tomek b1f6f0ba-f618-00d6-6d24-8410a9219c95}}]
which is:
TimeRange, UserName and UUID
Might be important: using "gorp" for DB managment
the scan function would be called on passed value type to dbmap.Select() so in your case, you need to implement Dashboard as scanner.

Using empty interfaces in go

I am trying to understand the code that is used at my company. I am new to go lang, and I have already gone through the tutorial on their official website. However, I am having a hard time wrapping my head around empty interfaces, i.e. interface{}. From various sources online, I figured out that the empty interface can hold any type. But, I am having a hard time figuring out the codebase, especially some of the functions. I will not be posting the entire thing here, but just the minimal functions in which it has been used. Please bear with me!
Function (I am trying to understand):
func (this *RequestHandler) CreateAppHandler(rw http.ResponseWriter, r *http.Request) *foo.ResponseError {
var data *views.Data = &views.Data{Attributes: &domain.Application{}}
var request *views.Request = &views.Request{Data: data}
if err := json.NewDecoder(r.Body).Decode(request); err != nil {
logrus.Error(err)
return foo.NewResponsePropogateError(foo.STATUS_400, err)
}
requestApp := request.Data.Attributes.(*domain.Application)
requestApp.CreatedBy = user
Setting some context, RequestHandler is a struct defined in the same package as this code. domain and views are seperate packages. Application is a struct in the package domain. The following two structs are part of the package views:
type Data struct {
Id string `json:"id"`
Type string `json:"type"`
Attributes interface{} `json:"attributes"`
}
type Request struct {
Data *Data `json:"data"`
}
The following are part of the package json:
func NewDecoder(r io.Reader) *Decoder {
return &Decoder{r: r}
}
func (dec *Decoder) Decode(v interface{}) error {
if dec.err != nil {
return dec.err
}
if err := dec.tokenPrepareForDecode(); err != nil {
return err
}
if !dec.tokenValueAllowed() {
return &SyntaxError{msg: "not at beginning of value"}
}
// Read whole value into buffer.
n, err := dec.readValue()
if err != nil {
return err
}
dec.d.init(dec.buf[dec.scanp : dec.scanp+n])
dec.scanp += n
// Don't save err from unmarshal into dec.err:
// the connection is still usable since we read a complete JSON
// object from it before the error happened.
err = dec.d.unmarshal(v)
// fixup token streaming state
dec.tokenValueEnd()
return err
}
type Decoder struct {
r io.Reader
buf []byte
d decodeState
scanp int // start of unread data in buf
scan scanner
err error
tokenState int
tokenStack []int
}
Now, I understood that, in the struct Data in package views, Application is being set as a type for the empty interface. After that, a pointer to Request in the same package is created which points to the variable data.
I have the following doubts:
What exactly does this keyword mean in Go? What is the purpose of writing this * RequestHandler?
Initialization of a structure in Go can be done while assigning it to a variable by specifying the values of all it's members. However, here, for the struct Data, only the empty interface value is assigned and the values for the other two fields are not assigned?
What is the advantage of assigning the Application struct to an empty interface? Does it mean I can use the struct members using the interface variable directly?
Can someone help me figure out the meaning of this statement? json.NewDecoder(r.Body).Decode(request)?
While I know this is too much, but I am having a hard time figuring out the meaning of interfaces in Go. Please help!
this is not a keyword in go; any variable name can be used there. That is called the receiver. A function declared in that way must be called like thing.func(params), where "thing" is an expression of the type of the receiver. Within the function, the receiver is set to the value of thing.
A struct literal does not have to contain values for all the fields (or any of them). Any fields not explicitly set will have the zero value for their types.
As you said, an empty interface can take on a value of any type. To use a value of type interface{}, you would use type assertion or a type switch to determine the type of the value, or you could use reflection to use the value without having to have code for the specific type.
What specifically about that statement do you not understand? json is the name of a package in which the function NewDecoder is declared. That function is called, and then the Decode function (which is implemented by the type of the return value of NewDecoder) is called on that return value.
You may want to take a look at Effective Go and/or The Go Programming Language Specification for more information.

Is it possible to dump golang db.Query() output to a string?

I have a small Heroku app in which i print out name and age from each rows after query execution.
I want to avoid looping rows.Next(),Scan().. and just want to show what database returned after query execution which may be some data or error.
Can we directly dump data to a string for printing?
rows, err := db.Query("SELECT name FROM users WHERE age = $1", age)
if err != nil {
log.Fatal(err)
}
for rows.Next() {
var name string
if err := rows.Scan(&name); err != nil {
log.Fatal(err)
}
fmt.Printf("%s is %d\n", name, age)
}
if err := rows.Err(); err != nil {
log.Fatal(err)
}
Pretty much: No.
The Query method is going to return a pointer to a Rows struct:
func (db *DB) Query(query string, args ...interface{}) (*Rows, error)
If you print that (fmt.Printf("%#v\n", rows)) you'll see something such as:
&sql.Rows{dc:(*sql.driverConn)(0xc8201225a0), releaseConn:(func(error)(0x4802c0), rowsi:(*pq.rows)(0xc820166700), closed:false, lastcols:[]driver.Value(nil), lasterr:error(nil), closeStmt:driver.Stmt(nil)}
...probably not what you want.
Those correspond to the Rows struct from the sql package (you'll notice the fields are not exported):
type Rows struct {
dc *driverConn // owned; must call releaseConn when closed to release
releaseConn func(error)
rowsi driver.Rows
closed bool
lastcols []driver.Value
lasterr error // non-nil only if closed is true
closeStmt driver.Stmt // if non-nil, statement to Close on close
}
You'll see []driver.Value (an interface from the driver package), that looks like where we can expect to find some useful, maybe even human readable data. But when directly printed it doesn't appear useful, it's even empty... So you have to somehow get at the underlying information. The sql package gives us the Next method to start with:
Next prepares the next result row for reading with the Scan method.
It returns true on success, or false if there is no next
result row or an error happened while preparing it. Err
should be consulted to distinguish between the two cases.
Every call to Scan, even the first one, must be preceded by a call to Next.
Next is going to make a []driver.Value the same size as the number of columns I have, which is accessible (within the sql package) through driver.Rows (the rowsi field) and populate it with values from the query.
After calling rows.Next() if you did the same fmt.Printf("%#v\n", rows) you should now see that []diver.Value is no longer empty but it's still not going to be anything that you can read, more likely something resembling:[]diver.Value{[]uint8{0x47, 0x65...
And since the field isn't exported you can't even try and convert it to something more meaningful. But the sql package gives us a means to do something with the data, which is Scan.
The Scan method is pretty concise, with lengthy comments that I won't paste here, but the really important bit is that it ranges over the columns in the current row you get from the Next method and calls convertAssign(dest[i], sv), which you can see here:
https://golang.org/src/database/sql/convert.go
It's pretty long but actually relatively simple, it essentially switches on the type of the source and destination and converts where it can, and copies from source to destination; the function comments tell us:
convertAssign copies to dest the value in src, converting it if possible. An error is returned if the copy would result in loss of information. dest should be a pointer type.
So now you have a method (Scan) which you can call directly and which hands you back converted values. Your code sample above is fine (except maybe the call to Fatal() on a Scan error).
It's important to realize that the sql package has to work with a specific driver, which is in turn implemented for specific database software, so there is quite some work going on behind the scenes.
I think your best bet if you want to hide/generalize the whole Query() ---> Next() ---> Scan() idiom is to drop it into another function which does it behind the scenes... write a package in which you abstract away that higher level implementation, as the sql package abstracts away some of the driver-specific details, the converting and copying, populating the Rows, etc.

golang - Rest update request using http patch semantics

type A struct {
Id int64
Email sql.NullString
Phone sql.NullString
}
Assume I have one record in the database
A{1, "x#x.com", "1112223333"}
Send an update request via PUT
curl -X PUT -d '{"Email": "y#y.com", "Phone": null}' http://localhost:3000/a/1
Here is the psuedo algorithm that would work with a full PUT request (i.e. update all fields of the record A - but it will cause difficulties with the PATCH request semantics - delta update)
-- Unmarshal json into empty record
a := A{}
json.Unmarshal([]byte(request.body), &a)
-- Load the record from the database
aFromDb = <assume we got record from db> //A{1, "x#x.com", "1112223333"}
-- Compare a and aFromDB
-- Notice the email change and set it on aFromDb - ok
-- Notice the phone number change -- but wait! Was it set to NULL in the JSON explicitly or was it not even included in the JSON? i.e. was the json request - {"Email": "y#y.com", "Phone": null} or was it {"Email": "y#y.com"}?
How can we tell by just looking at the unmarshaled json into the struct a?
Is there another method to do the update via rest (with patch semantics)? I am looking for a generic way to do it (not tied to a particular struct).
I created a separate datatype for this purpose. This example is for an int64 (actually string-encoded int64), but you can easily change it to a string as well. The idea behind it is, that the UnmarshalJSON method will only be called if the value is present in the JSON. The type will implement the Marshaler and the Unmarshaler.
// Used as a JSON type
//
// The Set flag indicates whether an unmarshaling actually happened on the type
type RequiredInt64 struct {
Set bool
Int64 int64
}
func (r RequiredInt64) MarshalJSON() ([]byte, error) {
lit := strconv.FormatInt(r.Int64, 10)
return json.Marshal(lit)
}
func (r *RequiredInt64) UnmarshalJSON(raw []byte) error {
var lit string
var err error
if err = json.Unmarshal(raw, &lit); err != nil {
return err
}
r.Int64, err = strconv.ParseInt(lit, 10, 64)
if err != nil {
return err
}
r.Set = true
return nil
}
So, if Set is false, you know that the value was not present in the JSON.
Try adding this tags to the struct:
type A struct {
Id int64 `json:"Id,omitempty"`
Email sql.NullString `json:"Email,omitempty"`
Phone sql.NullString `json:"Phone,omitempty"`
}
In this way if you are serializing and the field is empty then the json will not contain the field.
When deserializing though the field will have a either a value or it will have the default value for the type (Nil for the pointer or empty string for strings).
You could potentially write your own marshalling/uinmarshalling of your struct and react to the raw response within, although it might be non-obvious witgh inspection what that those functions are manipulating.
Alternatively, you could not omitempty within your fields and force null populated fields.
Or, maybe leveraging a different flavor of patching, perhaps http://jsonpatch.com/, which is more explicit in the nature of your modifications. This would require the client to be more understanding of the state of changes than say for a put.

Golang gorilla sessions preserving form data after redirect

From logic point of view I am trying to preserve partial form data between redirects for better user experience so user won't have to fill entire form again, just the part that was invalid.
From programing point of view I am trying to save request.PostForm data structure in gorilla session's flashes. The only thing I manage to retrieve after redirect is string representation of memory address like this [0xc2001c8b10].
Here is the part where I save flashes data after validation error (request.ParseForm() was executed before this):
session, _ := store.Get(request, "test")
session.AddFlash(err.Error(), "messages")
session.AddFlash(request.PostForm, "form_data")
session.Save(request, response)
http.Redirect(response, request, "/", http.StatusFound)
return
Also I tried registering structure with gob without effect:
func init() {
gob.Register(&url.Values{})
}
Form values are in lower case, eg. "first_name", "last_name" if that could have any influence on this behavior.
Please keep in mind that I successfully manage to retrieve "messages" after redirect, only problem I have is with structural data.
Am I doing something wrong or is there maybe another approach to fill partial forms after redirect that I am not aware of?
Your problem is that you're working with values of type interface{}, which is the generic type
and used when there can be more than one type. This is the case for gorilla's session.Flashes()
method as it can return arbitrary user data (whatever you put in).
You can reproduce what you're experiencing with this code (on play):
type MyData struct {
X int
}
// Simulate Flashes() from gorilla, which returns a slice of interface{} values.
func Flashes() []interface{} {
x := &MyData{2}
// Convert x to type interface{}
interfaceValue := interface{}(x)
// Put converted x into a slice of type []interface{}
return []interface{}{interfaceValue}
}
func main() {
// See that [0xSOMETHING] is printed
fmt.Println("Some data:", Flashes())
}
When running this program you will see output like this:
Some data: [0xc010000000]
This is the same you're experiencing. The reason for this is that fmt.Println does
not step through all levels of abstraction of pointers and interfaces and stops at
a certain level unless you tell it to print everything. So if you use
fmt.Printf("Some data: %#v\n", Flashes())
you will indeed see your data:
Some data: []interface {}{(*main.MyData)(0xc010000000)}
What you have to do to access the data is to match the resulted data for the type
you're expecting. You have to do a type assertion (example on play):
func main() {
value := Flashes()[0]
v, ok := value.(*MyData)
if ok {
fmt.Println("Some data:", v)
} else {
fmt.Println("Oh no, there's something else stored than expected")
}
}
In the example above the first flash returned by Flashes() is used and asserted to be
of type *MyData. If it is indeed this type, then it's value is printed to the console.
Otherwise an error message (albeit not a good one) is printed to the console.
After asserting a variable of being some type, the asserted value is of the asserted
type. That is the v in the example above is of type *MyType.

Resources