I have two servers
User Server: handle all the user CRUD operation
Product Server: Handle Product CRUD operation and fetch user info from the user server via gRPC call
type User struct {
ID string `json:"id"`
FirstName string `json:"firstName"`
MiddleName *string `json:"middleName,omitempty"`
LastName string `json:"lastName"`
Email string `json:"email"`
Disabled bool `json:"disabled"`
LastSignedInAt *time.Time `json:"lastSignedInAt,omitempty"`
Bio *string `json:"bio,omitempty"`
BirthDate *time.Time `json:"birthDate,omitempty"`
}
Here some fields are optionals and as I am using cockroachDB(extended postgreSQL), I kept them as a pointer so it's easy in scaning variable form query result.
And here is my proto file:
message User {
int64 id = 1;
string firstName = 2;
string lastName = 3;
string email = 5;
bool disabled = 6;
string lastSignedInAt = 8;
string bio = 9;
string birthdate = 10;
string profession = 14;
}
Now generated model from above proto file is like this:"
type User struct {
Id int64 `protobuf:"varint,1,opt,name=id,proto3" json:"id,omitempty"`
FirstName string `protobuf:"bytes,2,opt,name=firstName,proto3" json:"firstName,omitempty"`
LastName string `protobuf:"bytes,3,opt,name=lastName,proto3" json:"lastName,omitempty"`
Email string `protobuf:"bytes,4,opt,name=email,json=email,proto3" json:"email,omitempty"`
Disabled bool `protobuf:"varint,6,opt,name=disabled,proto3" json:"disabled,omitempty"`
LastSignedInAt string `protobuf:"bytes,8,opt,name=lastSignedInAt,proto3" json:"lastSignedInAt,omitempty"`
Bio string `protobuf:"bytes,9,opt,name=bio,proto3" json:"bio,omitempty"`
Birthdate string `protobuf:"bytes,10,opt,name=birthdate,proto3" json:"birthdate,omitempty"`
}
Now the problem is as I am using a pointer for the optional field it will store null in case of no value but on the opposite site gRPC won't understand the null value and throw the error.
I have tried google.protobuf.StringValue value as a grpc type like this
google.protobuf.StringValue lastSignedInAt = 8;
This works but the problem is I have to write condition for each field in the handler:
if lastSignedInAt != nil {
user.LastSignedInAt = &wrappers.StringValue{Value:*lastSignedInAt}
}
What is the best way to tackle this issue? Should I change the database model or any changes in the gRPC model?
As another answer above noted, if the protocol buffer message has fields that can be nil it is on you to check for them. There isn't much you can do about this, unless there's a utility package out there that does it.
If you want defaults above and beyond the defaults protocol buffers generates, you'll have to do exactly what you noted and check for nil.
I do have some questions though:
Why use pointers for the "optional" fields in first User struct? If you used plain-old string they would be populated with the empty string on construction and untouched by the json unmarshal call if the field is missing. Same for the time fields. In this case the default value for string (the empty string) should be an invalid middle name, and the default value for time (0001-01-01 00:00:00 +0000 UTC) is an invalid timestamp (probably?). Getting rid of the pointers allows you to use the defaults.
For the timestamp in the proto struct you could still use string and check for the empty string. Or you could use google.protobuf.Timestamp and use ptypes to handle the conversions to/from the non-proto struct. See https://godoc.org/github.com/golang/protobuf/ptypes#TimestampProto. You'll have to check for nil in this case.
You cannot set the null value instead of you you can use
oneof Examples {
Example1 example1 = 1;
Example2 example2 = 2;
}
when you use oneof you have to set only one value either you can set example1 or example2 you cannot use both at same time. This will resolve your issue as compared to setting the nil value.
Approach 2:
And by default gRPC have all the variable has initial value ex: string: ""
One thing you can also do don't set nil value check with the condition if your value is nil then set nothing.
grpc not recommended to use point field, If you insist on use pointers
the one way is use reflect to convert it
func ToGrpcData(in, out interface{}) error {
inVal := reflect.ValueOf(in)
if inVal.Kind() == reflect.Ptr {
inVal = inVal.Elem()
}
inTyp := inVal.Type()
outVal := reflect.ValueOf(out)
if outVal.Kind() != reflect.Ptr {
return errors.New("out data must be point value")
}
outVal = outVal.Elem()
outTyp := outVal.Type()
strWrapperType := reflect.TypeOf(wrappers.StringValue{})
// range all 'in' fields
for i := 0; i < inVal.NumField(); i++ {
fTyp := inTyp.Field(i)
fVal := inVal.Field(i)
if fTyp.Type.Kind() == reflect.Ptr {
switch fTyp.Type.Elem().Kind() {
case reflect.String: // only implement string in this test
oFTyp, ok := outTyp.FieldByName(fTyp.Name)
if ok == false {
return errors.New("not match field in out value")
}
if oFTyp.Type.Elem() != strWrapperType {
return errors.New("not match field in out value")
}
if fVal.IsNil() == false {
outVal.FieldByName(fTyp.Name).Set(
reflect.ValueOf(&wrappers.StringValue{
Value: fVal.Elem().String(),
}),
)
}
}
} else {
outVal.FieldByName(fTyp.Name).Set(fVal)
}
}
return nil
}
usage
type User struct {
Name string
Value *string
}
type ServerUser struct {
Name string
Value *wrappers.StringValue
}
usValue := "123"
u1 := User{
Name: "tom",
Value: &usValue,
}
u2 := ServerUser{}
err := ToGrpcData(&u1, &u2)
// error process ...
fmt.Println(u2)
Related
Given the code below, is it possible to transform first name while it's being unmarshaled? Say I want to always have it be lowercase whether it is in the actual json or not.
type Person struct {
FirstName string `json:"first_name"`
LastName string `json:"last_name"`
}
func main() {
jsonText := GetJsonFromSomewhere()
var v Person
json.Unmarshal(jsonText, &v)
}
One way to do this is to create a custom type that implements the Unmarshaler interface from the encoding/json package. Here's a link to this interface. Any type which implements Unmarshaler can be used as the type of a struct field when doing JSON unmarshalling. When doing the unmarshalling, encoding/json will use your implementation of the interface's UnmarshalJSON function to convert the JSON bytes to the field type.
So you could write an UnmarshalJSON function which includes changing the string values to lowercase.
Here's an example of what that could look like:
type LowerCaseString string
func (l *LowerCaseString) UnmarshalJSON(bytes []byte) error {
lowerCasedString := strings.ToLower(string(bytes))
*l = LowerCaseString(lowerCasedString)
return nil
}
Then, in your struct for JSON mapping, you can use your custom type instead of string:
type Person struct {
FirstName LowerCaseString `json:"first_name"`
LastName LowerCaseString `json:"last_name"`
}
If you unmarshal into this struct, the values of FirstName and LastName will be lowercased (also note that you'll need to type convert them back to string to use them as strings).
testJSON := `{"first_name" : "TestFirstNAME", "last_name": "TestLastNAME"}`
var result Person
err := json.Unmarshal([]byte(testJSON), &result)
if err != nil { /*handle the error*/ }
fmt.Println(result.FirstName) // prints "testfirstname"
var stringLastName string
stringLastName = string(result.LastName) // need to type convert from LowerCaseString to string
fmt.Println(stringLastName) // prints "testlastname"
Here is the above code running in Go Playground.
I am creating a gorm model
// Day is a corresponding day entry
type Day struct {
gorm.Model
Dateday string `json:"dateday" gorm:"type:date;NOT NULL"`
Nameday string `json:"nameday" gorm:"type:varchar(100);NOT NULL"`
Something sql.NullString `json:"salad"`
Holyday bool `json:"holyday"`
}
I am using sql.NullString for the field Something cause it may be NULL.
So when I try to execute a typical gorm example to validate my setup works:
db.Create(&Day{
Nameday: "Monday",
Dateday: "23-10-2019",
Something: "a string goes here",
Holyday: false,
})
I get:
cannot use "a string goes here", (type string) as type sql.NullString in field value
What type should I use for the Something field given it may be NULL?
The sql.NullString type is not actually a string type but a struct type. It's defined as:
type NullString struct {
String string
Valid bool // Valid is true if String is not NULL
}
Therefore you need to initialize it as such:
db.Create(&Day{
Nameday: "Monday",
Dateday: "23-10-2019",
Something: sql.NullString{String: "a string goes here", Valid: true},
Holyday: false,
})
As an alternative, if you want to keep using the simpler syntax when initializing a nullable string, you could declare your own nullable string type, have it implement the sql.Scanner and driver.Valuer interfaces, and leverage the null byte to signal a NULL value.
type MyString string
const MyStringNull MyString = "\x00"
// implements driver.Valuer, will be invoked automatically when written to the db
func (s MyString) Value() (driver.Value, error) {
if s == MyStringNull {
return nil, nil
}
return []byte(s), nil
}
// implements sql.Scanner, will be invoked automatically when read from the db
func (s *String) Scan(src interface{}) error {
switch v := src.(type) {
case string:
*s = String(v)
case []byte:
*s = String(v)
case nil:
*s = StringNull
}
return nil
}
With this, if you declare the field Something to be of type MyString you can initialize it as you originally intended.
db.Create(&Day{
Nameday: "Monday",
Dateday: "23-10-2019",
// here the string expression is an *untyped* string constant
// that will be implicitly converted to MyString because
// both `string` and `MyString` have the same *underlying* type.
Something: "a string goes here",
Holyday: false,
})
Just keep in mind that this works only with untyped constants, once you have a constant or variable of type string, to be able to assign that to a MyString you'll need to use an explicit conversion.
var s string
var ms MyString
s = "a string goes here"
ms = s // won't compile because s is not an untyped constant
ms = MyString(s) // you have to explicitly convert
package main
import (
"github.com/guregu/null"
)
func main() {
db.Create(&Day{
Nameday: null.StringFrom("Monday"),
})
}
I have such struct in Golang application:
type Employee struct {
ID int `json:"employee_id"`
Email *string `json:"employee_email"`
LastName *string `json:"employee_last_name"`
FirstName *string `json:"employee_first_name"`
Sex *string `json:"employee_sex"`
}
Some string fields of this struct can be empty. If I use *string application return me "". If use sql.NullString it return me for example such result:
"employee_last_name": {
String: "",
Valid: true
}
I want to show null is string field is empty.
In documentation I found such code:
type NullString struct {
String string
Valid bool
}
func (ns *NullString) Scan(value interface{}) error {
if value == nil {
ns.String, ns.Valid = "", false
return nil
}
ns.Valid = true
return convertAssign(&ns.String, value)
}
func (ns NullString) Value() (driver.Value, error) {
if !ns.Valid {
return nil, nil
}
return ns.String, nil
}
As I understand this code could help me to solve my problem, right?! What I need to import in my application to use convertAssign function?
In the Go language specification nil is not a valid value for type string. the default, zero value is "".
The top answer for this question goes into more detail.
Edit:
What you want to do is not possible according to the oracle go driver docs:
sql.NullString
sql.NullString is not supported: Oracle DB does not
differentiate between an empty string ("") and a NULL
Original Answer:
Need more details on the SQL database type you are connecting to. I know the go SQLite3 pkg - github.com/mattn/go-sqlite3 - supports setting nil for values of type *string.
Check the details of the driver you are using to connect to the database. For instance with MySQL, getting SQL date fields into go's native time.Time struct type is not set by default - it must be turned on at the driver level.
Since empty string is the zero/default value for Go string, I decided to define all such fields as interface{} instead. for example
type student struct {
FirstName interface{} `json:"first_name"`
MiddleName interface{} `json:"middle_name"`
LastName interface{} `json:"last_name"`
}
The application I am sending my data expect a null instead of an empty string if value is not available for that specific field.
Is this the correct approach or can someone please point me to something better than this.
In json package documentation :
Pointer values encode as the value pointed to. A nil pointer encodes as the null JSON object.
So you can store a pointer to a string which will be encoded as a string if not nil and will be encoded as "null" if nil
type student struct {
FirstName *string `json:"first_name"`
MiddleName *string `json:"middle_name"`
LastName *string `json:"last_name"`
}
For the case of a json object with null strings, its easiest to use the omitempty decorator on the field.
type student struct {
FirstName string `json:"first_name,omitempty"`
MiddleName string `json:"middle_name,omitempty"`
LastName string `json:"last_name"`
}
With the above declaration, only if first_name was assigned will that key show up in the resultant json. last_name, on the otherhand, will always show up in the result with a value of "" if not assigned.
Now when you start including numeric fields where 0 could be a value, using omitempty doesn't do what one would expect. A 0 value always drops the field, and we need to be able to differentiate between a 0 value and an unassigned value. Here use of library such as https://github.com/guregu/null may be advisable.
More discussion here: https://www.sohamkamani.com/blog/golang/2018-07-19-golang-omitempty/
Can be used https://github.com/guregu/null
type student struct {
FirstName null.String `json:"first_name"`
MiddleName null.String `json:"middle_name"`
LastName null.String `json:"last_name"`}
Another way actually is a workaround with using the MarhshalJSON and UnmarshalJSON interface method offered by the json lib of golang. The code is as below:
type MyType string
type MyStruct struct {
A MyType `json:"my_type"`
}
func (c MyType) MarshalJSON() ([]byte, error) {
var buf bytes.Buffer
if len(string(c)) == 0 {
buf.WriteString(`null`)
} else {
buf.WriteString(`"` + string(c) + `"`) // add double quation mark as json format required
}
return buf.Bytes(), nil
}
func (c *MyType)UnmarshalJSON(in []byte) error {
str := string(in)
if str == `null` {
*c = ""
return nil
}
res := MyType(str)
if len(res) >= 2 {
res = res[1:len(res)-1] // remove the wrapped qutation
}
*c = res
return nil
}
then when using json.Marshal, the MyType value will be marshaled as null.
You could use something like sql.NullString,use 'Valid' to check if it is a nil value.
NullString represents a string that may be null. NullString implements
the Scanner interface so it can be used as a scan destination:
type NullString struct {
String string
Valid bool // Valid is true if String is not NULL
}
var s NullString
err := db.QueryRow("SELECT name FROM foo WHERE id=?", id).Scan(&s)
...
if s.Valid {
// use s.String
} else {
// NULL value
}
Please DO NOT use pointers like below:
type student struct {
FirstName *string `json:"first_name"`
MiddleName *string `json:"middle_name"`
LastName *string `json:"last_name"`
}
Because you need check nil value and dereference like below everywhere,and maybe cause unexpected crash somewhere:
if obj.FirstName != nil {
fmt.Print("%s", *obj.FirstName)
}
I have compared two solution and choose former method in my extensive production code... it works ok.
Since empty string is the zero/default value for Go string, I decided to define all such fields as interface{} instead. for example
type student struct {
FirstName interface{} `json:"first_name"`
MiddleName interface{} `json:"middle_name"`
LastName interface{} `json:"last_name"`
}
The application I am sending my data expect a null instead of an empty string if value is not available for that specific field.
Is this the correct approach or can someone please point me to something better than this.
In json package documentation :
Pointer values encode as the value pointed to. A nil pointer encodes as the null JSON object.
So you can store a pointer to a string which will be encoded as a string if not nil and will be encoded as "null" if nil
type student struct {
FirstName *string `json:"first_name"`
MiddleName *string `json:"middle_name"`
LastName *string `json:"last_name"`
}
For the case of a json object with null strings, its easiest to use the omitempty decorator on the field.
type student struct {
FirstName string `json:"first_name,omitempty"`
MiddleName string `json:"middle_name,omitempty"`
LastName string `json:"last_name"`
}
With the above declaration, only if first_name was assigned will that key show up in the resultant json. last_name, on the otherhand, will always show up in the result with a value of "" if not assigned.
Now when you start including numeric fields where 0 could be a value, using omitempty doesn't do what one would expect. A 0 value always drops the field, and we need to be able to differentiate between a 0 value and an unassigned value. Here use of library such as https://github.com/guregu/null may be advisable.
More discussion here: https://www.sohamkamani.com/blog/golang/2018-07-19-golang-omitempty/
Can be used https://github.com/guregu/null
type student struct {
FirstName null.String `json:"first_name"`
MiddleName null.String `json:"middle_name"`
LastName null.String `json:"last_name"`}
Another way actually is a workaround with using the MarhshalJSON and UnmarshalJSON interface method offered by the json lib of golang. The code is as below:
type MyType string
type MyStruct struct {
A MyType `json:"my_type"`
}
func (c MyType) MarshalJSON() ([]byte, error) {
var buf bytes.Buffer
if len(string(c)) == 0 {
buf.WriteString(`null`)
} else {
buf.WriteString(`"` + string(c) + `"`) // add double quation mark as json format required
}
return buf.Bytes(), nil
}
func (c *MyType)UnmarshalJSON(in []byte) error {
str := string(in)
if str == `null` {
*c = ""
return nil
}
res := MyType(str)
if len(res) >= 2 {
res = res[1:len(res)-1] // remove the wrapped qutation
}
*c = res
return nil
}
then when using json.Marshal, the MyType value will be marshaled as null.
You could use something like sql.NullString,use 'Valid' to check if it is a nil value.
NullString represents a string that may be null. NullString implements
the Scanner interface so it can be used as a scan destination:
type NullString struct {
String string
Valid bool // Valid is true if String is not NULL
}
var s NullString
err := db.QueryRow("SELECT name FROM foo WHERE id=?", id).Scan(&s)
...
if s.Valid {
// use s.String
} else {
// NULL value
}
Please DO NOT use pointers like below:
type student struct {
FirstName *string `json:"first_name"`
MiddleName *string `json:"middle_name"`
LastName *string `json:"last_name"`
}
Because you need check nil value and dereference like below everywhere,and maybe cause unexpected crash somewhere:
if obj.FirstName != nil {
fmt.Print("%s", *obj.FirstName)
}
I have compared two solution and choose former method in my extensive production code... it works ok.