I have an optional field on my struct called ExpireTime. It has a time.Time type and a json:"expire_time,omitempty" tag to not send it, when it is empty. This part works perfectly fine.
When I want to use the same field via GRPC, I run into an issue when converting it to the protobuf timestamp format.
type Timestamp struct {
// Represents seconds of UTC time since Unix epoch
// 1970-01-01T00:00:00Z. Must be from 0001-01-01T00:00:00Z to
// 9999-12-31T23:59:59Z inclusive.
Seconds int64 `protobuf:"varint,1,opt,name=seconds,proto3" json:"seconds,omitempty"`
// Non-negative fractions of a second at nanosecond resolution. Negative
// second values with fractions must still have non-negative nanos values
// that count forward in time. Must be from 0 to 999,999,999
// inclusive.
Nanos int32 `protobuf:"varint,2,opt,name=nanos,proto3" json:"nanos,omitempty"`
// contains filtered or unexported fields
}
ExpireTime *timestamppb.Timestamp `protobuf:"bytes,1,opt,name=expire_time,json=expireTime,proto3" json:"expire_time,omitempty"`
The issue is that an empty time.Time{} object will be converted to a negative seconds value corresponding to 0001-01-01T00:00:00Z. Having the omitEmpty flag will not be applied in this case as the value is not zeroed out. What could I do to omit this field, when it is actually empty? Thanks!
As you say time.Time{} converts to 0001-01-01T00:00:00Z; this is working as intended. Note that you also need to be careful converting in the opposite direction (a zero TimeStamp will become 1970-01-01T00:00:00Z).
However generally the Timestamp will be part of a message, for example:
message MyMessage{
google.protobuf.Timestamp comm_time = 1;
}
Running this through protoc will result in something like:
type MyMessage struct {
state protoimpl.MessageState
sizeCache protoimpl.SizeCache
unknownFields protoimpl.UnknownFields
CommTime *timestamppb.Timestamp `protobuf:"bytes, 1,opt,name=comm_time,json=commTime,proto3" json:"comm_time,omitempty"`
}
This means you should be able to achieve the result you are looking for with CommTime=nil; e.g.
sourceTime := time.Time{} // Whatever time you want to encode
var commTime *timestamp.Timestamp
if !sourceTime.IsZero() {
commTime = timestamppb.New(sourceTime)
}
msg := MyMessage{
CommTime: commTime,
}
Related
Go has methods to extract almost every component of a timestamp, eg time.Second(), time.Nano(), but none to extract the millisecond portion of a timestamp.
How does one extract the millisecond value of a timestamp.
eg, in the case of a timestamp like:
2021-01-07 10:33:06.511
i want to extract 511
To access the fraction seconds, you may use time.Nanosecond(). And if we convert it to time.Duration (time.Duration is exactly the nanoseconds count), we can take advantage of its Duration.Milliseconds() method (which of course does no magic but code will be clearer and easier to read):
func extractMs(t time.Time) int64 {
return time.Duration(t.Nanosecond()).Milliseconds()
}
Try it on the Go Playground.
there is an answer in the comments, but i want to post here to be cannonical:
func extractMillisecond(t time.Time) int {
ms := time.Duration(t.Nanosecond()) / time.Millisecond
return int(ms)
}
I am creating a Go application which will use GRPC and protobuf. My RPC service shall take a message containing the type google.protobuf.Timestamp, parse it and eventually save it in a database or perform some more operations on it.
I am confused as to what is considered valid input for the type google.protobuf.Timestamp. I wish to use the following format for the datetimestamp with the timezone offset.
2019-02-15T13:00:00+01:00
Here is the proto file I am using.
syntax = "proto3"
package example;
import "google/protobuf/timestamp.proto"
service Tester {
rpc ParseDateTimeStamp(TSRequest) returns (TSReply) {}
}
message TSRequest {
google.protobuf.Timestamp dts = 1;
}
message TSReply {
string message = 1;
}
The issue is that when I send a message to the GRPC server containing the datetimestamp. I would expect that the type *tsbp.Timestamp for the 2019-02-15T13:00:00+01:00 datetimestamp given to be valid and give me the appropriate seconds from epoch. (After invoking GetSeconds() from the timestamp.go)
Invoking ptypes.TimestampString(ts *tspb.Timestamp) returns 1970-01-01T00:00:00Z for the example input above.
Does the google.protobuf.Timestamp accept datetimestamps with the +- offset?
Or do I have to have the input in a String type and then parse to time.Time with time.Format instead of using the timestamp variable type in protobuf? If so could you provide an example of this?
A gRPC message type of google.protobuf.Timestamp, internally, is just two int64's
message Timestamp {
// Represents seconds of UTC time since Unix epoch
// 1970-01-01T00:00:00Z. Must be from 0001-01-01T00:00:00Z to
// 9999-12-31T23:59:59Z inclusive.
int64 seconds = 1;
// Non-negative fractions of a second at nanosecond resolution. Negative
// second values with fractions must still have non-negative nanos values
// that count forward in time. Must be from 0 to 999,999,999
// inclusive.
int32 nanos = 2;
}
So when in this format-type, there is nothing to parse.
One typically takes:
a string format like your 2019-02-15T13:00:00+01:00 and converts to a time.Time using time.Parse
then convert the time.Time to a *tspb.Timestamp using ptypes.TimestampProto()
FYI in your cited output, you are seeing a zero timestamp (i.e. both seconds and nanos are zero) - hence the "1970-01-01T00:00:00Z" output.
Implementing the flow above:
ts, err := time.Parse(time.RFC3339, "2019-02-15T13:00:00+01:00")
pbts, err := ptypes.TimestampProto(ts) // ptypes.Timestamp:"seconds:1550232000 "
fmt.Println(ptypes.TimestampString(pbts)) // "2019-02-15T12:00:00Z"
Playground
Note: the ptype.Timestamp is stripped of any timezone - Z so UTC time. So if you need to preserve timezone of a time.Time, the offset would need to be sent in your gRPC message in addition to your google.protobuf.Timestamp message.
I've read both Optional Parameters? and Golang pass nil as optional argument to a function?
And still wonder if my case is more specific.
What I've got is:
type Periodical struct {
Interval *interval.Interval
StartsAt time.Time
EndsAt time.Time
}
to represent periodical event which has a start date and may or may not have an end date (periodical event runs for indefinite amount of time).
eachYear := Periodical{
interval.Years(1),
StartsAt: time.Parse("2 Jan 2006", "1 Jan 1970")}
Will throw
periodical/periodical.go:31:39: cannot use nil as type time.Time in field value
Which is understood, - I didn't specify EndsAt time.Time.
But what do I really do there then?
Am I forced to have a special flag to neglect EndsAt like so?
type Periodical struct {
Interval *interval.Interval
StartsAt time.Time
EndsAt time.Time
isIndefinite bool // This looks ugly already
}
and then if I want Yearly / Anually I do something like
eachYear := Periodical{
interval.Years(1),
time.Parse("2 Jan 2006", "1 Jan 1970"),
time.Parse("2 Jan 2006", "1 Jan 1970"),
isIndefinite: true}
Although, I can then account for this flag in business logic, but this EndsAt set to the same (or any other) date looks kind of dull.
I also define a method on periodical package which allows to have a shorthand periodical event like so:
func Monthly(s, e time.Time) Periodical {
return Periodical{StartsAt: s, EndsAt: e, Interval: interval.Months(1)}
}
What do I do to omit end (the second param)? Am I forced to either have separate method for that or do something that looks a bit funky and lacks readability:
func Monthly(s time.Time, end ...time.Time) Periodical {
if len(end) == 1 {
return Periodical{
StartsAt: s,
EndsAt: end[0],
Interval: interval.Months(1),
isIndefinite: false}
} else if len(end) > 1 {
panic("Multiple end dates are not allowed, don't know what to do with those")
} else {
return Periodical{
StartsAt: s,
EndsAt: time.Now(),
Interval: interval.Months(1),
isIndefinite: true}
}
}
Although it does the trick, it looks ugly, isn't it? My concise one-liner is now scattered along several lines of code.
So, that's why I wonder, what's the go's idiomatic way of achieving what I'm trying to do?
time.Time is a struct. Its zero value–although being a valid time value–is like never used. You can utilize the zero value time to signal the missing or undefined time value. There is even a Time.IsZero() method which tells you if a time.Time value is the zero value.
Note that the zero value time is not Jan 1, 1970 like in some other languages, but Jan 1, year 1, as you can see on the Go Playground. This is also documented at time.Time:
The zero value of type Time is January 1, year 1, 00:00:00.000000000 UTC. As this time is unlikely to come up in practice, the IsZero method gives a simple way of detecting a time that has not been initialized explicitly.
Also, when creating a Periodical value, use keyed composite literal, and you can omit fields which you don't want to set, thus leaving them at their zero value:
eachYear := Periodical{
Interval: interval.Years(1),
StartsAt: time.Date(1970, 1, 1, 0, 0, 0, 0, time.UTC),
}
Note that you can't use time.Parse() in this composite literal as that returns 2 values (a time.Time and an error). Either use time.Date() as in the above example, or create the time value prior (handle error), and just use the time value.
To tell if EndsAt is specified:
if eachYear.EndsAt.IsZero() {
fmt.Println("EndsAt is missing")
}
Should you need to zero an already set (non-zero) time value, you may use the time.Time{} composite literal:
eachYear.StartsAt = time.Time{}
Also note though that when marshaling a time.Time value, even if it's the zero value (since it is a valid time value), it will be sent even if you use the omitempty option. In those cases you must use a *time.Time pointer or write custom marshalers. For details, see Golang JSON omitempty With time.Time Field.
In my RESTFUL web service which is an online game, I'm storing starting time of every question in an global variable like this: var MyTime time.Time which I should update it after every level of the game. My application is distributed, so I want to make sure all of my apps are not updating it at the same time. That's why I've decided to make it atomic.
Actually I'm familiar with Golang sync/atomic package.
I tried to use atomic.LoadPointer() method but it needs specific argument type which isn't safe. Do you any other way for this?
Update:
Okay I solved my problem like this.
I defined time variable as atomic.Value and used atomic Load and Store methods. This is the code:
var myTime atomic.Value
myTime.Store(newTime) and load myTime.Load().(time.Time).
Consider that Load() method returns interface, so you should write (time.Time) at the end in order to convert it to time.Time type.
This can't be done, as such, because time.Time is a compound type:
type Time struct {
// wall and ext encode the wall time seconds, wall time nanoseconds,
// and optional monotonic clock reading in nanoseconds.
//
// From high to low bit position, wall encodes a 1-bit flag (hasMonotonic),
// a 33-bit seconds field, and a 30-bit wall time nanoseconds field.
// The nanoseconds field is in the range [0, 999999999].
// If the hasMonotonic bit is 0, then the 33-bit field must be zero
// and the full signed 64-bit wall seconds since Jan 1 year 1 is stored in ext.
// If the hasMonotonic bit is 1, then the 33-bit field holds a 33-bit
// unsigned wall seconds since Jan 1 year 1885, and ext holds a
// signed 64-bit monotonic clock reading, nanoseconds since process start.
wall uint64
ext int64
// loc specifies the Location that should be used to
// determine the minute, hour, month, day, and year
// that correspond to this Time.
// The nil location means UTC.
// All UTC times are represented with loc==nil, never loc==&utcLoc.
loc *Location
}
However, you can do this with pointers, so *time.Time would be possible, if this suits your needs. But of course, this is discouraged, by virtue of the fact that atomic.LoadPointer and atomic.StorePointer use the unsafe package to accomplish their magic.
A much better approach, if it will work for you, is just to use a mutex to protect your value. There are many ways to do this, but one minimal example:
type MyTime struct {
t time.Time
mu sync.RWMutex
}
func (t *MyTime) Time() time.Time {
t.mu.RLock()
defer t.mu.RUnlock()
return t.t
}
func (t *MyTime) SetTime(tm time.Time) {
t.mu.Lock()
defer t.mu.Unlock()
t.t = tm
}
You can keep unix time https://golang.org/pkg/time/#example_Time_Unix as atomic which is int64. Then convert to go time after you've read atomic value.
If you are only interested in the timestamp, you could simply keep a reference to the unix time, which is int64 and atomically update that.
var ts int64
func main() {
atomic.StoreInt64(&ts, time.Now().Unix())
t := time.Unix(atomic.LoadInt64(&ts), 0)
fmt.Println(t)
}
Instead, if you need the entire Time struct, read on.
Go 1.19 (still in beta)
If you are okay with storing a pointer to a time.Time object, you can use atomic.Pointer, which is a generic struct that abstracts atomic.LoadPointer and atomic.StorePointer. As a simple example:
// instantiate generic struct with time.Time
var at = atomic.Pointer[time.Time]{}
func main() {
t := time.Now()
at.Store(&t)
t = *at.Load()
fmt.Println(t)
}
Playground: https://go.dev/play/p/KwTMgvJIenx?v=gotip
Note that in Go 1.19 you can also use atomic.Int64. The advantage of these atomic types instead of top-level functions is that it's more fool-proof; it's impossible to access the value non-atomically as it's hidden behind the struct.
I have code similar to the following
package main
import (
"fmt"
"time"
"encoding/json"
)
type Message struct {
Time time.Time `json:"timestamp,omitempty"`
}
func main() {
t, _ := time.Parse("2006-01-02T15:04:05.999Z07:00", "2017-05-01T15:04:05.630Z")
msg := Message{
Time: t,
}
bs, _ := json.Marshal(msg)
fmt.Println(string(bs[:]))
}
This prints
{"timestamp":"2017-05-01T15:04:05.63Z"}
How can I make json marshalling keep the trailing 0? I.e., to print this?
{"timestamp":"2017-05-01T15:04:05.630Z"}
Edit:
Here's the playground https://play.golang.org/p/9p3kWeiwu2
time.Time always marshals to RFC 3339, only including sub-second precision if present: https://golang.org/pkg/time/#Time.MarshalJSON
You can write your own custom version using a named time.Time type, or you can define a custom marshal function for your structure, or you can make your structure hold a string in that place instead. In any case, if you want to use the same format, but including trailing zeros, you need to use a modified version of the RFC3339Nano constant.
Its value is: "2006-01-02T15:04:05.999999999Z07:00".
The 9's at the end mean "include until the rightmost non-zero value, omit after that". If you change those 9's to 0's, it will always include them. For example, if you always want millisecond precision (and nothing after that, regardless of whether it's non-zero or not), you would use:
"2006-01-02T15:04:05.000Z07:00"
If you feed that to Format() on your time.Time value, you'll get out the string you want, and can thus include it in the JSON.
Functioning example: https://play.golang.org/p/oqwnma6odw