Best method to stream results from web service - go

Question

I'm currently writing against an xml service in which batched results are returned. I currently have the following:
type QueryEnvelope struct {
XMLName xml.Name `xml:"http://schemas.xmlsoap.org/soap/envelope/ Envelope"`
Body *QueryBody `xml:"http://schemas.xmlsoap.org/soap/envelope/ Body"`
}
type QueryBody struct {
QueryResult *QueryResult `xml:"queryResponse>result"`
}
type QueryResult struct {
QueryLocator QueryLocator `xml:"queryLocator"`
Done bool `xml:"done"`
Size int `xml:"size"`
Records interface{} `xml:"records"`
}
type QueryLocator string
func (r *Resource) Query(sql string, r interface{}) error {
// builds xml request
// sends xml request through Resource which contains the net client
// decodes the result into r (type for records in QueryResult)
}
func (r *Resource) QueryMore(q QueryLocator, r interface{}) error {
// same as above except takes queryLocator and makes call to different endpoint in order to continue retrieving results, when QueryResult.Done == true, done fetching results
}
Obviously this needs refactored since the Client needs to see whether Done == true so they can continue to fetch. I was thinking of adding the following the following, and moving Query and QueryMore to be a method of Querier:
type Querier struct {
r *Resource
Done bool
QueryLocator QueryLocator
}
func New(r *Resource) *Querier {}
Client side would then behave as such:
err := q.Query("sql statement", r)
if err != nil {
// handle
}
// do stuff with r
for q.Done == false {
q.QueryMore(r)
// do stuff with r
}
I'm wondering what the idiomatic approach would be here in order to best "stream" the results.

Answer 1

One option is to use the pattern the stdlib sql package uses to iterate over rows. This way you can unify the initial query with subsequent calls.
There's a Next() method, which populates the internal structure with the query results, and returns true if there's a result pending. In your case, you would still need an initial constructor, like Query, but that would just setup the data-structure. Calling Next() does the real work, and calling Scan(r) (or whatever you'd like to call reading the result) just presents to result. Once iteration is done, you have an Err() method to check for any operational errors.
Altering the example from the sql package a bit:
// setup the query, but delay any action until we start iterating.
query, err := NewQuery(queryParameters)
// check err of course
defer query.Close()
for query.Next() {
err = query.Scan(&r)
...
}
err = query.Err() // get any error encountered during iteration
You can also check out other drivers like mgo for variations on this pattern.