How can I output CSV with martini? - go

I would like to print CSV-data to the output with martini. Currently, I have always used r.JSON(200, somestruct) where r is a render.Render from github.com/martini-contrib.
Now I have an slice of structs and I would like to print them as CSV (stringify each field of a single struct and print one struct at one line).
Currently, I do it like this:
r.Data(200, []byte("id,Latitude,Longitude\n"))
for _, packet := range tour.Packets {
r.Data(200, []byte(strconv.FormatInt(packet.Id, 10)+","+strconv.FormatFloat(packet.Latitude, 'f', 6, 64)+","+strconv.FormatFloat(packet.Longitude, 'f', 6, 64)+"\n"))
}
But I don't like the way I do it for the following reasons:
It is downloaded directly and not printed to the screen.
I get http: multiple response.WriteHeader calls
I would prefer not to make this manually (the struct has much more fields, but all fields are either ìnt64, float64 or time.Time.
How can I implement the CSV export option in a simpler way?

Use the standard library. There is no general solution without reflection, but you can simplify it.
func handler(rw http.ResponseWriter) {
rw.Header().Add("Content-Type", "text/csv")
wr := csv.NewWriter(rw)
err := wr.Write([]string{"id", "Latitude", "Longitude"})
if err != nil {
...
}
for _, packet := range tour.Packets {
err := wr.Write([]string{
strconv.FormatInt(packet.Id, 10),
strconv.FormatFloat(packet.Latitude, 'f', 6, 64),
strconv.FormatFloat(packet.Longitude, 'f', 6, 64),
})
if err != nil {
...
}
}
}
If you need a general solution for any struct, it will require reflect.
See here.
// structToStringSlice takes a struct value and
// creates a string slice of all the values in that struct
func structToStringSlice(i interface{}) []string {
v := reflect.ValueOf(i)
n := v.NumField()
out := make([]string, n)
for i := 0; i < n; i++ {
field := v.Field(i)
switch field.Kind() {
case reflect.String:
out[i] = field.String()
case reflect.Int:
out[i] = strconv.FormatInt(field.Int(), 10)
// add cases here to support more field types.
}
}
return out
}
// writeToCSV prints a slice of structs as csv to a writer
func writeToCSV(w io.Writer, i interface{}) {
wr := csv.NewWriter(w)
v := reflect.ValueOf(i)
// Get slice's element type (some unknown struct type)
typ := v.Type().Elem()
numFields := typ.NumField()
fieldSet := make([]string, numFields)
for i := 0; i < numFields; i++ {
fieldSet[i] = typ.Field(i).Name
}
// Write header row
wr.Write(fieldSet)
// Write data rows
sliceLen := v.Len()
for i := 0; i < sliceLen; i++ {
wr.Write(structToStringSlice(v.Index(i).Interface()))
}
wr.Flush()
}
so then your example is just:
func handler(rw http.ResponseWriter) {
....
writeToCSV(rw, tour.Packets)
}
The function I've written will only work for int or string fields. You can easily extend this to more types by adding cases to the switch in structToStringSlice. See here for reflect docs on the other Kinds.

Related

How to convert interface{} that's really a slice of types whose kind is reflect.Int32 into slice of int32?

I have the following:
type Int32A int32
type Int32B int32
and would like to implement a function that can accept any slice of types whose kind is reflect.Int32 and convert it into []int32. For example:
func ConvertTypeSliceToInt32Slice(es «es-type») []int32 {
result := make([]int32, len(es))
for i := 0; i < len(result); i++ {
result[i] = es[i].(int32)
}
return result
}
func caller() {
Int32as := Int32A{1, 2}
Int32bs := Int32B{3, 5}
int32as := ConvertTypeSliceToInt32Slice(Int32as)
int32bs := ConvertTypeSliceToInt32Slice(Int32bs)
}
How can this be done far any arbitrary type definition whose kind is reflect.Int32? (Context: this function will be used to convert slices of proto enums; ie the full set of types is unknown and unbounded so performing a switch on each type isn't feasible).
Also, I'm using 1.17 so I can't use parameterized types (aka templates).
One attempt that doesn't work (it panics at is.([]interface{})):
func ConvertTypeSliceToInt32Slice(is interface{}) []int32 {
es := is.([]interface{})
result := make([]int32, len(es))
for i := 0; i < len(result); i++ {
result[i] = es[i].(int32)
}
return result
}
int32 in this case is an "underlying type", and ~ in a type parameter declaration is how you specify a constraint to an underlying type.
For example: https://go.dev/play/p/8-WAu9KlXl5
func ConvertTypeSliceToInt32Slice[T ~int32](es []T) []int32 {
result := make([]int32, len(es))
for i := 0; i < len(result); i++ {
result[i] = int32(es[i])
}
return result
}
If you need to use reflection, you can convert the type of each element before appending to the final slice:
func ConvertTypeSliceToInt32Slice(es interface{}) []int32 {
v := reflect.ValueOf(es)
int32ty := reflect.TypeOf(int32(0))
result := make([]int32, v.Len())
for i := 0; i < v.Len(); i++ {
result[i] = v.Index(i).Convert(int32ty).Interface().(int32)
}
return result
}
And if you need to ensure other convertible numeric types are not converted, you can compare the element type and panic or error as necessary:
if v.Type().Elem().Kind() != reflect.Int32 {
...

How to create a Golang struct instance from an array?

Let's say I have a bunch of array of strings, for example:
data := [4]string{"a", "3.0", "2.5", "10.7"}
And a struct definition:
type Record struct {
name string
x float64
y float64
mag float64
}
I'd like to create an instance of this struct from each array.
I need to match the first item of the array to the first field of the struct and so on. Is it possible to do this?
Each array corresponds to one line of a file, so I can actually decide how to read these values in case a different approach is better.
An easy way is to use reflection to iterate over the fields of the struct, obtain their address (pointer), and use fmt.Sscan() to scan the string value into the field. fmt.Sscan() will handle the different types of fields for you. This is in no way an efficient solution, it is just a short, easy and flexible solution. If you need an efficient solution, you have to handle all fields explicitly, manually.
This only works if the fields of the struct are exported, e.g.:
type Record struct {
Name string
X float64
Y float64
Mag float64
Age int
}
The function that loads a string slice into a struct value:
func assign(recordPtr interface{}, data []string) error {
v := reflect.ValueOf(recordPtr).Elem()
max := v.NumField()
if max > len(data) {
max = len(data)
}
for i := 0; i < max; i++ {
if _, err := fmt.Sscan(data[i], v.Field(i).Addr().Interface()); err != nil {
return err
}
}
return nil
}
Note that this implementation tries to fill as many fields as possible (e.g. it does not return an error if the struct has more or less fields than input data provided). Also note that this assign() function can fill any other structs, not just Record, that's why it's flexible.
Example testing it:
data := []string{"a", "3.0", "2.5", "10.7", "23"}
var r Record
if err := assign(&r, data); err != nil {
fmt.Println("error:", err)
}
fmt.Printf("%+v", r)
Output (try it on the Go Playground):
{Name:a X:3 Y:2.5 Mag:10.7 Age:23}
There is no simple way to do this. You have to assign struct members one by one.
for _, x := range data {
x, err := strconv.ParseFloat(x[1])
y, err := strconv.ParseFloat(x[2])
max, err := strconv.ParseFloat(x[3])
strData = append(strData, Record{name: x[0], x: x, y: y, mag: mag})
}
You also have to deal with parse errors.
Of course it it possible to do that in Go.
The following code example will assign to s the record filled with the fields of data.
package main
import (
"fmt"
"strconv"
)
type Record struct {
name string
x float64
y float64
mag float64
}
func main() {
data := [4]string{"a", "3.0", "2.5", "10.7"}
x, err := strconv.ParseFloat(data[1], 64)
if err != nil {
panic(err)
}
y, err := strconv.ParseFloat(data[2], 64)
if err != nil {
panic(err)
}
mag, err := strconv.ParseFloat(data[3], 64)
if err != nil {
panic(err)
}
s := Record{ name: data[0], x: x, y: y, mag: mag}
fmt.Println(s)
}

golang - converting [ ]Interface to [ ]strings or joined string

How can I convert []interface to []strings or just as a joined single string with all elements in []interface ? Below is the screenshot showing exact value of "scope" which of type []interface with length of 2. In below code, where case is "slice" Currently i am doing this using reflect but this is not looking good. Wondering there should be some good way to join a slice\array of interface elements.
I also did try using json unmarshall like this "json.Unmarshal([]byte(jwtResp),&mymap)" but having trouble with type issues in converting jwt.MapClaims to byte[]
parsedkey, _ := jwt.ParseRSAPublicKeyFromPEM([]byte(key))
parsedToken, jwtErr := jwt.Parse(bearerToken, func(token *jwt.Token) (interface{}, error) {
return parsedkey, nil
})
jwtValues = make(map[string]string)
// we dont know which data types we are dealing with here, so using swtich case based on value data type
jwtResp := parsedToken.Claims.(jwt.MapClaims)
for k, v := range jwtResp {
switch reflect.TypeOf(v).Kind() {
case reflect.Slice:
fmt.Println("tp is : ", reflect.TypeOf(v)) // this is []interface{}
fmt.Println("type is : ", reflect.TypeOf(v).Kind()) // this is slice
s := reflect.ValueOf(v)
for i := 0; i < s.Len(); i++ {
jwtValues[k] += s.Index(i).Interface().(string)
jwtValues[k] += "|"
}
case reflect.String:
jwtValues[k] = v.(string)
default:
fmt.Println("unknown datatype")
}
}
Thanks for the suggestions . Below is the closest solution i found replacing 'switch' with 'if' condition along with type assertions. 'reflect' is costly. Using below code, i am finding if values of jwtResp is a slice, or string or something else. If slice, traverse through the values([]interface with string type elements) and join those as one concatenated string.
for k, v := range jwtResp {
if s, ok := v.(string); ok {
JwtValues[k] = s
} else if s, ok := v.([]interface{}); ok {
sslice := make([]string, len(s))
for i, v := range s {
sslice[i] = v.(string)
}
JwtValues[k] = strings.Join(sslice, "|")
} else {
logger.Log(util.LogDebug, "unknown data type")
}
}
Not sure to have fully understood your question, but it seems you're on the right track.
parts := make([]string, s.Len())
for i := 0; i < s.Len(); i++ {
parts = append(parts, s.Index(i).String())
}
jwtValues[k] = strings.Join(parts, "|")

Golang read data from file and indicate a missing value

I am parsing a csv file which contains integer values, some of them might be missing:
1,2,3
1,2,
1,2,3
In my code I'm populating a struct with the data:
type Line struct {
One *int
Two *int
Three *int
}
My current guess to handle missing values would be to use a pointer to an int in order to show whether the value is missing:
// if nil, then no value in file
l := &Line{}
// read and parse...
l.Three = nil
However, using this approach makes assingment to *int cumbersome:
l := &Line{}
// NOT POSSIBLE, NOT COMPILING (cannot use 5 (type int) as type *int in assignment)
l.One = 5
// FEELS WRONG, BUT WORKS
tmpInt := 5
l.One = &tmpInt
How to handle missing integer values?
You could use a function to build your Line{} from a []string, a simple example:
func NewLine(s []string) (l *Line) {
fmt.Println(len(s))
if len(s) < 2 {
return
}
l = &Line{}
if i, err := strconv.Atoi(s[0]); err == nil {
l.One = &i
}
if i, err := strconv.Atoi(s[1]); err == nil {
l.Two = &i
}
if len(s) == 3 {
if i, err := strconv.Atoi(s[2]); err == nil {
l.Three = &i
}
}
return
}

Reading bytes into structs using reflection

I'm trying to write functions that will allow me to marshal/unmarshal simple structs into byte arrays. I've succeeded in writing Marshal, with help from the kind folks at #go-nuts, but I'm running into trouble writing Unmarshal.
// Unmarshal unpacks the binary data and stores it in the packet using
// reflection.
func Unmarshal(b []byte, t reflect.Type) (pkt interface{}, err error) {
buf := bytes.NewBuffer(b)
p := reflect.New(t)
v := reflect.ValueOf(p)
for i := 0; i < t.NumField(); i++ {
f := v.Field(i)
switch f.Kind() {
case reflect.String:
// length of string
var l int16
var e error
e = binary.Read(buf, binary.BigEndian, &l)
if e != nil {
err = e
return
}
// read length-of-string bytes from the buffer
raw := make([]byte, l)
_, e = buf.Read(raw)
if e != nil {
err = e
return
}
// convert the bytes to a string
f.SetString(bytes.NewBuffer(raw).String())
default:
e := binary.Read(buf, binary.BigEndian, f.Addr())
if e != nil {
err = e
return
}
}
}
pkt = p
return
}
The problem with the code above is that the call to f.Addr() near the end is apparently trying to get the address of an unaddressable value.
If there is an alternative solution, I would appreciate that as well. Either way, any help would be much appreciated.
Thanks!
I think you should use
v := p.Elem() // Get the value that 'p' points to
instead of
v := reflect.ValueOf(p)
Working example with lots of assumptions and a trivial data format:
package main
import (
"fmt"
"reflect"
"strconv"
)
// example marshalled format. lets say that marshalled data will have
// four bytes of a formatted floating point number followed by two more
// printable bytes.
type m42 []byte
// example struct we'd like to unmarshal into.
type packet struct {
S string // exported fields required for reflection
F float64
}
// example usage
func main() {
var p packet
if err := Unmarshal(m42("3.14Pi"), &p); err == nil {
fmt.Println(p)
} else {
fmt.Println(err)
}
}
func Unmarshal(data m42, structPtr interface{}) error {
vp := reflect.ValueOf(structPtr)
ve := vp.Elem() // settable struct Value
vt := ve.Type() // type info for struct
nStructFields := ve.NumField()
for i := 0; i < nStructFields; i++ {
fv := ve.Field(i) // settable field Value
sf := vt.Field(i) // StructField type information
// struct field name indicates which m42 field to unmarshal.
switch sf.Name {
case "S":
fv.SetString(string(data[4:6]))
case "F":
s := string(data[0:4])
if n, err := strconv.ParseFloat(s, 64); err == nil {
fv.SetFloat(n)
} else {
return err
}
}
}
return nil
}
Appropriate alternative solutions would depend heavily on the real data you need to support.
I'm going to bet that the reason f.Addr() has the problem because it actually isn't addressable.
the reflect package Type object has a method that will tell you if the type is addressable called CanAddr(). Assuming the field is addressable if it's not a string is not always true. If the struct is not passed in as a pointer to a struct then it's fields won't be addressable. For more details about what is and isn't addressable see: http://weekly.golang.org/pkg/reflect/#Value.CanAddr which outlines the correct rules.
Essentially for your code to work I think you need to ensure you always call it with a pointer to a struct.

Resources