Golang - String with many delimiters to slice or struct - go

I have the following string returned as response from a TCP connection :
220 Connected.\ncommand:connect\nemail:ERROR_MAIL_MISSING\nstatus:CMD_ERROR\nend
I want actually to transform this response to the following golang struct :
type Message struct {
Key string
Value string
}
type Response struct {
Connect string
Messages []Message
}
If 220 Connected is always present means :
Response.Connect => TRUE
All response between 220 Connected. and end can be accessed using :
Response[0].Key => "command"
Response[0].Value => "connect"
This is what I actually achieved :
result, err := ioutil.ReadAll(conn)
data := strings.Split(string(result), "\n")
for i := range data {
Reply := new(Response)
if data[i] == "220 Connected." {
Reply.Connect = "TRUE"
Response = append(Response, Reply)
}
}
Any hint how this can be achieved ? I'm a newbie with Golang

You need to do a bit more work to extract the data you want. Here's a sample program that correctly populates the Messages array. I'll update with some notes on what's I did to move from your attempt to what you're looking for.
package main
import "fmt"
import "strings"
type Message struct {
Key string
Value string
}
type Response struct {
Connect string
Messages []Message
}
func main() {
result := `220 Connected.\ncommand:connect\nemail:ERROR_MAIL_MISSING\nstatus:CMD_ERROR\nend`
data := strings.Split(string(result), "\\")
r := &Response{}
for i := range data {
if data[i] == "220 Connected." {
r.Connect = "TRUE"
} else {
tokens := strings.Split(data[i], ":")
if len(tokens) == 2 {
m := Message{tokens[0], tokens[1]}
r.Messages = append(r.Messages, m)
}
}
}
for i := range r.Messages {
fmt.Println(r.Messages[i])
}
}
https://play.golang.org/p/Hs8aqYPyuM
Alright, so first lets list some problems. In your attempt the for loop looks like this;
for i := range data {
Reply := new(Response)
if data[i] == "220 Connected." {
Reply.Connect = "TRUE"
Response = append(Response, Reply)
}
}
This isn't consistent with your types/data layout. On every iteration you create a new Response instance, you actually only need one. What you want on each iteration is a new Message instance. This causes a few problems, firstly the data you're looking at isn't a response, it's a message, secondly you keep overwriting the previous one (that object is only scoped for the loop). To correct this we instantiate the Response object before the loop. Secondly, you need a second split in order to get your data out. So inside the loop we split on a colon to separate the key and value of each message. Then I have a quick check on the length before adding it (I got a panic on first run, the first if probably failed, meaning you need to edit that a bit. If you don't get 220 for connected you want to set that value to false, you should do the message split an a final else but simply not being 220 Connected. isn't sufficient to assume the current item is a Message which is why I added bounds check). Note that inside this loop I instantiate a new message for each key value pair. We use append there to append to the Messages array, not appending Response's onto eachother (that will never work since it's a struct, not a slice or map). This also makes it so the Message persists rather than going out of scope each time we hit the bottom of the loop (m will go out of scope but the instance in r.Messages will still be around).

Related

Get data from Twitter Library search into a struct in Go

How do I append output from a twitter search to the field Data in the SearchTwitterOutput{} struct.
Thanks!
I am using a twitter library to search twitter base on a query input. The search returns an array of strings(I believe), I am able to fmt.println the data but I need the data as a struct.
type SearchTwitterOutput struct {
Data string
}
func (SearchTwitter) execute(input SearchTwitterInput) (*SearchTwitterOutput, error) {
credentials := Credentials{
AccessToken: input.AccessToken,
AccessTokenSecret: input.AccessTokenSecret,
ConsumerKey: input.ConsumerKey,
ConsumerSecret: input.ConsumerSecret,
}
client, err := GetUserClient(&credentials)
if err != nil {
return nil, err
}
// search through the tweet and returns a
search, _ , err := client.Search.Tweets(&twitter.SearchTweetParams{
Query: input.Text,
})
if err != nil {
println("PANIC")
panic(err.Error())
return &SearchTwitterOutput{}, err
}
for k, v := range search.Statuses {
fmt.Printf("Tweet %d - %s\n", k, v.Text)
}
return &SearchTwitterOutput{
Data: "test", //data is a string for now it can be anything
}, nil
}
//Data field is a string type for now it can be anything
//I use "test" as a placeholder, bc IDK...
Result from fmt.Printf("Tweet %d - %s\n", k, v.Text):
Tweet 0 - You know I had to do it to them! #JennaJulien #Jenna_Marbles #juliensolomita #notjulen Got my first hydroflask ever…
Tweet 1 - RT #brenna_hinshaw: I was in J2 today and watched someone fill their hydroflask with vanilla soft serve... what starts here changes the wor…
Tweet 2 - I miss my hydroflask :(
This is my second week working with go and new to development. Any help would be great.
It doesn't look like the client is just returning you a slice of strings. The range syntax you're using (for k, v := range search.Statuses) returns two values for each iteration, the index in the slice (in this case k), and the object from the slice (in this case v). I don't know the type of search.Statuses - but I know that strings don't have a .Text field or method, which is how you're printing v currently.
To your question:
Is there any particular reason to return just a single struct with a Data field rather than directly returning the output of the twitter client?
Your function signature could look like this instead:
func (SearchTwitter) execute(input SearchTwitterInput) ([]<client response struct>, error)
And then you could operate on the text in those objects in wherever this function was called.
If you're dead-set on placing the data in your own struct, you could return a slice of them ([]*SearchTwitterOutput), in which case you could build a single SearchTwitterOutput in the for loop you're currently printing the tweets in and append it to the output list. That might look like this:
var output []*SearchTwitterOutput
for k, v := range search.Statuses {
fmt.Printf("Tweet %d - %s\n", k, v.Text)
output = append(output, &SearchTwitterOutput{
Data: v.Text,
})
}
return output, nil
But if your goal really is to return all of the results concatenated together and placed inside a single struct, I would suggest building a slice of strings (containing the text you want), and then joining them with the delimiter of your choosing. Then you could place the single output string in your return object, which might look something like this:
var outputStrings []string
for k, v := range search.Statuses {
fmt.Printf("Tweet %d - %s\n", k, v.Text)
outputStrings = append(outputStrings, v.Text)
}
output = strings.Join(outputStrings, ",")
return &SearchTwitterOutput{
Data: output,
}, nil
Though I would caution, it might be tricky to find a delimiter that will never show up in a tweet..

Avoid typecasting a slice value in loop

I'm reading an API (JSON) response from a server and I'm supposed to get(if the status is 200 ok) the following responses back.
// If I sent a wrong data ..
{
error: "some value",
message: "... description of the error"
}
or
// if all is good
{
events: [{key1: 1}, {key2: "two"} ... ]
}
because I'm not sure about the type of response.
I'm decoding the response as a map[string]interface{}.
resp := make(map[string]interface{}, 0)
json.NewDecoder(response.Body).Decode(&resp)
Later in the code flows I reach a stage where I know that response is an all good one. I need to typecast the interface{} back to []map[string]interface{}.
but I think this does not work ..
// This does not work I guess.
// events := resp["events"].([]map[string]interface{})
The only sane way I manage to do this is using.
interFace := resp["events"].([]interface{})
for mapInterface := range interFace {
row := MapInterface.(map[string]interface{})
// now use the row hence forth
}
Can this be done in any other way or any Other approach1 that would avoid the casting in inside the loop.
1: Maybe creating a Response struct over here can work. But I would really like to hear any other approaches that I can try before forward with a struct approach.
You could define a struct that has both the error values and non error values in your struct.
type Response struct {
ErrorMessage string `json:"message"`
Error string `json:"error"`
Events []map[string]interface{} `json:"events"`
}

GoRoutines and passing struct to original context

I have a configuration that defines a number of instances (SomeConfigItems) which have a thing() created for each of them.
That thing is a struct returned by an included package, which contains, among other things, a Price (float64) and a nested struct. The nested struct maintains a map of trades.
The problem is that I am able to loop through the thing.Streams.Trades and see all trades happening in real time from my main()'s for{} loop. I am not able to see an updated thing.Price even though it is set in the Handler on occasion.
I am having a hard time understanding how the nested structs can contain data but not Price. I feel as though I am missing something with scoping, goroutines, or possibly pointers for instantiation of new objects.
Any help would be appreciated, I will continue reading in the meantime. I've reduced the code to what seems relevant.
main.go:
package main
import thing
var Things []thing.Handler
for _, name := range SomeConfigItems {
handler := thing.New(name)
Things = append(Things, handler)
}
for {
for _, t := range Things {
log.Info("Price: ", t.Price) // This is set to 0 every iteration, but I can actively data in thing.Streams.Trades
}
}
thing.go:
package thing
import streams
type Handler struct {
Name string
Price float64
Streams streams.Streams
}
func New(name string) (h Handler, err error) {
stream, err := streams.New(strings.ToLower(name))
h = Handler{
Name: name,
Price: "0.0"
Streams: stream,
}
go h.handler()
return h, err
}
func (bot *Handler) handler() {
var currentPrice float64
for {
currentPrice = external.GetPrice(bot.Name).Price // Validated that this returns a float64
bot.Price = currentPrice // Verified that this is updated immediately after in this context.
// Unable to see Price updated from outer context.
}
}
streams.go:
package streams
type Streams struct {
Trades
}
type State struct {
Price string `json:"p"`
Quantity string `json:"q"`
}
type Trades struct {
Trades map[float64]float64
TradeMutex sync.Mutex
Updates chan State
}
func New(name string) (s Streams, err error) {
p := newTradeStream(name)
s = Streams{
Trades: p,
}
return s, err
}
func newTradeStream(name string) (ts Trades) {
ts = Trades{}
ts.Trades = make(map[float64]float64, MaxDepth)
ts.Updates = make(chan State, 500)
// ... Other watchdog code
return ts
}
Note:
I am added some debug logging in multiple locations. From within the Bot Handler, the price was printed (successfully), then updated, and then printed (successfully) again -- Showing no gap in the setting of Price from within the handler() function.
When adding the same type of debugging to the main() for{} loop, I tried setting an incrementing counter and assigning the value of thing.Price -- Printing thing.Price on each loop results in 0, even if I set the price (and validate it gets set) in the same loop, it is back to 0 on the next iteration.
This behavior is why I think that I am missing something very fundamental.
In Go, arguments are passed to functions by value -- meaning what the function gets is a copy of the value, not a reference to the variable. The same is true of the function receiver, and also the return list.
It's not the most elegant description, but for the sake of explanation, let's call this the "function wall." If the value being passed one way or the other is a pointer, the function still gets a copy, but it's a copy of a memory address, and so the pointer can be used to change the value of the variable on the other side of the wall. If it is a reference type, which uses a pointer in the implementation of the type, then again a change to the thing being pointed to can cross that wall. But otherwise the change does not cross the wall, which is one reason so many Go functions are written to return values instead of just modifying values.
Here's a runnable example:
package main
import (
"fmt"
)
type Car struct {
Color string
}
func (c Car) Change() { // c was passed by value, it's a copy
c.Color = "Red"
}
func main() {
ride := Car{"Blue"}
ride.Change()
fmt.Println(ride.Color)
}
Prints "Blue"
But two small changes:
func (c *Car) Change() { // here
c.Color = "Red"
}
func main() {
ride := &Car{"Blue"} // and here
ride.Change()
fmt.Println(ride.Color)
}
And now it prints "Red". Struct is not a reference type. So if you want modifications to a struct to cross the wall without using the return list to do it, use a pointer. Of course this only applies to values being passed via argument, return list, or receiver; and not to variables that are in scope on both sides of the wall; or to modifying the underlying value behind a reference type.
See also "Pointers Versus Values" in Effective Go, and "Go Data Structures" by Russ Cox.

Writing a struct's fields and values of different types to a file in Go

I'm writing a simple program that takes in input from a form, populates an instance of a struct with the received data and the writes this received data to a file.
I'm a bit stuck at the moment with figuring out the best way to iterate over the populated struct and write its contents to the file.
The struct in question contains 3 different types of fields (ints, strings, []strings).
I can iterate over them but I am unable to get their actual type.
Inspecting my posted code below with print statements reveals that each of their types is coming back as structs rather than the aforementioned string, int etc.
The desired output format is be plain text.
For example:
field_1="value_1"
field_2=10
field_3=["a", "b", "c"]
Anyone have any ideas? Perhaps I'm going about this the wrong way entirely?
func (c *Config) writeConfigToFile(file *os.File) {
listVal := reflect.ValueOf(c)
element := listVal.Elem()
for i := 0; i < element.NumField(); i++ {
field := element.Field(i)
myType := reflect.TypeOf(field)
if myType.Kind() == reflect.Int {
file.Write(field.Bytes())
} else {
file.WriteString(field.String())
}
}
}
Instead of using the Bytes method on reflect.Value which does not work as you initially intended, you can use either the strconv package or the fmt to format you fields.
Here's an example using fmt:
var s string
switch fi.Kind() {
case reflect.String:
s = fmt.Sprintf("%q", fi.String())
case reflect.Int:
s = fmt.Sprintf("%d", fi.Int())
case reflect.Slice:
if fi.Type().Elem().Kind() != reflect.String {
continue
}
s = "["
for j := 0; j < fi.Len(); j++ {
s = fmt.Sprintf("%s%q, ", s, fi.Index(i).String())
}
s = strings.TrimRight(s, ", ") + "]"
default:
continue
}
sf := rv.Type().Field(i)
if _, err := fmt.Fprintf(file, "%s=%s\n", sf.Name, s); err!= nil {
panic(err)
}
Playground: https://play.golang.org/p/KQF3CicVzA
Why not use the built-in gob package to store your struct values?
I use it to store different structures, one per line, in files. During decoding, you can test the type conversion or provide a hint in a wrapper - whichever is faster for your given use case.
You'd treat each line as a buffer when Encoding and Decoding when reading back the line. You can even gzip/zlib/compress, encrypt/decrypt, etc the stream in real-time.
No point in re-inventing the wheel when you have a polished and armorall'd wheel already at your disposal.

Writing generic data access functions in Go

I'm writing code that allows data access from a database. However, I find myself repeating the same code for similar types and fields. How can I write generic functions for the same?
e.g. what I want to achieve ...
type Person{FirstName string}
type Company{Industry string}
getItems(typ string, field string, val string) ([]interface{}) {
...
}
var persons []Person
persons = getItems("Person", "FirstName", "John")
var companies []Company
cs = getItems("Company", "Industry", "Software")
So you're definitely on the right track with the idea of returning a slice of nil interface types. However, you're going to run into problems when you try accessing specific members or calling specific methods, because you're not going to know what type you're looking for. This is where type assertions are going to come in very handy. To extend your code a bit:
getPerson(typ string, field string, val string) []Person {
slice := getItems(typ, field, val)
output := make([]Person, 0)
i := 0
for _, item := range slice {
// Type assertion!
thing, ok := item.(Person)
if ok {
output = append(output, thing)
i++
}
}
return output
}
So what that does is it performs a generic search, and then weeds out only those items which are of the correct type. Specifically, the type assertion:
thing, ok := item.(Person)
checks to see if the variable item is of type Person, and if it is, it returns the value and true, otherwise it returns nil and false (thus checking ok tells us if the assertion succeeded).
You can actually, if you want, take this a step further, and define the getItems() function in terms of another boolean function. Basically the idea would be to have getItems() run the function pass it on each element in the database and only add that element to the results if running the function on the element returns true:
getItem(critera func(interface{})bool) []interface{} {
output := make([]interface{}, 0)
foreach _, item := range database {
if criteria(item) {
output = append(output, item)
}
}
}
(honestly, if it were me, I'd do a hybrid of the two which accepts a criteria function but also accepts the field and value strings)
joshlf13 has a great answer. I'd expand a little on it though to maintain some additional type safety. instead of a critera function I would use a collector function.
// typed output array no interfaces
output := []string{}
// collector that populates our output array as needed
func collect(i interface{}) {
// The only non typesafe part of the program is limited to this function
if val, ok := i.(string); ok {
output = append(output, val)
}
}
// getItem uses the collector
func getItem(collect func(interface{})) {
foreach _, item := range database {
collect(item)
}
}
getItem(collect) // perform our get and populate the output array from above.
This has the benefit of not requiring you to loop through your interface{} slice after a call to getItems and do yet another cast.

Resources