My aim is to create a logging function that lists the name of a function and the list of passed parameters.
An example would be the following:
func MyFunc(a string, b int){
... some code ...
if err != nil{
errorDescription := myLoggingFunction(err)
fmt.Println(errorDescription)
}
}
func main(){
MyFunc("hello", 42)
}
// where MyLoggingFunction should return something like:
// "MyFunc: a = hello, b = 42, receivedError = "dummy error description"
So far it seems that in Go there is no way to get the name of the parameters of a function at runtime, as answered in this question, but I could give up this feature.
I've managed to get the function name and the memory address of the passed parameters by analysing the stack trace, but I'm hitting a wall when it comes to print somehow the parameters starting from their address (I understand that it might not be trivial depending on the type of the parameters, but even something very simple will do for now)
This is an implementation of the logging function I'm building (you can test it on this playground), is there away to print the parameter values?
func MyLoggingFunction(err error) string {
callersPCs := make([]uintptr, 10)
n := runtime.Callers(2, callersPCs) //skip first 2 entries, (Callers, GetStackTrace)
callersPCs = callersPCs[:n]
b := make([]byte, 1000)
runtime.Stack(b, false)
stackString := string(b)
frames := runtime.CallersFrames(callersPCs)
frame, _ := frames.Next()
trimmedString := strings.Split(strings.Split(stackString, "(")[2], ")")[0]
trimmedString = strings.Replace(trimmedString, " ", "", -1)
parametersPointers := strings.Split(trimmedString, ",")
return fmt.Sprintf("Name: %s \nParameters: %s \nReceived Error: %s", frame.Function, parametersPointers, err.Error())
}
If there are other ideas for building such logging function without analysing the stack trace, except the one that consists in passing a map[string]interface{} containing all the passed parameter names as keys and their values as values (that is my current implementation and is tedious since I'd like to log errors very often), I'd be glad to read them.
Related
The Problem:
Right now, I'm logging my SQL query and the args that related to that query, but what will happen if my args weight a lot? say 100MB?
The Solution:
I want to iterate over the args and once they exceeded the 0.5MB I want to take the args up till this point and only log them (of course I'll use the entire args set in the actual SQL query).
Where am stuck:
I find it hard to find the size on the disk of an interface{}.
How can I print it? (there is a nicer way to do it than %v?)
The concern is mainly focused on the first section, how can I find the size, I need to know the type, if its an array, stack, heap, etc..
If code helps, here is my code structure (everything sits in dal pkg in util file):
package dal
import (
"fmt"
)
const limitedLogArgsSizeB = 100000 // ~ 0.1MB
func parsedArgs(args ...interface{}) string {
currentSize := 0
var res string
for i := 0; i < len(args); i++ {
currentEleSize := getSizeOfElement(args[i])
if !(currentSize+currentEleSize =< limitedLogArgsSizeB) {
break
}
currentSize += currentEleSize
res = fmt.Sprintf("%s, %v", res, args[i])
}
return "[" + res + "]"
}
func getSizeOfElement(interface{}) (sizeInBytes int) {
}
So as you can see I expect to get back from parsedArgs() a string that looks like:
"[4378233, 33, true]"
for completeness, the query that goes with it:
INSERT INTO Person (id,age,is_healthy) VALUES ($0,$1,$2)
so to demonstrate the point of all of this:
lets say the first two args are equal exactly to the threshold of the size limit that I want to log, I will only get back from the parsedArgs() the first two args as a string like this:
"[4378233, 33]"
I can provide further details upon request, Thanks :)
Getting the memory size of arbitrary values (arbitrary data structures) is not impossible but "hard" in Go. For details, see How to get memory size of variable in Go?
The easiest solution could be to produce the data to be logged in memory, and you can simply truncate it before logging (e.g. if it's a string or a byte slice, simply slice it). This is however not the gentlest solution (slower and requires more memory).
Instead I would achieve what you want differently. I would try to assemble the data to be logged, but I would use a special io.Writer as the target (which may be targeted at your disk or at an in-memory buffer) which keeps track of the bytes written to it, and once a limit is reached, it could discard further data (or report an error, whatever suits you).
You can see a counting io.Writer implementation here: Size in bits of object encoded to JSON?
type CounterWr struct {
io.Writer
Count int
}
func (cw *CounterWr) Write(p []byte) (n int, err error) {
n, err = cw.Writer.Write(p)
cw.Count += n
return
}
We can easily change it to become a functional limited-writer:
type LimitWriter struct {
io.Writer
Remaining int
}
func (lw *LimitWriter) Write(p []byte) (n int, err error) {
if lw.Remaining == 0 {
return 0, io.EOF
}
if lw.Remaining < len(p) {
p = p[:lw.Remaining]
}
n, err = lw.Writer.Write(p)
lw.Remaining -= n
return
}
And you can use the fmt.FprintXXX() functions to write into a value of this LimitWriter.
An example writing to an in-memory buffer:
buf := &bytes.Buffer{}
lw := &LimitWriter{
Writer: buf,
Remaining: 20,
}
args := []interface{}{1, 2, "Looooooooooooong"}
fmt.Fprint(lw, args)
fmt.Printf("%d %q", buf.Len(), buf)
This will output (try it on the Go Playground):
20 "[1 2 Looooooooooooon"
As you can see, our LimitWriter only allowed to write 20 bytes (LimitWriter.Remaining), and the rest were discarded.
Note that in this example I assembled the data in an in-memory buffer, but in your logging system you can write directly to your logging stream, just wrap it in LimitWriter (so you can completely omit the in-memory buffer).
Optimization tip: if you have the arguments as a slice, you may optimize the truncated rendering by using a loop, and stop printing arguments once the limit is reached.
An example doing this:
buf := &bytes.Buffer{}
lw := &LimitWriter{
Writer: buf,
Remaining: 20,
}
args := []interface{}{1, 2, "Loooooooooooooooong", 3, 4, 5}
io.WriteString(lw, "[")
for i, v := range args {
if _, err := fmt.Fprint(lw, v, " "); err != nil {
fmt.Printf("Breaking at argument %d, err: %v\n", i, err)
break
}
}
io.WriteString(lw, "]")
fmt.Printf("%d %q", buf.Len(), buf)
Output (try it on the Go Playground):
Breaking at argument 3, err: EOF
20 "[1 2 Loooooooooooooo"
The good thing about this is that once we reach the limit, we don't have to produce the string representation of the remaining arguments that would be discarded anyway, saving some CPU (and memory) resources.
How do I append output from a twitter search to the field Data in the SearchTwitterOutput{} struct.
Thanks!
I am using a twitter library to search twitter base on a query input. The search returns an array of strings(I believe), I am able to fmt.println the data but I need the data as a struct.
type SearchTwitterOutput struct {
Data string
}
func (SearchTwitter) execute(input SearchTwitterInput) (*SearchTwitterOutput, error) {
credentials := Credentials{
AccessToken: input.AccessToken,
AccessTokenSecret: input.AccessTokenSecret,
ConsumerKey: input.ConsumerKey,
ConsumerSecret: input.ConsumerSecret,
}
client, err := GetUserClient(&credentials)
if err != nil {
return nil, err
}
// search through the tweet and returns a
search, _ , err := client.Search.Tweets(&twitter.SearchTweetParams{
Query: input.Text,
})
if err != nil {
println("PANIC")
panic(err.Error())
return &SearchTwitterOutput{}, err
}
for k, v := range search.Statuses {
fmt.Printf("Tweet %d - %s\n", k, v.Text)
}
return &SearchTwitterOutput{
Data: "test", //data is a string for now it can be anything
}, nil
}
//Data field is a string type for now it can be anything
//I use "test" as a placeholder, bc IDK...
Result from fmt.Printf("Tweet %d - %s\n", k, v.Text):
Tweet 0 - You know I had to do it to them! #JennaJulien #Jenna_Marbles #juliensolomita #notjulen Got my first hydroflask ever…
Tweet 1 - RT #brenna_hinshaw: I was in J2 today and watched someone fill their hydroflask with vanilla soft serve... what starts here changes the wor…
Tweet 2 - I miss my hydroflask :(
This is my second week working with go and new to development. Any help would be great.
It doesn't look like the client is just returning you a slice of strings. The range syntax you're using (for k, v := range search.Statuses) returns two values for each iteration, the index in the slice (in this case k), and the object from the slice (in this case v). I don't know the type of search.Statuses - but I know that strings don't have a .Text field or method, which is how you're printing v currently.
To your question:
Is there any particular reason to return just a single struct with a Data field rather than directly returning the output of the twitter client?
Your function signature could look like this instead:
func (SearchTwitter) execute(input SearchTwitterInput) ([]<client response struct>, error)
And then you could operate on the text in those objects in wherever this function was called.
If you're dead-set on placing the data in your own struct, you could return a slice of them ([]*SearchTwitterOutput), in which case you could build a single SearchTwitterOutput in the for loop you're currently printing the tweets in and append it to the output list. That might look like this:
var output []*SearchTwitterOutput
for k, v := range search.Statuses {
fmt.Printf("Tweet %d - %s\n", k, v.Text)
output = append(output, &SearchTwitterOutput{
Data: v.Text,
})
}
return output, nil
But if your goal really is to return all of the results concatenated together and placed inside a single struct, I would suggest building a slice of strings (containing the text you want), and then joining them with the delimiter of your choosing. Then you could place the single output string in your return object, which might look something like this:
var outputStrings []string
for k, v := range search.Statuses {
fmt.Printf("Tweet %d - %s\n", k, v.Text)
outputStrings = append(outputStrings, v.Text)
}
output = strings.Join(outputStrings, ",")
return &SearchTwitterOutput{
Data: output,
}, nil
Though I would caution, it might be tricky to find a delimiter that will never show up in a tweet..
I need to know the name of the go-package and function (including the receiver name) of the calling function.
This is my current code:
func retrieveCallInfo() {
pc, _, _, _ := runtime.Caller(1)
funcName := runtime.FuncForPC(pc).Name()
lastDot := strings.LastIndexByte(funcName, '.')
fmt.Printf(" Package: %s\n", funcName[:lastDot])
fmt.Printf(" Func: %s\n", funcName[lastDot+1:])
}
However, the code doesn't behave exactly as it should.
// When called from a conventional (free) function:
runtime.FuncForPC(pc).Name() // returns <package-path>.<funcName>
// When called from a method receiver function:
runtime.FuncForPC(pc).Name() // returns <package-path>.<receiverName>.<funcName>
When called from a receiver function, the receiver name is part of the package name, rather than the function name - which is not what I want.
Here's a demonstration: https://play.golang.org/p/-99sZXr4ptD
In the second example, I want the package name to be main and the function name to be empty.f.
Since dots are also valid parts of a package name, I can't simply split at another dot - maybe it's actually not the receiver, but part of the package name.
Hence, the information returned by runtime.FuncForPC() is ambiguous and not enough.
How can I get the correct results?
The results are correct. You'll need to do some parsing to format the results the way you want them; for example, try splitting on dots after the last slash in the string:
pc, _, _, _ := runtime.Caller(1)
funcName := runtime.FuncForPC(pc).Name()
lastSlash := strings.LastIndexByte(funcName, '/')
if lastSlash < 0 {
lastSlash = 0
}
lastDot := strings.LastIndexByte(funcName[lastSlash:], '.') + lastSlash
fmt.Printf("Package: %s\n", funcName[:lastDot])
fmt.Printf("Func: %s\n", funcName[lastDot+1:])
Playground: https://play.golang.org/p/-Nbos0a1Ifp
I'm learning Golang and I come from PHP background. I have a bit of trouble understanding some of the core functionalities at times.
Specifically, right now I'm building a Hearths game and I've created a CardStack type, that has some convenient methods one might use in a card stack (read: player's hand, discard pile...) such as DrawCards(...), AppendCards(...)...
The problem I have is that the function func (c* CardStack) DrawCards(cards []deck.Card) ([]deck.Card, error) {...} changes the argument cards []deck.Card and I cannot figure out why or how to avoid this.
This is my CardStack:
type CardStack struct {
cards []deck.Card
}
This is my DrawCards method:
func (c *CardStack) DrawCards(cards []deck.Card) ([]deck.Card, error) {
return c.getCardsSlice(cards, true)
}
// Returns cards that are missing
func (c *CardStack) getCardsSlice(cards []deck.Card, rm bool) ([]deck.Card, error) {
var err error
var returnc = []deck.Card{}
for _, card := range cards {
fmt.Println("BEFORE c.findCard(cards): ")
deck.PrintCards(cards) // In my example this will print out {Kc, 8d}, which is what I expect it to be
_, err = c.findCard(card, rm) // AFTER THIS LINE THE cards VAR IS CHANGED
fmt.Println("AFTER c.findCard(cards): ")
deck.PrintCards(cards) // In my example this will print out {8d, 8d}, which is not at all what I expected
if err != nil {
return returnc, err
}
}
return returnc, nil
}
// Expects string like "Ts" or "2h" (1. face 2. suit)
func (c *CardStack) findCard(cc deck.Card, rm bool) (deck.Card, error) {
for i, card := range c.GetCards() {
if cc == card {
return c.cardByIndex(i, rm)
}
}
return deck.Card{}, fmt.Errorf("Card not found")
}
func (c *CardStack) cardByIndex(n int, rm bool) (deck.Card, error) {
if n > len(c.GetCards()) {
return deck.Card{}, fmt.Errorf("Index out of bounds")
}
card := c.GetCards()[n]
if rm {
c.SetCards(append(c.GetCards()[:n], c.GetCards()[n+1:]...))
}
return card, nil
}
To explain a bit more - specifically the findCard(...) method that gets called in getCardsSlice messes with the original value (I've added comments to indicate where it happens).
If it's of any help, this is part of my main() method that I use for debugging:
// ...
ss, _ := cards.SubStack(1, 3) // ss now holds {Kc, 8d}
ss.Print() // Prints {Kc, 8d}
cards.Print() // Prints {5c, Kc, 8d} (assigned somewhere up in the code)
cards.DrawCards(ss) // Draws {Kc, 8d} from {5c, Kc, 8d}
cards.Print() // Prints {5c} - as expected
ss.Print() // Prints {8d, 8d} - ???
What am I doing wrong and how should I go about doing this.
Any kind of help is appreciated.
Edit:
The whole CardStack file: http://pastebin.com/LmhryfGc
Edit2:
I was going to put it on github sooner or later (was hoping after the code looks semi-ok), here it is - https://github.com/d1am0nd/hearths-go/tree/cardstack/redo
In your example, the value of cards in DrawCards is a sub-slice of the CardsStack.cards slice, which is referencing values in the same backing array.
When you call findCard and remove a card from the CardStack.cards slice, you are manipulating the same array that the cards argument is using.
When you want a copy of a slice, you need to allocate a new slice and copy each element. To do this in your example, you could:
ssCopy := make([]deck.Card, len(ss))
copy(ssCopy, ss)
cards.DrawCards(ssCopy)
In go there are functions which return two values or more values, commonly one is an error. Suppose that I want to store the first return value into an already initialized variable, but I would like to initialize the variable to contain the error inline. Is there a way to do this?
For example, say I had this code
var a int
//This code doesn't compile because err doesn't exist
a, err = SomeFuncWithTwoReturnValues()
//This code doesn't compile either
a, err := SomeFuncWithTwoReturnValues()
I know you could do this, but I was hoping there was a way to do it all inline
var a int
var err error
a, err = SomeFuncWithTwoReturnValues()
or
a, err := SomeFuncWithTwoReturnValues()
EDIT: The code above actually compiles, so I looked back at my code to drill down more and have created a quick sample that actually replicates the problem (not just in my mind...).
package main
func myfunc() (int, int) {
return 1, 1
}
func main() {
a := make([]int, 1)
a[0], b := myfunc()
a[0] = b
}
Compiler says main.go|9| non-name a[0] on left side of :=. If I make it = instead of := though then b is never created. I get the feeling that there is not shorthand way to do it though.
As you've mentioned in the comments, you'll need to use the = operator in order to assign to a variable you've already declared. The := operator is used to simultaneously declare and assign a variable. The two are the same:
var x int
x = 5
//is the same as
x := 5
This solution will at least compile:
package main
func myfunc() (int, int) {
return 1, 1
}
func main() {
var b int
a := make([]int, 1)
a[0], b = myfunc()
a[0] = b
}
To answer your question, I don't think there is a way to simultaneously use an undeclared and a declared variable when returning multiple values. That would be trying to use two different operators simultaneously.
Edit: just saw your example from the code that compiles, so it appears you're already familiar with go's assignment operators. I'll leave the example up anyway.
Golang is not a very consistent language. This is a good example. At the beginning I was confused and it would be much simpler if they would always allow the := operator. The compiler is smart enough to detect already declared variables:
package main
import "fmt"
func testFunc() (int,error) {
return 42,fmt.Errorf("Test Error")
}
func main() {
number1,err := testFunc() // OK
number2,err := testFunc() // OK, even if err is already defined
number1,err = testFunc() // OK
// number1,err := testFunc() // ERROR: no new variables on left side of :=
fmt.Println(number1,number2,err)
}
Playground Link: https://play.golang.org/p/eZVB-kG6RtX
It's not consistent, because golang allows you to use := for already declared variables if you assign to them while also introducing a new variable. So the compiler can detect that variables already exists and skip their declaration. But the golang developers decided to allow that only if you introduce at least one new value. The last example shows that.
I ran into this situation like this:
package main
import "os"
func main() {
var cache struct { dir string }
// undefined: err
cache.dir, err = os.UserCacheDir()
// non-name cache.dir on left side of :=
cache.dir, err := os.UserCacheDir()
if err != nil {
panic(err)
}
println(cache.dir)
}
as you discovered, this issue does not have a clean solution. You can declare
an extra variable:
dir, err := os.UserCacheDir()
if err != nil {
panic(err)
}
cache := userCache{dir}
Or, while more verbose, you can declare the error beforehand. This can save
memory, as Go does not use a Rust ownership model:
var (
cache struct { dir string }
err error
)
cache.dir, err = os.UserCacheDir()
As mention in the spec, while using:=, if one of the variables is new, then the old one will just be assigned with the new data.
Unlike regular variable declarations, a short variable declaration may redeclare variables provided they were originally declared earlier in the same block (or the parameter lists if the block is the function body) with the same type, and at least one of the non-blank variables is new. As a consequence, redeclaration can only appear in a multi-variable short declaration. Redeclaration does not introduce a new variable; it just assigns a new value to the original.
field1, offset := nextField(str, 0)
field2, offset := nextField(str, offset) // redeclares offset
As mentioned by the other answers you cannot use assignment and declaration in the same return statement. You have to use either.
However I guess the main reason for your question is cleaning up the code so you don't have to declare an extra err variable above the method or function statement.
You can solve this in two ways:
Declare a global var err error variable and use it in the assignment:
var err error
func MyFunc(someInput string) {
var a int
a, err = someOtherFunction()
}
If your method or function returns an error you can use the declared return variable
func MyFunc(someInput string) (err error) {
var a int
a, err = someOtherFunction()
return
}
I mainly have the problem in methods when I want to assign something to a struct member, e.g.:
type MyStruct struct {
so string
}
func (m *MyStruct) SomeMethod() (err error) {
m.so, err = SomeFunction()
// handle error and continue or return it
return
}