I've seen there is a struct testing.BenchmarkResult in Go to accesss the result of a benchmark but I found very little documentation or examples to help me use it.
So far I have only been benchmarking my functions like this:
func BenchmarkMyFunction(b *testing.B) {
// call to myFunction
}
And then running:
go test -bench=".*"
Here the results are printed to the console but I'd like to store them in a separate file. How can I use the BenchmarkResult type to do this?
For example:
package main
import (
"fmt"
"testing"
"time"
)
func Add(a, b int) int {
time.Sleep(10 * time.Microsecond) // Just to make the test take some time
return a + b
}
func BenchAdd(b *testing.B) {
for i := 0; i < b.N; i++ {
_ = Add(1, 2)
}
}
func main() {
res := testing.Benchmark(BenchAdd)
fmt.Printf("%s\n%#[1]v\n", res)
}
Produces:
120000 10000 ns/op
testing.BenchmarkResult{N:120000, T:1200000000, Bytes:0, MemAllocs:0x0, MemBytes:0x0, Extra:map[string]float64{}}
Playground.
You can easily write these results out to a file using ioutil.WriteFile.
Playground w/ WriteFile.
Related
Here is a simple concurrent map that I wrote for learning purpose
package concurrent_hashmap
import (
"hash/fnv"
"sync"
)
type ConcurrentMap struct {
buckets []ThreadSafeMap
bucketCount uint32
}
type ThreadSafeMap struct {
mapLock sync.RWMutex
hashMap map[string]interface{}
}
func NewConcurrentMap(bucketSize uint32) *ConcurrentMap {
var threadSafeMapInstance ThreadSafeMap
var bucketOfThreadSafeMap []ThreadSafeMap
for i := 0; i <= int(bucketSize); i++ {
threadSafeMapInstance = ThreadSafeMap{sync.RWMutex{}, make(map[string]interface{})}
bucketOfThreadSafeMap = append(bucketOfThreadSafeMap, threadSafeMapInstance)
}
return &ConcurrentMap{bucketOfThreadSafeMap, bucketSize}
}
func (cMap *ConcurrentMap) Put(key string, val interface{}) {
bucketIndex := hash(key) % cMap.bucketCount
bucket := cMap.buckets[bucketIndex]
bucket.mapLock.Lock()
bucket.hashMap[key] = val
bucket.mapLock.Unlock()
}
// Helper
func hash(s string) uint32 {
h := fnv.New32a()
h.Write([]byte(s))
return h.Sum32()
}
I am trying to write a simple benchmark and I find that synchronize access will work correctly but concurrent access will get
fatal error: concurrent map writes
Here is my benchmark run with go test -bench=. -race
package concurrent_hashmap
import (
"testing"
"runtime"
"math/rand"
"strconv"
"sync"
)
// Concurrent does not work
func BenchmarkMyFunc(b *testing.B) {
var wg sync.WaitGroup
runtime.GOMAXPROCS(runtime.NumCPU())
my_map := NewConcurrentMap(uint32(4))
for n := 0; n < b.N; n++ {
go insert(my_map, wg)
}
wg.Wait()
}
func insert(my_map *ConcurrentMap, wg sync.WaitGroup) {
wg.Add(1)
var rand_int int
for element_num := 0; element_num < 1000; element_num++ {
rand_int = rand.Intn(100)
my_map.Put(strconv.Itoa(rand_int), rand_int)
}
defer wg.Done()
}
// This works
func BenchmarkMyFuncSynchronize(b *testing.B) {
my_map := NewConcurrentMap(uint32(4))
for n := 0; n < b.N; n++ {
my_map.Put(strconv.Itoa(123), 123)
}
}
The WARNING: DATA RACE is saying that bucket.hashMap[key] = val is causing the problem, but I am confused on why that is possible, since I lock that logic whenever write is happening.
I think I am missing something basic, can someone point out my mistake?
Thanks
Edit1:
Not sure if this helps but here is what my mutex looks like if I don't lock anything
{{0 0} 0 0 0 0}
Here is what it looks like if I lock the write
{{1 0} 0 0 -1073741824 0}
Not sure why my readerCount is a low negative number
Edit:2
I think I find where the issue is at, but not sure why I have to code that way
The issue is
type ThreadSafeMap struct {
mapLock sync.RWMutex // This is causing problem
hashMap map[string]interface{}
}
it should be
type ThreadSafeMap struct {
mapLock *sync.RWMutex
hashMap map[string]interface{}
}
Another weird thing is that in Put if I put print statement inside lock
bucket.mapLock.Lock()
fmt.Println("start")
fmt.Println(bucket)
fmt.Println(bucketIndex)
fmt.Println(bucket.mapLock)
fmt.Println(&bucket.mapLock)
bucket.hashMap[key] = val
defer bucket.mapLock.Unlock()
The following prints is possible
start
start
{0x4212861c0 map[123:123]}
{0x4212241c0 map[123:123]}
Its weird because each start printout should be follow with 4 lines of bucket info since you cannot have start back to back because that would indicate that multiple thread is access the line inside lock
Also for some reason each bucket.mapLock have different address even if I make the bucketIndex static, that indicate that I am not even accessing the same lock.
But despite the above weirdness changing mutex to pointer solves my problem
I would love to find out why I need pointers for mutex and why the prints seem to indicate multiple thread is accessing the lock and why each lock has different address.
The problem is with the statement
bucket := cMap.buckets[bucketIndex]
bucket now contains copy of the ThreadSafeMap at that index. As sync.RWMutex is stored as value, a copy of it is made while assigning. But map maps hold references to an underlying data structure, so the copy of the pointer or the same map is passed. The code locks a copy of the lock while writing to a single map, which cause the problem.
Thats why you don't face any problem when you change sync.RWMutex to *sync.RWMutex. It's better to store reference to structure in map as shown.
package concurrent_hashmap
import (
"hash/fnv"
"sync"
)
type ConcurrentMap struct {
buckets []*ThreadSafeMap
bucketCount uint32
}
type ThreadSafeMap struct {
mapLock sync.RWMutex
hashMap map[string]interface{}
}
func NewConcurrentMap(bucketSize uint32) *ConcurrentMap {
var threadSafeMapInstance *ThreadSafeMap
var bucketOfThreadSafeMap []*ThreadSafeMap
for i := 0; i <= int(bucketSize); i++ {
threadSafeMapInstance = &ThreadSafeMap{sync.RWMutex{}, make(map[string]interface{})}
bucketOfThreadSafeMap = append(bucketOfThreadSafeMap, threadSafeMapInstance)
}
return &ConcurrentMap{bucketOfThreadSafeMap, bucketSize}
}
func (cMap *ConcurrentMap) Put(key string, val interface{}) {
bucketIndex := hash(key) % cMap.bucketCount
bucket := cMap.buckets[bucketIndex]
bucket.mapLock.Lock()
bucket.hashMap[key] = val
bucket.mapLock.Unlock()
}
// Helper
func hash(s string) uint32 {
h := fnv.New32a()
h.Write([]byte(s))
return h.Sum32()
}
It's possible to validate the scenario by modifying the function Put as follows
func (cMap *ConcurrentMap) Put(key string, val interface{}) {
//fmt.Println("index", key)
bucketIndex := 1
bucket := cMap.buckets[bucketIndex]
fmt.Printf("%p %p\n", &(bucket.mapLock), bucket.hashMap)
}
How to get function arguments in Go at runtime, all I know is only how to get function name:
pc, file, line, ok := runtime.Caller(2)
rt := runtime.FuncForPC(pc)
return rt.Name() // Foo
What I need is something like this:
Foo(1,2,3)
// Foo_1_2_3
Not a full answer, but maybe this can help :
package main
import (
"fmt"
"reflect"
)
func main() {
fmt.Println(reflect.TypeOf(f1))
for index := 0; index < reflect.TypeOf(f1).NumIn(); index++ {
fmt.Println(reflect.TypeOf(f1).In(index))
}
}
func f1(a int, b string) {}
prints :
func(int, string)
int
string
Below is a simple go example. I have omitted error handling etc intentionally to make the example short. I have a simple for loop calling the writeOutput function 5 times using the go keyword to make the function run concurrently.
What I expect to happen is 5 files are created in /tmp/ with the contents of test.
What happens is that no files are created.
However if I remove the go keyword the code executes as expected. Im overlooking something super obvious. My background is dynamically typed languages like PHP/Ruby so just getting to grips with go and can't understand why 5 files are created when the go keyword exists.
package main
import (
"os"
"math/rand"
"strconv"
)
func main() {
for i := 0; i < 5; i++ {
go writeOutput()
}
}
func writeOutput() {
filename := strconv.Itoa(rand.Intn(10000))
file, _ := os.Create("/tmp/" + filename)
defer file.Close()
file.WriteString("test")
}
I managed to solve this with a wait group as suggested in the comments.
package main
import (
"math/rand"
"os"
"strconv"
"sync"
)
func main() {
var wg sync.WaitGroup
for i := 0; i < 5; i++ {
wg.Add(1)
go func() {
defer wg.Done()
writeOutput()
}()
}
wg.Wait()
}
func writeOutput() {
filename := strconv.Itoa(rand.Intn(10000))
file, _ := os.Create("/tmp/" + filename)
defer file.Close()
file.WriteString("test")
}
I am currently learming Go. I am readging the book An Introduction to programming in go
I am at the concurrency section and form what I understand I can see two way to define an infinite loop a go program.
func pinger(c chan string) {
for i := 0; ; i++ {
c <- "ping"
}
}
func printer(c chan string) {
for {
msg := <- c
fmt.Println(msg)
time.Sleep(time.Second * 1)
}
}
I am wondering what is the use of the i variable in the pinger function. What is the best "go" way to declare an infinite loop ? I would say the the one in the printer function is better but as I am new to I might miss something with the declaration in the pinger function.
Thanks for all people who will help.
The i in the first loop is redundant; it's always best to get rid of unused variables therefore You should use a for{} in the pinger() function as well.
Here is a working example:
package main
import(
"time"
"fmt"
)
func main() {
c := make(chan string)
go printer(c)
go pinger(c)
time.Sleep(time.Second * 60)
}
func pinger(c chan string) {
for{
c <- "ping"
}
}
func printer(c chan string) {
for {
msg := <- c
fmt.Println(msg)
time.Sleep(time.Second * 1)
}
}
Run on playground
The "best" way is to write code that is easy to read and maintain. Your variable i in func pinger serves no purpose and someone stumbling upon that code later on will have a hard time understand what it's for.
I would just do
func pinger(c chan string) {
for {
c <- "ping"
}
}
I am trying to remove multiple items from a slice by using the Delete examples from here: http://code.google.com/p/go-wiki/wiki/SliceTricks
Here is the code I have:
package main
import "fmt"
import "net"
func main() {
a := []string{"72.14.191.202", "69.164.200.202", "72.14.180.202", "2600:3c00::22", "2600:3c00::32", "2600:3c00::12"}
fmt.Println(a)
for index, element := range a {
if net.ParseIP(element).To4() == nil {
//a = append(a[:index], a[index+1:]...)
a = a[:index+copy(a[index:], a[index+1:])]
}
}
fmt.Println(a)
}
While the code works fine if I have only one IPv6 address in the slice, it fails if there are more than one IPv6 address. It fails with the error "panic: runtime error: slice bounds out of range". What should I do to fix this code so it's able to delete all IPv6 addresses?
Your problem is that you are modifying the slice that you are iterating over. Below is your code a bit modified:
package main
import (
"fmt"
"net"
)
func main() {
a := []string{"72.14.191.202", "69.164.200.202", "72.14.180.202", "2600:3c00::22", "2600:3c00::32", "2600:3c00::12"}
fmt.Println(a)
for i := 0; i < len(a); i++ {
if net.ParseIP(a[i]).To4() == nil {
a = append(a[:i], a[i+1:]...)
//a = a[:i+copy(a[i:], a[i+1:])]
i-- // Since we just deleted a[i], we must redo that index
}
}
fmt.Println(a)
}
Playground
Just to raise the point : it is always tricky to alter the structure on which you are iterating.
A common way to avoid this is to build the end result in a new variable :
package main
import (
"fmt"
"net"
)
func main() {
a := []string{"72.14.191.202", "69.164.200.202", "72.14.180.202", "2600:3c00::22", "2600:3c00::32", "2600:3c00::12"}
fmt.Println(a)
var b []string
for _, ip := range a {
if net.ParseIP(ip).To4() != nil {
b = append(b, ip)
}
}
fmt.Println(b)
}
http://play.golang.org/p/7CLMPw_FQi