I am trying to remove multiple items from a slice by using the Delete examples from here: http://code.google.com/p/go-wiki/wiki/SliceTricks
Here is the code I have:
package main
import "fmt"
import "net"
func main() {
a := []string{"72.14.191.202", "69.164.200.202", "72.14.180.202", "2600:3c00::22", "2600:3c00::32", "2600:3c00::12"}
fmt.Println(a)
for index, element := range a {
if net.ParseIP(element).To4() == nil {
//a = append(a[:index], a[index+1:]...)
a = a[:index+copy(a[index:], a[index+1:])]
}
}
fmt.Println(a)
}
While the code works fine if I have only one IPv6 address in the slice, it fails if there are more than one IPv6 address. It fails with the error "panic: runtime error: slice bounds out of range". What should I do to fix this code so it's able to delete all IPv6 addresses?
Your problem is that you are modifying the slice that you are iterating over. Below is your code a bit modified:
package main
import (
"fmt"
"net"
)
func main() {
a := []string{"72.14.191.202", "69.164.200.202", "72.14.180.202", "2600:3c00::22", "2600:3c00::32", "2600:3c00::12"}
fmt.Println(a)
for i := 0; i < len(a); i++ {
if net.ParseIP(a[i]).To4() == nil {
a = append(a[:i], a[i+1:]...)
//a = a[:i+copy(a[i:], a[i+1:])]
i-- // Since we just deleted a[i], we must redo that index
}
}
fmt.Println(a)
}
Playground
Just to raise the point : it is always tricky to alter the structure on which you are iterating.
A common way to avoid this is to build the end result in a new variable :
package main
import (
"fmt"
"net"
)
func main() {
a := []string{"72.14.191.202", "69.164.200.202", "72.14.180.202", "2600:3c00::22", "2600:3c00::32", "2600:3c00::12"}
fmt.Println(a)
var b []string
for _, ip := range a {
if net.ParseIP(ip).To4() != nil {
b = append(b, ip)
}
}
fmt.Println(b)
}
http://play.golang.org/p/7CLMPw_FQi
Related
Here is a simple concurrent map that I wrote for learning purpose
package concurrent_hashmap
import (
"hash/fnv"
"sync"
)
type ConcurrentMap struct {
buckets []ThreadSafeMap
bucketCount uint32
}
type ThreadSafeMap struct {
mapLock sync.RWMutex
hashMap map[string]interface{}
}
func NewConcurrentMap(bucketSize uint32) *ConcurrentMap {
var threadSafeMapInstance ThreadSafeMap
var bucketOfThreadSafeMap []ThreadSafeMap
for i := 0; i <= int(bucketSize); i++ {
threadSafeMapInstance = ThreadSafeMap{sync.RWMutex{}, make(map[string]interface{})}
bucketOfThreadSafeMap = append(bucketOfThreadSafeMap, threadSafeMapInstance)
}
return &ConcurrentMap{bucketOfThreadSafeMap, bucketSize}
}
func (cMap *ConcurrentMap) Put(key string, val interface{}) {
bucketIndex := hash(key) % cMap.bucketCount
bucket := cMap.buckets[bucketIndex]
bucket.mapLock.Lock()
bucket.hashMap[key] = val
bucket.mapLock.Unlock()
}
// Helper
func hash(s string) uint32 {
h := fnv.New32a()
h.Write([]byte(s))
return h.Sum32()
}
I am trying to write a simple benchmark and I find that synchronize access will work correctly but concurrent access will get
fatal error: concurrent map writes
Here is my benchmark run with go test -bench=. -race
package concurrent_hashmap
import (
"testing"
"runtime"
"math/rand"
"strconv"
"sync"
)
// Concurrent does not work
func BenchmarkMyFunc(b *testing.B) {
var wg sync.WaitGroup
runtime.GOMAXPROCS(runtime.NumCPU())
my_map := NewConcurrentMap(uint32(4))
for n := 0; n < b.N; n++ {
go insert(my_map, wg)
}
wg.Wait()
}
func insert(my_map *ConcurrentMap, wg sync.WaitGroup) {
wg.Add(1)
var rand_int int
for element_num := 0; element_num < 1000; element_num++ {
rand_int = rand.Intn(100)
my_map.Put(strconv.Itoa(rand_int), rand_int)
}
defer wg.Done()
}
// This works
func BenchmarkMyFuncSynchronize(b *testing.B) {
my_map := NewConcurrentMap(uint32(4))
for n := 0; n < b.N; n++ {
my_map.Put(strconv.Itoa(123), 123)
}
}
The WARNING: DATA RACE is saying that bucket.hashMap[key] = val is causing the problem, but I am confused on why that is possible, since I lock that logic whenever write is happening.
I think I am missing something basic, can someone point out my mistake?
Thanks
Edit1:
Not sure if this helps but here is what my mutex looks like if I don't lock anything
{{0 0} 0 0 0 0}
Here is what it looks like if I lock the write
{{1 0} 0 0 -1073741824 0}
Not sure why my readerCount is a low negative number
Edit:2
I think I find where the issue is at, but not sure why I have to code that way
The issue is
type ThreadSafeMap struct {
mapLock sync.RWMutex // This is causing problem
hashMap map[string]interface{}
}
it should be
type ThreadSafeMap struct {
mapLock *sync.RWMutex
hashMap map[string]interface{}
}
Another weird thing is that in Put if I put print statement inside lock
bucket.mapLock.Lock()
fmt.Println("start")
fmt.Println(bucket)
fmt.Println(bucketIndex)
fmt.Println(bucket.mapLock)
fmt.Println(&bucket.mapLock)
bucket.hashMap[key] = val
defer bucket.mapLock.Unlock()
The following prints is possible
start
start
{0x4212861c0 map[123:123]}
{0x4212241c0 map[123:123]}
Its weird because each start printout should be follow with 4 lines of bucket info since you cannot have start back to back because that would indicate that multiple thread is access the line inside lock
Also for some reason each bucket.mapLock have different address even if I make the bucketIndex static, that indicate that I am not even accessing the same lock.
But despite the above weirdness changing mutex to pointer solves my problem
I would love to find out why I need pointers for mutex and why the prints seem to indicate multiple thread is accessing the lock and why each lock has different address.
The problem is with the statement
bucket := cMap.buckets[bucketIndex]
bucket now contains copy of the ThreadSafeMap at that index. As sync.RWMutex is stored as value, a copy of it is made while assigning. But map maps hold references to an underlying data structure, so the copy of the pointer or the same map is passed. The code locks a copy of the lock while writing to a single map, which cause the problem.
Thats why you don't face any problem when you change sync.RWMutex to *sync.RWMutex. It's better to store reference to structure in map as shown.
package concurrent_hashmap
import (
"hash/fnv"
"sync"
)
type ConcurrentMap struct {
buckets []*ThreadSafeMap
bucketCount uint32
}
type ThreadSafeMap struct {
mapLock sync.RWMutex
hashMap map[string]interface{}
}
func NewConcurrentMap(bucketSize uint32) *ConcurrentMap {
var threadSafeMapInstance *ThreadSafeMap
var bucketOfThreadSafeMap []*ThreadSafeMap
for i := 0; i <= int(bucketSize); i++ {
threadSafeMapInstance = &ThreadSafeMap{sync.RWMutex{}, make(map[string]interface{})}
bucketOfThreadSafeMap = append(bucketOfThreadSafeMap, threadSafeMapInstance)
}
return &ConcurrentMap{bucketOfThreadSafeMap, bucketSize}
}
func (cMap *ConcurrentMap) Put(key string, val interface{}) {
bucketIndex := hash(key) % cMap.bucketCount
bucket := cMap.buckets[bucketIndex]
bucket.mapLock.Lock()
bucket.hashMap[key] = val
bucket.mapLock.Unlock()
}
// Helper
func hash(s string) uint32 {
h := fnv.New32a()
h.Write([]byte(s))
return h.Sum32()
}
It's possible to validate the scenario by modifying the function Put as follows
func (cMap *ConcurrentMap) Put(key string, val interface{}) {
//fmt.Println("index", key)
bucketIndex := 1
bucket := cMap.buckets[bucketIndex]
fmt.Printf("%p %p\n", &(bucket.mapLock), bucket.hashMap)
}
How to get function arguments in Go at runtime, all I know is only how to get function name:
pc, file, line, ok := runtime.Caller(2)
rt := runtime.FuncForPC(pc)
return rt.Name() // Foo
What I need is something like this:
Foo(1,2,3)
// Foo_1_2_3
Not a full answer, but maybe this can help :
package main
import (
"fmt"
"reflect"
)
func main() {
fmt.Println(reflect.TypeOf(f1))
for index := 0; index < reflect.TypeOf(f1).NumIn(); index++ {
fmt.Println(reflect.TypeOf(f1).In(index))
}
}
func f1(a int, b string) {}
prints :
func(int, string)
int
string
I am trying to implement linked list of struct in Go using list package available. This is the sample code below -
package main
import (
"container/list"
"fmt"
)
type A struct{
B int
}
func main() {
l := list.New()
for i:=0;i<5;i++ {
c := A{i}
l.PushFront(c)
}
for e := l.Front(); e != nil; e = e.Next() {
fmt.Println(e.Value)
}
}
Output I get is in form {0} and same for other iterations. How can I access the field to get the integer value ?
You have to cast the list item back to A. You can see it running here.
package main
import (
"container/list"
"fmt"
)
type A struct {
B int
}
func main() {
l := list.New()
for i := 0; i < 5; i++ {
c := A{i}
l.PushFront(c)
}
for e := l.Front(); e != nil; e = e.Next() {
v := e.Value.(A)
fmt.Println(v.B)
}
}
You are getting access to the structure through e.Value, but you are not getting access to the structures value. You will need to do e.Value.B
How can I access a capture group from inside ReplaceAllFunc()?
package main
import (
"fmt"
"regexp"
)
func main() {
body := []byte("Visit this page: [PageName]")
search := regexp.MustCompile("\\[([a-zA-Z]+)\\]")
body = search.ReplaceAllFunc(body, func(s []byte) []byte {
// How can I access the capture group here?
})
fmt.Println(string(body))
}
The goal is to replace [PageName] with PageName.
This is the last task under the "Other tasks" section at the bottom of the Writing Web Applications Go tutorial.
I agree that having access to capture group while inside of your function would be ideal, I don't think it's possible with regexp.ReplaceAllFunc.
Only thing that comes to my mind right now regard how to do this with that function is this:
package main
import (
"fmt"
"regexp"
)
func main() {
body := []byte("Visit this page: [PageName] [OtherPageName]")
search := regexp.MustCompile("\\[[a-zA-Z]+\\]")
body = search.ReplaceAllFunc(body, func(s []byte) []byte {
m := string(s[1 : len(s)-1])
return []byte("" + m + "")
})
fmt.Println(string(body))
}
EDIT
There is one other way I know how to do what you want. First thing you need to know is that you can specify non capturing group using syntax (?:re) where re is your regular expression. This is not essential, but will reduce number of not interesting matches.
Next thing to know is regexp.FindAllSubmatcheIndex. It will return slice of slices, where each internal slice represents ranges of all submatches for given matching of regexp.
Having this two things, you can construct somewhat generic solution:
package main
import (
"fmt"
"regexp"
)
func ReplaceAllSubmatchFunc(re *regexp.Regexp, b []byte, f func(s []byte) []byte) []byte {
idxs := re.FindAllSubmatchIndex(b, -1)
if len(idxs) == 0 {
return b
}
l := len(idxs)
ret := append([]byte{}, b[:idxs[0][0]]...)
for i, pair := range idxs {
// replace internal submatch with result of user supplied function
ret = append(ret, f(b[pair[2]:pair[3]])...)
if i+1 < l {
ret = append(ret, b[pair[1]:idxs[i+1][0]]...)
}
}
ret = append(ret, b[idxs[len(idxs)-1][1]:]...)
return ret
}
func main() {
body := []byte("Visit this page: [PageName] [OtherPageName][XYZ] [XY]")
search := regexp.MustCompile("(?:\\[)([a-zA-Z]+)(?:\\])")
body = ReplaceAllSubmatchFunc(search, body, func(s []byte) []byte {
m := string(s)
return []byte("" + m + "")
})
fmt.Println(string(body))
}
If you want to get group in ReplaceAllFunc, you can use ReplaceAllString to get the subgroup.
package main
import (
"fmt"
"regexp"
)
func main() {
body := []byte("Visit this page: [PageName]")
search := regexp.MustCompile("\\[([a-zA-Z]+)\\]")
body = search.ReplaceAllFunc(body, func(s []byte) []byte {
// How can I access the capture group here?
group := search.ReplaceAllString(string(s), `$1`)
fmt.Println(group)
// handle group as you wish
newGroup := "<a href='/view/" + group + "'>" + group + "</a>"
return []byte(newGroup)
})
fmt.Println(string(body))
}
And when there are many groups, you are able to get each group by this way, then handle each group and return desirable value.
You have to call ReplaceAllFunc first and within the function call FindStringSubmatch on the same regex again. Like:
func (p parser) substituteEnvVars(data []byte) ([]byte, error) {
var err error
substituted := p.envVarPattern.ReplaceAllFunc(data, func(matched []byte) []byte {
varName := p.envVarPattern.FindStringSubmatch(string(matched))[1]
value := os.Getenv(varName)
if len(value) == 0 {
log.Printf("Fatal error substituting environment variable %s\n", varName)
}
return []byte(value)
});
return substituted, err
}
Is there a way to scan a big.Int directly from the standard input in Go? Right now I'm doing this:
package main
import (
"fmt"
"math/big"
)
func main() {
w := new(big.Int)
var s string
fmt.Scan(&s)
fmt.Sscan(s, w)
fmt.Println(w)
}
I also could have used .SetString. But, is there a way to Scan the big.Int directly from the standard input without scanning a string or an integer first?
For example,
package main
import (
"fmt"
"math/big"
)
func main() {
w := new(big.Int)
n, err := fmt.Scan(w)
fmt.Println(n, err)
fmt.Println(w.String())
}
Input (stdin):
295147905179352825857
Output (stdout):
1 <nil>
295147905179352825857
As far as I know - no, there's no other way. In fact, what you've got is the default example they have for scanning big.Int in the documentation.
package main
import (
"fmt"
"log"
"math/big"
)
func main() {
// The Scan function is rarely used directly;
// the fmt package recognizes it as an implementation of fmt.Scanner.
i := new(big.Int)
_, err := fmt.Sscan("18446744073709551617", i)
if err != nil {
log.Println("error scanning value:", err)
} else {
fmt.Println(i)
}
}
You can see the relevant section here - http://golang.org/pkg/math/big/#Int.Scan