some confusions about lambda function/closure in Golang - go

package main
import (
"fmt"
)
func main(){
f,val,val1:=fibonacci()
fmt.Println(val,val1)
for i:=0;i<=10;i++ {
fmt.Println(f(i),val,val1)
}
_,val,val1=fibonacci()
fmt.Println(val,val1)
}
func fibonacci()(func(n int)int,int,int){
var val int
var val1 int
f:=func(n int)int{
if n==0||n==1{
val,val1=1,1
}else{
val,val1=val+val1,val
}
return val
}
fmt.Println("fibonacci val =",val,"val1 =",val1)
return f,val,val1
}
Here is my code on sloving fibonacci without using recursion when I
read about lambda function/closure. And the Go Documentary says a
closure will capture some external state. My understanding is the
closure will keep a copy of state of the function which it is
declared. These states are just copies whatever I do on them won't
modify the original, is that so?

from your test code here: https://play.golang.org/p/pajT2bAIe2
your fibonacci function is working out the nth numbers in the sequence provided it's called in an incremental fashion as you are doing. but the values you return from the initial call to fibonacci are not pointers (or references) to those integer values they are just the values of those integers at that time, think of them as being copied out of the function, try using integer pointers instead like this: https://play.golang.org/p/-vLja7Fpsq
package main
import (
"fmt"
)
func main() {
f, val, val1 := fibonacci()
fmt.Println(val, val1)
for i := 0; i <= 10; i++ {
fmt.Println(f(i), *val, *val1) //dereference the pointer to get its value at the current time
}
_, val, val1 = fibonacci()
fmt.Println(*val, *val1)
}
func fibonacci() (func(n int) int, *int, *int) {
var val int
var val1 int
f := func(n int) int {
if n == 0 || n == 1 {
val, val1 = 1, 1
} else {
val, val1 = val+val1, val
}
return val
}
fmt.Println("fibonacci val =", val, "val1 =", val1)
return f, &val, &val1 // return pointers to the closured values instead of just the values
}

Although you have accepted the above answer i'm giving another explanation. The reason why you receive the last value of the loop operation has to do with the Go's lexical scoping. The for loop introduces a new lexical block in which the value is referenced by it's memory address, so by pointer and not by it's value. In order to get the value, you have to de-reference.
Each time the for loop makes an iteration the value processed is pointing to the same memory address. All the function values created by this loop "capture" and share the same variable - and addressable storage location, not it's value at that particular moment. Thus when the last iteration is finished, the variable holds the value from the final step.
A much better approach for these kind of operations would be to use goroutines, because in these cases you are not communicating through sharing the same memory address, but you are sharing the memory through communication.
Here is a more elegant solution using goroutine:
package main
import (
"fmt"
)
func fibonacci(ch chan interface{}, quit chan struct{}) {
x, y := 1, 1
for {
select {
case ch <- x:
x, y = y, x+y
fmt.Println(x , y)
case <-quit:
fmt.Println("Quiting...")
return
}
}
}
func main() {
ch := make(chan interface{})
quit := make(chan struct{})
go func() {
for i := 0; i < 10; i++ {
<-ch
}
quit <- struct{}{}
}()
fibonacci(ch, quit)
}
https://play.golang.org/p/oPQgXWyV9u

Related

What does the underscore(_) do in for loop Golang?

I am just getting started learning the Golang language!
In for loop, I saw sometimes adding an underscore or without underscore.
Whatever add _ or not, I got the same result.
package main
import (
"fmt"
)
func main() {
doSomething()
sum := addValues(5, 8)
fmt.Println("The sum is", sum)
multiSum, multiCount := addAllValues(4, 7, 9)
fmt.Println("multisum", multiSum)
fmt.Println("multiCount", multiCount)
}
func doSomething() {
fmt.Println("Doing Something")
}
func addValues(value1 int, value2 int) int {
return value1 + value2
}
func addAllValues(values ...int) (int, int) {
total := 0
for _, v := range values {
total += v
}
return total, len(values)
}
func addAllValues(values ...int) (int, int) {
total := 0
for v := range values {
total += v
}
return total, len(values)
}
All I know is I don't care about the index. Is that all? or there is something more what I have to know??
I really appreciate your help!
For range over slices:
In for v := range values { the v is the index of the element in the slice.
In for _, v := range values { the v is the actual element value.
In for i, v := range values { the i is the index and the v is the element.
In for i, _ := range values { the i is the index of the element in the slice.
You can run this playground example to see the differences.
Range expression 1st value 2nd value
array or slice a [n]E, *[n]E, or []E index i int a[i] E
string s string type index i int see below rune
map m map[K]V key k K m[k] V
channel c chan E, <-chan E element e E
For more details see the spec.
If you don't want to use the variable that iterates in the loop, you can use _ to simply let Go ignore it:
mySlice := [int]{1,3,4,59,5}
for _,x := range mySlice {
fmt.Println(x)
}
By placing underscore you are telling the compiler this:
Ok, I'm aware that this function is returning something but I don't care! For example:
package main
import "fmt"
func main() {
mul1, add1 := test_function(2, 3)
fmt.Println(mul1, add1)
mul2, _ := test_function(4, 5)
fmt.Println(mul2)
_, add3 := test_function(7, 8)
fmt.Println(add3)
}
func test_function(a int, b int) (mul int, add int) {
return a * b, a + b
}
just to add to the amazing answer above:
I think one of the main benefits is to maintain readability in your program: if you replace the blank identifier with a variable then you have to use it or your program will not compile.
also this decrease memory allocation be neglecting one of the returned parameters...

Im trying to understand why this case is always going off

I'm trying to understand why in this select statement the first case always goes off and does not wait for the channel to be filled. For this program I'm trying to get the program to wait till all the channels have been filled and whenever a channel is filled by the method it is put in the first available space in the array of channels
I tried putting the line <-res[i] in the case statement but for some reason this case always goes off regardless of whether or not the channels have a value.
package main
import (
"fmt"
"math"
"math/rand"
"time"
)
func numbers(sz int) (res chan float64) {
res = make(chan float64)
go func() {
defer close(res)
num := 0.0
time.Sleep(time.Duration(rand.Intn(1000)) *time.Microsecond)
for i := 0; i < sz; i++ {
num += math.Sqrt(math.Abs(rand.Float64()))
}
num /= float64(sz)
res <- num
return
}()
return
}
func main() {
var nGo int
rand.Seed(42)
fmt.Print("Number of Go routines: ")
fmt.Scanf("%d \n", &nGo)
res := make([]chan float64, nGo)
j:=0
for i := 0; i < nGo; i++ {
res[i] =numbers(1000)
}
for true{
for i := 0; i < nGo; {
select {
case <-res[i]:{
res[j]=res[i]//this line
j++
}
default:
i++
}
}
if j==nGo{
break
}
}
fmt.Println(<-res[nGo-1])
}
The print line should print some float.
<-res[i] in the case statement but for some reason this case always goes off regarless of wether or not the channels has a value
It will only not choose this case if the channel's buffer is full (i.e. a value cannot be sent without blocking). Your channel has a buffer length equal to the number of values you're sending on it, so it will never block, giving it no reason to ever take the default case.

How to create global counter in highly concurrent system

I'm creating global counter, which can be shared between goroutines.
Referring to this question, following code may satisfy my needs.
However if there ware lots of concurrent requests, could it happen that the same number is assigned to more than two goroutines?
If so how can I avoid this?
This question is different from the link I pasted, as what I want to know about is how I can avoid duplication using channel counter. if the only possible solution is other implementation like sync.Mutex or atomic, I'll use it. however, according to the link (again), channel seems to be the best option. Any comment or answer really helpful. thanks in advance.
I'm new to multithread coding and also go, might be silly question. sorry for that.
package main
import (
"fmt"
"time"
)
var counter int
var counter_chan chan int
func main() {
counter_chan = make(chan int, 100)
counter = 0
go func() {
for {
select {
case chanc := <-counter_chan:
counter += chanc
fmt.Printf("%d \n", counter)
}
}
}()
for i := 0; i < 10; i++ {
go AddCounter(counter_chan)
}
time.Sleep(time.Second)
fmt.Printf("Total Count is ... %d \n", GetCount())
}
func AddCounter(ch chan int) {
ch <- 1
}
func GetCount() int {
return counter
}
func ResetCount() {
if counter > 8190 {
counter = 0
}
}
-- Edit 05/14 2018
Assume following code is thread-safe for getting and resetting value. Am I right?
package main
import (
"fmt"
"time"
)
var counter int
var addCounterChan chan int
var readCounterChan chan int
func main() {
addCounterChan = make(chan int, 100)
readCounterChan = make(chan int, 100)
counter = 0
go func() {
for {
select {
case val := <-addCounterChan:
counter += val
if counter > 5 {
counter = 0
}
readCounterChan <- counter
fmt.Printf("%d \n", counter)
}
}
}()
for i := 0; i < 10; i++ {
go AddCounter(addCounterChan)
}
time.Sleep(time.Second)
for i := 0; i < 10; i++ {
fmt.Printf("Total Count #%d is ... %d \n", (i + 1), GetCount(readCounterChan))
}
}
// Following two functions will be implemented in another package in real case.
func AddCounter(ch chan int) {
ch <- 1
}
func GetCount(ch chan int) int {
r := <-ch
return r
}
The direct answer to your question is: The code you've pasted updates the counter safely, but doesn't read or reset it safely.
Contrary to the accepted answer in the question you linked to, however, the easiest, most efficient way to implement a shared counter is with the atomic package. It can be used to atomically increment several common types. Example:
var globalCounter *int32 = new(int32)
// .. later in your code
currentCount := atomic.AddInt32(globalCounter, 1)
Use a sync.Mutex to create a counter with add, get and reset operations as shown in the question.
type counter struct {
mu sync.Mutex
n int
}
func (c *counter) Add() {
c.mu.Lock()
c.n++
c.mu.Unlock()
}
func (c *counter) Get() int {
c.mu.Lock()
n := c.n
c.mu.Unlock()
return n
}
func (c *counter) Reset() {
c.mu.Lock()
if c.n > 8190 {
c.n = 0
}
c.mu.Unlock()
}
If the reset function is not needed, then use the sync/atomic.
type counter struct {
n int32
}
func (c *counter) Add() {
atomic.AddInt32(&c.n, 1)
}
func (c *counter) Get() int {
return int(atomic.LoadInt32(&c.n))
}
Go 1.19
The sync/atomic package now includes atomic types, such as atomic.Int32, which you can use to manage a value that can only be accessed atomically.
This basically accomplishes the same thing as having a custom struct with a mutex, or using top-level atomic functions to read and write a "naked" numerical type. Instead of rolling your own, you can simply rely on the standard library.
A simple example:
package main
import (
"fmt"
"sync"
"sync/atomic"
)
// zero value is 0
var counter = atomic.Int32{}
func main() {
wg := &sync.WaitGroup{}
wg.Add(100)
for i := 0; i < 100; i++ {
go func() {
counter.Add(1)
wg.Done()
}()
}
wg.Wait()
fmt.Println(counter.Load())
}
Playground: https://go.dev/play/p/76xM3xXTAM5?v=gotip

get sum of bigInt number golang

Hi i am new to the golang programming language. I can get the bigint
value from the factoral function but it is not working with the add
function.
i have had the add function accepting bigint but when i try to add a
.Mod and .Div methods it returns 0 fro some reason. the if else
statement in the add function is an old statement i was using when i
had int values coming from the factoral function.
it worked perfectly when it was as int value. When i attempted to
alter the if else statement to accept bigint values i couldn't get it
to work at all.
I have tried the .Mod and .Div methods and they are printing out the
correct values. But when i try to .Add them together it always returns
0. even if the values are "22", "2". I've put the if else statement back to the original int values for now.
if anyone can help me out with this one i would be very greatful.
package main
import (
"fmt"
"math/big"
)
func factoral(n uint64) (r *big.Int) {
one, bn := big.NewInt(1), new(big.Int).SetUint64(n)
r = big.NewInt(1)
if bn.Cmp(one) <= 0 {
return
}
for i := big.NewInt(2); i.Cmp(bn) <= 0; i.Add(i, one) {
r.Mul(r, i)
}
return
}
func add(number *big.Int) *big.Int {
//this the statement that works with normal int values
if number/10 < 10 {
return sum + number/10
} else {
return sum + add(number/10)
}
}
func main() {
fmt.Println(add(factoral(100)))
}
Fixed it
package main
import (
"fmt"
"math/big"
)
func factoral(n uint64) (r *big.Int) {
one, bn := big.NewInt(1), new(big.Int).SetUint64(n)
r = big.NewInt(1)
if bn.Cmp(one) <= 0 {
return
}
for i := big.NewInt(2); i.Cmp(bn) <= 0; i.Add(i, one) {
r.Mul(r, i)
}
return
}
func add(number *big.Int) *big.Int {
ten := big.NewInt(10)
sum := big.NewInt(0)
mod := big.NewInt(0)
for ten.Cmp(number)<0 {
sum.Add(sum, mod.Mod(number,ten))
number.Div(number,ten)
}
sum.Add(sum,number)
return sum
}
func main() {
fmt.Println(add(factoral(100)))
}
Seems your issue was likely with the way the Big Int object whose method you invoke is going to be the one the value is assigned to, and not necessarily one of the arguments.
See: https://golang.org/pkg/math/big/#Int.Div

Golang share big chunk of data between goroutines

I have a need to read structure fields set from another goroutine, afaik doing so directly even when knowing for sure there will be no concurrent access(write finished before read occurred, signaled via chan struct{}) may result in stale data
Will sending a pointer to the structure(created in the 1st goroutine, modified in the 2nd, read by the 3rd) resolve the possible staleness issue, considering I can guarantee no concurrent access?
I would like to avoid copying as structure is big and contains huge Bytes.Buffer filled in the 2nd goroutine, I need to read from the 3rd
There is an option for locking, but seems like an overkill considering I know that there will be no concurrent access
There are many answers to this, and it depends to your data structure and program logic.
see: How to lock/synchronize access to a variable in Go during concurrent goroutines?
and: How to use RWMutex in Golang?
1- using Stateful Goroutines and channels
2- using sync.Mutex
3- using sync/atomic
4- using WaitGroup
5- using program logic(Semaphore)
...
1: Stateful Goroutines and channels:
I simulated very similar sample(imagine you want to read from one SSD and write to another SSD with different speed):
In this sample code one goroutine (named write) does some job prepares data and fills the big struct, and another goroutine (named read) reads data from big struct then do some job, And the manger goroutine, guarantee no concurrent access to same data.
And communication between three goroutines done with channels. And in your case you can use pointers for channel data, or global struct like this sample.
output will be like this:
mean= 36.6920166015625 stdev= 6.068973186592054
I hope this helps you to get the idea.
Working sample code:
package main
import (
"fmt"
"math"
"math/rand"
"runtime"
"sync"
"time"
)
type BigStruct struct {
big []uint16
rpos int
wpos int
full bool
empty bool
stopped bool
}
func main() {
wg.Add(1)
go write()
go read()
go manage()
runtime.Gosched()
stopCh <- <-time.After(5 * time.Second)
wg.Wait()
mean := Mean(hist)
stdev := stdDev(hist, mean)
fmt.Println("mean=", mean, "stdev=", stdev)
}
const N = 1024 * 1024 * 1024
var wg sync.WaitGroup
var stopCh chan time.Time = make(chan time.Time)
var hist []int = make([]int, 65536)
var s *BigStruct = &BigStruct{empty: true,
big: make([]uint16, N), //2GB
}
var rc chan uint16 = make(chan uint16)
var wc chan uint16 = make(chan uint16)
func next(pos int) int {
pos++
if pos >= N {
pos = 0
}
return pos
}
func manage() {
dataReady := false
var data uint16
for {
if !dataReady && !s.empty {
dataReady = true
data = s.big[s.rpos]
s.rpos++
if s.rpos >= N {
s.rpos = 0
}
s.empty = s.rpos == s.wpos
s.full = next(s.wpos) == s.rpos
}
if dataReady {
select {
case rc <- data:
dataReady = false
default:
runtime.Gosched()
}
}
if !s.full {
select {
case d := <-wc:
s.big[s.wpos] = d
s.wpos++
if s.wpos >= N {
s.wpos = 0
}
s.empty = s.rpos == s.wpos
s.full = next(s.wpos) == s.rpos
default:
runtime.Gosched()
}
}
if s.stopped {
if s.empty {
wg.Done()
return
}
}
}
}
func read() {
for {
d := <-rc
hist[d]++
}
}
func write() {
for {
wc <- uint16(rand.Intn(65536))
select {
case <-stopCh:
s.stopped = true
return
default:
runtime.Gosched()
}
}
}
func stdDev(data []int, mean float64) float64 {
sum := 0.0
for _, d := range data {
sum += math.Pow(float64(d)-mean, 2)
}
variance := sum / float64(len(data)-1)
return math.Sqrt(variance)
}
func Mean(data []int) float64 {
sum := 0.0
for _, d := range data {
sum += float64(d)
}
return sum / float64(len(data))
}
5: another way(faster) for some use cases:
here another way to use shared data structure for read job/write job/ processing job which it was separated in first post, now here doing same 3 jobs without channels and without mutex.
working sample:
package main
import (
"fmt"
"math"
"math/rand"
"time"
)
type BigStruct struct {
big []uint16
rpos int
wpos int
full bool
empty bool
stopped bool
}
func manage() {
for {
if !s.empty {
hist[s.big[s.rpos]]++ //sample read job with any time len
nextPtr(&s.rpos)
}
if !s.full && !s.stopped {
s.big[s.wpos] = uint16(rand.Intn(65536)) //sample wrire job with any time len
nextPtr(&s.wpos)
}
if s.stopped {
if s.empty {
return
}
} else {
s.stopped = time.Since(t0) >= 5*time.Second
}
}
}
func main() {
t0 = time.Now()
manage()
mean := Mean(hist)
stdev := StdDev(hist, mean)
fmt.Println("mean=", mean, "stdev=", stdev)
d0 := time.Since(t0)
fmt.Println(d0) //5.8523347s
}
var t0 time.Time
const N = 100 * 1024 * 1024
var hist []int = make([]int, 65536)
var s *BigStruct = &BigStruct{empty: true,
big: make([]uint16, N), //2GB
}
func next(pos int) int {
pos++
if pos >= N {
pos = 0
}
return pos
}
func nextPtr(pos *int) {
*pos++
if *pos >= N {
*pos = 0
}
s.empty = s.rpos == s.wpos
s.full = next(s.wpos) == s.rpos
}
func StdDev(data []int, mean float64) float64 {
sum := 0.0
for _, d := range data {
sum += math.Pow(float64(d)-mean, 2)
}
variance := sum / float64(len(data)-1)
return math.Sqrt(variance)
}
func Mean(data []int) float64 {
sum := 0.0
for _, d := range data {
sum += float64(d)
}
return sum / float64(len(data))
}
To prevent concurrent modifications to a struct while retaining the ability to read, you'd typically embed a sync.RWMutex. This is no exemption. You can simply lock your struct for writes while it is in transit and unlock it at a point in time of your convenience.
package main
import (
"fmt"
"sync"
"time"
)
// Big simulates your big struct
type Big struct {
sync.RWMutex
value string
}
// pump uses a groutine to take the slice of pointers to Big,
// locks the underlying structs and sends the pointers to
// the locked instances of Big downstream
func pump(bigs []*Big) chan *Big {
// We make the channel buffered for this example
// for illustration purposes
c := make(chan *Big, 3)
go func() {
for _, big := range bigs {
// We lock the struct before sending it to the channel
// so it can not be changed via pointer while in transit
big.Lock()
c <- big
}
close(c)
}()
return c
}
// sink reads pointers to the locked instances of Big
// reads them and unlocks them
func sink(c chan *Big) {
for big := range c {
fmt.Println(big.value)
time.Sleep(1 * time.Second)
big.Unlock()
}
}
// modify tries to achieve locks to the instances and modify them
func modify(bigs []*Big) {
for _, big := range bigs {
big.Lock()
big.value = "modified"
big.Unlock()
}
}
func main() {
bigs := []*Big{&Big{value: "Foo"}, &Big{value: "Bar"}, &Big{value: "Baz"}}
c := pump(bigs)
// For the sake of this example, we wait until all entries are
// send into the channel and hence are locked
time.Sleep(1 * time.Second)
// Now we try to modify concurrently before we even start to read
// the struct of which the pointers were sent into the channel
go modify(bigs)
sink(c)
// We use sleep here to keep waiting for modify() to finish simple.
// Usually, you'd use a sync.waitGroup
time.Sleep(1 * time.Second)
for _, big := range bigs {
fmt.Println(big.value)
}
}
Run on playground

Resources