Multiple senders to single channel in Golang - go

The concept seems simple to explain, but a tad harder to implement ("correctly").
The tl;dr is I want to run multiple functions that push output into a single channel.
As a sample working test (with multiple channels), to elaborate on my question https://play.golang.org/p/1ztCvPFLXKv
package main
import (
"fmt"
"time"
)
type intTest struct {
ID int
Number int
}
func modify1(channelID string, res chan []intTest) {
s := []intTest{}
for i := 0; i < 10; i++ {
fmt.Printf("Adding inside: %s\n", channelID)
s = append(s, intTest{i, 0})
time.Sleep(100 * time.Millisecond)
}
res <- s
}
func modify2(channelID string, res chan []intTest) {
s := []intTest{}
for i := 10; i < 20; i++ {
fmt.Printf("Adding inside: %s\n", channelID)
s = append(s, intTest{i, 0})
time.Sleep(200 * time.Millisecond)
}
res <- s
}
func modify3(channelID string, res chan []intTest) {
s := []intTest{}
for i := 20; i < 30; i++ {
fmt.Printf("Adding inside: %s\n", channelID)
s = append(s, intTest{i, 0})
time.Sleep(300 * time.Millisecond)
}
res <- s
}
func main() {
channelA := make(chan []intTest)
channelB := make(chan []intTest)
channelC := make(chan []intTest)
go modify1("A", channelA)
go modify2("B", channelB)
go modify3("C", channelC)
b := append(<-channelA, <-channelB...)
b = append(b, <-channelC...)
fmt.Println(b)
}
Output:
Adding inside: C
Adding inside: A
Adding inside: B
..snip..
Adding inside: C
Adding inside: C
Adding inside: C
[{0 0} {1 0} {2 0} {3 0} {4 0} {5 0} {6 0} {7 0} {8 0} {9 0} {10 0} {11 0} {12 0} {13 0} {14 0} {15 0} {16 0} {17 0} {18 0} {19 0} {20 0} {21 0} {22 0} {23 0} {24 0} {25 0} {26 0} {27 0} {28 0} {29 0}]
However, I would like to achieve something like this: https://play.golang.org/p/qvC88LwkanY
Output:
Adding inside: C
Adding inside: A
Adding inside: B
..snip
Adding inside: B
Adding inside: A
Adding inside: C
[{0 0} {1 0} {2 0} {3 0} {4 0} {5 0} {6 0} {7 0} {8 0} {9 0}]
but as shown the functions modify2 & modify3 visually seems like they never get added.
Is this possible, or is the top sample more feasible?

I would like to achieve something like this
channelA := make(chan []intTest)
go modify1("A", channelA)
go modify2("B", channelA)
go modify3("C", channelA)
Is this possible, or is the top sample more feasible?
Yes: you can use a single channel across multiple goroutines -- that's what channels are designed for.
but as shown the functions modify2 & modify3 visually seems like they never get added.
The problem you have there is that you only called the receive operator once:
b := append(<-channelA)
fmt.Println(b)
The other two goroutines were either blocked waiting to send their result, or still building their result, when your main goroutine exited.
If you change your main() function to this, you can see that all three workers will send their results over the channel if another routine is ready to receive (because you used an unbuffered channel, sends will block until a receiver is ready):
func main() {
ch := make(chan []intTest)
go modify1("A", ch)
go modify2("B", ch)
go modify3("C", ch)
fmt.Println(<-ch)
fmt.Println(<-ch)
fmt.Println(<-ch)
}
Which outputs:
Adding inside: C
Adding inside: A
Adding inside: B
Adding inside: A
Adding inside: A
Adding inside: B
Adding inside: C
Adding inside: A
Adding inside: B
Adding inside: A
Adding inside: A
Adding inside: B
Adding inside: C
Adding inside: A
Adding inside: A
Adding inside: B
Adding inside: A
Adding inside: C
Adding inside: A
Adding inside: B
[{0 0} {1 0} {2 0} {3 0} {4 0} {5 0} {6 0} {7 0} {8 0} {9 0}]
Adding inside: C
Adding inside: B
Adding inside: B
Adding inside: C
Adding inside: B
Adding inside: C
Adding inside: B
[{10 0} {11 0} {12 0} {13 0} {14 0} {15 0} {16 0} {17 0} {18 0} {19 0}]
Adding inside: C
Adding inside: C
Adding inside: C
[{20 0} {21 0} {22 0} {23 0} {24 0} {25 0} {26 0} {27 0} {28 0} {29 0}]
You can then change that code to append received elements to a single list prior to output, or otherwise format the received elements in whatever way you like.
I thought it's almost like popping onto a stack and then pulling the whole stack
With an initialized, not-closed, buffered channel, a send statement puts one element onto a queue, and a receive operation pops one element off the queue and returns it. It's first in first out (FIFO) order, so it's a queue rather than a stack. In the examples above the channel is unbuffered, so a send has to wait for a goroutine ready to receive, and a receive has to wait for a goroutine ready to send. Main is a goroutine as well.

Related

How to sort a slice in go without mutating the original slice

In Golang, I am trying to sort a slice without mutating the original value
func main() {
originalArray := []int{4, 2, 1, 1, 2}
newArray := originalArray
sort.Ints(newArray)
fmt.Printf("%v", originalArray) // prints [1 1 2 2 4]
}
How can I sort a slice in golang without mutating the original value?
You need to make a copy of the original slice.
Use:
newArray := make([]int, len(originalArray))
copy(newArray, originalArray)
or:
newArray := append([]int{}, originalArray...)

Change golang slice in another function

I have a slice, if i remove one element from it directly in a main function the length of slice would be cut by one. But do the remove in another function and called it in main, the length of the slice is still keep origin. Who can explain it for me? Thanks!
package main
import "fmt"
func main() {
a := []int{1, 2, 3, 4}
i := 0
//copy(a[i:], a[i+1:])
//a[len(a)-1] = 0
//a = a[:len(a)-1]
//fmt.Println(a) //outputs: [2 3 4], this is correct
f(a, i)
fmt.Println(a) //outputs: [2 3 4 0], this is wrong!
}
func f(a []int, i int) {
copy(a[i:], a[i+1:])
a[len(a)-1] = 0
a = a[:len(a)-1]
fmt.Println(a) //outputs: [2 3 4], here still correct
}
Go Playground Link
The slice is passed by value, so changing it in your function f won't change it in function main. You can pass by pointer, like this:
package main
import "fmt"
func main() {
a := []int{1, 2, 3, 4}
i := 0
f(&a, i)
fmt.Println(a) //outputs: [2 3 4], correct
}
func f(a *[]int, i int) {
b := *a
copy(b[i:], b[i+1:])
// The following line seems pointless, but ok...
b[len(b)-1] = 0
b = b[:len(b)-1]
fmt.Println(b) //outputs: [2 3 4], here still correct
*a = b
}
Go Playground
As suggested by #zerkms in the comments, you could also return the new slice, avoiding the use of pointers:
package main
import "fmt"
func main() {
a := []int{1, 2, 3, 4}
i := 0
a = f(a, i)
fmt.Println(a)
}
func f(a []int, i int) []int {
copy(a[i:], a[i+1:])
// The following line seems pointless, but ok...
a[len(a)-1] = 0
a = a[:len(a)-1]
fmt.Println(a) //outputs: [2 3 4], here still correct
return a
}
Not providing new solution, just trying to explain why your program is behaving the way you asked:
Let us try to understand first how the built in function ‘copy’ works

Ref: [https://golang.org/pkg/builtin/#copy]
func copy(dst, src []Type) int
The copy built-in function copies elements from a source slice into a destination slice. (As a special case, it also will copy bytes from a string to a slice of bytes.) The source and destination may overlap. Copy returns the number of elements copied, which will be the minimum of len(src) and len(dst).
Two things:
1. 
First comment the line : //a[len(a)-1] = 0
Second: As you are using the same array i.e as source and destination you are getting [2,3,4,4] as output as the destination array is {1,2,3,4} which got overwritten to {2,3,4,4(which is already present)}

you can try with different array’s to make it more clear to you

Two goroutine in the same Channel - how are they executed?

When I take the tour of Golang from golang.org, there's one code snippet I don't understand:
func sum(a []int, c chan int, order int) {
sum := 0
for _, v := range a {
sum += v
}
fmt.Println(order, a)
c <- sum // 将和送入 c
}
func main() {
a := []int{7, 2, 8, -9, 4, 0}
c := make(chan int)
a1, a2 := a[:len(a)/2], a[len(a)/2:]
go sum(a1, c, 1)
x := <-c
go sum(a2, c, 2)
y := <-c
//x := <-c
//y := <-c
// x, y := <-c, <-c // 从 c 中获取
fmt.Println(x, y, x+y)
}
This is the output I expected:
1 [7 2 8]
2 [-9 4 0]
17 -5 12
and when I changed the code:
func main() {
a := []int{7, 2, 8, -9, 4, 0}
c := make(chan int)
a1, a2 := a[:len(a)/2], a[len(a)/2:]
go sum(a1, c, 1)
//x := <-c
go sum(a2, c, 2)
//y := <-c
x := <-c
y := <-c
// x, y := <-c, <-c // 从 c 中获取
fmt.Println(x, y, x+y)
}
Why is the output like this:
2 [-9 4 0]
1 [7 2 8]
-5 17 12
the thing is u've missed the concept of concurrency
there is no guarantee in executing functions in exact order of calling them with go
the reason first code snippet works orderly is the x := <-c part
this line force your app to wait until c channel is filled with data so second goroutine wont be called
let me help you with that.
in your second scenario:
2 [-9 4 0]
1 [7 2 8]
-5 17 12
As you can see second goroutine executes faster than first one and in result we have x = result of second goroutine (-5).
When you do run function/method with go keyword it runs in concurrency which means main function doesn't wait it execution (if do not use sync.WaitGroup{} or select{}).
you can find more details from docs:
https://golang.org/pkg/sync/#WaitGroup

Go: read bytes into array

I'm trying to append bytes to an array in the following way:
Go
func readBytes() {
b := make([]byte, 1)
a := [][]byte{}
for i := 0, i < 4, i++ {
conn.Read(b)
a = append(a, b)
fmt.Println(b)
}
fmt.Println(a)
}
Result from fmt.Println(b):
[2]
[5]
[5]
[3]
Result from fmt.Println(a):
[[3], [3], [3], [3]]
Why does it only print out the last byte sent?? Am I missing something?
b is a slice - and you're therefore updating the same underlying array each time you pass it to conn.Read. You can look at this Golang blog post to understand how this works.
Once you call fmt.Println(a) .. each slice is looking at the same underlying array.
You could fix this by instantiating the buffer b in the loop, or using an array instead of a slice.
Here's a working example that re-allocates the b slice inside the loop: http://play.golang.org/p/cN1BE8WSFE
It is essentially (with an int slice):
for i := 0; i < 5; i++ {
b = []int{i, i + 1, i + 2, i + 3, i + 4}
a = append(a, b)
}

What is the point of copying a slice?

What is the point of this snippet of code:
t := make([]byte, len(s), (cap(s)+1)*2) // +1 in case cap(s) == 0
for i := range s {
t[i] = s[i]
}
s = t
It's from this page: http://blog.golang.org/go-slices-usage-and-internals, and is supposed to grow a slice. However, above that code snippet is a diagram which depicts a slice as a struct with a pointer, a length, and a capacity. Why does each individual entry have to be copied over instead of something like:
t := make([]byte, len(s), (cap(s)+1)*2) // +1 in case cap(s) == 0
t = s[:]
s = t
And if the problem is that the capacity of t is changed to be the same as s, why isn't there another way of setting the pointers to be the same. Or does a slice have a pointer to every single element in the array within its bounds?
Edit: I read a little further and got to this snippet of code:
func CopyDigits(filename string) []byte {
b, _ := ioutil.ReadFile(filename)
b = digitRegexp.Find(b)
c := make([]byte, len(b))
copy(c, b)
return c
}
Its purpose is to stop referencing the file after c is returned by using copy. Does this imply that copy copies the underlying array as well as the slice?
To construct a new, higher capacity underlying array with the same length and values as the old underlying array. The old underlying array will be reclaimed by the garbage collector. For example,
package main
import "fmt"
func main() {
s := []byte{0, 1, 2, 3, 4}[:3]
fmt.Printf("s: %p %d %v %d %v\n", &s[0], len(s), s, cap(s), s[:cap(s)])
t := make([]byte, len(s), (cap(s)+1)*2) // +1 in case cap(s) == 0
fmt.Printf("t: %p %d %v %d %v\n", &t[0], len(t), t, cap(t), t[:cap(t)])
for i := range s {
t[i] = s[i]
}
s = t
fmt.Printf("s: %p %d %v %d %v\n", &s[0], len(s), s, cap(s), s[:cap(s)])
fmt.Printf("t: %p %d %v %d %v\n", &t[0], len(t), t, cap(t), t[:cap(t)])
}
Output:
s: 0x10500168 3 [0 1 2] 5 [0 1 2 3 4]
t: 0x1052e130 3 [0 0 0] 12 [0 0 0 0 0 0 0 0 0 0 0 0]
s: 0x1052e130 3 [0 1 2] 12 [0 1 2 0 0 0 0 0 0 0 0 0]
t: 0x1052e130 3 [0 1 2] 12 [0 1 2 0 0 0 0 0 0 0 0 0]
The Go Programming Language Specification
Appending to and copying slices
The function copy copies slice elements from a source src to a
destination dst and returns the number of elements copied. Both
arguments must have identical element type T and must be assignable to
a slice of type []T. The number of elements copied is the minimum of
len(src) and len(dst).
Examples:
var a = [...]int{0, 1, 2, 3, 4, 5, 6, 7}
var s = make([]int, 6)
var b = make([]byte, 5)
n1 := copy(s, a[0:]) // n1 == 6, s == []int{0, 1, 2, 3, 4, 5}
n2 := copy(s, s[2:]) // n2 == 4, s == []int{2, 3, 4, 5, 4, 5}
n3 := copy(b, "Hello, World!") // n3 == 5, b == []byte("Hello")
If we return a reference to b, we pin the whole underlying array for b. Since b refers to a file, that could easily be megabytes or gigabytes. By returning a new underlying array c, which is the exact size of the number, a few bytes, there will no longer be a reference to the large underlying array for b and it will be reclaimed by the garbage collector. The copy built-in function copies values from b to c. For example,
package main
import "fmt"
func Copy() []byte {
b := []byte{0, 1, 2, 3, 4, 5, 6, 7}
fmt.Printf("b: %p %d %v %d %v\n", &b[0], len(b), b, cap(b), b[:cap(b)])
b = b[:2]
fmt.Printf("b: %p %d %v %d %v\n", &b[0], len(b), b, cap(b), b[:cap(b)])
c := make([]byte, len(b))
copy(c, b)
fmt.Printf("c: %p %d %v %d %v\n", &c[0], len(c), c, cap(c), c[:cap(c)])
return c
}
func main() {
d := Copy()
fmt.Printf("d: %p %d %v %d %v\n", &d[0], len(d), d, cap(d), d[:cap(d)])
}
Output:
b: 0x10500168 8 [0 1 2 3 4 5 6 7] 8 [0 1 2 3 4 5 6 7]
b: 0x10500168 2 [0 1] 8 [0 1 2 3 4 5 6 7]
c: 0x10500178 2 [0 1] 2 [0 1]
d: 0x10500178 2 [0 1] 2 [0 1]

Resources