I like to program providing interfaces as restrictively as possible, both to avoid bad usage as to be explicit and self-documented.
So, I like to provide directional channels when they are supposed to be used unidirectionally by the user, but of course, internally I have a bidirectional channel copy.
Assigment for the following works:
var internal chan int
var external <-chan int
external = internal
But now I want to provide to the user a <-chan chan<- int type (in the return of a function), but the following won't work:
var internal chan chan int
var external <-chan chan<- int
external = internal // this fails
I have two questions:
Exactly why that doesn't work?
So, I can declare a variable of <-chan chan<- type, but... can't use such a type in any practical sense? (Because even though there're directional channels, they're AFAIK aways used in orchestration with bidirectional ones, and since assignment is not possible, they can't be used this way)
The reason why this does not work
The specification says this about channel assignability:
A [channel] value x is assignable to a variable of type T ("x is assignable to T") [when] x is a bidirectional channel value, T is a channel type, x's type V and T have identical element types, and at least one of V or T is not a named type.
This reflects exactly what you're experiencing:
chan (chan int) to <- chan (chan int) works
chan (chan int) to <- chan (chan<- int) does not
The reason for this is that the element types (the ones after the chan keyword)
are not equal.
I can declare it but not use it?
You can use it but not the way you want to. It is not possible to assign the variables
the way you do but by correcting the element types you can indeed use it:
var internal chan chan<- int
var external <-chan chan<- int
external = internal
If you only have your chan chan int type you need to copy your values (Examples on play):
var internal chan chan int
var internalCopy chan chan<- int
go func() { for e := range internal { internalCopy <- e } }()
var external <-chan chan<- int
external = internalCopy
This is a case somewhat akin to the problems of covariance and contravariance encountered in languages with user-level generics. You can also encounter it in Go when using its internal equivalents of generic types (they're called 'composite types'). For instance:
type A struct{}
// All `B`s are convertible to `A`s
type B A
Or even:
type A interface{}
// B embeds A
type B interface{ A }
var b B
// This works without problem
var a A = A(b)
But consider the following case:
var bs []B
var as []A = ([]A)(bs)
Here, the compilation fails with error cannot use bs (type []B) as type []A in assignment. Although any B can be converted to the equivalent A, this does not hold true for []B and []A (or chan B and chan A, or map[string]B and map[string]A, or func(a A) and func(b B) or any other generic type using As and Bs in their definition). Although types are convertible to one another, they are not the same, and the way these generics work in Go is, from the spec:
Each type T has an underlying type: If T is a predeclared type or a type literal, the corresponding underlying type is T itself. Otherwise, T's underlying type is the underlying type of the type to which T refers in its type declaration.
type T1 string
type T2 T1
type T3 []T1
type T4 T3
The underlying type of string, T1, and T2 is string. The underlying type of []T1, T3, and T4 is []T1.
Note here that the underlying type of []T1 is []T1, not []string. This means that the underlying type of []T2 will be []T2, not []string or []T1, which makes conversion between them impossible.
Basically, you are trying to do something like:
var internal chan Type1
var external <-chan Type2
external = internal
Which fails as Type1 and Type2 are two different types, as far as the type system is concerned.
Covariance and contravariance are very difficult problems, as the length of the wikipedia article or any time spent untangling layers of bounded generics in Java or C# will tell you. It is one of the reason generics are so difficult to implement and raise so many debate.
You can get the behaviour you want by going one level deeper on your aliases between read-only and read/write channels, exactly as you did with your internal/external channel on your first example:
package main
import "fmt"
// This has the correct signature you wanted
func ExportedFunction(c <-chan (chan<- int)) {
// Sends 1 to the channel it receives
(<-c)<- 1
}
func main() {
// Note that this is a READ/WRITE channel of WRITE-ONLY channels
// so that the types are correct
internal := make(chan (chan<- int))
var external <-chan (chan<- int)
// This works because the type of elements in the channel is the same
external = internal
// This channel is internal, so it is READ/WRITE
internal2 := make(chan int)
// This is typically called externally
go ExportedFunction(external)
fmt.Println("Sending channel...")
// The type of internal makes the receiving end of internal/external
// see a WRITE-ONLY channel
internal <- internal2
fmt.Println("We received:")
fmt.Println(<-internal2)
}
The same thing on the playground.
Basically the same thing as your first example, except that you have to go one level deeper in the 'read/write' vs 'read(or write) only' aliases.
I doubt that this is possible as you would have to convert the type of the outer and inner channel at the same time. One at a time works.
I like to think of these send/receive-only channels a a nice way to limit stuff you can do in a function: You have a var c chan int and you pass it to a func f(ro <-chan int) and now in f you are save from sending to ro. No explicit type conversion needed. Same for returning c: You may just return c in func g() <-chan int. But on any case you must agree what type to transfer via your your channel: This can be an other bidirectional channel, a send-only or a receive-only channel. Once you got that sent channel out you may convert the channel.
chan chan int is a channel of integer channels. You can convert this to a receive-only channel of integer channels <-chan chan int or a send-only channel of integer channels chan<- chan int. Anyway: What you send or receive is always a integer channel chan int. Each such can be converted to send/receive-only before/after sending/receiving but you cannot convert chan chan int to chan chan<-int as this is like converting chan chan int to chan chan int32 or even chan string.
Related
I am currently trying to send data over a channel to a goroutine, which will then process it further. My problem is that I want the channel to be able to work with any type. To do this I was looking into the newly introduced generics in Go 1.18.
My problem is that I need to tell the goroutine when I am starting it what type the channel will be, which is impossible to tell since it could hold any data.
This is what I got right now:
Thread:
func StartController[T any](sender chan Packet[T]) {
go runThread(sender)
}
func runThread[T any](sender chan Packet[T]) {
fmt.Println("in thread")
for true {
data := <- sender
fmt.Println(data)
}
}
And my test function is this:
func main() {
sender := make(chan Packet)
StartController(sender)
sender <- Packet[int]{
Msg: Message[int]{
Data: 1,
},
}
sender <- Packet[string]{
Msg: Message[string]{
Data: "asd",
},
}
for true {}
}
Types:
type Message[T any] struct {
Data T
}
type Packet[T any] struct {
Msg Message[T]
}
Right now this code doesn't compile because of
.\test.go:8:22: cannot use generic type Packet[T interface{}] without instantiation
Is there a way to do this properly?
I was looking into not using generics and just using interface{} as the type, but that would make the entire logic messy since it requires parsing (and might not even be possible since the data could be fairly complex (nested structs))
That's a mistaken way of using generics.
A parametrized type, like chan T must be instantiated with a concrete type parameter before you can use it. Given a defined chan type:
type GenericChan[T any] chan T
you would still need to instantiate it with a concrete type:
c := make(GenericChan[int])
which makes using type parameters a bit moot.
I don't know what your background is, but consider a language where generics have been a stable presence since a long time. E.g. Java. And consider the typical Java generic collector List<T>. What you usually do is instantiating that with a type:
var list = new ArrayList<String>();
What you are attempting to do here is to declare a channel which can take any type. In Java, what would be a list that can hold any type?
var list = new ArrayList<Object>();
And in Go that would be nothing else than
c := make(chan interface{})
You can look at it another way: how do you expect this generic chan to work on receive operations?
c := make(GenericChan) // wrong syntax: instantiating without type param
c <- "a string" // let's pretend you can send anything into it
// ...
foo := <-c
At this point what is foo? Is it a string? Or an int? You can send anything into it. That's why a generic chan like in your example can not work the way you intend. It must be chan interface{} and then you type-assert the received item like you do now without generics.
The point of generics is to write code that consumes arbitrary types, while maintaining type safety:
func receiveAny[T any](c chan T) T {
return <-c
}
which you can call with either a chan int or a chan string.
I'm digging around the use of channels in Go, and in this example from Tour of Go, we have this line:
func sum(s []int, c chan int) {
I'm familiar with the syntax: variableName type in Go.
But what does this mean? c chan int
Is this a channel type, or an int type, or a chan int type? And what's with the weird syntax?
I cannot search for the answer, if this is a duplicate, kindly give me a link to the original post in the comment, and I'll delete the question.
I'm familiar with the syntax: variableName type in Go. Is this a channel type, or an int type, or a chan int type? And what's with the weird syntax?
chan int is the type. It's just as "weird" as []int.
I want to organize communication between two functions via channels. The callee can only send data to the channel while the caller will wait for it in the select. And I want to show this restriction in the callee signature. Another thing that I want is working with type aliases for channels. For example, instead of having chan string I want to work with MsgChan defined as type MsgChan chan string. And I faced with the problem - the code below won't compile if uncomment line test1(make(Ch)):
package main
import "fmt"
type Ch chan int
type ChIn chan<- int
func test1(in ChIn) {
fmt.Println(in)
}
func test2(in chan<- int) {
fmt.Println(in)
}
func main() {
//test1(make(Ch))
test1(make(chan int))
test2(make(Ch))
test2(make(ChIn))
}
I don't understand why I can't use such approach?
test1() has one parameter of type ChIn. This is a named type. You want to pass a value of type Ch which is a bidirectional channel type, and is also a named type.
So in order for this to compile, a value of Ch should be assignable to type ChIn. This is not allowed by the language specification.
Quoting Assignability (highlighted the one that applies here):
A value x is assignable to a variable of type T ("x is assignable to T") in any of these cases:
x's type is identical to T.
x's type V and T have identical underlying types and at least one of V or T is not a named type.
T is an interface type and x implements T.
x is a bidirectional channel value, T is a channel type, x's type V and T have identical element types, and at least one of V or T is not a named type.
x is the predeclared identifier nil and T is a pointer, function, slice, map, channel, or interface type.
x is an untyped constant representable by a value of type T.
You can make it work if you try to pass a value of unnamed type but having the same underlying type, which can be achieved by using a type conversion, e.g.:
test1((chan int)(make(Ch)))
But the above conversion would defeat the purpose of having the named Ch type (as you have to repeat its type literal in order to convert it to an unnamed type just so you can pass it to test1()).
What you should do is don't hide that the type is a channel (don't include chan in the type literal of the type declaration), only create a new type for the element type of the channel, e.g.:
type Msg int
func test(in chan<- Msg) {
fmt.Println(in)
}
func main() {
test(make(chan Msg))
test(make(chan<- Msg))
ch := make(chan Msg)
chIn := (chan<- Msg)(ch)
test(chIn)
}
Try it on the Go Playground.
My program has a pipeline structure, and I just implemented a caching filter that sends stuff directly to output if the already processed version of data is in the cache.
func Run(in chan downloader.ReadyDownload) chan CCFile {
out := make(chan CCFile)
processQueue := make(chan downloader.ReadyDownload)
go cache.BypassFilter(in, processQueue, out)
// writes the cached, already processed version to out if it exists
// otherwise redirects the input to processQueue
go process(processQueue, out)
return out
}
The problem is that my program has multiple places like this, and many kind of structs (like ReadyDownload and CCFile in this snippet) are being passed through the channels. They all implement this interface
type ProcessItem interface {
Source() string
Target() string
Key() string
}
so my BypassFilter() function signature looks like this:
func (c Cache) BypassFilter(in chan ProcessItem, process chan ProcessItem, bypass chan ProcessItem)
But this brings about the following error:
cannot use in (type chan downloader.ReadyDownload) as type chan utils.ProcessItem in function argument
Although ReadyDownload certainly implements ProcessItem. For example, this works without problems:
foo := downloader.ReadyDownload{}
var bar utils.ProcessItem
bar = foo
So, my (yet) very limited understanding of Go types and interfaces brings me to ask this question: Is it the fact that they are channels of something and something else, that makes the types incompatible? What should I do to make it work? Let's say that I've got a channel of ReadyDownloads. Is the only way to forward the data to a function that takes, let's say channel of interface{}s as a parameter, to create a new channel of interface{}s, pass that to the function and read stuff from the channel of ReadyDownloads and feed them to the other channel?
These two are different types:
processQueue chan ReadyDownload
process chan ProcessItem
You can put a ReadyDownloader value in a channel of type chan ProcessItem (if it implements the interface), but you cannot convert one channel type to another, in the same way that you cannot convert a []T slice into a []interface{} slice, another common confusion similar to this one.
What you need to do is make all the channels of type chan ProcessItem:
func Run(in chan ProcessItem) chan CCFile {
out := make(chan CCFile)
processQueue := make(chan ProcessItem)
go cache.BypassFilter(in, processQueue, out)
// writes the cached, already processed version to out if it exists
// otherwise redirects the input to processQueue
go process(processQueue, out)
return out
}
To read more about why this is (for slices, but the same applies for channels), you can read the following go-wiki page:
http://code.google.com/p/go-wiki/wiki/InterfaceSlice
Changing every channels to struct channels might work here, but in general, you might want to treat your struct type as interfaces for processing down the road. Fortunately, go gives us many solutions. Here is one.
Consider this very simple set up, where we want to use a Object struct type as several interfaces:
// Get the objects
func ParseFile(fileName string, co chan Object) {
for _, object := range DoStuff(fileName) {
co <- object
}
}
// Use some saving functionality that is defined elsewhere as:
func Archive(cs chan Saveable) {
for saveable := range cs {
saveable.Save()
}
}
type Saveable interface {
Save()
}
//Implement the interfaces...
func (*Object) Save() {
fmt.Println("Naa, I'm lazy")
}
// Or some throwing functionality?
func ThrowOnTheWall(ct chan Throwable) {
for throwable := range cs {
throwable.Throw()
}
}
//...
co := make(chan Object)
go ParseFile("file.xml", co)
Archive(co) // Will NOT work, co is of the wrong type.
Here, using everywhere some chan Object is not suitable, because you might want to throw on the wall something different than an object (e.g., type Defecation struct {...} that you would implement as a Throwable too.).
You could use a go routine to do the casting in the background:
func ObjectToSaveable(from chan Object) chan Saveable {
to := make(chan Saveable)
go func() {
for object := range from {
to <- &object
}
close(to)
}()
return to
}
And then use it to encapsulate the initial channel:
co := make(chan Object)
go ParseFile("file.xml", co)
Archive(ObjectToSaveable(co))
I am writing a DNS protocol parser in golang, the idea is to use a map like this
var tidMap map[uint16] (chan []byte)
So for the tidMap map, key is the tid (transaction ID), value is a byte array channel.
The idea is that a goroutine will try get value from the channel, another goroutine will try read bytes by listening every imcoming packet, and once found transaction ID, will set response data to the tidMap, so the former goroutine will continue handle the response.
One problem with the design is that I need the make sure the channel has buffer length of 1, so extra values can be pushed into channel without blocking.
So how can I specify channel buffer length in tidMap declaration?
var tidMap map[int] make(chan int, 1)
You can't use make() there.
The length of the channel buffer doesn't convey type, so you will have to add logic to test if the map entry exists, if it doesn't:
tidMap[0] = make(chan int, 1)
The short answer: you can't. When you make a map, you define the data types of its keys and values, and the capacity of a channel is not part of its type.
The longer answer is: create an abstract data type that hides this implementation detail. Something like this:
type ChannelMap struct {
tidMap map[int](chan []byte)
}
func NewChannelMap() *ChannelMap { ... }
func (c *ChannelMap) Put(tid int) (chan int) {
res := make(chan int, 1)
c.tidMap[tid] = res
return res
}
func (c *ChannelMap) Get(tid int) (chan int) {
return c.tidMap[tid]
}
And just to be sure: giving the channel a capacity of 1 does not ensure that senders will never block; if your channel consumers are too slow, producers can fill the channel up to its capacity and will then block until the channel has room again.