I am currently trying to send data over a channel to a goroutine, which will then process it further. My problem is that I want the channel to be able to work with any type. To do this I was looking into the newly introduced generics in Go 1.18.
My problem is that I need to tell the goroutine when I am starting it what type the channel will be, which is impossible to tell since it could hold any data.
This is what I got right now:
Thread:
func StartController[T any](sender chan Packet[T]) {
go runThread(sender)
}
func runThread[T any](sender chan Packet[T]) {
fmt.Println("in thread")
for true {
data := <- sender
fmt.Println(data)
}
}
And my test function is this:
func main() {
sender := make(chan Packet)
StartController(sender)
sender <- Packet[int]{
Msg: Message[int]{
Data: 1,
},
}
sender <- Packet[string]{
Msg: Message[string]{
Data: "asd",
},
}
for true {}
}
Types:
type Message[T any] struct {
Data T
}
type Packet[T any] struct {
Msg Message[T]
}
Right now this code doesn't compile because of
.\test.go:8:22: cannot use generic type Packet[T interface{}] without instantiation
Is there a way to do this properly?
I was looking into not using generics and just using interface{} as the type, but that would make the entire logic messy since it requires parsing (and might not even be possible since the data could be fairly complex (nested structs))
That's a mistaken way of using generics.
A parametrized type, like chan T must be instantiated with a concrete type parameter before you can use it. Given a defined chan type:
type GenericChan[T any] chan T
you would still need to instantiate it with a concrete type:
c := make(GenericChan[int])
which makes using type parameters a bit moot.
I don't know what your background is, but consider a language where generics have been a stable presence since a long time. E.g. Java. And consider the typical Java generic collector List<T>. What you usually do is instantiating that with a type:
var list = new ArrayList<String>();
What you are attempting to do here is to declare a channel which can take any type. In Java, what would be a list that can hold any type?
var list = new ArrayList<Object>();
And in Go that would be nothing else than
c := make(chan interface{})
You can look at it another way: how do you expect this generic chan to work on receive operations?
c := make(GenericChan) // wrong syntax: instantiating without type param
c <- "a string" // let's pretend you can send anything into it
// ...
foo := <-c
At this point what is foo? Is it a string? Or an int? You can send anything into it. That's why a generic chan like in your example can not work the way you intend. It must be chan interface{} and then you type-assert the received item like you do now without generics.
The point of generics is to write code that consumes arbitrary types, while maintaining type safety:
func receiveAny[T any](c chan T) T {
return <-c
}
which you can call with either a chan int or a chan string.
Related
With the following select statement I want to ensure that some none-blocking function is only executed one by one:
select {
case <-available:
default:
fmt.Println("busy")
return
}
go func() {
defer func() { available <- true }()
doSomethingOneByOne()
}()
Currently I'm using bool as a channel type and it works as expected.
What I don't like is that using bool suggests that it matters if the value is true or false. But actually it doesn't matter in this case. In my opinion this makes understanding the code a bit harder because it is misleading.
Is there a convention for which type to use when the value doesn't matter?
chan struct{} is a valid choice — struct{} is a valid type, but a value of this type contains no data and has zero size, and all struct{} values are indistinguishable, making it a unit type for Go. To construct a value of type struct{} to send on the channel, you can use the literal struct{}{}.
I'm writing a package to control a Canon DSLR using their EDSDK DLL from Go.
This is a personal project for a photo booth to use at our wedding at my partners request, which I'll be happy to post on GitHub when complete :).
Looking at the examples of using the SDK elsewhere, it isn't threadsafe and uses thread-local resources, so I'll need to make sure I'm calling it from a single thread during usage. While not ideal, it looks like Go provides a "runtime.LockOSThread" function for doing just that, although this does get called by the core DLL interop code itself, so I'll have to wait and find out if that interferes or not.
I want the rest of the application to be able to call the SDK using a higher level interface without worrying about the threading, so I need a way to pass function call requests to the locked thread/Goroutine to execute there, then pass the results back to the calling function outside of that Goroutine.
So far, I've come up with this working example of using very broad function definitions using []interface{} arrays and passing back and forward via channels. This would take a lot of mangling of input/output data on every call to do type assertions back out of the interface{} array, even if we know what we should expect for each function ahead of time, but it looks like it'll work.
Before I invest a lot of time doing it this way for possibly the worst way to do it - does anyone have any better options?
package edsdk
import (
"fmt"
"runtime"
)
type CanonSDK struct {
FChan chan functionCall
}
type functionCall struct {
Function func([]interface{}) []interface{}
Arguments []interface{}
Return chan []interface{}
}
func NewCanonSDK() (*CanonSDK, error) {
c := &CanonSDK {
FChan: make(chan functionCall),
}
go c.BackgroundThread(c.FChan)
return c, nil
}
func (c *CanonSDK) BackgroundThread(fcalls <-chan functionCall) {
runtime.LockOSThread()
for f := range fcalls {
f.Return <- f.Function(f.Arguments)
}
runtime.UnlockOSThread()
}
func (c *CanonSDK) TestCall() {
ret := make(chan []interface{})
f := functionCall {
Function: c.DoTestCall,
Arguments: []interface{}{},
Return: ret,
}
c.FChan <- f
results := <- ret
close(ret)
fmt.Printf("%#v", results)
}
func (c *CanonSDK) DoTestCall([]interface{}) []interface{} {
return []interface{}{ "Test", nil }
}
For similar embedded projects I've played with, I tend to create a single goroutine worker that listens on a channel to perform all the work over that USB device. And any results sent back out on another channel.
Talk to the device with channels only in Go in a one-way exchange. LIsten for responses from the other channel.
Since USB is serial and polling, I had to setup a dedicated channel with another goroutine that justs picks items off the channel when they were pushed into it from the worker goroutine that just looped.
I'm running into a slight architectural problem with Golang right now that's causing me to copy/paste a bit more code than I'd prefer. I feel like there must be a solution, so please let me know if this is perhaps possible:
When I pass things through an interface {}-typed function parameter, I start getting errors such as "expected struct or slice", etc. ... even though what I passed was previously a struct or a slice. I realize that I could manually convert these to another type after receiving them in that function, but then that become tedious in instances such as this:
local interface type *interface {} can only be decoded from remote
interface type; received concrete type
... In this case, the receiving function seems like it'd need to be hard-coded to convert all interface {} items back to their respective original types in order to work properly, because the receiving function needs to know the exact type in order to process the item correctly.
Is there a way to dynamically re-type Golang interface {} typed variables back to their original type? Something like this, How to I convert reflect.New's return value back to the original type ... maybe?
EDIT: To clarify, basically, I'm passing &out to a function and it needs to be its original type by the time it reaches another inner function call.
Example code:
// NOTE: This is sort of pseudo-Golang code, not meant to be compiled or taken too seriously.
func PrepareTwoDifferentThings(keyA string, keyB string) {
var somethingA TypeA;
var somethingB TypeB;
loadFromCache(keyA, &somethingA, nil);
loadFromCache(keyB, &somethingB, nil);
fmt.Printf("Somethings: %v, %v", somethingA, somethingB);
}
func loadFromCache(key string, isNew, out interface {}, saveNewData interface {}) {
if err := cache.load(key, &out); err!=nil { // NOTE: Current issue is that this expects "&out" to be `TypeA`/`TypeB` not "interface {}", but I don't want to copy and paste this whole function's worth of code or whatever.
panic("oh no!");
}
if (saveNewData!=nil) {
cache.save(key, saveNewData); // This doesn't seem to care if "saveNewData" is "interface {}" when saving, but later cache fetches above using the "load()" method to an "interface {}"-typed `&out` parameter throw an exception that the "interface {}" type on `&out` does not match the original when it was saved here (`TypeA`/`TypeB`).
}
}
To change the type of an interface into its rightful type, you can use type assertions:
package main
import r "reflect"
type A struct {
Name string
}
func main() {
// No pointer
aa := A{"name"}
var ii interface{} = aa
bb := ii.(A)
// main.A
// Pointer
a := &A{"name"}
var i interface{} = a
b := *i.(*A)
// main.A
c := i.(*A)
// *main.A
d := r.Indirect(r.ValueOf(i)).Interface().(A)
// main.A
}
Playground 1
When using type assertions, you have to know the underlying type of your interface. In Go, there is no way to use type assertion with a dynamic type. reflect.Type is not a type, it's an interface representing a type. So no, you can't use it this way.
If you have several type possibilities, the solution is the type switch:
package main
import "fmt"
type TypeA struct {
A string
}
type TypeB struct {
B string
}
func doSomethingA(t TypeA) {
fmt.Println(t.A)
}
func doSomethingB(t TypeB) {
fmt.Println(t.B)
}
func doSomething(t interface{}) {
switch t := t.(type) {
case TypeA:
doSomethingA(t)
case TypeB:
doSomethingB(t)
default:
panic("Unrecognized type")
}
}
func main() {
a := TypeA{"I am A"}
b := TypeB{"I am B"}
doSomething(a)
// I am A
doSomething(b)
// I am B
}
Playground 2
It turns out that using JSON instead of Gob for serialization avoids the error that I was encountering entirely. Other functions can handle passing into interfaces, etc.
My program has a pipeline structure, and I just implemented a caching filter that sends stuff directly to output if the already processed version of data is in the cache.
func Run(in chan downloader.ReadyDownload) chan CCFile {
out := make(chan CCFile)
processQueue := make(chan downloader.ReadyDownload)
go cache.BypassFilter(in, processQueue, out)
// writes the cached, already processed version to out if it exists
// otherwise redirects the input to processQueue
go process(processQueue, out)
return out
}
The problem is that my program has multiple places like this, and many kind of structs (like ReadyDownload and CCFile in this snippet) are being passed through the channels. They all implement this interface
type ProcessItem interface {
Source() string
Target() string
Key() string
}
so my BypassFilter() function signature looks like this:
func (c Cache) BypassFilter(in chan ProcessItem, process chan ProcessItem, bypass chan ProcessItem)
But this brings about the following error:
cannot use in (type chan downloader.ReadyDownload) as type chan utils.ProcessItem in function argument
Although ReadyDownload certainly implements ProcessItem. For example, this works without problems:
foo := downloader.ReadyDownload{}
var bar utils.ProcessItem
bar = foo
So, my (yet) very limited understanding of Go types and interfaces brings me to ask this question: Is it the fact that they are channels of something and something else, that makes the types incompatible? What should I do to make it work? Let's say that I've got a channel of ReadyDownloads. Is the only way to forward the data to a function that takes, let's say channel of interface{}s as a parameter, to create a new channel of interface{}s, pass that to the function and read stuff from the channel of ReadyDownloads and feed them to the other channel?
These two are different types:
processQueue chan ReadyDownload
process chan ProcessItem
You can put a ReadyDownloader value in a channel of type chan ProcessItem (if it implements the interface), but you cannot convert one channel type to another, in the same way that you cannot convert a []T slice into a []interface{} slice, another common confusion similar to this one.
What you need to do is make all the channels of type chan ProcessItem:
func Run(in chan ProcessItem) chan CCFile {
out := make(chan CCFile)
processQueue := make(chan ProcessItem)
go cache.BypassFilter(in, processQueue, out)
// writes the cached, already processed version to out if it exists
// otherwise redirects the input to processQueue
go process(processQueue, out)
return out
}
To read more about why this is (for slices, but the same applies for channels), you can read the following go-wiki page:
http://code.google.com/p/go-wiki/wiki/InterfaceSlice
Changing every channels to struct channels might work here, but in general, you might want to treat your struct type as interfaces for processing down the road. Fortunately, go gives us many solutions. Here is one.
Consider this very simple set up, where we want to use a Object struct type as several interfaces:
// Get the objects
func ParseFile(fileName string, co chan Object) {
for _, object := range DoStuff(fileName) {
co <- object
}
}
// Use some saving functionality that is defined elsewhere as:
func Archive(cs chan Saveable) {
for saveable := range cs {
saveable.Save()
}
}
type Saveable interface {
Save()
}
//Implement the interfaces...
func (*Object) Save() {
fmt.Println("Naa, I'm lazy")
}
// Or some throwing functionality?
func ThrowOnTheWall(ct chan Throwable) {
for throwable := range cs {
throwable.Throw()
}
}
//...
co := make(chan Object)
go ParseFile("file.xml", co)
Archive(co) // Will NOT work, co is of the wrong type.
Here, using everywhere some chan Object is not suitable, because you might want to throw on the wall something different than an object (e.g., type Defecation struct {...} that you would implement as a Throwable too.).
You could use a go routine to do the casting in the background:
func ObjectToSaveable(from chan Object) chan Saveable {
to := make(chan Saveable)
go func() {
for object := range from {
to <- &object
}
close(to)
}()
return to
}
And then use it to encapsulate the initial channel:
co := make(chan Object)
go ParseFile("file.xml", co)
Archive(ObjectToSaveable(co))
I like to program providing interfaces as restrictively as possible, both to avoid bad usage as to be explicit and self-documented.
So, I like to provide directional channels when they are supposed to be used unidirectionally by the user, but of course, internally I have a bidirectional channel copy.
Assigment for the following works:
var internal chan int
var external <-chan int
external = internal
But now I want to provide to the user a <-chan chan<- int type (in the return of a function), but the following won't work:
var internal chan chan int
var external <-chan chan<- int
external = internal // this fails
I have two questions:
Exactly why that doesn't work?
So, I can declare a variable of <-chan chan<- type, but... can't use such a type in any practical sense? (Because even though there're directional channels, they're AFAIK aways used in orchestration with bidirectional ones, and since assignment is not possible, they can't be used this way)
The reason why this does not work
The specification says this about channel assignability:
A [channel] value x is assignable to a variable of type T ("x is assignable to T") [when] x is a bidirectional channel value, T is a channel type, x's type V and T have identical element types, and at least one of V or T is not a named type.
This reflects exactly what you're experiencing:
chan (chan int) to <- chan (chan int) works
chan (chan int) to <- chan (chan<- int) does not
The reason for this is that the element types (the ones after the chan keyword)
are not equal.
I can declare it but not use it?
You can use it but not the way you want to. It is not possible to assign the variables
the way you do but by correcting the element types you can indeed use it:
var internal chan chan<- int
var external <-chan chan<- int
external = internal
If you only have your chan chan int type you need to copy your values (Examples on play):
var internal chan chan int
var internalCopy chan chan<- int
go func() { for e := range internal { internalCopy <- e } }()
var external <-chan chan<- int
external = internalCopy
This is a case somewhat akin to the problems of covariance and contravariance encountered in languages with user-level generics. You can also encounter it in Go when using its internal equivalents of generic types (they're called 'composite types'). For instance:
type A struct{}
// All `B`s are convertible to `A`s
type B A
Or even:
type A interface{}
// B embeds A
type B interface{ A }
var b B
// This works without problem
var a A = A(b)
But consider the following case:
var bs []B
var as []A = ([]A)(bs)
Here, the compilation fails with error cannot use bs (type []B) as type []A in assignment. Although any B can be converted to the equivalent A, this does not hold true for []B and []A (or chan B and chan A, or map[string]B and map[string]A, or func(a A) and func(b B) or any other generic type using As and Bs in their definition). Although types are convertible to one another, they are not the same, and the way these generics work in Go is, from the spec:
Each type T has an underlying type: If T is a predeclared type or a type literal, the corresponding underlying type is T itself. Otherwise, T's underlying type is the underlying type of the type to which T refers in its type declaration.
type T1 string
type T2 T1
type T3 []T1
type T4 T3
The underlying type of string, T1, and T2 is string. The underlying type of []T1, T3, and T4 is []T1.
Note here that the underlying type of []T1 is []T1, not []string. This means that the underlying type of []T2 will be []T2, not []string or []T1, which makes conversion between them impossible.
Basically, you are trying to do something like:
var internal chan Type1
var external <-chan Type2
external = internal
Which fails as Type1 and Type2 are two different types, as far as the type system is concerned.
Covariance and contravariance are very difficult problems, as the length of the wikipedia article or any time spent untangling layers of bounded generics in Java or C# will tell you. It is one of the reason generics are so difficult to implement and raise so many debate.
You can get the behaviour you want by going one level deeper on your aliases between read-only and read/write channels, exactly as you did with your internal/external channel on your first example:
package main
import "fmt"
// This has the correct signature you wanted
func ExportedFunction(c <-chan (chan<- int)) {
// Sends 1 to the channel it receives
(<-c)<- 1
}
func main() {
// Note that this is a READ/WRITE channel of WRITE-ONLY channels
// so that the types are correct
internal := make(chan (chan<- int))
var external <-chan (chan<- int)
// This works because the type of elements in the channel is the same
external = internal
// This channel is internal, so it is READ/WRITE
internal2 := make(chan int)
// This is typically called externally
go ExportedFunction(external)
fmt.Println("Sending channel...")
// The type of internal makes the receiving end of internal/external
// see a WRITE-ONLY channel
internal <- internal2
fmt.Println("We received:")
fmt.Println(<-internal2)
}
The same thing on the playground.
Basically the same thing as your first example, except that you have to go one level deeper in the 'read/write' vs 'read(or write) only' aliases.
I doubt that this is possible as you would have to convert the type of the outer and inner channel at the same time. One at a time works.
I like to think of these send/receive-only channels a a nice way to limit stuff you can do in a function: You have a var c chan int and you pass it to a func f(ro <-chan int) and now in f you are save from sending to ro. No explicit type conversion needed. Same for returning c: You may just return c in func g() <-chan int. But on any case you must agree what type to transfer via your your channel: This can be an other bidirectional channel, a send-only or a receive-only channel. Once you got that sent channel out you may convert the channel.
chan chan int is a channel of integer channels. You can convert this to a receive-only channel of integer channels <-chan chan int or a send-only channel of integer channels chan<- chan int. Anyway: What you send or receive is always a integer channel chan int. Each such can be converted to send/receive-only before/after sending/receiving but you cannot convert chan chan int to chan chan<-int as this is like converting chan chan int to chan chan int32 or even chan string.