So I'm super new to Golang and seeing as the big buzz around the language seems to be concurrency I figured a good way to get my toes wet would be to write a generalized map function. In psudo code:
map(F funtion,A array){
return([F(k) for k in A])
}
And obviously I want the calculations for each F(k) occur concurrently. For organization I have a main file with my implementation and a supporting file Mr with my required definitions.
.
├── main.go
└── Mr
└── Mr.go
Main is a simple test implementation which should convert an array of strings to an array of ints where each member of the result is the length of the corresponding string in the input array.
package main
import(
"fmt"
"./Mr"
)
func exfunc(i int, c chan int){
c<-i
}
func main(){
data := make(map[int]string)
data[1]="these"
data[2]="are"
data[3]="some"
data[4]="words"
data[5]="and a few more..."
f := func(w string)int{
return(len(w))
}
testMr := Mr.Map(f,data) // this is line 22 (matters later)
fmt.Println(testMr)
}
Which works great with my Mr.Map function given that I specify all the types explicitly.
package Mr
type result struct{
key,value int
}
func wrapper(f func(string) int,k int,v string, c chan result){
c <- result{k,f(v)}
}
func Map(f func(string) int,m map[int]string) map[int]int{
c := make(chan result)
ret := make(map[int]int)
n := 0
for k := range m{
go wrapper(f,k,m[k],c)
n++
}
for ;n>0; {
r := <-c
ret[r.key]=r.value
n--
}
return(ret)
}
However I was hoping that I would be able to generalize this mapping with the empty interface.
package Mr
type T interface{}
type result struct{
key,value T
}
func wrapper(f func(T) T,k T,v T, c chan result){
c <- result{k,f(v)}
}
func Map(f func(T) T,m map[T]T) map[T]T{
c := make(chan result)
ret := make(map[T]T)
n := 0
for k := range m{
go wrapper(f,k,m[k],c)
n++
}
for ;n>0; {
r := <-c
ret[r.key]=r.value
n--
}
return(ret)
}
Unfortunately when I run main with this generalize Mr.Map I get the following error.
# command-line-arguments
./main.go:22: cannot use f (type func(string) int) as type func(Mr.T) Mr.T in argument to Mr.Map
./main.go:22: cannot use data (type map[int]string) as type map[Mr.T]Mr.T in argument to Mr.Map
So yeah, obviously I understand what the errors are telling me but it seems wild that I would have to re-write my Map function for each possible combination of key and value types.
Is there a work around here, or is this just the nature of the beast?
No real workaround there, that's the nature of the beast.
The language was designed after struggling with C++ for some time, and the idea of the creators was simplifying all non-vital things but at the same time make key additions to make the language more expressive.
You can read a bit about their reasoning here, which I believe is quite interesting even if you don't agree with all the decisions they made:
https://commandcenter.blogspot.com.ar/2012/06/less-is-exponentially-more.html
In your example, if you wanted to, you could make your maps and functions use interface{} (which by the way is called the empty interface and not "nil" interface).
But of course you would lose compile-time type checking and would have to add casts all around.
You can also try to find an interface to express the commonalities of the types you want to use (which might not be so easy or even possible at all), and then build your mapping API around that interface.
The philosophy of Go is not compatible with generalized functions such as is the style with popular dynamic languages. To properly inform the compiler of what you are trying to do, you should express your needed map through an interface or simply by writing it for each type you are using it with.
Mapping requires allocating an array, iterating through a collection, and adding to the array some data element for each element in the collection. If you need a map for a slice of structs, as is common in the application layer, you can express this tersely in Go:
https://play.golang.org/p/pk3Tl_BdlD
Dynamic languages build a "type tree" of "generic" types that allow for terse programming, such as functions like map being called by one symbol over any possible type. This provides a ton of developer productivity because code can be written loosely to allow easy experimentation.
Go is designed for writing semi-permanent software. It performs well because it requires more information to be supplied to the compiler. Map is only about three lines of code, so the cost/benefit of developer productivity v. efficiency lands on the performance side for Go. Functions like map, reduce and filter should be written explicitly as needed.
To evaluate the language, I would encourage you to try to solve a problem with a Go program and see where that takes you.
Related
I am trying to create a Map with String and functions as key and Value. It works if all functions are of same signature but my requirement is to store functions of different signature in the same map. Is this possible in Go?
package main
import "fmt"
func main() {
functions := buildFunctions()
f := functions["isInValid"]
// f("hello")
}
func buildFunctions() map[string]func() bool {
functions := map[string]func() bool{
"isInValid": isInValid,
"isAvailable": isAvailable,
}
return functions
}
func isInValid(s string) bool {
fmt.Println("Invalid ", s)
return true
}
func isAvailable(s string, s1 string) bool {
return true
}
https://play.golang.org/p/ocwCgEpa_0G
Go is a strongly typed language. So, it's not possible the way, it is possible with, say python. But just like python, once you do this, you loose the benefit of compile time error checks, and your runtime error checking has to be full-proof.
Here's what you can do:
Use map[string]interface{} type for your function map, which enables you to store anything. But then you are responsible to correctly type assertion at the time of calling. Problem is that, in most cases, if while calling a function, you could know the type of function, may be you might not need a map in the first place.
Use a map[string]string or map[string]interface{} as the argument, and return type in all the functions that are supposed to go into this map. Or at least put all the variable arguments into this map.
eg.
map[string](func (name string, age int, other_attributes
map[string]interface{}) (map[string]interface{}, error))
But again, each function call should provide the correct arguments, and there should also be checks inside the functions, to see (with non-panic version of map lookup), if the parameters are correctly provided, if not, you can return a custom error like ErrInvalidParametersPassed. (playing the role of an interpreter). But you will still have lesser chances of messing up, compared to first option. As the burden of type assertion will be on the function implementation, and not the caller. Caller just needs to fetch it's required values, which it anyways needs to know about.
But yet, best option would be to redesign your actual solution in a way, so that it can be done without going this road. As #bereal suggested in comments, it's good to have separate maps if possible, or maybe use a superset of arguments if they aren't too different, or too many. If there are just a few arguments, even switch case could be clean enough. Look for ways that cheat/bypass compile time checks, when you are truly convinced that there is no other elegant way.
I want to express a function that can take any slice. I thought that I could do this:
func myFunc(list []interface{}) {
for _, i := range list {
...
some_other_fun(i)
...
}
}
where some_other_fun(..) itself takes an interface{} type. However, this doesn't work because you can't pass []DEFINITE_TYPE as []interface{}. See: https://golang.org/doc/faq#convert_slice_of_interface which notes that the representation of an []interface{} is different. This answer sums up why but with respect to pointers to interfaces instead of slices of interfaces, but the reason is the same: Why can't I assign a *Struct to an *Interface?.
The suggestion provided at the golang.org link above suggests rebuilding a new interface slice from the DEFINITE_TYPE slice. However, this is not practical to do everywhere in the code that I want to call this function (This function is itself meant to abbreviate only 9 lines of code, but those 9 lines appear quite frequently in our code).
In every case that I want to invoke the function I would be passing a []*DEFINITE_TYPE which I at first thought would be easier to abstract until, again, I discovered Why can't I assign a *Struct to an *Interface? (also linked above).
Further, everytime I want to invoke the function it is with a different DEFINITE_TYPE so implementing n examples for the n types would not save me any lines of code or make my code any clearer (quite the contrary!).
It is frustrating that I can't do this since the 9 lines are idiomatic in our code and a mistype could easily introduce a bug. I'm really missing generics. Is there really no way to do this?!!
In the case you provided, you would have to create your slice as a slice of interface e.g. s := []interface{}{}. At which point you could literally put any type you wanted into the slice (even mixing types). But then you would have to do all sorts of type assertions and everything gets really nasty.
Another technique that is commonly used by unmarshalers is a definition like this:
func myFunc(list interface{})
Because a slice fits an interface, you can indeed pass a regular slice into this. You would still need to do some validation and type assertions in myFunc, but you would be doing single assertions on the entire list type, instead of having to worry about a list that could possibly contain mixed types.
Either way, due to being a statically typed language, you eventually have to know the type that is passed in via assertions. It's just the way things are. In your case, I would probably use the func signature as above, then use a type switch to handle the different cases. See this document https://newfivefour.com/golang-interface-type-assertions-switch.html
So, something like this:
func myFunc(list interface{}) {
switch v := list.(type) {
case []string:
// do string thing
case []int32, []int64:
// do int thing
case []SomeCustomType:
// do SomeCustomType thing
default:
fmt.Println("unknown")
}
}
No there is no easy way to deal with it. Many people miss generics in Go.
Maybe you can get inspired by sort.Sort function and sort.Interface to find a reasonable solution that would not require copying slices.
Probably the best thing to do is to define an interface that encapsulates what myFunc needs to do with the slice (i.e., in your example, get the nth element). Then the argument to the function is that interface type and you define the interface method(s) for each type you want to pass to the function.
You can also do it with the reflect package, but that's probably not a great idea since it will panic if you pass something other than a slice (or array or string).
func myFunc(list interface{}) {
listVal := reflect.ValueOf(list)
for i := 0; i < listVal.Len(); i++ {
//...
some_other_fun(listVal.Index(i).Interface())
//...
}
}
See https://play.golang.org/p/TyzT3lBEjB.
Now with Go 1.18+, you can use the generics feature to do that:
func myFunc[T any](list []T) {
for _, item := range list {
doSomething(item)
}
}
Is it a good idea to create own type from a slice in Golang?
Example:
type Trip struct {
From string
To string
Length int
}
type Trips []Trip // <-- is this a good idea?
func (trips *Trips) TotalLength() int {
ret := 0
for _, i := range *trips {
ret += i.Length
}
return ret
}
Is it somehow a convention in Golang to create types like Trips in my example? Or it is better to use []Trip in the whole project? Any pros and cons?
There's no convention, as far as I am aware of. It's OK to create a slice type if you really need it. In fact, if you ever want to sort your data, this is pretty much the only way: create a type and define the sort.Interface methods on it.
Also, in your example there is no need to take the address of Trips since slice is already a "fat pointer" of a kind. So you can simplify your method to:
func (trips Trips) TotalLength() (tl int) {
for _, l := range trips {
tl += l.Length
}
return tl
}
If this is what your type is (a slice), it's just fine. It gives you an easy access to underlying elements (and allows for range iteration) while providing additional methods.
Of course you probably should only keep essential set of methods on this type and not bloating it with everything that would take []Trip as an argument. (For example I would suggest having DrawTripsOnTheGlobe(t Trips) rather than having it as a Trips' method.)
To calm your mind there are plenty of such slice-types in standard packages:
http://golang.org/pkg/net/#IP
http://golang.org/pkg/sort/#Float64Slice
http://golang.org/pkg/sort/#IntSlice
http://golang.org/pkg/encoding/json/#RawMessage
Go has no unions. But unions are necessary in many places. XML makes excessive use of unions or choice types. I tried to find out, which is the preferred way to work around the missing unions. As an example I tried to write Go code for the non terminal Misc in the XML standard which can be either a comment, a processing instruction or white space.
Writing code for the three base types is quite simple. They map to character arrays and a struct.
type Comment Chars
type ProcessingInstruction struct {
Target *Chars
Data *Chars
}
type WhiteSpace Chars
But when I finished the code for the union, it got quite bloated with many redundant functions. Obviously there must be a container struct.
type Misc struct {
value interface {}
}
In order to make sure that the container holds only the three allowed types I made the value private and I had to write for each type a constructor.
func MiscComment(c *Comment) *Misc {
return &Misc{c}
}
func MiscProcessingInstruction (pi *ProcessingInstruction) *Misc {
return &Misc{pi}
}
func MiscWhiteSpace (ws *WhiteSpace) *Misc {
return &Misc{ws}
}
In order to be able to test the contents of the union it was necessary to write three predicates:
func (m Misc) IsComment () bool {
_, itis := m.value.(*Comment)
return itis
}
func (m Misc) IsProcessingInstruction () bool {
_, itis := m.value.(*ProcessingInstruction)
return itis
}
func (m Misc) IsWhiteSpace () bool {
_, itis := m.value.(*WhiteSpace)
return itis
}
And in order to get the correctly typed elements it was necessary to write three getters.
func (m Misc) Comment () *Comment {
return m.value.(*Comment)
}
func (m Misc) ProcessingInstruction () *ProcessingInstruction {
return m.value.(*ProcessingInstruction)
}
func (m Misc) WhiteSpace () *WhiteSpace {
return m.value.(*WhiteSpace)
}
After this I was able to create an array of Misc types and use it:
func main () {
miscs := []*Misc{
MiscComment((*Comment)(NewChars("comment"))),
MiscProcessingInstruction(&ProcessingInstruction{
NewChars("target"),
NewChars("data")}),
MiscWhiteSpace((*WhiteSpace)(NewChars(" \n")))}
for _, misc := range miscs {
if (misc.IsComment()) {
fmt.Println ((*Chars)(misc.Comment()))
} else if (misc.IsProcessingInstruction()) {
fmt.Println (*misc.ProcessingInstruction())
} else if (misc.IsWhiteSpace()) {
fmt.Println ((*Chars)(misc.WhiteSpace()))
} else {
panic ("invalid misc");
}
}
}
You see there is much code looking almost the same. And it will be the same for any other union. So my question is: Is there any way to simplify the way to deal with unions in Go?
Go claims to simplify programing work by removing redundancy. But I think the above example shows the exact opposite. Did I miss anything?
Here is the complete example: http://play.golang.org/p/Zv8rYX-aFr
As it seems that you're asking because you want type safety, I would firstly argue that your initial
solution is already unsafe as you have
func (m Misc) Comment () *Comment {
return m.value.(*Comment)
}
which will panic if you haven't checked IsComment before. Therefore this solution has no benefits over
a type switch as proposed by Volker.
Since you want to group your code you could write a function that determines what a Misc element is:
func IsMisc(v {}interface) bool {
switch v.(type) {
case Comment: return true
// ...
}
}
That, however, would bring you no compiler type checking either.
If you want to be able to identify something as Misc by the compiler then you should
consider creating an interface that marks something as Misc:
type Misc interface {
ImplementsMisc()
}
type Comment Chars
func (c Comment) ImplementsMisc() {}
type ProcessingInstruction
func (p ProcessingInstruction) ImplementsMisc() {}
This way you could write functions that are only handling misc. objects and get decide later
what you really want to handle (Comments, instructions, ...) in these functions.
If you want to mimic unions then the way you wrote it is the way to go as far as I know.
I think this amount of code might be reduced, e.g. I personally do not think that safeguarding type Misc against containing "illegal" stuff is really helpful: A simple type Misc interface{} would do, or?
With that you spare the constructors and all the Is{Comment,ProcessingInstruction,WhiteSpace} methods boil down to a type switch
switch m := misc.(type) {
Comment: fmt.Println(m)
...
default: panic()
}
Thats what package encoding/xml does with Token.
I am not sure to understand your issue. The 'easy' way to do it would be like the encoding/xml package with interface{}. If you do not want to use interfaces, then you can do something like you did.
However, as you stated, Go is a typed language and therefore should be use for typed needs.
If you have a structured XML, Go can be a good fit, but you need to write your schema. If you want a variadic schema (one given field can have multiple types), then you might be better off with an non-typed language.
Very useful tool for json that could easily rewritten for xml:
http://mholt.github.io/json-to-go/
You give a json input and it gives you the exact Go struct. You can have multiple types, but you need to know what field has what type. If you don't, you need to use the reflection and indeed you loose a lot of the interest of Go.
TL;DR You don't need a union, interface{} solves this better.
Unions in C are used to access special memory/hardware. They also subvert the type system. Go does not have the language primitives access special memory/hardware, it also shunned volatile and bit-fields for the same reason.
In C/C++ unions can also be used for really low level optimization / bit packing. The trade off: sacrifice the type system and increase complexity in favor of saving some bits. This of course comes with all the warnings about optimizations.
Imagine Go had a native union type. How would the code be better? Rewrite the code with this:
// pretend this struct was a union
type MiscUnion struct {
c *Comment
pi *ProcessingInstruction
ws *WhiteSpace
}
Even with a builtin union accessing the members of MiscUnion requires a runtime check of some kind. So using an interface is no worse off. Arguably the interface is superior as the runtime type checking is builtin (impossible to get wrong) and has really nice syntax for dealing with it.
One advantage of a union type is static type check to make sure only proper concrete types where put in a Misc. The Go way of solving this is "New..." functions, e.g. MiscComment, MiscProcessingInstruction, MiscWhiteSpace.
Here is a cleaned up example using interface{} and New* functions: http://play.golang.org/p/d5bC8mZAB_
In Go, in order to iterate over an array/slice, you would write something like this:
for _, v := range arr {
fmt.Println(v)
}
However, I want to iterate over array/slice which includes different types (int, float64, string, etc...). In Python, I can write it out as follows:
a, b, c = 1, "str", 3.14
for i in [a, b, c]:
print(i)
How can I do such a work in Go? As far as I know, both array and slice are supposed to allow only same-type object, right? (say, []int allows only int type object.)
Thanks.
As Go is a statically typed language, that won't be as easy as in Python. You will have to resort to type assertions, reflection or similar means.
Take a look at this example:
package main
import (
"fmt"
)
func main() {
slice := make([]interface{}, 3)
slice[0] = 1
slice[1] = "hello"
slice[2] = true
for _, v := range slice {
switch v.(type) {
case string:
fmt.Println("We have a string")
case int:
fmt.Println("That's an integer!")
// You still need a type assertion, as v is of type interface{}
fmt.Printf("Its value is actually %d\n", v.(int))
default:
fmt.Println("It's some other type")
}
}
}
Here we construct a slice with the type of an empty interface (any type implements it), do a type switch and handle the value based on the result of that.
Unfortunately, you'll need this (or a similar method) anywhere where you'll be dealing with arrays of unspecified type (empty interface). Moreover, you'll probably need a case for every possible type, unless you have a way to deal with any object you could get.
One way would be to make all of the types you want to store implement some interface of yours and then only use those objects through that interface. That's kind of how fmt handles generic arguments – it simply calls String() on any object to get its string representation.