Go 1.8 supports Go plugins.
I have created two plugins as follows.
As I understand, the plugin exposes only the functions and variables in the main package. i.e. plugin.Lookup() will fail for non-main variable/function.
But I wanted to test if a plugin can internally invoke a method from another plugin, similar to how a C++ library can invoke another library.
So I tested as follows:
plugin1 github.com/vimal/testplugin
$ cat myplugin.go
package main
import "C"
import "fmt"
import help "github.com/vimal/testplugin1/plug"
func init() {
fmt.Printf("main.init invoked\n")
}
// TestPlugin
func TestPlugin() string {
return help.Help()
}
plugin2 github.com/vimal/testplugin1
$ cat myplugin.go
package main
import "C"
func HelperFunc() string {
return "help"
}
$ cat plug/helper.go
package help
func Help() string {
return "help234"
}
Idea here is that plugin1 invokes an internal, non-main function of plugin2.
main program
Main program loads a number of plugins given as arguments, and invokes TestPlugin() from the last plugin.
test 1:
Build both plugins, and load both plugins, and invoke TestPlugin(), the output contains "help234", i.e. the inner function is invoked. This can be understood, since both plugins are loaded, one plugin can invoke another plugin's inner code.
test 2:
Load only plugin1, and invoke TestPlugin(), the output contains "help234", i.e. the inner function is invoked. The same output is observed as in test1. Perhaps this time the method is found from GOPATH.
test 3:
Rename the folder "github.com/vimal/testplugin1" to "github.com/vimal/junk1", delete the plugin2, and load only plugin1, and invoke TestPlugin(). the output still contains "help234", i.e. the inner function is invoked.
I am not able to understand how test3 produces the same output. Does plugin1 contain plugin2 code also? How can I understand the Go plugin dependency on other Go plugins?
Go version : go version go1.8rc3 linux/amd64
You're not doing exactly what you think you are.
Your plugin1 imports and uses a package, namely github.com/vimal/testplugin1/plug. This is not "equal" to plugin2!
What happens here is that when you build plugin1, all its dependencies are built into the plugin file, including the .../testplugin1/plug package. And when you load plugin1, all its dependencies are also loaded from the plugin file, including the plug package. After this, it's not surprising that it works no matter the loaded status of plugin2. These 2 plugins are independent from each another.
The -buildmode=plugin instructs the compiler that you want to build a plugin and not a standalone app, but it doesn't mean dependencies must not be included. They have to be, because the plugin cannot have any guarantee what Go app will load it, and what packages that Go app will have. Because a runnable app also only contains packages even from the standard library that the app itself references explicitly.
The only way to guarantee that the plugin will have everything it needs and so that it will work if it also contains all its dependencies, including those from the standard library. (This is the reason why building simple plugins generate relatively big files, similarly to building simple Go executables resulting in big files.)
Few things that don't need to be added to plugins include the Go runtime for example, because a running Go app that will load the plugin will already have a Go runtime running. (Note that you can only load a plugin from an app compiled with the same version of Go.) But beyond that, the plugin has to contain everything it needs.
Go is a statically linked language. Once a Go app or plugin is compiled, they do not rely nor check the value of GOPATH, it is only used by the Go tool during building them.
Deeper insight
It's possible that your main app and a plugin refers to the same package ("same" by import path). In such cases only one "instance" of the package will be used.
This can be tested if this commonly referred package has "state", e.g a global variable. Let's assume a common, shared package called mymath:
package mymath
var S string
func SetS(s string) {
S = s
}
And a plugin called pg that uses it:
package main
import (
"C"
"mymath"
"fmt"
)
func Start() {
fmt.Println("pg:mymath.S", mymath.S)
mymath.SetS("pghi")
fmt.Println("pg:mymath.S", mymath.S)
}
And the main app that uses mymath and loads pg (which uses it):
package main
import (
"plugin"
"mymath"
"fmt"
)
func main() {
fmt.Println("mymath.S", mymath.S)
mymath.SetS("hi")
fmt.Println("mymath.S", mymath.S)
p, err := plugin.Open("../pg/pg.so")
if err != nil {
panic(err)
}
start, err := p.Lookup("Start")
if err != nil {
panic(err)
}
start.(func())()
fmt.Println("mymath.S", mymath.S)
}
Building the plugin:
cd pg
go build -buildmode=plugin
Running the main app, the output is:
mymath.S
mymath.S hi
pg:mymath.S hi
pg:mymath.S pghi
mymath.S pghi
Analysis: first the main app plays with mymath.S, sets it to "hi" eventually. Then comes the plugin, which prints it (we see the value set by the main app "hi"), then changes it to "pghi". Then comes again the main app and prints mymath.S, and again, sees the last value set by the plugin: "pghi".
So there is only one "instance" of mymath. Now if you go ahead and change mymath, e.g. rename myMath.SetS() to mymath.SetS2(), and you update the call in the main app (to mymath.SetS2("hi")), and without rebuilding the plugin, just running the main app, you get the following output:
mymath.S
mymath.S hi
panic: plugin.Open: plugin was built with a different version of package mymath
goroutine 1 [running]:
main.main()
<GOPATH>/src/play/play.go:16 +0x4b5
exit status 2
As you can see, when building the main app and the plugin, the package version (which is most likely a hash) is recorded, which must match if their import paths match in the main app and in the plugin.
(Note that you will also get the above error if you don't change the exported identifiers (and signatures) of the used mymath package, only the implementation; e.g. func SetS(s string) { S = s + "+" }.)
Related
I'm learning Go so I'm writing some basic utils and libraries for fun that I can reuse in other modules.
When importing them inside another module (using go get for example), I can use them just fine.
As I understand, Go will look for a package main and a main function to know the entrypoint when trying to run a go app/module.
The thing is that the libraries are not in "package main", not in main.go and have no main function, so how am I supposed to use the cli and use go run . to test the behavior of my function?
Do I absolutely need to create another module with a main.go file inside a package main?
Or do I need to create a test file and use go test ?
Thank you!
Non-main packages (i.e., library packages) are not runnable.
To run the code in them, you must either run a main package that imports and runs their code, or write test functions in _test.go files to run the code. Note that test files are not just for unit tests; you can write whatever you want in them, and they can often be easier than creating a new main package for testing.
Modern IDEs now will allow you to easily run individual tests in debugging mode, so you can step through your code and see if it's working how you expect. If you prefer print debugging and doing everything from the command-line, you can still run an individual test and see the print output like this:
// something_test.go
package mylibrary
func TestSomeFunction(t *testing.T) {
SomeFunction()
}
go test mylibrary -v -run TestSomeFunction
go test is the preferred way to test Go code.
If you do not want to use go test for some reason, then use build constraints and the go run command to execute a main program from the same directory as your package.
Here's an example program with a build constraint. Place the file in the same directory as the package source files. The comment //go:build ignore is a build constraint that excludes the file from the package sources.
//go:build ignore
package main
import (
"your/package/import/path" // <-- modify to your path
"fmt"
)
func main() {
// do something with your package here.
fmt.Println("example")
}
Assuming that the above file is named program.go, run the program from the package directory with the command:
go run program.go
The go run command is OK for running toy or test programs like this. Prefer the go build command for anything else.
I have a small go demo project in Gogland with the structure:
awsomeProject
->src
->awsomeProject
->configuration.go
->main.go
Configuration file has a simple structure just for demo:
configuration.go:
package main
type Config struct {
Data int
}
Main file just uses the Config struct:
main.go
package main
import "fmt"
func main(){
var cfg Config
cfg.Data = 1
fmt.Println("lalala")
}
The error that I have is:
/usr/local/go/bin/go run /Users/lapetre/Work/awsomeProject/src/awsomeProject/main.go
command-line-arguments
src/awsomeProject/main.go:6: undefined: Config
Process finished with exit code 2
Any idea why the Config is not seen in main?
Thanks
When you build reusable pieces of code, you will develop a package as a shared library. But when you develop executable programs, you will use the package “main” for making the package as an executable program. The package “main” tells the Go compiler that the package should compile as an executable program instead of a shared library. The main function in the package “main” will be the entry point of our executable program.
That's why you should use the following structure:
awsomeProject
->src
->awsomeProject
->configuration.go
->main.go
with main.go
package main
import "fmt"
func main(){
var cfg awsomeProject.Config
cfg.Data = 1
fmt.Println("lalala")
}
and configuration.go
package awsomeProject
type Config struct {
Data int
}
For more details:
https://golang.org/doc/code.html
https://thenewstack.io/understanding-golang-packages
How are you calling go run? If you're calling it like
go run main.go
then that's the problem.
Go run only runs the file(s) you tell it to. So you need to tell it to also run configuration.go, or if you have several go files to run you can use
go run *.go
as eXMoor suggested.
There are some limits/drawbacks to "go run *.go" however, so the better alternative is to use go build instead of go run.
go build
Will compile everything. Then, to run the executable:
./awesomeProject
To combine all of this into one command that will compile and run whatever app you're working on, you can use:
go build && ./${PWD##*/}
I have it aliased to
gorunapp
just to make it easier.
This may not actually be the answer to the problem you're having, but hopefully you'll find it useful information regardless.
Try to run your application with command:
go run *.go
Without a background in C and only "beginner" experience in Go I'm trying to work out whether main.go is actually required or is just a convention.
I'm looking to create a simple web API but could someone clarify this for me?
main.go as a file is not required.
However, a main package with a func main() is required for executables.
Your file name can be called whatever you want.
E.g
myawesomeapp.go
package main
func main() {
fmt.Println("Hello World")
}
Running go run myawesomeapp.go will work as expected.
For a web server (an executable) you need to have a package main with a func main(), but it doesn't need to be called main.go - the file name can be anything you want it to be. From the language spec:
Program execution
A complete program is created by linking a single, unimported package
called the main package with all the packages it imports,
transitively. The main package must have package name main and declare
a function main that takes no arguments and returns no value.
func main() { … }
Program execution begins by initializing the main package and then
invoking the function main. When that function invocation returns, the
program exits. It does not wait for other (non-main) goroutines to
complete.
I'm new to Go (but not at programming), I love the language but I have a bit of trouble fully understanding the way I'm supposed to make internal libraries in an application through packages. For reference, getting external packages and then importing/using them is fine.
Let's say I'm making an application A.
/home/me/A/a.go (package main)
Then, I realize a.go start to be rather big, so I cut it into two parts
/home/me/A/a.go (package main)
/home/me/A/b.go (package main)
How am I supposed to import/include b.go from a.go to make its function available ?
As a continuation of the question, in the A I'm manipulation lots of objects O, so I figure it would be a lot better if I just give them their own package and encapsulate the functionalities in a public/exported api. How do I do that ?
I've tried creating ./lib/o.go (package o) and import lib/o but I keep getting error like
./a.go:6: imported and not used: "o"
./a.go:43: undefined: o
I have no GOPATH in my env but I tried export GOPATH=$GOPATH:/home/me/A and it didn't change the result.
I've tried to read the article on "go layout" but it felt a bit too overwhelming at once and I would really love a simpler explanation of that one "small" step I am trying to make
Thanks !
GOPATH/src/me/a/a.go:
package main
func main() {
test()
}
GOPATH/src/me/a/test.go:
package main
import "fmt"
func test() {
fmt.Println("test func !")
}
Exec:
$ go run a.go
# command-line-arguments
./a.go:4: undefined: test
EDIT: got my answer here: https://groups.google.com/forum/?fromgroups#!topic/golang-nuts/qysy2bM_o1I
Either list all files in go run (go run a.go test.go) or use go build and run the resulting executable.
You're trying to use the Go build system while not following the necessaary required directory layouts. You will benefit a lot from reading this document.
In short, these are, wrt the go tool, the show stoppers:
You must have a valid, exported GOPATH
Package files with import path "example/foo" must be located in the $GOPATH/src/example/foo directory.
For more details please see the above linked article.
I'm playing with Google Go and I'm having fun (!), but I'm having some problems with package subsystem.
I'm running Go 1.0.1 on Mac OS X Lion. I've build also various single file programs without problems (I've also build a small webapp using html/templates without problems and it compiles and runs without any error).
I've defined a "reusable" package (even.go):
package even
func Even(i int) bool {
return i % 2 == 0
}
func Odd(i int) bool {
return i % 2 == 1
}
and a consumer program (useeven.go):
package main
import (
"./even"
"fmt"
)
func main() {
a := 5
b := 6
fmt.Printf("%d is even %v?\n", a, even.Even(a))
fmt.Printf("%d is odd %v?\n", b, even.Odd(b))
}
But when I compile the "library" using
go build even.go
I got nothing... No errors, no message... What happens?
How should I do this?
The answer to your question, "How should I do this?" is explained in How to Write Go Code. It's really pretty important stuff and worth taking a look at.
The behavior of go build might seem puzzling, but is actually conventional for command line programs--no output means that the program ran successfully. So what did it do? For that your answer is in go help build
... Otherwise build compiles the packages but discards the results,
serving only as a check that the packages can be built.
What, you wanted more? Of course. "How to Write Go Code" explains good ways of doing this. For a quick fix to your program, I'll explain that the go command expects each package and each executable program to be in a separate directory. If you just make a directory called even, immediately under the location of useeven.go, and move even.go to it, then go run useeven.go should run just as you have it.
go build only generates an output file when it builds a "single main package". If you want to output a file for your other package you should use go install. That will build and install that package to your pkgs directory.
As noted in a comment by the OP, go build and go run useeven.go work fine, once you put even.go in its own folder: ./even/even.go. There is a bit of Go magic in the ./ (in the import) that makes the "local package" build automatically without requiring it to be installed anywhere.
useeven/
├── even
│ └── even.go
└── useeven.go
If you wanted to make this work without the magic ./, then the library need to be installed. A quick-and-dirty way (at the risk of polluting your library namespace) of making this work for your project is to simply register the library's source location by using a symbolic link like so (in this example GOPATH is ~/go):
[ useeven ]
$ ln -s $(pwd)/even ~/go/src
Now when you build the program, it automatically performs a go get even to install an up-to-date version of your library.
Note that this doesn't make go install* work for the library, it just makes doing that unnecessary.
There are more idiomatic ways of doing this, if you're not in a hurry: How to import local packages in go?
The reason that output is not being generated here is because you're running the
go build even.go
command on the package and not the main go file. Doing so on the package will check for errors on the package, but because it's not a main with an output, no success messages will be generated. If there are issues with the package's contents, errors will be displayed, otherwise if it compiles fine there will be no output
Instead, you should run:
go build useeven.go
To create the binary, then to execute the binary and get output from the program:
./useeven