I want to be able to get the checksum of every package used by a go program, including packages used within modules.
runtime/debug in the standard library has ReadBuildInfo(), which is great, but it only gives data for modules, not for packages.
Example:
package pkgA
var Foo = 1
package pkgB
import "pkgA"
var Bar = pkgA.Foo
package main
import (
"fmt"
"runtime/debug"
"example/pkgB"
)
func main() {
_ = pkgB.Bar
b, ok := debug.ReadBuildInfo()
if !ok {
fmt.Println("not ok!")
return
}
for _, module := range b.Deps {
fmt.Println(module.Path, module.Sum)
}
}
The output is like
pkgB v0.0.0-20210225235400-92e28d816f64
There is no info on A. I believe this is because pkgB and pkgA both belong to the same module.
Question: Is there any way to access the checksum for pkgA?
The Go checksum database stores checksums for modules, not packages.
The debug information embedded in a binary does not include the mapping from packages to modules, but if you have access to the module's source you can use go list to report the mapping from packages to modules:
go list -f '{{if .Module}}{{.ImportPath}}: {{.Module}}{{end}}' all
You can use that mapping, in conjunction with the module-level checksums, to verify that each package has the correct source code. (Note that go mod verify already implements that verification.)
Related
I've tried many times to set up a REAL go package with the modules system and store code in pkg. All the tutorials I've found are too basic, creating a module with go files stores at the top level, and I keep getting no Go files in /usr/local/go/github.com/me/mypackage.
I've tried a bunch of different things, but I can't get it to work properly...
GOROOT is set to /usr/local/go. I created a package here /usr/local/go/github.com/me/mypackage.
go.mod
module github.com/me/mypackage
go 1.18
pkg/main.go
package mypackage
// Add is our function that sums two integers
func Add(x, y int) (res int) {
return x + y
}
// Subtract subtracts two integers
func Subtract(x, y int) (res int) {
return x - y
}
pkg/main_test.go
package mypackage
import "testing"
func TestAdd(t *testing.T){
got := Add(4, 6)
want := 10
if got != want {
t.Errorf("got %q, wanted %q", got, want)
}
}
And I run: go test
What am I doing wrong? I find Go so frustrating to set up because languages/runtimes like Rust and NodeJS have very friendly package managers and are real easy to setup.
I'm trying to structure a library as described in this guidance for structuring go packages.
Don't confuse modules with packages.
One module might hold many packages.
Like this:
module_dir/package1_dir
module_dir/package2_dir
Try this layout:
Repository: github.com/me/mymodule
mymodule/mypkg
mymodule/mypkg/mypkg_test.go
mymodule/mypkg/mypkg.go
mymodule/go.mod
In mypkg.go and mypkg_test.go declare package mypkg.
Otherwise, run this script and it will create a correct layout for you:
https://gist.github.com/udhos/695d3be51fb4c7d151b4e252cdec3c63
I am attempting to create named loggers automatically for HTTP handlers that I'm writing, where I am passed a function (pointer).
I'm using the code mentioned in this question to get the name of a function:
package utils
import (
"reflect"
"runtime"
)
func GetFunctionName(fn interface{}) string {
value := reflect.ValueOf(fn)
ptr := value.Pointer()
ffp := runtime.FuncForPC(ptr)
return ffp.Name()
}
I'm using this in my main function to try it out like so:
package main
import (
"github.com/naftulikay/golang-webapp/experiments/functionname/long"
"github.com/naftulikay/golang-webapp/experiments/functionname/long/nested/path"
"github.com/naftulikay/golang-webapp/experiments/functionname/utils"
"log"
)
type Empty struct{}
func main() {
a := long.HandlerA
b := path.HandlerB
c := path.HandlerC
log.Printf("long.HandlerA: %s", utils.GetFunctionName(a))
log.Printf("long.nested.path.HandlerB: %s", utils.GetFunctionName(b))
log.Printf("long.nested.path.HandlerC: %s", utils.GetFunctionName(c))
}
I see output like this:
github.com/naftulikay/golang-webapp/experiments/functionname/long.HandlerA
This is okay but I'd like an output such as long.HandlerA, long.nested.path.HandlerB, etc.
If I could get the Go module name (github.com/naftulikay/golang-webapp/experiments/functionname), I can then use strings.Replace to remove the module name to arrive at long/nested/path.HandlerB, then strings.Replace to replace / with . to finally get to my desired value, which is long.nested.path.HandlerB.
The first question is: can I do better than runtime.FuncForPC(reflect.ValueOf(fn).Pointer()) for getting the qualified path to a function?
If the answer is no, is there a way to get the current Go module name using runtime or reflect so that I can transform the output of runtime.FuncForPC into what I need?
Once again, I'm getting values like:
github.com/naftulikay/golang-webapp/experiments/functionname/long.HandlerA
github.com/naftulikay/golang-webapp/experiments/functionname/long/nested/path.HandlerB
github.com/naftulikay/golang-webapp/experiments/functionname/long/nested/path.HandlerC
And I'd like to get values like:
long.HandlerA
long.nested.path.HandlerB
long.nested.path.HandlerC
EDIT: It appears that Go does not have a runtime representation of modules, and that's okay, if I can do it at compile time that would be fine too. I've seen the codegen documentation and I'm having a hard time figuring out how to write my own custom codegen that can be used from go generate.
The module info is included in the executable binary, and can be acquired using the debug.ReadBuildInfo() function (the only requirement is that the executable must be built using module support, but this is the default in the current version, and likely the only in future versions).
BuildInfo.Path is the current module's path.
Let's say you have the following go.mod file:
module example.com/foo
Example reading the build info:
bi, ok := debug.ReadBuildInfo()
if !ok {
log.Printf("Failed to read build info")
return
}
fmt.Println(bi.Main.Path)
// or
fmt.Println(bi.Path)
This will output (try it on the Go Playground):
example.com/foo
example.com/foo
See related: Golang - How to display modules version from inside of code
If your goal is to just have the name of the module available in your program, and if you are okay with setting this value at link time, then you may use the -ldflags build option.
You can get the name of the module with go list -m from within the module directory.
You can place everything in a Makefile or in a shell script:
MOD_NAME=$(go list -m)
go build -ldflags="-X 'main.MODNAME=$MOD_NAME'" -o main ./...
With main.go looking like:
package main
import "fmt"
var MODNAME string
func main() {
fmt.Println(MODNAME) // example.com
}
With the mentioned "golang.org/x/mod/modfile" package, an example might look like:
package main
import (
"fmt"
"golang.org/x/mod/modfile"
_ "embed"
)
//go:embed go.mod
var gomod []byte
func main() {
f, err := modfile.Parse("go.mod", gomod, nil)
if err != nil {
panic(err)
}
fmt.Println(f.Module.Mod.Path) // example.com
}
However embedding the entire go.mod file in your use case seems overkill. Of course you could also open the file at runtime, but that means you have to deploy go.mod along with your executable. Setting the module name with -ldflags is more straightforward IMO.
How can I specify to import/use a package from the vendor instead from the GOPATH/GOROOT?
$GOPATH/src/
$GOPATH/src/github.com/
$GOPATH/src/github.com/myproject/mypkg
$GOPATH/src/github.com/myproject/mypkg/mypkgfile1.go
package mypkg
import "github.com/someproject/somepkg" // importing from vendor
type MyStruct struct {
Config somepkg.SomeStruct1
}
func New(config somepkg.SomeStruct1) MyStruct {...}
func (m *MyStruct) DoSomething() {
a := somepkg.SomeStruct1{}
b := somepkg.SomeStruct2{}
// do something with 'a' and 'b'
out := somepkg.SomeFunc(a)
}
func (m *MyStruct) MyFunc(input SomeStruct1) (output SomeStruct2, err error) {...}
$GOPATH/src/github.com/myproject/mypkg/mypkgfile2.go
$GOPATH/src/github.com/myproject/mypkg/vendor/github.com/someproject/somepkg/
$GOPATH/src/github.com/myproject/mypkg/vendor/github.com/someproject/somepkg/somepkgfile1.go
package somepkg
type SomeStruct1 struct {...}
type SomeStruct2 struct {...}
func SomeFunc(input SomeStruct1) (output SomeStruct2) {...}
$GOPATH/src/github.com/myproject/mypkg/go.mod
module gitHub.com/myproject/mypkg
go 1.1.4
require github.com/someproject/somepkg v1.0.0
$GOPATH/src/github.com/someproject/somepkg/somepkgfile1.go
package somepkg
type SomeStruct1 struct {...}
type SomeStruct2 struct {...}
func SomeFunc() {...}
$GOPATH/src/github.com/someproject/somepkg/go.mod
module gitHub.com/someproject/somepkg
go 1.1.4
require github.com/someproject/somepkg v1.0.0
$GOPATH/src/github.com/anotherproject/anotherpkpg/somepkgfile1.go
package main
import (
"github.com/someproject/somepkg"
"github.com/myproject/mypkg"
)
func main() {
// do something with somepkg
somepkg.SomeFunc()
s := somepkg.SomeStruct1{...}
myData := mypkg.New(s)
m := mypkg.MyFunc()
x := somepkg.SomeStruct1{...}
y := mypkg.MyFunc(x)
}
$GOPATH/src/github.com/someproject/somepkg/go.mod
module gitHub.com/someproject/somepkg
go 1.1.4
require (
github.com/myproject/mypkg v1.0.0
github.com/someproject/somepkg v1.0.0
)
When I'm building/running anotherpkpg/main.go I keep getting a type mismatch error like:
cannot use &s (type *"someproject/somepkg".SomeStruct1) as type *"myproject/mypkg/vendor/github.com/someproject/somepkg".SomeStruct1 in argument to mypkg.New
Its not possible at all to be able do this? I get it that type mismatch can occur if the somepkg are of different version/releases. But There is no way to reference the vendored somepkg? I would think it would get even more complex when i
vendor directories work differently in GOPATH mode than in module mode.
Since github.com/myproject/mypkg/go.mod exists, you are presumably building github.com/myproject/mypkg and its dependencies in module mode. In module mode, only the vendor contents for the main module are used, and vendor directories for other packages are always ignored, so each package has exactly one location and one definition. (With -mod=vendor each package is loaded from your vendor directory; with -mod=readonly or -mod=mod each package is loaded from the module cache.)
Since github.com/anotherproject/anotherpkg/go.mod does not exist, Go 1.15 and earlier will by default build it in GOPATH mode. In GOPATH mode, the import statements within each directory refer to the packages in the vendor subtrees of all parent directories, and the vendored packages are treated as distinct packages even if they share the same import path.
The right long-term fix for your situation is probably to start building github.com/anotherproject/anotherpkg in module mode, and to use a replace directive if needed to point it at your working copy of github.com/myproject/mypkg.
You can set that up using a sequence of commands like:
# Enable module mode for anotherpkg.
cd $GOPATH/src/github.com/anotherproject/anotherpkg
go mod init github.com/anotherproject/anotherpkg
# Use the local copy of github.com/myproject/mypkg instead of the latest GitHub snapshot.
go mod edit -replace github.com/myproject/mypkg=$GOPATH/src/github.com/myproject/mypkg
go get -d github.com/myproject/mypkg
# Resolve any additional missing dependencies to their latest versions.
go mod tidy
I'm building a simple app and after reading the doc on structuring go applications, I'm still confused.
I want this structure:
practice
models (packaged as models)
a
b
routers (packaged as routers)
a
b
app.go
Inside of app.go, I have the following:
package main
import (
"net/http"
// I have tried the following:
"practice/models/a"
"practice/models/b"
"practice/models"
"$GOPATH/practice/models/a"
"$GOPATH/practice/models/b"
"$GOPATH/practice/models"
...
)
func main() {
http.HandleFunc("/a", AHandler)
http.HandleFunc("/b", BHandler)
http.ListenAndServe(":8080", nil)
}
The A and B models look like this:
package models
import "net/http"
func AHandler(w http.ResponseWriter, r *http.Request) {
// code
}
Two questions:
What in the world is the right way to import these files? Do I really have to push them to github in order to be able to reference them? I understand the $GOPATH is the namespace for the entire go workspace on a local machine. My $GOPATH is set to include this directory.
Do I need to define a main method inside of these files? Can I just export a single function and have that be the handling function?
I have consulted the docs
See How to Write Go Code.
Use this directory structure:
- practice
- go.mod
- app.go
- models
- a.go
- b.go
- routers
- a.go
- b.go
where go.mod is created with the command go mod init practice where practice is the module path.
Import the packages as follows:
import (
"practice/routers"
"practice/models"
...
)
Use the imported packages like this:
func main() {
http.HandleFunc("/a", models.AHandler)
http.HandleFunc("/b", models.BHandler)
http.ListenAndServe(":8080", nil)
}
You do not need to push to github.com, even if you use github.com in the module path.
The main function in the main package is the entry point for the application. Do not define main functions in packages other than main.
What follows is the original answer based on GOPATH workspaces:
See How to Write Go Code.
Create your directory structure under $GOPATH/src.
$GOPATH
src
practice
models
routers
Import the packages as follows:
import (
"practice/routers"
"practice/models"
...
)
Use the imported packages like this:
func main() {
http.HandleFunc("/a", models.AHandler)
http.HandleFunc("/b", models.BHandler)
http.ListenAndServe(":8080", nil)
}
You do not need to push to github.com, even if you use 'github.com' in the file path.
The main function in the main package is the entry point for the application. Do not define main functions in packages other than main.
I think the other answer is out of date, you don't need to use GOPATH anymore.
Run:
go mod init yellow
Then create a file yellow.go:
package yellow
func Mix(s string) string {
return s + "Yellow"
}
Then create a file orange/orange.go:
package main
import "yellow"
func main() {
s := yellow.Mix("Red")
println(s)
}
Then build:
go build
https://golang.org/doc/code.html
I am new to go and coming from a Ruby background. I am trying to understand code structuring in a world without classes and am probably making the mistake wanting to do it "the Ruby way" in Go.
I am trying to refactor my code to make it more modular / readable so I moved the loading of the configuration file to its own package. Good idea?
package configuration
import "github.com/BurntSushi/toml"
type Config struct {
Temperatures []struct {
Degrees int
Units string
}
}
func Load() Config {
var cnf Config
_, err := toml.DecodeFile("config", &cnf)
if err != nil {
panic(err)
}
return cnf
}
Now, in my main package:
package main
import "./configuration"
var conf Configuration = configuration.Load()
Gives undefined: Config. I understand why. I could copy the struct definition in the main package but that's not very DRY.
It's my understanding passing around structs like this is a bad practice as it makes your code harder to understand (now everyone needs to know about my Config struct).
Is hiding logic in a package like I am trying to do here a good idea in Go? If so, what's the "Go" way to pass this Config struct around?
In your main package you should specify
var conf configuration.Config = configuration.Load()
configuration refers to your imported package and Config is the exported struct (uppercase name) from that package. But you can also omit this, as the type can be inferred
var conf = configuration.Load()
As a side note: please don't use relative imports
in Go imports you always declare the full path of you package, dont use relative paths in imports, best example is that toml import import "github.com/BurntSushi/toml" that exist in:
GOPATH/src/github.com/BurntSushi/toml
GOPATH/pkg/_/github.com/BurntSushi/toml
Then build you package and main.go
package main
import "mypackage/configuration"
func main() {
// configuration contain all funcs & structs
var conf configuration.Config = configuration.Load()
}
Go it is not ruby.
Ref Packages: https://golang.org/doc/code.html
why don't you just import the configuration package and then do Go's variable declaration/instatiation shortcut? Maybe I'm missing something.
package main
import "mypackage/configuration"
func main() {
conf := configuration.Load()
}