I'm having trouble achieving what should be an easy task. I understand the GitHub model for code organization (ie a library repo and an app repo that consumes the library). I think it's fantastic. But I find often that I want mylib to come bundled with a simple executable in a single main.go file. The main.go should be package main and should import mylib. In other words, it should be an exact documentation of how to build an app that consumes this library.
My point is, since it is often enough convenient to provide a simple command line interface that wraps your library, there should be an easy way to do this without having to make another repo, and golang should help.
I'd like something like the following:
$GOPATH/src/github.com/me/mylib
mylib.go
mylib_also.go
main.go
where mylib is the library (package mylib) and main.go is package main and on running go install it generates bin/mylib and pkg/mylib.a.
Either main.go should import "github.com/me/mylib" (if I do that now, I get cyclical import error) or go would understand what's happening since this feature should be built in and the one main.go in the repo generates the exec. Probably requiring the import (and dropping the cyclical error) is the better way.
Right now, I have to do
$GOPATH/src/github.com/me/mylib
mylib/
mylib.go
main.go
So I have to import github.com/me/mylib/mylib which is ridiculous.
In sum, go install should allow the special case of a package and a main which imports the package and provides a simple cli that demonstrates the packages API. The GitHub model promotes two repos, but it should be easy to do simple clis in one!
You can't have multiple packages per folder. Go operates on a package-level, not a file level.
Convention in this case—a binary that consumes your library—is to create a cmd folder with your package main - i.e. as per https://github.com/bradfitz/camlistore/tree/master/cmd or https://github.com/boltdb/bolt
In your case this would be:
$GOPATH/src/github.com/me/mylib
mylib/
mylib.go
README.md
cmd/
your-lib-tool/
main.go
If you just want a convenient, little wrapper, just some command line interface to your lib for demonstration purpose and it is okay if the main program is not built on go get (and not during simple go build and go install) but you are okay by running it via go run main.go or building it manually e.g. with 6g: Just use a build tag (see http://golang.org/pkg/go/build/#hdr-Build_Constraints) in main.go:
// +build ignore
and you won't need a different folder or repo.
Related
I am trying to port a program from C to Go, so using a lot of stuff from the syscall package is required.
I am trying to use https://pkg.go.dev/syscall#EpollCreate, but VSCode refuses to autocomplete it for me or recognize that it is a defined function. FWIW, I get autocomplete for many other things in the syscall package.
My project is using Go 1.14. I am unsure how to tell what version of Go things were introduced in, so I am wondering if that is my problem.
I tried creating a dummy project that uses Go 1.17 and still no luck.
I am writing the code on a Mac, but it will eventually be compiled for ARM Linux, if that matters.
So is this a Go problem or a VSCode problem? Both? Neither?
Sample dummy project:
go.mod:
module epoll-noodle
go 1.14
main.go:
package main
import (
"syscall"
)
func main() {
syscall.EpollCreate(4)
}
It turns out that if you are developing for a platform that differs from your own, some things may be unsupported (like EpollCreate in this case, as #JimB said). The Go extension does let you change your env vars though, so you can work around this in VSCode. In your extensions settings, put something like:
{
"go.toolsEnvVars": {"GOOS" : "linux"}
}
and your project will be built for that OS.
The settings file can be found in the .vscode folder in the root of your project. If there isn't one, you can go to the Go extension in VSCode and fiddle with some settings (for the workspace, not the user) and it will create one for you. Then edit as necessary.
See also: https://github.com/microsoft/vscode-go/issues/632
I want to use a function from a parent module for this project https://github.com/eregnier/beuss
however (and for the repository in the current state), all my tries fails.
in my root folder I have the following go.mod file
module github.com/eregnier/beuss
go 1.17
in the ./cmd folder I have the following go.mod file
module beuss/cmd
go 1.17
I tried various combination of go.mod file for the cmd folder like
module github.com/eregnier/beuss/cmd
go 1.17
I also tried some imports from the https://github.com/eregnier/beuss/blob/main/cmd/cmd.go file looking like
import (
"fmt"
"io/ioutil"
"log"
"os"
"github.com/eregnier/beuss"
// The following line does not work either
// "github.com/eregnier/beuss" beuss
)
//try to use parent module functions
func main () {
connPUT, err := beuss.newClient(beuss.MESSAGE_PUT)
}
nothing work.
I tried the following commands desperately
go mod init
go mod tidy
go get github.com/eregnier/beuss
go install github.com/eregnier/beuss
I got the follwing errors :
go get: github.com/eregnier/beuss#v0.0.0-20220101172233-d7ecaadb1d81: parsing go.mod:
module declares its path as: beuss
but was required as: github.com/eregnier/beuss
with not understanding the real issue. I googled errors, I had a look at this which looks intersting https://go.dev/doc/code#Workspaces
In the end, I am just loosing patience for this issue where I already spend lot of time without really understanding the real good practice / why it should be done like this or that.
I am sorry if this is a redundant question. but this topic looks nebulous to me where I just want to resolve a dependency. I guess I am lost between local imports / remote imports / modules and pacakges concepts. I guess I just want an entry point to understand all of this where in javascript a simple require('../code.js') does the trick.
What I would like in the end is to be able to better understand go imports system, and pragmatically, how to solve my linked project dependency so I can use functions from parent "folder" (ideally without namespacing)
As the question was unclear because I did not understand what was wrong, I'll try here to synthetise what worked for me.
I followed andrey dyatlov advice by simply removing the go.mod from my cmd folder and It is possible to make this folder a standalone program without module management. So I can go run / build the command file.
Also I slightly changed imports system and reorganized my code folders by understanding how imports works a bit better (I did not practised go since a while and I am a relatively new commer to go world)
So for this last point, doing imports with github.com/eregnier/beuss does the trick, but I missed to export my "lib" functions by just capitalizing their name. Doing so let me import and use them elsewhere in the code.
For the next person that this question might help, it should be possible to see the state of the project (wip) from the link in the question to see how it works now.
Go version: 1.12.9
Here is the structure of a simple demo project:
So we have a module domain, which contains two modules: activity and person.
I would like to use domain with all nested modules in the main file => modules.go.
I know how to import domain in the main go.mod:
module modules
go 1.12
replace modules/domain v0.0.0 => ./domain
require modules/domain v0.0.0
So after that can use code from domain/domain.go, but I cannot access code from activity and person modules.
Yes, I could manually import nested modules, for example:
(main go.mod):
module modules
go 1.12
replace modules/domain v0.0.0 => ./domain
replace modules/domain/activity v0.0.0 => ./domain/activity
require (
modules/domain v0.0.0
modules/domain/activity v0.0.0
)
but I don't want to manually import all submodules.
How to configure modules to import domain with all submodules?
Yes, I could manually import nested modules [...]
but I don't want to manually import all submodules.
How to configure modules to import domain with all submodules ?
You simply cannot do this.
Some background:
There is no real notion of "sub"-module, all modules are equal.
Same for packages: all(*) packages are equal. If you want to use a package you must import it.
The layout in the filesystem does not imply any kind of technical relation between packages (e.g. net/http/cookiejar is as much related to net/http as it is related crypto/md5: not at all).
Being part of a module doesn't mean much: Modules are just sets of packages versioned together and doesn't add any additional relation between these packages.
The rule is dead simple: If you want to import a package you have to import it.
The fact that there are no magical wildcard imports might seem annoying (see below) but prevents unintended imports: Adding a package doesn't magically import it (and execute its init functions!).
In real live having to import all packages is not that annoying because "normal" Go code doesn't use tiny packages. This is a common mistake for someone indoctrinated by Java/C#/PHP/Node "project architectures" with lots of folders and tiny classes: Don't do that in Go. It is not helping. It often even leads to circular import and hurts.
Same for modules. A Go module is probably not what you think it is. A module is a set of packages versioned together. I doubt there is a reason to provide different versions for package person and package domain at the same time. (It is allowed to have several modules in one repository/project, but having more than one is a very rare case, yours clearly isn't).
Best advice (in order of relevance):
Stop making every package its own module.
Stop making tiny packages.
Stop trying to mimick a source code layout ("architecture") you might be used from other languages.
(*) internal and vendored packages are an exception which does not apply to your problem.
go 1.18 has the support for the new multi-workspace feature by go.work files.
for more info, check golang workspace proposal
I faced a similar situation with Go 1.20. Here is How I solved it.
Package
I have used a repository as package in my Go project. This repository is uses a sub-module.
Here is the .gitmodules file inside my repository my-repository/services:
Change
What I was missing:
After the introduction of Workspace, submodules were treated differently. So now I need to create a go.work file. Needed to be placed at same level as go.mod.
go.work
go 1.20
use ./validator/validate
Usage
Now I want to use this package in my main repo. Imported it as,
This solved my issue. You can find the documentation at: Setting up your workspace
I'm writing a library in Go. I'm planning to distribute it, and with a main requirement of 'without source codes'.
For testing, I have created two workspaces like following,
WS1
bin/
pkg/linux_amd64/lib.a
src/lib/src.go
WS2
bin/
pkg/
src/main/main.go
My first workspace (WS1) is the actual dummy library, which has some utility functions. Second workspace (WS2) has main function which uses the package (lib.a) from WS1.
Everything was working good until I remove the sources from WS1. If I remove the directory /lib/src.go in WS1, I'm getting the following error during go build,
main.go:5:2: cannot find package "lib" in any of:
/usr/local/go/src/pkg/lib (from $GOROOT) ../Testing/ws1/src/lib
(from $GOPATH)
The above message indicates us that we should keep the source files as well. Precompiled binary packages alone cannot be used directly.
Based on few suggestions online, we may keep some dummy sources with timestamp value lesser than binary packages' timestamp. But, this doesn't seems to be a feasible solution for us. What happens if timestamp of the dummy sources got updated unfortunately?
I have seen similar issue discussed here,
https://github.com/golang/go/issues/2775
My Questions:
Distributing the sources is the only possibility in Golang?
Why Go is not providing a provision for using '.a' files directly?
If keeping the source is mandatory for Go, why this small thing is
not mentioned anywhere in Go? (or) Am I missing something here?
Thanks in advance for your help guys!
The Go compiler just needs the .a files. If you ship them anybody will be able to use your package without the source code.
BUT your users will have to invoke the compiler (e.g. 6g, not the go tool) manually. If you ship a myfoo.a file and a dummy source myfoo.go containing just package myfoo and the timestamp of myfoo.a is newer than that of myfoo.go (and you put everything in place) you may use the go tool.
Update: Newer version of the go tool detect deleted files and require all files (possibly empty) with the proper filenames and older timestamps in the src folder.
Managing a timestamp should not be a dealbreaker.
Don't get fooled that the go tool is Go: It is a dead convenient tool to build, test, get, whatever your Go code, but it is neither the language nor the compiler nor the linker.
BTW: There is really no point in not distributing the sources.
The binary-only packages will be available in go1.7 (August 2016) - https://tip.golang.org/doc/go1.7
This release adds experimental, minimal support for building programs using binary-only packages, packages distributed in binary form without the corresponding source code. This feature is needed in some commercial settings but is not intended to be fully integrated into the rest of the toolchain. For example, tools that assume access to complete source code will not work with such packages, and there are no plans to support such packages in the “go get” command.
The proposal is at https://github.com/golang/proposal/blob/master/design/2775-binary-only-packages.md , https://tip.golang.org/pkg/go/build/#hdr-Binary_Only_Packages has more information about the new feature.
The binary-only packages is supported in go 1.7 now.
You can only provide .a files and fake go files without source code to distribute it now.
Here is a detailed example and a script of Go1.7 binary package generator.
myframework/frameImplement.go
package myframework
import "fmt"
func Hello(name string) string {
return fmt.Sprintf("Hello, %s!", name)
}
main/main.go
package main
import (
"fmt"
"golang-binary-package-generator/myframework"
)
func main() {
fmt.Println(" start program ")
fmt.Println(" print program :", myframework.Hello("print something now"))
}
If I want to hide my framework's source code, just build it with go build -i -o $GOPATH/pkg/framework.a, then modify your source code to
//go:binary-only-package
package framework
//you can add function prototype here for go doc etc, but no necessary.
, which you can use my binary package generator(script) to help you.
The decision to drop the feature:
cmd/go: drop support for binary-only packages
I have a utility package that many other packages use. I also created some test structures that implement those interfaces. And I put them in interfaces_test.go
I'd like to be able to import those test structures in other packages in my *_test.go files.
I saw something like that in http://golang.org/src/pkg/os/export_test.go but whatever I try I get an error similar to this one:
go test something/mypackage
mypackage/ant_lat_lon_test.go:46: undefined: rutl.TestAntenner
FAIL something/mypackage [build failed]
Is there a way to do it?
Files matching *_test.go are only compiled when testing the package they are part of. If you are testing package A that uses package B, you won't have access to the _test.go code from package B.
So the two options are:
Always compile the test support code into package B.
If the test support code only depends on the exported interface of B, consider splitting it out into a separate package.