According to the Go documentation they would like you to have a workspace that you should put all their projects in.1 However, as far as I can tell, this all falls apart as soon as you want to make a project that does not use Go exclusively.
Take a project where it is made up of many micoservices for example. Lets say that it is structured like this:
app/
authentication/ (Using rust)
users/ (Using NodeJS)
posts/ (Using Go)
Only one part of the app would be written in Go, and that part is nested in a subdirectory of the app. How would I apply the Go workspace philosophy to this situation?
https://golang.org/doc/code.html#Workspaces
Using a different GOPATH per project is a very good and simple approach. In my experience this also works better than vendor since you can also install binaries and keep them on different versions.
vg is a simple tool that helps managing workspaces, it integrates with your shell and detects automatically workspaces when you cd them.
Disclaimer: I am one of the authors of the tool.
As of Go 1.11, go now has modules. Amongst other things, modules enable you to have isolated source trees (with any number of packages and their own dependencies) outside of your $GOPATH.
You create a new module by running go mod init <module name> (you must be outside of $GOPATH/src to do this). This will create a go.mod file in the current folder, and any go command you run in that folder (or any folder beneath) will use that folder as your project root.
You can read more about using go modules as workspaces in this post: https://aliceh75.github.io/using-modules-for-workspaces-in-golang (disclaimer: I wrote it), and you can read more about Go modules on the Go Modules Wiki:
https://github.com/golang/go/wiki/Modules
You can put app/ in $GOPATH/src. Then whenever you're ready to build, you specify the path of your source files, relative to where they are in GOPATH.
For example:
if your app source is in $GOPATH/src/app/ and your .go files are in $GOPATH/src/app/posts/ then you can build a source (lets say posts.go in app/posts/) with go build $GOPATH/src/app/posts/posts.go or better go build posts/posts.go with app/ as your current working directory.
just set GOPATH according to your go files:
GOPATH=$PROJECT_PATH/app/posts
then put your source codes under
$PROJECT_PATH/app/posts/src/package
Related
I have a project with mutliple main methods.
When running go build program1/main1.go which has a different set of dependencies than program2/main2.go, my first go build seems to alter my go.mod file and removes the dependencies it thinks it does not need. Yet main2 would need these dependencies.
I've tried using go build ... but that also created a different set of dependencies. Specifically, it seems like all the //indirect dependencies get removed and cause program2 to fail.
Is there a way of running go build or go run without updating the go.mod file? Using go build -mod=readonly program1/main1.go it tells me that it fails because the dependencies need to be updated..
I believe you're looking for submodules. See this walktrhough.
TLDR: you'll want a separate go.mod in each of your tools's cmd dir, and you can use the replace directive to point dependncies from these tools to your local module.
This Go Issue and others linked from it suggest that figuring out "the one right way" to do this is still WIP, though I think your use case is simple enough.
Using submodules is a way to nest multiple Go module projects you can edit.
But Go 1.18 might include the notion of Go workspace, which means you don't need submodules anymore: one Go project can include multiple modules you can edit.
See golang/go issue 45713: "proposal: cmd/go: add a workspace mode " and its design document.
Background
Users often want to make changes across multiple modules: for instance, to introduce a new interface in a package in one module along with a usage of that interface in another module.
Normally, the go command recognizes a single "main" module the user can edit.
Other modules are read-only and are loaded from the module cache.
The go mod replace directive is the exception: it allows users to replace the resolved version of a module with a working version on disk.
But working with the replace directive can often be awkward: each module developer might have working versions at different location on disk, so having the directive in a file that needs to be distributed with the module isn't a good fit for all use cases.
Proposal
This proposal describes a new workspace mode in the go command for editing multiple modules.
The presence of a go.work file in the working directory or a containing directory will put the go command into workspace mode.
The go.work file specifies a set of local modules that comprise a workspace. When invoked in workspace mode, the go command will always select these modules and a consistent set of dependencies.
Main modules: The module the user is working in.
Before this proposal, this is the single module containing the directory where the go command is invoked. This module is used as the starting point when running MVS.
This proposal proposes allowing multiple main modules.
See for instance CL 334934 (CL = Change List)
[dev.cmdgo] cmd/go: add the workspace mode
This change adds the outline of the implementation of the workspace mode.
The go command will now locate go.work files, and read them to determine
which modules are in the workspace.
It will then put those modules in the root of the workspace when building the build list.
It supports building, running, testing, and listing in workspaces.
You can initiate a multiple-module project with go mod initwork
Again, this is not before Go 1.18 (Q1 2022) and will probably be opt-in in Go 1.19 (Q3 2022).
Say I wanted to integrate a go project into another repo, or wanted to develop a temporary project on my desktop. My current understanding is that in order to develop anything, it must be within my gopath.
Is this unique to Go?
Is there a way to, for example, go init a folder and use that as a self contained project folder?
As I illustrated in Automatically defining GOPATH on a per project basis, it depends on your IDE.
Visual Studio Code will detect automatically a Go project structure (myProject/src) and set a GOPATH just for that project.
See "GOPATH in the VS Code Go extension"
You would find the same feature in GoLand, the IntelliJ editor (not free).
See this thread.
Global GOPATH: this allows you to configure a GOPATH (which can have multiple values, of course) for all the projects you are using the IDE with. You'll configure it once and reuse it across the projects
Project GOPATH: has the same properties as the Global GOPATH but this one applies only to the current project. This means you can configure different GOPATH values for different projects.
I'm new to Golang. As I understand, when you want to create a new Go project, we just need to create a directory. Then we point the environment variable GOPATH to this directory. Inside this directory, we create three subdirectories pkg, src and bin. Then when we execute go get ..., the third-party package will be installed in the pkg subdirectory. Later if I want to create another Go project, I create a new dir called project2 and point GOPATH to project2. At this time go get ... will download third-party package in the pkg subdirectory of project2. My question is, whether Go has a central repository? If not, the same package will be downloaded twice if they are used in two different projects. Is that true?
I guess now there is https://gocenter.jfrog.com/
More info in this blog https://jfrog.com/blog/go-at-full-speed-with-gocenter
There is no central repository of go packages. Go always is looking for packages either in GOPATH or GOROOT. go get simply downloads packages using git or mercurial. I recommend you to read
https://golang.org/doc/code.html
and https://peter.bourgon.org/go-best-practices-2016/#repository-structure
GOPATH simply tells go compiler where to search for src, pkg directories.
Later if I want to create another Go project, I create a new dir called project2 and point GOPATH to project2
…
My question is, whether Go has a central repository? If not, the same package will be downloaded twice if they are used in two different projects. Is that true?
No, there is no central repository for Go code. However, it is also not true that the packages will always be downloaded twice.
The misconception here is that GOPATH points to an individual project: it does not. Instead, GOPATH points to an environment where all of your packages live; it is where go get will download packages, and where go build will look for packages when building.
Instead of changing GOPATH for every project, you should set GOPATH once and put all of your projects in $GOPATH/src/ (your projects don't contain an src/ directory, they go in the src/ directory).
So for example, the entire tree might look like:
$GOPATH/src/bitbucket.org/ (or GitHub, or your website, or whatever)
├── YourProject
└── AnotherProject
Update
It is worth noting that this answer is no longer correct. Now that Go Modules are the normal versioning mechanism for Go code and $GOPATH is being phased out, a central proxy has been setup that routes all requests for packages through Google servers where the various tagged versions of the package can be cached. A separate checksum database keeps hashes for every package that are audit-able and can help you detect if a package author has changed an already released tag. All of this isn't a central repository in the same sense that PyPi (in the Python world) or NPM (for JavaScript) are a repo: the packages are still fetched from their source control, but because all packages are routed through the proxy by default it serves a similar purpose. For more information see https://proxy.golang.org/
Recently, a new site that collects information about Go packages has emerged:
https://go.dev/.
go.dev is the hub for Go users providing centralized and curated resources from across the Go ecosystem.
It is an official companion website to golang.org. It does not qualify for a repository, such as cpan, nmpjs, nuget or crates. For external packages, it simply links to their respective Github pages.
Go.dev is currently in MVP status. We’re proud of what we’ve built and excited to share it with the community. We hope you find value and joy in using go.dev. Go.dev only has a small portion of features we intend to build, and we are actively seeking feedback
But as is written it the about page, it is still in early development. Maybe one day (hopefully) it shall become a fully featured code repository.
I am using GoDep to resolve a project dependencies.
My problem is that repositories for dependencies maight be removed and my project wouldn't build.
I am trying to find any solution to store dependencies at Artifactory or another solution.
Please advice.
Regards.
Okay so GoDeps may be the standard way of doing this, but I usually found it a bit complicated. In my opinion, use a Makefile which sets a custom GoPath and just include dependencies with your code (remove their .git folder). This way the version freezes and no one needs to do a godep restore or something similar.
You can make recipes like make deploy that builds your code, runs GoFmt, cleans the pkg files, installs it to your custom GoPath bin/ and then you just go and run the binary.
You can have another one like make install that will install any missing dependencies.
I've managed to create a watch using this on my Makefile to keep on looking for changes on a linux based system using inotify-tools and call rebuild.
Internally all commands will be using standard go commands but you'll get rid of the GoDeps and maintaining JSON. To upgrade a dependency, it may be a bit of a problem as you'd have to manually copy the whole directory into your custom path and remove the .git/ folder.
Our company uses this method and seems to work quite nice for us.
Plus this method basically gets you away the $GOPATH/src/github.com/repoName/ kind of paths.
If i seem unclear, let me know, I'll add a gist on github.
Suppose I have a project "MyFramework" that has some code, which is used across quite a few solutions. Each solution has its own source control management (SVN).
MyFramework is an internal product and doesn't have a formal release schedule, and same goes for the solutions.
I'd prefer not having to build and copy the DLLs to all 12 projects, i.e. new developers should to be able to just do a svn-checkout, and get to work.
What is the best way to share MyFramework across all these solutions?
Since you mention SVN, you could use externals to "import" the framework project into the working copy of each solution that uses it. This would lead to a layout like this:
C:\Projects
MyFramework
MyFramework.csproj
<MyFramework files>
SolutionA
SolutionA.sln
ProjectA1
<ProjectA1 files>
MyFramework <-- this is a svn:externals definition to "import" MyFramework
MyFramework.csproj
<MyFramework files>
With this solution, you have the source code of MyFramework available in each solution that uses it. The advantage is, that you can change the source code of MyFramework from within each of these solutions (without having to switch to a different project).
BUT: at the same time this is also a huge disadvantage, since it makes it very easy to break MyFramwork for some solutions when modifiying it for another.
For this reason, I have recently dropped that approach and am now treating our framework projects as a completely separate solution/product (with their own release-schedule). All other solutions then include a specific version of the binaries of the framework projects.
This ensures that a change made to the framework libraries does not break any solution that is reusing a library. For each solution, I can now decide when I want to update to a newer version of the framework libraries.
That sounds like a disaster... how do you cope with developers undoing/breaking the work of others...
If I were you, I'd put MyFrameWork in a completely seperate solution. When a developer wants to develop one of the 12 projects, he opens that project solution in one IDE & opens MyFrameWork in a seperate IDE.
If you strong name your MyFramework Assemby & GAC it, and reference it in your other projects, then the "Copying DLLs" won't be an issue.
You just Build MyFrameWork (and a PostBuild event can run GacUtil to put it in the asssembly cache) and then Build your other Project.
The "best way" will depend on your environment. I worked in a TFS-based, continuous integration environment, where the nightly build deployed the binaries to a share. All the dependent projects referred to the share. When this got slow, I built some tools to permit developers to have a local copy of the shared binaries, without changing the project files.
Does work in any of the 12 solutions regularly require changes to the "framework" code?
If so your framework is probably new and just being created, so I'd just include the framework project in all of the solutions. After all, if work dictates that you have to change the framework code, it should be easy to do so.
Since changes in the framework made from one solution will affect all the other solutions, breaks will happen, and you will have to deal with them.
Once you rarely have to change the framework as you work in the solutions (this should be your goal) then I'd include a reference to a framework dll instead, and update the dll in each solution only as needed.
svn:externals will take care of this nicely if you follow a few rules.
First, it's safer if you use relative URIs (starting with a ^ character) for svn:externals definitions and put the projects in the same repository if possible. This way the definitions will remain valid even if the subversion server is moved to a new URL.
Second, make sure you follow the following hint from the SVN book. Use PEG-REVs in your svn:externals definitions to avoid random breakage and unstable tags:
You should seriously consider using
explicit revision numbers in all of
your externals definitions. Doing so
means that you get to decide when to
pull down a different snapshot of
external information, and exactly
which snapshot to pull. Besides
avoiding the surprise of getting
changes to third-party repositories
that you might not have any control
over, using explicit revision numbers
also means that as you backdate your
working copy to a previous revision,
your externals definitions will also
revert to the way they looked in that
previous revision ...
I agree with another poster - that sounds like trouble. But if you can't want to do it the "right way" I can think of two other ways to do it. We used something similar to number 1 below. (for native C++ app)
a script or batch file or other process that is run that does a get and a build of the dependency. (just once) This is built/executed only if there are no changes in the repo. You will need to know what tag/branch/version to get. You can use a bat file as a prebuild step in your project files.
Keep the binaries in the repo (not a good idea). Even in this case the dependent projects have to do a get and have to know about what version to get.
Eventually what we tried to do for our project(s) was mimic how we use and refer to 3rd party libraries.
What you can do is create a release package for the dependency that sets up a path env variable to itself. I would allow multiple versions of it to exist on the machine and then the dependent projects link/reference specific versions.
Something like
$(PROJ_A_ROOT) = c:\mystuff\libraryA
$(PROJ_A_VER_X) = %PROJ_A_ROOT%\VER_X
and then reference the version you want in the dependent solutions either by specific name, or using the version env var.
Not pretty, but it works.
A scalable solution is to do svn-external on the solution directory so that your imported projects appear parallel to your other projects. Reasons for this are given below.
Using a separate sub-directory for "imported" projects, e.g. externals, via svn-external seems like a good idea until you have non-trivial dependencies between projects. For example, suppose project A depends on project on project B, and project B on project C. If you then have a solution S with project A, you'll end up with the following directory structure:
# BAD SOLUTION #
S
+---S.sln
+---A
| \---A.csproj
\---externals
+---B <--- A's dependency
| \---B.csproj
\---externals
\---C <--- B's dependency
\---C.csproj
Using this technique, you may even end up having multiple copies of a single project in your tree. This is clearly not what you want.
Furthermore, if your projects use NuGet dependencies, they normally get loaded within packages top-level directory. This means that NuGet references of projects within externals sub-directory will be broken.
Also, if you use Git in addition to SVN, a recommended way of tracking changes is to have a separate Git repository for each project, and then a separate Git repository for the solution that uses git submodule for the projects within. If a Git submodule is not an immediate sub-directory of the parent module, then Git submodule command will make a clone that is an immediate sub-directory.
Another benefit of having all projects on the same layer is that you can then create a "super-solution", which contains projects from all of your solutions (tracked via Git or svn-external), which in turn allows you to check with a single Solution-rebuild that any change you made to a single project is consistent with all other projects.
# GOOD SOLUTION #
S
+---S.sln
+---A
| \---A.csproj
+---B <--- A's dependency
| \---B.csproj
\---C <--- B's dependency
\---C.csproj