I am making a Go project containing a few packages. Those are data structures (and there will be algorithms as well). My project root looks like this:
C:.
├───array
├───binary_tree
├───heap
└───list
The thing is, I want to add like a CI. So I would have a job checking if all packages build and if all tests pass.
Unfortunately, I can't run go build on project root. I have to pass whole path to it. I mean I could write a script which calls go build X/Y/foo and then go build X/Y/bar, but the CI on GitLab (docker image) won't have those paths, it will just git clone my repo and that's it (cuz I cannot run it on a relative path from project root, but a relative path from GOPATH, so like github.com/dabljues/project_name/array). And what about the test?
So the question there would be: Can I somehow run go build and go test for all the packages in the Go project? (located in separate folders)
Use a Makefile. Define the targets and instruction to compile them.
For example define the following in a Makefile
build:
go build -o bin/main main.go
run:
go run main.go
To run just invoke : make build
Related
In some build systems like gradle or blaze I can generate code (ent or proto) on a build stage and don't add in to a repository.
Is it possible to do the same for the go build command?
Yes, if you add "go generate" as your pre-build step in CI script.
$ go generate
$ go build
$ go test
But I would recommend more practical approach: to store your generated code in your repo and check it on CI - run go generate and assert that there is no changes.
Links
https://go.dev/blog/generate
When I run dep init in project folder, the error occurs:
init failed: unable to detect the containing GOPATH: D:\projects\foo is not within a known GOPATH/src
My projects are located on another drive and not %GOPATH%/src (i.e. %USERPROFILE%\go\src).
It's a known error but it's unclear what is the solution.
How can I use dep without moving Go projects to %GOPATH%/src?
Go makes this choice so that there is nothing like a CLASSPATH (ie: Java) to deal with. You specify a $GOPATH that has a consistent src tree inside of it. If your repo makes references to particular git commits (rather than the ones checked out into $GOPATH/src/github.com/$githubUser/$githubProjectName), then those will be in the ./vendor directory of your project.
If you have a different Go project that uses a completely different set of checkouts, due to versioning issues, then you can have multiple $GOPATH values to deal with that.
How can I use dep without moving Go projects to %GOPATH%/src?
Not at all.
Go projects require that your project is within its path..
So first do a
$ go env
to find out where that is. Lets say it says /home/turgut/go
move the project that you downloaded that needs the dep to:
/home/turgut/go/src/myproject
then cd /home/turgut/go/src/myproject
and try the
dep ensure
command now.
what does go env command say your GOPATH is? Set GOPATH for your environment as per this doc
Follow steps here https://deployer.org/docs/7.x/installation
you can use this command vendor/bin/dep init instead of dep init
Background
Related question: Google Container Builder: How to install govendor dependencies during build step?
I am trying to use Google Cloud Container Builder to automate the building of my Docker containers using Build Triggers.
My code is in Go, and I have a vendor folder (checked in to Git) in my project root which contains all of my Go dependencies.
My project has four binaries that need to be Dockerized, structured as follows:
vendor/
...
program1/
program1.go
main/
main.go
Dockerfile
program2/
program2.go
main/
main.go
Dockerfile
...
Each program's Dockerfile is simple:
FROM alpine
ADD main main
ENTRYPOINT ["/main"]
I have a Build Trigger set up to track my master branch. The trigger runs the following build request (cloudbuild.yaml), which makes use of the open sourced Docker build step:
steps:
- name: 'gcr.io/cloud-builders/docker'
args: ['build', '-t', 'gcr.io/$PROJECT_ID/program1:0.1.15-$SHORT_SHA', '.']
dir: 'program1/main'
... (repeated for each program)
(images, tags omitted)
To summarize, my current build process is as follows:
Edit code.
Build each Go executable using go build. The executable is called main, and is saved in the programX/main/ directory, alongside main.go.
Commit and push code (since the main executables are tracked by Git) to my master branch.
Build Trigger makes four Docker images using the main file built in step 1.
Goal
I would like to eliminate Step 1 from my build process, so that I no longer need to compile my executables locally, and do not need to track my main executables in Git.
In sum, here is my ideal process:
Edit code, commit, push to remote.
Build Trigger compiles all four programs, builds all four images.
Relax :)
Attempted solution
I used the open source Go build step, as follows:
cloudbuild.yaml: (updated)
steps:
- id: 'build-program1'
name: 'gcr.io/cloud-builders/go'
args: ['build', '-a', '-installsuffix', 'cgo', '-ldflags', '''-w''', '-o', 'main', './main.go']
env: ['PROJECT_ROOT=/workspace', 'CGO_ENABLED=0', 'GOOS=linux']
dir: 'program1/main'
- name: 'gcr.io/cloud-builders/docker'
args: ['build', '-t', 'gcr.io/$PROJECT_ID/program1:0.1.15-$SHORT_SHA', '.']
dir: 'program1/main'
waitFor: ['build-program1']
... (repeated for each program)
(images, tags omitted)
After trying various combinations of setting PROJECT_ROOT and GOPATH in the env field of the build-programX, I kept getting the same error for every single package used in my project (file path varies):
cannot find package "github.com/acoshift/go-firebase-admin" in any of
Step #0 - "build-program1": /usr/local/go/src/github.com/acoshift/go-firebase-admin (from $GOROOT)
Step #0 - "build-program1": /workspace/auth/main/gopath/src/github.com/acoshift/go-firebase-admin (from $GOPATH)
It's not even looking for a vendor directory?
What next?
My guess is that one of the following is true:
I am not specifying the GOPATH/PROJECT_ROOT correctly in the build request file. But if so, what is the correct setting?
My project is not structured correctly.
What I am trying to do is impossible :(*
I need to make a custom build step, somehow.
The version of Go used is old - but how can I check this?
I can find no examples online of what I am trying to achieve, and I find GCP's documentation on this subject quite lacking.
Any help would be greatly appreciated!
The issue is with #1: PROJECT_ROOT refers to the desired import path of your binaries. For example, if in program1/main/main.go you import "github.com/foo/bar/program1" to get the package defined in program1/program1.go, you'd set PROJECT_ROOT=github.com/foo/bar.
Fixed the problem (but not exactly sure how...), thanks to these changes:
Set PROJECT_ROOT to my_root, such that the code I want to compile sits at my_root/program1/main/main.go (thanks to John Asmuth for his answer: https://stackoverflow.com/a/46526875/6905609)
Remove the dirs field for the Go build step
Set the -o flag to ./program1/main/main, and the final build arg to ./program1/main/main.go
Previously I was cding into the program1/main directory during the build step, and for some reason go build was looking for packages within my_root/program1/main instead of my_root. Weird!
I am creating a project in go. Since there is already a lot of things at the root of the repository of the project (doc, README.md...), I want all the source code to go in a folder src, and all the test code in a folder named test :
\go
\src
\github.com
\user
\my_project
\src
main.go
some_func.go
\test
test_main.go
test_some_func.go
\doc
README.md
But I have two issues :
The build command is not working while I am in the my_project folder. I have to go in the my_project/src to successfully run build. I want to do it from the my_project folder. How to inforce go to understand that the source for my_project is in the src code ?
Then the executable produced by the go install command is named src. How to change the name of that executable ?
I want all the source code to go in a folder src, and all the test code in a folder named test :
Go has a way that it organizes source code. Do not fight this. It is how the system works. Organize your code the way Go wants you to. Do not try to force Go to work the way you have learned working in some other language. Every language has its own ways of doing things. None of them are "correct." Like Java, Go has very specific ideas of what you're supposed to do. Do it that way. (This isn't an argument about whether Go is "right" or Go is "wrong." Go is Go, and it does things in the Go way.)
In particular, you should not create another "src" directory. There is already a "src" directory at the top of the "go" tree. If you create another, redundant, "src" directory, then the package name for your project is "github.com/user/my_project/src" which is likely not what you want.
To make the executable be named what you want, put it in a directory named what you want (probably "my_project"). Put test files with the files they test. This is how go works.
So your tree should look like:
\go
\src
\github.com
\user
\my_project
main.go
some_func.go
main_test.go
some_func_test.go
\doc
README.md
Attempts to do something other than this is going to blow up over and over again, and questions of "how do I make the build system do this other thing" will continually return "put your code in the way the build system expects."
For details on what Go expects, and how you should organize your code, see GOPATH environment variable in the Command Go documentation. Once you've built your system this way for a while, you will start to see where you can deviate from it (like creating other directories for test utilities, but not test cases). Don't start deviating until you've tried it the standard Go way.
I'm working on a Go project that has the following structure (schematic):
./lib1
./prog1
./...
./prog2
./...
Where:
lib1 is a library
prog1 and prog2 are executables (both depend on lib1)
In a nutshell: How do I create Dockerfiles for prog1 and prog2?
I tried several approaches, all to no avail:
Creating a Dockerfile in prog1/
Failed because docker cannot ADD ../lib1 to the container since it is out of the context (see http://docs.docker.com/reference/builder/#add).
Creating a Dockerfile in the root directory
Ignoring the fact that I actually need two Dockerfiles (for prog1, and for prog2), I tried placing the Dockerfile in the root directory of the project.
However, the docker image I need to use (golang:1.4.1-onbuild) fails to find any Go files (since they are in ./prog1, and not in the root):
+ exec go install -v
can't load package: package app: no buildable Go source files in /go/src/app
What is the recommended project structure for the golang:1.4.1-onbuild image?
Edit: Better asked here: https://stackoverflow.com/questions/29209202/dockerizing-gos-hello-world
Most pull the files in via their VCS or just go get instead of using ADD. You add the the source tree in the same way regardless of what tool your building, since you need the structure of $GOPATH to be the same for each.
Note, you can also name your docker file via docker build -f, so if you want to use ADD, you can have multiple DOCKERFILEs in your root directory.