My application uses json configuration files and other resources. Where should I place them in my project hierarchy?
I could not find the answer in http://golang.org/doc/code.html (How to Write Go Code)
Upd:
The question is not about automatic distribution of resources with application but much simpler: Where should I keep my resources in project hierarchy? Is there some standard place anyone expects them to be?
There is no single correct answer, nor are there any strong conventions assumed or enforced by any Go tooling at this time.
Typically I start by assuming that the files I need are located in the same directory from where the program will be run. For instance, suppose I need conf.json for myprog.go; then both of those files live together in the same directory and it works to just run something like
go build -o myprog && ./myprog
When I deploy the code, the myprog binary and conf.json live together on the server. The run/supervisor script needs to cd to that directory and then run the program.
This also works when you have a lot of resources; for instance, if you have a webserver with JS, CSS, and images, you just assume they're relative to cwd in the code and deploy the resource directories along with the server binary.
Another alternative to assuming a cwd is to have a -conf flag which the user can use to specify a configuration file. I typically use this for distributing command-line tools and open-source server applications that require a single configuration file. You could even use an -assets flag or something to point to a whole tree of resource files, if you wanted.
Finally, one more approach is to not have any resource files. go-bindata is a useful tool that I've used for this purpose -- it just encodes some data as bytes into a Go source file. That way it's all baked into your binary. I think this method is most useful when the resource data will rarely or never change, and is pretty small. (Otherwise you're going to be shipping around huge binaries.) One (kind of silly) example of when I've used go-bindata in the past was for baking a favicon into a really simple server which didn't otherwise require any extra files besides the server binary.
For static resource it might be the most convenient solution to include them in the binary similar to resources in Java. Newer Go version, at least 1.18 are providing the //go:embed directive to include content:
import (
_ "embed"
)
//go:embed myfile.txt
var myfile string
You can now use myfile in your code. E.g. IntelliJ provides also support for this.
There are also other options to include the content, e.g. as binary or dynamically, see the link.
Related
My project is currently structured such that the HTML templates are in the root of the project within a templates/ directory.
My HTTP request handlers are several levels down the file system tree.
go.mod
templates/webpp/*.html
templates/admin-dashboard/*html
cmd/webapp/main.go
cmd/admin-dashboard/main.go
app/webapp/handlers/handler.go
Ideally, I would like to be able to use go:embed in the following way:
//go:embed ../../templates
or
///go:embed $PROJECT_ROOT/templates
But unfortunately neither of these methodologies exist. The go doc embed says:
The patterns are interpreted relative to the package
directory containing the source file. The path separator is a forward slash,
even on Windows systems. Patterns may not contain ‘.’ or ‘..’ or empty path
elements, nor may they begin or end with a slash.
I could relocate the templates (something I'd rather not do since in some modes my webapp expects to see them relative to the running binary). e.g.
webapp <-- binary uses templates at runtime
templates/*.html <-- editable HTML files
So I could have my build script copy them in place just before compilation, but this seem unnecessarily convoluted; if I bypasses the build script or forget or mess up the build sequence, I could end up with a v2 binary with v1 "old templates" embedded.
Or, I could make the templates/ directory into a package and export an embed.FS and then inject it as a dependency to my application, but now my code is re-structured just to accomodate the embedding process.
Is there any likelyhood that go:embed will allow some type of embedding relative to the project root in future, or would this raise security issues?
go:embed is a great feature, but it feels like I'm beholden to it- either forced to copy, relocate, split up my sets of templates into separate directories or adjust my application to use dependency injection.
What's the best practice?
I am writing a Go command-line tool that generates some files based on templates.
The templates are located in the Git repository alongside the code for the command-line tool itself.
I wanted to allow the following:
The binary, wherever it is called from, should always find the templates directory.
The templates directory can be overriden by the user if need be.
Since this is a Go application, I went with something like:
templateRoot := filepath.Join(
os.Getenv("GOPATH"),
"src/github.com/myuser/myproject/templates",
)
But being rather new to Go, I wonder if this approach is reliable enough: is it guaranteed that my application template will always be accessible at that path ?
What if someone vendorize my application into their own project ? Does that even make sense for a command-line tool ?
Because of 2., I obviously can't/won't use go-bindata because I want to allow for the templates to be overriden if need be.
In summary: what is a good strategy to reliably refer to non-go, static files in a Go command-line tool ?
GOPATH is used for building the application. While you could look for GOPATH and check relative locations to each GOPATH entry at runtime, you can't be sure it will exist (unless of course you make it a prerequisite for running your application).
go get itself is a convenience for developers to fetch and build a go package. It relies on having a GOPATH (though there's a default now in go1.8), and GOBIN in your PATH. Many programs require extra steps not covered by the simple go tool, and have scripts or Makefiles to do the build. If you're targeting users that aren't developers, you need to provide a way to install into standard system paths anyway.
Do what any regular program would do, and use some well-known path to locate the template files. You can certainly add some logic in your program to check for a hierarchy of locations: relative to $GOPATH, relative to the binary, working directory, $HOME, etc; just provide a list of locations to the user that your program will look for templates.
I'm playing with golang and made a tool for password generation. Initially it was intended to be used as a command line tool. I later separated the generation logic into a separate package (still same github repository) and left the main function in the root of the project. Now I also want to add a simple web frontend (nothing fancy), but I don't know how to structure the packages.
Am I supposed to put both the command line entry point as well as the web UI into their own packages in the same project (which leaves the root empty). Or maybe I should move the actual generation library to the root and the UIs in separate packages. I guess the other option is to have the UIs in separate projects on github, but they are only going to be used for this library, so it does not seem like a good idea.
I remember seeing in some projects packages named cmd for example, but never have I encountered one, with multiple front ends. Is there a go(-gettable-)way for doing this?
I agree that there's not much point in making separate projects/repositories if they're only going to be used for this library. I would just have a cmd directory with a subdirectory for each executable you're building.
Something like this:
github.com/user/project
generation
cmd
cmdline
main.go
web
main.go
The main.go files can use the functionality that you've broken out into your "generation" package.
The name of the executables produced by go build will be the name of the parent directory, so cmdline and web in this example (you would want to choose better names).
Note: you don't actually have a package cmdline or web. The files in those directories would all be in [their own separate] package main.
I have a go app which relies heavily on static resources like images and jars. I want to install that go executable in different platforms like linux, mac and windows.
I first thought of bundling the resources using https://github.com/jteeuwen/go-bindata, but since the files(~100) have size ~ 20MB or so, it takes a really long time to build the executable. I thought having a single executable is an easy way for people to download the executable and run it. But seems like that is not an effective way.
I then thought of writing a installation package for each of the platform like creating a .rpm or .deb packages? So these packages contain all the resources and puts it into some platform specific pre defined locations and the go executable can reference them. But the only thing is that I have to handle that in the go code. I have to see if it is windows then load the files from say c:\go-installs or if it is linux then load the files from say /usr/local/share/go-installs. I want the go code to be as platform agnostic as it can be.
Or is there some other strategy for this?
Thanks
Possibly does not qualify as real answer but still…
As to your point №2, one way to handle this is to exploit Go's way to do conditional compilation: you might create a set of files like res_linux.go, res_windows.go etc and put a set of the same variables in each, pointing to different locations, like
var InstallsPath = `C:\go-installs`
in res_windows.go and
var InstallsPath = `/usr/share/myapp`
in res_linux.go and so on. Then in the rest of the program just reference the res.InstallsPath variable and use the path/filepath package to construct full pathnames of actual resources.
Of course, another way to go is to do a runtime switch on runtime.GOOS variable—possibly in an init() function in one of the source files.
Pack everything in a zip archive and read your resource files from it using archive/zip. This way you'll have to distribute just two files—almost "xcopy deployment".
Note that while on Windows you could just have your executable extract the directory from the pathname of itself (os.Args[0]) and assume the resource file is located in the same directory, on POSIX platforms (GNU/Linux and *BSD etc) the resource file should still be located under /usr/share/myapp or a similar place dictated by FHS (or particular distro's rules), so some logic to locate that file will still be required.
All in all, if this is supposed to be a piece of FOSS, I'd go with the first variant to let the downstream packagers tweak the pathnames. If this is a proprietary (or just niche) software the second idea appears to be rather OK as you'll play the role of downstream packagers yourself.
I have a folder which contains an executable file (Exec.exe) and a lot of files that Exec.exe needs to run. Currently, it's pretty ugly having all of those files there when I only need to run the one executable. Is there any way to bundle them all into another executable that runs Exec.exe and also contains all of the files Exec.exe needs to run? Thanks for any help!
Yes, but I would recommend you only do it if you need to.
You can achieve this by adding your files as resources in your exe project, so they are added into the exe's binary at compile time. You can then access the files directly from your exe at runtime by using LoadResource and related functions. I'd recommend reading up on the Portable Executable (PE) file format if you're considering this route.
This is the way to do it if you, and again I stress, need to have only a single binary where you can still access your files. There are obvious downsides to doing this, such as it's much more coding to access the data as it's embedded in your application binary, and you can't easily update the files (check out resource hacker tool) without re-compiling your binary to include the new data.
If the only reason you want to do this is because it's "pretty ugly" seeing the additional files in the same directory as your exe, consider moving them into another directory, for example,
from:
MyExeDir
--myExe.exe
--myFile1.txt
--myFile2.dll
--myFile3.dat
to:
MyExeDir
--myExe.exe
--dat
----myFile1.txt
----myFile2.png
----myFile3.dat
or:
MyExeDir
--bin
----myExe.exe
--dat
----myFile1.txt
----myFile2.png
----myFile3.dat
So all the "ugly" looking files are out of the way.
Why don't you create a shortcut of "Exec.exe" and keep it somewhere handy ? If whats that you want ?
Or if you want to distribute your app, you can use Winrar/Winzip (winrar is the best) to create a compressed .exe of your entire folder, making "Exec.exe" as your startup app. Use the SFX option in winrar.