How can I open files relative to my GOPATH? - go

Question

I'm using io/ioutil to read a small text file:
fileBytes, err := ioutil.ReadFile("/absolute/path/to/file.txt")
And that works fine, but this isn't exactly portable. In my case, the files I want to open are in my GOPATH, for example:
/Users/matt/Dev/go/src/github.com/mholt/mypackage/data/file.txt
Since the data folder rides right alongside the source code, I'd love to just specify the relative path:
data/file.txt
But then I get this error:
panic: open data/file.txt: no such file or directory
How can I open files using their relative path, especially if they live alongside my Go code?
(Note that my question is specifically about opening files relative to the GOPATH. Opening files using any relative path in Go is as easy as giving the relative path instead of an absolute path; files are opened relative to the compiled binary's working directory. In my case, I want to open files relative to where the binary was compiled. In hindsight, this is a bad design decision.)

Answer 1

Hmm... the path/filepath package has Abs() which does what I need (so far) though it's a bit inconvenient:
absPath, _ := filepath.Abs("../mypackage/data/file.txt")
Then I use absPath to load the file and it works fine.
Note that, in my case, the data files are in a package separate from the main package from which I'm running the program. If it was all in the same package, I'd remove the leading ../mypackage/. Since this path is obviously relative, different programs will have different structures and need this adjusted accordingly.
If there's a better way to use external resources with a Go program and keep it portable, feel free to contribute another answer.

Answer 2

this seems to work pretty well:
import "os"
import "io/ioutil"
pwd, _ := os.Getwd()
txt, _ := ioutil.ReadFile(pwd+"/path/to/file.txt")

Answer 3

I wrote gobundle to solve exactly this problem. It generates Go source code from data files, which you then compile into your binary. You can then access the file data through a VFS-like layer. It's completely portable, supports adding entire file trees, compression, etc.
The downside is that you need an intermediate step to build the Go files from the source data. I usually use make for this.
Here's how you'd iterate over all files in a bundle, reading the bytes:
for _, name := range bundle.Files() {
r, _ := bundle.Open(name)
b, _ := ioutil.ReadAll(r)
fmt.Printf("file %s has length %d\n", name, len(b))
}
You can see a real example of its use in my GeoIP package. The Makefile generates the code, and geoip.go uses the VFS.

Answer 4

Starting from Go 1.16, you can use the embed package. This allows you to embed the files in the running go program.
Given the file structure:
-- main.go
-- data
\- file.txt
You can reference the file using a go directive
package main
import (
"embed"
"fmt"
)
//go:embed data/file.txt
var content embed.FS
func main() {
text, _ := content.ReadFile("data/file.txt")
fmt.Println(string(text))
}
This program will run successfully regardless of where the program is executed. This is useful in case the file could be called from multiple different locations, for instance, from a test directory.

Answer 5

I think Alec Thomas has provided The Answer, but in my experience it isn't foolproof. One problem I had with compiling resources into the binary is that compiling may require a lot of memory depending on the size of your assets. If they're small, then it's probably nothing to worry about. In my particular scenario, a 1MB font file was causing compilation to require somewhere around 1GB of memory to compile. It was a problem because I wanted it to be go gettable on a Raspberry Pi. This was with Go 1.0; things may have improved in Go 1.1.
So in that particular case, I opt to just use the go/build package to find the source directory of the program based on the import path. Of course, this requires that your targets have a GOPATH set up and that the source is available. So it isn't an ideal solution in all cases.