FilePaths in Go - go

So this is the example from Programming in Go by Mark Summerfield.
package main
import (
"fmt"
"io/ioutil"
"os"
"path/filepath"
"strings"
)
var britishAmerican = "british-american.txt"
func init() {
dir, _ := filepath.Split(os.Args[0])
britishAmerican = filepath.Join(dir, britishAmerican)
}
func main() {
rawBytes, err := ioutil.ReadFile(britishAmerican)
if err != nil {
fmt.Println(err)
}
text := string(rawBytes)
usForBritish := make(map[string]string)
lines := strings.Split(text, "\n")
fmt.Println(lines)
for _, line := range lines {
fields := strings.Fields(line)
if len(fields) == 2 {
usForBritish[fields[0]] = fields[1]
}
}
fmt.Println(usForBritish)
}
When I run this code with the init() func commented out, it works perfectly fine. If I leave it in I get this error:
open /var/folders/l6/rdqtyrfd303dw1cz8qvlfcvc0000gn/T/go- build652175567/command-line-arguments/_obj/exe/british-american.txt: no such file or directory exit status 1
My question is, why does the init() func not grab the file from the appropriate directory?

You change the variable britishAmerican in the init function. Without init(), the program looks in the current directory (no path given, only the file name). With init(), it looks in the path where the executable is (os.Args[0]). And with go run main.go, the directory with the executable is not the current working directory.
You should use go build to build the binary and then run it, or you should tell us what you want to achieve (as written by #RoninDev).
The MCVE I've mentioned could look like this:
package main
import (
"io/ioutil"
"log"
"os"
"path/filepath"
)
var filename = "foo.txt"
func init() {
// change to true and things break
if false {
dir, _ := filepath.Split(os.Args[0])
filename = filepath.Join(dir, filename)
}
}
func main() {
// requires a file 'foo.txt' in the current directory
_, err := ioutil.ReadFile(filename)
if err != nil {
log.Fatal(err)
}
}
It can (of course) be even shorter, but this should be enough for the others in the community to see what is going on.

It looks to me like the program is expecting a file called british-american.txt in the directory that the executable is in.
That is what the code in init() does - it finds the path the the executable and constructs a path to the dictionary relative to that.
I can see from your error message that you are using go run to run the code. This makes a temporary executable in /tmp and runs that. If you leave the init() code in then it will look for the dictionary in the /tmp directory and it won't find it. If you take the init() code out it will look for the dictionary in the current directory and it will succeed.
If you want to use it as the author intended then use go build to build a binary and then run it - that will work.

Related

go run command doesn't pick up non-go files in the same path

Folder structure:
- dev.env
- go.mod
- main.go
dev.env:
ENV="DEV"
PASSWORD="DEV!##$%"
main.go:
package main
import (
"fmt"
"log"
"os"
"path/filepath"
"github.com/joho/godotenv"
)
var environment string
func init() {
fmt.Println("ENV: ", environment)
dir, err := filepath.Abs(filepath.Dir(os.Args[0]))
if err != nil {
log.Fatal(err)
}
devEnvPath := filepath.Join(dir, "dev.env")
_ = godotenv.Load(devEnvPath)
}
func main() {
fmt.Println("PASSWORD", os.Getenv("PASSWORD"))
}
Command:
go run -ldflags="-X 'main.environment=DEV'" .
Output:
ENV: DEV
PASSWORD
It works with go build but curious why it doesn't work with go run
As a general rule, don't use go run unless in the most trivial of use cases - it is the most common footgun in the Go community.
As #Marc pointed out the error stems from the go run binary is built in a temporary directory. To keep things simple, just use a relative path in your directory. This will work for both go build & go run:
dir := "."
devEnvPath := filepath.Join(dir, "dev.env")
err := godotenv.Load(devEnvPath)
if err != nil {
log.Fatal(err)
}
As of Go 1.16, the simplest way to access files from the directory containing a package's source code is to use //go:embed and the embed package to embed those files into the compiled binary:
//go:embed dev.env
var devEnv string
func init() {
m, err := godotenv.Unmarshal(devEnv)
…
}

Get current file name after creating executable

I can get the current directory and the name of the current .go file but if i build a .exe from the file with go build then i still get the .go name.
.../Project/Test/testfile.go
After building testfile.go with go build i have these:
1) .../Project/Test/testfile.go
2) .../Project/Test/main.exe
When i execute main.exe i get testfile But i want to get the main name after executing main.exe
package main
import (
"errors"
"fmt"
"path/filepath"
"runtime"
"strings"
)
func GetFileName() string {
_, fpath, _, ok := runtime.Caller(0)
if !ok {
err := errors.New("failed to get filename")
panic(err)
}
filename := filepath.Base(fpath)
filename = strings.Replace(filename, ".go", "", 1)
return filename
}
func main() {
fmt.Print(GetFileName())
}
The os package provides os.Args[0] to obtain the executable name, or if you want the whole path you could use os.Executable, as #Peter has suggested

go test load files from root directory? [duplicate]

I'm writing some unit tests for my application in Go. The tests fail however because it cannot find the configuration files. Normally the binary looks for the configuration files in the working directory under the path conf/*.conf.
I figured that browsing to the directory that has conf/ and running go test in it would solve it, but it still reports that the file system cannot find the path specified.
How can I tell go test to use a certain directory as the working directory so that the tests may actually be executed?
You may be able to use the Caller to get the path to the current test source file, like this:
package sample
import (
"testing"
"runtime"
"fmt"
)
func TestGetFilename(t *testing.T) {
_, filename, _, _ := runtime.Caller(0)
t.Logf("Current test filename: %s", filename)
}
I do not believe this is possible. I have not been able to find documentation stating this explicitly, but I believe go test always uses the package directory (containing the go source files) as the working directory.
As a workaround, I compiled the test and execute the test from the current directory.
go test -c && ./<mypackage>.test
Or, if you want a generic command that you can use, you can rename the test file with -o option.
go test -c -o xyz.test && ./xyz.test
While not really convenient, you can always pass it as a command line variable, for example :
package blah_test
import (
"flag"
"fmt"
"os"
"testing"
)
var (
cwd_arg = flag.String("cwd", "", "set cwd")
)
func init() {
flag.Parse()
if *cwd_arg != "" {
if err := os.Chdir(*cwd_arg); err != nil {
fmt.Println("Chdir error:", err)
}
}
}
func TestBlah(t *testing.T) {
t.Errorf("cwd: %+q", *cwd_arg)
}
Then run it like :
┌─ oneofone#Oa [/tmp]
└──➜ go test . -cwd="$PWD"
--- FAIL: TestBlah (0.00 seconds)
blah_test.go:16: cwd: "/tmp"
No matter where the work directory is. It must be under your project Dir. So my solution is
wd, _ := os.Getwd()
for !strings.HasSuffix(wd, "<yourProjectDirName>") {
wd = filepath.Dir(wd)
}
raw, err := ioutil.ReadFile(fmt.Sprintf("%s/src/conf/conf.dev.json", wd))
Your path should always start from your project Dir. Every time you read the file in a package and accessed by main.go or your another package unit test. It will always work.
You can use the os package.
You would want to do something like this
func TestMyFunction(t *testing.T) {
os.Chdir("./path")
//TEST FUNCTION
os.Chdir("..")
}
There are several possibilities in the os package.
To add init function into *_test.go under your test package.
Test package will run this function before test function start.
func init() {
_, filename, _, _ := runtime.Caller(0)
// The ".." may change depending on you folder structure
dir := path.Join(path.Dir(filename), "..")
err := os.Chdir(dir)
if err != nil {
panic(err)
}
}
I know this is an old question but I had the same problem trying to use migrations for the database on my tests, and maybe this solution helps someone.
Since there is no native way of getting the project directory, you could identify some file or directory that you know it's only in the root of the project (in my case, it was the relative directory database/migrations). Once you have this unique relative directory, you could have a function like the following to obtain the project root directory. It just gets the current working directory (assuming it's inside the project's directory) and starts to navigate all the way up until it finds a dir that has the relative directory you know it's on the root of the project:
func FindMyRootDir() string {
workingDirectory, err := os.Getwd()
if err != nil {
panic(err)
}
lastDir := workingDirectory
myUniqueRelativePath := "database/migrations"
for {
currentPath := fmt.Sprintf("%s/%s", lastDir, myUniqueRelativePath)
fi, err := os.Stat(currentPath)
if err == nil {
switch mode := fi.Mode(); {
case mode.IsDir():
return currentPath
}
}
newDir := filepath.Dir(lastDir)
// Ooops, we couldn't find the root dir. Check that your "myUniqueRelativePath" really exists
if newDir == "/" || newDir == lastDir {
return ""
}
lastDir = newDir
}
}
Of course it's not the most beautiful solution, but it works.
I've had a similar problem and found the solution on this blog
Basically you can change the folder that the test is running using a similar function:
package main
import (
"os"
"path"
"runtime"
)
func MakeFunctionRunOnRootFolder() {
_, filename, _, _ := runtime.Caller(0)
// The ".." may change depending on you folder structure
dir := path.Join(path.Dir(filename), "..")
err := os.Chdir(dir)
if err != nil {
panic(err)
}
}
Go 1.20 is getting new -C arguments for "go subcommands" so this should help:
go test -C directory/ ...
It's a common practice in Go to place test fixtures in same package inside testdata folder.
Some examples from standard library:
debug/elf
net/http
image
Also, there is a post from Dave Cheney, where he suggests following code:
f, err := os.Open("testdata/somefixture.json")
I currently use a neat solution for this problem, instead of opening the file directly by calling os.Open(), I use the embed package in a smart way:
First I create a global variable in my root package called:
//go:embed config/* otherdirectories/*
var RootFS embed.FS
Then I just open the files inside my tests by using this global variable, e.g.:
func TestOpenConfig(t *testing.T) {
configFile, err := rootpkg.RootFS.ReadFile("config/env")
if err != nil {
t.Fatalf("unable to open config/env file: %s", err)
}
if string(configFile) != "FOO=bar\n" {
t.Fatalf("config file contents differ from expected: %s", string(configFile))
}
}
This is a neat trick because now you can always work with relative paths from your root package, which is what I used to do in other programming languages.
Of course, this has the restriction that you will need to import your root package, which depending on your package layout might not be ideal because of cyclic imports. If this is your case you might just create a embed.go file inside the config directory itself and call
your configs by name.
One other drawback is that you are embedding test files in your binary, this is probably ok if your test files are not very big, like megabytes big, so I don't really mind this issue.
I also created a repository for illustrating this solution:
https://github.com/VinGarcia/golang-reading-files-from-tests
I would use an Environment Variable for the location of your application. It seems to be the best way when running go tools, as test programs can be run from a temporary location.
// get home dir of app, use MYAPPHOME env var if present, else executable dir.
func exeDir() string {
dir, exists := os.LookupEnv("MYAPPHOME")
if exists {
return dir
} else {
ex, err := os.Executable()
if err != nil {
panic(err)
}
exPath := path.Dir(ex)
return exPath
}
}

Finding imports and dependencies of a go program

The go list -json command run from the command line will tell you the imports and dependencies of a go program ( in json format). Is there a way to get this information from within a go program I.e at runtime, either by running the 'go list' command somehow or another way?
The following code uses the go/build to get the imports for the application in the current working directory.
p, err := build.Default.Import(".", ".", 0)
if err != nil {
// handle error
}
for _, i := range p.Imports {
fmt.Println(i)
}
You can build a list of all dependencies using a simple recursive function.
To get the imports for a specific path, use:
p, err := build.Default.Import(path, ".", 0)
if err != nil {
// handle error
}
for _, i := range p.Imports {
fmt.Println(i)
}
I don't think you can do it without using the go binary since go needs to analyze your source code.
It's pretty easy to do but it must have access to go and your source code at run time. Heres a quick example:
package main
import (
"encoding/json"
"fmt"
"os/exec"
)
func main() {
cmd := exec.Command("go", "list", "-json")
stdout, err := cmd.Output()
if err != nil {
println(err.Error())
return
}
var list GoList
err = json.Unmarshal(stdout, &list)
for _, d := range list.Deps {
fmt.Printf(" - %s\n", d)
}
}
type GoList struct {
Dir string
ImportPath string
Name string
Target string
Stale bool
Root string
GoFiles []string
Imports []string
Deps []string
}
No, I don't think this is possible without the source in a reliable way. Go binaries on different platforms, compiled with different compilers may or may not have (or may not have in the future) these informations compiled in.
But as Go programs are compiled anyway: Why not record this information while you do have access to the source code?

How to get the directory of the currently running file?

In nodejs I use __dirname . What is the equivalent of this in Golang?
I have googled and found out this article http://andrewbrookins.com/tech/golang-get-directory-of-the-current-file/ . Where he uses below code
_, filename, _, _ := runtime.Caller(1)
f, err := os.Open(path.Join(path.Dir(filename), "data.csv"))
But is it the right way or idiomatic way to do in Golang?
EDIT: As of Go 1.8 (Released February 2017) the recommended way of doing this is with os.Executable:
func Executable() (string, error)
Executable returns the path name for the executable that started the current process. There is no guarantee that the path is still pointing to the correct executable. If a symlink was used to start the process, depending on the operating system, the result might be the symlink or the path it pointed to. If a stable result is needed, path/filepath.EvalSymlinks might help.
To get just the directory of the executable you can use path/filepath.Dir.
Example:
package main
import (
"fmt"
"os"
"path/filepath"
)
func main() {
ex, err := os.Executable()
if err != nil {
panic(err)
}
exPath := filepath.Dir(ex)
fmt.Println(exPath)
}
OLD ANSWER:
You should be able to use os.Getwd
func Getwd() (pwd string, err error)
Getwd returns a rooted path name corresponding to the current directory. If the current directory can be reached via multiple paths (due to symbolic links), Getwd may return any one of them.
For example:
package main
import (
"fmt"
"os"
)
func main() {
pwd, err := os.Getwd()
if err != nil {
fmt.Println(err)
os.Exit(1)
}
fmt.Println(pwd)
}
This should do it:
import (
"fmt"
"log"
"os"
"path/filepath"
)
func main() {
dir, err := filepath.Abs(filepath.Dir(os.Args[0]))
if err != nil {
log.Fatal(err)
}
fmt.Println(dir)
}
Use package osext
It's providing function ExecutableFolder() that returns an absolute path to folder where the currently running program executable reside (useful for cron jobs). It's cross platform.
Online documentation
package main
import (
"github.com/kardianos/osext"
"fmt"
"log"
)
func main() {
folderPath, err := osext.ExecutableFolder()
if err != nil {
log.Fatal(err)
}
fmt.Println(folderPath)
}
I came from Node.js to Go. The Node.js equivalent to __dirname in Go is:
_, filename, _, ok := runtime.Caller(0)
if !ok {
return errors.New("unable to get the current filename")
}
dirname := filepath.Dir(filename)
Some other mentions in this thread and why they're wrong:
os.Executable() will give you the filepath of the currently running executable. This is equivalent to process.argv[0] in Node. This is not true if you want to take the __dirname of a sub-package.
os.Getwd() will give you the current working directory. This is the equivalent to process.cwd() in Node. This will be wrong when you run your program from another directory.
Lastly, I'd recommend against pulling in a third-party package for this use case. Here's a package you can use:
package current
// Filename is the __filename equivalent
func Filename() (string, error) {
_, filename, _, ok := runtime.Caller(1)
if !ok {
return "", errors.New("unable to get the current filename")
}
return filename, nil
}
// Dirname is the __dirname equivalent
func Dirname() (string, error) {
filename, err := Filename()
if err != nil {
return "", err
}
return filepath.Dir(filename), nil
}
Note that I've adjusted runtime.Caller(1) to 1 because we want to get the directory of the package that called current.Dirname(), not the directory containing the current package.
filepath.Abs("./")
Abs returns an absolute representation of path. If the path is not
absolute it will be joined with the current working directory to turn
it into an absolute path.
As stated in the comment, this returns the directory which is currently active.
os.Executable: https://tip.golang.org/pkg/os/#Executable
filepath.EvalSymlinks: https://golang.org/pkg/path/filepath/#EvalSymlinks
Full Demo:
package main
import (
"fmt"
"os"
"path/filepath"
)
func main() {
var dirAbsPath string
ex, err := os.Executable()
if err == nil {
dirAbsPath = filepath.Dir(ex)
fmt.Println(dirAbsPath)
return
}
exReal, err := filepath.EvalSymlinks(ex)
if err != nil {
panic(err)
}
dirAbsPath = filepath.Dir(exReal)
fmt.Println(dirAbsPath)
}
if you use this way :
dir, err := filepath.Abs(filepath.Dir(os.Args[0]))
if err != nil {
log.Fatal(err)
}
fmt.Println(dir)
you will get the /tmp path when you are running program using some IDE like GoLand because the executable will save and run from /tmp
i think the best way for getting the currentWorking Directory or '.' is :
import(
"os"
"fmt"
"log"
)
func main() {
dir, err := os.Getwd()
if err != nil {
log.Fatal(err)
}
fmt.Println(dir)
}
the os.Getwd() function will return the current working directory.
and its all without using of any external library :D
If you use package osext by kardianos and you need to test locally, like Derek Dowling commented:
This works fine until you'd like to use it with go run main.go for
local development. Not sure how best to get around that without
building an executable beforehand each time.
The solution to this is to make a gorun.exe utility instead of using go run. The gorun.exe utility would compile the project using "go build", then run it right after, in the normal directory of your project.
I had this issue with other compilers and found myself making these utilities since they are not shipped with the compiler... it is especially arcane with tools like C where you have to compile and link and then run it (too much work).
If anyone likes my idea of gorun.exe (or elf) I will likely upload it to github soon..
Sorry, this answer is meant as a comment, but I cannot comment due to me not having a reputation big enough yet.
Alternatively, "go run" could be modified (if it does not have this feature already) to have a parameter such as "go run -notemp" to not run the program in a temporary directory (or something similar). But I would prefer just typing out gorun or "gor" as it is shorter than a convoluted parameter. Gorun.exe or gor.exe would need to be installed in the same directory as your go compiler
Implementing gorun.exe (or gor.exe) would be trivial, as I have done it with other compilers in only a few lines of code... (famous last words ;-)
Sometimes this is enough, the first argument will always be the file path
package main
import (
"fmt"
"os"
)
func main() {
fmt.Println(os.Args[0])
// or
dir, _ := os.Getwd()
fmt.Println(dir)
}
dir, err := os.Getwd()
if err != nil {
fmt.Println(err)
}
this is for golang version: go version go1.13.7 linux/amd64
works for me, for go run main.go. If I run go build -o fileName, and put the final executable in some other folder, then that path is given while running the executable.
Do not use the "Answer recommended by Go Language" with runtime.Caller(0).
That works when you go build or go install a program, because you are re-compiling it.
But when you go build a program and then distribute it (copy) on your colleagues' workstations (who don't have Go, and just need the executable), the result of runtime.Caller(0) would still be the path of where you built it (from your computer).
Ie a path which would likely not exist on their own computer.
os.Args[0] or, better, os.Executable() (mentioned here) and kardianos/osext (mentioned here and here), are more reliable.
None of the answers here worked for me, at least on go version go1.16.2 darwin/amd64. This is the only thing close to the __dirname functionality in node
This was posted by goland engineer Daniil Maslov in the jetbrains forums
pasted below for easier reading:
The trick is actually very simple and is to get the current executing and add .. to the project root.
Create a new directory and file like testing_init.go with the following content:
package testing_init
import (
"os"
"path"
"runtime"
)
func init() {
_, filename, _, _ := runtime.Caller(0)
dir := path.Join(path.Dir(filename), "..")
err := os.Chdir(dir)
if err != nil {
panic(err)
}
}
After that, just import the package into any of the test files:
package main_test
import (
_ "project/testing_init"
)
Now you can specify paths from the project root
// GetCurrentDir
func GetCurrentDir() string {
p, _ := os.Getwd()
return p
}
// GetParentDir
func GetParentDir(dirctory string) string {
return substr(dirctory, 0, strings.LastIndex(dirctory, "/"))
}
func substr(s string, pos, length int) string {
runes := []rune(s)
l := pos + length
if l > len(runes) {
l = len(runes)
}
return string(runes[pos:l])
}
If your file is not in the main package then the above answers won't work
I tried different approaches to find find the directory of the currently running file but failed.
The best possible answer is in the question itself this is how I find the current working directory of the file which is not in the main package.
_, filename, _, _ := runtime.Caller(1)
pwd := path.Dir(filename)
Gustavo Niemeyer's answer is great.
But in Windows, runtime proc is mostly in another dir, like this:
"C:\Users\XXX\AppData\Local\Temp"
If you use relative file path, like "/config/api.yaml", this will use your project path where your code exists.

Resources