This question already has an answer here:
Write and Read File with same *os.File in Go
(1 answer)
Closed 8 months ago.
This post was edited and submitted for review 8 months ago and failed to reopen the post:
Original close reason(s) were not resolved
In Go, I am trying to write data to a temp file that I then turn around and read but have not been successful. Below is a stripped down test program. I have verified that the data are being written to the file by inspecting the temporary file. So, at least I know that data are making it into the file. I just am then unable to read it out.
Thank you for your help in advance
package main
import (
"bufio"
"fmt"
"io/ioutil"
"log"
"os"
"path/filepath"
)
func main() {
tmpFile, err := ioutil.TempFile("", fmt.Sprintf("%s-", filepath.Base(os.Args[0])))
if err != nil {
log.Fatal("Could not create temporary file", err)
}
fmt.Println("Created temp file: ", tmpFile.Name())
// defer os.Remove(tmpFile.Name())
fmt.Println("Writing some data to the temp file")
if _, err = tmpFile.WriteString("test data"); err != nil {
log.Fatal("Unable to write to temporary file", err)
} else {
fmt.Println("data should have been written")
}
fmt.Println("Trying to read the temp file now")
s := bufio.NewScanner(tmpFile)
for s.Scan() {
fmt.Println(s.Text())
}
err = s.Err()
if err != nil {
log.Fatal("error reading temp file", err)
}
}
ioutil.TempFile creates a temp file and opens the file for reading and writing and returns the resulting *os.File (file descriptor). So when you're writing inside the file, the pointer is moved to that offset, i.e., it's currently at the end of the file.
But as your requirement is read from the file, you need to Seek back to the beginning or wherever desired offset using *os.File.Seek method. So, adding tmpFile.Seek(0, 0) will give you the desired behaviour.
Also, as a good practice, do not forget to close the file. Notice I've used defer tmpFile.Close() which closes the file before exiting.
Refer the following example:
package main
import (
"bufio"
"fmt"
"io/ioutil"
"log"
"os"
"path/filepath"
)
func main() {
tmpFile, err := ioutil.TempFile("", fmt.Sprintf("%s-", filepath.Base(os.Args[0])))
if err != nil {
log.Fatal("Could not create temporary file", err)
}
defer tmpFile.Close()
fmt.Println("Created temp file: ", tmpFile.Name())
fmt.Println("Writing some data to the temp file")
if _, err = tmpFile.WriteString("test data"); err != nil {
log.Fatal("Unable to write to temporary file", err)
} else {
fmt.Println("Data should have been written")
}
fmt.Println("Trying to read the temp file now")
// Seek the pointer to the beginning
tmpFile.Seek(0, 0)
s := bufio.NewScanner(tmpFile)
for s.Scan() {
fmt.Println(s.Text())
}
if err = s.Err(); err != nil {
log.Fatal("error reading temp file", err)
}
}
Update:
Comment from OP:
Is the deferred close needed given that deleting the actual file is also deferred? If so, I imagine order of deferral would matter.
So, that's a nice question. So the basic rule of thumb would be to close the file and then remove. So, it might even be possible to delete first and later close it, but that is OS-dependent.
If you refer C++'s doc:
If the file is currently open by the current or another process, the behavior of this function is implementation-defined (in particular, POSIX systems unlink the file name, although the file system space is not reclaimed even if this was the last hardlink to the file until the last running process closes the file, Windows does not allow the file to be deleted)
So, on Windows, that's a problem for sure if you try deleting it first without closing it.
So, as defer's are stacked, so the order of execution would be
defer os.Remove(tmpFile.Name()) // Called 2nd
defer tmpFile.Close() // Called 1st
package main
import (
"fmt"
"io/ioutil"
"log"
)
func main() {
content, err := ioutil.ReadFile("testdata/hello")
if err != nil {
log.Fatal(err)
}
fmt.Printf("File contents: %s", content)
according to the official golang docs.
I am starting out with golang, and as I am starting to build out my applciation I want to add logging from the start, and that is where I am running into issues.
If I open a file and use the standard logging library, I am able to write to a file. Like so.
package main
import (
"os"
"fmt"
"log"
)
func main() {
// open a file
f, err := os.OpenFile("test.log", os.O_APPEND | os.O_CREATE | os.O_RDWR, 0666)
if err != nil {
fmt.Printf("error opening file: %v", err)
}
// don't forget to close it
defer f.Close()
// assign it to the standard logger
log.SetOutput(f)
log.Output(1, "this is an event")
}
I will get my test.log with the log line in it. However if I try to adapt this to support logrus https://github.com/Sirupsen/logrus like this
package main
import (
"os"
"fmt"
log "github.com/Sirupsen/logrus"
)
func init() {
// open a file
f, err := os.OpenFile("testlogrus.log", os.O_APPEND | os.O_CREATE | os.O_RDWR, 0666)
if err != nil {
fmt.Printf("error opening file: %v", err)
}
// don't forget to close it
defer f.Close()
// Log as JSON instead of the default ASCII formatter.
log.SetFormatter(&log.JSONFormatter{})
// Output to stderr instead of stdout, could also be a file.
log.SetOutput(f)
// Only log the warning severity or above.
log.SetLevel(log.DebugLevel)
}
func main() {
log.WithFields(log.Fields{
"Animal": "Logrus",
}).Info("A logrus appears")
}
All I will ever see are errors.
Failed to write to log, write testlogrus.log: bad file descriptor
All I ever get is the bad file descriptor error. Any ideas what I am doing wrong?
Thanks
Craig
Since you setup the file in the init function and you have defer f.Close(), the file gets closed after init returns.
you either have to keep the file open or move the whole thing into main.
On Linux
In development you can write to stdout and pipe to a file.
In production write to syslog.
Both ways you dont need to handle the file yourself.
If you use logrus you should better use hooks which is recommended by documentation.
See : https://github.com/rifflock/lfshook for examples.
In my opinon moving the whole thing to the main is generally not a good idea, instead we want well-segmented code.
I have a program that reads a filename from the console and executes go run filename.go.
// main.go
package main
import (
"bufio"
"fmt"
"log"
"os"
"os/exec"
)
func main() {
console := bufio.NewReader(os.Stdin)
fmt.Print("Enter a filename: ")
input, err := console.ReadString('\n')
if err != nil {
log.Fatalln(err)
}
input = input[:len(input)-1]
gorun := exec.Command("go", "run", input)
result, err := gorun.Output()
if err != nil {
log.Println(err)
}
fmt.Println("---", input, "Result ---")
fmt.Println(string(result))
}
In the same directory, I have another file like this.
// hello.go
package main
import "fmt"
func main() {
fmt.Println("Hello, World!")
}
When I input "hello.go" in the console, that file is run, and its output gets returned to the parent Go process. However, I have another program like this.
// count.go
package main
import (
"fmt"
"time"
)
func main() {
i := 0
for {
time.Sleep(time.Second)
i++
fmt.Println(i)
}
}
Except, because this program never returns, my parent process is left hanging forever. Is there a way to communicate with different Go processes? I'm thinking something like channels for goroutines, but for processes. I need to be able to receive live stdout from the child process.
The problem I'm trying to solve is dynamically executing Go programs from a directory. Go files will be added, removed, and modified daily. I'm kind of trying to make something like Go Playgrounds. The main process is a webserver serving webpages, so I can't shut it down all the time to modify code.
Don't use go run, you need to do what go run is doing yourself to have the go program be a direct child of your server process.
Using go build -o path_to/binary source_file.go will give you more control. Then you can can directly execute and communicate with the resulting binary.
Let's say I have a program than outputs things on file descriptor 3; something like this:
package main
import "os"
func main() {
fd3 := os.NewFile(3, "fd3")
fd3.Write([]byte("FOOBAR\n"))
fd3.Close()
}
Now, I want to get the output sent to file descriptor 3 from a Go program:
package main
import (
"bufio"
"fmt"
"os/exec"
"os"
)
func main() {
cmd := exec.Command("./client")
cmd.Stderr = os.Stderr
fd3 := os.NewFile(3, "fd3")
defer fd3.Close()
cmd.ExtraFiles = []*os.File{fd3}
err := cmd.Start()
if err != nil {
panic(err)
}
go func() {
for {
reader := bufio.NewReader(fd3)
line, err := reader.ReadString('\n')
if err != nil {
panic(err)
}
fmt.Print(line)
}
}()
cmd.Wait()
fmt.Println("--- END ---")
}
But that does not work as it outputs the following error:
panic: read fd3: bad file descriptor
I don't understand what's wrong with my code. Anyone willing to help?
os.NewFile doesn't actually open a file descriptor. It's really an API to wrap a fd that was given to you.
look at the godoc: http://golang.org/pkg/os/#Create
(click the name Create, which currently points to this)
I think you want to call os.Create(name) and pass the fd to the child process
or potentiall os.Open / os.OpenFile if you need to set mode and stuff
In nodejs I use __dirname . What is the equivalent of this in Golang?
I have googled and found out this article http://andrewbrookins.com/tech/golang-get-directory-of-the-current-file/ . Where he uses below code
_, filename, _, _ := runtime.Caller(1)
f, err := os.Open(path.Join(path.Dir(filename), "data.csv"))
But is it the right way or idiomatic way to do in Golang?
EDIT: As of Go 1.8 (Released February 2017) the recommended way of doing this is with os.Executable:
func Executable() (string, error)
Executable returns the path name for the executable that started the current process. There is no guarantee that the path is still pointing to the correct executable. If a symlink was used to start the process, depending on the operating system, the result might be the symlink or the path it pointed to. If a stable result is needed, path/filepath.EvalSymlinks might help.
To get just the directory of the executable you can use path/filepath.Dir.
Example:
package main
import (
"fmt"
"os"
"path/filepath"
)
func main() {
ex, err := os.Executable()
if err != nil {
panic(err)
}
exPath := filepath.Dir(ex)
fmt.Println(exPath)
}
OLD ANSWER:
You should be able to use os.Getwd
func Getwd() (pwd string, err error)
Getwd returns a rooted path name corresponding to the current directory. If the current directory can be reached via multiple paths (due to symbolic links), Getwd may return any one of them.
For example:
package main
import (
"fmt"
"os"
)
func main() {
pwd, err := os.Getwd()
if err != nil {
fmt.Println(err)
os.Exit(1)
}
fmt.Println(pwd)
}
This should do it:
import (
"fmt"
"log"
"os"
"path/filepath"
)
func main() {
dir, err := filepath.Abs(filepath.Dir(os.Args[0]))
if err != nil {
log.Fatal(err)
}
fmt.Println(dir)
}
Use package osext
It's providing function ExecutableFolder() that returns an absolute path to folder where the currently running program executable reside (useful for cron jobs). It's cross platform.
Online documentation
package main
import (
"github.com/kardianos/osext"
"fmt"
"log"
)
func main() {
folderPath, err := osext.ExecutableFolder()
if err != nil {
log.Fatal(err)
}
fmt.Println(folderPath)
}
I came from Node.js to Go. The Node.js equivalent to __dirname in Go is:
_, filename, _, ok := runtime.Caller(0)
if !ok {
return errors.New("unable to get the current filename")
}
dirname := filepath.Dir(filename)
Some other mentions in this thread and why they're wrong:
os.Executable() will give you the filepath of the currently running executable. This is equivalent to process.argv[0] in Node. This is not true if you want to take the __dirname of a sub-package.
os.Getwd() will give you the current working directory. This is the equivalent to process.cwd() in Node. This will be wrong when you run your program from another directory.
Lastly, I'd recommend against pulling in a third-party package for this use case. Here's a package you can use:
package current
// Filename is the __filename equivalent
func Filename() (string, error) {
_, filename, _, ok := runtime.Caller(1)
if !ok {
return "", errors.New("unable to get the current filename")
}
return filename, nil
}
// Dirname is the __dirname equivalent
func Dirname() (string, error) {
filename, err := Filename()
if err != nil {
return "", err
}
return filepath.Dir(filename), nil
}
Note that I've adjusted runtime.Caller(1) to 1 because we want to get the directory of the package that called current.Dirname(), not the directory containing the current package.
filepath.Abs("./")
Abs returns an absolute representation of path. If the path is not
absolute it will be joined with the current working directory to turn
it into an absolute path.
As stated in the comment, this returns the directory which is currently active.
os.Executable: https://tip.golang.org/pkg/os/#Executable
filepath.EvalSymlinks: https://golang.org/pkg/path/filepath/#EvalSymlinks
Full Demo:
package main
import (
"fmt"
"os"
"path/filepath"
)
func main() {
var dirAbsPath string
ex, err := os.Executable()
if err == nil {
dirAbsPath = filepath.Dir(ex)
fmt.Println(dirAbsPath)
return
}
exReal, err := filepath.EvalSymlinks(ex)
if err != nil {
panic(err)
}
dirAbsPath = filepath.Dir(exReal)
fmt.Println(dirAbsPath)
}
if you use this way :
dir, err := filepath.Abs(filepath.Dir(os.Args[0]))
if err != nil {
log.Fatal(err)
}
fmt.Println(dir)
you will get the /tmp path when you are running program using some IDE like GoLand because the executable will save and run from /tmp
i think the best way for getting the currentWorking Directory or '.' is :
import(
"os"
"fmt"
"log"
)
func main() {
dir, err := os.Getwd()
if err != nil {
log.Fatal(err)
}
fmt.Println(dir)
}
the os.Getwd() function will return the current working directory.
and its all without using of any external library :D
If you use package osext by kardianos and you need to test locally, like Derek Dowling commented:
This works fine until you'd like to use it with go run main.go for
local development. Not sure how best to get around that without
building an executable beforehand each time.
The solution to this is to make a gorun.exe utility instead of using go run. The gorun.exe utility would compile the project using "go build", then run it right after, in the normal directory of your project.
I had this issue with other compilers and found myself making these utilities since they are not shipped with the compiler... it is especially arcane with tools like C where you have to compile and link and then run it (too much work).
If anyone likes my idea of gorun.exe (or elf) I will likely upload it to github soon..
Sorry, this answer is meant as a comment, but I cannot comment due to me not having a reputation big enough yet.
Alternatively, "go run" could be modified (if it does not have this feature already) to have a parameter such as "go run -notemp" to not run the program in a temporary directory (or something similar). But I would prefer just typing out gorun or "gor" as it is shorter than a convoluted parameter. Gorun.exe or gor.exe would need to be installed in the same directory as your go compiler
Implementing gorun.exe (or gor.exe) would be trivial, as I have done it with other compilers in only a few lines of code... (famous last words ;-)
Sometimes this is enough, the first argument will always be the file path
package main
import (
"fmt"
"os"
)
func main() {
fmt.Println(os.Args[0])
// or
dir, _ := os.Getwd()
fmt.Println(dir)
}
dir, err := os.Getwd()
if err != nil {
fmt.Println(err)
}
this is for golang version: go version go1.13.7 linux/amd64
works for me, for go run main.go. If I run go build -o fileName, and put the final executable in some other folder, then that path is given while running the executable.
Do not use the "Answer recommended by Go Language" with runtime.Caller(0).
That works when you go build or go install a program, because you are re-compiling it.
But when you go build a program and then distribute it (copy) on your colleagues' workstations (who don't have Go, and just need the executable), the result of runtime.Caller(0) would still be the path of where you built it (from your computer).
Ie a path which would likely not exist on their own computer.
os.Args[0] or, better, os.Executable() (mentioned here) and kardianos/osext (mentioned here and here), are more reliable.
None of the answers here worked for me, at least on go version go1.16.2 darwin/amd64. This is the only thing close to the __dirname functionality in node
This was posted by goland engineer Daniil Maslov in the jetbrains forums
pasted below for easier reading:
The trick is actually very simple and is to get the current executing and add .. to the project root.
Create a new directory and file like testing_init.go with the following content:
package testing_init
import (
"os"
"path"
"runtime"
)
func init() {
_, filename, _, _ := runtime.Caller(0)
dir := path.Join(path.Dir(filename), "..")
err := os.Chdir(dir)
if err != nil {
panic(err)
}
}
After that, just import the package into any of the test files:
package main_test
import (
_ "project/testing_init"
)
Now you can specify paths from the project root
// GetCurrentDir
func GetCurrentDir() string {
p, _ := os.Getwd()
return p
}
// GetParentDir
func GetParentDir(dirctory string) string {
return substr(dirctory, 0, strings.LastIndex(dirctory, "/"))
}
func substr(s string, pos, length int) string {
runes := []rune(s)
l := pos + length
if l > len(runes) {
l = len(runes)
}
return string(runes[pos:l])
}
If your file is not in the main package then the above answers won't work
I tried different approaches to find find the directory of the currently running file but failed.
The best possible answer is in the question itself this is how I find the current working directory of the file which is not in the main package.
_, filename, _, _ := runtime.Caller(1)
pwd := path.Dir(filename)
Gustavo Niemeyer's answer is great.
But in Windows, runtime proc is mostly in another dir, like this:
"C:\Users\XXX\AppData\Local\Temp"
If you use relative file path, like "/config/api.yaml", this will use your project path where your code exists.