I am starting out with golang, and as I am starting to build out my applciation I want to add logging from the start, and that is where I am running into issues.
If I open a file and use the standard logging library, I am able to write to a file. Like so.
package main
import (
"os"
"fmt"
"log"
)
func main() {
// open a file
f, err := os.OpenFile("test.log", os.O_APPEND | os.O_CREATE | os.O_RDWR, 0666)
if err != nil {
fmt.Printf("error opening file: %v", err)
}
// don't forget to close it
defer f.Close()
// assign it to the standard logger
log.SetOutput(f)
log.Output(1, "this is an event")
}
I will get my test.log with the log line in it. However if I try to adapt this to support logrus https://github.com/Sirupsen/logrus like this
package main
import (
"os"
"fmt"
log "github.com/Sirupsen/logrus"
)
func init() {
// open a file
f, err := os.OpenFile("testlogrus.log", os.O_APPEND | os.O_CREATE | os.O_RDWR, 0666)
if err != nil {
fmt.Printf("error opening file: %v", err)
}
// don't forget to close it
defer f.Close()
// Log as JSON instead of the default ASCII formatter.
log.SetFormatter(&log.JSONFormatter{})
// Output to stderr instead of stdout, could also be a file.
log.SetOutput(f)
// Only log the warning severity or above.
log.SetLevel(log.DebugLevel)
}
func main() {
log.WithFields(log.Fields{
"Animal": "Logrus",
}).Info("A logrus appears")
}
All I will ever see are errors.
Failed to write to log, write testlogrus.log: bad file descriptor
All I ever get is the bad file descriptor error. Any ideas what I am doing wrong?
Thanks
Craig
Since you setup the file in the init function and you have defer f.Close(), the file gets closed after init returns.
you either have to keep the file open or move the whole thing into main.
On Linux
In development you can write to stdout and pipe to a file.
In production write to syslog.
Both ways you dont need to handle the file yourself.
If you use logrus you should better use hooks which is recommended by documentation.
See : https://github.com/rifflock/lfshook for examples.
In my opinon moving the whole thing to the main is generally not a good idea, instead we want well-segmented code.
Related
This question already has an answer here:
Write and Read File with same *os.File in Go
(1 answer)
Closed 8 months ago.
This post was edited and submitted for review 8 months ago and failed to reopen the post:
Original close reason(s) were not resolved
In Go, I am trying to write data to a temp file that I then turn around and read but have not been successful. Below is a stripped down test program. I have verified that the data are being written to the file by inspecting the temporary file. So, at least I know that data are making it into the file. I just am then unable to read it out.
Thank you for your help in advance
package main
import (
"bufio"
"fmt"
"io/ioutil"
"log"
"os"
"path/filepath"
)
func main() {
tmpFile, err := ioutil.TempFile("", fmt.Sprintf("%s-", filepath.Base(os.Args[0])))
if err != nil {
log.Fatal("Could not create temporary file", err)
}
fmt.Println("Created temp file: ", tmpFile.Name())
// defer os.Remove(tmpFile.Name())
fmt.Println("Writing some data to the temp file")
if _, err = tmpFile.WriteString("test data"); err != nil {
log.Fatal("Unable to write to temporary file", err)
} else {
fmt.Println("data should have been written")
}
fmt.Println("Trying to read the temp file now")
s := bufio.NewScanner(tmpFile)
for s.Scan() {
fmt.Println(s.Text())
}
err = s.Err()
if err != nil {
log.Fatal("error reading temp file", err)
}
}
ioutil.TempFile creates a temp file and opens the file for reading and writing and returns the resulting *os.File (file descriptor). So when you're writing inside the file, the pointer is moved to that offset, i.e., it's currently at the end of the file.
But as your requirement is read from the file, you need to Seek back to the beginning or wherever desired offset using *os.File.Seek method. So, adding tmpFile.Seek(0, 0) will give you the desired behaviour.
Also, as a good practice, do not forget to close the file. Notice I've used defer tmpFile.Close() which closes the file before exiting.
Refer the following example:
package main
import (
"bufio"
"fmt"
"io/ioutil"
"log"
"os"
"path/filepath"
)
func main() {
tmpFile, err := ioutil.TempFile("", fmt.Sprintf("%s-", filepath.Base(os.Args[0])))
if err != nil {
log.Fatal("Could not create temporary file", err)
}
defer tmpFile.Close()
fmt.Println("Created temp file: ", tmpFile.Name())
fmt.Println("Writing some data to the temp file")
if _, err = tmpFile.WriteString("test data"); err != nil {
log.Fatal("Unable to write to temporary file", err)
} else {
fmt.Println("Data should have been written")
}
fmt.Println("Trying to read the temp file now")
// Seek the pointer to the beginning
tmpFile.Seek(0, 0)
s := bufio.NewScanner(tmpFile)
for s.Scan() {
fmt.Println(s.Text())
}
if err = s.Err(); err != nil {
log.Fatal("error reading temp file", err)
}
}
Update:
Comment from OP:
Is the deferred close needed given that deleting the actual file is also deferred? If so, I imagine order of deferral would matter.
So, that's a nice question. So the basic rule of thumb would be to close the file and then remove. So, it might even be possible to delete first and later close it, but that is OS-dependent.
If you refer C++'s doc:
If the file is currently open by the current or another process, the behavior of this function is implementation-defined (in particular, POSIX systems unlink the file name, although the file system space is not reclaimed even if this was the last hardlink to the file until the last running process closes the file, Windows does not allow the file to be deleted)
So, on Windows, that's a problem for sure if you try deleting it first without closing it.
So, as defer's are stacked, so the order of execution would be
defer os.Remove(tmpFile.Name()) // Called 2nd
defer tmpFile.Close() // Called 1st
package main
import (
"fmt"
"io/ioutil"
"log"
)
func main() {
content, err := ioutil.ReadFile("testdata/hello")
if err != nil {
log.Fatal(err)
}
fmt.Printf("File contents: %s", content)
according to the official golang docs.
I need to read golang code from the play.golang.org link and save to a .go file. I'm wondering if there is any public API support for play.golang.org. I googled but no hints. Has anyone attempted anything similar?
To get the text of a shared playground program, append ".go" to the URL. For example, you can get the text of the program at https://play.golang.org/p/HmnNoBf0p1z with https://play.golang.org/p/HmnNoBf0p1z.go.
You can upload a program by posting the program text to https://play.golang.org/share. The response is the ID of the shared program. This program uploads stdin to the playground and prints the ID of the uploaded program to stdout:
package main
import (
"io"
"log"
"net/http"
"os"
)
func main() {
req, err := http.NewRequest("POST", "https://play.golang.org/share", os.Stdin)
if err != nil {
log.Fatal(err)
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
log.Fatal(err)
}
defer resp.Body.Close()
io.Copy(os.Stdout, resp.Body)
}
Assuming the above program is in upload.go, the following shell script prints HmnNoBf0p1z.
go run upload.go << EOF
package main
import (
"fmt"
)
func main() {
fmt.Println("Hello, playground")
}
EOF
If you want to download the program as a file using a browser, then add the query ?download=true to the .go URL. Example: https://play.golang.org/p/HmnNoBf0p1z.go?download=true
There are two ways that I know one of which is described by #ThunderCat. Another simple solution is go to the URL https://play.golang.org/p/HmnNoBf0p1z and Press Ctrl+save on the page it will be downloaded as a .go file.
There are certain fmt.Print statements that I want to save into a .txt file.
I don't want to store all print statments. Can I do this ?
package main
import (
"fmt"
"io"
"log"
"os"
)
func main() {
file, err := os.Create("myfile")
if err != nil {
log.Fatal(err)
}
mw := io.MultiWriter(os.Stdout, file)
fmt.Fprintln(mw, "This line will be written to stdout and also to a file")
}
Use the fmt.Fprint() method for calls you want to save to a file. There are also fmt.Fprintf() and fmt.Fprintln().
These functions take a destination io.Writer as the first argument, to which you can pass your file (*os.File).
For example:
f, err := os.Open("data.txt")
if err != nil {
log.Fatal(err)
}
defer f.Close()
fmt.Println("This goes to standard output.")
fmt.Fprintln(f, "And this goes to the file")
fmt.Fprintf(f, "Also to file, with some formatting. Time: %v, line: %d\n",
time.Now(), 2)
If you want all fmt.PrintXX() calls to go to the file which you have no control over (e.g. you can't change them to fmt.FprintXX() because they are part of another library), you may change os.Stdout temporarily, so all further fmt.PrintXX() calls will write to the output you set, e.g.:
// Temporarily set your file as the standard output (and save the old)
old, os.Stdout = os.Stdout, f
// Now all fmt.PrintXX() calls output to f
somelib.DoSomething()
// Restore original standard output
os.Stdout = old
I started learning and playing around with Go to see what it is like to make some more complex console/cli type tools instead of using shells or Python. I want to execute commands and display the output. I figured out how to print the output like this:
out, err := exec.Command("pwd").Output()
print(string(out))
Is there a way to execute the commands and have it default to stdout like a shell script, or do I need to make a helper function for this?
Update: After getting IntelliJ and the Go plugin, I poked around in the Go source and agree there is currently no way to do with without a helper method.
It is not possible to reuse a Cmd object as per this comment in the exec.go source code:
// A Cmd cannot be reused after calling its Run, Output or CombinedOutput
// methods.
I did incorporate the stdout option into my own helper, including other options like shell integration. I will try turn that into open source if I can make it useful. An interesting first day of Go.
The solution
Actually, it is pretty easy. You can set the stdout of the command to os.Stdout and Bob's your uncle:
package main
import (
"os"
"os/exec"
)
func main() {
cmd := exec.Command("pwd")
cmd.Stdout = os.Stdout
err := cmd.Run()
if err != nil {
panic(err)
}
}
What's happening here?
By default, the output of a command is stored in a bytes.Buffer if cmd.Stdout is not set to another io.Writer. The call of cmd.Output() then runs the command and saves the output to said buffer.
Since os.Stdout implements io.Writer interface, we simply set cmd.Stdout to be os.Stdout. Now when .Run() is called, the output of the command gets written to the io.Writer defined in cmd.Stdout, which happens to be os.Stdout and the output gets written in the shell.
EDIT: As per comment, if all commands should write to os.Stdout, there of course is no way to prevent some helper. I'd do it like this:
package main
import (
"os"
"os/exec"
)
func CmdToStdout( c string ) (err error){
cmd := exec.Command(c)
cmd.Stdout = os.Stdout
err = cmd.Run()
return
}
func main() {
err := CmdToStdout("pwd")
if err != nil {
panic(err)
}
}
You have to create a helper if you need this often (and 5 lines looks too much). Based on the documentation this is a recommended way:
package main
import (
"fmt"
"log"
"os/exec"
)
func main() {
out, err := exec.Command("date").Output()
if err != nil {
log.Fatal(err)
}
fmt.Printf("The date is %s\n", out)
}
Let's say I have a program than outputs things on file descriptor 3; something like this:
package main
import "os"
func main() {
fd3 := os.NewFile(3, "fd3")
fd3.Write([]byte("FOOBAR\n"))
fd3.Close()
}
Now, I want to get the output sent to file descriptor 3 from a Go program:
package main
import (
"bufio"
"fmt"
"os/exec"
"os"
)
func main() {
cmd := exec.Command("./client")
cmd.Stderr = os.Stderr
fd3 := os.NewFile(3, "fd3")
defer fd3.Close()
cmd.ExtraFiles = []*os.File{fd3}
err := cmd.Start()
if err != nil {
panic(err)
}
go func() {
for {
reader := bufio.NewReader(fd3)
line, err := reader.ReadString('\n')
if err != nil {
panic(err)
}
fmt.Print(line)
}
}()
cmd.Wait()
fmt.Println("--- END ---")
}
But that does not work as it outputs the following error:
panic: read fd3: bad file descriptor
I don't understand what's wrong with my code. Anyone willing to help?
os.NewFile doesn't actually open a file descriptor. It's really an API to wrap a fd that was given to you.
look at the godoc: http://golang.org/pkg/os/#Create
(click the name Create, which currently points to this)
I think you want to call os.Create(name) and pass the fd to the child process
or potentiall os.Open / os.OpenFile if you need to set mode and stuff