I am getting a bad file descriptor when trying to append to a logging file within my go routine.
write ./log.log: bad file descriptor
The file exists and has 666 for permissions. At first I thought well maybe it is because each one of them is trying to open the file at the same time. I implemented a mutex to try and avoid that but got the same issue so I removed it.
logCh := make(chan string, 150)
go func() {
for {
msg, ok := <-logCh
if ok {
if f, err := os.OpenFile("./log.log", os.O_APPEND, os.ModeAppend); err != nil {
panic(err)
} else {
logTime := time.Now().Format(time.RFC3339)
if _, err := f.WriteString(logTime + " - " + msg); err != nil {
fmt.Print(err)
}
f.Close()
}
} else {
fmt.Print("Channel closed! \n")
break
}
}
}()
You need to add the O_WRONLY flag :
if f, err := os.OpenFile("./log.log", os.O_APPEND|os.O_WRONLY, os.ModeAppend); err != nil { /*[...]*/ }
To explain, here is the linux documentation for open: http://man7.org/linux/man-pages/man2/openat.2.html :
The argument flags must include one of the following access modes:
O_RDONLY, O_WRONLY, or O_RDWR. These request opening the file read-
only, write-only, or read/write, respectively.
If you check /usr/local/go/src/syscall/zerrors_linux_amd64.go:660, you can see that:
O_RDONLY = 0x0
O_RDWR = 0x2
O_WRONLY = 0x1
So by default you get a read-only file descriptor.
it used for me
the code before:
os.OpenFile(fileName, os.O_CREATE|os.O_APPEND, os.ModePerm)
and it occured the error: bad file descriptor,
then i add the os.O_WRONLY into the function
the code after:
os.OpenFile(fileName, os.O_CREATE|os.O_WRONLY|os.O_APPEND, os.ModePerm)
and it did't occured the problem.
This appears to be a discrepancy between windows and linux.
On windows os.O_APPEND implies write access, as can be seen in the code in syscall_windows.go.
if mode&O_APPEND != 0 {
access &^= GENERIC_WRITE
access |= FILE_APPEND_DATA
}
in linux the openflags are passed as-is to a linux syscall
So in DOS/WINDOWS you do not need to explicitly add a write flag with an append flag, as it is implied. (this has been default behaviour since the DOS days)
Go in linux does need the extra flag added to work.
(...but imo it should not need this flag, as appending implicitly implies that I want to write. Should I bugreport this to golang ?)
Related
Sorry for reformatting my question by focusing on the real issue as follows:
I am trying to create file and write to it on a network mapped drive, which I can access, create, delete and edit files using windows explorer or CMD (Windows 10/Server 2016).
The following code should do the task:
package main
import (
"bufio"
"fmt"
"os"
)
func main() {
//The following is the file name on a network mapped drive H:
out, errc := os.OpenFile("H:/00_SC/Dest01.txt", os.O_CREATE|os.O_APPEND|os.O_RDWR, 0666)
if errc != nil {
fmt.Println("Error Creating/Wrting to Dest file :", errc)
}
defer out.Close()
wr := bufio.NewWriter(out)
mystring := "another line here"
d, err := wr.WriteString(mystring)
if err != nil {
fmt.Println("Error writing by Writer: ", err)
}
errflush := wr.Flush()
if errflush != nil {
fmt.Println("Error Flushing the write to file", errflush)
}
wrtd, errw := out.Write([]byte("Write something in the file"))
if errw != nil {
fmt.Println("Error of Writte call", errw)
}
fmt.Println("Length of mystring = ", len(mystring))
fmt.Println("Bytes written to buffer = ", d)
fd, errf := fmt.Fprintf(out, "Another line to the file using Fprintf %d", d)
if errf != nil {
fmt.Println("Fprintf Error: ", errf)
}
fmt.Println("Bytes written to file by Fprintf = ", fd)
fmt.Println("Bytes written to file using Write", wrtd)
}
The file was created successfully, however, neither of the methods used write to it.
Bufio.WriteString neither write nor return an error !!
Fprintf failed with unclear error message which I searched and could not find a meaning or reason.
Here the output I got when running the compiled file:
Error Flushing the write to file write H:/00_SC/Dest01.txt: The parameter is incorrect.
Error of Write call write H:/00_SC/Dest01.txt: The parameter is incorrect.
Length of mystring = 17
Bytes writen to buffer = 17
Fprintf Error: write H:/00_SC/Dest01.txt: The parameter is incorrect.
Bytes written to file by Fprintf = 0
Bytes written to file using Write 0
I would appreciate your help and guidance finding the cause of this error.
Thanks
I have the following piece of code which creates an output file on a local drive and required to do the same on a network mapped drive let's call it [H:].
The file name (full path name) entered from command line as argument[1].
I am using Windows 10/Server 2016
// The following will create and append to the file when required.
sourcefile, errf := os.OpenFile(os.Args[1], s.O_CREATE|os.O_APPEND|os.O_RDWR, 0666)
if erro != nil {
panic(erro)
}
defer outfile.Close()
I use the following function to write a map into this file.
func map2Linpro(inp map[string][]string, outfile io.Writer) {
for k, v := range inp {
_, err := fmt.Fprintf(outfile, "%s %s=%s %s\n", v[0], k, v[1], v[2])
if err != nil {
fmt.Println("Error Writing to File: ", err)
}
}
}
Everything is working just fine if the output file is on the local Drive, but when using full path with the Mapped Drive letter, I received the following error:
Error: write h://00_sc//dest01.txt: The parameter is incorrect.
I searched for any reason, but could not find one.
I would appreciate if someone help
The following is the Error I got after adding Panic(erro) after OpenFile.
Which proves that the error source is fmt.Fprintf
Error Writing to File: write H:/00_sc/dest01.txt: The parameter is incorrect.
Thanks to all.
outfile, _ := os.OpenFile(os.Args[2], os.O_CREATE|os.O_APPEND, 0666)
should read
outfile, err := os.OpenFile(os.Args[2], os.O_CREATE|os.O_APPEND, 0666)
if err != nil {
panic(err)
}
rewrite those lines and the resulting error message should give a clue as to the cause
I have a Go function that appends a line to a file:
func AppendLine(p string, s string) error {
f, err := os.OpenFile(p, os.O_APPEND|os.O_WRONLY, 0600)
defer f.Close()
if err != nil {
return errors.WithStack(err)
}
_, err = f.WriteString(s + "\n")
return errors.WithStack(err)
}
I'm wondering if the flags os.O_APPEND|os.O_WRONLY make this a safe operation. Is there a guarantee that no matter what happens (even if the process gets shut off in the middle of writing) the existing file contents cannot be deleted?
os package is a wrapper around systems calls so you have guarantees provided by operation system. In this case linux OS guarantees that file opened with O_APPEND flag would be processed atomically http://man7.org/linux/man-pages/man2/open.2.html
If I do
s, err := os.Stat(path)
and err != nil I need to know if the file doesn't exist vs I don't have permission to access it, etc. How do I get the underlying error code? Reading the os package docs it seems to suggest that I read the text of the error string - surely not?
What FUZxxl says.
From the os.Stat documentation:
Stat returns a FileInfo describing the named file. If there is an error, it will be of type *PathError.
PathError is documented on the same page, stating that it holds the operation that caused the error, the path to the file that caused it and the underlying system's error. In case the file was not found when calling os.Stat, the returned error would be something like this:
&PathError{"stat", "/your/file", syscall.Errno(2)}
Since the underlying error is inherently depending on the OS you use, the only thing that you can do is to
understand PathError.Err. For UNIX systems the syscall package has the Errno error type returned by syscalls like syscall.Stat. You can compare this value with the constants in the syscall package and handle the error (Click to play):
stat, err := os.Stat(file)
if perr, ok := err.(*os.PathError); ok {
switch perr.Err.(syscall.Errno) {
case syscall.ENOENT: fmt.Println("No such file or directory.")
default: panic("Unknown error")
}
}
The shorter way of doing this is to use os.IsNotExist which does pretty much the above
and is, most importantly, platform independent:
stat, err := os.Stat(file)
if err != nil && os.IsNotExist(err) {
// ...
}
The other answer is great, but I wanted add a note about this suggestion:
stat, err := os.Stat(file)
if err != nil && os.IsNotExist(err) {
// ...
}
I found that in many cases, I was needing to take different actions depending on
each test, so in reality you have three branches here. Here is code I use for
that:
stat, err := os.Stat(file)
if os.IsNotExist(err) {
// branch one
} else if err != nil {
// branch two
}
// branch three
I am trying to take input from the keyboard and then store it in a text file but I am a bit confused on how to actually do it.
My current code is as follow at the moment:
// reads the file txt.txt
bs, err := ioutil.ReadFile("text.txt")
if err != nil {
panic(err)
}
// Prints out content
textInFile := string(bs)
fmt.Println(textInFile)
// Standard input from keyboard
var userInput string
fmt.Scanln(&userInput)
//Now I want to write input back to file text.txt
//func WriteFile(filename string, data []byte, perm os.FileMode) error
inputData := make([]byte, len(userInput))
err := ioutil.WriteFile("text.txt", inputData, )
There are so many functions in the "os" and "io" packages. I am very confused about which one I actually should use for this purpose.
I am also confused about what the third argument in the WriteFile function should be. In the documentation is says of type " perm os.FileMode" but since I am new to programming and Go I am a bit clueless.
Does anybody have any tips on how to proced?
Thanks in advance,
Marie
// reads the file txt.txt
bs, err := ioutil.ReadFile("text.txt")
if err != nil { //may want logic to create the file if it doesn't exist
panic(err)
}
var userInput []string
var err error = nil
var n int
//read in multiple lines from user input
//until user enters the EOF char
for ln := ""; err == nil; n, err = fmt.Scanln(ln) {
if n > 0 { //we actually read something into the string
userInput = append(userInput, ln)
} //if we didn't read anything, err is probably set
}
//open the file to append to it
//0666 corresponds to unix perms rw-rw-rw-,
//which means anyone can read or write it
out, err := os.OpenFile("text.txt", os.O_APPEND, 0666)
defer out.Close() //we'll close this file as we leave scope, no matter what
if err != nil { //assuming the file didn't somehow break
//write each of the user input lines followed by a newline
for _, outLn := range userInput {
io.WriteString(out, outLn+"\n")
}
}
I've made sure this compiles and runs on play.golang.org, but I'm not at my dev machine, so I can't verify that it's interacting with Stdin and the file entirely correctly. This should get you started though.
For example,
package main
import (
"fmt"
"io/ioutil"
"os"
)
func main() {
fname := "text.txt"
// print text file
textin, err := ioutil.ReadFile(fname)
if err == nil {
fmt.Println(string(textin))
}
// append text to file
f, err := os.OpenFile(fname, os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0666)
if err != nil {
panic(err)
}
var textout string
fmt.Scanln(&textout)
_, err = f.Write([]byte(textout))
if err != nil {
panic(err)
}
f.Close()
// print text file
textin, err = ioutil.ReadFile(fname)
if err != nil {
panic(err)
}
fmt.Println(string(textin))
}
If you simply want to append the user's input to a text file, you could just read the
input as you've already done and use ioutil.WriteFile, as you've tried to do.
So you already got the right idea.
To make your way go, the simplified solution would be this:
// Read old text
current, err := ioutil.ReadFile("text.txt")
// Standard input from keyboard
var userInput string
fmt.Scanln(&userInput)
// Append the new input to the old using builtin `append`
newContent := append(current, []byte(userInput)...)
// Now write the input back to file text.txt
err = ioutil.WriteFile("text.txt", newContent, 0666)
The last parameter of WriteFile is a flag which specifies the various options for
files. The higher bits are options like file type (os.ModeDir, for example) and the lower
bits represent the permissions in form of UNIX permissions (0666, in octal format, stands for user rw, group rw, others rw). See the documentation for more details.
Now that your code works, we can improve it. For example by keeping the file open
instead of opening it twice:
// Open the file for read and write (O_RDRW), append to it if it has
// content, create it if it does not exit, use 0666 for permissions
// on creation.
file, err := os.OpenFile("text.txt", os.O_RDWR|os.O_APPEND|os.O_CREATE, 0666)
// Close the file when the surrounding function exists
defer file.Close()
// Read old content
current, err := ioutil.ReadAll(file)
// Do something with that old content, for example, print it
fmt.Println(string(current))
// Standard input from keyboard
var userInput string
fmt.Scanln(&userInput)
// Now write the input back to file text.txt
_, err = file.WriteString(userInput)
The magic here is, that you use the flag os.O_APPEND while opening the file,
which makes file.WriteString() append. Note that you need to close the file after
opening it, which we do after the function exists using the defer keyword.