This is my first attempt to log into files with golang.
file, _ := os.Open("logfile")
log.SetOutput(file)
log.Println("foo")
these lines build, but are not working. Why?
os.Open calls OpenFile(name, O_RDONLY, 0). This does not have the flag O_CREATE to create the file if it does not exist. Therefore you need to call OpenFile with the O_CREATE flag. I have commented out the error code:
package main
import (
"log"
"os"
"syscall"
)
const (
O_RDONLY int = syscall.O_RDONLY // open the file read-only.
O_RDWR int = syscall.O_RDWR // open the file read-write.
O_CREATE int = syscall.O_CREAT // create a new file if none exists.
O_APPEND int = syscall.O_APPEND // append data to the file when writing.
)
func main() {
/*f1, err := os.OpenFile("testlogfile1", O_RDONLY, 0) // Equivalent to os.Open("testlogfile1")
if err != nil {
log.Fatalf("error opening file1: %v", err)
}
// ** error opening file: open testlogfile1: no such file or directory exit status 1 **
defer f1.Close()
log.SetOutput(f1)
log.Println("This is a test for log 1")*/
f2, err := os.OpenFile("testlogfile2", O_RDWR | O_CREATE | O_APPEND, 0644)
/* Note that you want:
* O_RDWR to write to the file
* O_CREATE to create file if it does not exist
* O_APPEND to add additional information to existing file */
if err != nil {
log.Fatalf("error opening file2: %v", err)
}
defer f2.Close()
log.SetOutput(f2)
log.Println("This is a test for log 2")
}
Always check those errors!
Why log.Println(“does not log into file”)?
The reason is because you didn't check the file is exist or not form your code.
file, _ := os.Open("logfile")
you're using _ and didn't check the error. This is important if you wanted to write something in to file. For example look at this code :
f, err := os.OpenFile(filePath+fileName, os.O_RDWR|os.O_CREATE|os.O_APPEND, os.ModePerm)
if err != nil {
// if error then you need to create the file first
err = os.MkdirAll(filePath, os.ModePerm)
}
from above code you can see the error checking using if err != nil. if the file is not exist yet then create the file first. using os.MkdirAll().
file, err := os.Open("file.go") // For read access.
if err != nil {
// do something
}
// and try to include the file path...
Related
I am trying to write a simple Go program which connects to an FTP server, list the files in a specified directory and pulls them.
The code is this:
package main
import (
"bytes"
"fmt"
"github.com/secsy/goftp"
"io/ioutil"
"log"
"os"
"path"
"time"
)
func main() {
config := goftp.Config{
User: "anonymous",
Password: "root#local.me",
ConnectionsPerHost: 21,
Timeout: 10 * time.Second,
Logger: os.Stderr,
}
// Connecting to the server
client, dailErr := goftp.DialConfig(config, "ftp.example.com")
if dailErr != nil {
log.Fatal(dailErr)
panic(dailErr)
}
// setting the search directory
dir := "/downloads/"
files, err := client.ReadDir(dir)
if err != nil {
for _, file := range files {
if file.IsDir() {
path.Join(dir, file.Name())
} else {
fmt.Println("the file is %s", file.Name())
}
}
}
// this section works , I am setting a file name and I can pull it
// if I mark the search part
ret_file := "example.PDF"
fmt.Println("Retrieving file: ", ret_file)
buf := new(bytes.Buffer)
fullPathFile := dir + ret_file
rferr := client.Retrieve(fullPathFile, buf)
if rferr != nil {
panic(rferr)
}
fmt.Println("writing data to file", ret_file)
fmt.Println("Opening file", ret_file, "for writing")
w, _ := ioutil.ReadAll(buf)
ferr := ioutil.WriteFile(ret_file, w, 0644)
if ferr != nil {
log.Fatal(ferr)
panic(ferr)
} else {
fmt.Println("Writing", ret_file, " completed")
}
}
For some reason I am getting an error on the ReadDir function.
I need to grab the files names so I can download them.
You're attempting to loop through files when ReadDir() returns an error. That will never work, as any time an error is returned files is nil.
This is pretty standard behavior and can be confirmed by reading the implementation of ReadDir().
I'm guessing you may have used the the example from the project used to demonstrate ReadDir() as a starting point. Within the example, the error handling is involved because it's deciding whether or not to continue walking the directory tree. However, note that when ReadDir() returns an error that doesn't result in stopping the program, the subsequent for loop is a no-op, since files is nil.
Here's a small program that demonstrates successfully using the results of Readdir() in a straightforward manner:
package main
import (
"fmt"
"github.com/secsy/goftp"
)
const (
ftpServerURL = "ftp.us.debian.org"
ftpServerPath = "/debian/"
)
func main() {
client, err := goftp.Dial(ftpServerURL)
if err != nil {
panic(err)
}
files, err := client.ReadDir(ftpServerPath)
if err != nil {
panic(err)
}
for _, file := range files {
fmt.Println(file.Name())
}
}
It outputs (which matches the current listing at http://ftp.us.debian.org/debian/):
$ go run goftp-test.go
README
README.CD-manufacture
README.html
README.mirrors.html
README.mirrors.txt
dists
doc
extrafiles
indices
ls-lR.gz
pool
project
tools
zzz-dists
Sorry for reformatting my question by focusing on the real issue as follows:
I am trying to create file and write to it on a network mapped drive, which I can access, create, delete and edit files using windows explorer or CMD (Windows 10/Server 2016).
The following code should do the task:
package main
import (
"bufio"
"fmt"
"os"
)
func main() {
//The following is the file name on a network mapped drive H:
out, errc := os.OpenFile("H:/00_SC/Dest01.txt", os.O_CREATE|os.O_APPEND|os.O_RDWR, 0666)
if errc != nil {
fmt.Println("Error Creating/Wrting to Dest file :", errc)
}
defer out.Close()
wr := bufio.NewWriter(out)
mystring := "another line here"
d, err := wr.WriteString(mystring)
if err != nil {
fmt.Println("Error writing by Writer: ", err)
}
errflush := wr.Flush()
if errflush != nil {
fmt.Println("Error Flushing the write to file", errflush)
}
wrtd, errw := out.Write([]byte("Write something in the file"))
if errw != nil {
fmt.Println("Error of Writte call", errw)
}
fmt.Println("Length of mystring = ", len(mystring))
fmt.Println("Bytes written to buffer = ", d)
fd, errf := fmt.Fprintf(out, "Another line to the file using Fprintf %d", d)
if errf != nil {
fmt.Println("Fprintf Error: ", errf)
}
fmt.Println("Bytes written to file by Fprintf = ", fd)
fmt.Println("Bytes written to file using Write", wrtd)
}
The file was created successfully, however, neither of the methods used write to it.
Bufio.WriteString neither write nor return an error !!
Fprintf failed with unclear error message which I searched and could not find a meaning or reason.
Here the output I got when running the compiled file:
Error Flushing the write to file write H:/00_SC/Dest01.txt: The parameter is incorrect.
Error of Write call write H:/00_SC/Dest01.txt: The parameter is incorrect.
Length of mystring = 17
Bytes writen to buffer = 17
Fprintf Error: write H:/00_SC/Dest01.txt: The parameter is incorrect.
Bytes written to file by Fprintf = 0
Bytes written to file using Write 0
I would appreciate your help and guidance finding the cause of this error.
Thanks
I'm new to golang and using ioutil.ReadFile(os.Args[1]) to fetch a file path from the cli and then processing each line using:
package main
import (
"bufio"
"fmt"
"log"
"os"
)
func main() {
file, err := os.Open(os.Args[1])
if err != nil {
log.Fatal(err)
}
defer file.Close()
scanner := bufio.NewScanner(file)
for scanner.Scan() {
fmt.Println(scanner.Text())
}
if err := scanner.Err(); err != nil {
log.Fatal(err)
}
}
Now I can get the output at stdout. But also get the following ERROR in the end:
msg:"BRUMBRUM";reference:cve,CVE-2007-2810;reference:blah;
msg:"WAKANDA";reference:cve,CVE-2007-2810;reference:blah; file name too long
exit status 1
My file path input is data/srulz.tcl
. FYI, the error message is not a part of the text file.
I need to know where is this going wrong here and how can this be improved?
UPDATE:
Provided issue reproducible code.
Go-ing with flag pkg for now until this mystery is solved
In the first line,
filePath, err := ioutil.ReadFile(os.Args[1])
Above step will read the whole file contents and return slice of byte and error. filePath variable will not store the file path instead its storing the content of file in bytes. I am wondering why are you not getting compile time error as filepath variable is slice of bytes whereas os.Open(filepath) the argument to os.Open will be string.
I am currently learning Go and I am trying to send the contents of a directory to another machine over a plain tcp connection using Go's net package.
It works fine with individual files and small folders, but I run into issues if the folder contains many subfolders and larger files. I am using the filepath.Walk function to traverse over all files in the given directory. For each file or directory I send, I also send a header that provides the receiver with file name, file size, isDir properties so I know for how long I need to read for when reading the content. The issue I am having is that after a while when reading the header, I am reading actual file content of the previous file even though I already read that file from the connection
Here is the writer side. I simply traverse over the directory.
func transferDir(session *Session, dir string) error {
return filepath.Walk(dir, func(path string, info os.FileInfo, err error) error {
if err != nil {
return err
}
header := Header{Name: info.Name(), Size: info.Size(), Path: path}
if info.IsDir() {
header.SetDirBit()
session.WriteHeader(header)
return nil // nothing more to write
}
// content is a file. write the file now byte by byte
file, err := os.Open(path)
inf, err := file.Stat()
header.Size = inf.Size() // get the true size of the file
session.WriteHeader(header)
defer file.Close()
if err != nil {
return err
}
buf := make([]byte, BUF_SIZE)
for {
n, err := file.Read(buf)
if err != nil {
if err == io.EOF {
session.Write(buf[:n])
session.Flush()
break
} else {
log.Println(err)
return err
}
}
session.Write(buf[:n])
session.Flush()
}
return nil
})
And here is the reader part
func (c *Clone) readFile(h Header) error {
file, err := os.Create(h.Path)
defer file.Close()
if err != nil {
return err
}
var receivedByts int64
fmt.Printf("Reading File: %s Size: %d\n", h.Name, h.Size)
for {
if (h.Size - receivedByts) < BUF_SIZE {
n, err := io.CopyN(file, c.sesh, (h.Size - receivedByts))
fmt.Println("Written: %d err: %s\n", n, err)
break
}
n, err := io.CopyN(file, c.sesh, BUF_SIZE)
fmt.Println("Written: %d err: %s\n", n, err)
receivedByts += BUF_SIZE
fmt.Println("Bytes Read: ", receivedByts)
}
return nil
}
Now the weird part is that when I am looking at the print statements I see something like:
Reading File: test.txt Size: 14024
Written 1024 nil
Bytes Read 1024
... This continues all the way to the break statement
And the total of the Bytes read equals the actual file size. Yet, the subsequent read for the header will return content from the test.txt file. Almost like there is still stuff in the buffer, but I think I read it already....
I am trying to take input from the keyboard and then store it in a text file but I am a bit confused on how to actually do it.
My current code is as follow at the moment:
// reads the file txt.txt
bs, err := ioutil.ReadFile("text.txt")
if err != nil {
panic(err)
}
// Prints out content
textInFile := string(bs)
fmt.Println(textInFile)
// Standard input from keyboard
var userInput string
fmt.Scanln(&userInput)
//Now I want to write input back to file text.txt
//func WriteFile(filename string, data []byte, perm os.FileMode) error
inputData := make([]byte, len(userInput))
err := ioutil.WriteFile("text.txt", inputData, )
There are so many functions in the "os" and "io" packages. I am very confused about which one I actually should use for this purpose.
I am also confused about what the third argument in the WriteFile function should be. In the documentation is says of type " perm os.FileMode" but since I am new to programming and Go I am a bit clueless.
Does anybody have any tips on how to proced?
Thanks in advance,
Marie
// reads the file txt.txt
bs, err := ioutil.ReadFile("text.txt")
if err != nil { //may want logic to create the file if it doesn't exist
panic(err)
}
var userInput []string
var err error = nil
var n int
//read in multiple lines from user input
//until user enters the EOF char
for ln := ""; err == nil; n, err = fmt.Scanln(ln) {
if n > 0 { //we actually read something into the string
userInput = append(userInput, ln)
} //if we didn't read anything, err is probably set
}
//open the file to append to it
//0666 corresponds to unix perms rw-rw-rw-,
//which means anyone can read or write it
out, err := os.OpenFile("text.txt", os.O_APPEND, 0666)
defer out.Close() //we'll close this file as we leave scope, no matter what
if err != nil { //assuming the file didn't somehow break
//write each of the user input lines followed by a newline
for _, outLn := range userInput {
io.WriteString(out, outLn+"\n")
}
}
I've made sure this compiles and runs on play.golang.org, but I'm not at my dev machine, so I can't verify that it's interacting with Stdin and the file entirely correctly. This should get you started though.
For example,
package main
import (
"fmt"
"io/ioutil"
"os"
)
func main() {
fname := "text.txt"
// print text file
textin, err := ioutil.ReadFile(fname)
if err == nil {
fmt.Println(string(textin))
}
// append text to file
f, err := os.OpenFile(fname, os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0666)
if err != nil {
panic(err)
}
var textout string
fmt.Scanln(&textout)
_, err = f.Write([]byte(textout))
if err != nil {
panic(err)
}
f.Close()
// print text file
textin, err = ioutil.ReadFile(fname)
if err != nil {
panic(err)
}
fmt.Println(string(textin))
}
If you simply want to append the user's input to a text file, you could just read the
input as you've already done and use ioutil.WriteFile, as you've tried to do.
So you already got the right idea.
To make your way go, the simplified solution would be this:
// Read old text
current, err := ioutil.ReadFile("text.txt")
// Standard input from keyboard
var userInput string
fmt.Scanln(&userInput)
// Append the new input to the old using builtin `append`
newContent := append(current, []byte(userInput)...)
// Now write the input back to file text.txt
err = ioutil.WriteFile("text.txt", newContent, 0666)
The last parameter of WriteFile is a flag which specifies the various options for
files. The higher bits are options like file type (os.ModeDir, for example) and the lower
bits represent the permissions in form of UNIX permissions (0666, in octal format, stands for user rw, group rw, others rw). See the documentation for more details.
Now that your code works, we can improve it. For example by keeping the file open
instead of opening it twice:
// Open the file for read and write (O_RDRW), append to it if it has
// content, create it if it does not exit, use 0666 for permissions
// on creation.
file, err := os.OpenFile("text.txt", os.O_RDWR|os.O_APPEND|os.O_CREATE, 0666)
// Close the file when the surrounding function exists
defer file.Close()
// Read old content
current, err := ioutil.ReadAll(file)
// Do something with that old content, for example, print it
fmt.Println(string(current))
// Standard input from keyboard
var userInput string
fmt.Scanln(&userInput)
// Now write the input back to file text.txt
_, err = file.WriteString(userInput)
The magic here is, that you use the flag os.O_APPEND while opening the file,
which makes file.WriteString() append. Note that you need to close the file after
opening it, which we do after the function exists using the defer keyword.