Reading from a text file in Golang? - go

Question

I want to do this:
Read a line from a text file.
Process the line.
Delete the line.
My first thought was to read the entire file into memory with ioutil.Readfile(),
but I'm not sure how to update the text file after the line has been processed,
and what happens if extra lines is added to the text file after it has been read into memory?
I normally write shell scripts and would do something like this:
while read -r line; do
echo "${line}"
sed -i 1d "${myList}"
done < "${myList}"
What is the best way to do this in Golang?

Answer 1

Use the bufio package.
Here's the basic syntax for opening a text file and looping through each line.
package main
import (
"bufio"
"fmt"
"os"
)
func main() {
// Open the file.
f, _ := os.Open("C:\\programs\\file.txt")
// Create a new Scanner for the file.
scanner := bufio.NewScanner(f)
// Loop over all lines in the file and print them.
for scanner.Scan() {
line := scanner.Text()
fmt.Println(line)
}
}

Answer 2

you have some options:
1- read file, process it, then write it back (you need to lock that file).
2- use binary file and invent (make use of) special data structure (like linked list) to optimize text processing (with line locking).
3- use ready made databases.
4- use Virtual filesystem inside your file, and treat each line like one file, see: https://github.com/lotrfan/vfs and https://github.com/blang/vfs
using file manager (like database server) solves the file locking dilemma.
and if the purpose of using file is one way communication which sender program just adds new line and receiver program just removes it, it is better to use os pipes (named pipe (FIFO)) or other interop methods.
see for Linux: Unix FIFO in go?
for Windows: https://github.com/natefinch/npipe
sample file writer:
package main
import (
"bufio"
"fmt"
"os"
"time"
)
func main() {
f, err := os.OpenFile("/tmp/file.txt", os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0666)
if err != nil {
panic(err)
}
defer f.Close()
for i := 0; ; i++ {
w := bufio.NewWriter(f)
_, err := fmt.Fprintln(w, i)
if err != nil {
panic(err)
}
w.Flush() // Flush writes any buffered data to the underlying io.Writer.
f.Sync() // commit the current contents of the file to stable storage.
fmt.Println("write", i)
time.Sleep(500 * time.Millisecond)
}
}
sample file reader:
package main
import (
"fmt"
"os"
"time"
)
func main() {
f, err := os.OpenFile("/tmp/file.txt", os.O_RDWR, 0666)
if err != nil {
panic(err)
}
defer f.Close()
i := 0
for {
n, err := fmt.Fscanln(f, &i)
if n == 1 {
fmt.Println(i)
}
if err != nil {
fmt.Println(err)
return
}
time.Sleep(500 * time.Millisecond)
}
}