I have this go code:
func readTwoLines() {
reader := bufio.NewReader(os.Stdin)
line, _ := reader.ReadString('\n')
fmt.Println(line)
line, _ = reader.ReadString('\n')
fmt.Println(line)
}
For the input:
hello
bye
the output is:
hello
bye
Everything ok. But now, if I create one reader per line:
func readTwoLines() {
line, _ := bufio.NewReader(os.Stdin).ReadString('\n')
fmt.Println(line)
line, err := bufio.NewReader(os.Stdin).ReadString('\n')
if err != nil {
fmt.Println(err)
}
fmt.Println(line)
}
there is an EOF error, in the second line reading.
Why is it happening?
For simple uses, a Scanner may be more convenient.
You should not use two readers, first read, buffers 4096 bytes of input:
// NewReader returns a new Reader whose buffer has the default size.
func NewReader(rd io.Reader) *Reader {
return NewReaderSize(rd, defaultBufSize)
}
and defaultBufSize = 4096
and even your input contains 4000 bytes, still second read got nothing to read.
but if you enter input more than 4096 bytes it will work.
If ReadString encounters an error before finding a delimiter, it
returns the data read before the error and the error itself (often
io.EOF).
it is by design, see doc:
// ReadString reads until the first occurrence of delim in the input,
// returning a string containing the data up to and including the delimiter.
// If ReadString encounters an error before finding a delimiter,
// it returns the data read before the error and the error itself (often io.EOF).
// ReadString returns err != nil if and only if the returned data does not end in
// delim.
// For simple uses, a Scanner may be more convenient.
func (b *Reader) ReadString(delim byte) (string, error) {
bytes, err := b.ReadBytes(delim)
return string(bytes), err
}
try this:
package main
import (
"bufio"
"fmt"
"os"
)
func main() {
scanner := bufio.NewScanner(os.Stdin)
for scanner.Scan() {
fmt.Println(scanner.Text()) // Println will add back the final '\n'
}
if err := scanner.Err(); err != nil {
fmt.Fprintln(os.Stderr, "reading standard input:", err)
}
}
run:
go run m.go < in.txt
output:
hello
bye
in.txt file:
hello
bye
I hope this helps.
Related
I need to automate the input of a code segment like bellow where the inputs of the ReadString are distinct.
package main
import (
"bufio"
"fmt"
"os"
)
func main() {
buf := bufio.NewReader(os.Stdin)
value, err := buf.ReadString('\n')
if err != nil {
fmt.Println(err)
} else {
fmt.Println(value)
}
buf = bufio.NewReader(os.Stdin)
value, err = buf.ReadString('\n')
if err != nil {
fmt.Println(err)
} else {
fmt.Println(value)
}
}
I have tried several formats like the bellow one following answers of this question, unfortunately, none worked.
>> echo "data1
data2" | go run main.go
output: data1
EOF
Here data1 and data2 and input of the separate ReadString methods. I don't have control over the source code. So, I can only try changing the bash input. How to resolve this issue?
This is happening because the second string does not end with a newline. Looking at the documentation for ReadString:
If ReadString encounters an error before finding a delimiter, it returns the data read before the error and the error itself (often io.EOF).
So, even though error is non-nil, you have the data. The following change should work for this specific case:
buf = bufio.NewReader(os.Stdin)
value, err = buf.ReadString('\n')
if value!="" {
fmt.Println(value)
}
if err != nil {
fmt.Println(err)
}
In general, even if you get an error from ReadString, you may still have nonempty data returned from the function.
I'm trying to read a line and words from stdin using the same bufio.Scanner instance.
I've tried using two bufio.Scanner first but it the second scanner can't read anything.
package main
import (
"bufio"
"fmt"
"os"
"strings"
)
func main() {
r := strings.NewReader("first line\n2nd line")
scanner := bufio.NewScanner(r)
scanner2 := bufio.NewScanner(r)
fmt.Println("scanning line")
if scanner.Scan() {
fmt.Println("Scanned line: ", scanner.Text())
}
if err := scanner.Err(); err != nil {
fmt.Fprintln(os.Stderr, "reading standard input:", err)
}
scanner2.Split(bufio.ScanWords)
fmt.Println("scanning word")
if scanner2.Scan() {
// i'm expecting that this would read "2nd"
fmt.Println("Scanned word: ", scanner2.Text())
}
if err := scanner2.Err(); err != nil {
fmt.Fprintln(os.Stderr, "reading standard input:", err)
}
}
Then I tried setting the scanner.Split , but it doesn't allow calling the scanner.Split after calling scanner.Scan.
Is there a way that I can read a line and then words from the same bufio.Scanner?
First. You can't read same line from reader multiple times. You read it once reader going to be empty, that is why the second scanner can't read anything. In your example you are using strings.Reader no os.Stdin. You can use something like io.TeeReader to duplicate to read the reader more times but this isn't your case.
Second. You cant call scanner.Split after scanner.Scan was called, you need to create a new scanner with os.Stdin as input.
For example:
// this gonna read from stdin complete lines
scanner := bufio.NewScanner(os.Stdin)
// scanning lines until Q is typed
for scanner.Scan() {
t := scanner.Text()
if t == "Q" {
break
}
fmt.Println(scanner.Text())
}
// new scanner to read but this time words
scanner = bufio.NewScanner(os.Stdin)
scanner.Split(bufio.ScanWords)
for scanner.Scan() {
fmt.Println(scanner.Text()) // Println will add back the final '\n'
}
I start learning Go and I am a bit puzzled by the fact it includes the EOF when using the ioutil.ReadFile function. I want, for example, to read a file and parse all its lines on a field separator.
Sample input File:
CZG;KCZG;some text
EKY;KEKY;some text
A50;KA50;some text
UKY;UCFL;some text
MIC;KMIC;some text
K2M;K23M;some text
This is what I do to read and parse that file:
import(
"fmt"
"log"
"io/ioutil"
"strings"
)
func main() {
/* Read file */
airportsFile := "/path/to/file/ad_iata"
content, err := ioutil.ReadFile(airportsFile)
if err != nil {
log.Fatal(err)
}
/* split content on EOL */
lines := strings.Split(string(content), "\n")
/* split line on field separator ; */
for _, line := range lines {
lineSplit := strings.Split(line, ";")
fmt.Println(lineSplit)
}
}
The string.Split function adds a empty element at the end of the lineSplit slice when it sees the EOF (nothing to parse). Therefore, if I want to access the second index of that slice (lineSplit[1]) I run into a panic: runtime error: index out of range. I have to restrict the range by doing this
/* split line on field separator ; */
lenLines := len(lines) -1
for _, line := range lines[:lenLines] {
lineSplit := strings.Split(line, ";")
fmt.Println(lineSplit[1])
}
Is there a better way if I want to keep using ReadFile for its terseness ?
The same problem occurs when using ioutil.ReadAll
There is no such thing as an "EOF byte" or "EOF character". What you are seeing is probably caused by a line break character ('\n') at the very end of the file.
To read a file line by line, it's more idiomatic to use bufio.Scanner instead:
file, err := os.Open(airportsFile)
if err != nil {
log.Fatal(err)
}
defer file.Close()
scanner := bufio.NewScanner(file)
for scanner.Scan() {
line := scanner.Text()
// ... use line as you please ...
}
if err := scanner.Err(); err != nil {
log.Fatal(err)
}
And this actually addresses your problem, because Scanner will read the final newline without starting a new line, as evidenced by this playground example.
Your input File seeems to be CSV file, so you can use encoding/csv
airportsFile := "/path/to/file/ad_iata"
content, err := os.Open(airportsFile)
if err != nil {
log.Fatal(err)
}
r := csv.NewReader(content)
r.Comma = ';'
records, err := r.ReadAll() /* split line on field separator ; */
if err != nil {
log.Fatal(err)
}
fmt.Println(records)
which looks terse enough for me and provide correct output
[[CZG KCZG some text] [EKY KEKY some text] [A50 KA50 some text] [UKY UCFL some text] [MIC KMIC some text] [K2M K23M some text]]
You may use scanner.Err() to check for errors on file read.
// Err returns the first non-EOF error that was encountered by the Scanner.
func (s *Scanner) Err() error {
if s.err == io.EOF {
return nil
}
return s.err
}
In general in go the idiomatic way to read and parse a file is to use bufio.NewScanner which accept as an input parameter the file to read and returns a new Scanner.
Considering the above remarks here is a way you can read and parse a file:
package main
import (
"bufio"
"fmt"
"os"
)
func main() {
input, err := os.Open("example.txt")
if err != nil {
panic("Error happend during opening the file. Please check if file exists!")
os.Exit(1)
}
defer input.Close()
scanner := bufio.NewScanner(input)
for scanner.Scan() {
line := scanner.Text()
fmt.Printf("%v\n", line)
}
if err := scanner.Err(); err != nil {
fmt.Fprintln(os.Stderr, "reading input:", err)
}
}
I'm experimenting getting keyboard input in a test case, basically I followed this example,
https://www.socketloop.com/tutorials/golang-read-input-from-console-line
and in my unit test case, it always result in error of "EOF" without giving me a chance to type in from keyboard.
Is there any special in Go unit test environment? Or I should look into another better option?
My code looks like,
func (o *Player) consoleReadLn() (line string) {
consoleReader := bufio.NewReader(os.Stdin)
line, err := consoleReader.ReadString('\n')
if err != nil {
panic(err.Error()) // it just panic: EOF
}
return
}
Firstly, your code should be corrected to:
import "testing/iotest"
func (o *Player) consoleReadLn() string {
consoleReader := bufio.NewReader(os.Stdin)
s := ""
for {
s1, err := consoleReader.ReadString('\n')
if err == io.EOF {
break
}
if err != nil && err != iotest.ErrTimeout {
panic("GetLines: " + err.Error())
}
s += s1
}
return s
}
Because you expect using \n as delimiter of a line of string, so it will return the data with EOF which is \n in Unix OS, see godoc bufio#Reader.ReadString:
ReadString reads until the first occurrence of delim in the input, returning a string containing the data up to and including the delimiter. If ReadString encounters an error before finding a delimiter, it returns the data read before the error and the error itself (often io.EOF). ReadString returns err != nil if and only if the returned data does not end in delim. For simple uses, a Scanner may be more convenient.
However, I suggest reading this answer Read from initial stdin in GO?
Secondly, it is hard to test STDIN in unit test context like go test. I found this mail saying:
the new go test runs tests with standard input connected
to /dev/null.
So I think it is hard to test os.Stdin via go test directly, for example, the following code confirmed it doesn't read /dev/stdin at all when running command echo this is stdin | go test ./:
import "io/ioutil"
import "testing"
import "fmt"
func TestSTDIN(t *testing.T) {
bytes, err := ioutil.ReadAll(os.Stdin)
if err != nil {
t.Fatal(err)
}
fmt.Println(string(bytes))
}
I currently have this piece of code that will read a file line by line (delimited by a \n)
file, _ := os.Open(filename) //deal with the error later
defer file.Close()
buf := bufio.NewReader(file)
for line, err := buf.ReadString('\n'); err != io.EOF; line, err = buf.ReadString('\n')
{
fmt.Println(strings.TrimRight(line, "\n"))
}
However I don't feel comfortable with writing buf.ReadString("\n") twice in the for loop, does anyone have any suggestions for improvement?
bufio.ReadString reads until the first occurrence of delim in the input,
returning a string containing the data up to and including the
delimiter. If ReadString encounters an error before finding a
delimiter, it returns the data read before the error and the error
itself (often io.EOF). ReadString returns err != nil if and only if
the returned data does not end in delim.
If buf.ReadString('\n') returns an error other than io.EOF, for example bufio.ErrBufferFull, you will be in an infinite loop. Also, if the file doesn't end in a '\n', you silently ignore the data after the last '\n'.
Here's a more robust solution, which executes buf.ReadString('\n') once.
package main
import (
"bufio"
"fmt"
"io"
"os"
"strings"
)
func main() {
filename := "FileName"
file, err := os.Open(filename)
if err != nil {
fmt.Println(err)
return
}
defer file.Close()
buf := bufio.NewReader(file)
for {
line, err := buf.ReadString('\n')
if err != nil {
if err != io.EOF || len(line) > 0 {
fmt.Println(err)
return
}
break
}
fmt.Println(strings.TrimRight(line, "\n"))
}
}
Most code that reads line by line can be improved by not reading line by line. If your goal is to read the file and access the lines, something like the following is almost always better.
package main
import (
"fmt"
"io/ioutil"
"log"
"strings"
)
func main() {
b, err := ioutil.ReadFile("filename")
if err != nil {
log.Fatal(err)
}
s := string(b) // convert []byte to string
s = strings.TrimRight(s, "\n") // strip \n on last line
ss := strings.Split(s, "\n") // split to []string
for _, s := range ss {
fmt.Println(s)
}
}
Any errors come to you at a single point so error handling is simplified. Stripping a newline off the last line allows for files that may or may not have that final newline, as Peter suggested. Most text files are tiny compared to available memory these days, so reading these in one gulp is appropriate.