I need to automate the input of a code segment like bellow where the inputs of the ReadString are distinct.
package main
import (
"bufio"
"fmt"
"os"
)
func main() {
buf := bufio.NewReader(os.Stdin)
value, err := buf.ReadString('\n')
if err != nil {
fmt.Println(err)
} else {
fmt.Println(value)
}
buf = bufio.NewReader(os.Stdin)
value, err = buf.ReadString('\n')
if err != nil {
fmt.Println(err)
} else {
fmt.Println(value)
}
}
I have tried several formats like the bellow one following answers of this question, unfortunately, none worked.
>> echo "data1
data2" | go run main.go
output: data1
EOF
Here data1 and data2 and input of the separate ReadString methods. I don't have control over the source code. So, I can only try changing the bash input. How to resolve this issue?
This is happening because the second string does not end with a newline. Looking at the documentation for ReadString:
If ReadString encounters an error before finding a delimiter, it returns the data read before the error and the error itself (often io.EOF).
So, even though error is non-nil, you have the data. The following change should work for this specific case:
buf = bufio.NewReader(os.Stdin)
value, err = buf.ReadString('\n')
if value!="" {
fmt.Println(value)
}
if err != nil {
fmt.Println(err)
}
In general, even if you get an error from ReadString, you may still have nonempty data returned from the function.
Related
I can not take input from the user in Golang by use fmt.scan().
package main
import "fmt"
func main() {
fmt.Print("Enter text: ")
var input string
e, _ := fmt.Scanln(&input)
fmt.Println(input)
fmt.Println(e)
}
image of code
After stopping the debugger:
image of code
The err added to code, but nothing happened.
func main() {
fmt.Print("Enter text: ")
var input string
e, err := fmt.Scanln(&input)
if err != nil {
fmt.Fprintln(os.Stderr, err)
return
}
fmt.Println(input)
fmt.Println(e)
}
Image after add err in my Code. What is "not available" in Next line (after my Input value: "51213")
You code has no problem. If you build your code with go build and run the binary directly in a terminal, you will see your code runs.
The problem you hit is because of the Delve and vscode debug console. The vscode debug console doesn't support read from stdin. You can check this issue: Cannot debug programs which read from STDIN for details.
No need for 'e'. Replace it with an underscore and remove the print statement.
import (
"fmt"
"os"
)
func main() {
fmt.Print("Enter text: \n")
var input string
_, err := fmt.Scanln(&input)
if err != nil {
fmt.Fprintln(os.Stderr, err)
return
}
fmt.Println(input)
}
For the record. If you want to input a numeric value, fmt.Scan stores the value in a variable as a string, if you would like perform any mathematical operation with it, you need to convert it either to int or float. A quick example:
func main() {
fmt.Println("Type your age: ")
var input string
_, err := fmt.Scanln(&input)
if err != nil {
fmt.Fprintln(os.Stderr, err)
return
}
fmt.Printf("%T\n", input) // outputs string
inputInt,_ := strconv.Atoi(input)
fmt.Printf("%T\n", inputInt) // outputs int
fmt.Printf("You were born in %d\n", 2021-inputInt)
}
Took me a while to figure it out, hope it helps!
I'm new to Golang, starting out with some examples. Currently, what I'm trying to do is reading a file line by line and replace it with another string in case it meets a certain condition.
The file is use for testing purposes contains four lines:
one
two
three
four
The code working on that file looks like this:
func main() {
file, err := os.OpenFile("test.txt", os.O_RDWR, 0666)
if err != nil {
panic(err)
}
reader := bufio.NewReader(file)
for {
fmt.Print("Try to read ...\n")
pos,_ := file.Seek(0, 1)
log.Printf("Position in file is: %d", pos)
bytes, _, _ := reader.ReadLine()
if (len(bytes) == 0) {
break
}
lineString := string(bytes)
if(lineString == "two") {
file.Seek(int64(-(len(lineString))), 1)
file.WriteString("This is a test.")
}
fmt.Printf(lineString + "\n")
}
file.Close()
}
As you can see in the code snippet, I want to replace the string "two" with "This is a test" as soon as this string is read from the file.
In order to get the current position within the file I use Go's Seek method.
However, what happens is that always the last line gets replaced by This is a test, making the file looking like this:
one
two
three
This is a test
Examining the output of the print statement which writes the current file position to the terminal, I get that kind of output after the first line has been read:
2016/12/28 21:10:31 Try to read ...
2016/12/28 21:10:31 Position in file is: 19
So after the first read, the position cursor already points to the end of my file, which explains why the new string gets appended to the end. Does anyone understand what is happening here or rather what is causing that behavior?
The Reader is not controller by the file.Seek. You have declared the reader as: reader := bufio.NewReader(file) and then you read one line at a time bytes, _, _ := reader.ReadLine() however the file.Seek does not change the position that the reader is reading.
Suggest you read about the ReadSeeker in the docs and switch over to using that. Also there is an example using the SectionReader.
Aside from the incorrect seek usage, the difficulty is that the line you're replacing isn't the same length as the replacement. The standard approach is to create a new (temporary) file with the modifications. Assuming that is successful, replace the original file with the new one.
package main
import (
"bufio"
"io"
"io/ioutil"
"log"
"os"
)
func main() {
// file we're modifying
name := "text.txt"
// open original file
f, err := os.Open(name)
if err != nil {
log.Fatal(err)
}
defer f.Close()
// create temp file
tmp, err := ioutil.TempFile("", "replace-*")
if err != nil {
log.Fatal(err)
}
defer tmp.Close()
// replace while copying from f to tmp
if err := replace(f, tmp); err != nil {
log.Fatal(err)
}
// make sure the tmp file was successfully written to
if err := tmp.Close(); err != nil {
log.Fatal(err)
}
// close the file we're reading from
if err := f.Close(); err != nil {
log.Fatal(err)
}
// overwrite the original file with the temp file
if err := os.Rename(tmp.Name(), name); err != nil {
log.Fatal(err)
}
}
func replace(r io.Reader, w io.Writer) error {
// use scanner to read line by line
sc := bufio.NewScanner(r)
for sc.Scan() {
line := sc.Text()
if line == "two" {
line = "This is a test."
}
if _, err := io.WriteString(w, line+"\n"); err != nil {
return err
}
}
return sc.Err()
}
For more complex replacements, I've implemented a package which can replace regular expression matches. https://github.com/icholy/replace
import (
"io"
"regexp"
"github.com/icholy/replace"
"golang.org/x/text/transform"
)
func replace2(r io.Reader, w io.Writer) error {
// compile multi-line regular expression
re := regexp.MustCompile(`(?m)^two$`)
// create replace transformer
tr := replace.RegexpString(re, "This is a test.")
// copy while transforming
_, err := io.Copy(w, transform.NewReader(r, tr))
return err
}
OS package has Expand function which I believe can be used to solve similar problem.
Explanation:
file.txt
one
two
${num}
four
main.go
package main
import (
"fmt"
"os"
)
var FILENAME = "file.txt"
func main() {
file, err := os.ReadFile(FILENAME)
if err != nil {
panic(err)
}
mapper := func(placeholderName string) string {
switch placeholderName {
case "num":
return "three"
}
return ""
}
fmt.Println(os.Expand(string(file), mapper))
}
output
one
two
three
four
Additionally, you may create a config (yml or json) and
populate that data in the map that can be used as a lookup table to store placeholders as well as their replacement strings and modify mapper part to use this table to lookup placeholders from input file.
e.g map will look like this,
table := map[string]string {
"num": "three"
}
mapper := func(placeholderName string) string {
if val, ok := table[placeholderName]; ok {
return val
}
return ""
}
References:
os.Expand documentation: https://pkg.go.dev/os#Expand
Playground
I start learning Go and I am a bit puzzled by the fact it includes the EOF when using the ioutil.ReadFile function. I want, for example, to read a file and parse all its lines on a field separator.
Sample input File:
CZG;KCZG;some text
EKY;KEKY;some text
A50;KA50;some text
UKY;UCFL;some text
MIC;KMIC;some text
K2M;K23M;some text
This is what I do to read and parse that file:
import(
"fmt"
"log"
"io/ioutil"
"strings"
)
func main() {
/* Read file */
airportsFile := "/path/to/file/ad_iata"
content, err := ioutil.ReadFile(airportsFile)
if err != nil {
log.Fatal(err)
}
/* split content on EOL */
lines := strings.Split(string(content), "\n")
/* split line on field separator ; */
for _, line := range lines {
lineSplit := strings.Split(line, ";")
fmt.Println(lineSplit)
}
}
The string.Split function adds a empty element at the end of the lineSplit slice when it sees the EOF (nothing to parse). Therefore, if I want to access the second index of that slice (lineSplit[1]) I run into a panic: runtime error: index out of range. I have to restrict the range by doing this
/* split line on field separator ; */
lenLines := len(lines) -1
for _, line := range lines[:lenLines] {
lineSplit := strings.Split(line, ";")
fmt.Println(lineSplit[1])
}
Is there a better way if I want to keep using ReadFile for its terseness ?
The same problem occurs when using ioutil.ReadAll
There is no such thing as an "EOF byte" or "EOF character". What you are seeing is probably caused by a line break character ('\n') at the very end of the file.
To read a file line by line, it's more idiomatic to use bufio.Scanner instead:
file, err := os.Open(airportsFile)
if err != nil {
log.Fatal(err)
}
defer file.Close()
scanner := bufio.NewScanner(file)
for scanner.Scan() {
line := scanner.Text()
// ... use line as you please ...
}
if err := scanner.Err(); err != nil {
log.Fatal(err)
}
And this actually addresses your problem, because Scanner will read the final newline without starting a new line, as evidenced by this playground example.
Your input File seeems to be CSV file, so you can use encoding/csv
airportsFile := "/path/to/file/ad_iata"
content, err := os.Open(airportsFile)
if err != nil {
log.Fatal(err)
}
r := csv.NewReader(content)
r.Comma = ';'
records, err := r.ReadAll() /* split line on field separator ; */
if err != nil {
log.Fatal(err)
}
fmt.Println(records)
which looks terse enough for me and provide correct output
[[CZG KCZG some text] [EKY KEKY some text] [A50 KA50 some text] [UKY UCFL some text] [MIC KMIC some text] [K2M K23M some text]]
You may use scanner.Err() to check for errors on file read.
// Err returns the first non-EOF error that was encountered by the Scanner.
func (s *Scanner) Err() error {
if s.err == io.EOF {
return nil
}
return s.err
}
In general in go the idiomatic way to read and parse a file is to use bufio.NewScanner which accept as an input parameter the file to read and returns a new Scanner.
Considering the above remarks here is a way you can read and parse a file:
package main
import (
"bufio"
"fmt"
"os"
)
func main() {
input, err := os.Open("example.txt")
if err != nil {
panic("Error happend during opening the file. Please check if file exists!")
os.Exit(1)
}
defer input.Close()
scanner := bufio.NewScanner(input)
for scanner.Scan() {
line := scanner.Text()
fmt.Printf("%v\n", line)
}
if err := scanner.Err(); err != nil {
fmt.Fprintln(os.Stderr, "reading input:", err)
}
}
I'm experimenting getting keyboard input in a test case, basically I followed this example,
https://www.socketloop.com/tutorials/golang-read-input-from-console-line
and in my unit test case, it always result in error of "EOF" without giving me a chance to type in from keyboard.
Is there any special in Go unit test environment? Or I should look into another better option?
My code looks like,
func (o *Player) consoleReadLn() (line string) {
consoleReader := bufio.NewReader(os.Stdin)
line, err := consoleReader.ReadString('\n')
if err != nil {
panic(err.Error()) // it just panic: EOF
}
return
}
Firstly, your code should be corrected to:
import "testing/iotest"
func (o *Player) consoleReadLn() string {
consoleReader := bufio.NewReader(os.Stdin)
s := ""
for {
s1, err := consoleReader.ReadString('\n')
if err == io.EOF {
break
}
if err != nil && err != iotest.ErrTimeout {
panic("GetLines: " + err.Error())
}
s += s1
}
return s
}
Because you expect using \n as delimiter of a line of string, so it will return the data with EOF which is \n in Unix OS, see godoc bufio#Reader.ReadString:
ReadString reads until the first occurrence of delim in the input, returning a string containing the data up to and including the delimiter. If ReadString encounters an error before finding a delimiter, it returns the data read before the error and the error itself (often io.EOF). ReadString returns err != nil if and only if the returned data does not end in delim. For simple uses, a Scanner may be more convenient.
However, I suggest reading this answer Read from initial stdin in GO?
Secondly, it is hard to test STDIN in unit test context like go test. I found this mail saying:
the new go test runs tests with standard input connected
to /dev/null.
So I think it is hard to test os.Stdin via go test directly, for example, the following code confirmed it doesn't read /dev/stdin at all when running command echo this is stdin | go test ./:
import "io/ioutil"
import "testing"
import "fmt"
func TestSTDIN(t *testing.T) {
bytes, err := ioutil.ReadAll(os.Stdin)
if err != nil {
t.Fatal(err)
}
fmt.Println(string(bytes))
}
I currently have this piece of code that will read a file line by line (delimited by a \n)
file, _ := os.Open(filename) //deal with the error later
defer file.Close()
buf := bufio.NewReader(file)
for line, err := buf.ReadString('\n'); err != io.EOF; line, err = buf.ReadString('\n')
{
fmt.Println(strings.TrimRight(line, "\n"))
}
However I don't feel comfortable with writing buf.ReadString("\n") twice in the for loop, does anyone have any suggestions for improvement?
bufio.ReadString reads until the first occurrence of delim in the input,
returning a string containing the data up to and including the
delimiter. If ReadString encounters an error before finding a
delimiter, it returns the data read before the error and the error
itself (often io.EOF). ReadString returns err != nil if and only if
the returned data does not end in delim.
If buf.ReadString('\n') returns an error other than io.EOF, for example bufio.ErrBufferFull, you will be in an infinite loop. Also, if the file doesn't end in a '\n', you silently ignore the data after the last '\n'.
Here's a more robust solution, which executes buf.ReadString('\n') once.
package main
import (
"bufio"
"fmt"
"io"
"os"
"strings"
)
func main() {
filename := "FileName"
file, err := os.Open(filename)
if err != nil {
fmt.Println(err)
return
}
defer file.Close()
buf := bufio.NewReader(file)
for {
line, err := buf.ReadString('\n')
if err != nil {
if err != io.EOF || len(line) > 0 {
fmt.Println(err)
return
}
break
}
fmt.Println(strings.TrimRight(line, "\n"))
}
}
Most code that reads line by line can be improved by not reading line by line. If your goal is to read the file and access the lines, something like the following is almost always better.
package main
import (
"fmt"
"io/ioutil"
"log"
"strings"
)
func main() {
b, err := ioutil.ReadFile("filename")
if err != nil {
log.Fatal(err)
}
s := string(b) // convert []byte to string
s = strings.TrimRight(s, "\n") // strip \n on last line
ss := strings.Split(s, "\n") // split to []string
for _, s := range ss {
fmt.Println(s)
}
}
Any errors come to you at a single point so error handling is simplified. Stripping a newline off the last line allows for files that may or may not have that final newline, as Peter suggested. Most text files are tiny compared to available memory these days, so reading these in one gulp is appropriate.