godoc Example output with CRLF - go

I have written the following method:
func (c *Component) Encode(w io.Writer){
//encodes c and writes the bytes into w, containing a few CRLF linebreaks
}
I also wrote the function demonstrating the encoder:
func ExampleComponent_Encode() {
c := &Component{
Name: "DESCRIPTION",
}
c.Encode(os.Stdout)
//Output:
//BEGIN:DESCRIPTION
//END:DESCRIPTION
}
The Problem is now that this example fails the go test command because the linebreaks in the comments are \n linebreaks (I'm on Linux) while the linebreaks generated by c.Encode have to be \r\n(CRLF) linebreaks (as defined by some spec).
How can i get the example to not fail go test while also keeping it straightforward? Is there perhaps a way to hint go test/godoc on the linebreaks or get them to be more lenient?
I propably can hand-edit the linebreaks on these two lines or possibly the whole codebase but that will be pretty fragile and I want to avoid this solution.

Redirect the Encode io.Writer to a buffer. In the buffer, replace CRLF (\r\n) with LF (\n) for the example output. For example,
example_test.go:
package main
import (
"bytes"
"fmt"
"io"
"os"
"strings"
)
type Component struct{ Name string }
func (c *Component) Encode(w io.Writer) {
//encodes c and writes the bytes into w, containing a few CRLF linebreaks
w.Write([]byte("BEGIN:" + c.Name + "\r\n"))
w.Write([]byte("END:" + c.Name + "\r\n"))
}
func ExampleComponent_Encode() {
var buf bytes.Buffer
c := &Component{
Name: "DESCRIPTION",
}
c.Encode(&buf)
output := strings.Replace(buf.String(), "\r\n", "\n", -1)
fmt.Fprintf(os.Stdout, "%s", output)
//Output:
//BEGIN:DESCRIPTION
//END:DESCRIPTION
}
Output:
$ go test -v example_test.go
=== RUN ExampleComponent_Encode
--- PASS: ExampleComponent_Encode (0.00s)
PASS

Related

Go: CSV NewReader not getting the correct number of fields

How to get the correct number of fields when using NewReader ?
package main
import (
"encoding/csv"
"fmt"
"log"
"strings"
)
func main() {
parser := csv.NewReader(strings.NewReader(`||""FOO""||`))
parser.Comma = '|'
parser.LazyQuotes = true
record, err := parser.Read()
if err != nil {
log.Fatal(err)
}
fmt.Printf("record length: %v\n", len(record))
}
https://go.dev/play/p/gg-KYRciWFH
It should return 5, but instead I'm getting 3:
record length: 3
Program exited.
EDIT
I'm actually working with a big CSV file containing many double quotes.
After examining your code, I decided to modify it slightly and then print the results:
package main
import (
"encoding/csv"
"fmt"
"log"
"strings"
)
func main() {
parser := csv.NewReader(strings.NewReader(`x||""FOO""|x|x\n`))
parser.Comma = '|'
parser.LazyQuotes = true
record, err := parser.Read()
if err != nil {
log.Fatal(err)
}
fmt.Printf("record length: %v, Data: %v\n", len(record), strings.Join(record, ", "))
}
When you run this, the data is printed as x, , "FOO"||x|x\n". My thought is that when you end your entry with two double-quotes, the parser is assuming the string is still being quoted and therefore lumps the rest of the line into the third entry. This appears to be a bug with how lazy-quoting works in the csv package, however, when examining the documentation for LazyQuotes, you'll see this:
If LazyQuotes is true, a quote may appear in an unquoted field and a non-doubled quote may appear in a quoted field.
This doesn't mention anything about finding double quotes within double quotes. To fix this, you should either remove the quotes altogether or replace the double double-quotes ("") with double quotes (").
One other thing you might consider would be using the gocsv package. I've worked with this package in the past and it's reasonably stable. I'm not sure how it would respond to this specific issue, but it might be worth your time checking it out.
Note:
The encoding/csv package implements the RFC 4180 standard. If you have such input, that's not an RFC 4180 compliant CSV file and encoding/csv will not parse it properly.
You're misusing the quotes. Quoting a single field FOO is like this:
parser := csv.NewReader(strings.NewReader(`||"FOO"||`))
If you want the field to have the "FOO" value, you have to use 2 double quotes in a quoted field, so it should be:
parser := csv.NewReader(strings.NewReader(`||"""FOO"""||`))
This will output 5. Try it on the Go Playground.
What you have is this:
parser := csv.NewReader(strings.NewReader(`||""FOO""||`))
Since the second " character is not followed by a separator character, the field is not interrupted and the rest is processed as the content of the quoted field (which will terminate at the end of the line).
If you print the record:
fmt.Println(record)
fmt.Printf("%#v", record)
Output will be (try it on the Go Playground):
[ "FOO"||]
[]string{"", "", "\"FOO\"||"}
Quotes are a part of csv format.
There is a problem with go/csv shielding, you can try something like this:
package main
import (
"encoding/csv"
"fmt"
"log"
"strings"
)
func main() {
parser := csv.NewReader(strings.NewReader(`||FOO||`))
parser.Comma = '|'
parser.LazyQuotes = true
record, err := parser.Read()
if err != nil {
log.Fatal(err)
}
fmt.Printf("record length: %v\n", len(record))
fmt.Println(strings.Join(record, " /SEP/ "))
}
or like this:
package main
import (
"encoding/csv"
"fmt"
"log"
"strings"
)
func main() {
parser := csv.NewReader(strings.NewReader(`||"""FOO"""||`))
parser.Comma = '|'
parser.LazyQuotes = true
record, err := parser.Read()
if err != nil {
log.Fatal(err)
}
fmt.Printf("record length: %v\n", len(record))
fmt.Println(strings.Join(record, " SEP "))
}

Strange performing of map data type

I am trying to add a bunch of values in a map data type and after that trying to print it out. But it is performing strangely. When I am directly calling the map with the key it is giving me the correct output but not giving me any output when I am storing the key in a variable and then calling it. I am not been able to figure it out what is happening and why am I getting this kind of output. Can Somebody help me with the same.
package main
import (
"bufio"
"fmt"
"os"
)
func main(){
type Authentication struct {
password string
}
var authentication = map[string]Authentication{}
var user1 Authentication
user1.password = "abc"
authentication["def"] = user1
reader := bufio.NewReader(os.Stdin)
usid := readString(reader)
fmt.Println(authentication)
fmt.Println(authentication[usid])
fmt.Println(authentication["def"])
}
// Reading input functions
func readString(reader *bufio.Reader) string {
s, _ := reader.ReadString('\n')
for i := 0; i < len(s); i++ {
if s[i] == '\n' {
return s[:i]
}
}
return s
}
Input:
def
Output:
map[def:{abc}]
{abc}
You're trying to do the same thing twice in readString. But all you have to do is to cut it by one byte.
func readString(reader *bufio.Reader) string {
s, _ := reader.ReadString('\n')
return s[:len(s)-1]
}
The program in the question does not work when \r\n is used as the line terminator in stdin. The program removes the trailing \n from the line, but not the \r.
Fix by using bufio.Scanner instead of bufio.Reader to read lines from the input. The bufio.Scanner type removes line terminators.
func main() {
type Authentication struct {
password string
}
var authentication = map[string]Authentication{}
var user1 Authentication
user1.password = "abc"
authentication["def"] = user1
scanner := bufio.NewScanner(os.Stdin)
if !scanner.Scan() {
log.Fatal(scanner.Err())
}
usid := scanner.Text()
fmt.Println(authentication)
fmt.Println(authentication[usid])
fmt.Println(authentication["def"])
}
There can always be a better way of reading string, but I see your code works too. I ran it in my local and it gives the expected output:
From your description, I presume you are using go playground or any such platform. If that is so, the thing is, go playground doesn't take standard input, and your code has reader on os.Stdin. When I copy your code to playground and add the following line to check,
fmt.Printf("Length of usid: %d\nusid: %q\n", len(usid), usid)
I see the following output:
Length of usid: 0
usid: ""
Conclusion: There is no issue with variables, map or code, but just the stdin.

Go strings.Replace(input, "\n", "", -1) did not recognized

I'm trying to process the strings inputted by users, and wrote following code.
import (
"bufio"
"fmt"
"os"
"strings"
)
func main() {
var input string
fileScanner := bufio.NewScanner(os.Stdin)
fileScanner.Scan()
input = fileScanner.Text()
replaced := strings.Replace(input, "\n", "", -1)
fmt.Println(replace)
}
But I found "\n" was not replaced to "". "\n" does not seems as a string.
I tried it "." or "," instead of "\n" and it works.
I just started learning Go and this question might be too fundamental, but I appreciate for any advice.
"\" characters are processed as escape characters within a formated string and hence wont be replaced by strings.Replace
If you are really trying to replace the \n character itself, not the new line putting it inside a raw string literal should help. Refer the code below:
package main
import (
"fmt"
"strings"
)
func main() {
var input string
input = `\naaa`
replaced := strings.Replace(input, `\n`, "", -1)
fmt.Println(replaced)
}

Why does slicing the result of a ReadString() operation result in weird output?

package main
import (
"bufio"
"fmt"
"os"
)
func main() {
reader := bufio.NewReader(os.Stdin)
fmt.Printf("Input: ")
input, _ := reader.ReadString('\n')
fmt.Println("thing\n"[:5] + "\"")
fmt.Println(input[:len(input)-1] + "\"")
return
}
Running the code:
Input: thing
thing"
"hing
Why does the second concatenation behave oddly? It should produce identical results, assuming the ReadString() operation returns a string with a newline at the end. Please explain what is going on here.
That's because you're presumably on windows.
The actual input you make from your keyboard is not thing\n but thing\r\n
So when you do fmt.Println(input[:len(input)-1] + "\"") it only truncates the latest \n and leaves \r.
So the terminal prints thing, then reaches \r that returns carriage to the beginning of the string, then you print a double quote. But the carriage is in the first position now, and it effectively overwrites the first character of the line, leaving you with "hing

Text processing in Go - how to convert string to byte?

I'm writing a small pragram to number the paragraph:
put paragraph number in front of each paragraph in the form of [1]..., [2]....
Article title should be excluded.
Here is my program:
package main
import (
"fmt"
"io/ioutil"
)
var s_end = [3]string{".", "!", "?"}
func main() {
b, err := ioutil.ReadFile("i_have_a_dream.txt")
if err != nil {
panic(err)
}
p_num, s_num := 1, 1
for _, char := range b {
fmt.Printf("[%s]", p_num)
p_num += 1
if char == byte("\n") {
fmt.Printf("\n[%s]", p_num)
p_num += 1
} else {
fmt.Printf(char)
}
}
}
http://play.golang.org/p/f4S3vQbglY
I got this error:
prog.go:21: cannot convert "\n" to type byte
prog.go:21: cannot convert "\n" (type string) to type byte
prog.go:21: invalid operation: char == "\n" (mismatched types byte and string)
prog.go:25: cannot use char (type byte) as type string in argument to fmt.Printf
[process exited with non-zero status]
How to convert string to byte?
What is the general practice to process text? Read in, parse it by byte, or by line?
Update
I solved the problem by converting the buffer byte to string, replacing strings by regular expression. (Thanks to #Tomasz Kłak for the regexp help)
I put the code here for reference.
package main
import (
"fmt"
"io/ioutil"
"regexp"
)
func main() {
b, err := ioutil.ReadFile("i_have_a_dream.txt")
if err != nil {
panic(err)
}
s := string(b)
r := regexp.MustCompile("(\r\n)+")
counter := 1
repl := func(match string) string {
p_num := counter
counter++
return fmt.Sprintf("%s [%d] ", match, p_num)
}
fmt.Println(r.ReplaceAllStringFunc(s, repl))
}
Using "\n" causes it to be treated as an array, use '\n' to treat it as a single char.
A string cannot be converted into a byte in a meaningful way. Use one of the following approaches:
If you have a string literal like "a", consider using a rune literal like 'a' which can be converted into a byte.
If you want to take a byte out of a string, use an index expression like myString[42].
If you want to interpret the content of a string as a (decimal) number, use strconv.Atoi() or strconv.ParseInt().
Please notice that it is customary in Go to write programs that can deal with Unicode characters. Explaining how to do this would be too much for this answer, but there are tutorials out there which explain what kind of things to pay attention to.

Resources