my go script should add one newline before matching the regEx-Search-String ^(.+[,]+\n).
The Prototype i had tested before into the editor:
i want add newlines before this lines: \n$1.
This works if i try it into the Text-Editor.
If i try this (see line 24) with my script it is changing nothing and sends no error.
Any ideas what i do wrong?
Example
i like to use PCRE like it works in this Example https://regex101.com/r/sB9wW6/17
Same Example here:
Example source
Dear sir,
Thanks for your interest.
expected result
#### here is a newline ####
Dear sir,
Thanks for your interest.
result is (produced by the script below)
Dear sir,
Thanks for your interest.
go script:
// replace in files and store the new copy of it.
package main
import (
"fmt"
"io/ioutil"
"os"
"path/filepath"
"regexp"
"strings"
"time"
)
func visit(path string, fi os.FileInfo, err error) error {
matched, err := filepath.Match("*.csv", fi.Name())
if err != nil {
panic(err)
return err
}
if matched {
read, err := ioutil.ReadFile(path)
if err != nil {
panic(err)
}
newContents := string(read)
newContents = regExRepl(`^(.+[,]+\n)`, newContents, `\n\n\n$1`)
var re = regexp.MustCompile(`[\W]+`)
t_yymmdd := regexp.MustCompile(`[\W]+`).ReplaceAllString(time.Now().Format(time.RFC3339), `-`)[:10]
t_hhss := re.ReplaceAllString(time.Now().Format(time.RFC3339), `-`)[11:19]
t_yymmddhhss := t_yymmdd + "_" + t_hhss
fmt.Println(t_yymmddhhss)
filePath := fileNameWithoutExtension(path) + t_yymmddhhss + ".csv"
err = ioutil.WriteFile(filePath, []byte(newContents), 0)
if err != nil {
panic(err)
}
}
return nil
}
func regExRepl(regExPatt string, newContents string, regExRepl string) string {
return regexp.MustCompile(regExPatt).ReplaceAllString(newContents, regExRepl)
}
func main() {
err := filepath.Walk("./november2020messages.csv", visit) // <== read all files in current folder 20:12:06 22:44:42
if err != nil {
panic(err)
}
}
func fileNameWithoutExtension(fileName string) string {
return strings.TrimSuffix(fileName, filepath.Ext(fileName))
}
for interpretation \n as newline don't us
`\n`` use "\n"
may use ^(.+[,]+) instead ^(.+[,]+\n) and ad (?m) before for multi-line replacements
this suggestion you could test here: https://play.golang.org/p/25_0GJ93oCT
The following example illustrates the difference (in golang-playground here https://play.golang.org/p/FkPwElhx-Xu ):
// example from:
package main
import (
"fmt"
"regexp"
)
func main() {
newContents := `line 1,
line 2
line a,
line b`
newContents1 := regexp.MustCompile(`^(.+[,]+\n)`).ReplaceAllString(newContents, `\n$1`)
fmt.Println("hi\n" + newContents1)
newContents1 = regexp.MustCompile(`(?m)^(.+[,]+\n)`).ReplaceAllString(newContents, "\n$1")
fmt.Println("ho\n" + newContents1)
}
Result:
hi
\nline 1,
line 2
line a,
line b
ho
line 1,
line 2
line a,
line b
Related
We can find the byte offset of a pattern from file by
"grep -ob pattern filename";
However, grep is not utf8 safe.
How do I find byte offset of a pattern in Go? The file is process log, which can be in TB.
This is what I want to get in Go:
$ cat fname
hello world
findme
hello 世界
findme again
...
$ grep -ob findme fname
12:findme
32:findme
FindAllStringIndex(s string, n int) returns byte start/finish indexes (i.e., slices) of all successive matches of the expression:
package main
import "fmt"
import "io/ioutil"
import "regexp"
func main() {
fname := "C:\\Users\\UserName\\go\\src\\so56798431\\fname"
b, err := ioutil.ReadFile(fname)
if err != nil {
panic(err)
}
re, err := regexp.Compile("findme")
if err != nil {
// handle error
}
fmt.Println(re.FindAllStringIndex(string(b), -1))
}
Output:
[[12 18] [32 38]]
Note: I did this on Microsoft Windows, but saved the file in UNIX format (linefeed); if input file saved in Windows format (carriage return & linefeed) the byte offsets would increment to 13 and 35, respectively.
UPDATE: for large files, use bufio.Scanner; for example:
package main
import (
"bufio"
"fmt"
"log"
"os"
"regexp"
)
func main() {
fname, err := os.Open("C:\\Users\\UserName\\go\\src\\so56798431\\fname")
if err != nil {
log.Fatal(err)
}
defer fname.Close()
re, err := regexp.Compile("findme")
if err != nil {
// handle error
}
scanner := bufio.NewScanner(fname)
bytesRead := 0
for scanner.Scan() {
b := scanner.Text()
//fmt.Println(b)
results := re.FindAllStringIndex(b, -1)
for _, result := range results {
fmt.Println(bytesRead + result[0])
}
// account for UNIX EOL marker
bytesRead += len(b) + 1
}
if err := scanner.Err(); err != nil {
log.Fatal(err)
}
}
Output:
12
32
I'm new to Golang, starting out with some examples. Currently, what I'm trying to do is reading a file line by line and replace it with another string in case it meets a certain condition.
The file is use for testing purposes contains four lines:
one
two
three
four
The code working on that file looks like this:
func main() {
file, err := os.OpenFile("test.txt", os.O_RDWR, 0666)
if err != nil {
panic(err)
}
reader := bufio.NewReader(file)
for {
fmt.Print("Try to read ...\n")
pos,_ := file.Seek(0, 1)
log.Printf("Position in file is: %d", pos)
bytes, _, _ := reader.ReadLine()
if (len(bytes) == 0) {
break
}
lineString := string(bytes)
if(lineString == "two") {
file.Seek(int64(-(len(lineString))), 1)
file.WriteString("This is a test.")
}
fmt.Printf(lineString + "\n")
}
file.Close()
}
As you can see in the code snippet, I want to replace the string "two" with "This is a test" as soon as this string is read from the file.
In order to get the current position within the file I use Go's Seek method.
However, what happens is that always the last line gets replaced by This is a test, making the file looking like this:
one
two
three
This is a test
Examining the output of the print statement which writes the current file position to the terminal, I get that kind of output after the first line has been read:
2016/12/28 21:10:31 Try to read ...
2016/12/28 21:10:31 Position in file is: 19
So after the first read, the position cursor already points to the end of my file, which explains why the new string gets appended to the end. Does anyone understand what is happening here or rather what is causing that behavior?
The Reader is not controller by the file.Seek. You have declared the reader as: reader := bufio.NewReader(file) and then you read one line at a time bytes, _, _ := reader.ReadLine() however the file.Seek does not change the position that the reader is reading.
Suggest you read about the ReadSeeker in the docs and switch over to using that. Also there is an example using the SectionReader.
Aside from the incorrect seek usage, the difficulty is that the line you're replacing isn't the same length as the replacement. The standard approach is to create a new (temporary) file with the modifications. Assuming that is successful, replace the original file with the new one.
package main
import (
"bufio"
"io"
"io/ioutil"
"log"
"os"
)
func main() {
// file we're modifying
name := "text.txt"
// open original file
f, err := os.Open(name)
if err != nil {
log.Fatal(err)
}
defer f.Close()
// create temp file
tmp, err := ioutil.TempFile("", "replace-*")
if err != nil {
log.Fatal(err)
}
defer tmp.Close()
// replace while copying from f to tmp
if err := replace(f, tmp); err != nil {
log.Fatal(err)
}
// make sure the tmp file was successfully written to
if err := tmp.Close(); err != nil {
log.Fatal(err)
}
// close the file we're reading from
if err := f.Close(); err != nil {
log.Fatal(err)
}
// overwrite the original file with the temp file
if err := os.Rename(tmp.Name(), name); err != nil {
log.Fatal(err)
}
}
func replace(r io.Reader, w io.Writer) error {
// use scanner to read line by line
sc := bufio.NewScanner(r)
for sc.Scan() {
line := sc.Text()
if line == "two" {
line = "This is a test."
}
if _, err := io.WriteString(w, line+"\n"); err != nil {
return err
}
}
return sc.Err()
}
For more complex replacements, I've implemented a package which can replace regular expression matches. https://github.com/icholy/replace
import (
"io"
"regexp"
"github.com/icholy/replace"
"golang.org/x/text/transform"
)
func replace2(r io.Reader, w io.Writer) error {
// compile multi-line regular expression
re := regexp.MustCompile(`(?m)^two$`)
// create replace transformer
tr := replace.RegexpString(re, "This is a test.")
// copy while transforming
_, err := io.Copy(w, transform.NewReader(r, tr))
return err
}
OS package has Expand function which I believe can be used to solve similar problem.
Explanation:
file.txt
one
two
${num}
four
main.go
package main
import (
"fmt"
"os"
)
var FILENAME = "file.txt"
func main() {
file, err := os.ReadFile(FILENAME)
if err != nil {
panic(err)
}
mapper := func(placeholderName string) string {
switch placeholderName {
case "num":
return "three"
}
return ""
}
fmt.Println(os.Expand(string(file), mapper))
}
output
one
two
three
four
Additionally, you may create a config (yml or json) and
populate that data in the map that can be used as a lookup table to store placeholders as well as their replacement strings and modify mapper part to use this table to lookup placeholders from input file.
e.g map will look like this,
table := map[string]string {
"num": "three"
}
mapper := func(placeholderName string) string {
if val, ok := table[placeholderName]; ok {
return val
}
return ""
}
References:
os.Expand documentation: https://pkg.go.dev/os#Expand
Playground
I've tried to search on Google for pattern matching function between file and string but I could not find it. I've also tried to use strings.Contains(), but it gives wrong result in large input file.
Is there any function in Go for searching string in some file?
If no, is there another way to resolve this problem?
Here is my code:
package main
import (
"bufio"
"fmt"
"io/ioutil"
"os"
"strings"
)
func main() {
reader := bufio.NewReader(os.Stdin)
fmt.Print("Enter text: ")
text, _ := reader.ReadString('\n')
// read the whole file at once
b, err := ioutil.ReadFile("input.txt")
if err != nil {
panic(err)
}
s := string(b)
length := len(s)
//check whether s contains substring text
fmt.Println(strings.Contains(s, text))
}
If I read your question correctly you want to read from a file and determine if a string entered at the command line is in that file... And I think the problem that you are seeing has to do with the string delimiter, the reader.ReadString('\n') bit, and not string.Contains().
In my opinion it will be a little bit easier to make what you want work with fmt.Scanln; it will simplify things and will return a result that I'm pretty sure is what you want. Try this variation of your code:
package main
import (
"fmt"
"io/ioutil"
"strings"
)
func main() {
var text string
fmt.Print("Enter text: ")
// get the sub string to search from the user
fmt.Scanln(&text)
// read the whole file at once
b, err := ioutil.ReadFile("input.txt")
if err != nil {
panic(err)
}
s := string(b)
// //check whether s contains substring text
fmt.Println(strings.Contains(s, text))
}
I am just adding a flag to use command line arguments. If nothing is passed it will prompt you :).
package main
import (
"flag"
"fmt"
"io/ioutil"
"strings"
)
//Usage go run filename -text=dataYouAreLookingfor
//if looking for Nissan in file the command will be
// go run filename -text=Nissan
func main() {
var text string
// use it as cmdline argument
textArg := flag.String("text", "", "Text to search for")
flag.Parse()
// if cmdline arg was not passed ask
if fmt.Sprintf("%s", *textArg) == "" {
fmt.Print("Enter text: ")
// get the sub string to search from the user
fmt.Scanln(&text)
} else {
text = fmt.Sprintf("%s", *textArg)
}
// read the whole file at once
b, err := ioutil.ReadFile("input.txt")
if err != nil {
panic(err)
}
s := string(b)
// //check whether s contains substring text
fmt.Println(strings.Contains(s, text))
}
How do I replace a line in a text file with a new line?
Assume I've opened the file and have every line in an array of string objects i'm now looping through
//find line with ']'
for i, line := range lines {
if strings.Contains(line, ']') {
//replace line with "LOL"
?
}
}
What matters here is not so much what you do in that loop. It's not like you're gonna be directly editing the file on the fly.
The most simple solution for you is to just replace the string in the array and then write the contents of the array back to your file when you're finished.
Here's some code I put together in a minute or two. It properly compiles and runs on my machine.
package main
import (
"io/ioutil"
"log"
"strings"
)
func main() {
input, err := ioutil.ReadFile("myfile")
if err != nil {
log.Fatalln(err)
}
lines := strings.Split(string(input), "\n")
for i, line := range lines {
if strings.Contains(line, "]") {
lines[i] = "LOL"
}
}
output := strings.Join(lines, "\n")
err = ioutil.WriteFile("myfile", []byte(output), 0644)
if err != nil {
log.Fatalln(err)
}
}
There's a gist too (with the same code)
https://gist.github.com/dallarosa/b58b0e3425761e0a7cf6
I want to ReadBytes until "\n" for a text file, not a bufio.
Is there a way to do this without converting to a bufio?
There are many ways to do it, but wrapping with bufio is what I would suggest. But if that doesn't work for you (why not?), you can go ahead and read single bytes like this:
Full working example:
package main
import (
"bytes"
"fmt"
"io"
)
// ReadLine reads a line delimited by \n from the io.Reader
// Unlike bufio, it does so rather inefficiently by reading one byte at a time
func ReadLine(r io.Reader) (line []byte, err error) {
b := make([]byte, 1)
var l int
for err == nil {
l, err = r.Read(b)
if l > 0 {
if b[0] == '\n' {
return
}
line = append(line, b...)
}
}
return
}
var data = `Hello, world!
I will write
three lines.`
func main() {
b := bytes.NewBufferString(data)
for {
line, err := ReadLine(b)
fmt.Println("Line: ", string(line))
if err != nil {
return
}
}
}
Output:
Line: Hello, world!
Line: I will write
Line: three lines.
Playground: http://play.golang.org/p/dfb0GHPpnm