Interrupt current tail and start new one - go

Question

So I am creating some log analyzer in golang and what I need is real-time tail -f of newly created files.
I am using tail package together with fsnotify package but I'm not very familiar with channels and routines in go so I need some assistance.
Currently program looks like this:
package main
import(
"fmt"
"github.com/hpcloud/tail"
"strings"
"log"
"net/smtp"
"time"
"github.com/fsnotify/fsnotify"
)
//this function needs to monitor for new files in directory
func newFileCheck() (newFilename chan string, err error) {
watcher, err := fsnotify.NewWatcher()
if err != nil {
return
}
err = watcher.Add("myDir")
if err != nil {
return
}
newFilename = make(chan string)
// Process events
go func() {
for {
select {
case ev := <-watcher.Events:
log.Println("event:", ev)
newFilename <- ev.Name // Relative path to the file
//t.Stop() //if I pass reference to t THIS IS NOT HAPPENING ?
case err := <-watcher.Errors:
log.Println("error:", err)
}
}
}()
return
}
func main() {
newFileName = "mylog_0000.log.txt"
fmt.Println("Processing log: ",newFileName)
newFilenameChan, err := newFileCheck()
if err != nil {
fmt.Println("ERR: ",err)
}
t := tailLog(newFileName)
go func() {
for {
select {
case name := <-newFilenameChan:
fmt.Println("New file created: ",name) //this will be printed only once and then on new events nothing is printed ?
//NONE of the lines abowe doesn't work
t.Stop()
t.Dead()
t.Done()
t = tailLog(name)
}
}
}()
}
func tailLog(fileName string) *tail.Tail{
var count = 0
// close the old one and read new file
t, err := tail.TailFile("/mydir/"+fileName, tail.Config{Follow: true, ReOpen: true})
for line := range t.Lines {
//fmt.Println("Line is:", line.Text)
//check do we have what we need
if strings.Contains(strings.ToLower(line.Text), "mfc"){
count++
//do other stuff
}
}
fmt.Println(err)
return t
}
So I can't figure out why newFileCheck functions prints event only for the fist time, and also I'm not able to figure out how to cancel active tail when new event happens, and then start tail again after that event ?

Answer 1

Consider this:
When the function main returns, the program exits. It does not wait
for other (non-main) goroutines to complete.
And look at this block of your code:
func main() {
newFileName = "mylog_0000.log.txt"
fmt.Println("Processing log: ",newFileName)
newFilenameChan, err := newFileCheck()
if err != nil {
fmt.Println("ERR: ",err)
}
t := tailLog(newFileName)
go func() {
for {
select {
case name := <-newFilenameChan:
fmt.Println("New file created: ",name) //this will be printed only once and then on new events nothing is printed ?
//NONE of the lines abowe doesn't work
t.Stop()
t.Dead()
t.Done()
t = tailLog(name)
}
}
}()
}
To handle this job with grace, you should learn more about concurrency in Golang. Use channels to control goroutines like stopping/starting/etc and WaitGroup if you are interested in order of executing/finishing tasks. When there is no control over your program flow, goroutines can lives their own lives and that is bad practice.

Answer 2

If you about this package https://github.com/hpcloud/tail
You need to create one goroutine after you are open file, then you need to check a new file.
I make it for checking a new logfile of SoftetherVPN-Server:
package main
import (
"fmt"
"github.com/hpcloud/tail"
"log"
"time"
)
func CheckFileName(filename string, f chan<- string) {
for {
newFilename := fmt.Sprintf("%v", time.Now().Format("vpn_20060102.log"))
// We have a new filename, that send it into channel
if newFilename != filename {
log.Println("A new logfile", newFilename)
f <- newFilename
return
}
}
}
func main() {
filenameChan := make(chan string)
for {
filename = fmt.Sprintf("%v", time.Now().Format("vpn_20060102.log"))
t, err := tail.TailFile(filename, tail.Config{Follow: true, ReOpen: true, MustExist: true})
if err != nil {
log.Println("No log file", filename, ". Waiting for 10 minute")
time.Sleep(time.Minute * 10)
continue
}
// Send goroutine for checking filename in loop
go CheckFileName(filename, filenameChan)
// Send goroutine that stops lines read in range
go func() {
for {
select {
case name := <-filenameChan:
log.Println("Received a new name for log:", name)
t.Stop()
return
}
}
}()
if err != nil {
log.Fatalln(err)
}
for line := range t.Lines {
fmt.Println(line)
}
}
}
So, that works for me