How to wait until a specific line is printed to os.Stderr? - go

I'm running a Goroutine which, after some delay, logs a specific line to os.Stderr. I'd like to wait until that line is logged. So far, what I've tried is
package main
import (
"bufio"
"log"
"os"
"strings"
"time"
)
func main() {
go func() {
time.Sleep(time.Second)
log.Println("Hello, world!")
}()
scanner := bufio.NewScanner(os.Stderr)
for scanner.Scan() {
if strings.Contains(scanner.Text(), "Hello, world!") {
break
}
}
}
However, if I run this, it just blocks:
> go run main.go
2022/04/17 00:31:43 Hello, world!
Should this scanner not capture the standard error output and hit the break statement?

If you want to intercept output, you can use a pipe, like so:
r, w, _ := os.Pipe()
log.SetOutput(w)
go func() {
time.Sleep(time.Second)
log.Println("Hello, world!")
}()
scanner := bufio.NewScanner(r)
https://go.dev/play/p/HdEs5tbDYDE

It seems that if the io.Reader is os.Stderr, scanner.Scan() blocks indefinitely, whereas if I set it to a custom reader like a *bytes.Buffer, it returns immediately before the Goroutine has a chance to write to it.
Since the context of this logic is a unit test, I worked around this by using assert.Eventually:
func TestScan(t *testing.T) {
buf := bytes.NewBuffer([]byte{})
log.SetOutput(buf)
go func() {
time.Sleep(time.Second)
log.Println("Hello, world!")
}()
assert.Eventually(t, func() bool {
scanner := bufio.NewScanner(buf)
for scanner.Scan() {
if strings.Contains(scanner.Text(), "Hello, world!") {
return true
}
}
return false
}, 10*time.Second, 100*time.Millisecond)
}
This test passes in 1.1 seconds as expected:
> go test ./... -v
=== RUN TestScan
--- PASS: TestScan (1.10s)
PASS
(Granted, this is not the most efficient solution as it presumably reads the entire output each time, but it should do the job).

Related

golang goroutine synchronization expected behaviour

The following piece of code try to send to the channel on the main goroutine and receive from another goroutine but a few times it returns as expected but a few times it exits without printing any on the console screen
package main
import "fmt"
func main() {
ch := make(chan bool)
go func() {
data := <-ch
fmt.Printf("Received: %t", data)
}()
ch <- true
}
At the same time, the following piece of code works as expected everytime, one difference is that an additional check has been added to check if the channel is closed or not which always throws the same expected output.
Does this ensure that a check on the channel is a must than optional ? or anything wrong with the code
package main
import "fmt"
func main() {
ch := make(chan bool)
go func() {
data, ok := <-ch
if !ok {
fmt.Println("Channel closed")
return
}
fmt.Printf("Received: %t", data)
}()
ch <- true
}
You should wait for goroutine to complete before main routine exit.
package main
import (
"fmt"
"sync"
)
func main() {
ch := make(chan bool)
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
data := <-ch
fmt.Printf("Received: %t", data)
}()
ch <- true
wg.Wait()
}
The thing is your second piece of code doesn't print Received: true every time. I tested it several times.
As #jub0bs mentioned there is no guarantee that your goroutine finishes before the main routine. You must control it yourself.

Capture SIGINT when running tests in GoLand IDE

When running tests from command line, capturing SIGINT works fine. However, is there a way to pass SIGINT signal to code when running tests from GoLand IDE?
When running from command line:
go test -v -run TestSth and then calling Ctrl + C it captures fine.
Example code:
EDIT: it seems my example code now captures SIGINT as intended (Goland 2022.3.2)
package main
import (
"fmt"
"os"
"os/signal"
"syscall"
"testing"
"time"
)
func TestMain(m *testing.M) {
terminate := make(chan os.Signal)
signal.Notify(terminate, syscall.SIGINT)
go func() {
<-terminate
fmt.Println()
fmt.Println("CAPTURED!!!") // want to get here when running tests from IDE
}()
exitCode := m.Run()
os.Exit(exitCode)
}
func TestSth(t *testing.T) {
time.Sleep(time.Second * 5)
}
Get the current process information by calling FindProcess on the current PID and signal the interrupt to it using Process.Signal
func TestSth(t *testing.T) {
go func() {
// Sleep added for demonstrative purposes, can be removed
time.Sleep(time.Second * 1)
p, err := os.FindProcess(os.Getpid())
if err != nil {
panic(err)
}
p.Signal(syscall.SIGINT)
}()
time.Sleep(time.Second * 5)
}

Writing concurrently with channels

I wrote a short script to write a file concurrently.
One goroutine is supposed to write strings to a file while the others are supposed to send the messages through a channel to it.
However, for some really strange reason the file is created but no message is added to it through the channel.
package main
import (
"fmt"
"os"
"sync"
)
var wg sync.WaitGroup
var output = make(chan string)
func concurrent(n uint64) {
output <- fmt.Sprint(n)
defer wg.Done()
}
func printOutput() {
f, err := os.OpenFile("output.txt", os.O_CREATE|os.O_RDWR|os.O_APPEND, 0666);
if err != nil {
panic(err)
}
defer f.Close()
for msg := range output {
f.WriteString(msg+"\n")
}
}
func main() {
wg.Add(2)
go concurrent(1)
go concurrent(2)
wg.Wait()
close(output)
printOutput()
}
The printOutput() goroutine is executed completely, if I tried to write something after the for loop it would actually get into the file. So this leads me to think that range output might be null
You need to have something taking from the output channel as it is blocking until something removes what you put on it.
Not the only/best way to do it but: I moved printOutput() to above the other funcs and run it as a go routine and it prevents the deadlock.
package main
import (
"fmt"
"os"
"sync"
)
var wg sync.WaitGroup
var output = make(chan string)
func concurrent(n uint64) {
output <- fmt.Sprint(n)
defer wg.Done()
}
func printOutput() {
f, err := os.OpenFile("output.txt", os.O_CREATE|os.O_RDWR|os.O_APPEND, 0666)
if err != nil {
panic(err)
}
defer f.Close()
for msg := range output {
f.WriteString(msg + "\n")
}
}
func main() {
go printOutput()
wg.Add(2)
go concurrent(1)
go concurrent(2)
wg.Wait()
close(output)
}
One of the reason why you get a null output is because channels are blocking for both send/receive.
According to your flow, the code snippet below will never reach wg.Done(), as sending channel is expecting a receiving end to pull the data out. This is a typical deadlock example.
func concurrent(n uint64) {
output <- fmt.Sprint(n) // go routine is blocked until data in channel is fetched.
defer wg.Done()
}
Let's examine the main func:
func main() {
wg.Add(2)
go concurrent(1)
go concurrent(2)
wg.Wait() // the main thread will be waiting indefinitely here.
close(output)
printOutput()
}
My take on the problem:
package main
import (
"fmt"
"os"
"sync"
)
var wg sync.WaitGroup
var output = make(chan string)
var donePrinting = make(chan struct{})
func concurrent(n uint) {
defer wg.Done() // It only makes sense to defer
// wg.Done() before you do something.
// (like sending a string to the output channel)
output <- fmt.Sprint(n)
}
func printOutput() {
f, err := os.OpenFile("output.txt", os.O_CREATE|os.O_RDWR|os.O_APPEND, 0666)
if err != nil {
panic(err)
}
defer f.Close()
for msg := range output {
f.WriteString(msg + "\n")
}
donePrinting <- struct{}{}
}
func main() {
wg.Add(2)
go printOutput()
go concurrent(1)
go concurrent(2)
wg.Wait()
close(output)
<-donePrinting
}
Each concurrent function will deduct from the wait-group.
After the two concurrent goroutines finish, the wg.Wait() will unblock, and the next instruction (close(output)) will be executed. You have to wait for the two goroutines to finish before closing the channel. If, instead, you try the following:
go printOutput()
go concurrent(1)
go concurrent(2)
close(output)
wg.Wait()
you could end up with the close(output) instruction executing before any one of the concurrent goroutines concludes. If the channel closes before the concurrent goroutines run, they will crash at runtime, (while trying to write to a closed channel).
If, then, you don’t wait up for the printOutput() goroutine to finish, you could actually quit main() before printOutput() has gotten the chance to finish writing to its file.
Because I want to wait for the printOutput() goroutine to finish before I quit the program, I also created a separate channel just to signal that printOutput() is done.
The <-donePrinting instruction blocks until main receives something over the donePrinting channel.
Once main receives anything (even the empty structure that printOutput() sends), it will unblock and run to conclusion.
https://play.golang.org/p/nXJoYLI758m

How can I properly write the `Read` and `Write` the `net.Pipe()`

I'm trying out the net.Pipe(). I thought writing the "haha" string and then reading it back might be a good experiment.
Here is my first version. It blocks on the Write
func TestNetPipe(t *testing.T) {
out1 := make([]byte, 10)
c1, c2 := net.Pipe()
c1.Write([]byte("haha"))
c2.Read(out1)
}
I tried to use a goroutine
func TestNetPipe(t *testing.T) {
out1 := make([]byte, 10)
c1, c2 := net.Pipe()
go func() {
c1.Write([]byte("haha"))
}()
fmt.Printf("%v\n", out1)
c2.Read(out1)
fmt.Printf("%v\n", out1)
}
It works. But I felt there is no guarantee that the Read will read the whole "haha" string. It might only read the "hah" part.
I'm wondering if there is a better way to demo the usage of net.Pipe()
Use ReadAll function from package io/ioutil.
As ReadAll function blocks until EOF the following code needs no synchronization of goroutines. The call of close method causes the EOF on the stream.
package main
import (
"fmt"
"io/ioutil"
"log"
"net"
)
func main() {
r, w := net.Pipe()
go func() {
w.Write([]byte("haha"))
w.Close()
}()
b, err := ioutil.ReadAll(r)
if err != nil {
log.Fatalf(err.Error())
}
fmt.Println(string(b))
}
Playground

Reading from stdin in golang

I'm trying to read from Stdin in Golang as I'm trying to implement a driver for Erlang. I have the following code:
package main
import (
"fmt"
"os"
"bufio"
"time"
)
func main() {
go func() {
stdout := bufio.NewWriter(os.Stdin)
p := []byte{121,100,125,'\n'}
stdout.Write(p)
}()
stdin := bufio.NewReader(os.Stdin)
values := make([]byte,4,4)
for{
fmt.Println("b")
if read_exact(stdin) > 0 {
stdin.Read(values)
fmt.Println("a")
give_func_write(values)
}else{
continue
}
}
}
func read_exact(r *bufio.Reader) int {
bits := make([]byte,3,3)
a,_ := r.Read(bits)
if a > 0 {
r.Reset(r)
return 1
}
return -1
}
func give_func_write(a []byte) bool {
fmt.Println("Yahu")
return true
}
However it seems that the give_func_write is never reached. I tried to start a goroutine to write to standard input after 2 seconds to test this.
What am I missing here?
Also the line r.Reset(r). Is this valid in go? What I tried to achieve is simply restart the reading from the beginning of the file. Is there a better way?
EDIT
After having played around I was able to find that the code is stuck at a,_ := r.Read(bits) in the read_exact function
I guess that I will need to have a protocol in which I send a \n to
make the input work and at the same time discard it when reading it
No, you don't. Stdin is line-buffered only if it's bound to terminal. You can run your program prog < /dev/zero or cat file | prog.
bufio.NewWriter(os.Stdin).Write(p)
You probably don't want to write to stdin. See "Writing to stdin and reading from stdout" for details.
Well, it's not particular clear for me what you're trying to achieve. I'm assuming, that you just want to read data from stdin by fixed-size chunks. Use io.ReadFull for this. Or if you want to use buffers, you can use Reader.Peek or Scanner to ensure, that specific number of bytes is available. I've changed your program to demonstrate the usage of io.ReadFull:
package main
import (
"fmt"
"io"
"time"
)
func main() {
input, output := io.Pipe()
go func() {
defer output.Close()
for _, m := range []byte("123456") {
output.Write([]byte{m})
time.Sleep(time.Second)
}
}()
message := make([]byte, 3)
_, err := io.ReadFull(input, message)
for err == nil {
fmt.Println(string(message))
_, err = io.ReadFull(input, message)
}
if err != io.EOF {
panic(err)
}
}
You can easily split it in two programs and test it that way. Just change input to os.Stdin.

Resources