log.New(os.Stdin, "in", 1).Println("TT")
The following code out stream looks like it should use stdout or stderr, but I'm using the stdin input stream.
It should go into some panic, but output to the console normally.
Why is this working fine?
I don't know why.
um Isn't stream input used to type on the keyboard? But since I input stream input to output, I don't think this should work. stdout:console,
stderr:console,
stdin:keyboard
TL;DR - From the source code:
var (
Stdin = NewFile(uintptr(syscall.Stdin), "/dev/stdin")
Stdout = NewFile(uintptr(syscall.Stdout), "/dev/stdout")
Stderr = NewFile(uintptr(syscall.Stderr), "/dev/stderr")
)
Stdin, Stdout, and Stderr are open Files pointing to the standard
input, standard output, and standard error file descriptors.
what this actually means is that Go treats Stdin, Stdout and Stderr the same, just as regular Files. and why shouldn't it? that's really just what they are. If you run echo sometext > /dev/stdin in your shell of choice, you'll see that this is valid and the text "sometext" will be printed to the terminal
Digging Deeper
Let's examine the log.New function declaration:
func New(out io.Writer, prefix string, flag int) *Logger
New creates a new Logger. The out variable sets the destination to
which log data will be written. The prefix appears at the beginning of
each generated log line, or after the log header if the Lmsgprefix
flag is provided. The flag argument defines the logging properties.
You can see that the first argument is the io.Writer interface - which is declared as :
type Writer interface {
Write(p []byte) (n int, err error)
}
This effectively means that log.New accepts every struct that has a Write(p []byte) (n int, err error) method. And because os.Stdin is of type os.File, it matches the io.Writer interface specification and the signature of log.New.
Related
I am using zerolog package in my go project and I need to log the content in both file as well as stdout. I took reference from here. The only change I have made is, instead of creating new variable named logger, I am directly affecting the global logger.
runLogFile, _ := os.OpenFile(
"myapp.log",
os.O_APPEND|os.O_CREATE|os.O_WRONLY,
0664,
)
fileLogger := zerolog.New(runLogFile).With().Logger()
multi := zerolog.MultiLevelWriter(os.Stdout, fileLogger)
log.Logger = zerolog.New(multi).With().Timestamp().Logger()
log.Info().Msg("Hello World!")
It produces output in the stdout as shown below, which is expected.
{"level":"info","time":"2022-09-15T08:10:28-04:00","message":"Hello World!"}
But, the file content becomes messy with additional message field which wraps the above output again.
{"message":"{\"level\":\"info\",\"time\":\"2022-09-15T08:10:28-04:00\",\"message\":\"Hello World!\"}"}
How can I enforce the zerolog to log the content without additional message field ?
MultiLevelWriter accept io.Writer. So you can input runLogFile to this.
In the end, the code look like this :
runLogFile, _ := os.OpenFile(
"myapp.log",
os.O_APPEND|os.O_CREATE|os.O_WRONLY,
0664,
)
multi := zerolog.MultiLevelWriter(os.Stdout, runLogFile)
log.Logger = zerolog.New(multi).With().Timestamp().Logger()
log.Info().Msg("Hello World!")
For printing, justified and fixed length, seems like what everyone asks about and there are many examples that I have found, like...
package main
import "fmt"
func main() {
values := []string{"Mustang", "10", "car"}
for i := range(values) {
fmt.Printf("%10v...\n", values[i])
}
for i := range(values) {
fmt.Printf("|%-10v|\n", values[i])
}
}
Situation
But what if I need to WRITE to a file with fixed length bytes?
For example: what if I have requirement that states, write this line to a file that must be 32 bytes, left justified and padded to the right with 0's
Question
So, how do you accomplish this when writing to a file?
There are analogous functions to fmt.PrintXX() functions, ones that start with an F, take the form of fmt.FprintXX(). These variants write the result to an io.Writer which may be an os.File as well.
So if you have the fmt.Printf() statements which you want to direct to a file, just change them to call fmt.Fprintf() instead, passing the file as the first argument:
var f *os.File = ... // Initialize / open file
fmt.Fprintf(f, "%10v...\n", values[i])
If you look into the implementation of fmt.Printf():
func Printf(format string, a ...interface{}) (n int, err error) {
return Fprintf(os.Stdout, format, a...)
}
It does exactly this: it calls fmt.Fprintf(), passing os.Stdout as the output to write to.
For how to open a file, see How to read/write from/to file using Go?
See related question: Format a Go string without printing?
I am trying to do functional testing of a cli app similar to this way.
As the command asks a few input on command prompt, I am putting them in a file and setting it as os.Stdin.
cmd := exec.Command(path.Join(dir, binaryName), "myArg")
tmpfile := setStdin("TheMasterPassword\nSecondAnswer\n12121212\n")
cmd.Stdin = tmpfile
output, err := cmd.CombinedOutput()
The setStdin just creates a tmpFile, write the string in file and returns the *os.File.
Now, I am expecting TheMasterPassword to be first input, and it's working. But for the second input always getting Critical Error: EOF.
The function I am using for asking and getting user input this :
func Ask(question string, minLen int) string {
reader := bufio.NewReader(os.Stdin)
for {
fmt.Printf("%s: ", question)
response, err := reader.ReadString('\n')
ExitIfError(err)
if len(response) >= minLen {
return strings.TrimSpace(response)
} else {
fmt.Printf("Provide at least %d character.\n", minLen)
}
}
}
Can you please help me to find out what's going wrong?
Thanks a lot!
Adding setStdin as requested
func setStdin(userInput string) *os.File {
tmpfile, err := ioutil.TempFile("", "test_stdin_")
util.ExitIfError(err)
_, err = tmpfile.Write([]byte(userInput))
util.ExitIfError(err)
_, err = tmpfile.Seek(0, 0)
util.ExitIfError(err)
return tmpfile
}
It pretty much looks like in your app your call Ask() whenever you want a single input line.
Inside Ask() you create a bufio.Reader to read from os.Stdin. Know that bufio.Reader–as its name suggests–uses buffered reading, meaning it may read more data from its source than what is returned by its methods (Reader.ReadString() in this case). Which means if you just use it to read one (or some) lines and you throw away the reader, you will throw away buffered, unread data.
So next time you call Ask() again, attempting to read from os.Stdin, you will not continue from where you left off...
To fix this issue, only create a single bufio.Reader from os.Stdin, store it in a global variable for example, and inside Ask(), always use this single reader. So buffered and unread data will not be lost between Ask() calls. Of course this solution will not be valid to call from multiple goroutines, but reading from a single os.Stdin isn't either.
For example:
var reader = bufio.NewReader(os.Stdin)
func Ask(question string, minLen int) string {
// use the global reader here...
}
Also note that using bufio.Scanner would be easier in your case. But again, bufio.Scanner may also read more data from its source than needed, so you have to use a shared bufio.Scanner here too. Also note that Reader.ReadString() returns you a string containing the delimeter (a line ending with \n in your case) which you probably have to trim, while Scanner.Text() (with the default line splitting function) will strip that first before returning the line. That's also a simplification you can take advantage of.
I am using logrus for logging and have a few custom format loggers. Each is initialized to write to a different file like:
fp, _ := os.OpenFile(path, os.O_APPEND|os.O_WRONLY|os.O_CREATE, 0755)
// error handling left out for brevity
log.Out = fp
Later in the application, I need to change the file the logger is writing to (for a log rotation logic). What I want to achieve is to properly close the current file before changing the logger's output file. But the closest thing to the file handle logrus provides me is a Writer() method that returns a io.PipeWriter pointer. So would calling Close() on the PipeWriter also close the underlying file?
If not, what are my options to do this, other than keeping the file pointer stored somewhere.
For the record, twelve-factor tells us that applications should not concern themselves with log rotation. If and how logs are handled best depends on how the application is deployed. Systemd has its own logging system, for instance. Writing to files when deployed in (Docker) containers is annoying. Rotating files are annoying during development.
Now, pipes don't have an "underlying file". There's a Reader end and a Writer end, and that's it. From the docs for PipeWriter:
Close closes the writer; subsequent reads from the read half of the pipe will return no bytes and EOF.
So what happens when you close the writer depends on how Logrus handles EOF on the Reader end. Since Logger.Out is an io.Writer, Logrus cannot possibly call Close on your file.
Your best bet would be to wrap *os.File, perhaps like so:
package main
import "os"
type RotatingFile struct {
*os.File
rotate chan struct{}
}
func NewRotatingFile(f *os.File) RotatingFile {
return RotatingFile{
File: f,
rotate: make(chan struct{}, 1),
}
}
func (r RotatingFile) Rotate() {
r.rotate <- struct{}{}
}
func (r RotatingFile) doRotate() error {
// file rotation logic here
return nil
}
func (r RotatingFile) Write(b []byte) (int, error) {
select {
case <-r.rotate:
if err := r.doRotate(); err != nil {
return 0, err
}
default:
}
return r.File.Write(b)
}
Implementing log file rotation in a robust way is surprisingly tricky. For instance, closing the old file before creating the new one is not a good idea. What if the log directory permissions changed? What if you run out of inodes? If you can't create a new log file you may want to keep writing to the current file. Are you okay with ripping lines apart, or do you only want to rotate after a newline? Do you want to rotate empty files? How do you reliably remove old logs if someone deletes the N-1th file? Will you notice the Nth file or stop looking at the N-2nd?
The best advice I can give you is to leave log rotation to the pros. I like svlogd (part of runit) as a standalone log rotation tool.
The closing of io.PipeWriter will not affect actual Writer behind it. The chain of close execution:
PipeWriter.Close() -> PipeWriter.CloseWithError(err error) ->
pipe.CloseWrite(err error)
and it doesn't influence underlying io.Writer.
To close actual writer you need just to close Logger.Out that is an exported field.
I'm facing with a weird golang issue. The following code will clarify:
package main
import (
"os/exec"
"io"
"fmt"
"os"
)
var (
pw io.WriteCloser
pr io.ReadCloser
)
func main() {
term := exec.Command("/bin/sh")
// Get stdin writer pipe
pw, _ = term.StdinPipe()
pr, _ = term.StdoutPipe()
term.Start()
run("cd ~")
pwd := run("pwd");
// Do something with pwd output
...
term.Wait()
}
func run(c string) string {
io.WriteString(pw, fmt.Sprintln(c))
buf := make([]byte, 32 * 1024)
pr.Read(buf)
return string(buf)
}
I'd like to run some commands in a shell env and read their output. There's no problem on write/run command but it seems that there're some limitations while reading:
you can't know if a command doesn't output anything or not;
there's no way to check if stdout is ready to be read or not.
The pr.Read(dest) method will block the code flow until something is read from stdout. As said, the goal is to read sequentially (without using a go routine and/or an infinite loop). This means that if we send a cd command the func end is never reached.
Setting the non-block flag through unix.SetNonblock on stdout file descriptor seems to solve the above issue but you can't know prior if it's ready or not and an error saying "resource temporary not available" is returned from .Read call.
As Cerise Limón mentioned go functions whould be the way to go here, since these sorts of interactive scripting exercises are traditionally done with expect.
You can wrap the the parrellel execution into a library to it might still look like sequencial code, so this might be helpful: https://github.com/ThomasRooney/gexpect
From the readme:
child, err := gexpect.Spawn("python")
if err != nil { panic(err) }
child.Expect(">>>")
child.SendLine("print 'Hello World'")
child.Interact()
child.Close()