Writing to file from cmd output - go

I am trying to write a small code in Go that will collect and save stats from IPFS.
So my Go code will execute IPFS command and save its output in .txt file and keep updating that .txt file.
I am having trouble doing that.
This is my code:
package main
import (
"fmt"
"io"
"log"
"os"
"os/exec"
"time"
)
func ipfsCommand() (ipfsOutput string) {
// output and error
out, err := exec.Command("ipfs","stats","bitswap","--human").Output()
// if there are errors, print/log them
if err != nil {
log.Printf("error!")
log.Fatal(err)
} else {
log.Printf("no error, printing output")
fmt.Printf("%s", out)
}
return
}
func writeToFile(message string) error {
f, err := os.Create("outputTest2_2.txt")
if err != nil {
return err
}
defer f.Close()
l, err := io.WriteString(f, message)
if err != nil {
fmt.Println(err)
f.Close()
return err
}
fmt.Println(l, "bytes written successfully")
return f.Sync()
}
func main() {
// get current time
currentTime := time.Now()
fmt.Println("YYYY.MM.DD : ", currentTime.Format("2006.01.02 15:04:05"))
writeToFile(currentTime)
// get output from ipfs command
msg := ipfsCommand()
// write the output to file
writeToFile(msg)
fmt.Println("file written!!!")
/* // write to file many times
for i:=0;i<3;i++{
// get output from ipfs command
msg := ipfsCommand()
// write the output to file
writeToFile(msg)
}*/
}
When the above code is run, this is the error:
# command-line-arguments
.\test2.go:49:13: cannot use currentTime (type time.Time) as type string in argument to writeToFile
Again, I want to get output from IPFS and save it to .txt file along with current time. I want to do this in loop because I want to save output from IPFS over a long period of time.

I tried to fix your script as is, but it just has too many issues. Here is a
rewrite, maybe you can use it as a new starting point:
package main
import (
"os"
"os/exec"
"time"
)
func main() {
f, err := os.Create("outputTest2_2.txt")
if err != nil {
panic(err)
}
defer f.Close()
currentTime, err := time.Now().MarshalText()
if err != nil {
panic(err)
}
f.Write(append(currentTime, '\n'))
msg, err := exec.Command("go", "env").Output()
if err != nil {
panic(err)
}
f.Write(msg)
}

Related

Can't find a public file from url in go

I am trying to get the content of a publicly available file using ioutil.ReadFile() but it doesn't find the file: panic: open http://www.pdf995.com/samples/pdf.pdf: No such file or directory
Here's my code:
// Reading and writing files are basic tasks needed for
// many Go programs. First we'll look at some examples of
// reading files.
package main
import (
"fmt"
"io/ioutil"
)
// Reading files requires checking most calls for errors.
// This helper will streamline our error checks below.
func check(e error) {
if e != nil {
panic(e)
}
}
func main() {
fileInUrl, err := ioutil.ReadFile("http://www.pdf995.com/samples/pdf.pdf")
if err != nil {
panic(err)
}
fmt.Printf("HERE --- fileInUrl: %+v", fileInUrl)
}
Here's a go playground example
ioutil.ReadFile() does not support http.
If you look at the source code(https://golang.org/src/io/ioutil/ioutil.go?s=1503:1549#L42), open the file using os.Open.
I think I can do this coding.
package main
import (
"io"
"net/http"
"os"
)
func main() {
fileUrl := "http://www.pdf995.com/samples/pdf.pdf"
if err := DownloadFile("example.pdf", fileUrl); err != nil {
panic(err)
}
}
func DownloadFile(filepath string, url string) error {
// Get the data
resp, err := http.Get(url)
if err != nil {
return err
}
defer resp.Body.Close()
// Create the file
out, err := os.Create(filepath)
if err != nil {
return err
}
defer out.Close()
// Write the body to file
_, err = io.Copy(out, resp.Body)
return err
}
but, go playgound not protocol(go error dial tcp: Protocol not available).
so, You have to do it PC.

How to write log into .log files in golang?

I'am new use go language, still learn and need help. How can I write my application logs in go language into ".log" files similiar like PHP phalcon framework ?
I have searching google and get go language tutorials, but there is no simple understanding example. I have try some example but the logs text doesn't write continously. Here's the example that I used to learn for logging log, and I think it's completly wrong.
package main
import (
"bufio"
"fmt"
"io/ioutil"
"os"
"time"
)
func check(e error) {
if e != nil {
panic(e)
}
}
func main() {
now := time.Now()
dt := now.Format("2006-01-02")
dt2 := now.Format("2006-01-02 15:04:05")
// To start, here's how to dump a string (or just
// bytes) into a file.
d1 := []byte("hello\ngo11\n" + dt2)
err := ioutil.WriteFile("/Users/my/Documents/work/src/logs/log-"+dt+".log", d1, 0644)
check(err)
// For more granular writes, open a file for writing.
f, err := os.Create("/Users/my/Documents/work/src/logs/log1.log")
check(err)
// It's idiomatic to defer a `Close` immediately
// after opening a file.
defer f.Close()
// You can `Write` byte slices as you'd expect.
d2 := []byte{115, 111, 109, 101, 10}
n2, err := f.Write(d2)
check(err)
fmt.Printf("wrote %d bytes\n", n2)
// A `WriteString` is also available.
n3, err := f.WriteString("writes\n" + dt)
fmt.Printf("wrote %d bytes\n", n3)
// Issue a `Sync` to flush writes to stable storage.
f.Sync()
// `bufio` provides buffered writers in addition
// to the buffered readers we saw earlier.
w := bufio.NewWriter(f)
n4, err := w.WriteString("buffered\n")
fmt.Printf("wrote %d bytes\n", n4)
// Use `Flush` to ensure all buffered operations have
// been applied to the underlying writer.
w.Flush()
}
You're trying so many different things it's hard to tell what you're aiming for specifically, but if you simply want to write logs to a file, here is an example:
package main
import (
"log"
"os"
)
func main() {
//create your file with desired read/write permissions
f, err := os.OpenFile("filename", os.O_WRONLY|os.O_CREATE|os.O_APPEND, 0644)
if err != nil {
log.Fatal(err)
}
//defer to close when you're done with it, not because you think it's idiomatic!
defer f.Close()
//set output of logs to f
log.SetOutput(f)
//test case
log.Println("check to make sure it works")
}
I recommend taking a look at Lumberjack. It handles rolling logs with several file conditions quite nicely. I'm using it for a streaming network sensor.
I have a small project that uses the log.Logger as following:
package main
import (
"errors"
"log"
"os"
)
const (
logNone = iota
logInfo
logWarning
logError
logVerbose
logDebug
)
type myFileLogger struct {
logger *log.Logger
logFile *os.File
logLevel int
}
func newFileLogger() *myFileLogger {
return &myFileLogger{
logger: nil,
logFile: nil,
logLevel: logNone,
}
}
func (myLogger *myFileLogger) startLog(level int, file string) error {
f, err := os.OpenFile(file, os.O_WRONLY|os.O_CREATE|os.O_APPEND, os.ModePerm)
if err != nil {
return err
}
myLogger.logger = log.New(f, "", 0)
myLogger.logLevel = level
myLogger.logFile = f
return nil
}
func (myLogger *myFileLogger) stopLog() error {
if myLogger.logFile != nil {
return myLogger.logFile.Close()
}
return nil
}
// You can add a log of auxiliary functions here to make the log more easier
func (myLogger *myFileLogger) log(level int, msg string) error {
if myLogger.logger == nil {
return errors.New("myFileLogger is not initialized correctly")
}
if level >= myLogger.logLevel {
myLogger.logger.Print(msg) // maybe you want to include the loglevel here, modify it as you want
}
return nil
}
func main() {
logger := newFileLogger()
if err := logger.startLog(logError, "myLogFile.log"); err != nil {
panic(err.Error())
}
defer func() {
logger.stopLog()
}()
logger.log(logInfo, "Info level log msg\n") // this will be ignored
logger.log(logError, "Error: error message\n") // this should included in the log file
}
import (
"os/exec"
)
func main() {
// check error here...
exec.Command("/bin/sh", "-c", "echo "+err.Error()+" >> log.log").Run()
}
I have my logger do this, here is an example.
Send message sends error to my IRC and sendSMS sends error to my phone.
in my config file I have:
var ErrorFile = "error.log"
type errorLog struct {
}
func (e errorLog) Write(p []byte) (n int, err error) {
fmt.Println("Error: " + string(p))
if config.Verbose == 0 {
file, _ := os.OpenFile(config.ErrorFile, os.O_RDWR|os.O_APPEND|os.O_CREATE, 0666)
file.WriteString(string(p))
sendMSG(string(p))
sendSMS(string(p))
// Close the file when the surrounding function exists
defer file.Close()
}
return n, err
}
// Logger is a helpper method to print out a more useful error message
var Logger = log.New(errorLog{}, "", log.Lmicroseconds|log.Lshortfile)

How to read a text file? [duplicate]

This question already has answers here:
How can I read a whole file into a string variable
(7 answers)
Closed 4 years ago.
I'm trying to read "file.txt" and put the contents into a variable using Golang. Here is what I've tried...
package main
import (
"fmt"
"os"
"log"
)
func main() {
file, err := os.Open("file.txt")
if err != nil {
log.Fatal(err)
}
fmt.Print(file)
}
The file gets read successfully and the return from os.Open returns a type of *os.File
It depends on what you are trying to do.
file, err := os.Open("file.txt")
fmt.print(file)
The reason it outputs &{0xc082016240}, is because you are printing the pointer value of a file-descriptor (*os.File), not file-content. To obtain file-content, you may READ from a file-descriptor.
To read all file content(in bytes) to memory, ioutil.ReadAll
package main
import (
"fmt"
"io/ioutil"
"os"
"log"
)
func main() {
file, err := os.Open("file.txt")
if err != nil {
log.Fatal(err)
}
defer func() {
if err = file.Close(); err != nil {
log.Fatal(err)
}
}()
b, err := ioutil.ReadAll(file)
fmt.Print(b)
}
But sometimes, if the file size is big, it might be more memory-efficient to just read in chunks: buffer-size, hence you could use the implementation of io.Reader.Read from *os.File
func main() {
file, err := os.Open("file.txt")
if err != nil {
log.Fatal(err)
}
defer func() {
if err = file.Close(); err != nil {
log.Fatal(err)
}
}()
buf := make([]byte, 32*1024) // define your buffer size here.
for {
n, err := file.Read(buf)
if n > 0 {
fmt.Print(buf[:n]) // your read buffer.
}
if err == io.EOF {
break
}
if err != nil {
log.Printf("read %d bytes: %v", n, err)
break
}
}
}
Otherwise, you could also use the standard util package: bufio, try Scanner. A Scanner reads your file in tokens: separator.
By default, scanner advances the token by newline (of course you can customise how scanner should tokenise your file, learn from here the bufio test).
package main
import (
"fmt"
"os"
"log"
"bufio"
)
func main() {
file, err := os.Open("file.txt")
if err != nil {
log.Fatal(err)
}
defer func() {
if err = file.Close(); err != nil {
log.Fatal(err)
}
}()
scanner := bufio.NewScanner(file)
for scanner.Scan() { // internally, it advances token based on sperator
fmt.Println(scanner.Text()) // token in unicode-char
fmt.Println(scanner.Bytes()) // token in bytes
}
}
Lastly, I would also like to reference you to this awesome site: go-lang file cheatsheet. It encompassed pretty much everything related to working with files in go-lang, hope you'll find it useful.

Interact with external application from within code

I need to be able to run an external application and interact with it as though I was manually running it from the command-line. All the examples I find only deal with running the program and capturing the output.
Below is a very simple example that I hope illustrates what I am trying to accomplish.
package main
import (
"fmt"
"log"
"os/exec"
)
func main() {
cmd := exec.Command("rm", "-i", "somefile.txt")
out, err := cmd.CombinedOutput()
if err != nil {
log.Fatal(err)
}
if string(out) == "Remove file 'somefile.txt'?" {
// send the response 'y' back to the rm process
}
// program completes normally...
}
I've tried to tweak various examples that I've found to accomplish this with zero success. It seems that even though 'rm' is waiting for a response, Go closes the process.
Any examples, articles, or advice you can provide would be greatly appreciated. Many thanks in advance.
You have two possibilities. First is to use ReadLine() but that works only if application output is full lines, and you can wait for \n. This is not the case with rm, so you have to develop a custom SplitFunction for Scanner. Both versions can be found below.
Please note that you can not use CombinedOutput, as it can not be Scanned. You have to use the pipes.
package main
import (
"bufio"
//"fmt"
"log"
"os/exec"
)
func main() {
cmd := exec.Command("rm", "-i", "somefile.txt")
// Stdout + stderr
out, err := cmd.StderrPipe() // rm writes the prompt to err
if err != nil {
log.Fatal(err)
}
r := bufio.NewReader(out)
// Stdin
in, err := cmd.StdinPipe()
if err != nil {
log.Fatal(err)
}
defer in.Close()
// Start the command!
err = cmd.Start()
if err != nil {
log.Fatal(err)
}
line, _, err := r.ReadLine()
for err != nil {
if string(line) == "Remove file 'somefile.txt'?" {
in.Write([]byte("y\n"))
}
line, _, err = r.ReadLine()
}
// program completes normally...s
}
This is a second version with the scanner, and it uses both \n and ? as line delimiters:
package main
import (
"bufio"
"bytes"
"fmt"
"log"
"os/exec"
)
// Ugly hack, this is bufio.ScanLines with ? added as an other delimiter :D
func new_scanner(data []byte, atEOF bool) (advance int, token []byte, err error) {
if atEOF && len(data) == 0 {
return 0, nil, nil
}
if i := bytes.IndexByte(data, '\n'); i >= 0 {
// We have a full newline-terminated line.
fmt.Printf("nn\n")
return i + 1, data[0:i], nil
}
if i := bytes.IndexByte(data, '?'); i >= 0 {
// We have a full ?-terminated line.
return i + 1, data[0:i], nil
}
// If we're at EOF, we have a final, non-terminated line. Return it.
if atEOF {
return len(data), data, nil
}
// Request more data.
return 0, nil, nil
}
func main() {
cmd := exec.Command("rm", "-i", "somefile.txt")
// Stdout + stderr
out, err := cmd.StderrPipe() // Again, rm writes prompts to stderr
if err != nil {
log.Fatal(err)
}
scanner := bufio.NewScanner(out)
scanner.Split(new_scanner)
// Stdin
in, err := cmd.StdinPipe()
if err != nil {
log.Fatal(err)
}
defer in.Close()
// Start the command!
err = cmd.Start()
if err != nil {
log.Fatal(err)
}
// Start scanning
for scanner.Scan() {
line := scanner.Text()
if line == "rm: remove regular empty file ‘somefile.txt’" {
in.Write([]byte("y\n"))
}
}
// Report scanner's errors
if err := scanner.Err(); err != nil {
log.Fatal(err)
}
// program completes normally...s
}

Why cant I open a text file when I use the wrong file extension while trying to open it?

I'm trying to read a .txt file using Go, but I keep getting told that the program cannot find the specified file, even when I use the complete path to it. However, the code I have has no problem reading .go files.
Help?
package main
import (
"bufio"
"fmt"
"log"
"os"
)
func main() {
// Open an input file, exit on error.
inputFile, err := os.Open("main.go");
if err != nil {
log.Fatal("Error opening input file:", err)
}
defer inputFile.Close()
scanner := bufio.NewScanner(inputFile)
for scanner.Scan() {
fmt.Println(scanner.Text())
}
if err := scanner.Err(); err != nil {
log.Fatal(scanner.Err())
}
}
You can see where you are adding this snippet of code at the beginning of your main function:
http://play.golang.org/p/DqnivLi1Z2
cwd, err := os.Getwd()
if err != nil {
log.Fatal("os.Getwd ", err)
}
log.Println("Current Directory", cwd)
files, err := ioutil.ReadDir(cwd)
if err != nil {
log.Fatal("ioutil.ReadDir", err)
}
for idx, finfo := range files {
log.Println(idx, " ", finfo.Name)
}

Resources