Interact with external application from within code - go

I need to be able to run an external application and interact with it as though I was manually running it from the command-line. All the examples I find only deal with running the program and capturing the output.
Below is a very simple example that I hope illustrates what I am trying to accomplish.
package main
import (
"fmt"
"log"
"os/exec"
)
func main() {
cmd := exec.Command("rm", "-i", "somefile.txt")
out, err := cmd.CombinedOutput()
if err != nil {
log.Fatal(err)
}
if string(out) == "Remove file 'somefile.txt'?" {
// send the response 'y' back to the rm process
}
// program completes normally...
}
I've tried to tweak various examples that I've found to accomplish this with zero success. It seems that even though 'rm' is waiting for a response, Go closes the process.
Any examples, articles, or advice you can provide would be greatly appreciated. Many thanks in advance.

You have two possibilities. First is to use ReadLine() but that works only if application output is full lines, and you can wait for \n. This is not the case with rm, so you have to develop a custom SplitFunction for Scanner. Both versions can be found below.
Please note that you can not use CombinedOutput, as it can not be Scanned. You have to use the pipes.
package main
import (
"bufio"
//"fmt"
"log"
"os/exec"
)
func main() {
cmd := exec.Command("rm", "-i", "somefile.txt")
// Stdout + stderr
out, err := cmd.StderrPipe() // rm writes the prompt to err
if err != nil {
log.Fatal(err)
}
r := bufio.NewReader(out)
// Stdin
in, err := cmd.StdinPipe()
if err != nil {
log.Fatal(err)
}
defer in.Close()
// Start the command!
err = cmd.Start()
if err != nil {
log.Fatal(err)
}
line, _, err := r.ReadLine()
for err != nil {
if string(line) == "Remove file 'somefile.txt'?" {
in.Write([]byte("y\n"))
}
line, _, err = r.ReadLine()
}
// program completes normally...s
}
This is a second version with the scanner, and it uses both \n and ? as line delimiters:
package main
import (
"bufio"
"bytes"
"fmt"
"log"
"os/exec"
)
// Ugly hack, this is bufio.ScanLines with ? added as an other delimiter :D
func new_scanner(data []byte, atEOF bool) (advance int, token []byte, err error) {
if atEOF && len(data) == 0 {
return 0, nil, nil
}
if i := bytes.IndexByte(data, '\n'); i >= 0 {
// We have a full newline-terminated line.
fmt.Printf("nn\n")
return i + 1, data[0:i], nil
}
if i := bytes.IndexByte(data, '?'); i >= 0 {
// We have a full ?-terminated line.
return i + 1, data[0:i], nil
}
// If we're at EOF, we have a final, non-terminated line. Return it.
if atEOF {
return len(data), data, nil
}
// Request more data.
return 0, nil, nil
}
func main() {
cmd := exec.Command("rm", "-i", "somefile.txt")
// Stdout + stderr
out, err := cmd.StderrPipe() // Again, rm writes prompts to stderr
if err != nil {
log.Fatal(err)
}
scanner := bufio.NewScanner(out)
scanner.Split(new_scanner)
// Stdin
in, err := cmd.StdinPipe()
if err != nil {
log.Fatal(err)
}
defer in.Close()
// Start the command!
err = cmd.Start()
if err != nil {
log.Fatal(err)
}
// Start scanning
for scanner.Scan() {
line := scanner.Text()
if line == "rm: remove regular empty file ‘somefile.txt’" {
in.Write([]byte("y\n"))
}
}
// Report scanner's errors
if err := scanner.Err(); err != nil {
log.Fatal(err)
}
// program completes normally...s
}

Related

Writing to file from cmd output

I am trying to write a small code in Go that will collect and save stats from IPFS.
So my Go code will execute IPFS command and save its output in .txt file and keep updating that .txt file.
I am having trouble doing that.
This is my code:
package main
import (
"fmt"
"io"
"log"
"os"
"os/exec"
"time"
)
func ipfsCommand() (ipfsOutput string) {
// output and error
out, err := exec.Command("ipfs","stats","bitswap","--human").Output()
// if there are errors, print/log them
if err != nil {
log.Printf("error!")
log.Fatal(err)
} else {
log.Printf("no error, printing output")
fmt.Printf("%s", out)
}
return
}
func writeToFile(message string) error {
f, err := os.Create("outputTest2_2.txt")
if err != nil {
return err
}
defer f.Close()
l, err := io.WriteString(f, message)
if err != nil {
fmt.Println(err)
f.Close()
return err
}
fmt.Println(l, "bytes written successfully")
return f.Sync()
}
func main() {
// get current time
currentTime := time.Now()
fmt.Println("YYYY.MM.DD : ", currentTime.Format("2006.01.02 15:04:05"))
writeToFile(currentTime)
// get output from ipfs command
msg := ipfsCommand()
// write the output to file
writeToFile(msg)
fmt.Println("file written!!!")
/* // write to file many times
for i:=0;i<3;i++{
// get output from ipfs command
msg := ipfsCommand()
// write the output to file
writeToFile(msg)
}*/
}
When the above code is run, this is the error:
# command-line-arguments
.\test2.go:49:13: cannot use currentTime (type time.Time) as type string in argument to writeToFile
Again, I want to get output from IPFS and save it to .txt file along with current time. I want to do this in loop because I want to save output from IPFS over a long period of time.
I tried to fix your script as is, but it just has too many issues. Here is a
rewrite, maybe you can use it as a new starting point:
package main
import (
"os"
"os/exec"
"time"
)
func main() {
f, err := os.Create("outputTest2_2.txt")
if err != nil {
panic(err)
}
defer f.Close()
currentTime, err := time.Now().MarshalText()
if err != nil {
panic(err)
}
f.Write(append(currentTime, '\n'))
msg, err := exec.Command("go", "env").Output()
if err != nil {
panic(err)
}
f.Write(msg)
}

The process cannot access the file because it is being used by another process in Golang

The process cannot access the file ... because it is being used by another process
I can't Remover Zip file with this code ..
it's possible? extract and delete the file in one code.
Code
package main
import (
"archive/zip"
"fmt"
"io"
"log"
"net/http"
"os"
"path/filepath"
"strings"
)
func main() {
url := "https://230c07c8-77b2-4c0d-9b82-8c6501a5bc45.filesusr.com/archives/b7572a_9ec985e0031042ef912cb40cafbe6376.zip?dn=7.zip"
out, _ := os.Create("E:\\experi\\1234567890.zip")
defer out.Close()
resp, _ := http.Get(url)
defer resp.Body.Close()
_, _ = io.Copy(out, resp.Body)
files, err := Unzip("E:\\experi\\1234567890.zip", "E:\\experi\\1234567890")
if err != nil {
log.Fatal(err)
}
fmt.Println("Unzipped the following files:\n" + strings.Join(files, "\n"))
}
func Unzip(src string, destination string) ([]string, error) {
var filenames []string
r, err := zip.OpenReader(src)
if err != nil {
return filenames, err
}
defer r.Close()
for _, f := range r.File {
fpath := filepath.Join(destination, f.Name)
if !strings.HasPrefix(fpath, filepath.Clean(destination)+string(os.PathSeparator)){
return filenames, fmt.Errorf("%s is an illegal filepath", fpath)
}
filenames = append(filenames, fpath)
if f.FileInfo().IsDir() {
os.MkdirAll(fpath, os.ModePerm)
continue
}
if err = os.MkdirAll(filepath.Dir(fpath), os.ModePerm); err != nil {
return filenames, err
}
outFile, err := os.OpenFile(fpath,
os.O_WRONLY|os.O_CREATE|os.O_TRUNC | os.O_RDWR,
f.Mode())
if err != nil {
return filenames, err
}
rc, err := f.Open()
if err != nil {
return filenames, err
}
_, err = io.Copy(outFile, rc)
outFile.Close()
rc.Close()
if err != nil {
return filenames, err
}
}
removeFile()
return filenames, nil
}
func removeFile() {
error := os.Remove("E:\\experi\\1234567890.zip")
if error != nil {
log.Fatal(error)
}
}
Output
output text
2020/10/28 13:09:04 remove E:\experi\1234567890.zip: The process cannot access the file because it is being used by another process.
Process finished with exit code 1
Any other way to do this same thing ?
Did I go wrong anywhere?
Help Would be Much Appreciated. Thanks in Advance. :)
out, _ := os.Create("E:\\experi\\1234567890.zip") creates or truncates the file and returns you a *File (so the file is open).
defer out.Close() closes the file "the moment the surrounding function returns" (spec).
So at the time you call Unzip you have the file open. To fix this call out.Close() before the call to Unzip (and please don't assume that calls complete without error).
If you close using the defer, it is closed after performing up to the last line of the function. You must explicitly close the file before remove it.

How to call an external program and process its output?

I am trying to call an external command (e.g., seq 10) and take its output, process the output then print out the processed results. But the following code does not work. Could you please let me know how to make it work?
// vim: set noexpandtab tabstop=2:
package main
import (
"bufio"
"io"
"os"
"log"
"os/exec"
)
func main() {
cmd := exec.Command("seq", "10")
stdin := bufio.NewReader(cmd.Stdout)
err := cmd.Run()
if err != nil {
log.Fatalf("cmd.Run() failed with %s\n", err)
}
for {
line, err := stdin.ReadBytes('\n')
if err == io.EOF {
if len(line) == 0 { break }
} else {
if err != nil { log.Fatal(err) }
line = line[:(len(line)-1)]
}
os.Stdout.Write(line)
os.Stdout.Write([]byte{'\n'})
}
}
$ $ go run main.go
# command-line-arguments
./main.go:15:30: cannot use cmd.Stdout (type io.Writer) as type io.Reader in argument to bufio.NewReader:
io.Writer does not implement io.Reader (missing Read method)
EDIT: I also tried this. But it also has error. Could anybody show me a working example.
// vim: set noexpandtab tabstop=2:
package main
import (
"bufio"
"io"
"os"
"log"
"os/exec"
)
func main() {
cmd := exec.Command("seq", "10")
stdout, err := cmd.StdoutPipe()
if err != nil { log.Fatal(err) }
stdin := bufio.NewReader(stdout)
err = cmd.Run()
if err != nil {
log.Fatalf("cmd.Run() failed with %s\n", err)
}
for {
line, err := stdin.ReadBytes('\n')
if err == io.EOF {
if len(line) == 0 { break }
} else {
if err != nil { log.Fatal(err) }
line = line[:(len(line)-1)]
}
os.Stdout.Write(line)
os.Stdout.Write([]byte{'\n'})
}
}
Another method (and a cleaner one) is to use bufio.Scanner which handles \n (or any other delimiter) automatically. Another advantage is that this method doesn't have race issues (been there, done that):
package main
import (
"bufio"
"fmt"
"log"
"os/exec"
)
func main() {
cmd := exec.Command("seq", "10")
stdout, err := cmd.StdoutPipe()
if err != nil {
log.Fatal(err)
}
err = cmd.Start()
if err != nil {
log.Fatalf("cmd.Start() failed with %s\n", err)
}
stdin := bufio.NewScanner(stdout)
for stdin.Scan() {
fmt.Println(stdin.Text())
}
cmd.Wait()
}
stdin.Scan() returns false on EOF, which is given once the process exits. cmd.Wait() will close StdoutPipe, and you can read err.(exec.ExitError).ExitCode() to get the exit code (if exited non-zero).
You need to pipe the standard out to the reader using .StdoutPipe(), you also need to use exec.Command(..).Start() to read incrementally (.Run() waits for the process to exit).
Here is the working code:
// vim: set noexpandtab tabstop=2:
package main
import (
"bufio"
"fmt"
"io"
"log"
"os"
"os/exec"
)
func main() {
cmd := exec.Command("seq", "10")
cmdStdOut, err := cmd.StdoutPipe()
cmdStdErr, err := cmd.StderrPipe()
defer cmdStdOut.Close()
if err != nil {
log.Fatalf("command failed with %s\n", err)
}
stdoutReader := bufio.NewReader(cmdStdOut)
stderrReader := bufio.NewReader(cmdStdErr)
err = cmd.Start()
if err != nil {
log.Fatalf("cmd.Run() failed with %s\n", err)
}
// Read stdout
for {
line, err := stdoutReader.ReadBytes('\n')
if err == io.EOF {
if len(line) == 0 {
break
}
} else {
if err != nil {
log.Fatal(err)
}
line = line[:(len(line) - 1)]
}
os.Stdout.Write(line)
os.Stdout.Write([]byte{'\n'})
}
// Read stderr
for {
line, err := stderrReader.ReadBytes('\n')
if err == io.EOF {
if len(line) == 0 {
break
}
} else {
if err != nil {
log.Fatal(err)
}
line = line[:(len(line) - 1)]
}
os.Stderr.Write(line)
os.Stderr.Write([]byte{'\n'})
}
cmd.Wait()
fmt.Println(cmd.ProcessState.ExitCode())
}

Is it possible to extract a tar.xz package in golang?

Is it possible to extract a tar.xz package in golang? My understanding is it's possible to use the library for tar and sending it to an xz go library.
I recently created an XZ decompression package so it is now
possible to extract a tar.xz using only Go code.
The following code extracts the file myfile.tar.xz to the current
directory:
package main
import (
"archive/tar"
"fmt"
"io"
"log"
"os"
"github.com/xi2/xz"
)
func main() {
// Open a file
f, err := os.Open("myfile.tar.xz")
if err != nil {
log.Fatal(err)
}
// Create an xz Reader
r, err := xz.NewReader(f, 0)
if err != nil {
log.Fatal(err)
}
// Create a tar Reader
tr := tar.NewReader(r)
// Iterate through the files in the archive.
for {
hdr, err := tr.Next()
if err == io.EOF {
// end of tar archive
break
}
if err != nil {
log.Fatal(err)
}
switch hdr.Typeflag {
case tar.TypeDir:
// create a directory
fmt.Println("creating: " + hdr.Name)
err = os.MkdirAll(hdr.Name, 0777)
if err != nil {
log.Fatal(err)
}
case tar.TypeReg, tar.TypeRegA:
// write a file
fmt.Println("extracting: " + hdr.Name)
w, err := os.Create(hdr.Name)
if err != nil {
log.Fatal(err)
}
_, err = io.Copy(w, tr)
if err != nil {
log.Fatal(err)
}
w.Close()
}
}
f.Close()
}
http://golang.org/pkg/archive/tar/#example_
also you can do
import "os/exec"
cmd := exec.Command("tar", "-x", "/your/archive.tar.xz")
err := cmd.Run()
There is no Lempel-Ziv-Markow encoder or decoder in the Go standard library. If you are allowed to assume that the platform your code runs on provides the xz utility, you could use stub functions like these:
import "os/exec"
// decompress xz compressed data stream r.
func UnxzReader(r io.Reader) (io.ReadCloser, error) {
unxz := exec.Command("xz", "-d")
unxz.Stdin = r
out, err := unxz.StdoutPipe()
if err != nil {
return nil, err
}
err = unxz.Start()
if err != nil {
return nil, err
}
// we are not interested in the exit status, but we should really collect
// that zombie process
go unxz.Wait()
return out, nil
}

Read text file into string array (and write)

The ability to read (and write) a text file into and out of a string array is I believe a fairly common requirement. It is also quite useful when starting with a language removing the need initially to access a database. Does one exist in Golang?
e.g.
func ReadLines(sFileName string, iMinLines int) ([]string, bool) {
and
func WriteLines(saBuff[]string, sFilename string) (bool) {
I would prefer to use an existing one rather than duplicate.
As of Go1.1 release, there is a bufio.Scanner API that can easily read lines from a file. Consider the following example from above, rewritten with Scanner:
package main
import (
"bufio"
"fmt"
"log"
"os"
)
// readLines reads a whole file into memory
// and returns a slice of its lines.
func readLines(path string) ([]string, error) {
file, err := os.Open(path)
if err != nil {
return nil, err
}
defer file.Close()
var lines []string
scanner := bufio.NewScanner(file)
for scanner.Scan() {
lines = append(lines, scanner.Text())
}
return lines, scanner.Err()
}
// writeLines writes the lines to the given file.
func writeLines(lines []string, path string) error {
file, err := os.Create(path)
if err != nil {
return err
}
defer file.Close()
w := bufio.NewWriter(file)
for _, line := range lines {
fmt.Fprintln(w, line)
}
return w.Flush()
}
func main() {
lines, err := readLines("foo.in.txt")
if err != nil {
log.Fatalf("readLines: %s", err)
}
for i, line := range lines {
fmt.Println(i, line)
}
if err := writeLines(lines, "foo.out.txt"); err != nil {
log.Fatalf("writeLines: %s", err)
}
}
Note: ioutil is deprecated as of Go 1.16.
If the file isn't too large, this can be done with the ioutil.ReadFile and strings.Split functions like so:
content, err := ioutil.ReadFile(filename)
if err != nil {
//Do something
}
lines := strings.Split(string(content), "\n")
You can read the documentation on ioutil and strings packages.
Cannot update first answer.
Anyway, after Go1 release, there are some breaking changes, so I updated as shown below:
package main
import (
"os"
"bufio"
"bytes"
"io"
"fmt"
"strings"
)
// Read a whole file into the memory and store it as array of lines
func readLines(path string) (lines []string, err error) {
var (
file *os.File
part []byte
prefix bool
)
if file, err = os.Open(path); err != nil {
return
}
defer file.Close()
reader := bufio.NewReader(file)
buffer := bytes.NewBuffer(make([]byte, 0))
for {
if part, prefix, err = reader.ReadLine(); err != nil {
break
}
buffer.Write(part)
if !prefix {
lines = append(lines, buffer.String())
buffer.Reset()
}
}
if err == io.EOF {
err = nil
}
return
}
func writeLines(lines []string, path string) (err error) {
var (
file *os.File
)
if file, err = os.Create(path); err != nil {
return
}
defer file.Close()
//writer := bufio.NewWriter(file)
for _,item := range lines {
//fmt.Println(item)
_, err := file.WriteString(strings.TrimSpace(item) + "\n");
//file.Write([]byte(item));
if err != nil {
//fmt.Println("debug")
fmt.Println(err)
break
}
}
/*content := strings.Join(lines, "\n")
_, err = writer.WriteString(content)*/
return
}
func main() {
lines, err := readLines("foo.txt")
if err != nil {
fmt.Println("Error: %s\n", err)
return
}
for _, line := range lines {
fmt.Println(line)
}
//array := []string{"7.0", "8.5", "9.1"}
err = writeLines(lines, "foo2.txt")
fmt.Println(err)
}
You can use os.File (which implements the io.Reader interface) with the bufio package for that. However, those packages are build with fixed memory usage in mind (no matter how large the file is) and are quite fast.
Unfortunately this makes reading the whole file into the memory a bit more complicated. You can use a bytes.Buffer to join the parts of the line if they exceed the line limit. Anyway, I recommend you to try to use the line reader directly in your project (especially if do not know how large the text file is!). But if the file is small, the following example might be sufficient for you:
package main
import (
"os"
"bufio"
"bytes"
"fmt"
)
// Read a whole file into the memory and store it as array of lines
func readLines(path string) (lines []string, err os.Error) {
var (
file *os.File
part []byte
prefix bool
)
if file, err = os.Open(path); err != nil {
return
}
reader := bufio.NewReader(file)
buffer := bytes.NewBuffer(make([]byte, 1024))
for {
if part, prefix, err = reader.ReadLine(); err != nil {
break
}
buffer.Write(part)
if !prefix {
lines = append(lines, buffer.String())
buffer.Reset()
}
}
if err == os.EOF {
err = nil
}
return
}
func main() {
lines, err := readLines("foo.txt")
if err != nil {
fmt.Println("Error: %s\n", err)
return
}
for _, line := range lines {
fmt.Println(line)
}
}
Another alternative might be to use io.ioutil.ReadAll to read in the complete file at once and do the slicing by line afterwards. I don't give you an explicit example of how to write the lines back to the file, but that's basically an os.Create() followed by a loop similar to that one in the example (see main()).
func readToDisplayUsingFile1(f *os.File){
defer f.Close()
reader := bufio.NewReader(f)
contents, _ := ioutil.ReadAll(reader)
lines := strings.Split(string(contents), '\n')
}
or
func readToDisplayUsingFile1(f *os.File){
defer f.Close()
slice := make([]string,0)
reader := bufio.NewReader(f)
for{
str, err := reader.ReadString('\n')
if err == io.EOF{
break
}
slice = append(slice, str)
}

Resources