I am trying to do in golang the bash equivalent:
cat image.jpg | exiftool -author=some_auth - > updated_image.jpg
The exiftool '-' option makes it read from stdin, but say I have the image stored in a variable like
var img []bytes //in golang
I want stdin to contain the bytes from img and the system call to exiftool to read these from stdin, save the result (stdout) in another []byte - I am new to golang, how what do i approach this ?
I don't want to save to temp files on disk.
Thanks
Something like
out, err := os.Create("updated_image.jpg")
if err != nil {
log.Fatal(err)
}
cmd := exec.Command("exiftool", "-author=some_auth", "-")
cmd.Stdout = out
cmd.Stdin = bytes.NewReader(img)
err = cmd.Run()
if err != nil {
log.Fatal(err)
}
out.Close()
should work.
Note that I haven't tested the code.
Related
Code in function to run a fzf against an input, while debugging i discovered my code doesn't return errors, this code runs successfully:
reader := strings.NewReader(listOutput.String())
r, w, _ := os.Pipe()
os.Stdout = w
cmd := exec.Command("fzf", "--multi")
cmd.Stdin = reader
cmd.Stderr = os.Stderr
if err := cmd.Run(); err != nil {
fmt.Println("Couldn't call fzf: %v", err)
}
w.Close()
So i changed the command to something that doesn't exist, but the code still doesn't return "couldn't call command: command not found", just exits.
reader := strings.NewReader(listOutput.String())
r, w, _ := os.Pipe()
os.Stdout = w
cmd := exec.Command("idontexist")
cmd.Stdin = reader
cmd.Stderr = os.Stderr
if err := cmd.Run(); err != nil {
fmt.Println("Couldn't call command: %v", err)
}
w.Close()
I don't have an idea what could be wrong.
cmd.Run() does return an error, and your if block gets properly executed, but since you change the standard output os.Stdout = w you just don't see the result on your console / terminal.
The fmt package writes to the standard output.
If you use the log package, you will see the error as the log package writes to the standard error (which you didn't change):
log.Printf("Couldn't call command: %v", err)
This will output something like (note the default log format includes the timestamp too):
2022/12/07 13:46:19 Couldn't call command: exec: "idontexist": executable file not found in $PATH
Or don't change the standard output.
Also do note that log.Println() and fmt.Println() do not require a format string. Do use log.Printf() and fmt.Printf() when you specify a format string and arguments.
I am trying to write a file from a bash command into a file in Go.
Note there are several reasons for using Go over bash here: I have some more logic such as parsing configuration files, I would like to run that code for multiple DBs in parallele and finally performing some more complex data manipulation after.
dumpStr := fmt.Sprintf("pg_dump -U %s -h %s %s | gzip", DbUserName, DbHost, DbName)
cmd := exec.Command("bash", "-c", dumpStr)
cmd.Env = append(cmd.Env, "PGPASSWORD="+DbPassword)
outfile, err := os.Create(DbName + ".gz")
if err != nil {
panic(err)
}
outfile = cmd.Stdout
defer outfile.Close()
err = cmd.Start()
if err != nil {
panic(err)
}
cmd.Wait()
However, I am getting an emtpy result.
I am getting data if I am executing dumpStr from the CLI but not from that code...
What am I missing?
As Flimzy said, you're not capturing the output of pg_dump. You can do that with Go, or you can use pg_dump-s --file. It can also compress with --compress so no need to pipe to gzip. Then there's no need for bash and you can avoid shell quoting issues.
cmd := exec.Command(
"pg_dump",
"--compress=9",
"--file="+DbName + ".gz",
"-U"+DbUserName,
"-h"+DbHost,
DbName,
)
log.Print("Running pg_dump...")
if err := cmd.Run(); err != nil {
log.Fatal(err)
}
Much simpler and more secure.
For illustration here's how you'd do it all in Go.
Use Cmd.StdoutPipe to get an open IO reader to pg_dump's stdout. Then use io.Copy to copy from stdout to your open file.
#Peter points out that since Cmd.Stdout is an io.Reader it's simpler to assign the open file to cmd.Stdout and let cmd write to it directly.
// Same as above, but no --file.
cmd := exec.Command(
"pg_dump",
"--compress=9",
"-U"+DbUserName,
"-h"+DbHost,
DbName,
)
// Open the output file
outfile, err := os.Create(DbName + ".gz")
if err != nil {
log.Fatal(err)
}
defer outfile.Close()
// Send stdout to the outfile. cmd.Stdout will take any io.Writer.
cmd.Stdout = outfile
// Start the command
if err = cmd.Start(); err != nil {
log.Fatal(err)
}
log.Print("Waiting for command to finish...")
// Wait for the command to finish.
if err = cmd.Wait(); err != nil {
log.Fatal(err)
}
In addition, you're only checking if the command started, not if it successfully ran.
From the docs for Cmd.Start.
Start starts the specified command but does not wait for it to complete.
The Wait method will return the exit code and release associated resources once the command exits.
You're checking cmd.Start for an error, but not cmd.Wait. Checking the error from cmd.Start only means the command started. If there is an error while the program is running you won't know what it is.
You need to actually use the output of your command. You're not doing that. To do so, use the StdoutPipe method, then you can copy the stdout from your program, into your file.
There's a library that exports a file but I'd like to capture the contents of the file. I'd like to pass a writer to the library and be able to read what the writer wrote to the file. Eventually i want to augment the library to skip writing this file.
Is this possible with io.Copy or io.Pipe?
The library code creates a *File and uses this handle as an io.Writer.
I tried using io.Copy but only 0 bytes were read.
func TestFileCopy(t *testing.T) {
codeFile, err := os.Create("test.txt")
if err != nil {
t.Error(err)
}
defer codeFile.Close()
codeFile.WriteString("Hello World")
n, err := io.Copy(os.Stdout, codeFile)
if err != nil {
t.Error(err)
}
log.Println(n, "bytes")
}
If you want to capture the bytes as they are written, use an io.MultiWriter with a bytes.Buffer as the second writer.
var buf bytes.Buffer
w := io.MultiWriter(codeFile, &buf)
or to see the file on stdout as it's written:
w := io.MultiWriter(codeFile, os.Stdout)
Otherwise, if you want to re-read the same file, you need to seek back to the start after writing:
codeFile.Seek(0, 0)
I have the following code which executes an external command and output to the console two fields waiting for the user input.
One for the username and other for the password, and then I have added them manually.
Could anyone give me a hint about how to write to stdin in order to enter these inputs from inside the program?
The tricky part for me is that there are two different fields waiting for input, and I'm having trouble to figure out how to fill one after the other.
login := exec.Command(cmd, "login")
login.Stdout = os.Stdout
login.Stdin = os.Stdin
login.Stderr = os.Stderr
err := login.Run()
if err != nil {
fmt.Fprintln(os.Stderr, err)
}
SOLUTION:
login := exec.Command(cmd, "login")
var b bytes.Buffer
b.Write([]byte(username + "\n" + pwd + "\n"))
login.Stdout = os.Stdout
login.Stdin = &b
login.Stderr = os.Stderr
I imagine you could use a bytes.Buffer for that.
Something like that:
login := exec.Command(cmd, "login")
buffer := bytes.Buffer{}
buffer.Write([]byte("username\npassword\n"))
login.Stdin = &buffer
login.Stdout = os.Stdout
login.Stderr = os.Stderr
err := login.Run()
if err != nil {
fmt.Fprintln(os.Stderr, err)
}
The trick is that the stdin is merely a char buffer, and when reading the credentials, it will simply read chars until encountering a \n character (or maybe \n\r). So you can write them in a buffer in advance, and feed the buffer directly to the command.
I am trying to take input from the keyboard and then store it in a text file but I am a bit confused on how to actually do it.
My current code is as follow at the moment:
// reads the file txt.txt
bs, err := ioutil.ReadFile("text.txt")
if err != nil {
panic(err)
}
// Prints out content
textInFile := string(bs)
fmt.Println(textInFile)
// Standard input from keyboard
var userInput string
fmt.Scanln(&userInput)
//Now I want to write input back to file text.txt
//func WriteFile(filename string, data []byte, perm os.FileMode) error
inputData := make([]byte, len(userInput))
err := ioutil.WriteFile("text.txt", inputData, )
There are so many functions in the "os" and "io" packages. I am very confused about which one I actually should use for this purpose.
I am also confused about what the third argument in the WriteFile function should be. In the documentation is says of type " perm os.FileMode" but since I am new to programming and Go I am a bit clueless.
Does anybody have any tips on how to proced?
Thanks in advance,
Marie
// reads the file txt.txt
bs, err := ioutil.ReadFile("text.txt")
if err != nil { //may want logic to create the file if it doesn't exist
panic(err)
}
var userInput []string
var err error = nil
var n int
//read in multiple lines from user input
//until user enters the EOF char
for ln := ""; err == nil; n, err = fmt.Scanln(ln) {
if n > 0 { //we actually read something into the string
userInput = append(userInput, ln)
} //if we didn't read anything, err is probably set
}
//open the file to append to it
//0666 corresponds to unix perms rw-rw-rw-,
//which means anyone can read or write it
out, err := os.OpenFile("text.txt", os.O_APPEND, 0666)
defer out.Close() //we'll close this file as we leave scope, no matter what
if err != nil { //assuming the file didn't somehow break
//write each of the user input lines followed by a newline
for _, outLn := range userInput {
io.WriteString(out, outLn+"\n")
}
}
I've made sure this compiles and runs on play.golang.org, but I'm not at my dev machine, so I can't verify that it's interacting with Stdin and the file entirely correctly. This should get you started though.
For example,
package main
import (
"fmt"
"io/ioutil"
"os"
)
func main() {
fname := "text.txt"
// print text file
textin, err := ioutil.ReadFile(fname)
if err == nil {
fmt.Println(string(textin))
}
// append text to file
f, err := os.OpenFile(fname, os.O_CREATE|os.O_APPEND|os.O_WRONLY, 0666)
if err != nil {
panic(err)
}
var textout string
fmt.Scanln(&textout)
_, err = f.Write([]byte(textout))
if err != nil {
panic(err)
}
f.Close()
// print text file
textin, err = ioutil.ReadFile(fname)
if err != nil {
panic(err)
}
fmt.Println(string(textin))
}
If you simply want to append the user's input to a text file, you could just read the
input as you've already done and use ioutil.WriteFile, as you've tried to do.
So you already got the right idea.
To make your way go, the simplified solution would be this:
// Read old text
current, err := ioutil.ReadFile("text.txt")
// Standard input from keyboard
var userInput string
fmt.Scanln(&userInput)
// Append the new input to the old using builtin `append`
newContent := append(current, []byte(userInput)...)
// Now write the input back to file text.txt
err = ioutil.WriteFile("text.txt", newContent, 0666)
The last parameter of WriteFile is a flag which specifies the various options for
files. The higher bits are options like file type (os.ModeDir, for example) and the lower
bits represent the permissions in form of UNIX permissions (0666, in octal format, stands for user rw, group rw, others rw). See the documentation for more details.
Now that your code works, we can improve it. For example by keeping the file open
instead of opening it twice:
// Open the file for read and write (O_RDRW), append to it if it has
// content, create it if it does not exit, use 0666 for permissions
// on creation.
file, err := os.OpenFile("text.txt", os.O_RDWR|os.O_APPEND|os.O_CREATE, 0666)
// Close the file when the surrounding function exists
defer file.Close()
// Read old content
current, err := ioutil.ReadAll(file)
// Do something with that old content, for example, print it
fmt.Println(string(current))
// Standard input from keyboard
var userInput string
fmt.Scanln(&userInput)
// Now write the input back to file text.txt
_, err = file.WriteString(userInput)
The magic here is, that you use the flag os.O_APPEND while opening the file,
which makes file.WriteString() append. Note that you need to close the file after
opening it, which we do after the function exists using the defer keyword.