I am trying to run the find command using exec.Command:
cmd := exec.Command("find", "/usr/bin", "-maxdepth",
"2", "-iname", "'*go*'", "|", "head", "-10")
out, err := cmd.CombinedOutput()
fmt.Println(err)
fmt.Println(string(out))
Unfortunately this fails with the following output:
exit status 1
find: paths must precede expression: |
Usage: find [-H] [-L] [-P] [-Olevel] [-D help|tree|search|stat|rates|opt|exec] [path...] [expression]
What am I missing here?
EDIT:
Even if I try it without piping this still fails:
cmd := exec.Command("find", "/usr/bin", "-maxdepth", "2", "-iname", "'*go*'")
out, err := cmd.CombinedOutput()
fmt.Println(err)
fmt.Println(string(out))
Output:
<nil>
You're using | which pipes the output of the previous command into the next. Instead you probably need two commands in Go. Use string(out) as input to the second one rather than trying to pipe it and compose the two bash commands into one Go command.
// this is pseudo code
cmd2 := exec.Command("head", "args1", string(out))
Basically, you have to do the piping yourself and use two separate commands rather than trying to invoke command once with commands that are composed using pipe.
Related
I'm unable to echo a string into a new file , it works if I use a filename with some extension(.txt, .go, etc) but it doesn't work when I just use a filename without file extension
I want the below bash commands to be executed in golang
echo "testDir/*"> .git/info/sparse-checkout
git checkout <Branch Name>
code snippet:
// Remove the redirect from command
cmd := exec.Command("echo", "testDir/*")
// Make test file
testFile, err := os.Create(".git/info/sparse-checkout")
if err != nil {
panic(err)
}
defer testFile.Close()
// Redirect the output here (this is the key part)
cmd.Stdout = testFile
err = cmd.Start(); if err != nil {
panic(err)
}
cmd.Wait()
branchCmd := exec.Command("git checkout <Branch Name>")
The echo command just print the list of arguments. The command interpreter (shell) expand the * based on globing existing filenames.
Without the shell, you can’t echo anything. You may need to call the shell (bash for instance) to expand and echo it, or use the io package to list files based on this pattern
If I understand correctly : in your bash command, echo "testDir/*" doesn't expand anything, it just outputs the string "testDir/*". echo ... > file is just one of the gazillion ways to set the file content to a given value from your shell.
If you want to write a fixed string to a file in go, just write it :
_, err := testFile.Write([]byte("testDir/*\n"))
no need to start some external process to echo a value on stdout.
I try to get video file duration with ffprobe. When I run this code I get error:
exit status 1:
var out bytes.Buffer
var stderr bytes.Buffer
cmdArgs := []string{"-i", "bunny.mp4", "-show_entries", "format=duration", "-v", "quiet", "-of", `csv="p=0"`}
cmd := exec.Command("ffprobe", cmdArguments...)
cmd.Stdout = &out
cmd.Stderr = &stderr
if err := cmd.Run(); err != nil {
fmt.Println(fmt.Sprint(err) + ": " + stderr.String())
}
fmt.Printf("command output: %q\n", out.String())
But when I pass argruments without -of csv="p=0" like this:
cmdArgs := []string{"-i", "bunny.mp4", "-show_entries", "format=duration", "-v", "quiet"}
It works and returns the result (but in bad format):
command output: "[FORMAT]\nduration=3.008000\n[/FORMAT]\n"
So what is the problem and how to solve it ?
Try formatting the argument like this (use double quotes for the string instead of backticks and remove the inner double quotes):
cmdArgs := []string{..., "csv=p=0"}
The Go exec package does not invoke the sytem shell to process the arguments, so you do not need to take the same precautions when specifying them. In this case, there's no need to wrap the portion after the first "=" in quotes.
From the package documentation:
Unlike the "system" library call from C and other languages, the
os/exec package intentionally does not invoke the system shell and
does not expand any glob patterns or handle other expansions,
pipelines, or redirections typically done by shells. The package
behaves more like C's "exec" family of functions. To expand glob
patterns, either call the shell directly, taking care to escape any
dangerous input, or use the path/filepath package's Glob function. To
expand environment variables, use package os's ExpandEnv.
In my shell, I can exec command acme.sh --issue --dns -d exmaple.com --yes-I-know-dns-manual-mode-enough-go-ahead-please and get output.
now I want to do that in go, and my code like:
cmd := exec.Command("bash", "-c", "acme.sh --issue --dns -d exmaple.com --yes-I-know-dns-manual-mode-enough-go-ahead-please");
out, err := cmd.CombinedOutput()
if err != nil {
log.Fatalf("issue failed with error: %s\n", err)
}
fmt.Printf("combined out:\n%s\n", string(out))
but I got error exit status 1.
and as the comment said, I separate argument:
exec.Command("bash", "-c", "acme.sh", "--issue", "--dns", "-d exmaple.com", "--yes-I-know-dns-manual-mode-enough-go-ahead-please");
but the result is that it exec acme.sh without parameters.
Using this script as acme.sh
#!/bin/bash
echo "$*"
with the program as given in the same directory the error you report occurs
However, if I add the current directory to the shell PATH
export PATH=.:$PATH
Then the program executes as hoped for, in the case of my version
$ go run c.go
combined out:
--issue --dns -d exmaple.com --yes-I-know-dns-manual-mode-enough-go-ahead-please
Ok so that's the case when the bash -c takes a single string as the command (more about that in a moment)
If the command is issued like this
cmd := exec.Command("acme.sh", "--issue", "--dns", "-d exmaple.com", "--
yes-I-know-dns-manual-mode-enough-go-ahead-please")
Then as the later edits to your question state, the command acme.sh is run without arguments.
The problem is with the way that bash -c behaves.
From the man page
bash interprets the following options when it
is invoked:
-c If the -c option is present, then commands are read from the
first non-option argument command_string. If there are argu‐
ments after the command_string, they are assigned to the
positional parameters, starting with $0.
In your case what this means is that the first arg to bash -c is accepted as a command. The other args are lost as they are positional args to the new bash shell and not to the acme.sh command
See this for more details on bash -c https://unix.stackexchange.com/questions/144514/add-arguments-to-bash-c
Lastly, what I would do in this situation: Skip the "bash" "-c", ensure that the script has a correct bang line and rely on the kernel binfmt handler
exclude exit status 1 from err will get right result.
cmd := exec.Command("bash", "-c", "acme.sh --issue --dns -d exmaple.com --yes-I-know-dns-manual-mode-enough-go-ahead-please");
out, err := cmd.CombinedOutput()
if err != nil && err.Error() != "exit status 1" {
log.Fatalf("issue failed with error: %s\n", err)
}
fmt.Printf("combined out:\n%s\n", string(out))
I am trying to invoke a git log from go and redirect the output to a given file.
cmdArgs = []string{"log", "--numstat", "--reverse", fmt.Sprintf("%s..HEAD", "89c98f5ec48c8ac383ea9e27d792c3dc77fa6240"), `--pretty="format:=%P %H %an %ae %ad %at %s %b"`}
cmdArgs = append(cmdArgs, ">> "+workingDir+"/logs/"+repoName+".log && cat "+workingDir+"/logs/"+repoName+".log")
cmd := exec.Command("git", cmdArgs...)
cmd.Dir = workingDir + repoName
var out bytes.Buffer
var stderr bytes.Buffer
cmd.Stdout = &out
cmd.Stderr = &stderr
err := cmd.Run()
if err != nil {
fmt.Println("git", strings.Join(cmdArgs, " "), "in", workingDir+repoName)
fmt.Println(fmt.Sprint(err) + ": " + stderr.String())
panic(err)
}
fails with
git log --numstat --reverse --pretty="format:=%P %H %an %ae %ad %at %s %b" 89c98f5ec48c8ac383ea9e27d792c3dc77fa6240..HEAD > /someplace/xx-tmp.log && cat /someplace/xx-tmp.log in /someplace
exit status 128: fatal: ambiguous argument ' > /someplace/xx-tmp.log && cat /someplace/xx-tmp.log: unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
Executing the command in bash directly poses no problem.
Go's cmd.Run() acts similarly to the C fork() and exec() process for starting a new program. This doesn't implicitly invoke a shell -- which is, from a security perspective, an extremely good thing; unnecessary shell invocation often leads to command injection vulnerabilities.
If you want capabilities that a shell can add -- here, redirection and compound command syntax -- but want to avoid the security risks, pass your data out-of-band from code:
cmdArgs = []string{
"-c", // tells interpreter that script is next argument
`outfile=$1; shift; "$#" >"$outfile" && cat "$outfile"`, // script to execute
"_", // this is $0
workingDir+"/logs/"+repoName+".log", // $1, becomes outfile
"git", "log", "--numstat", "--reverse", // remaining args are in "$#"
fmt.Sprintf("%s..HEAD", "89c98f5ec48c8ac383ea9e27d792c3dc77fa6240"),
"--pretty=format:=%P %H %an %ae %ad %at %s %b"
}
cmd := exec.Command("sh", cmdArgs...)
The above is equivalent to the following shell script:
#!/bin/sh
# ^^- not /bin/bash; this only guarantees support for POSIX syntax
outfile=$1 # assign first positional argument to variable "$outfile"
shift # rename $2 to $1, $3 to $2, etc
if "$#" >"$outfile"; then # run remaining arguments as a single command, stdout to outfile
cat "$outfile" # if that succeeded, then cat our "$outfile" to stdout
fi
Note that I removed the literal quotes from inside --pretty=. That's because when you run the command in a shell, those quotes are treated as syntax by the shell -- an instruction not to split on the spaces within the format string. Here, there is no shell parsing that string as code; if we left the quotes in, they would become a literal part of your format string.
The above being said, there's no good reason to use a shell for any of this. You can do the redirection in native Go trivially:
cmdArgs = []string{
"log", "--numstat", "--reverse",
fmt.Sprintf("%s..HEAD", "89c98f5ec48c8ac383ea9e27d792c3dc77fa6240"),
`--pretty=format:=%P %H %an %ae %ad %at %s %b`}
cmd := exec.Command("git", cmdArgs...)
cmd.Dir = workingDir + repoName
outputFile, err = os.OpenFile(workingDir+"/logs/"+repoName+".log",
os.O_APPEND | os.O_CREATE | os.O_WRONLY, 0644)
if err != nil { panic(err) }
defer outputFile.Close()
cmd.Stdout = outputFile
...and then run the cat as a separate exec.Command instance (if you have a good reason not to just implement it in native Go).
I write simple script using Golang to grep a log file with some parameters. Here's my shell command
grep CRON var/log/sys | tail -5 | grep "cd /home/raka/repo && git status"
I want to run command above in Golang using os/exec package. Here's my code sniped.
var (
reader io.Reader
out []byte
err error
commandName string = "grep"
)
args := []string{"CRON", "/var/log/syslog", "| tail -6", "| grep \"git status\""}
cmd := exec.Command(commandName, args...)
r, err = cmd.StdoutPipe()
err = cmd.Start()
out, err = ioutil.ReadAll(r)
err = cmd.Wait()
return strings.Split(string(out), "\n")```
Currently, the sniped above doesn't work, because of exit status 2.
Any of you guys/ladies have solution for this problem? thank you so much.
Pipes (|) are implemented by a shell program (like bash). If you want to use them you should execute shell passing a command containing piped program invocations:
exec.Command("/bin/sh", "-c",
"grep CRON var/log/sys | tail -5 | grep \"cd /home/raka/repo && git status\"")