I'm unable to echo a string into a new file , it works if I use a filename with some extension(.txt, .go, etc) but it doesn't work when I just use a filename without file extension
I want the below bash commands to be executed in golang
echo "testDir/*"> .git/info/sparse-checkout
git checkout <Branch Name>
code snippet:
// Remove the redirect from command
cmd := exec.Command("echo", "testDir/*")
// Make test file
testFile, err := os.Create(".git/info/sparse-checkout")
if err != nil {
panic(err)
}
defer testFile.Close()
// Redirect the output here (this is the key part)
cmd.Stdout = testFile
err = cmd.Start(); if err != nil {
panic(err)
}
cmd.Wait()
branchCmd := exec.Command("git checkout <Branch Name>")
The echo command just print the list of arguments. The command interpreter (shell) expand the * based on globing existing filenames.
Without the shell, you can’t echo anything. You may need to call the shell (bash for instance) to expand and echo it, or use the io package to list files based on this pattern
If I understand correctly : in your bash command, echo "testDir/*" doesn't expand anything, it just outputs the string "testDir/*". echo ... > file is just one of the gazillion ways to set the file content to a given value from your shell.
If you want to write a fixed string to a file in go, just write it :
_, err := testFile.Write([]byte("testDir/*\n"))
no need to start some external process to echo a value on stdout.
Related
I have a bash script in which I am receiving response of an api in string format. The response is in the following format:
foo bar test stack over flow
Now I am having following bash script to convert it to array and process further:
#!/bin/bash
result=$(curl "API URL")
resp=($result)
for i in "${resp[#]}"
do
echo "$i"
done
Problem:
If I run this script manually in the terminal (by making it executable) it works fine. But when I try to run it by using Golang sh command
ExecuteCommand("sh /path/to/directory/test.sh")
func ExecuteCommand(command string) error{
cmd := exec.Command("sh", "-c",command)
var out bytes.Buffer
var stderr bytes.Buffer
cmd.Stdout = &out
cmd.Stderr = &stderr
err := cmd.Run()
if err != nil {
fmt.Println(fmt.Sprint(err) + ": " + stderr.String())
return err
}
fmt.Println("Result: " + out.String())
return nil
}
It gives me error:
test.sh: Syntax error: "(" unexpected
can someone help me what am I doing wrong ?
change these lines in your example
ExecuteCommand("/path/to/directory/test.sh")
func ExecuteCommand(command string) error{
cmd := exec.Command(command)
The kernel shebang will intercept the #! line and correctly run the script
U can also create a shortcut from "test.sh" file :
#!/bin/bash
$ sudo ln -s /path/to/directory/test.sh /usr/bin/testSH
If u arent root user give permission to the shortcut :
sudo chmod 777 /usr/bin/testSH
command := "testSH"
cmd := exec.Command("sh", "-c",command).Run()
I try to get video file duration with ffprobe. When I run this code I get error:
exit status 1:
var out bytes.Buffer
var stderr bytes.Buffer
cmdArgs := []string{"-i", "bunny.mp4", "-show_entries", "format=duration", "-v", "quiet", "-of", `csv="p=0"`}
cmd := exec.Command("ffprobe", cmdArguments...)
cmd.Stdout = &out
cmd.Stderr = &stderr
if err := cmd.Run(); err != nil {
fmt.Println(fmt.Sprint(err) + ": " + stderr.String())
}
fmt.Printf("command output: %q\n", out.String())
But when I pass argruments without -of csv="p=0" like this:
cmdArgs := []string{"-i", "bunny.mp4", "-show_entries", "format=duration", "-v", "quiet"}
It works and returns the result (but in bad format):
command output: "[FORMAT]\nduration=3.008000\n[/FORMAT]\n"
So what is the problem and how to solve it ?
Try formatting the argument like this (use double quotes for the string instead of backticks and remove the inner double quotes):
cmdArgs := []string{..., "csv=p=0"}
The Go exec package does not invoke the sytem shell to process the arguments, so you do not need to take the same precautions when specifying them. In this case, there's no need to wrap the portion after the first "=" in quotes.
From the package documentation:
Unlike the "system" library call from C and other languages, the
os/exec package intentionally does not invoke the system shell and
does not expand any glob patterns or handle other expansions,
pipelines, or redirections typically done by shells. The package
behaves more like C's "exec" family of functions. To expand glob
patterns, either call the shell directly, taking care to escape any
dangerous input, or use the path/filepath package's Glob function. To
expand environment variables, use package os's ExpandEnv.
I am trying to run a command with go. The command is in a string.
package main
import (
"log"
"os"
"os/exec"
"strings"
"github.com/davecgh/go-spew/spew"
)
func main() {
commandToRun := `echo $HOME`
log.Printf("Running %s\n", commandToRun)
args := strings.Fields(commandToRun)
spew.Dump(args[1:len(args)])
command := exec.Command(args[0], args[1:len(args)]...)
command.Stdout = os.Stdout
command.Stdin = os.Stdin
command.Stderr = os.Stderr
err := command.Run()
if err != nil {
log.Printf("Command finished with error: %v", err)
}
}
The output is:
2018/11/14 09:41:22 Running echo $HOME
([]string) (len=1 cap=1) {
(string) (len=5) "$HOME"
}
$HOME
What I'd like to have is:
2018/11/14 09:41:22 Running echo $HOME
([]string) (len=1 cap=1) {
(string) (len=5) "$HOME"
}
/home/whatever
Looks like go is sanitizing the string somehow. So the $HOME is not expanded. Is there any way of running the string exactly as if it was typed into the shell?
This is the important part. Ideally I'd like to turn from string to type in the current shell.
EDIT: The example below solve the simplest scenario but doesn't cover the "running the string exactly as if it was typed into the shell" part.
If I switch to expandenv:
commandToRun := os.ExpandEnv(`echo "$HOME"`)
I get:
2018/11/14 11:45:44 Running echo "/Users/rafael"
([]string) (len=1 cap=1) {
(string) (len=15) "\"/home/whatever\""
}
"/home/whatever"
What I'd get in the shell is:
$ > echo "$HOME"
/home/whatever
without the quotes.
This is close to what I want but not exactly it.
$HOME (and all other env variables) are expanded by the shell. You're not executing a shell, so they don't get expanded.
You need to look up the env variable directly in go, with something like:
command := exec.Command("echo", os.Getenv("HOME"))
or this:
commandToRun := os.ExpandEnv("echo $HOME")
args := strings.Fields(commandToRun)
command := exec.Command(args[0], args[1:]...)
Note that this last approach won't work if $HOME expands to a string containing whitespace, so the os.Getenv method is generally safer/preferred for this use case.
Before executing the command, you can actively expand all env vars in the string using os.ExpandEnv:
os.ExpandEnv("echo $HOME")
From the docs:
ExpandEnv replaces ${var} or $var in the string according to the values of the current environment variables. References to undefined variables are replaced by the empty string.
If you want to get the output of $ echo $HOME, the minimal code you need is
fmt.Println(os.Getenv("HOME"))
Nothing more is needed.
If you use os.ExpandEnv("echo $HOME"), then first $HOME var will be expanded and then it will give you a string like echo /home/<user>
If you use command := exec.Command("echo", os.Getenv("HOME")), then it will be resolved as command := exec.Command("echo", "/home/<user>") and finally which will give output /home/<user>
If you use
commandToRun := os.ExpandEnv("echo $HOME")
command := exec.Command(strings.Fields(commandToRun)...)
then it will be process like previous cases.
So better way is using only fmt.Println(os.Getenv("HOME")).
I am trying to invoke a git log from go and redirect the output to a given file.
cmdArgs = []string{"log", "--numstat", "--reverse", fmt.Sprintf("%s..HEAD", "89c98f5ec48c8ac383ea9e27d792c3dc77fa6240"), `--pretty="format:=%P %H %an %ae %ad %at %s %b"`}
cmdArgs = append(cmdArgs, ">> "+workingDir+"/logs/"+repoName+".log && cat "+workingDir+"/logs/"+repoName+".log")
cmd := exec.Command("git", cmdArgs...)
cmd.Dir = workingDir + repoName
var out bytes.Buffer
var stderr bytes.Buffer
cmd.Stdout = &out
cmd.Stderr = &stderr
err := cmd.Run()
if err != nil {
fmt.Println("git", strings.Join(cmdArgs, " "), "in", workingDir+repoName)
fmt.Println(fmt.Sprint(err) + ": " + stderr.String())
panic(err)
}
fails with
git log --numstat --reverse --pretty="format:=%P %H %an %ae %ad %at %s %b" 89c98f5ec48c8ac383ea9e27d792c3dc77fa6240..HEAD > /someplace/xx-tmp.log && cat /someplace/xx-tmp.log in /someplace
exit status 128: fatal: ambiguous argument ' > /someplace/xx-tmp.log && cat /someplace/xx-tmp.log: unknown revision or path not in the working tree.
Use '--' to separate paths from revisions, like this:
'git <command> [<revision>...] -- [<file>...]'
Executing the command in bash directly poses no problem.
Go's cmd.Run() acts similarly to the C fork() and exec() process for starting a new program. This doesn't implicitly invoke a shell -- which is, from a security perspective, an extremely good thing; unnecessary shell invocation often leads to command injection vulnerabilities.
If you want capabilities that a shell can add -- here, redirection and compound command syntax -- but want to avoid the security risks, pass your data out-of-band from code:
cmdArgs = []string{
"-c", // tells interpreter that script is next argument
`outfile=$1; shift; "$#" >"$outfile" && cat "$outfile"`, // script to execute
"_", // this is $0
workingDir+"/logs/"+repoName+".log", // $1, becomes outfile
"git", "log", "--numstat", "--reverse", // remaining args are in "$#"
fmt.Sprintf("%s..HEAD", "89c98f5ec48c8ac383ea9e27d792c3dc77fa6240"),
"--pretty=format:=%P %H %an %ae %ad %at %s %b"
}
cmd := exec.Command("sh", cmdArgs...)
The above is equivalent to the following shell script:
#!/bin/sh
# ^^- not /bin/bash; this only guarantees support for POSIX syntax
outfile=$1 # assign first positional argument to variable "$outfile"
shift # rename $2 to $1, $3 to $2, etc
if "$#" >"$outfile"; then # run remaining arguments as a single command, stdout to outfile
cat "$outfile" # if that succeeded, then cat our "$outfile" to stdout
fi
Note that I removed the literal quotes from inside --pretty=. That's because when you run the command in a shell, those quotes are treated as syntax by the shell -- an instruction not to split on the spaces within the format string. Here, there is no shell parsing that string as code; if we left the quotes in, they would become a literal part of your format string.
The above being said, there's no good reason to use a shell for any of this. You can do the redirection in native Go trivially:
cmdArgs = []string{
"log", "--numstat", "--reverse",
fmt.Sprintf("%s..HEAD", "89c98f5ec48c8ac383ea9e27d792c3dc77fa6240"),
`--pretty=format:=%P %H %an %ae %ad %at %s %b`}
cmd := exec.Command("git", cmdArgs...)
cmd.Dir = workingDir + repoName
outputFile, err = os.OpenFile(workingDir+"/logs/"+repoName+".log",
os.O_APPEND | os.O_CREATE | os.O_WRONLY, 0644)
if err != nil { panic(err) }
defer outputFile.Close()
cmd.Stdout = outputFile
...and then run the cat as a separate exec.Command instance (if you have a good reason not to just implement it in native Go).
For fun and to better learn Go, I'm trying to re-implement antigen in Go.
Problem is: source is a shell built-in function, so I can't call it with os/exec Command function, because it expects an executable in PATH.
How can I do this? And, is it possible to make a source from inside a go program affect the user shell?
You can write the command directly in the terminal device. But, to do that, first you need to know which device is using the user. A script that executes your program can be a solution.
#!/bin/bash
echo Running from foo script, pid = $$
go run foo.go `tty`
Then, the program has to write the commands to the terminal device.
package main
import (
"C"
"fmt"
"os"
"syscall"
"unsafe"
)
func main() {
// Get tty path
if len(os.Args) < 2 {
fmt.Printf("no tty path\n")
os.Exit(1)
}
ttyPath := os.Args[1]
// Open tty
tty, err := os.Open(ttyPath)
if err != nil {
fmt.Printf("error opening tty: %s\n", err.Error())
os.Exit(2)
}
defer tty.Close()
// Write a command
cmd := "echo Hello from go, pid = $$\n"
cmdstr := C.CString(cmd)
cmdaddr := uintptr(unsafe.Pointer(cmdstr))
for i := range []byte(cmd) {
_, _, err := syscall.Syscall(syscall.SYS_IOCTL, tty.Fd(), syscall.TIOCSTI, cmdaddr+uintptr(i))
if uintptr(err) != 0 {
fmt.Printf("syscall error: %s\n", err.Error())
os.Exit(3)
}
}
}
Here is an example output:
$ echo $$
70318
$ ./foo
Running from foo script, pid = 83035
echo Hello from go, pid = $$
$ echo Hello from go, pid = $$
Hello from go, pid = 70318
Note that I am executing the script with ./ not source, so the PID of the script differs. But later, the command executed by the go program has the same PID.