What is the difference between os.Stdout and syscall.Stdout? - go

I have been trying to work ForkExec() and I am not able to get this one work, is there a difference between syscall.Stdout and os.Stdout?
Here is a small example of the code I am trying to run.
command := "/usr/bin/echo"
args := []string{"Hello there."}
attr := new(syscall.ProcAttr)
attr.Env = os.Environ()
attr.Files = []uintptr{uintptr(syscall.Stdin), uintptr(syscall.Stdout), uintptr(syscall.Stderr)}
pid , err := syscall.ForkExec(command, args, attr)
if err != nil {
log.Fatal(err)
}
fmt.Println(pid)
The output is not showing up on the screen.
Thanks a lot for your help in advance.

os.Stdout is a *os.File. It works with go functions that want an io.Writer or similar interfaces. syscall.Stdout is an integer constant. It's the file descriptor number of stdout, which is useful for low-level syscalls.
syscall.ForkExec does indeed want file descriptor numbers... but it's unclear why you're using that instead of os/exec.Cmd which is much more straightforward.

Related

Golang: unit testing fuction depenent on io.File.Write()

Hеllo, fellow Gophers!
Learning how to write unit tests and got stuck. Please, don't beat me up, cause I am new to this. I've googled and read a lot, but I still couldn't come up with a proper solution.
The exercise is to write a unit test for a function I wrote:
Filecopy(pathfrom, pathto string, limit, offset int64) error
The simplified function will look like this
source, err := os.Open(pathfrom)
destination, err := os.Create(pathto)
buf := make([]byte, *buffersize)
for {
n, err := source.Read(buf)
if err != nil && err != io.EOF {
return err
}
if _, err := destination.Write(buf[pos:n]); err != nil {
return err
if n == 0 {
break
}
}
}
Now I have to unit test it. And it's not going well. There are two ways I could test it:
Provide it a real temp file with a predetermined contents, so the test function would know what to expect after the execution. Should work, but I read this option is NOT the way to unit test properly as it touches the FS.
Use fake FS. I've tried testing files generated by mapfs and afero, but no luck, my function doesn't work with it (it works with real files though), either I am doing something wrong.
Please tell me how to unit test my function properly. Any help will be appreciated.

Porting from ruby to go: find and replace in a file

I am porting some ruby code to golang. I'm having difficulty finding a good equivalent for the below line and wondered if someone knew of a better solution than what I have below. The basic premise is find a line in a file that has a lot of spaces and remove the line.
I also thought of just using exec to call sed -i but when I tried that it didn't work, and the below did finally work.
Ruby:
File.write(filename, File.read(filename).gsub(/^\s*$/,""))
Golang:
b, err := ioutil.ReadFile(filename)
if err != nil {
return
}
// I happen to know that there will be at least 30 spaces,
// but I would really prefer to not use a hardcoded value here.
// I was just never able to make using '^\s*$' work in the regex.
r := regexp.MustCompile(`[ ]{30,}`) // there's a space in the []
newb := r.ReplaceAll(b, []byte(""))
err = ioutil.WriteFile(filename, newb, 0666)
if err != nil {
fmt.Printf("Unable to write to file (%+v)\n", err)
return
}
Turn on multiline mode, and your original pattern will work:
r := regexp.MustCompile(`(?m)^\s*$`)
Demo using strings: https://play.golang.org/p/6TsfgB83WgX

Copy exec.Command output to file as the buffer receives data

I have a script that dumps quite a bit of text into STDOUT when run. I'm trying to execute this script and write the output to a file without holding the entire buffer in memory at one time. (We're talking many megabytes of text that this script outputs at one time.)
The following works, but because I'm doing this across several goroutines, my memory consumption shoots up to > 5GB which I would really like to avoid:
var out bytes.Buffer
cmd := exec.Command("/path/to/script/binary", "arg1", "arg2")
cmd.Stdout = &out
err := cmd.Run()
if err != nil {
log.Fatal(err)
}
out.WriteTo(io) // io is the writer connected to the new file
Ideally as out fills up, I want to be emptying it into my target file to keep memory usage low. I've tried changing this to:
cmd := exec.Command("/path/to/script/binary", "arg1", "arg2")
cmd.Start()
stdout, _ := cmd.StdoutPipe()
r := *bufio.NewReader(stdout)
r.WriteTo(io)
cmd.Wait()
However when I print out these variables stdout is <nil>, r is {[0 0 0 0 0...]}, and r.WriteTo panics: invalid memory address or nil pointer dereference.
Is it possible to write the output of cmd as it is generated to keep memory usage down? Thanks!
Why don't you just write to a file directly?
file, _ := os.Create("/some/file")
cmd.Stdout = file
Or use your io thing (that's a terrible name for a variable, by the way, since it's a) the name of a standard library package, b) ambiguous--what does it mean?)
cmd.Stdout = io

Working in the console via exec.Command

Please help. I have to pass the console commando with a certain number of parameters. There are many.
That is, ideally, should be as follows:
test.go --distr
For example:
test.go --distr mc curl cron
i create function
func chroot_create() {
cmd := exec.Command("urpmi",
"--urpmi-root",
*fldir,
"--no-verify-rpm",
"--nolock",
"--auto",
"--ignoresize",
"--no-suggests",
"basesystem-minimal",
"rpm-build",
"sudo",
"urpmi",
"curl")
if err := cmd.Run(); err != nil {
log.Println(err)
}
}
And catch parameter distr through flag.Parse ()
How do I get rid of
"rpm-build",
"sudo",
"urpmi",
"curl")
That would not be tied to count packets. Please forgive me for stupidity, I'm just starting to learn golang. Especially when there was a problem.
Full code http://pastebin.com/yeuKy8Cc
You are looking for the ... operator.
func lsElements(elems ...string) {
cmd := exec.Command("ls", append([]string{"-l", "-h", "/root"}, elems...)...)
if err := cmd.Run(); err != nil {
log.Println(err)
}
}
You receive as function parameter ...string which is in really a []string, except that when you call the function, you pass the strings separately.
In order to use it, (and it works with any slices), you can "transform" your slice into list of element with ... suffix.
In the case of exec, you could use elem... directly if you had only this. However, has you have fixed parameters as well, you need to build your slice with append and extend it back with ...
Example: http://play.golang.org/p/180roQGL4a

Go language how detect file changing?

I need to know how to detect when a file changes using Go. I know that Unix provides a function named fcntl() which notifies when a specific file is changed but I have not found this one in Go.
Please help me.
Here's a simple cross-platform version:
func watchFile(filePath string) error {
initialStat, err := os.Stat(filePath)
if err != nil {
return err
}
for {
stat, err := os.Stat(filePath)
if err != nil {
return err
}
if stat.Size() != initialStat.Size() || stat.ModTime() != initialStat.ModTime() {
break
}
time.Sleep(1 * time.Second)
}
return nil
}
And usage would be like this:
doneChan := make(chan bool)
go func(doneChan chan bool) {
defer func() {
doneChan <- true
}()
err := watchFile("/path/to/file")
if err != nil {
fmt.Println(err)
}
fmt.Println("File has been changed")
}(doneChan)
<-doneChan
Not as efficient as a proper system call but it's simple and works everywhere, and might be enough for some uses.
I would add to the peterSO's answer that if you in fact want to read the data appended to a file by some other process(es) — what tail program does in Unix, — it's probably better to just make tail itself do the hard job of monitoring the file and consume what it outputs. This can be achieved by running tail using the StdoutPipe function from the exec package.
Using tail for this kind of task is preferable in my eyes because tail has been taught to use a bunch of clever tricks including detection of file replacements (commonly occuring when one monitors a log file which is being rotated by logrotate or something similar).
There is currently an experimental package here. It should be merged into core as os/fsnotify in go1.3
Have a look at https://github.com/howeyc/fsnotify. It wraps the inotify subsystem of the Linux kernel and should work in Go1.
As of Go1 inotify has been removed from the package. Have a look at the syscall package now...
Package inotify implements a wrapper for the Linux inotify
system.

Resources