exec command every one minute and serve the output via http - go

Question

I want to run a command in the Bash shell every one minute, and serve the output via http, on http://localhost:8080/feed
package main
import (
"fmt"
"os/exec"
)
func main() {
cmd := `<a piped command>`
out, err := exec.Command("bash", "-c", cmd).Output()
if err != nil {
fmt.Sprintf("Failed to execute command: %s", cmd)
}
fmt.Println(string(out))
}
UPDATE :
package main
import (
"fmt"
"log"
"net/http"
"os/exec"
)
func handler(w http.ResponseWriter, r *http.Request) {
cmd := `<a piped command>`
out, err := exec.Command("bash", "-c", cmd).Output()
if err != nil {
fmt.Sprintf("Failed to execute command: %s", cmd)
}
fmt.Fprintf(w, string(out))
}
func main() {
http.HandleFunc("/feed", handler)
log.Fatal(http.ListenAndServe(":8080", nil))
}
With the above code, the command is run every time http://localhost:8080/feed is accessed. How do I make it cache(?) the output of the command for one minute, and then run the command again only after the cache time is expired?

Answer 1

If output is not too big, you can keep it in memory variable.
I took approach to wait for result in case that script is executed (using mutex):
package main
import (
"fmt"
"net/http"
"os/exec"
"sync"
"time"
)
var LoopDelay = 60*time.Second
type Output struct {
sync.Mutex
content string
}
func main() {
var output *Output = new(Output)
go updateResult(output)
http.HandleFunc("/feed", initHandle(output))
err := http.ListenAndServe(":8080", nil)
if err != nil {
fmt.Println("ERROR", err)
}
}
func initHandle(output *Output) func(http.ResponseWriter, *http.Request) {
return func(respw http.ResponseWriter, req *http.Request) {
output.Lock()
defer output.Unlock()
_, err := respw.Write([]byte(output.content))
if err != nil {
fmt.Println("ERROR: Unable to write response: ", err)
}
}
}
func updateResult(output *Output) {
var execFn = func() { /* Extracted so that 'defer' executes at the end of loop iteration */
output.Lock()
defer output.Unlock()
command := exec.Command("bash", "-c", "date | nl ")
output1, err := command.CombinedOutput()
if err != nil {
output.content = err.Error()
} else {
output.content = string(output1)
}
}
for {
execFn()
time.Sleep(LoopDelay)
}
}
Executing
date; curl http://localhost:8080/feed
gives output (on multiple calls):
1 dimanche 13 octobre 2019, 09:41:40 (UTC+0200)
http-server-cmd-output> date; curl http://localhost:8080/feed
dimanche 13 octobre 2019, 09:42:05 (UTC+0200)
1 dimanche 13 octobre 2019, 09:41:40 (UTC+0200)
Few things to consider:
- Used 'date | nl' as command with pipe example
- If output too big, write to file
- Very likely it is good idea to keep mutex only for content update (no need to wait during script execution) - you can try to improve it
- Go routine may have variable (channel) to exit on signal (ex: when program ends)
EDIT: variable moved to main function

Answer 2

How do I make it cache(?) the output of the command for one minute,
and then run the command again only after the cache time is expired?
In below solution two goroutines are declared.
First goroutine loop until the context is done to execute the command at regular interval and send a copy to the second go routine.
The second routine, tries to get from the first goroutine, or distribute its value to other goroutines.
The http handler, third goroutine, only queries the value from the getter and does something with it.
The reason to use three routines instead of two in this example is to prevent blocking the http routines if the command is being executed. With that additional routines, http requests only wait for the synchronization to occur.
type state is a dummy struct to transport the values within channels.
We prevent race conditions because of two facts, the state is passed by value, cmd.Output() allocates a new buffer each time it runs.
To retrieve original command in the http handler, OP should build a custom error type and attach those information to the recorded error, within the http handler, OP should type assert the error to its custom type and get the specific details from there.
package main
import (
"context"
"fmt"
"log"
"net/http"
"os/exec"
"time"
)
type state struct {
out []byte
err error
}
func main() {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
set := make(chan state, 1)
go func() {
ticker := time.NewTicker(2 * time.Second)
cmd := []string{"date"}
s := state{}
s.out, s.err = exec.Command(cmd[0], cmd[1:]...).Output()
set <- s
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
s.out, s.err = exec.Command(cmd[0], cmd[1:]...).Output()
set <- s
}
}
}()
get := make(chan state)
go func() {
s := state{}
for {
select {
case <-ctx.Done():
return
case s = <-set: // get the fresh value, if any
case get <- s: // distribute your copy
}
}
}()
http.HandleFunc("/feed", func(w http.ResponseWriter, r *http.Request) {
state := <-get
if state.err != nil {
fmt.Printf("Failed to execute command: %v", state.err)
}
fmt.Fprintf(w, string(state.out))
})
log.Fatal(http.ListenAndServe(":8080", nil))
}

Answer 3

You could:
write the last command instance (the "string(out)") to a log file: see "Go/Golang write log to file"
serve that log file (and only the file) in a Go http server: see "How to serve a file to a specific URL path with the FileServer function in go/golang"
That way, any time you go to your server, you will get the last command output.
How do I make it cache(?) the output of the command for one minute, and then run the command again only after the cache time is expired?
By keeping the execution of the command separate from the command output being served.
That is why you must kept the two asynchronous, typically through a goroutine, a a time.NewTicker.
See "Is there a way to do repetitive tasks at intervals?"