I have a system I need to operate as a daemon, which has been developed in go. Ordinarily I could just manage this though something like systemd or even just something as simple as './project &' but this particular project needs to get some input before it can become a daemon - it needs a password to be entered.
This password cannot be provided as command line args, placed in a config file, hard coded in the source etc. It must be manually provided by someone who has knowledge of the password at system startup.
At startup, the project needs to collect the password, verify that things are correct, and only if the correct password has been provided (verified by attempting to decrypt a certificate) can it then actually begin to operate as the daemon.
Is there any way in go for the project to startup, accept input from stdin, perform whatever validation is needed etc, and only then become a daemon process (detaching from stdin etc)?
Currently I can simulate the required behavior by starting it, and after the password has been entered using 'ctrl+z' to suspend it and bg to push the process to the background. Surely there has to be a way of doing that from within the process itself...
here is a program that does what I think you meant.
so the program only checks the password. The actual daemon is webserver which gets called only when password checks out.
package main
import (
"golang.org/x/crypto/ssh/terminal"
"os/exec"
)
func main() {
pwd, err := terminal.ReadPassword(1)
if err != nil {
panic(err)
}
if string(pwd) == "test" {
c := exec.Command("./webserver")
c.Run()
}
return
}
the daemon program I used here was taken from here
https://github.com/sevlyar/go-daemon/blob/master/examples/cmd/gd-simple/simple.go
Its not possible to run a single program both as waiting for input and as a daemon.
You could use flags and carefully control your app flow and run itself as a daemon if a given flag (such as encrypted credentials) is passed. Or even store them in a temporary file, database or anywhere.
func main() {
cred := flag.String("cred", "", "raw cred")
flag.Parse()
if *cred == "" {
fmt.Print("Enter credentials:\n")
decryptedCred, _ := bufio.NewReader(os.Stdin).ReadString('\n')
if !validCred(decryptedCred) {
os.Exit(1)
}
encryptedCred := encryptCredentials(decryptedCred)
cmd := exec.Command("./project", fmt.Sprintf("-cred=%s", encryptedCred), "&")
cmd.Start()
fmt.Printf("Started project with pid: %d\n", cmd.Process.Pid)
os.Exit(0)
}
for {
// start app
// use *cred here
}
}
Whatever the approach I would probably keep track of the pid, etc.
./project
# Enter credentials:
myCredentialsString
# Started project with pid: 11702
ps ax | grep project
# 11667 s001 R 0:02.47 ./project -cred=N/esPq8wsWn4/+Gco16ddl9UnJ0= &\012
Hope this helps
Related
exec.Command() works for executing C:\Windows\System32\notepad.exe
But exec.Command() doesn't work for executing C:\Users\<username>\AppData\Local\Microsoft\WindowsApps\winget.exe. Fails with the error message:
exec: "C:\\Users\\<username>\\AppData\\Local\\Microsoft\\WindowsApps\\winget.exe": file does not exist
However, os.StartProcess() works for executing C:\Users\<username>\AppData\Local\Microsoft\WindowsApps\winget.exe
Can someone tell me why?
This code fragment does not work. winget.exe isn't launched.
wingetPath := filepath.Join(os.Getenv("LOCALAPPDATA"),
"Microsoft\\WindowsApps\\winget.exe")
cmd := exec.Command(wingetPath, "--version")
err := cmd.Start()
fmt.Println(err)
// exec: "C:\\Users\\<username>\\AppData\\Local\\Microsoft\\WindowsApps\\winget.exe": file does not exist
But this works:
wingetPath := filepath.Join(os.Getenv("LOCALAPPDATA"),
"Microsoft\\WindowsApps\\winget.exe")
procAttr := new(os.ProcAttr)
procAttr.Files = []*os.File{nil, nil, nil}
// The argv slice will become os.Args in the new process,
// so it normally starts with the program name
_, err := os.StartProcess(wingetPath, []string{wingetPath, "--version"}, procAttr)
fmt.Println(err)
// <nil>
Go version:
> go version
go version go1.18 windows/amd64
Bug in Golang
So apparently this is a bug in the Windows implementation of Go and has been filed on GitHub many times - the oldest I could find is this issue which was filed years ago.
The bug is caused by the fact that exec.Command() internally uses os.Stat() which does not read files with reparse points correctly. os.Lstat() can.
Windows Store apps use App Execution Aliases, which are essentially zero-byte files with reparse points. This post has some additional details.
Workarounds
Workaround is to use os.StartProces() - a lower level API which can be a bit painful to use especially when compared to os.Exec().
Important: In os.StartProcess(), the argv slice will become os.Args in the new process, so you should normally pass the program name as the first argument:
wingetPath := filepath.Join(os.Getenv("LOCALAPPDATA"),
"Microsoft\\WindowsApps\\winget.exe")
procAttr := new(os.ProcAttr)
procAttr.Files = []*os.File{nil, nil, nil}
/*
To redirect IO, pass in stdin, stdout, stderr as required
procAttr.Files = []*os.File{os.Stdin, os.Stdout, os.Stderr}
*/
args = []string { "install", "git.git" }
// The argv slice will become os.Args in the new process,
// so it normally starts with the program name
proc, err := os.StartProcess(wingetPath,
append([]string{wingetPath}, arg...), procAttr)
fmt.Println(err) // nil
Another approach to work around this bug is to (create and) execute a .cmd file (for example) which would (correctly resolve and) execute the file with reparse points. See this (and also this directory) for an example.
I am working on a small web server that serves files and provides access to each user's home directory.
If the source was to be in C I had the option of answering each request under different threads and to make sure each thread gets to run with the user of the caller as its users.
Is there any approach to achieve something similar to that in Go?
Ideally, the part of the code that handles the request, the goroutine or the method that gets called should be run under the user account of the caller.
I have done some research and it seems in Go we can stick a single goroutine to the current thread but I can't see how it is possible to create a new thread and then attach a goroutine to that thread.
It is not possible to run a goroutine or method as a different user because they both run within the same context as the parent process. Goroutines are equivalent to green threads and don't even necessarily spawn off proper OS thread per routine.
This answer might also depend on OS, but I don't think this will work on windows either.
if you are spawning another process via the cmd package, then this answer may be useful
Running external commands through os/exec under another user
Yes, you can do that with the use of the Linux syscall setuid (not the built in function setuid). I just found this question and thought that it has to be possible, as I use this in other programming languages too. So I got my problem solved and wanted to report back how to do this.
However, it is correct what SJP wrote about the threads and there lies exactly the answer to my problem, but it will not solve your problem, due to the threading issue - whole story in this very long issue 1435. Therein is also a suggestion in how to solve a specific subset of the setuid problem and that solved my problem.
But back to code ... you need to call LockOSThread in order to fix the current go routine to the thread you're currently executing in and in that, you can change the context with the syscall setuid.
Here is a working example for Linux:
package main
import (
"fmt"
"log"
"os"
"runtime"
"sync"
"syscall"
"time"
)
func printUID() {
fmt.Printf("Real UID: %d\n", syscall.Getuid())
fmt.Printf("Effective UID: %d\n", syscall.Geteuid())
}
func main() {
printUID()
var wg sync.WaitGroup
wg.Add(2)
go func(wg *sync.WaitGroup) {
defer wg.Done()
time.Sleep(2 * time.Second)
printUID()
}(&wg)
go func(wg *sync.WaitGroup) {
runtime.LockOSThread()
defer runtime.UnlockOSThread()
defer wg.Done()
_, _, serr := syscall.Syscall(syscall.SYS_SETUID, 1, 0, 0)
if serr != 0 {
log.Fatal(serr)
os.Exit(1)
}
printUID()
}(&wg)
wg.Wait()
printUID()
}
You will receive operation not supported if you use syscall.Setuid:
serr := syscall.Setuid(1)
instead of
_, _, serr := syscall.Syscall(syscall.SYS_SETUID, 1, 0, 0)
[This answer is similar to the one by #A.Steinel but, alas, I have insufficient reputation to actually comment on that one. Hopefully, this offers a little more of a complete worked example and, importantly, demonstrates keeping the runtime free of the confusion of threads running with different UIDs.]
First, to strictly do what you asked requires a number of hacks and isn't all that secure...
[Go likes to operate with POSIX semantics, and what you want to do is break POSIX semantics by operating with two or more UIDs at the same time in a single process. Go wants POSIX semantics because it runs goroutines on whatever thread is available, and the runtime needs them to all behave the same for this to work reliably. Since Linux's setuid() syscall doesn't honor POSIX semantics, Go opted to not implement syscall.Setuid() until very recently when it became possible to implement it with POSIX semantics in go1.16.
Note, glibc, if you call setuid(), wraps the syscall itself with a fix-up mechanism (glibc/nptl/setxid) and will change the UID values for all the threads in the program simultaneously. So, even in C, you will have to do some hacking to work around this detail.]
That being said, you can make goroutines work the way you want with the runtime.LockOSThread() call, but not confuse the Go runtime by discarding the locked threads immediately after each specialized use.
Something like this (call it uidserve.go):
// Program uidserve serves content as different uids. This is adapted
// from the https://golang.org/pkg/net/http/#ListenAndServe example.
package main
import (
"fmt"
"log"
"net/http"
"runtime"
"syscall"
)
// Simple username to uid mapping.
var prefixUIDs = map[string]uintptr{
"apple": 100,
"banana": 101,
"cherry": 102,
}
type uidRunner struct {
uid uintptr
}
func (u *uidRunner) ServeHTTP(w http.ResponseWriter, r *http.Request) {
runtime.LockOSThread()
// Note, we never runtime.UnlockOSThread().
if _, _, e := syscall.RawSyscall(syscall.SYS_SETUID, u.uid, 0, 0); e != 0 {
http.Error(w, "permission problem", http.StatusInternalServerError)
return
}
fmt.Fprintf(w, "query %q executing as UID=%d\n", r.URL.Path, syscall.Getuid())
}
func main() {
for u, uid := range prefixUIDs {
h := &uidRunner{uid: uid}
http.Handle(fmt.Sprint("/", u, "/"), h)
}
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "general query %q executing as UID=%d\n", r.URL.Path, syscall.Getuid())
})
log.Fatal(http.ListenAndServe(":8080", nil))
}
Build it like this:
$ go build uidserve.go
Next, to get this to work, you have to grant this program some privilege. That is do one or the other of (setcap is a tool from the libcap suite):
$ sudo /sbin/setcap cap_setuid=ep ./uidserve
or, alternatively, the more traditional way of running setuid-root:
$ sudo chown root ./uidserve
$ sudo chmod +s ./uidserve
Now, if you run ./uidserve and connect to your browser to localhost:8080 you can try fetching the following URLs:
localhost:8080/something which shows something like general query "/something" executing as UID=your UID here.
localhost:8080/apple/pie which shows something like query "/apple/pie" executing as UID=100.
etc.
Hope that helps show how to do what you asked. [Since it involves lots of hacks, however, I wouldn't recommend doing this for real though...]
I'm facing with a weird golang issue. The following code will clarify:
package main
import (
"os/exec"
"io"
"fmt"
"os"
)
var (
pw io.WriteCloser
pr io.ReadCloser
)
func main() {
term := exec.Command("/bin/sh")
// Get stdin writer pipe
pw, _ = term.StdinPipe()
pr, _ = term.StdoutPipe()
term.Start()
run("cd ~")
pwd := run("pwd");
// Do something with pwd output
...
term.Wait()
}
func run(c string) string {
io.WriteString(pw, fmt.Sprintln(c))
buf := make([]byte, 32 * 1024)
pr.Read(buf)
return string(buf)
}
I'd like to run some commands in a shell env and read their output. There's no problem on write/run command but it seems that there're some limitations while reading:
you can't know if a command doesn't output anything or not;
there's no way to check if stdout is ready to be read or not.
The pr.Read(dest) method will block the code flow until something is read from stdout. As said, the goal is to read sequentially (without using a go routine and/or an infinite loop). This means that if we send a cd command the func end is never reached.
Setting the non-block flag through unix.SetNonblock on stdout file descriptor seems to solve the above issue but you can't know prior if it's ready or not and an error saying "resource temporary not available" is returned from .Read call.
As Cerise Limón mentioned go functions whould be the way to go here, since these sorts of interactive scripting exercises are traditionally done with expect.
You can wrap the the parrellel execution into a library to it might still look like sequencial code, so this might be helpful: https://github.com/ThomasRooney/gexpect
From the readme:
child, err := gexpect.Spawn("python")
if err != nil { panic(err) }
child.Expect(">>>")
child.SendLine("print 'Hello World'")
child.Interact()
child.Close()
I'm working with a new go service and I have a SetupLogger utility function that creates a new instance of go-kit's logging struct log.Logger.
Is this method safe to invoke from code that's handling requests inside separate go-routines?
package utils
import (
"fmt"
"github.com/go-kit/kit/log"
"io"
"os"
"path/filepath"
)
// If the environment-specified directory for writing log files exists, open the existing log file
// if it already exists or create a log file if no log file exists.
// If the environment-specified directory for writing log files does not exist, configure the logger
// to log to process stdout.
// Returns an instance of go-kit logger
func SetupLogger() log.Logger {
var logWriter io.Writer
var err error
LOG_FILE_DIR := os.Getenv("CRAFT_API_LOG_FILE_DIR")
LOG_FILE_NAME := os.Getenv("CRAFT_API_LOG_FILE_NAME")
fullLogFilePath := filepath.Join(
LOG_FILE_DIR,
LOG_FILE_NAME,
)
if dirExists, _ := Exists(&ExistsOsCheckerStruct{}, LOG_FILE_DIR); dirExists {
if logFileExists, _ := Exists(&ExistsOsCheckerStruct{}, fullLogFilePath); !logFileExists {
os.Create(fullLogFilePath)
}
logWriter, err = os.OpenFile(fullLogFilePath, os.O_RDWR|os.O_CREATE|os.O_APPEND, 0666)
if err != nil {
fmt.Println("Could not open log file. ", err)
}
} else {
logWriter = os.Stdout
}
return log.NewContext(log.NewJSONLogger(logWriter)).With(
"timestamp", log.DefaultTimestampUTC,
"caller", log.DefaultCaller,
)
}
First recommendation: Use the -race flag to go build and go test. It will almost always be able to tell you if you have a race condition. Although it might not in this case since you could end up calling your os.Create() and your os.OpenFile() simultaneously.
So, second recommendation is to avoid, if at all possible, the "If it exists/matches/has permissions Then open/delete/whatever" pattern.
That pattern leads to the TOCTTOU (Time of Check To Time Of Use) bug, which is often a security bug and can at the very least lead to data loss.
To avoid it either wrap the check and use into the same mutex, or use atomic operations, such as an OpenFile call that creates the file or returns an error if it already existed (although to be technical, its locked inside the OS kernel. Just like how atomic CPU ops are locked in the hardware bus.).
In your case here I am not quite sure why you have two Open calls since it looks like just one would do the job.
Since your setting up of your Logger only involves library instantiation, creating a log file if it doesn't exist, opening the log file and no writing involved there will be no problem calling it from different go-routines since the shared data is not getting tampered with.
Side note:
Design wise it makes sense (assuming Logger is writing to the same file) to pass around the only instantiated instance of Logger for logging which would prevent two go-routines calling your setup function at the same time.
I have an application that usually runs silent in the background, so I compile it with
go build -ldflags -H=windowsgui <gofile>
To check the version at the command line, I wanted to pass a -V flag to the command line to get the string holding the version to be printed to the command prompt then have the application exit. I added the flag package and code. When I test it with
go run <gofile> -V
...it prints the version fine. When I compile the exe, it just exits, printing nothing. I suspect it's the compilation flag causing it to not access the console and sending my text into the bit bucket.
I've tried variations to print to stderr and stdout, using println and fprintf and os.stderr.write, but nothing appears from the compiled application. How should I try printing a string to the command prompt when compiled with those flags?
The problem is that when a process is created using an executable which has the "subsystem" variable in its PE header set to "Windows", the process has its three standard handles closed and it is not associated with any console—no matter if you run it from the console or not. (In fact, if you run an executable which has its subsystem set to "console" not from a console, a console is forcibly created for that process and the process is attached to it—you usually see it as a console window popping up all of a sudden.)
Hence, to print anything to the console from a GUI process on Windows you have to explicitly connect that process to the console which is attached to its parent process (if it has one), like explained here for instance. To do this, you call the AttachConsole API function. With Go, this can be done using the syscall package:
package main
import (
"fmt"
"syscall"
)
const (
ATTACH_PARENT_PROCESS = ^uint32(0) // (DWORD)-1
)
var (
modkernel32 = syscall.NewLazyDLL("kernel32.dll")
procAttachConsole = modkernel32.NewProc("AttachConsole")
)
func AttachConsole(dwParentProcess uint32) (ok bool) {
r0, _, _ := syscall.Syscall(procAttachConsole.Addr(), 1, uintptr(dwParentProcess), 0, 0)
ok = bool(r0 != 0)
return
}
func main() {
ok := AttachConsole(ATTACH_PARENT_PROCESS)
if ok {
fmt.Println("Okay, attached")
}
}
To be truly complete, when AttachConsole() fails, this code should probably take one of these two routes:
Call AllocConsole() to get its own console window created for it.
It'd say this is pretty much useless for displaying version information as the process usually quits after printing it, and the resulting user experience will be a console window popping up and immediately disappearing; power users will get a hint that they should re-run the application from the console but mere mortals won't probably cope.
Post a GUI dialog displaying the same information.
I think this is just what's needed: note that displaying help/usage messages in response to the user specifying some command-line argument is quite often mentally associated with the console, but this is not a dogma to follow: for instance, try running msiexec.exe /? at the console and see what happens.
One problem with the solutions already posted here is that they redirect all output to the console, so if I run ./myprogram >file, the redirection to file gets lost. I've written a new module, github.com/apenwarr/fixconsole, that avoids this problem. You can use it like this:
import (
"fmt"
"github.com/apenwarr/fixconsole"
"os"
)
func main() {
err := fixconsole.FixConsoleIfNeeded()
if err != nil {
fmt.Fatalf("FixConsoleOutput: %v\n", err)
}
os.Stdout.WriteString(fmt.Sprintf("Hello stdout\n"))
os.Stderr.WriteString(fmt.Sprintf("Hello stderr\n"))
}
Answer above was helpful but alas it did not work for me out of the box. After some additional research I came to this code:
// go build -ldflags -H=windowsgui
package main
import "fmt"
import "os"
import "syscall"
func main() {
modkernel32 := syscall.NewLazyDLL("kernel32.dll")
procAllocConsole := modkernel32.NewProc("AllocConsole")
r0, r1, err0 := syscall.Syscall(procAllocConsole.Addr(), 0, 0, 0, 0)
if r0 == 0 { // Allocation failed, probably process already has a console
fmt.Printf("Could not allocate console: %s. Check build flags..", err0)
os.Exit(1)
}
hout, err1 := syscall.GetStdHandle(syscall.STD_OUTPUT_HANDLE)
hin, err2 := syscall.GetStdHandle(syscall.STD_INPUT_HANDLE)
if err1 != nil || err2 != nil { // nowhere to print the error
os.Exit(2)
}
os.Stdout = os.NewFile(uintptr(hout), "/dev/stdout")
os.Stdin = os.NewFile(uintptr(hin), "/dev/stdin")
fmt.Printf("Hello!\nResult of console allocation: ")
fmt.Printf("r0=%d,r1=%d,err=%s\nFor Goodbye press Enter..", r0, r1, err0)
var s string
fmt.Scanln(&s)
os.Exit(0)
}
The key point: after allocating/attaching the console, there is need to get stdout handle, open file using this handle and assign it to os.Stdout variable. If you need stdin you have to repeat the same for stdin.
You can get the desired behavior without using -H=windowsgui; you'd basically create a standard app (with its own console window), and hide it until the program exits.
func Console(show bool) {
var getWin = syscall.NewLazyDLL("kernel32.dll").NewProc("GetConsoleWindow")
var showWin = syscall.NewLazyDLL("user32.dll").NewProc("ShowWindow")
hwnd, _, _ := getWin.Call()
if hwnd == 0 {
return
}
if show {
var SW_RESTORE uintptr = 9
showWin.Call(hwnd, SW_RESTORE)
} else {
var SW_HIDE uintptr = 0
showWin.Call(hwnd, SW_HIDE)
}
}
And then use it like this:
func main() {
Console(false)
defer Console(true)
...
fmt.Println("Hello World")
...
}
If you build a windowless app you can get output with PowerShell command Out-String
.\\main.exe | out-string
your build command may look like:
cls; go build -i -ldflags -H=windowsgui main.go; .\\main.exe | out-string;
or
cls; go run -ldflags -H=windowsgui main.go | out-string
No tricky syscalls nor kernel DLLs needed!