How to gently defer execution of a function that might return an error? - go

Most cleanup functions, especially those related to the IO operations, return an error, and normally we'd prefer to defer their execution in case if we'd not forget to call them when we're done with acquired resources. For example, at some point in the code we might write something like this:
var r *SomeResource
var err error
if r, err = Open(/* parameters */); err != nil {
return nil, err
}
defer r.Close() // This might return an error
It seems that if Close function returns an error, it'll be ignored. How can we gently process the returned error from such a function?

Using defer with a func() {}() like so.
var r *SomeResource
var err error
if r, err = Open(/* parameters */); err != nil {
return nil, err
}
defer func() {
if err = r.Close(); err != nil {
fmt.Printf("ERROR: %v", err)
}
}()

Fail gracefully with an error. Report the first error. Don't overwrite earlier errors. For example,
package main
import (
"fmt"
"os"
)
func demo() (name string, err error) {
filename := `test.file`
f, err := os.Open(filename)
if err != nil {
return "", err
}
defer func() {
e := f.Close()
if e != nil {
if err == nil {
err = e
}
}
}()
// do someting with the file
name = f.Name()
fi, err := f.Stat()
if err != nil {
return name, err
}
if fi.Size() == 0 {
err = fmt.Errorf("%s: empty file", filename)
return name, err
}
return name, err
}
func main() {
name, err := demo()
fmt.Println(name, err)
}

We can handle this in ways like:
way-1:
func myFn() error {
var err error
if r, err = Open(/* parameters */); err != nil {
return nil, err
}
defer func() {
if cErr = r.Close(); cErr != nil {
err = cErr
}
}()
return err
}
way-2:
func myFn() error {
var err error
if r, err = Open(/* parameters */); err != nil {
return nil, err
}
defer func() {
if cErr = r.Close(); cErr != nil {
// we can log the error
// or
// whatever we want to do
}
}()
return err
}
I have also find a nice blog on this topic, i mean handling error when defer func returns an error. Check here https://blog.learngoprogramming.com/5-gotchas-of-defer-in-go-golang-part-iii-36a1ab3d6ef1.

Related

Create function that receives any function with specific amount of parameters

Say I have several different gRPC servers, for example x.Server, y.Server and z.Server, and in order to spin them up, I have a lot of repeated code inside their main function, e.g.:
func main() {
if err := config.EnsureArgLength(1); err != nil {
log.Fatalln(err.Error())
}
srv := &x.Server{}
if err := srv.ReadServerConfig(os.Args[1]); err != nil {
log.Fatalln(err.Error())
}
if err := srv.RegisterListener(); err != nil {
log.Fatalln(err.Error())
}
if err := srv.RegisterClients(); err != nil {
log.Fatalln(err.Error())
}
s := grpc.NewServer()
proto.RegisterXServer(s, srv)
if err := srv.Serve(s); err != nil {
log.Fatalf("failed to serve: %s", err.Error())
}
}
I would love to refactor this main function to make it one or two lines long, something like the following:
func main() {
srv := x.Server{}
if err := srv.RegisterAndServe(); err != nil {
log.Fatal("failed to serve: %s", err.Error())
}
}
But each server will have an auto-generated function proto.RegisterXServer which is not part of x.Server struct, and I'm also not able to modify the file which contains it, since it is auto generated. How should I proceed?
in regards to op changes, which was radical,
I can suggest using a reducer pattern like this.
package main
import (
"fmt"
)
func main() {
fail(reduce(sayHello(), sayGoodbye))
}
func sayHello() func() error {
return func() error { fmt.Println("Hello, playground"); return nil }
}
func sayGoodbye() error {
fmt.Println("Goodbye from the playground")
return nil
}
func reduce(h ...func() error) error {
for _, hh := range h {
if err := hh(); err != nil {
return err
}
}
return nil
}
func fail(err error) {
if err != nil {
panic(err)
}
}

Having issues with multipart.NewWriter using io.pipe

I'm having issues with requests being sent randomly empty. It doesn't always happen, but sometimes out of the blue, it will not send any of the multipart fields. I thought it might have to do with the upload server, so I created a local upload server to print out the request that's being sent, and it comes out empty.
I have added error checks everywhere, but no errors are being returned.
I tried to run the code with -race, but no race condition has been reported.
Edit: Update the code to use CloseWithError()
package main
import (
"fmt"
"io"
"log"
"mime/multipart"
"net/http"
"os"
)
var (
upload_url string = "https://upload.imagekit.io/api/v1/files/upload"
file_name string = "favicon-516140983.ico"
api_secret_key string = "PRIVATE_KEY"
)
func UploadMultipartFile(client *http.Client, uri, key, path string) (*http.Response, error) {
body, writer := io.Pipe()
req, err := http.NewRequest(http.MethodPost, uri, body)
if err != nil {
log.Println(err)
return nil, err
}
mwriter := multipart.NewWriter(writer)
req.Header.Add("Content-Type", mwriter.FormDataContentType())
req.SetBasicAuth(api_secret_key, "")
go func() {
var err error
defer func() {
if err != nil {
writer.CloseWithError(err)
} else {
writer.Close()
}
}()
var file *os.File
file, err = os.Open(path)
if err != nil {
return
}
defer file.Close()
if err = mwriter.WriteField("fileName", file_name); err != nil {
return
}
var w io.Writer
w, err = mwriter.CreateFormFile("file", path)
if err != nil {
return
}
var written int64
if written, err = io.Copy(w, file); err != nil {
err = fmt.Errorf("error copying %s (%d bytes written): %v", path, written, err)
return
}
if err = mwriter.Close(); err != nil {
return
}
}()
resp, err := client.Do(req)
if err != nil {
return nil, err
}
return resp, nil
}
func main() {
path, _ := os.Getwd()
path += "/" + file_name
client := &http.Client{}
resp, err := UploadMultipartFile(client, upload_url, "file", path)
if err != nil {
log.Println(err)
} else {
fmt.Println(resp.StatusCode)
fmt.Println(resp.Header)
_, err := io.Copy(os.Stdout, resp.Body)
if err != nil {
log.Fatal(err)
}
resp.Body.Close()
}
}

Copy a folder in go

Is there an easy way to copy a directory in go?
I have the following function:
err = CopyDir("sourceFolder","destinationFolder")
Nothing so far has worked, including libraries such as github.com/cf-guardian/guardian/kernel/fileutils
One important thing to note is that I need to preserve directory structure, including the sourceFolder itself, not simply copy all contents of the folder.
I believe that docker implementation can be considered as complete solution for handling edge cases:
https://github.com/moby/moby/blob/master/daemon/graphdriver/copy/copy.go
There are following good things:
unsupported file type rise error
preserving permissions and ownership
preserving extended attributes
preserving timestamp
but because of a lot of imports your tiny application becomes huge.
I've tried to combine several solutions but use stdlib and for Linux only:
func CopyDirectory(scrDir, dest string) error {
entries, err := os.ReadDir(scrDir)
if err != nil {
return err
}
for _, entry := range entries {
sourcePath := filepath.Join(scrDir, entry.Name())
destPath := filepath.Join(dest, entry.Name())
fileInfo, err := os.Stat(sourcePath)
if err != nil {
return err
}
stat, ok := fileInfo.Sys().(*syscall.Stat_t)
if !ok {
return fmt.Errorf("failed to get raw syscall.Stat_t data for '%s'", sourcePath)
}
switch fileInfo.Mode() & os.ModeType{
case os.ModeDir:
if err := CreateIfNotExists(destPath, 0755); err != nil {
return err
}
if err := CopyDirectory(sourcePath, destPath); err != nil {
return err
}
case os.ModeSymlink:
if err := CopySymLink(sourcePath, destPath); err != nil {
return err
}
default:
if err := Copy(sourcePath, destPath); err != nil {
return err
}
}
if err := os.Lchown(destPath, int(stat.Uid), int(stat.Gid)); err != nil {
return err
}
fInfo, err := entry.Info()
if err != nil {
return err
}
isSymlink := fInfo.Mode()&os.ModeSymlink != 0
if !isSymlink {
if err := os.Chmod(destPath, fInfo.Mode()); err != nil {
return err
}
}
}
return nil
}
func Copy(srcFile, dstFile string) error {
out, err := os.Create(dstFile)
if err != nil {
return err
}
defer out.Close()
in, err := os.Open(srcFile)
defer in.Close()
if err != nil {
return err
}
_, err = io.Copy(out, in)
if err != nil {
return err
}
return nil
}
func Exists(filePath string) bool {
if _, err := os.Stat(filePath); os.IsNotExist(err) {
return false
}
return true
}
func CreateIfNotExists(dir string, perm os.FileMode) error {
if Exists(dir) {
return nil
}
if err := os.MkdirAll(dir, perm); err != nil {
return fmt.Errorf("failed to create directory: '%s', error: '%s'", dir, err.Error())
}
return nil
}
func CopySymLink(source, dest string) error {
link, err := os.Readlink(source)
if err != nil {
return err
}
return os.Symlink(link, dest)
}
This package seems to do exactly what you want to do, give it a try.
From the readme:
err := Copy("your/source/directory", "your/destination/directory")
Not satisfied with the already listed options which include using sketchy libraries, or vastly bloated libraries.
In my case, I opted to do things the old fashioned way. With shell commands!
import (
"os/exec"
)
func main() {
// completely arbitrary paths
oldDir := "/home/arshbot/"
newDir := "/tmp/"
cmd := exec.Command("cp", "--recursive", oldDir, newDir)
cmd.Run()
}
This solution copies a directory recursively, including symbolic links. Trying to be efficient in the actual copy stage using streams.
Also it's fairly easy to handle more of irregular files if needed.
// CopyDir copies the content of src to dst. src should be a full path.
func CopyDir(dst, src string) error {
return filepath.Walk(src, func(path string, info fs.FileInfo, err error) error {
if err != nil {
return err
}
// copy to this path
outpath := filepath.Join(dst, strings.TrimPrefix(path, src))
if info.IsDir() {
os.MkdirAll(outpath, info.Mode())
return nil // means recursive
}
// handle irregular files
if !info.Mode().IsRegular() {
switch info.Mode().Type() & os.ModeType {
case os.ModeSymlink:
link, err := os.Readlink(path)
if err != nil {
return err
}
return os.Symlink(link, outpath)
}
return nil
}
// copy contents of regular file efficiently
// open input
in, _ := os.Open(path)
if err != nil {
return err
}
defer in.Close()
// create output
fh, err := os.Create(outpath)
if err != nil {
return err
}
defer fh.Close()
// make it the same
fh.Chmod(info.Mode())
// copy content
_, err = io.Copy(fh, in)
return err
})
}
I've come up with a relatively shorter answer which uses path/filepath's Walk method:
import (
"io/ioutil"
"path/filepath"
"os"
"strings"
)
func copy(source, destination string) error {
var err error = filepath.Walk(source, func(path string, info os.FileInfo, err error) error {
var relPath string = strings.Replace(path, source, "", 1)
if relPath == "" {
return nil
}
if info.IsDir() {
return os.Mkdir(filepath.Join(destination, relPath), 0755)
} else {
var data, err1 = ioutil.ReadFile(filepath.Join(source, relPath))
if err1 != nil {
return err1
}
return ioutil.WriteFile(filepath.Join(destination, relPath), data, 0777)
}
})
return err
}
Also this might be a solution:
available on github.com/floscodes/golang-tools
import (
"fmt"
"io/ioutil"
"os"
)
func CopyDir(src string, dest string) error {
if dest[:len(src)] == src {
return fmt.Errorf("Cannot copy a folder into the folder itself!")
}
f, err := os.Open(src)
if err != nil {
return err
}
file, err := f.Stat()
if err != nil {
return err
}
if !file.IsDir() {
return fmt.Errorf("Source " + file.Name() + " is not a directory!")
}
err = os.Mkdir(dest, 0755)
if err != nil {
return err
}
files, err := ioutil.ReadDir(src)
if err != nil {
return err
}
for _, f := range files {
if f.IsDir() {
err = CopyDir(src+"/"+f.Name(), dest+"/"+f.Name())
if err != nil {
return err
}
}
if !f.IsDir() {
content, err := ioutil.ReadFile(src + "/" + f.Name())
if err != nil {
return err
}
err = ioutil.WriteFile(dest+"/"+f.Name(), content, 0755)
if err != nil {
return err
}
}
}
return nil
}

Create a service as autorun

OS: windows/7/8/8.1/10 32bit
I have one question. How to create a service that would work like autorun?
Most applications install themselves in autorun through the registry or through C:\Users\Anon\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup. But there are those that are installing through the services, or rather as a service.
I have a code:
package main
import (
"fmt"
"strings"
"time"
"syscall"
"golang.org/x/sys/windows/svc"
"golang.org/x/sys/windows/svc/mgr"
"golang.org/x/sys/windows/svc/debug"
"log"
"os"
"path/filepath"
"golang.org/x/sys/windows/svc/eventlog"
)
var elog debug.Log
type myservice struct{}
func (m *myservice) Execute(args []string, r <-chan svc.ChangeRequest, changes chan<- svc.Status) (ssec bool, errno uint32) {
const cmdsAccepted = svc.AcceptStop | svc.AcceptShutdown | svc.AcceptPauseAndContinue
changes <- svc.Status{State: svc.StartPending}
fasttick := time.Tick(500 * time.Millisecond)
slowtick := time.Tick(2 * time.Second)
tick := fasttick
changes <- svc.Status{State: svc.Running, Accepts: cmdsAccepted}
elog.Info(1, strings.Join(args, "-"))
loop:
for {
select {
case <-tick:
beep()
elog.Info(1, "beep")
case c := <-r:
switch c.Cmd {
case svc.Interrogate:
changes <- c.CurrentStatus
// Testing deadlock from https://code.google.com/p/winsvc/issues/detail?id=4
time.Sleep(100 * time.Millisecond)
changes <- c.CurrentStatus
case svc.Stop, svc.Shutdown:
break loop
case svc.Pause:
changes <- svc.Status{State: svc.Paused, Accepts: cmdsAccepted}
tick = slowtick
case svc.Continue:
changes <- svc.Status{State: svc.Running, Accepts: cmdsAccepted}
tick = fasttick
default:
elog.Error(1, fmt.Sprintf("unexpected control request #%d", c))
}
}
}
changes <- svc.Status{State: svc.StopPending}
return
}
func runService(name string, isDebug bool) {
var err error
if isDebug {
elog = debug.New(name)
} else {
elog, err = eventlog.Open(name)
if err != nil {
return
}
}
defer elog.Close()
elog.Info(1, fmt.Sprintf("starting %s service", name))
run := svc.Run
if isDebug {
run = debug.Run
}
err = run(name, &myservice{})
if err != nil {
elog.Error(1, fmt.Sprintf("%s service failed: %v", name, err))
return
}
elog.Info(1, fmt.Sprintf("%s service stopped", name))
}
func startService(name string) error {
m, err := mgr.Connect()
if err != nil {
return err
}
defer m.Disconnect()
s, err := m.OpenService(name)
if err != nil {
return fmt.Errorf("could not access service: %v", err)
}
defer s.Close()
err = s.Start("is", "auto-started")
if err != nil {
return fmt.Errorf("could not start service: %v", err)
}
return nil
}
func controlService(name string, c svc.Cmd, to svc.State) error {
m, err := mgr.Connect()
if err != nil {
return err
}
defer m.Disconnect()
s, err := m.OpenService(name)
if err != nil {
return fmt.Errorf("could not access service: %v", err)
}
defer s.Close()
status, err := s.Control(c)
if err != nil {
return fmt.Errorf("could not send control=%d: %v", c, err)
}
timeout := time.Now().Add(10 * time.Second)
for status.State != to {
if timeout.Before(time.Now()) {
return fmt.Errorf("timeout waiting for service to go to state=%d", to)
}
time.Sleep(300 * time.Millisecond)
status, err = s.Query()
if err != nil {
return fmt.Errorf("could not retrieve service status: %v", err)
}
}
return nil
}
func main() {
const svcName = "Best Service"
isIntSess, err := svc.IsAnInteractiveSession()
if err != nil {
log.Fatalf("failed to determine if we are running in an interactive session: %v", err)
}
if !isIntSess {
runService(svcName, false)
return
}
/*err = controlService(svcName, svc.Stop, svc.Stopped)
err = removeService(svcName)*/
err = installService(svcName, "Best Service")
runService(svcName, true)
if err != nil {
log.Fatalf("failed to %s: %v", svcName, err)
}
return
}
func exePath() (string, error) {
prog := os.Args[0]
p, err := filepath.Abs(prog)
if err != nil {
return "", err
}
fi, err := os.Stat(p)
if err == nil {
if !fi.Mode().IsDir() {
return p, nil
}
err = fmt.Errorf("%s is directory", p)
}
if filepath.Ext(p) == "" {
p += ".exe"
fi, err := os.Stat(p)
if err == nil {
if !fi.Mode().IsDir() {
return p, nil
}
err = fmt.Errorf("%s is directory", p)
}
}
return "", err
}
func installService(name, desc string) error {
exepath, err := exePath()
if err != nil {
return err
}
m, err := mgr.Connect()
if err != nil {
return err
}
defer m.Disconnect()
s, err := m.OpenService(name)
if err == nil {
s.Close()
return fmt.Errorf("service %s already exists", name)
}
s, err = m.CreateService(name, exepath, mgr.Config{DisplayName: desc, Description: "BB service"}, "is", "auto-started")
if err != nil {
return err
}
defer s.Close()
err = eventlog.InstallAsEventCreate(name, eventlog.Error|eventlog.Warning|eventlog.Info)
if err != nil {
s.Delete()
return fmt.Errorf("SetupEventLogSource() failed: %s", err)
}
return nil
}
func removeService(name string) error {
m, err := mgr.Connect()
if err != nil {
return err
}
defer m.Disconnect()
s, err := m.OpenService(name)
if err != nil {
return fmt.Errorf("service %s is not installed", name)
}
defer s.Close()
err = s.Delete()
if err != nil {
return err
}
err = eventlog.Remove(name)
if err != nil {
return fmt.Errorf("RemoveEventLogSource() failed: %s", err)
}
return nil
}
var (
beepFunc = syscall.MustLoadDLL("user32.dll").MustFindProc("MessageBeep")
)
func beep() {
beepFunc.Call(0xffffffff)
}
Application is installed and every time I exit the application the service stops. I need that even after restarting the PC the service worked and the application started. How can I do it?
maybe it's not actual but during the creation of the service you should extend and pass Config
mgr.Config{DisplayName: desc, StartType: mgr.StartAutomatic}
like here:
s, err = m.CreateService(name, exepath, mgr.Config{DisplayName: desc, StartType: mgr.StartAutomatic}, "is", "auto-started")
if err != nil {
return err
}
Here you can find all necessary constants and functions:
https://github.com/golang/sys/blob/master/windows/svc/mgr/config.go
On Windows 10 go to Task Scheduler > Task Scheduler Library > Create Basic Task > Trigger: when the computer starts > Action: Start a program.
When running the task, use the following user account: SYSTEM.
There is a package in the standard library that does this:
https://godoc.org/golang.org/x/sys/windows/svc
example: https://github.com/golang/sys/tree/master/windows/svc/example

Golang streams and readers

I am writing a simple script to get download unzip the tar.gz file and then remove it. Whenever I try to remove it I get an error:
The process cannot access the file because it is being used by another process.
I assume the error is in how I pass the file to the extractTarGz function, but I am not sure.
Here is the code:
package main
import (
"archive/tar"
"compress/gzip"
"io"
"log"
"os"
)
func main() {
f, err := os.Open("file.tar.gz")
if err != nil {
panic(err)
}
defer f.Close()
extractTarGz(f)
err = os.Remove("file.tar.gz")
}
func extractTarGz(gzipStream io.Reader) {
uncompressedStream, err := gzip.NewReader(gzipStream)
if err != nil {
log.Fatal("ExtractTarGz: NewReader failed")
}
tarReader := tar.NewReader(uncompressedStream)
for true {
header, err := tarReader.Next()
if err == io.EOF {
break
}
if err != nil {
log.Fatalf("ExtractTarGz: Next() failed: %s", err.Error())
}
switch header.Typeflag {
case tar.TypeDir:
if err := os.Mkdir(header.Name, 0755); err != nil {
log.Fatalf("ExtractTarGz: Mkdir() failed: %s", err.Error())
}
case tar.TypeReg:
outFile, err := os.Create(header.Name)
if err != nil {
log.Fatalf("ExtractTarGz: Create() failed: %s", err.Error())
}
defer outFile.Close()
if _, err := io.Copy(outFile, tarReader); err != nil {
log.Fatalf("ExtractTarGz: Copy() failed: %s", err.Error())
}
default:
log.Fatalf(
"ExtractTarGz: uknown type: %s in %s",
header.Typeflag,
header.Name)
}
}
}
You should first close the file, and then attempt to remove it. Since you close it using defer, that will / would be called after the os.Remove() call.
Try it like this:
name := "file.tar.gz"
defer func() {
if err = os.Remove(name); err != nil {
log.Printf("Failed to remove %s: %v", name, err)
}
}()
f, err := os.Open(name)
if err != nil {
panic(err)
}
defer f.Close()
extractTarGz(f)
Deferred functions are executed in LIFO (last-in-first-out) order, so first f.Close() will be called, and then the other which tries to remove the file. Quoting from Spec: Deferred statements:
...deferred functions are invoked immediately before the surrounding function returns, in the reverse order they were deferred.
f, err := os.Open("file.tar.gz")
if err != nil {
panic(err)
}
defer f.Close()
extractTarGz(f)
err = os.Remove("file.tar.gz")
At the very least, you need to close the file before you removeit.
err = f.Close()
if err != nil {
panic(err)
}
err = os.Remove("file.tar.gz")
defer f.Close() won't run until the end of the function.

Resources