I have a cli that I am making that is more for learning purposes and creating my own cli, that does stuff. Anyways, I am testing the delete function and it works fine and gives me the right answer. However, I don't believe that it is the best practice and was wondering if you could let me know if it's ok, or not.
Test file
func TestDeleteConfig(t *testing.T) {
err := cm.DeleteConfig()
if err != nil {
t.Errorf("error when deleting the folder: %s", err)
}
usr, err := user.Current()
if err != nil {
t.Errorf("error when getting user current: %s", err)
}
fp := filepath.Join(usr.HomeDir, ".config", "godot", "config.json")
fmt.Println("the path of the config file", fp)
if _, e := os.Stat(fp); !os.IsNotExist(e) {
t.Errorf("error path still exists: %v", e)
}
}
function being tested
func DeleteConfig() error {
usr, err := user.Current()
if err != nil {
return err
}
err = os.RemoveAll(filepath.Join(usr.HomeDir, ".config", "godot"))
if err != nil {
return err
}
return nil
}
The problem is that I don't want the DeleteConfig() to take any arguments as this is a hard path. I have a separate function for deleting a single file in which the help with this will solve that as well func DeleteFile(p string) error {}.
So for testing purposes should I just create a foo directory at a separate path (within the test), delete said directory, and assume that if it works on foo path then it should work with the godot directory?
It's a bit philosophical.
If you absolutely do not want to stub/mock any part of your code to replace accessing a real file system to do testing, then we're talking about what you'd call system or integration testing. That's fine in itself: for instance, you could run such tests in a throw-away container as part of a CI pipeline.
But with this approach you can't sensibly do what is called unit-testing.
To unit-test your function you need to replace something it uses with something "virtualized". Exactly how to do that is an open question.
Approach: make is possible to override os.RemoveAll
For instance, you can have a private global variable containing the "remove whole directory" function—like this:
var removeAll = os.RemoveAll
func DeleteConfig() error {
usr, err := user.Current()
if err != nil {
return err
}
err = removeAll(filepath.Join(usr.HomeDir, ".config", "godot"))
if err != nil {
return err
}
return nil
}
Then to test it you'd just monkey-patch the function with your own implementation which would have the same signature as os.RemoveAll, like this:
func TestDeleteConfig(t *testing.T) {
var actualPath string
removeAll = func(path string) {
actualPath = path
return nil
}
defer func() { removeAll = os.RemoveAll }()
DeleteConfig()
if actualPath != expectedPath {
t.Errorf("unexpected path: want %s, got %s", expectedPath, actualPath)
}
}
This approach also allows to test how your function handles errors: just replace it with something which generates an error and then afterwards check that your function under test returned that error (or wrapped it in an expected way, whatever).
Still, it's a bit low-tech for a number of reasons:
A global variable is involved, so you have to be sure no two tests which monkey-patch it run concurrently, or that all patching is done before running those tests.
If different tests need to set it to different value, they must be serialized.
Approach: do not hard-code the concept of "the user" or "the config"
Another approach is to notice that basically the problem with testing stems from the fact you hard-coded getting of the user.
Leaving aside the flawed approach you've taken to getting the place of configuration (you should be using something which implements the XDG spec), if you could easily override getting of the "root" directory (which is the user's home directory in your code), you could easily target your function to operate on the result of calling io/ioutil.TempDir.
So may be a way to go is to have an interface type like
type ConfigStore interface {
Dir() string
}
of which the Dir() method is supposed to return the path to the configuration store's root directory.
Your DeleteConfig() would then start to accept a single argument of type ConfigStore, and in your program you'd have a concrete implementation of it, and in your testing code — a stub implementing the same interface and managing a temporary directory.
Approach: go full-on virtualized
Right now, a work is being done on bringing filesystem virtualization right into the Go standard library, but while it's not there yet, 3rd-party packages which do that exist for ages, — for instance, github.com/spf13/afero.
Basically, they allow you to not use os directly but write all your code in a way so that instead of the os package it calls methods on an instance of a type implementing particular interface: in the production code that object is a thin shim for the os package, and in the testing code it's replaced by whatever you wish; afero has a readily-available in-memory FS backend to do this.
Writing a unit test for filesystem checking is not trivial. You should NOT create a real file on the system, because then your test will depend on the I/O of the file system itself. The last resort is mocking the filesystem. There are quite a few powerful libraries like spf13/afero for this purpose (mocking of a filesystem). These packages will create temporary files in the background and clean up afterward.
main.go
package main
import (
"log"
"os/user"
"path/filepath"
iowrap "github.com/spf13/afero"
)
var (
// FS is simulated filesystem interface
FS iowrap.Fs
// FSUtil is the struct of the simulated interface
FSUtil *iowrap.Afero
)
func init() {
FS = iowrap.NewOsFs()
FSUtil = &iowrap.Afero{Fs: FS}
}
// DeleteConfig removes ~/.config/godot if exists
func DeleteConfig() error {
usr, err := user.Current()
if err != nil {
return err
}
path := filepath.Join(usr.HomeDir, ".config", "godot")
log.Println(path)
err = FSUtil.RemoveAll(path)
return err
}
func main() {
err := DeleteConfig()
if err != nil {
log.Fatal(err)
}
}
main_test.go
package main
import (
"os/user"
"path/filepath"
"testing"
iowrap "github.com/spf13/afero"
)
func init() {
FS = iowrap.NewMemMapFs()
FSUtil = &iowrap.Afero{Fs: FS}
usr, _ := user.Current()
pathDir := filepath.Join(usr.HomeDir, ".config")
filePath := filepath.Join(pathDir, "godot")
FS.MkdirAll(pathDir, 0755)
iowrap.WriteFile(FS, filePath, []byte("0-7\n"), 0644)
}
const (
succeed = "\u2713"
failed = "\u2717"
)
func TestDeleteConfig(t *testing.T) {
t.Log("Given the need to test downloading a webpage content")
{
usr, _ := user.Current()
pathDir := filepath.Join(usr.HomeDir, ".config")
filePath := filepath.Join(pathDir, "godot")
t.Logf("\tTest 0:\tWhen deleting the %v with 0644 permissions", filePath)
{
err := DeleteConfig()
if err != nil {
t.Fatalf("\t%s\tThe file couldn't be deleted: %v", failed, err)
}
t.Logf("\t%s\tThe file has been successfully deleted.", succeed)
}
}
}
Functional test:
touch C:/Users/drpan/.config/godot
ls -l C:/Users/drpan/.config/godot Output: -rw-r--r-- 1 drpan 197609 0 Nov 2 19:38 C:/Users/drpan/.config/godot
./deletedirectory.exe
ls -l C:/Users/drpan/.config/godot Output: ls: cannot access 'C:/Users/drpan/.config/godot': No such file or directory
Unit Test:
$ touch C:/Users/drpan/.config/godot
$ go test
2020/11/02 19:55:35 C:\Users\drpan\.config\godot
PASS
ok github.com/drpaneas/deletedirectory 0.162s
$ ls -l C:/Users/drpan/.config/godot Output: -rw-r--r-- 1 drpan 197609 0 Nov 2 19:55 C:/Users/drpan/.config/godot
Related
I used Lambda functions before, and if I remember correctly I'm supposed to have ~500Mb of (ephemeral) space in /tmp.
Nevertheless, my Go lambda function doesn't seem to interact with the fs properly:
exec.Command("ls -la /").Output() returns empty
exec.Command("rm -rf /tmp/xxx").Run() returns fork/exec : no such file or directory
exec.Command("mkdir -p /tmp/xxx").Run() returns fork/exec : no such file or directory
It's really weird.
It's using the go1.x environment (thus, I guess amazonlinux:2)
UPDATE
I CAN access the fs using Go os functions:
os.RemoveAll("/tmp/xxx")
if _, err := os.Stat("/tmp/xxx"); os.IsNotExist(err) {
if err := os.Mkdir("/tmp/xxx", os.ModePerm); err != nil {
return err
}
}
BUT I really need exec to run afterwards (a binary command), and write a file in that tmp folder. The error in that case is the same (no such file or directory). Even though I've just created the folder with the above commands.
You are close. The way you use exec.Command() is not yet 100% correct. Try the following:
package main
import (
"fmt"
"os"
"os/exec"
)
func main() {
o, err := exec.Command("ls", "-la", "/tmp").Output()
if err != nil {
fmt.Println(err)
os.Exit(1)
}
fmt.Printf("%s\n", o)
}
The first argument to Command() is the program you want to run and all the following arguments are the programs arguments.
See https://play.golang.org/p/WaVOU0IESmZ
I have a function which writes/updates a json. But I need to stop the executable, run go build again and re-run the executable to get the json updated in url.
For example, I have a Handler.go file which takes argument from URL as key and runs an if condition and updates the json. So If json value before building the executable is {"Name":"Sneha"} and i pass parameter "Nair" in the url, the json gets updated in the server as {"Name":"Nair"}, but doesnt get updated in the URL. So I have to stop the executable, run go build again and run the executable again to reflect the new json value {"Name":"Nair"} in the URL.
1. Can somebody please suggest an alternative idea ?
2. Can we run go build or go update inside a function?
Help much appreciated.
PS: I have got URL for goagain.go. But am not sure if that matches my requirement.
Handler.go
func handler(w http.ResponseWriter, r *http.Request) {
keys, ok := r.URL.Query()["key"]
if !ok || len(keys) < 1 {
log.Println("Url Param 'key' is missing")
return
}
key := keys[0]
log.Println("Url Param 'key' is: " + string(key))
if key == "java" {
commands := []string{
"Version=`java -version`",
"sed -i -e 's/^\\( *Name: *\\) .*$/ Name:\"Java\",/' Handler.go",
"sed -i -e 's/^\\( *Version: *\\) .*$/ Version:\" '$Version'\",/' Handler.go",
}
exe_cmd(commands)
}
if key == "go" {
commands := []string{
"Version=`go version`",
"sed -i -e 's/^\\( *Name: *\\) .*$/ Name:\"Go\",/' Handler.go",
"sed -i -e 's/^\\( *Version: *\\) .*$/ Version:\" '$Version'\",/' Handler.go",
}
exe_cmd(commands)
}
GetHands := GetHand{
Name:"java",
Version:" 1.7.0_71",
}
if err := json.NewEncoder(w).Encode(GetHands); err != nil {
panic(err)
}
So on running this package, the url shows json value : {"name":"java","version":" 1.7.0_71"}
If I call url : http://localhost:8080/?key=go this Handler.go gets updated to,
GetHands := GetHand{
Name:"go",
Version:" 1.9",
}
If I stop the executable,
run go build again and run executable again the url gets returned as :{"name":"go","version":" 1.9"}
So basically I need dynamic url which on hitting the http:/localhost/?key=go would return go's corresponding value annd htpp://localhost/?key=java would return java's corresponding value. This should be attained without restarting the executable or re-running the go build
Its quite difficult to understand exactly what you want. But I suspect that is essence you simply want to extract the output from a shell command and write it to JSON.
For this there is no need to modify the Handler.go file or do go build. You can simply write the output directly into the GetHand structure.
A basic example is as follows :
package main
import (
"fmt"
"os/exec"
)
var cmds = map[string][]string{
"go": []string{"/usr/local/go/bin/go", "version"},
"java": []string{"java", "-version"},
}
type GetHand struct {
Name string
Version string
}
func handleKey(key string) (*GetHand, error) {
cmd := cmds[key]
if cmd == nil {
return nil, fmt.Errorf("No such key : %v", key)
}
b, err := exec.Command("/usr/local/go/bin/go", "version").Output()
if err != nil {
return nil, err
}
return &GetHand{
Name: key,
Version: string(b),
}, nil
}
func main() {
h, err := handleKey("go")
if err != nil {
fmt.Println(err)
return
}
fmt.Println(h)
h, err = handleKey("java")
if err != nil {
fmt.Println(err)
return
}
fmt.Println(h)
}
Your current code with all those sed tried to modify yor source code. It does not work that way. Some ancient practices (usually using bash) auctually did that kind of staff but it is outdated and problemtic.
As for go, you sure know go is a compiled language, which compiles source code into binaries. In that way, any modification to the source code will not affect binaries, and resulting the process running unaware of that. In that sense, modifying the source code by the process it self is wrong and uneffective. Even if it is possible to use compiler to rebuild binaries, it is going to be both dangerous and inefficient. Do not do it that way.
In your specified case, there is a much better and a much common practice: variable. You sure use it everyday. It would be the simplest to just set the fields of GetHands to needed value. By doing that, your response to each request (what you call URL, but it is wrong) will change because the fields of GetHands change as variable.
I write the code below:
import (
"os/exec"
"regex"
"runtime"
)
var pattern = regexp.MustCompile(`version \"(.+)\"`)
func GetJavaVersion() string {
cmd := exec.Command("Java", "-version")
out, err := cmd.Output()
if err != nil {
panic(err)
}
return string(pattern.FindSubmatch(out)[1])
}
func GetGoVersion() string {
return runtime.Version()[2:]
}
func handler(w http.ResponseWriter, r *http.Request) {
keys, ok := r.URL.Query()["key"]
if !ok || len(keys) < 1 {
log.Println("Url Param 'key' is missing")
return
}
key := keys[0]
log.Println("Url Param 'key' is: " + string(key))
var GetHands GetHand
if key == "java" {
GetHands = GetHand{Name: "java", Version: GetJavaVersion()}
}
if key == "go" {
GetHands = GetHand{Name: "go", Version: GetGoVersion()}
}
if err := json.NewEncoder(w).Encode(GetHands); err != nil {
panic(err)
}
}
So, for go, you can auctually get the version info from package runtime, though the info started with "go". You can simply slice it. And for java, you would have to run a command: The same as your orignal code: java -version. But instead of passing it to sed, you get the output from it by using Command.Output and then use a regexp to find the version information. regexp is a very useful tool to get info from strings. You can read about it here
I'm writing some unit tests for my application in Go. The tests fail however because it cannot find the configuration files. Normally the binary looks for the configuration files in the working directory under the path conf/*.conf.
I figured that browsing to the directory that has conf/ and running go test in it would solve it, but it still reports that the file system cannot find the path specified.
How can I tell go test to use a certain directory as the working directory so that the tests may actually be executed?
You may be able to use the Caller to get the path to the current test source file, like this:
package sample
import (
"testing"
"runtime"
"fmt"
)
func TestGetFilename(t *testing.T) {
_, filename, _, _ := runtime.Caller(0)
t.Logf("Current test filename: %s", filename)
}
I do not believe this is possible. I have not been able to find documentation stating this explicitly, but I believe go test always uses the package directory (containing the go source files) as the working directory.
As a workaround, I compiled the test and execute the test from the current directory.
go test -c && ./<mypackage>.test
Or, if you want a generic command that you can use, you can rename the test file with -o option.
go test -c -o xyz.test && ./xyz.test
While not really convenient, you can always pass it as a command line variable, for example :
package blah_test
import (
"flag"
"fmt"
"os"
"testing"
)
var (
cwd_arg = flag.String("cwd", "", "set cwd")
)
func init() {
flag.Parse()
if *cwd_arg != "" {
if err := os.Chdir(*cwd_arg); err != nil {
fmt.Println("Chdir error:", err)
}
}
}
func TestBlah(t *testing.T) {
t.Errorf("cwd: %+q", *cwd_arg)
}
Then run it like :
┌─ oneofone#Oa [/tmp]
└──➜ go test . -cwd="$PWD"
--- FAIL: TestBlah (0.00 seconds)
blah_test.go:16: cwd: "/tmp"
No matter where the work directory is. It must be under your project Dir. So my solution is
wd, _ := os.Getwd()
for !strings.HasSuffix(wd, "<yourProjectDirName>") {
wd = filepath.Dir(wd)
}
raw, err := ioutil.ReadFile(fmt.Sprintf("%s/src/conf/conf.dev.json", wd))
Your path should always start from your project Dir. Every time you read the file in a package and accessed by main.go or your another package unit test. It will always work.
You can use the os package.
You would want to do something like this
func TestMyFunction(t *testing.T) {
os.Chdir("./path")
//TEST FUNCTION
os.Chdir("..")
}
There are several possibilities in the os package.
To add init function into *_test.go under your test package.
Test package will run this function before test function start.
func init() {
_, filename, _, _ := runtime.Caller(0)
// The ".." may change depending on you folder structure
dir := path.Join(path.Dir(filename), "..")
err := os.Chdir(dir)
if err != nil {
panic(err)
}
}
I know this is an old question but I had the same problem trying to use migrations for the database on my tests, and maybe this solution helps someone.
Since there is no native way of getting the project directory, you could identify some file or directory that you know it's only in the root of the project (in my case, it was the relative directory database/migrations). Once you have this unique relative directory, you could have a function like the following to obtain the project root directory. It just gets the current working directory (assuming it's inside the project's directory) and starts to navigate all the way up until it finds a dir that has the relative directory you know it's on the root of the project:
func FindMyRootDir() string {
workingDirectory, err := os.Getwd()
if err != nil {
panic(err)
}
lastDir := workingDirectory
myUniqueRelativePath := "database/migrations"
for {
currentPath := fmt.Sprintf("%s/%s", lastDir, myUniqueRelativePath)
fi, err := os.Stat(currentPath)
if err == nil {
switch mode := fi.Mode(); {
case mode.IsDir():
return currentPath
}
}
newDir := filepath.Dir(lastDir)
// Ooops, we couldn't find the root dir. Check that your "myUniqueRelativePath" really exists
if newDir == "/" || newDir == lastDir {
return ""
}
lastDir = newDir
}
}
Of course it's not the most beautiful solution, but it works.
I've had a similar problem and found the solution on this blog
Basically you can change the folder that the test is running using a similar function:
package main
import (
"os"
"path"
"runtime"
)
func MakeFunctionRunOnRootFolder() {
_, filename, _, _ := runtime.Caller(0)
// The ".." may change depending on you folder structure
dir := path.Join(path.Dir(filename), "..")
err := os.Chdir(dir)
if err != nil {
panic(err)
}
}
Go 1.20 is getting new -C arguments for "go subcommands" so this should help:
go test -C directory/ ...
It's a common practice in Go to place test fixtures in same package inside testdata folder.
Some examples from standard library:
debug/elf
net/http
image
Also, there is a post from Dave Cheney, where he suggests following code:
f, err := os.Open("testdata/somefixture.json")
I currently use a neat solution for this problem, instead of opening the file directly by calling os.Open(), I use the embed package in a smart way:
First I create a global variable in my root package called:
//go:embed config/* otherdirectories/*
var RootFS embed.FS
Then I just open the files inside my tests by using this global variable, e.g.:
func TestOpenConfig(t *testing.T) {
configFile, err := rootpkg.RootFS.ReadFile("config/env")
if err != nil {
t.Fatalf("unable to open config/env file: %s", err)
}
if string(configFile) != "FOO=bar\n" {
t.Fatalf("config file contents differ from expected: %s", string(configFile))
}
}
This is a neat trick because now you can always work with relative paths from your root package, which is what I used to do in other programming languages.
Of course, this has the restriction that you will need to import your root package, which depending on your package layout might not be ideal because of cyclic imports. If this is your case you might just create a embed.go file inside the config directory itself and call
your configs by name.
One other drawback is that you are embedding test files in your binary, this is probably ok if your test files are not very big, like megabytes big, so I don't really mind this issue.
I also created a repository for illustrating this solution:
https://github.com/VinGarcia/golang-reading-files-from-tests
I would use an Environment Variable for the location of your application. It seems to be the best way when running go tools, as test programs can be run from a temporary location.
// get home dir of app, use MYAPPHOME env var if present, else executable dir.
func exeDir() string {
dir, exists := os.LookupEnv("MYAPPHOME")
if exists {
return dir
} else {
ex, err := os.Executable()
if err != nil {
panic(err)
}
exPath := path.Dir(ex)
return exPath
}
}
I am using golang revel web framework and
I am trying to create a sqlite db in the current working directory.
model.go
func New(dbName string,table string) *Db {
_,filename,_,_ := runtime.Caller(1)
db , err := sql.Open("sqlite3",path.Join(path.Dir(filename),dbName))
if err != nil {
log.Fatal(err)
}
err = db.Ping()
if err != nil {
log.Panic(err)
}
database := &Db{Database:db}
_,err = db.Exec("create table %s" +
"( id integer primary key, " +
"name varchar(100),"+
"email varchar(100),"+
"branch varchar(100),"+
"help varchar(100)",)
if err != nil {
log.Fatal(err)
}
}
I have a test in place which just calls this function.
whenever i run the test using revel test or by going to the localhost:9000/#tests, the function Panics and the error message is
cannot open the database file.
The reason that is happening is because the filename returned by runtime.Caller(1) is /usr/local/go/src/runtime/asm_amd64.s for which the program has no permission.
if i directly write ./foo.db, even then the error shows.
I tried os.Getwd() which return empty string.
I also tried filepath.Abs(filepath.Dir(os.Args[0]))
but that returned /home/girish/GoProjects/bin/revel.d which is the revel binary.
So whats the best way to find the directory of the model.go?
It doesn't make sense to get the directory of the model.go file at runtime, because the compiled executable could be on a completely different filesystem.
You may want to get the directory of where the running executable was started from:
dir, err := filepath.Abs(filepath.Dir(os.Args[0]))
dir will be the folder where the program lives at runtime.
I am trying to create a set of nested directories from a Go executable such as 'dir1/dir2/dir3'. I have succeeded in creating a single directory with this line:
os.Mkdir("." + string(filepath.Separator) + c.Args().First(),0777);
However, I have no idea how to approach creating a predetermined nested set of directories inside of that directory.
os.Mkdir is used to create a single directory. To create a folder path, instead try using:
os.MkdirAll(folderPath, os.ModePerm)
Go documentation
func MkdirAll(path string, perm FileMode) error
MkdirAll creates a directory named path, along with any necessary parents, and returns nil, or else returns an error. The permission bits perm are used for all directories that MkdirAll creates. If path is already a directory, MkdirAll does nothing and returns nil.
Edit:
Updated to correctly use os.ModePerm instead.
For concatenation of file paths, use package path/filepath as described in #Chris' answer.
This way you don't have to use any magic numbers:
os.MkdirAll(newPath, os.ModePerm)
Also, rather than using + to create paths, you can use:
import "path/filepath"
path := filepath.Join(someRootPath, someSubPath)
The above uses the correct separators automatically on each platform for you.
If the issue is to create all the necessary parent directories, you could consider using os.MkDirAll()
MkdirAll creates a directory named path, along with any necessary parents, and returns nil, or else returns an error.
The path_test.go is a good illustration on how to use it:
func TestMkdirAll(t *testing.T) {
tmpDir := TempDir()
path := tmpDir + "/_TestMkdirAll_/dir/./dir2"
err := MkdirAll(path, 0777)
if err != nil {
t.Fatalf("MkdirAll %q: %s", path, err)
}
defer RemoveAll(tmpDir + "/_TestMkdirAll_")
...
}
(Make sure to specify a sensible permission value, as mentioned in this answer)
This is one alternative for achieving the same but it avoids race condition caused by having two distinct "check ..and.. create" operations.
package main
import (
"fmt"
"os"
)
func main() {
if err := ensureDir("/test-dir"); err != nil {
fmt.Println("Directory creation failed with error: " + err.Error())
os.Exit(1)
}
// Proceed forward
}
func ensureDir(dirName string) error {
err := os.MkdirAll(dirName, os.ModeDir)
if err == nil || os.IsExist(err) {
return nil
} else {
return err
}
}
An utility method like the following can be used to solve this.
import (
"os"
"path/filepath"
"log"
)
func ensureDir(fileName string) {
dirName := filepath.Dir(fileName)
if _, serr := os.Stat(dirName); serr != nil {
merr := os.MkdirAll(dirName, os.ModePerm)
if merr != nil {
panic(merr)
}
}
}
func main() {
_, cerr := os.Create("a/b/c/d.txt")
if cerr != nil {
log.Fatal("error creating a/b/c", cerr)
}
log.Println("created file in a sub-directory.")
}