I used Lambda functions before, and if I remember correctly I'm supposed to have ~500Mb of (ephemeral) space in /tmp.
Nevertheless, my Go lambda function doesn't seem to interact with the fs properly:
exec.Command("ls -la /").Output() returns empty
exec.Command("rm -rf /tmp/xxx").Run() returns fork/exec : no such file or directory
exec.Command("mkdir -p /tmp/xxx").Run() returns fork/exec : no such file or directory
It's really weird.
It's using the go1.x environment (thus, I guess amazonlinux:2)
UPDATE
I CAN access the fs using Go os functions:
os.RemoveAll("/tmp/xxx")
if _, err := os.Stat("/tmp/xxx"); os.IsNotExist(err) {
if err := os.Mkdir("/tmp/xxx", os.ModePerm); err != nil {
return err
}
}
BUT I really need exec to run afterwards (a binary command), and write a file in that tmp folder. The error in that case is the same (no such file or directory). Even though I've just created the folder with the above commands.
You are close. The way you use exec.Command() is not yet 100% correct. Try the following:
package main
import (
"fmt"
"os"
"os/exec"
)
func main() {
o, err := exec.Command("ls", "-la", "/tmp").Output()
if err != nil {
fmt.Println(err)
os.Exit(1)
}
fmt.Printf("%s\n", o)
}
The first argument to Command() is the program you want to run and all the following arguments are the programs arguments.
See https://play.golang.org/p/WaVOU0IESmZ
Related
I have a cli that I am making that is more for learning purposes and creating my own cli, that does stuff. Anyways, I am testing the delete function and it works fine and gives me the right answer. However, I don't believe that it is the best practice and was wondering if you could let me know if it's ok, or not.
Test file
func TestDeleteConfig(t *testing.T) {
err := cm.DeleteConfig()
if err != nil {
t.Errorf("error when deleting the folder: %s", err)
}
usr, err := user.Current()
if err != nil {
t.Errorf("error when getting user current: %s", err)
}
fp := filepath.Join(usr.HomeDir, ".config", "godot", "config.json")
fmt.Println("the path of the config file", fp)
if _, e := os.Stat(fp); !os.IsNotExist(e) {
t.Errorf("error path still exists: %v", e)
}
}
function being tested
func DeleteConfig() error {
usr, err := user.Current()
if err != nil {
return err
}
err = os.RemoveAll(filepath.Join(usr.HomeDir, ".config", "godot"))
if err != nil {
return err
}
return nil
}
The problem is that I don't want the DeleteConfig() to take any arguments as this is a hard path. I have a separate function for deleting a single file in which the help with this will solve that as well func DeleteFile(p string) error {}.
So for testing purposes should I just create a foo directory at a separate path (within the test), delete said directory, and assume that if it works on foo path then it should work with the godot directory?
It's a bit philosophical.
If you absolutely do not want to stub/mock any part of your code to replace accessing a real file system to do testing, then we're talking about what you'd call system or integration testing. That's fine in itself: for instance, you could run such tests in a throw-away container as part of a CI pipeline.
But with this approach you can't sensibly do what is called unit-testing.
To unit-test your function you need to replace something it uses with something "virtualized". Exactly how to do that is an open question.
Approach: make is possible to override os.RemoveAll
For instance, you can have a private global variable containing the "remove whole directory" function—like this:
var removeAll = os.RemoveAll
func DeleteConfig() error {
usr, err := user.Current()
if err != nil {
return err
}
err = removeAll(filepath.Join(usr.HomeDir, ".config", "godot"))
if err != nil {
return err
}
return nil
}
Then to test it you'd just monkey-patch the function with your own implementation which would have the same signature as os.RemoveAll, like this:
func TestDeleteConfig(t *testing.T) {
var actualPath string
removeAll = func(path string) {
actualPath = path
return nil
}
defer func() { removeAll = os.RemoveAll }()
DeleteConfig()
if actualPath != expectedPath {
t.Errorf("unexpected path: want %s, got %s", expectedPath, actualPath)
}
}
This approach also allows to test how your function handles errors: just replace it with something which generates an error and then afterwards check that your function under test returned that error (or wrapped it in an expected way, whatever).
Still, it's a bit low-tech for a number of reasons:
A global variable is involved, so you have to be sure no two tests which monkey-patch it run concurrently, or that all patching is done before running those tests.
If different tests need to set it to different value, they must be serialized.
Approach: do not hard-code the concept of "the user" or "the config"
Another approach is to notice that basically the problem with testing stems from the fact you hard-coded getting of the user.
Leaving aside the flawed approach you've taken to getting the place of configuration (you should be using something which implements the XDG spec), if you could easily override getting of the "root" directory (which is the user's home directory in your code), you could easily target your function to operate on the result of calling io/ioutil.TempDir.
So may be a way to go is to have an interface type like
type ConfigStore interface {
Dir() string
}
of which the Dir() method is supposed to return the path to the configuration store's root directory.
Your DeleteConfig() would then start to accept a single argument of type ConfigStore, and in your program you'd have a concrete implementation of it, and in your testing code — a stub implementing the same interface and managing a temporary directory.
Approach: go full-on virtualized
Right now, a work is being done on bringing filesystem virtualization right into the Go standard library, but while it's not there yet, 3rd-party packages which do that exist for ages, — for instance, github.com/spf13/afero.
Basically, they allow you to not use os directly but write all your code in a way so that instead of the os package it calls methods on an instance of a type implementing particular interface: in the production code that object is a thin shim for the os package, and in the testing code it's replaced by whatever you wish; afero has a readily-available in-memory FS backend to do this.
Writing a unit test for filesystem checking is not trivial. You should NOT create a real file on the system, because then your test will depend on the I/O of the file system itself. The last resort is mocking the filesystem. There are quite a few powerful libraries like spf13/afero for this purpose (mocking of a filesystem). These packages will create temporary files in the background and clean up afterward.
main.go
package main
import (
"log"
"os/user"
"path/filepath"
iowrap "github.com/spf13/afero"
)
var (
// FS is simulated filesystem interface
FS iowrap.Fs
// FSUtil is the struct of the simulated interface
FSUtil *iowrap.Afero
)
func init() {
FS = iowrap.NewOsFs()
FSUtil = &iowrap.Afero{Fs: FS}
}
// DeleteConfig removes ~/.config/godot if exists
func DeleteConfig() error {
usr, err := user.Current()
if err != nil {
return err
}
path := filepath.Join(usr.HomeDir, ".config", "godot")
log.Println(path)
err = FSUtil.RemoveAll(path)
return err
}
func main() {
err := DeleteConfig()
if err != nil {
log.Fatal(err)
}
}
main_test.go
package main
import (
"os/user"
"path/filepath"
"testing"
iowrap "github.com/spf13/afero"
)
func init() {
FS = iowrap.NewMemMapFs()
FSUtil = &iowrap.Afero{Fs: FS}
usr, _ := user.Current()
pathDir := filepath.Join(usr.HomeDir, ".config")
filePath := filepath.Join(pathDir, "godot")
FS.MkdirAll(pathDir, 0755)
iowrap.WriteFile(FS, filePath, []byte("0-7\n"), 0644)
}
const (
succeed = "\u2713"
failed = "\u2717"
)
func TestDeleteConfig(t *testing.T) {
t.Log("Given the need to test downloading a webpage content")
{
usr, _ := user.Current()
pathDir := filepath.Join(usr.HomeDir, ".config")
filePath := filepath.Join(pathDir, "godot")
t.Logf("\tTest 0:\tWhen deleting the %v with 0644 permissions", filePath)
{
err := DeleteConfig()
if err != nil {
t.Fatalf("\t%s\tThe file couldn't be deleted: %v", failed, err)
}
t.Logf("\t%s\tThe file has been successfully deleted.", succeed)
}
}
}
Functional test:
touch C:/Users/drpan/.config/godot
ls -l C:/Users/drpan/.config/godot Output: -rw-r--r-- 1 drpan 197609 0 Nov 2 19:38 C:/Users/drpan/.config/godot
./deletedirectory.exe
ls -l C:/Users/drpan/.config/godot Output: ls: cannot access 'C:/Users/drpan/.config/godot': No such file or directory
Unit Test:
$ touch C:/Users/drpan/.config/godot
$ go test
2020/11/02 19:55:35 C:\Users\drpan\.config\godot
PASS
ok github.com/drpaneas/deletedirectory 0.162s
$ ls -l C:/Users/drpan/.config/godot Output: -rw-r--r-- 1 drpan 197609 0 Nov 2 19:55 C:/Users/drpan/.config/godot
The following code gives ENOENT (2) Do you know how to get stat of a file containing tilde?
file := "~/.zshrc"
fileStat, err := os.Stat(file)
if err != nil {
return 0, err
}
The tilde is not something that the file system calls are able to interpret - it has a meaning only in shells like bash, which typically interprets it as $HOME. So you'll probably want to use os.Getenv("HOME") and then replace ~ with the result. Or, as suggested by Allon Guralnek in a comment, use os.UserHomeDir(), which reads the appropriate environment variable depending on your OS.
You can access the home directory of the current user using the os/user package.
Something like this will get you near what you want:
package main
import (
"fmt"
"log"
"os"
"os/user"
)
func main() {
usr, err := user.Current()
if err != nil {
log.Fatal(err)
}
fmt.Println(usr.HomeDir)
file := usr.HomeDir + "/.zshrc"
fileStat, err := os.Stat(file)
if err != nil {
log.Fatal(err)
}
fmt.Println(fileStat)
}
I have downloaded over 500 Gb of data to a single directory off of AWS.
Whenever I try to access that directory, the command line hangs and doesn't show me anything.
I'm trying to run some code that will interact with the files by printing out the path of each file but the command line hangs and then exits the program.
The program definitely starts execution because "Printing file path's" gets displayed to the console.
func main() {
fmt.Println("Printing file path's")
err := filepath.Walk(source,
func(fpath string, info os.FileInfo, err error) {
if !info.IsDir() && file path.Ext(fpath)==".txt" {
fmt.Println(fpath)
}
}
}
}
How should I handle the situation of being able to view all the files in the command line and why is this program not working?
UPDATE:
By using
files, err := dir.Readdir(10)
if err == io.EOF {
break
}
I was able to snap up the first 10 folders/files in the directory.
Using a loop I could keep doing this until I hit the end of the directory.
This doesn't rely on ordering the files/folders as does the walk function and so its more efficient.
The possible performance issue with filepath.Walk is clearly documented:
The files are walked in lexical order, which makes the output deterministic but means that for very large directories Walk can be inefficient.
Use os.File.Readdir to iterate files in filesystem order:
Readdir reads the contents of the directory associated with file and returns a slice of up to n FileInfo values, as would be returned by Lstat, in directory order. Subsequent calls on the same file will yield further FileInfos.
package main
import (
"fmt"
"io"
"log"
"os"
"time"
)
func main() {
dir, err := os.Open("/tmp")
if err != nil {
log.Fatal(err)
}
for {
files, err := dir.Readdir(10)
if err == io.EOF {
break
}
if err != nil {
log.Fatal(err)
}
for _, fi := range files {
classifier := ""
if fi.IsDir() {
classifier = "/"
}
fmt.Printf("%v %12d %s%s\n",
fi.ModTime().UTC().Truncate(time.Second),
fi.Size(),
fi.Name(), classifier,
)
}
}
}
I am trying to create a set of nested directories from a Go executable such as 'dir1/dir2/dir3'. I have succeeded in creating a single directory with this line:
os.Mkdir("." + string(filepath.Separator) + c.Args().First(),0777);
However, I have no idea how to approach creating a predetermined nested set of directories inside of that directory.
os.Mkdir is used to create a single directory. To create a folder path, instead try using:
os.MkdirAll(folderPath, os.ModePerm)
Go documentation
func MkdirAll(path string, perm FileMode) error
MkdirAll creates a directory named path, along with any necessary parents, and returns nil, or else returns an error. The permission bits perm are used for all directories that MkdirAll creates. If path is already a directory, MkdirAll does nothing and returns nil.
Edit:
Updated to correctly use os.ModePerm instead.
For concatenation of file paths, use package path/filepath as described in #Chris' answer.
This way you don't have to use any magic numbers:
os.MkdirAll(newPath, os.ModePerm)
Also, rather than using + to create paths, you can use:
import "path/filepath"
path := filepath.Join(someRootPath, someSubPath)
The above uses the correct separators automatically on each platform for you.
If the issue is to create all the necessary parent directories, you could consider using os.MkDirAll()
MkdirAll creates a directory named path, along with any necessary parents, and returns nil, or else returns an error.
The path_test.go is a good illustration on how to use it:
func TestMkdirAll(t *testing.T) {
tmpDir := TempDir()
path := tmpDir + "/_TestMkdirAll_/dir/./dir2"
err := MkdirAll(path, 0777)
if err != nil {
t.Fatalf("MkdirAll %q: %s", path, err)
}
defer RemoveAll(tmpDir + "/_TestMkdirAll_")
...
}
(Make sure to specify a sensible permission value, as mentioned in this answer)
This is one alternative for achieving the same but it avoids race condition caused by having two distinct "check ..and.. create" operations.
package main
import (
"fmt"
"os"
)
func main() {
if err := ensureDir("/test-dir"); err != nil {
fmt.Println("Directory creation failed with error: " + err.Error())
os.Exit(1)
}
// Proceed forward
}
func ensureDir(dirName string) error {
err := os.MkdirAll(dirName, os.ModeDir)
if err == nil || os.IsExist(err) {
return nil
} else {
return err
}
}
An utility method like the following can be used to solve this.
import (
"os"
"path/filepath"
"log"
)
func ensureDir(fileName string) {
dirName := filepath.Dir(fileName)
if _, serr := os.Stat(dirName); serr != nil {
merr := os.MkdirAll(dirName, os.ModePerm)
if merr != nil {
panic(merr)
}
}
}
func main() {
_, cerr := os.Create("a/b/c/d.txt")
if cerr != nil {
log.Fatal("error creating a/b/c", cerr)
}
log.Println("created file in a sub-directory.")
}
In nodejs I use __dirname . What is the equivalent of this in Golang?
I have googled and found out this article http://andrewbrookins.com/tech/golang-get-directory-of-the-current-file/ . Where he uses below code
_, filename, _, _ := runtime.Caller(1)
f, err := os.Open(path.Join(path.Dir(filename), "data.csv"))
But is it the right way or idiomatic way to do in Golang?
EDIT: As of Go 1.8 (Released February 2017) the recommended way of doing this is with os.Executable:
func Executable() (string, error)
Executable returns the path name for the executable that started the current process. There is no guarantee that the path is still pointing to the correct executable. If a symlink was used to start the process, depending on the operating system, the result might be the symlink or the path it pointed to. If a stable result is needed, path/filepath.EvalSymlinks might help.
To get just the directory of the executable you can use path/filepath.Dir.
Example:
package main
import (
"fmt"
"os"
"path/filepath"
)
func main() {
ex, err := os.Executable()
if err != nil {
panic(err)
}
exPath := filepath.Dir(ex)
fmt.Println(exPath)
}
OLD ANSWER:
You should be able to use os.Getwd
func Getwd() (pwd string, err error)
Getwd returns a rooted path name corresponding to the current directory. If the current directory can be reached via multiple paths (due to symbolic links), Getwd may return any one of them.
For example:
package main
import (
"fmt"
"os"
)
func main() {
pwd, err := os.Getwd()
if err != nil {
fmt.Println(err)
os.Exit(1)
}
fmt.Println(pwd)
}
This should do it:
import (
"fmt"
"log"
"os"
"path/filepath"
)
func main() {
dir, err := filepath.Abs(filepath.Dir(os.Args[0]))
if err != nil {
log.Fatal(err)
}
fmt.Println(dir)
}
Use package osext
It's providing function ExecutableFolder() that returns an absolute path to folder where the currently running program executable reside (useful for cron jobs). It's cross platform.
Online documentation
package main
import (
"github.com/kardianos/osext"
"fmt"
"log"
)
func main() {
folderPath, err := osext.ExecutableFolder()
if err != nil {
log.Fatal(err)
}
fmt.Println(folderPath)
}
I came from Node.js to Go. The Node.js equivalent to __dirname in Go is:
_, filename, _, ok := runtime.Caller(0)
if !ok {
return errors.New("unable to get the current filename")
}
dirname := filepath.Dir(filename)
Some other mentions in this thread and why they're wrong:
os.Executable() will give you the filepath of the currently running executable. This is equivalent to process.argv[0] in Node. This is not true if you want to take the __dirname of a sub-package.
os.Getwd() will give you the current working directory. This is the equivalent to process.cwd() in Node. This will be wrong when you run your program from another directory.
Lastly, I'd recommend against pulling in a third-party package for this use case. Here's a package you can use:
package current
// Filename is the __filename equivalent
func Filename() (string, error) {
_, filename, _, ok := runtime.Caller(1)
if !ok {
return "", errors.New("unable to get the current filename")
}
return filename, nil
}
// Dirname is the __dirname equivalent
func Dirname() (string, error) {
filename, err := Filename()
if err != nil {
return "", err
}
return filepath.Dir(filename), nil
}
Note that I've adjusted runtime.Caller(1) to 1 because we want to get the directory of the package that called current.Dirname(), not the directory containing the current package.
filepath.Abs("./")
Abs returns an absolute representation of path. If the path is not
absolute it will be joined with the current working directory to turn
it into an absolute path.
As stated in the comment, this returns the directory which is currently active.
os.Executable: https://tip.golang.org/pkg/os/#Executable
filepath.EvalSymlinks: https://golang.org/pkg/path/filepath/#EvalSymlinks
Full Demo:
package main
import (
"fmt"
"os"
"path/filepath"
)
func main() {
var dirAbsPath string
ex, err := os.Executable()
if err == nil {
dirAbsPath = filepath.Dir(ex)
fmt.Println(dirAbsPath)
return
}
exReal, err := filepath.EvalSymlinks(ex)
if err != nil {
panic(err)
}
dirAbsPath = filepath.Dir(exReal)
fmt.Println(dirAbsPath)
}
if you use this way :
dir, err := filepath.Abs(filepath.Dir(os.Args[0]))
if err != nil {
log.Fatal(err)
}
fmt.Println(dir)
you will get the /tmp path when you are running program using some IDE like GoLand because the executable will save and run from /tmp
i think the best way for getting the currentWorking Directory or '.' is :
import(
"os"
"fmt"
"log"
)
func main() {
dir, err := os.Getwd()
if err != nil {
log.Fatal(err)
}
fmt.Println(dir)
}
the os.Getwd() function will return the current working directory.
and its all without using of any external library :D
If you use package osext by kardianos and you need to test locally, like Derek Dowling commented:
This works fine until you'd like to use it with go run main.go for
local development. Not sure how best to get around that without
building an executable beforehand each time.
The solution to this is to make a gorun.exe utility instead of using go run. The gorun.exe utility would compile the project using "go build", then run it right after, in the normal directory of your project.
I had this issue with other compilers and found myself making these utilities since they are not shipped with the compiler... it is especially arcane with tools like C where you have to compile and link and then run it (too much work).
If anyone likes my idea of gorun.exe (or elf) I will likely upload it to github soon..
Sorry, this answer is meant as a comment, but I cannot comment due to me not having a reputation big enough yet.
Alternatively, "go run" could be modified (if it does not have this feature already) to have a parameter such as "go run -notemp" to not run the program in a temporary directory (or something similar). But I would prefer just typing out gorun or "gor" as it is shorter than a convoluted parameter. Gorun.exe or gor.exe would need to be installed in the same directory as your go compiler
Implementing gorun.exe (or gor.exe) would be trivial, as I have done it with other compilers in only a few lines of code... (famous last words ;-)
Sometimes this is enough, the first argument will always be the file path
package main
import (
"fmt"
"os"
)
func main() {
fmt.Println(os.Args[0])
// or
dir, _ := os.Getwd()
fmt.Println(dir)
}
dir, err := os.Getwd()
if err != nil {
fmt.Println(err)
}
this is for golang version: go version go1.13.7 linux/amd64
works for me, for go run main.go. If I run go build -o fileName, and put the final executable in some other folder, then that path is given while running the executable.
Do not use the "Answer recommended by Go Language" with runtime.Caller(0).
That works when you go build or go install a program, because you are re-compiling it.
But when you go build a program and then distribute it (copy) on your colleagues' workstations (who don't have Go, and just need the executable), the result of runtime.Caller(0) would still be the path of where you built it (from your computer).
Ie a path which would likely not exist on their own computer.
os.Args[0] or, better, os.Executable() (mentioned here) and kardianos/osext (mentioned here and here), are more reliable.
None of the answers here worked for me, at least on go version go1.16.2 darwin/amd64. This is the only thing close to the __dirname functionality in node
This was posted by goland engineer Daniil Maslov in the jetbrains forums
pasted below for easier reading:
The trick is actually very simple and is to get the current executing and add .. to the project root.
Create a new directory and file like testing_init.go with the following content:
package testing_init
import (
"os"
"path"
"runtime"
)
func init() {
_, filename, _, _ := runtime.Caller(0)
dir := path.Join(path.Dir(filename), "..")
err := os.Chdir(dir)
if err != nil {
panic(err)
}
}
After that, just import the package into any of the test files:
package main_test
import (
_ "project/testing_init"
)
Now you can specify paths from the project root
// GetCurrentDir
func GetCurrentDir() string {
p, _ := os.Getwd()
return p
}
// GetParentDir
func GetParentDir(dirctory string) string {
return substr(dirctory, 0, strings.LastIndex(dirctory, "/"))
}
func substr(s string, pos, length int) string {
runes := []rune(s)
l := pos + length
if l > len(runes) {
l = len(runes)
}
return string(runes[pos:l])
}
If your file is not in the main package then the above answers won't work
I tried different approaches to find find the directory of the currently running file but failed.
The best possible answer is in the question itself this is how I find the current working directory of the file which is not in the main package.
_, filename, _, _ := runtime.Caller(1)
pwd := path.Dir(filename)
Gustavo Niemeyer's answer is great.
But in Windows, runtime proc is mostly in another dir, like this:
"C:\Users\XXX\AppData\Local\Temp"
If you use relative file path, like "/config/api.yaml", this will use your project path where your code exists.