I would like to recursively create a directory and assign an owner and group for the folders and its parents that were created.
For example, assuming /var exists, I want to create /var/test1/test2/test3.
I am able to do this using os.MkdirAll("/var/test1/test2/test3", 0600).
However, I also want to set test1, test2, and test3's uid to user1 and gid to user1.
It's possible to do so using os.Chown, but that requires a lot of manually work. I would need build a tree of the folder and its parents that do not exist before creating the folder chain and then use os.Chown on each folder after creation.
Is there a simpler way?
A bit like this ChownR() type of function, the idea would be to filter the walk, and apply the chown only to folders which are part of a path passed in parameter (here "/var/test1/test2/test3)
Without filter:
func ChownR(path string, uid, gid int) error {
return filepath.Walk(path, func(name string, info os.FileInfo, err error) error {
if err == nil {
err = os.Chown(name, uid, gid)
}
return err
})
}
In other words, with filepath.Walk(), there should not be too much "manual work" involved here.
Switch the mkdir process to the user you wish to create directories for. This has the advantage of creating directories with correct permissions and ownership in a single command instead of creating loops or running multiple commands back to back.
Example:
import "os/exec"
import "syscall"
func main() {
// The command here can use flags to create all of the dirs at once
cmd := exec.Command('mkdir', '-p', '/var/test1/test2/test3/')
_ = syscall.Umask(0077) // Set umask for this process
cmd.SysProcAttr = &syscall.SysProcAttr{}
cmd.SysProcAttr.Credential = &syscall.Credential{Uid: uid, Gid: gid}
cmd.Run()
}
Since the subprocess "mkdir" runs as the user specified by uid, the folders would already have the correct owner. If you get the umask part right, each directory will also have correct permissions.
One caveat: this is not a very portable solution. Using syscall should only be your solution if you are certain your code will only be executing on a specific OS.
Related
myprogram/
|
|-main.go
|-dir1/
|-data/
|-datafile.json
|-runner.go
|-runner_test.go
In runner.go, I have a simple function that reads the datafile.json. Something like
func GetPayload() (string, err) {
dBytes, dErr := ioutil.ReadFile("dir1/data/datafile.json")
if dErr != nil { return nil, dErr}
return dBytes, nil
}
I'm using Go in a Lambda with a structure similar to above. When the Lambda runs in its actual environment, it starts at main.go, and then invokes GetPayload() from runner.go. However, I have a test in a simple worker node machine in runner_test.go that also hits GetPayload() .
During "normal" execution (from main.go) - this works OK. However, when GetPayload() is invoked from runner_test.go, it errors, saying
open dir1/data/datafile.json no such file or directory
This makes sense, because during the test, the working directory is the directory that houses runner_test.go, which is data/, so there is no dir1 as a child of it. I've been trying to play with using os.Getwd() and getting the paths from there like:
pwd, _ := os.Getwd()
dBytes, dErr := ioutil.ReadFile(pwd + "dir1/data/datafile.json")
But again, that won't work, because for runner_test.go pwd is user/myname/myprogram/dir1, but from main.go, it turns up as user/myname/myprogram.
Any idea how I can find and open datafile.json from within GetPayload() in any environment? I could pass an optional parameter to GetPayload() but if possible, it'd be great to avoid that.
If the file is static (meaning that it doesn't need to change after you build the program), you can embed it into the built program. This means you no longer have to worry about run-time file paths.
import (
"embed"
)
//go:embed data/*
var dataFiles embed.FS
func GetPayload() (string, err) {
dBytes, dErr := dataFiles.ReadFile(dataFiles, "data/datafile.json")
if dErr != nil { return nil, dErr}
return dBytes, nil
}
Now the files in your data/ directory are embedded in this variable dataFiles which acts as a read-only file system.
For more info:
Read more about embed in the package documentation overview.
Read my answer about "when to use embed"
For data files that your program needs during runtime, either use a fixed directory and refer to that, or accept a command line argument or some sort of configuration that tells you where the file is.
When running unit tests, the wd is the directory containing the test file. One convention is to use a testdata/ directory under the directory containing the test, and put all data files there. That way you can refer to that file from the test by using testdata/datafile.json.
You can use a copy of the file you need during runtime as your test file, or you can use a symlink from the runtime data file to the test file under the testdata/ dir.
For data files that your program needs during runtime, either use a fixed
directory and refer to that
Someone made this suggestion, which I agree with. To that end, you can use
something like this:
package main
import (
"os"
"path/filepath"
)
func main() {
d, err := os.UserCacheDir()
if err != nil {
panic(err)
}
d = filepath.Join(d, "file.json")
f, err := os.Open(d)
if err != nil {
panic(err)
}
defer f.Close()
os.Stdout.ReadFrom(f)
}
https://golang.org/pkg/os#UserCacheDir
https://golang.org/pkg/os#UserConfigDir
I am trying to implement the function mkdir in a fuse, written in Go,and I'm using Bazil library. I have successfully implemented a simple read-only fs, and I now want to be able to call mkdir inside any existing directory to make a new one.
I have made sure that all the existing directories are writable,(attr.Mode = os.ModeDir | 0777).
Right now I have just added the function:
func (d Dir) MkDir(ctx context.Context, req *fuse.MkdirRequest) (fs.Node, error) {
dir := &Dir{name: req.Name, files: 0, inode: 10 /*a random inode*/,mode: os.FileMode(0777),nextdir: nil, nextfile: nil}
d.nextdir = dir
return dir, nil
}
in my own implementation of the hello fs example of Bazil's library. But that doesn't seem to make any difference.
When I call mkdir new_dir_name from the terminal, I get the error: "mkdir: cannot create directory ‘new_dir_name’: Operation not permitted", even though I have added the mkdir function.
Any insights as to why this is happening, and what else should I add to my code to make this working would be great. Also, this is my first stackoverflow question, so I'm sorry if I didn't ask in a clear way.
the correct function for make directory like this
func (d *Dir) Mkdir(ctx context.Context, req *fuse.MkdirRequest) (fs.Node, error) {
}
I am creating Go command-line app and I need to generate some stuff in the current directory (the directory which the user execute the commands from)
to get the pwd I need to use
os.Getwd()
but this give me path like
/Users/s05333/go/src/appcmd
and I need path like this
/Users/s05333/go/src/
which option I've in this case?
Omit the last string after the / or there is better way in Go?
Take a look at the filepath package, particularly filepath.Dir:
wd,err := os.Getwd()
if err != nil {
panic(err)
}
parent := filepath.Dir(wd)
Per the docs:
Dir returns all but the last element of path, typically the path's directory.
Another option is the path package:
package main
import "path"
func main() {
s := "/Users/s05333/go/src/appcmd"
t := path.Dir(s)
println(t == "/Users/s05333/go/src")
}
https://golang.org/pkg/path#Dir
I've some command line program which I need to read files from parent folder, I mean
-parentDir
-- myproject
--- cmd
----main.go
--otherdir
-file.json
As you can see otherdir is like sibling to myproject and I need from my main.go read the file.json
what I've tried is like following
func visit(path string, f os.FileInfo, err error) error {
fmt.Printf("Visited: %s\n", path)
return nil
}
func main() {
flag.Parse()
root := flag.Arg(0)
err := filepath.Walk(root, visit)
fmt.Printf("filepath.Walk() returned %v\n", err)
}
I've also try to provide args(-1) which doesnt help...
Any idea how from command line program I can read some files that on level up from my executable ?
I've also tried with
import "github.com/mitchellh/go-homedir"
func Path(path string) (error, string) {
home, err := homedir.Dir()
}
this give the root directory which doesnt help either...
It doesn't matter where the binary is, it matters what the working directory is (the directory you're in when you execute the program). All relative paths will be relative to the current working directory. So, if you're executing from myproject, you'd use something like ../ as the root path to Walk.
That said, I would highly recommend you make the path configurable, rather than assuming the binary will always be executed from some particular location within the source tree.
I have the following structure:
project/
docs/
index.html
root.html
I'm trying to iterate through this project structure so that I can read the contents of each file to process them. So I want to say "search through the directory project", then it will search through all the files, and only the first level of directories and their files, so if there was another directory with a file inside of docs/, it would ignore it.
Currently, I've tried to accomplish this with the "path/filepath" library:
func traverse(path string, file os.FileInfo, err error) error {
if file, err := os.Open(file.Name()); err == nil {
defer file.Close()
if fileStat, err := file.Stat(); err == nil {
switch mode := fileStat.Mode(); {
case mode.IsDir():
fmt.Println("it be a directory! lets traverse", file.Name())
filepath.Walk(file.Name(), traverse)
case mode.IsRegular():
fmt.Println("the thingy ", file.Name(), " is a file")
}
} else {
return errors.New("failed to check status")
}
}
return errors.New("failed 2 open file/dir?")
}
Then I call it from here:
if err := filepath.Walk("project/", traverse); err != nil {
setupErr("%s", err)
}
Note that I run this executable relative to my test directory, so it's finding the directory okay. My problem is actually when I run it, I get the following:
it be a directory! lets traverse project
it be a directory! lets traverse project
# ^ printed about 20 more times ^
failed 2 open file/dir?
I think my recursion is a little off, and it's not changing into the directory perhaps? Any ideas, if you need any more information just ask and I'll update.
First, it looks like what you want do to contradicts with the code you have. You wrote:
So I want to say "search through the directory project", then it will search through all the files, and only the first level of directories and their files, so if there was another directory with a file inside of docs/, it would ignore it.
Does it mean that you want to iterate only two levels of directories (current and one below) and ignore the rest?
If so then you do not need a recursion, just a simple loop that executes search function over the files within the current directory and for every its subdirectory.
The code that you have walks over the filesystem directory subtree.
Basically, filepath.Walk that you use should do it for you. So you either implement recursive walking or use Walk, but not both.
Second, the recursion is implemented incorrectly in your code. It missing iterating over the directories.
So the code that prints the file names in the current directory and its subdirectories (but not further) is:
package main
import (
"fmt"
"io/ioutil"
)
func main() {
items, _ := ioutil.ReadDir(".")
for _, item := range items {
if item.IsDir() {
subitems, _ := ioutil.ReadDir(item.Name())
for _, subitem := range subitems {
if !subitem.IsDir() {
// handle file there
fmt.Println(item.Name() + "/" + subitem.Name())
}
}
} else {
// handle file there
fmt.Println(item.Name())
}
}
}
Walk walks the file tree rooted at root, calling walkFn for each file
or directory in the tree, including root. All errors that arise
visiting files and directories are filtered by walkFn. The files are
walked in lexical order, which makes the output deterministic but
means that for very large directories Walk can be inefficient. Walk
does not follow symbolic links.