I am trying to write a simple Go program which connects to an FTP server, list the files in a specified directory and pulls them.
The code is this:
package main
import (
"bytes"
"fmt"
"github.com/secsy/goftp"
"io/ioutil"
"log"
"os"
"path"
"time"
)
func main() {
config := goftp.Config{
User: "anonymous",
Password: "root#local.me",
ConnectionsPerHost: 21,
Timeout: 10 * time.Second,
Logger: os.Stderr,
}
// Connecting to the server
client, dailErr := goftp.DialConfig(config, "ftp.example.com")
if dailErr != nil {
log.Fatal(dailErr)
panic(dailErr)
}
// setting the search directory
dir := "/downloads/"
files, err := client.ReadDir(dir)
if err != nil {
for _, file := range files {
if file.IsDir() {
path.Join(dir, file.Name())
} else {
fmt.Println("the file is %s", file.Name())
}
}
}
// this section works , I am setting a file name and I can pull it
// if I mark the search part
ret_file := "example.PDF"
fmt.Println("Retrieving file: ", ret_file)
buf := new(bytes.Buffer)
fullPathFile := dir + ret_file
rferr := client.Retrieve(fullPathFile, buf)
if rferr != nil {
panic(rferr)
}
fmt.Println("writing data to file", ret_file)
fmt.Println("Opening file", ret_file, "for writing")
w, _ := ioutil.ReadAll(buf)
ferr := ioutil.WriteFile(ret_file, w, 0644)
if ferr != nil {
log.Fatal(ferr)
panic(ferr)
} else {
fmt.Println("Writing", ret_file, " completed")
}
}
For some reason I am getting an error on the ReadDir function.
I need to grab the files names so I can download them.
You're attempting to loop through files when ReadDir() returns an error. That will never work, as any time an error is returned files is nil.
This is pretty standard behavior and can be confirmed by reading the implementation of ReadDir().
I'm guessing you may have used the the example from the project used to demonstrate ReadDir() as a starting point. Within the example, the error handling is involved because it's deciding whether or not to continue walking the directory tree. However, note that when ReadDir() returns an error that doesn't result in stopping the program, the subsequent for loop is a no-op, since files is nil.
Here's a small program that demonstrates successfully using the results of Readdir() in a straightforward manner:
package main
import (
"fmt"
"github.com/secsy/goftp"
)
const (
ftpServerURL = "ftp.us.debian.org"
ftpServerPath = "/debian/"
)
func main() {
client, err := goftp.Dial(ftpServerURL)
if err != nil {
panic(err)
}
files, err := client.ReadDir(ftpServerPath)
if err != nil {
panic(err)
}
for _, file := range files {
fmt.Println(file.Name())
}
}
It outputs (which matches the current listing at http://ftp.us.debian.org/debian/):
$ go run goftp-test.go
README
README.CD-manufacture
README.html
README.mirrors.html
README.mirrors.txt
dists
doc
extrafiles
indices
ls-lR.gz
pool
project
tools
zzz-dists
Related
I have downloaded over 500 Gb of data to a single directory off of AWS.
Whenever I try to access that directory, the command line hangs and doesn't show me anything.
I'm trying to run some code that will interact with the files by printing out the path of each file but the command line hangs and then exits the program.
The program definitely starts execution because "Printing file path's" gets displayed to the console.
func main() {
fmt.Println("Printing file path's")
err := filepath.Walk(source,
func(fpath string, info os.FileInfo, err error) {
if !info.IsDir() && file path.Ext(fpath)==".txt" {
fmt.Println(fpath)
}
}
}
}
How should I handle the situation of being able to view all the files in the command line and why is this program not working?
UPDATE:
By using
files, err := dir.Readdir(10)
if err == io.EOF {
break
}
I was able to snap up the first 10 folders/files in the directory.
Using a loop I could keep doing this until I hit the end of the directory.
This doesn't rely on ordering the files/folders as does the walk function and so its more efficient.
The possible performance issue with filepath.Walk is clearly documented:
The files are walked in lexical order, which makes the output deterministic but means that for very large directories Walk can be inefficient.
Use os.File.Readdir to iterate files in filesystem order:
Readdir reads the contents of the directory associated with file and returns a slice of up to n FileInfo values, as would be returned by Lstat, in directory order. Subsequent calls on the same file will yield further FileInfos.
package main
import (
"fmt"
"io"
"log"
"os"
"time"
)
func main() {
dir, err := os.Open("/tmp")
if err != nil {
log.Fatal(err)
}
for {
files, err := dir.Readdir(10)
if err == io.EOF {
break
}
if err != nil {
log.Fatal(err)
}
for _, fi := range files {
classifier := ""
if fi.IsDir() {
classifier = "/"
}
fmt.Printf("%v %12d %s%s\n",
fi.ModTime().UTC().Truncate(time.Second),
fi.Size(),
fi.Name(), classifier,
)
}
}
}
I've been trying to figure out how to simply list the files and folders in a single directory in Go.
I've found filepath.Walk, but it goes into sub-directories automatically, which I don't want. All of my other searches haven't turned anything better up.
I'm sure that this functionality exists, but it's been really hard to find. Let me know if anyone knows where I should look. Thanks.
You can try using the ReadDir function in the os package. Per the docs:
ReadDir reads the named directory, returning all its directory entries sorted by filename.
The resulting slice contains os.DirEntry types, which provide the methods listed here. Here is a basic example that lists the name of everything in the current directory (folders are included but not specially marked - you can check if an item is a folder by using the IsDir() method):
package main
import (
"fmt"
"os"
"log"
)
func main() {
entries, err := os.ReadDir("./")
if err != nil {
log.Fatal(err)
}
for _, e := range entries {
fmt.Println(e.Name())
}
}
We can get a list of files inside a folder on the file system using various golang standard library functions.
filepath.Walk
ioutil.ReadDir
os.File.Readdir
package main
import (
"fmt"
"io/ioutil"
"log"
"os"
"path/filepath"
)
func main() {
var (
root string
files []string
err error
)
root := "/home/manigandan/golang/samples"
// filepath.Walk
files, err = FilePathWalkDir(root)
if err != nil {
panic(err)
}
// ioutil.ReadDir
files, err = IOReadDir(root)
if err != nil {
panic(err)
}
//os.File.Readdir
files, err = OSReadDir(root)
if err != nil {
panic(err)
}
for _, file := range files {
fmt.Println(file)
}
}
Using filepath.Walk
The path/filepath package provides a handy way to scan all the files
in a directory, it will automatically scan each sub-directories in the
directory.
func FilePathWalkDir(root string) ([]string, error) {
var files []string
err := filepath.Walk(root, func(path string, info os.FileInfo, err error) error {
if !info.IsDir() {
files = append(files, path)
}
return nil
})
return files, err
}
Using ioutil.ReadDir
ioutil.ReadDir reads the directory named by dirname and returns a
list of directory entries sorted by filename.
func IOReadDir(root string) ([]string, error) {
var files []string
fileInfo, err := ioutil.ReadDir(root)
if err != nil {
return files, err
}
for _, file := range fileInfo {
files = append(files, file.Name())
}
return files, nil
}
Using os.File.Readdir
Readdir reads the contents of the directory associated with file and
returns a slice of up to n FileInfo values, as would be returned by
Lstat, in directory order. Subsequent calls on the same file will
yield further FileInfos.
func OSReadDir(root string) ([]string, error) {
var files []string
f, err := os.Open(root)
if err != nil {
return files, err
}
fileInfo, err := f.Readdir(-1)
f.Close()
if err != nil {
return files, err
}
for _, file := range fileInfo {
files = append(files, file.Name())
}
return files, nil
}
Benchmark results.
Get more details on this Blog Post
Even simpler, use path/filepath:
package main
import (
"fmt"
"log"
"path/filepath"
)
func main() {
files, err := filepath.Glob("*")
if err != nil {
log.Fatal(err)
}
fmt.Println(files) // contains a list of all files in the current directory
}
Starting with Go 1.16, you can use the os.ReadDir function.
func ReadDir(name string) ([]DirEntry, error)
It reads a given directory and returns a DirEntry slice that contains the directory entries sorted by filename.
It's an optimistic function, so that, when an error occurs while reading the directory entries, it tries to return you a slice with the filenames up to the point before the error.
package main
import (
"fmt"
"log"
"os"
)
func main() {
files, err := os.ReadDir(".")
if err != nil {
log.Fatal(err)
}
for _, file := range files {
fmt.Println(file.Name())
}
}
Of interest: Go 1.17 (Q3 2021) includes fs.FileInfoToDirEntry():
func FileInfoToDirEntry(info FileInfo) DirEntry
FileInfoToDirEntry returns a DirEntry that returns information from info.
If info is nil, FileInfoToDirEntry returns nil.
Background
Go 1.16 (Q1 2021) will propose, with CL 243908 and CL 243914 , the ReadDir function, based on the FS interface:
// An FS provides access to a hierarchical file system.
//
// The FS interface is the minimum implementation required of the file system.
// A file system may implement additional interfaces,
// such as fsutil.ReadFileFS, to provide additional or optimized functionality.
// See io/fsutil for details.
type FS interface {
// Open opens the named file.
//
// When Open returns an error, it should be of type *PathError
// with the Op field set to "open", the Path field set to name,
// and the Err field describing the problem.
//
// Open should reject attempts to open names that do not satisfy
// ValidPath(name), returning a *PathError with Err set to
// ErrInvalid or ErrNotExist.
Open(name string) (File, error)
}
That allows for "os: add ReadDir method for lightweight directory reading":
See commit a4ede9f:
// ReadDir reads the contents of the directory associated with the file f
// and returns a slice of DirEntry values in directory order.
// Subsequent calls on the same file will yield later DirEntry records in the directory.
//
// If n > 0, ReadDir returns at most n DirEntry records.
// In this case, if ReadDir returns an empty slice, it will return an error explaining why.
// At the end of a directory, the error is io.EOF.
//
// If n <= 0, ReadDir returns all the DirEntry records remaining in the directory.
// When it succeeds, it returns a nil error (not io.EOF).
func (f *File) ReadDir(n int) ([]DirEntry, error)
// A DirEntry is an entry read from a directory (using the ReadDir method).
type DirEntry interface {
// Name returns the name of the file (or subdirectory) described by the entry.
// This name is only the final element of the path, not the entire path.
// For example, Name would return "hello.go" not "/home/gopher/hello.go".
Name() string
// IsDir reports whether the entry describes a subdirectory.
IsDir() bool
// Type returns the type bits for the entry.
// The type bits are a subset of the usual FileMode bits, those returned by the FileMode.Type method.
Type() os.FileMode
// Info returns the FileInfo for the file or subdirectory described by the entry.
// The returned FileInfo may be from the time of the original directory read
// or from the time of the call to Info. If the file has been removed or renamed
// since the directory read, Info may return an error satisfying errors.Is(err, ErrNotExist).
// If the entry denotes a symbolic link, Info reports the information about the link itself,
// not the link's target.
Info() (FileInfo, error)
}
src/os/os_test.go#testReadDir() illustrates its usage:
file, err := Open(dir)
if err != nil {
t.Fatalf("open %q failed: %v", dir, err)
}
defer file.Close()
s, err2 := file.ReadDir(-1)
if err2 != nil {
t.Fatalf("ReadDir %q failed: %v", dir, err2)
}
Ben Hoyt points out in the comments to Go 1.16 os.ReadDir:
os.ReadDir(path string) ([]os.DirEntry, error), which you'll be able to call directly without the Open dance.
So you can probably shorten this to just os.ReadDir, as that's the concrete function most people will call.
See commit 3d913a9 (Dec. 2020):
os: add ReadFile, WriteFile, CreateTemp (was TempFile), MkdirTemp (was TempDir) from io/ioutil
io/ioutil was a poorly defined collection of helpers.
Proposal #40025 moved out the generic I/O helpers to io.
This CL for proposal #42026 moves the OS-specific helpers to os,
making the entire io/ioutil package deprecated.
os.ReadDir returns []DirEntry, in contrast to ioutil.ReadDir's []FileInfo.
(Providing a helper that returns []DirEntry is one of the primary motivations for this change.)
ioutil.ReadDir is a good find, but if you click and look at the source you see that it calls the method Readdir of os.File. If you are okay with the directory order and don't need the list sorted, then this Readdir method is all you need.
From your description, what you probably want is os.Readdirnames.
func (f *File) Readdirnames(n int) (names []string, err error)
Readdirnames reads the contents of the directory associated with file and returns a slice of up to n names of files in the directory, in directory order. Subsequent calls on the same file will yield further names.
...
If n <= 0, Readdirnames returns all the names from the directory in a single slice.
Snippet:
file, err := os.Open(path)
if err != nil {
return err
}
defer file.Close()
names, err := file.Readdirnames(0)
if err != nil {
return err
}
fmt.Println(names)
Credit to SquattingSlavInTracksuit's comment; I'd have suggested promoting their comment to an answer if I could.
A complete example of printing all the files in a directory recursively using Readdirnames
package main
import (
"fmt"
"os"
)
func main() {
path := "/path/to/your/directory"
err := readDir(path)
if err != nil {
panic(err)
}
}
func readDir(path string) error {
file, err := os.Open(path)
if err != nil {
return err
}
defer file.Close()
names, _ := file.Readdirnames(0)
for _, name := range names {
filePath := fmt.Sprintf("%v/%v", path, name)
file, err := os.Open(filePath)
if err != nil {
return err
}
defer file.Close()
fileInfo, err := file.Stat()
if err != nil {
return err
}
fmt.Println(filePath)
if fileInfo.IsDir() {
readDir(filePath)
}
}
return nil
}
I'm new to Golang, starting out with some examples. Currently, what I'm trying to do is reading a file line by line and replace it with another string in case it meets a certain condition.
The file is use for testing purposes contains four lines:
one
two
three
four
The code working on that file looks like this:
func main() {
file, err := os.OpenFile("test.txt", os.O_RDWR, 0666)
if err != nil {
panic(err)
}
reader := bufio.NewReader(file)
for {
fmt.Print("Try to read ...\n")
pos,_ := file.Seek(0, 1)
log.Printf("Position in file is: %d", pos)
bytes, _, _ := reader.ReadLine()
if (len(bytes) == 0) {
break
}
lineString := string(bytes)
if(lineString == "two") {
file.Seek(int64(-(len(lineString))), 1)
file.WriteString("This is a test.")
}
fmt.Printf(lineString + "\n")
}
file.Close()
}
As you can see in the code snippet, I want to replace the string "two" with "This is a test" as soon as this string is read from the file.
In order to get the current position within the file I use Go's Seek method.
However, what happens is that always the last line gets replaced by This is a test, making the file looking like this:
one
two
three
This is a test
Examining the output of the print statement which writes the current file position to the terminal, I get that kind of output after the first line has been read:
2016/12/28 21:10:31 Try to read ...
2016/12/28 21:10:31 Position in file is: 19
So after the first read, the position cursor already points to the end of my file, which explains why the new string gets appended to the end. Does anyone understand what is happening here or rather what is causing that behavior?
The Reader is not controller by the file.Seek. You have declared the reader as: reader := bufio.NewReader(file) and then you read one line at a time bytes, _, _ := reader.ReadLine() however the file.Seek does not change the position that the reader is reading.
Suggest you read about the ReadSeeker in the docs and switch over to using that. Also there is an example using the SectionReader.
Aside from the incorrect seek usage, the difficulty is that the line you're replacing isn't the same length as the replacement. The standard approach is to create a new (temporary) file with the modifications. Assuming that is successful, replace the original file with the new one.
package main
import (
"bufio"
"io"
"io/ioutil"
"log"
"os"
)
func main() {
// file we're modifying
name := "text.txt"
// open original file
f, err := os.Open(name)
if err != nil {
log.Fatal(err)
}
defer f.Close()
// create temp file
tmp, err := ioutil.TempFile("", "replace-*")
if err != nil {
log.Fatal(err)
}
defer tmp.Close()
// replace while copying from f to tmp
if err := replace(f, tmp); err != nil {
log.Fatal(err)
}
// make sure the tmp file was successfully written to
if err := tmp.Close(); err != nil {
log.Fatal(err)
}
// close the file we're reading from
if err := f.Close(); err != nil {
log.Fatal(err)
}
// overwrite the original file with the temp file
if err := os.Rename(tmp.Name(), name); err != nil {
log.Fatal(err)
}
}
func replace(r io.Reader, w io.Writer) error {
// use scanner to read line by line
sc := bufio.NewScanner(r)
for sc.Scan() {
line := sc.Text()
if line == "two" {
line = "This is a test."
}
if _, err := io.WriteString(w, line+"\n"); err != nil {
return err
}
}
return sc.Err()
}
For more complex replacements, I've implemented a package which can replace regular expression matches. https://github.com/icholy/replace
import (
"io"
"regexp"
"github.com/icholy/replace"
"golang.org/x/text/transform"
)
func replace2(r io.Reader, w io.Writer) error {
// compile multi-line regular expression
re := regexp.MustCompile(`(?m)^two$`)
// create replace transformer
tr := replace.RegexpString(re, "This is a test.")
// copy while transforming
_, err := io.Copy(w, transform.NewReader(r, tr))
return err
}
OS package has Expand function which I believe can be used to solve similar problem.
Explanation:
file.txt
one
two
${num}
four
main.go
package main
import (
"fmt"
"os"
)
var FILENAME = "file.txt"
func main() {
file, err := os.ReadFile(FILENAME)
if err != nil {
panic(err)
}
mapper := func(placeholderName string) string {
switch placeholderName {
case "num":
return "three"
}
return ""
}
fmt.Println(os.Expand(string(file), mapper))
}
output
one
two
three
four
Additionally, you may create a config (yml or json) and
populate that data in the map that can be used as a lookup table to store placeholders as well as their replacement strings and modify mapper part to use this table to lookup placeholders from input file.
e.g map will look like this,
table := map[string]string {
"num": "three"
}
mapper := func(placeholderName string) string {
if val, ok := table[placeholderName]; ok {
return val
}
return ""
}
References:
os.Expand documentation: https://pkg.go.dev/os#Expand
Playground
I am trying to create a set of nested directories from a Go executable such as 'dir1/dir2/dir3'. I have succeeded in creating a single directory with this line:
os.Mkdir("." + string(filepath.Separator) + c.Args().First(),0777);
However, I have no idea how to approach creating a predetermined nested set of directories inside of that directory.
os.Mkdir is used to create a single directory. To create a folder path, instead try using:
os.MkdirAll(folderPath, os.ModePerm)
Go documentation
func MkdirAll(path string, perm FileMode) error
MkdirAll creates a directory named path, along with any necessary parents, and returns nil, or else returns an error. The permission bits perm are used for all directories that MkdirAll creates. If path is already a directory, MkdirAll does nothing and returns nil.
Edit:
Updated to correctly use os.ModePerm instead.
For concatenation of file paths, use package path/filepath as described in #Chris' answer.
This way you don't have to use any magic numbers:
os.MkdirAll(newPath, os.ModePerm)
Also, rather than using + to create paths, you can use:
import "path/filepath"
path := filepath.Join(someRootPath, someSubPath)
The above uses the correct separators automatically on each platform for you.
If the issue is to create all the necessary parent directories, you could consider using os.MkDirAll()
MkdirAll creates a directory named path, along with any necessary parents, and returns nil, or else returns an error.
The path_test.go is a good illustration on how to use it:
func TestMkdirAll(t *testing.T) {
tmpDir := TempDir()
path := tmpDir + "/_TestMkdirAll_/dir/./dir2"
err := MkdirAll(path, 0777)
if err != nil {
t.Fatalf("MkdirAll %q: %s", path, err)
}
defer RemoveAll(tmpDir + "/_TestMkdirAll_")
...
}
(Make sure to specify a sensible permission value, as mentioned in this answer)
This is one alternative for achieving the same but it avoids race condition caused by having two distinct "check ..and.. create" operations.
package main
import (
"fmt"
"os"
)
func main() {
if err := ensureDir("/test-dir"); err != nil {
fmt.Println("Directory creation failed with error: " + err.Error())
os.Exit(1)
}
// Proceed forward
}
func ensureDir(dirName string) error {
err := os.MkdirAll(dirName, os.ModeDir)
if err == nil || os.IsExist(err) {
return nil
} else {
return err
}
}
An utility method like the following can be used to solve this.
import (
"os"
"path/filepath"
"log"
)
func ensureDir(fileName string) {
dirName := filepath.Dir(fileName)
if _, serr := os.Stat(dirName); serr != nil {
merr := os.MkdirAll(dirName, os.ModePerm)
if merr != nil {
panic(merr)
}
}
}
func main() {
_, cerr := os.Create("a/b/c/d.txt")
if cerr != nil {
log.Fatal("error creating a/b/c", cerr)
}
log.Println("created file in a sub-directory.")
}
I've been trying to figure out how to simply list the files and folders in a single directory in Go.
I've found filepath.Walk, but it goes into sub-directories automatically, which I don't want. All of my other searches haven't turned anything better up.
I'm sure that this functionality exists, but it's been really hard to find. Let me know if anyone knows where I should look. Thanks.
You can try using the ReadDir function in the os package. Per the docs:
ReadDir reads the named directory, returning all its directory entries sorted by filename.
The resulting slice contains os.DirEntry types, which provide the methods listed here. Here is a basic example that lists the name of everything in the current directory (folders are included but not specially marked - you can check if an item is a folder by using the IsDir() method):
package main
import (
"fmt"
"os"
"log"
)
func main() {
entries, err := os.ReadDir("./")
if err != nil {
log.Fatal(err)
}
for _, e := range entries {
fmt.Println(e.Name())
}
}
We can get a list of files inside a folder on the file system using various golang standard library functions.
filepath.Walk
ioutil.ReadDir
os.File.Readdir
package main
import (
"fmt"
"io/ioutil"
"log"
"os"
"path/filepath"
)
func main() {
var (
root string
files []string
err error
)
root := "/home/manigandan/golang/samples"
// filepath.Walk
files, err = FilePathWalkDir(root)
if err != nil {
panic(err)
}
// ioutil.ReadDir
files, err = IOReadDir(root)
if err != nil {
panic(err)
}
//os.File.Readdir
files, err = OSReadDir(root)
if err != nil {
panic(err)
}
for _, file := range files {
fmt.Println(file)
}
}
Using filepath.Walk
The path/filepath package provides a handy way to scan all the files
in a directory, it will automatically scan each sub-directories in the
directory.
func FilePathWalkDir(root string) ([]string, error) {
var files []string
err := filepath.Walk(root, func(path string, info os.FileInfo, err error) error {
if !info.IsDir() {
files = append(files, path)
}
return nil
})
return files, err
}
Using ioutil.ReadDir
ioutil.ReadDir reads the directory named by dirname and returns a
list of directory entries sorted by filename.
func IOReadDir(root string) ([]string, error) {
var files []string
fileInfo, err := ioutil.ReadDir(root)
if err != nil {
return files, err
}
for _, file := range fileInfo {
files = append(files, file.Name())
}
return files, nil
}
Using os.File.Readdir
Readdir reads the contents of the directory associated with file and
returns a slice of up to n FileInfo values, as would be returned by
Lstat, in directory order. Subsequent calls on the same file will
yield further FileInfos.
func OSReadDir(root string) ([]string, error) {
var files []string
f, err := os.Open(root)
if err != nil {
return files, err
}
fileInfo, err := f.Readdir(-1)
f.Close()
if err != nil {
return files, err
}
for _, file := range fileInfo {
files = append(files, file.Name())
}
return files, nil
}
Benchmark results.
Get more details on this Blog Post
Even simpler, use path/filepath:
package main
import (
"fmt"
"log"
"path/filepath"
)
func main() {
files, err := filepath.Glob("*")
if err != nil {
log.Fatal(err)
}
fmt.Println(files) // contains a list of all files in the current directory
}
Starting with Go 1.16, you can use the os.ReadDir function.
func ReadDir(name string) ([]DirEntry, error)
It reads a given directory and returns a DirEntry slice that contains the directory entries sorted by filename.
It's an optimistic function, so that, when an error occurs while reading the directory entries, it tries to return you a slice with the filenames up to the point before the error.
package main
import (
"fmt"
"log"
"os"
)
func main() {
files, err := os.ReadDir(".")
if err != nil {
log.Fatal(err)
}
for _, file := range files {
fmt.Println(file.Name())
}
}
Of interest: Go 1.17 (Q3 2021) includes fs.FileInfoToDirEntry():
func FileInfoToDirEntry(info FileInfo) DirEntry
FileInfoToDirEntry returns a DirEntry that returns information from info.
If info is nil, FileInfoToDirEntry returns nil.
Background
Go 1.16 (Q1 2021) will propose, with CL 243908 and CL 243914 , the ReadDir function, based on the FS interface:
// An FS provides access to a hierarchical file system.
//
// The FS interface is the minimum implementation required of the file system.
// A file system may implement additional interfaces,
// such as fsutil.ReadFileFS, to provide additional or optimized functionality.
// See io/fsutil for details.
type FS interface {
// Open opens the named file.
//
// When Open returns an error, it should be of type *PathError
// with the Op field set to "open", the Path field set to name,
// and the Err field describing the problem.
//
// Open should reject attempts to open names that do not satisfy
// ValidPath(name), returning a *PathError with Err set to
// ErrInvalid or ErrNotExist.
Open(name string) (File, error)
}
That allows for "os: add ReadDir method for lightweight directory reading":
See commit a4ede9f:
// ReadDir reads the contents of the directory associated with the file f
// and returns a slice of DirEntry values in directory order.
// Subsequent calls on the same file will yield later DirEntry records in the directory.
//
// If n > 0, ReadDir returns at most n DirEntry records.
// In this case, if ReadDir returns an empty slice, it will return an error explaining why.
// At the end of a directory, the error is io.EOF.
//
// If n <= 0, ReadDir returns all the DirEntry records remaining in the directory.
// When it succeeds, it returns a nil error (not io.EOF).
func (f *File) ReadDir(n int) ([]DirEntry, error)
// A DirEntry is an entry read from a directory (using the ReadDir method).
type DirEntry interface {
// Name returns the name of the file (or subdirectory) described by the entry.
// This name is only the final element of the path, not the entire path.
// For example, Name would return "hello.go" not "/home/gopher/hello.go".
Name() string
// IsDir reports whether the entry describes a subdirectory.
IsDir() bool
// Type returns the type bits for the entry.
// The type bits are a subset of the usual FileMode bits, those returned by the FileMode.Type method.
Type() os.FileMode
// Info returns the FileInfo for the file or subdirectory described by the entry.
// The returned FileInfo may be from the time of the original directory read
// or from the time of the call to Info. If the file has been removed or renamed
// since the directory read, Info may return an error satisfying errors.Is(err, ErrNotExist).
// If the entry denotes a symbolic link, Info reports the information about the link itself,
// not the link's target.
Info() (FileInfo, error)
}
src/os/os_test.go#testReadDir() illustrates its usage:
file, err := Open(dir)
if err != nil {
t.Fatalf("open %q failed: %v", dir, err)
}
defer file.Close()
s, err2 := file.ReadDir(-1)
if err2 != nil {
t.Fatalf("ReadDir %q failed: %v", dir, err2)
}
Ben Hoyt points out in the comments to Go 1.16 os.ReadDir:
os.ReadDir(path string) ([]os.DirEntry, error), which you'll be able to call directly without the Open dance.
So you can probably shorten this to just os.ReadDir, as that's the concrete function most people will call.
See commit 3d913a9 (Dec. 2020):
os: add ReadFile, WriteFile, CreateTemp (was TempFile), MkdirTemp (was TempDir) from io/ioutil
io/ioutil was a poorly defined collection of helpers.
Proposal #40025 moved out the generic I/O helpers to io.
This CL for proposal #42026 moves the OS-specific helpers to os,
making the entire io/ioutil package deprecated.
os.ReadDir returns []DirEntry, in contrast to ioutil.ReadDir's []FileInfo.
(Providing a helper that returns []DirEntry is one of the primary motivations for this change.)
ioutil.ReadDir is a good find, but if you click and look at the source you see that it calls the method Readdir of os.File. If you are okay with the directory order and don't need the list sorted, then this Readdir method is all you need.
From your description, what you probably want is os.Readdirnames.
func (f *File) Readdirnames(n int) (names []string, err error)
Readdirnames reads the contents of the directory associated with file and returns a slice of up to n names of files in the directory, in directory order. Subsequent calls on the same file will yield further names.
...
If n <= 0, Readdirnames returns all the names from the directory in a single slice.
Snippet:
file, err := os.Open(path)
if err != nil {
return err
}
defer file.Close()
names, err := file.Readdirnames(0)
if err != nil {
return err
}
fmt.Println(names)
Credit to SquattingSlavInTracksuit's comment; I'd have suggested promoting their comment to an answer if I could.
A complete example of printing all the files in a directory recursively using Readdirnames
package main
import (
"fmt"
"os"
)
func main() {
path := "/path/to/your/directory"
err := readDir(path)
if err != nil {
panic(err)
}
}
func readDir(path string) error {
file, err := os.Open(path)
if err != nil {
return err
}
defer file.Close()
names, _ := file.Readdirnames(0)
for _, name := range names {
filePath := fmt.Sprintf("%v/%v", path, name)
file, err := os.Open(filePath)
if err != nil {
return err
}
defer file.Close()
fileInfo, err := file.Stat()
if err != nil {
return err
}
fmt.Println(filePath)
if fileInfo.IsDir() {
readDir(filePath)
}
}
return nil
}