Making a filename truncator in go - go

I am new to Go, and I've been trying to make a program that truncates filenames after (and including) the character "-" of all files in its directory. It's not working, and I don't know where I'm going wrong.
func changeFilename() {
file, err := os.Open(".")
if err != nil {
log.Fatalf("failed opening directory: %s", err)
}
defer file.Close()
oldNames, _ := file.Readdirnames(0)
var s string
for _, i := range oldNames {
for _, j := range i {
for j != '-' {
k := strconv.QuoteRune(j)
s += k
j++
}
}
err := os.Rename(i, s)
if err != nil {
log.Fatal("failed to rename: %s", err)
}
}
}
I expected filenames in the test directory to be changed when I ran the executable, but they didn't. The program builds without any error messages.

Fixing your logic to correspond to your description of the problem:
package main
import (
"fmt"
"log"
"os"
"path/filepath"
)
// changeFilename truncates filenames after (and including) the character "-"
// of all files in the directory. File extensions are preserved.
func changeFilename(dir string) {
file, err := os.Open(dir)
if err != nil {
log.Fatalf("failed opening directory: %s", err)
}
defer file.Close()
fis, err := file.Readdir(0)
if err != nil {
log.Fatalf("failed reading directory: %s", err)
}
for _, fi := range fis {
if !fi.Mode().IsRegular() {
continue
}
old := fi.Name()
ext := filepath.Ext(old)
for i, r := range old[:len(old)-len(ext)] {
if r == '-' {
new := old[:i] + ext
err := os.Rename(old, new)
if err != nil {
log.Fatalf("failed to rename: %s", err)
}
fmt.Printf("%q %q\n", old, new)
break
}
}
}
}
func main() {
changeFilename(".")
}
Output:
"trunc-ate.typ" "trunc.typ"
"hyp-hen.ext" "hyp.ext"

Your s variable should be moved into the outermost loop since its lifetime should be tied to a single oldName. The innermost loop has no exit condition once a non - character is encountered. That should be an if statement with a corresponding break if a - is found. As for building up the new filename, you ought to use += string(j) instead of quoting the rune.
func changeFilename() {
file, err := os.Open(".")
if err != nil {
log.Fatalf("failed opening directory: %s", err)
}
defer file.Close()
oldNames, _ := file.Readdirnames(0)
for _, i := range oldNames {
var s string
for _, j := range i {
if j != '-' {
s += string(j)
} else {
break
}
}
err := os.Rename(i, s)
if err != nil {
log.Fatal("failed to rename: %s", err)
}
}
}

Related

Refactor function that reading files from folders tree

I have the following structure of my project like that:
main.go
/data
/1
1.json
/2
2.json
/3
3.json
And I have the following function that works well
const (
data = "./data"
)
type Hello struct {
S string `json:"hello"`
}
func myFunc() {
folders, err := os.ReadDir(data)
if err != nil {
log.Fatalf("read dir: %s", err)
}
for _, folder := range folders {
path := filepath.Join(data, folder.Name())
files, err := os.ReadDir(path)
if err != nil {
log.Fatalf("read dir: %s", err)
}
for _, file := range files {
pathToFile := filepath.Join(path, file.Name())
b, err := os.ReadFile(pathToFile)
if err != nil {
log.Println(err)
}
h := Hello{}
err = json.Unmarshal(b, &h)
if err != nil {
log.Printf("unmarshall erro: %w\n", err)
}
fmt.Println(h)
}
}
}
I should refactor my data tree and divide data folder into two additional folders. So my structure should be like this:
main.go
/data
/dog
/1
1.json
/2
2.json
/cat
/3
3.json
So I need to fix my function according to this structure of the files.
I don't want to hardcode and make the solution with additional for loop. But I think it is not good solution.
func myfunc() {
subFolders, err := os.ReadDir(data)
if err != nil {
log.Fatalf("read dir: %s", err)
}
for _, subFolder := range subFolders {
pathToSubFolder := filepath.Join(data, subFolder.Name())
folders, err := os.ReadDir(pathToSubFolder)
if err != nil {
log.Fatalf("read dir1: %s", err)
}
for _, folder := range folders {
path := filepath.Join(pathToSubFolder, folder.Name())
files, err := os.ReadDir(path)
if err != nil {
log.Fatalf("read dir2: %s", err)
}
for _, file := range files {
pathToFile := filepath.Join(path, file.Name())
b, err := os.ReadFile(pathToFile)
if err != nil {
log.Println(err)
}
h := Hello{}
err = json.Unmarshal(b, &h)
if err != nil {
log.Printf("unmarshall erro: %w\n", err)
}
fmt.Println(h)
}
}
}
}
This soultion works but can anyone suggest another solution that could be better?
Thanks in advance

binary-to-text encoding that replace a byte into a predefined unique string in golang

I am trying to build a binary-to-text encoder with the ability to replace every byte into a predefined unique string and then use the same string set to decode back into binary.
I am able to make the encoder and decoder for simple .txt files but I want to make this workable for .zip files too.
Encoder :
package main
import (
"archive/zip"
"bufio"
"encoding/json"
"fmt"
"io/ioutil"
"log"
"os"
"strings"
)
func main() {
// keys.json is a 256 words dictionary for every byte
keysFile, err := os.Open("keys.json")
if err != nil {
log.Printf("unable to read keys.json file , error : %v", err)
return
}
var keys []string
byteValue, _ := ioutil.ReadAll(keysFile)
if err := json.Unmarshal(byteValue, &keys); err != nil {
log.Printf("unable to unmarshal array , error : %v", err)
return
}
Encoder(keys)
}
func listFiles(file *zip.File) ([]byte, error) {
fileToRead, err := file.Open()
if err != nil {
msg := "Failed to open zip %s for reading: %s"
return nil, fmt.Errorf(msg, file.Name, err)
}
b := make([]byte, file.FileInfo().Size())
fileToRead.Read(b)
defer fileToRead.Close()
if err != nil {
return nil, fmt.Errorf("failed to read zip file : %s for reading , error : %s", file.Name, err)
}
return b, nil
}
func Encoder(keys []string) {
read, err := zip.OpenReader("some.zip")
if err != nil {
msg := "Failed to open: %s"
log.Fatalf(msg, err)
}
defer read.Close()
var encodedBytes []byte
f, err := os.Create("result.txt")
w := bufio.NewWriter(f)
defer f.Close()
for _, file := range read.File {
readBytes, err := listFiles(file)
if err != nil {
log.Fatalf("Failed to read %s from zip: %s", file.Name, err)
continue
}
for i, b := range readBytes {
for _, eb := range []byte(keys[b] + " ") {
encodedBytes = append(encodedBytes, eb)
}
}
}
_, err = w.Write(encodedBytes)
if err != nil {
log.Printf("error :%v", err)
return
}
}
Decoder :
func Decoder(keys []string) {
inputFile, err := os.Open("result.txt")
if err != nil {
log.Printf("unable to read file , error : %v", err)
return
}
inputBytes, _ := ioutil.ReadAll(inputFile)
var (
current []byte
decoded []byte
)
for _, c := range inputBytes {
if c != 32 {
current = append(current, c)
} else {
for i, key := range keys {
if string(current) == key {
decoded = append(decoded, byte(i))
break
}
}
current = []byte{}
}
}
// here i want the decoded back into zip file
}
here is a similar one in nodejs.
Two things:
You are dealing with spaces correctly, but not with newlines.
Your decoder loop is wrong. As far as I can tell, it should look like the following:
for i, key := range keys {
if string(current)==key {
decoded=append(decoded,i)
break
}
}
Also, your decoded is an int-array, not a byte-array.

Move a file to a different drive with Go

I'm trying to move a file from my C-drive to my H-drive using os.Replace().
The code looks as follows:
func MoveFile(source string, destination string) {
err := os.Rename(source, destination)
if err != nil {
fmt.Println(err)
}
}
However, when I run the code I get the following error:
rename C:\old\path\to\file.txt H:\new\path\to\file.txt: The system cannot move the file to a different disk drive.
I found this issue on GitHub that specifies the problem but it appears that they will not change this function to allow it to move file on different disk drives.
I already searched for other possibilities to move files, but found nothing in the standard documentation or the internet.
So, what should I do now to be able to move files on different disk drives?
As the comment said, you'll need to create a new file on the other disk, copy the contents, and then remove the original. It's straightforward using os.Create, io.Copy, and os.Remove:
import (
"fmt"
"io"
"os"
)
func MoveFile(sourcePath, destPath string) error {
inputFile, err := os.Open(sourcePath)
if err != nil {
return fmt.Errorf("Couldn't open source file: %s", err)
}
outputFile, err := os.Create(destPath)
if err != nil {
inputFile.Close()
return fmt.Errorf("Couldn't open dest file: %s", err)
}
defer outputFile.Close()
_, err = io.Copy(outputFile, inputFile)
inputFile.Close()
if err != nil {
return fmt.Errorf("Writing to output file failed: %s", err)
}
// The copy was successful, so now delete the original file
err = os.Remove(sourcePath)
if err != nil {
return fmt.Errorf("Failed removing original file: %s", err)
}
return nil
}
You need to make sure that you handle all cases on both Linux and Windows. For example, for any size file,
package main
import (
"flag"
"fmt"
"io"
"os"
)
func MoveFile(source, destination string) (err error) {
src, err := os.Open(source)
if err != nil {
return err
}
defer src.Close()
fi, err := src.Stat()
if err != nil {
return err
}
flag := os.O_WRONLY | os.O_CREATE | os.O_TRUNC
perm := fi.Mode() & os.ModePerm
dst, err := os.OpenFile(destination, flag, perm)
if err != nil {
return err
}
defer dst.Close()
_, err = io.Copy(dst, src)
if err != nil {
dst.Close()
os.Remove(destination)
return err
}
err = dst.Close()
if err != nil {
return err
}
err = src.Close()
if err != nil {
return err
}
err = os.Remove(source)
if err != nil {
return err
}
return nil
}
func main() {
var src, dst string
flag.StringVar(&src, "src", "", "source file")
flag.StringVar(&dst, "dst", "", "destination file")
flag.Parse()
if src == "" || dst == "" {
flag.Usage()
os.Exit(1)
}
err := MoveFile(src, dst)
if err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
fmt.Printf("moved %q to %q\n", src, dst)
}
Output (Linux):
$ cp move.file src.file && go build movefile.go && ./movefile -src=src.file -dst=dst.file
moved "src.file" to "dst.file"
$
Output (Windows):
>copy /Y move.file src.file && go build movefile.go && movefile -src=src.file -dst=dst.file
moved "src.file" to "dst.file"
>
This solution Moves the file and preserves permissions:
func MoveFile(src, dst string) error {
in, err := os.Open(src)
if err != nil {
return fmt.Errorf("Couldn't open source file: %s", err)
}
out, err := os.Create(dst)
if err != nil {
in.Close()
return fmt.Errorf("Couldn't open dest file: %s", err)
}
defer out.Close()
_, err = io.Copy(out, in)
in.Close()
if err != nil {
return fmt.Errorf("Writing to output file failed: %s", err)
}
err = out.Sync()
if err != nil {
return fmt.Errorf("Sync error: %s", err)
}
si, err := os.Stat(src)
if err != nil {
return fmt.Errorf("Stat error: %s", err)
}
err = os.Chmod(dst, si.Mode())
if err != nil {
return fmt.Errorf("Chmod error: %s", err)
}
err = os.Remove(src)
if err != nil {
return fmt.Errorf("Failed removing original file: %s", err)
}
return nil
}
If only want to Copy the file without remove the original:
func CopyFile(src, dst string) error {
in, err := os.Open(src)
if err != nil {
return fmt.Errorf("Couldn't open source file: %s", err)
}
out, err := os.Create(dst)
if err != nil {
in.Close()
return fmt.Errorf("Couldn't open dest file: %s", err)
}
defer out.Close()
_, err = io.Copy(out, in)
in.Close()
if err != nil {
return fmt.Errorf("Writing to output file failed: %s", err)
}
err = out.Sync()
if err != nil {
return fmt.Errorf("Sync error: %s", err)
}
si, err := os.Stat(src)
if err != nil {
return fmt.Errorf("Stat error: %s", err)
}
err = os.Chmod(dst, si.Mode())
if err != nil {
return fmt.Errorf("Chmod error: %s", err)
}
return nil
}
Maybe you can use a magic approach, just using the syscall.MoveFile as follows.
func main() {
oldpath := "D:\\black.txt"
newpath := "E:\\black-new.txt"
from, _ := syscall.UTF16PtrFromString(oldpath)
to, _ := syscall.UTF16PtrFromString(newpath)
fmt.Println(*from, *to)
err := syscall.MoveFile(from, to)
if err != nil {
panic(err)
}
}
the program works.
if you want a cross-platform compatibility program, you can implement your own MoveFile.
func MoveFile(src string, dst string) error {
if runtime.GOOS == "windows" {
from, _ := syscall.UTF16PtrFromString(src)
to, _ := syscall.UTF16PtrFromString(dst)
return syscall.MoveFile(from, to)
} else {
return os.Rename(src, dst)
}
}

No output to error file

I'm coding a little Go program.
It reads files in a directory line by line, it only reads lines with a certain prefix, normalizes the data and outputs to one of two files, depending on whether the normalized record has certain number of elements.
Data is being outputted to the Data file, but errors are not being outputted to the Errors file.
Debugging I see no issue.
Any help is much appreciated.
Thanks,
Martin
package main
import (
"bufio"
"fmt"
"io/ioutil"
"log"
"os"
"strings"
)
func main() {
//Output file - Data
if _, err := os.Stat("allData.txt"); os.IsNotExist(err) {
var file, err = os.Create("allData.txt")
if err != nil {
fmt.Println(err)
return
}
defer file.Close()
}
file, err := os.OpenFile("allData.txt", os.O_WRONLY|os.O_APPEND, 0644)
if err != nil {
panic(err)
}
w := bufio.NewWriter(file)
//Output file - Errors
if _, err := os.Stat("errorData.txt"); os.IsNotExist(err) {
var fileError, err = os.Create("errorData.txt")
if err != nil {
fmt.Println(err)
return
}
defer fileError.Close()
}
fileError, err := os.OpenFile("errorData.txt", os.O_WRONLY|os.O_APPEND, 0644)
if err != nil {
panic(err)
}
z := bufio.NewWriter(fileError)
//Read Directory
files, err := ioutil.ReadDir("../")
if err != nil {
log.Fatal(err)
}
//Build file path
for _, f := range files {
fName := string(f.Name())
sPath := string("../" + fName)
sFile, err := os.Open(sPath)
if err != nil {
fmt.Println(err)
return
}
//Create scanner
scanner := bufio.NewScanner(sFile)
scanner.Split(bufio.ScanLines)
var lines []string
// This is the buffer now
for scanner.Scan() {
lines = append(lines, scanner.Text())
}
for _, line := range lines {
sRecordC := strings.HasPrefix((line), "DATA:")
if sRecordC {
splitted := strings.Split(line, " ")
splittedNoSpaces := deleteEmpty(splitted)
if len(splittedNoSpaces) == 11 {
splittedString := strings.Join(splittedNoSpaces, " ")
sFinalRecord := string(splittedString + "\r\n")
if _, err = fmt.Fprintf(w, sFinalRecord); err != nil {
}
}
if len(splittedNoSpaces) < 11 {
splitted := strings.Split(line, " ")
splittedNoSpaces := deleteEmpty(splitted)
splittedString := strings.Join(splittedNoSpaces, " ")
sFinalRecord := string(splittedString + "\r\n")
if _, err = fmt.Fprintf(z, sFinalRecord); err != nil {
}
err = fileError.Sync()
if err != nil {
log.Fatal(err)
}
}
}
}
}
err = file.Sync()
if err != nil {
log.Fatal(err)
}
}
//Delete Empty array elements
func deleteEmpty(s []string) []string {
var r []string
for _, str := range s {
if str != "" {
r = append(r, str)
}
}
return r
}
Don't open the file multiple times, and don't check for the file's existence before creating it, just use the os.O_CREATE flag. You're also not deferring the correct os.File.Close call, because it's opened multiple times.
When using a bufio.Writer, you should always call Flush() to ensure that all data has been written to the underlying io.Writer.

Unable to delete an unzipped folder using golang

I wrote code that unzips a file in a particular location then copies the contents of the folder to outside where the folder is unzipped then it removes the folder.
This is the Code I wrote:
package main
import (
"os"
"flag"
"fmt"
"io"
"path/filepath"
"os/exec"
"archive/zip"
"time"
)
func RemoveContents(dir string) error {
d, err := os.Open(dir)
if err != nil {
return err
}
names, err := d.Readdirnames(-1)
if err != nil {
return err
}
for _, name := range names {
err = os.RemoveAll(filepath.Join(dir, name))
if err != nil {
return err
}
}
d.Close()
return nil
}
func CopyFile(source string, dest string) (err error) {
sourcefile, err := os.Open(source)
if err != nil {
return err
}
defer sourcefile.Close()
destfile, err := os.Create(dest)
if err != nil {
return err
}
defer destfile.Close()
_, err = io.Copy(destfile, sourcefile)
if err == nil {
sourceinfo, err := os.Stat(source)
if err != nil {
err = os.Chmod(dest, sourceinfo.Mode())
}
}
return
}
func CopyDir(source string, dest string) (err error) {
// get properties of source dir
sourceinfo, err := os.Stat(source)
if err != nil {
return err
}
// create dest dir
err = os.MkdirAll(dest, sourceinfo.Mode())
if err != nil {
return err
}
directory, _ := os.Open(source)
objects, err := directory.Readdir(-1)
for _, obj := range objects {
sourcefilepointer := source + "/" + obj.Name()
destinationfilepointer := dest + "/" + obj.Name()
if obj.IsDir() {
// create sub-directories - recursively
err = CopyDir(sourcefilepointer, destinationfilepointer)
if err != nil {
fmt.Println(err)
}
} else {
// perform copy
err = CopyFile(sourcefilepointer, destinationfilepointer)
if err != nil {
fmt.Println(err)
}
}
}
return
}
func main() {
flag.Parse() // get the source and destination directory
source_dir := flag.Arg(0) // get the source directory from 1st argument
dest_dir := flag.Arg(1) // get the destination directory from the 2nd argument
os.MkdirAll("E:\\go\\copyDirectory\\myFile.zip",0777)
zipFilePath := "E:\\go\\copyDirectory\\myFile.zip"
tempWrkDir := "E:\\go\\copyDirectory\\"
//Read zip file and get path handle.
fileHandleReader, err := zip.OpenReader(zipFilePath)
if err != nil {
fmt.Println(err)
os.Exit(1)
}
//open zip file and read all the folder and files inside
for _, fileReadHandler := range fileHandleReader.Reader.File {
//read the file or folder handle inside zip
fileOpenHandle, err := fileReadHandler.Open()
if err != nil {
fmt.Println(err)
os.Exit(1)
}
defer fileOpenHandle.Close()
targetUnZipPath := filepath.Join(tempWrkDir, fileReadHandler.Name)
if fileReadHandler.FileInfo().IsDir() {
os.MkdirAll(targetUnZipPath, fileReadHandler.Mode())
//fmt.Println("Creating directory", path)
}else {
// create new dummy file to copy original file.
newTempFileHandle, err := os.OpenFile(targetUnZipPath, os.O_WRONLY|os.O_CREATE, fileReadHandler.Mode())
if err != nil {
fmt.Println(err)
os.Exit(1)
}
defer newTempFileHandle.Close()
//copying original file to dummy file.
if _, err = io.Copy(newTempFileHandle, fileOpenHandle); err != nil {
fmt.Println(err)
os.Exit(1)
}
}
}
time.Sleep(1000*time.Millisecond)
fmt.Println("Source :" + source_dir)
// check if the source dir exist
src, err := os.Stat(source_dir)
if err != nil {
panic(err)
}
if !src.IsDir() {
fmt.Println("Source is not a directory")
os.Exit(1)
}
// create the destination directory
fmt.Println("Destination :"+ dest_dir)
/*_, err = os.Open(dest_dir)
if !os.IsNotExist(err) {
fmt.Println("Destination directory already exists. Abort!")
os.Exit(1)
}*/
err = CopyDir(source_dir, dest_dir)
if err != nil {
fmt.Println(err)
} else {
fmt.Println("Directory copied")
}
err = RemoveContents("./myFiles")
if err != nil {
fmt.Println("ERRR:::",err)
}
//time.Sleep(10000*time.Millisecond)
}
The problem is that everything works fine except for deleting the folder. The folder has only one file in it. The location of the file is as follows:
E:\go\copyDirectory\myfile\mytextfile.txt
The Location of the zip file is as follows:
E:\go\copyDirectory\myfile.zip
The zip file has only one text file. The File inside the zip file is as follows:
E:\go\copyDirectory\myfile.zip\myfile\mytextfile.txt
The error I get is:
ERRR::: remove myfile\mytextfile.txt: The process cannot
access the file because it is being used by another process.
Thanks in advance.
You aren't closing the file. This:
defer newTempFileHandle.Close()
Is run when main finishes, which is after:
err = RemoveContents("./myFiles")
You can wrap that bit of code in an unnamed function:
func() {
//read the file or folder handle inside zip
fileOpenHandle, err := fileReadHandler.Open()
if err != nil {
fmt.Println(err)
os.Exit(1)
}
defer fileOpenHandle.Close()
targetUnZipPath := filepath.Join(tempWrkDir, fileReadHandler.Name)
if fileReadHandler.FileInfo().IsDir() {
os.MkdirAll(targetUnZipPath, fileReadHandler.Mode())
//fmt.Println("Creating directory", path)
} else {
// create new dummy file to copy original file.
newTempFileHandle, err := os.OpenFile(targetUnZipPath, os.O_WRONLY|os.O_CREATE, fileReadHandler.Mode())
if err != nil {
fmt.Println(err)
os.Exit(1)
}
defer newTempFileHandle.Close()
//copying original file to dummy file.
if _, err = io.Copy(newTempFileHandle, fileOpenHandle); err != nil {
fmt.Println(err)
os.Exit(1)
}
}
}()
And then your defer will happen before you try and remove the files. I would recommend pulling this out into a named function though.

Resources