I've written the following code to tar a file, code works but strangely if I untar the archive the file permissions are gone so I can't read it unless I then chmod the file:
package main
import (
"archive/tar"
"io/ioutil"
"log"
"os"
)
func main() {
c, err := os.Create("/path/to/tar/file/test.tar")
if err != nil {
log.Fatalln(err)
}
tw := tar.NewWriter(c)
f, err := os.Open("sample.txt")
if err != nil {
log.Fatalln(err)
}
fi, err := f.Stat()
if err != nil {
log.Fatalln(err)
}
hdr := &tar.Header{Name: f.Name(),
Size: fi.Size(),
}
if err := tw.WriteHeader(hdr); err != nil {
log.Fatalln(err)
}
r, err := ioutil.ReadFile("sample.txt")
if err != nil {
log.Fatalln(err)
}
if _, err := tw.Write(r); err != nil {
log.Fatalln(err)
}
if err := tw.Close(); err != nil {
log.Fatalln(err)
}
}
Any idea what I'm doing wrong?
You're not preserving the original permissions of the file. You're manually creating a header, and specifying only the name and size. Instead, use tar.FileInfoHeader to build the header.
package main
import (
"archive/tar"
"io/ioutil"
"log"
"os"
)
func main() {
c, err := os.Create("/path/to/tar/file/test.tar")
if err != nil {
log.Fatalln(err)
}
tw := tar.NewWriter(c)
f, err := os.Open("sample.txt")
if err != nil {
log.Fatalln(err)
}
fi, err := f.Stat()
if err != nil {
log.Fatalln(err)
}
// create header from FileInfo
hdr, err := tar.FileInfoHeader(fi, "")
if err != nil {
log.Fatalln(err)
}
if err := tw.WriteHeader(hdr); err != nil {
log.Fatalln(err)
}
// instead of reading the whole file into memory, prefer io.Copy
r, err := io.Copy(tw, f)
if err != nil {
log.Fatalln(err)
}
log.Printf("Wrote %d bytes\n", r)
}
Also note that I used io.Copy to copy data from the file (an io.Reader) to the tar writer (an io.Writer). This will work much better for larger files.
Also - pay special attention to this note from the docs:
Because os.FileInfo's Name method returns only the base name of the file it describes, it may be necessary to modify the Name field of the returned header to provide the full path name of the file.
In this simple example, you're just using sample.txt so you shouldn't run into trouble. If you wanted to preserve a directory structure in your tar, you may have to modify the Name field in the header.
Related
I am trying to create a .tar.gz file from folder that contains multiple files / folders. Once the .tar.gz file gets created, while extracting, the files are not not properly extracted. Mostly I think its because of large names or path exceeding some n characters, because same thing works when the filename/path is small. I referred this https://github.com/golang/go/issues/17630 and tried to add below code but it did not help.
header.Uid = 0
header.Gid = 0
I am using simple code seen below to create .tar.gz. The approach is, I create a temp folder, do some processing on the files and from that temp path, I create the .tar.gz file hence in the path below I am using pre-defined temp folder path.
package main
import (
"archive/tar"
"compress/gzip"
"fmt"
"io"
"log"
"os"
fp "path/filepath"
)
func main() {
// Create output file
out, err := os.Create("output.tar.gz")
if err != nil {
log.Fatalln("Error writing archive:", err)
}
defer out.Close()
// Create the archive and write the output to the "out" Writer
tmpDir := "C:/Users/USERNAME~1/AppData/Local/Temp/temp-241232063"
err = createArchive1(tmpDir, out)
if err != nil {
log.Fatalln("Error creating archive:", err)
}
fmt.Println("Archive created successfully")
}
func createArchive1(path string, targetFile *os.File) error {
gw := gzip.NewWriter(targetFile)
defer gw.Close()
tw := tar.NewWriter(gw)
defer tw.Close()
// walk through every file in the folder
err := fp.Walk(path, func(filePath string, info os.FileInfo, err error) error {
// ensure the src actually exists before trying to tar it
if _, err := os.Stat(filePath); err != nil {
return err
}
if err != nil {
return err
}
if info.IsDir() {
return nil
}
file, err := os.Open(filePath)
if err != nil {
return err
}
defer file.Close()
// generate tar header
header, err := tar.FileInfoHeader(info, info.Name())
header.Uid = 0
header.Gid = 0
if err != nil {
return err
}
header.Name = filePath //strings.TrimPrefix(filePath, fmt.Sprintf("%s/", fp.Dir(path))) //info.Name()
// write header
if err := tw.WriteHeader(header); err != nil {
return err
}
if _, err := io.Copy(tw, file); err != nil {
return err
}
return nil
})
return err
}
Please let me know what wrong I am doing.
I just tried to download webp image from url, but I found something different when I try to process the stored image.
If I download the image from the browser, it can be decoded using x/image/webp package, but if I store the image using http.Get() then create a new file then io.Copy() the image, it says:
"missing RIFF chunk header"
I assume that I need to write some RIFF chunk header when I store it using golang code.
func main(){
response, e := http.Get(URL)
if e != nil {
log.Fatal(e)
}
defer response.Body.Close()
//open a file for writing
file, err := os.Create('tv.webp')
if err != nil {
log.Fatal(err)
}
defer file.Close()
// Use io.Copy to just dump the response body to the file. This supports huge files
_, err = io.Copy(file, response.Body)
if err != nil {
log.Fatal(err)
}
fmt.Println("Success!")
imgData, err := os.Open("tv.webp")
if err != nil {
fmt.Println(err)
return
}
log.Printf("%+v", imgData)
image, err := webp.Decode(imgData)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(image.Bounds())
}
Here is the URL IMG URL
download file is not webp type. it's png.
package main
import (
"fmt"
"image"
"io"
"log"
"net/http"
"os"
_ "image/png"
)
func main() {
response, e := http.Get("https://www.sony.com/is/image/gwtprod/0abe7672ff4c6cb4a0a4d4cc143fd05b?fmt=png-alpha")
if e != nil {
log.Fatal(e)
}
defer response.Body.Close()
file, err := os.Create("dump")
if err != nil {
log.Fatal(err)
}
defer file.Close()
_, err = io.Copy(file, response.Body)
if err != nil {
log.Fatal(err)
}
fmt.Println("Success!")
imageFile, err := os.Open("dump")
if err != nil {
panic(err)
}
m, name, err := image.Decode(imageFile)
if err != nil {
panic(err)
}
fmt.Println("image type is ", name, m.Bounds())
}
I was following this document to create a key/string in windows registry with this code snippet:
package main
import (
"golang.org/x/sys/windows/registry"
"log"
)
func main() {
k, err := registry.OpenKey(registry.CURRENT_USER, `Software\Microsoft\Windows\CurrentVersion\Audio`, registry.QUERY_VALUE)
if err != nil {
log.Fatal(err)
}
k.SetStringValue("xyz", "blahblah")
err = k.Close()
if err != nil {
log.Fatal(err)
}
}
but nothing happens, without any errors.
Edit (clarification): It doesn't work, in any circumstances.
You are opening the key with only the QUERY_VALUE permission, but you also need SET_VALUE in order to successfully call SetStringValue.
You should also be checking the return value on SetStringValue, which would have likely informed you what the problem was.
k, err := registry.OpenKey(registry.CURRENT_USER, `Software\Microsoft\Windows\CurrentVersion\Audio`, registry.QUERY_VALUE|registry.SET_VALUE)
if err != nil {
log.Fatal(err)
}
if err := k.SetStringValue("xyz", "blahblah"); err != nil {
log.Fatal(err)
}
if err := k.Close(); err != nil {
log.Fatal(err)
}
My Go application outputs some amounts of text data and I need to pipe it to some external command (e.g. less). I haven't find any way to pipe this data to syscall.Exec'ed process.
As a workaround I write that text data to a temporary file and then use that file as an argument to less:
package main
import (
"io/ioutil"
"log"
"os"
"os/exec"
"syscall"
)
func main() {
content := []byte("temporary file's content")
tmpfile, err := ioutil.TempFile("", "example")
if err != nil {
log.Fatal(err)
}
defer os.Remove(tmpfile.Name()) // Never going to happen!
if _, err := tmpfile.Write(content); err != nil {
log.Fatal(err)
}
if err := tmpfile.Close(); err != nil {
log.Fatal(err)
}
binary, err := exec.LookPath("less")
if err != nil {
log.Fatal(err)
}
args := []string{"less", tmpfile.Name()}
if err := syscall.Exec(binary, args, os.Environ()); err != nil {
log.Fatal(err)
}
}
It works but leaves a temporary file on a file system, because syscall.Exec replaces the current Go process with another (less) one and deferred os.Remove won't run. Such behaviour is not desirable.
Is there any way to pipe some data to an external process without leaving any artefacts?
You should be using os/exec to build an exec.Cmd to execute, then you could supply any io.Reader you want as the stdin for the command.
From the example in the documentation:
cmd := exec.Command("tr", "a-z", "A-Z")
cmd.Stdin = strings.NewReader("some input")
var out bytes.Buffer
cmd.Stdout = &out
err := cmd.Run()
if err != nil {
log.Fatal(err)
}
fmt.Printf("in all caps: %q\n", out.String())
If you want to write directly to the command's stdin, then you call cmd.StdInPipe to get an io.WriteCloser you can write to.
If you really need to exec the process in place of your current one, you can simply remove the file before exec'ing, and provide that file descriptor as stdin for the program.
content := []byte("temporary file's content")
tmpfile, err := ioutil.TempFile("", "example")
if err != nil {
log.Fatal(err)
}
os.Remove(tmpfile.Name())
if _, err := tmpfile.Write(content); err != nil {
log.Fatal(err)
}
tmpfile.Seek(0, 0)
err = syscall.Dup2(int(tmpfile.Fd()), syscall.Stdin)
if err != nil {
log.Fatal(err)
}
binary, err := exec.LookPath("less")
if err != nil {
log.Fatal(err)
}
args := []string{"less"}
if err := syscall.Exec(binary, args, os.Environ()); err != nil {
log.Fatal(err)
}
Is it possible to extract a tar.xz package in golang? My understanding is it's possible to use the library for tar and sending it to an xz go library.
I recently created an XZ decompression package so it is now
possible to extract a tar.xz using only Go code.
The following code extracts the file myfile.tar.xz to the current
directory:
package main
import (
"archive/tar"
"fmt"
"io"
"log"
"os"
"github.com/xi2/xz"
)
func main() {
// Open a file
f, err := os.Open("myfile.tar.xz")
if err != nil {
log.Fatal(err)
}
// Create an xz Reader
r, err := xz.NewReader(f, 0)
if err != nil {
log.Fatal(err)
}
// Create a tar Reader
tr := tar.NewReader(r)
// Iterate through the files in the archive.
for {
hdr, err := tr.Next()
if err == io.EOF {
// end of tar archive
break
}
if err != nil {
log.Fatal(err)
}
switch hdr.Typeflag {
case tar.TypeDir:
// create a directory
fmt.Println("creating: " + hdr.Name)
err = os.MkdirAll(hdr.Name, 0777)
if err != nil {
log.Fatal(err)
}
case tar.TypeReg, tar.TypeRegA:
// write a file
fmt.Println("extracting: " + hdr.Name)
w, err := os.Create(hdr.Name)
if err != nil {
log.Fatal(err)
}
_, err = io.Copy(w, tr)
if err != nil {
log.Fatal(err)
}
w.Close()
}
}
f.Close()
}
http://golang.org/pkg/archive/tar/#example_
also you can do
import "os/exec"
cmd := exec.Command("tar", "-x", "/your/archive.tar.xz")
err := cmd.Run()
There is no Lempel-Ziv-Markow encoder or decoder in the Go standard library. If you are allowed to assume that the platform your code runs on provides the xz utility, you could use stub functions like these:
import "os/exec"
// decompress xz compressed data stream r.
func UnxzReader(r io.Reader) (io.ReadCloser, error) {
unxz := exec.Command("xz", "-d")
unxz.Stdin = r
out, err := unxz.StdoutPipe()
if err != nil {
return nil, err
}
err = unxz.Start()
if err != nil {
return nil, err
}
// we are not interested in the exit status, but we should really collect
// that zombie process
go unxz.Wait()
return out, nil
}