I need to zip/unzip folder that contains symlinks in a way that the structure will be saved and symlinks will be written as symlinks.
Is there a way doing this using Golang package "archive/zip"? or any other alternative way?
I tried to use this code, but 'io.Copy()' copies the target file content and we "lose" the symlink.
archive, err := os.Create("archive.zip")
if err != nil {
panic(err)
}
defer archive.Close()
zipWriter := zip.NewWriter(archive)
localPath := "../testdata/sym"
file, err := os.Open(localPath)
defer file.Close()
if err != nil {
panic(err)
}
w1 , err:= zipWriter.Create("w1")
if _, err = io.Copy(w1, file); err !=nil{
panic(err)
}
zipWriter.Close()
I used this PR : https://github.com/mholt/archiver/pull/92
and wrote symlink's target to writer.
I tested it on Linux and Windows.
archive, err := os.Create("archive.zip")
if err != nil {
panic(err)
}
defer archive.Close()
zipWriter := zip.NewWriter(archive)
defer zipWriter.Close()
localPath := "../testdata/sym"
symlinksTarget = "../a/b.in"
file, err := os.Open(localPath)
defer file.Close()
if err != nil {
panic(err)
}
info, err := os.Lstat(file.Name())
if err != nil {
panic(err)
}
header, err := zip.FileInfoHeader(info)
if err != nil {
panic(err)
}
header.Method = zip.Deflate
writer, err := zipWriter.CreateHeader(header)
if err != nil {
panic(err)
}
// Write symlink's target to writer - file's body for symlinks is the symlink target.
_, err = writer.Write([]byte(filepath.ToSlash(symlinksTarget)))
if err != nil {
panic(err)
}
Related
I need to write multicast listener on Go. I faces the problem of twicing packets when I read it. It seems that I need to set IP_MULTICAST_LOOP to false. It is not easy to do in Go.
I found this post. It seems that it should work. But I still get copies of the same host. How should it be done?
ipAddr, err := net.ResolveUDPAddr("udp", groupAddress)
if err != nil {...}
iface, err := net.InterfaceByName("en0")
if err != nil {...}
conn, err := net.ListenPacket("udp4", groupAddress)
if err != nil {...}
pc := ipv4.NewPacketConn(conn)
if err := pc.JoinGroup(iface, ipAddr); err != nil {...}
if err := pc.SetMulticastLoopback(false); err != nil {...}
if loop, err := pc.MulticastLoopback(); err == nil {...}
buf := make([]byte, 1024)
for {
n, _, addr, err := pc.ReadFrom(buf)
if err != nil {...}
fmt.Printf("recv from %v: [%s] \n", addr, buf[:n])
}
The simplest way is to use the ListenMulticastUDP wrapper in the net package, as actually already explained in the other SO answer you point to, How to set IP_MULTICAST_LOOP on multicast UDPConn in Golang.
If you follow the implementation of ListenMulticastUDP(), you will see that at a certain point it calls setIPv4MulticastLoopback(fd, false).
If you need something more advanced, the documentation of ListenMulticastUDP() suggests to look at https://godoc.org/golang.org/x/net/ipv4 and https://godoc.org/golang.org/x/net/ipv6, which document extensively how to do multicast in Go.
Here is some minimal code (tested on MacOS, but platform-independent) that shows how to use ListenMulticastUDP():
func main() {
// MDNS (https://en.wikipedia.org/wiki/Multicast_DNS)
groupAddress := "224.0.0.251:5353"
ifaceName := "en0"
if err := run(groupAddress, ifaceName); err != nil {
fmt.Fprintln(os.Stderr, "error:", err)
os.Exit(1)
}
}
func run(groupAddr string, ifaceName string) error {
iface, err := net.InterfaceByName(ifaceName)
if err != nil {
return err
}
gaddr, err := net.ResolveUDPAddr("udp", groupAddr)
if err != nil {
return err
}
conn, err := net.ListenMulticastUDP("udp", iface, gaddr)
if err != nil {
return err
}
buf := make([]byte, 1024)
for {
n, addr, err := conn.ReadFromUDP(buf)
if err != nil {
return err
}
fmt.Printf("recv %4d bytes from %v\n", n, addr)
}
}
newZip, err := os.Create("./temp/tosend.zip")
if err != nil {
return "", err
}
defer newZip.Close()
zipWriter := zip.NewWriter(newZip)
_, err = zipWriter.Create("archivos/")
if err != nil {
return "", err
}
defer zipWriter.Close()
You are almost there. A file name can contain directories.
Use this code:
zipWriter := zip.NewWriter(newZip)
w, err := zipWriter.Create("archivos/file.png") // <-- use full path here
if err != nil {
return "", err
}
w.Write(pngData)
Run it on the playground.
I followed the repo (https://github.com/360EntSecGroup-Skylar/excelize) tutorial to open a file:
f, err := excelize.OpenFile("./Book1.xlsx")
if err != nil {
fmt.Println(err)
return
}enter code here
But i couldn't find a tutorial about closing, something like:
defer f.Close()
Is there a way to do that?
You don't have to close it.
Just open it, and save it if you need to.
myFile, err := excelize.OpenFile("./Book1.xlsx")
if err != nil {
fmt.Println(err)
return
}
Then, do what you want.
Finally, just save it with the origin path :
err = myFile.Save()
if err != nil {
fmt.Println(err)
}
Or, an other path :
err := myFile.SaveAs("./myFolder/Book2.xlsx")
if err != nil {
fmt.Println(err)
}
I wonder if possible to copy running .exe file to another folder. I am trying to do this using usual copy approach in Go like that.
func copy(src, dst string) error {
in, err := os.Open(src)
if err != nil {
return err
}
defer in.Close()
out, err := os.Create(dst)
if err != nil {
return err
}
defer out.Close()
_, err = io.Copy(out, in)
if err != nil {
return err
}
return out.Close()
}
...
copyErr := copy(os.Args[0], "D:"+"\\"+"whs.exe")
if copyErr != nil {
log.Panicf("copy -> %v", copyErr)
}
The file copied with the same size but I can't open it correctly. I have only a fast cmd flash. After several milliseconds, cmd is closing and I can't see even any errors.
I was trying to write errors to log file but it's empty.
f, err := os.OpenFile("debug.log", os.O_RDWR|os.O_CREATE|os.O_APPEND, 0777)
if err != nil {
log.Panicf("setLogOutput -> %v", err)
}
defer f.Close()
log.SetOutput(f)
If I open not copied .exe file everything works correctly.
I've reduced my program to only one main method. The result was the same.
func main() {
log.Println("Starting...")
copyErr := copy(os.Args[0], "F:"+"\\"+"whs.exe")
if copyErr != nil {
log.Panicf("copy -> %v", copyErr)
}
os.Stdin.Read([]byte{0})
}
I have found an error.
The process cannot access the file because it is being used by another process.
I was trying to copy the .exe file to its own path.
func copy(src, dst string) error {
if _, err := os.Stat(dst); os.IsNotExist(err) {
in, err := os.Open(src)
if err != nil {
return err
}
defer in.Close()
out, err := os.Create(dst)
if err != nil {
return err
}
defer out.Close()
_, err = io.Copy(out, in)
if err != nil {
return err
}
}
return nil
}
I'm trying to download a remote file over ssh
The following approach works fine on shell
ssh hostname "tar cz /opt/local/folder" > folder.tar.gz
However the same approach on golang giving some difference in output artifact size. For example the same folders with pure shell produce artifact gz file 179B and same with go script 178B.
I assume that something has been missed from io.Reader or session got closed earlier. Kindly ask you guys to help.
Here is the example of my script:
func executeCmd(cmd, hostname string, config *ssh.ClientConfig, path string) error {
conn, _ := ssh.Dial("tcp", hostname+":22", config)
session, err := conn.NewSession()
if err != nil {
panic("Failed to create session: " + err.Error())
}
r, _ := session.StdoutPipe()
scanner := bufio.NewScanner(r)
go func() {
defer session.Close()
name := fmt.Sprintf("%s/backup_folder_%v.tar.gz", path, time.Now().Unix())
file, err := os.OpenFile(name, os.O_APPEND|os.O_WRONLY|os.O_CREATE, 0644)
if err != nil {
panic(err)
}
defer file.Close()
for scanner.Scan() {
fmt.Println(scanner.Bytes())
if err := scanner.Err(); err != nil {
fmt.Println(err)
}
if _, err = file.Write(scanner.Bytes()); err != nil {
log.Fatal(err)
}
}
}()
if err := session.Run(cmd); err != nil {
fmt.Println(err.Error())
panic("Failed to run: " + err.Error())
}
return nil
}
Thanks!
bufio.Scanner is for newline delimited text. According to the documentation, the scanner will remove the newline characters, stripping any 10s out of your binary file.
You don't need a goroutine to do the copy, because you can use session.Start to start the process asynchronously.
You probably don't need to use bufio either. You should be using io.Copy to copy the file, which has an internal buffer already on top of any buffering already done in the ssh client itself. If an additional buffer is needed for performance, wrap the session output in a bufio.Reader
Finally, you return an error value, so use it rather than panic'ing on regular error conditions.
conn, err := ssh.Dial("tcp", hostname+":22", config)
if err != nil {
return err
}
session, err := conn.NewSession()
if err != nil {
return err
}
defer session.Close()
r, err := session.StdoutPipe()
if err != nil {
return err
}
name := fmt.Sprintf("%s/backup_folder_%v.tar.gz", path, time.Now().Unix())
file, err := os.OpenFile(name, os.O_APPEND|os.O_WRONLY|os.O_CREATE, 0644)
if err != nil {
return err
}
defer file.Close()
if err := session.Start(cmd); err != nil {
return err
}
n, err := io.Copy(file, r)
if err != nil {
return err
}
if err := session.Wait(); err != nil {
return err
}
return nil
You can try doing something like this:
r, _ := session.StdoutPipe()
reader := bufio.NewReader(r)
go func() {
defer session.Close()
// open file etc
// 10 is the number of bytes you'd like to copy in one write operation
p := make([]byte, 10)
for {
n, err := reader.Read(p)
if err == io.EOF {
break
}
if err != nil {
log.Fatal("err", err)
}
if _, err = file.Write(p[:n]); err != nil {
log.Fatal(err)
}
}
}()
Make sure your goroutines are synchronized properly so output is completeky written to the file.