I am facing a weird issue. In the following code the code reaches to break point 1 but not break point 2. There is no error as well. The code just gets stuck there without throwing any error.
dbMaster := s.master
tx, err := dbMaster.Begin()
stmt, err := tx.Prepare("UPDATE service set status='deleted' where id=1")
if err != nil {
tx.Rollback()
log.Printf("Error while preparing statement: %s\n", err)
}
defer stmt.Close()
fmt.Println("break point 1")
result, err := stmt.Exec()
fmt.Println("break point 2")
if err != nil {
log.Printf("Error executing statement: %s\n", err)
}
Related
I am writing a simple script to get download unzip the tar.gz file and then remove it. Whenever I try to remove it I get an error:
The process cannot access the file because it is being used by another process.
I assume the error is in how I pass the file to the extractTarGz function, but I am not sure.
Here is the code:
package main
import (
"archive/tar"
"compress/gzip"
"io"
"log"
"os"
)
func main() {
f, err := os.Open("file.tar.gz")
if err != nil {
panic(err)
}
defer f.Close()
extractTarGz(f)
err = os.Remove("file.tar.gz")
}
func extractTarGz(gzipStream io.Reader) {
uncompressedStream, err := gzip.NewReader(gzipStream)
if err != nil {
log.Fatal("ExtractTarGz: NewReader failed")
}
tarReader := tar.NewReader(uncompressedStream)
for true {
header, err := tarReader.Next()
if err == io.EOF {
break
}
if err != nil {
log.Fatalf("ExtractTarGz: Next() failed: %s", err.Error())
}
switch header.Typeflag {
case tar.TypeDir:
if err := os.Mkdir(header.Name, 0755); err != nil {
log.Fatalf("ExtractTarGz: Mkdir() failed: %s", err.Error())
}
case tar.TypeReg:
outFile, err := os.Create(header.Name)
if err != nil {
log.Fatalf("ExtractTarGz: Create() failed: %s", err.Error())
}
defer outFile.Close()
if _, err := io.Copy(outFile, tarReader); err != nil {
log.Fatalf("ExtractTarGz: Copy() failed: %s", err.Error())
}
default:
log.Fatalf(
"ExtractTarGz: uknown type: %s in %s",
header.Typeflag,
header.Name)
}
}
}
You should first close the file, and then attempt to remove it. Since you close it using defer, that will / would be called after the os.Remove() call.
Try it like this:
name := "file.tar.gz"
defer func() {
if err = os.Remove(name); err != nil {
log.Printf("Failed to remove %s: %v", name, err)
}
}()
f, err := os.Open(name)
if err != nil {
panic(err)
}
defer f.Close()
extractTarGz(f)
Deferred functions are executed in LIFO (last-in-first-out) order, so first f.Close() will be called, and then the other which tries to remove the file. Quoting from Spec: Deferred statements:
...deferred functions are invoked immediately before the surrounding function returns, in the reverse order they were deferred.
f, err := os.Open("file.tar.gz")
if err != nil {
panic(err)
}
defer f.Close()
extractTarGz(f)
err = os.Remove("file.tar.gz")
At the very least, you need to close the file before you removeit.
err = f.Close()
if err != nil {
panic(err)
}
err = os.Remove("file.tar.gz")
defer f.Close() won't run until the end of the function.
While I was reading the example of "Prepared" statement in "transaction" in golang SQL/database example. One of the line says, "danger", yet the code example was provided without an alternative.
I wanted to have more clear explanation on below query, as not much information was provided on Wiki page at - http://go-database-sql.org/prepared.html
tx, err := db.Begin()
if err != nil {
log.Fatal(err)
}
defer tx.Rollback()
stmt, err := tx.Prepare("INSERT INTO foo VALUES (?)")
if err != nil {
log.Fatal(err)
}
defer stmt.Close() // danger!
for i := 0; i < 10; i++ {
_, err = stmt.Exec(i)
if err != nil {
log.Fatal(err)
}
}
err = tx.Commit()
if err != nil {
log.Fatal(err)
}
// stmt.Close() runs here!
If you see in defer stmt.Close() it mentions, it's dangerous and yet not commented out for users to remove it.
Though I see no issue in above code as "defer" is going to run the code at the end but do they mean, above code is wrong and it should be replaced with below code or other better alternatives code.
tx, err := db.Begin()
if err != nil {
log.Fatal(err)
}
defer tx.Rollback()
stmt, err := tx.Prepare("INSERT INTO foo VALUES (?)")
if err != nil {
log.Fatal(err)
}
// Commented out below line.
// defer stmt.Close()
for i := 0; i < 10; i++ {
_, err = stmt.Exec(i)
if err != nil {
log.Fatal(err)
}
}
err = tx.Commit()
if err != nil {
log.Fatal(err)
}
// Comment removed from below line to close the stmt
stmt.Close()
I see no difference in both of the code above, yet, I need expert advice on above if there is any difference or If I am missing something.
a defer statement is a good way to make sure something runs no matter how you exit the function.
In this particular case, it seems to not matter, since all the error handlers use log.Fatal. If you replace the log.Fatals with return statements, and remove the defers, you now have to cleanup in many places:
tx, err := db.Begin()
if err != nil {
return nil,err
}
stmt, err := tx.Prepare("INSERT INTO foo VALUES (?)")
if err != nil {
tx.Rollback()
return nil,err
}
defer
for i := 0; i < 10; i++ {
_, err = stmt.Exec(i)
if err != nil {
tx.Rollback()
return nil,err
}
}
err = tx.Commit()
if err != nil {
stmt.Close()
tx.Rollback()
return nil,err
}
stmt.Close()
return someValue, nil
If you use defer, it is harder to forget one place you need to clean something up.
I have started using Go for a web-service and have some database interaction (surprise!!!) and I have found this example:
tx, err := db.Begin()
if err != nil {
log.Fatal(err)
}
defer tx.Rollback()
stmt, err := tx.Prepare("INSERT INTO foo VALUES (?)")
if err != nil {
log.Fatal(err)
}
defer stmt.Close() // danger!
for i := 0; i < 10; i++ {
_, err = stmt.Exec(i)
if err != nil {
log.Fatal(err)
}
}
err = tx.Commit()
if err != nil {
log.Fatal(err)
}
// stmt.Close() runs here!
From http://go-database-sql.org/prepared.html
The example is well formulated an easy to understand. However, it leaves me with an unanswered question. Why defer the transaction Rollback call?
Why not just do the following:
err := tx.Commit()
if err != nil {
log.Error(err)
tx.Rollback()
}
Would defer tx.Rollback() not always attempt a rollback? Even if tx.Commit() was a success, or have I misunderstood something about defer?
The important thing is that if you defer tx.Rollback() you are sure that it will be called even if you do an early return and the "trick" is that calling tx.Rollback() on a committed transaction will not actually do the rollback, because once a transaction is committed, it's committed and there is no way to roll it back :)
So this is a neat trick on how to keep the code simple.
The example is a little bit misleading. It uses log.Fatal(err) for error handling. You wouldn't normally do that, and instead return err. So the deferred rollback is there to ensure that the transaction is rolled back in case of an early return.
I'm taking a dive into the networking side of Go, and I'd thought I'd start with a TCP Client and Server.
I am able to get the client to connect to the server and send a simple message ("Hello") successfully. However, I can not get the server to send back a response (or the get the client to read the response).
Here is the code.
Server
Address := "localhost:9999"
Addr, err := net.ResolveTCPAddr("tcp", Address)
if err != nil {
log.Fatal(err)
}
listener, err := net.ListenTCP("tcp", Addr)
if err != nil {
log.Fatal(err)
}
defer listener.Close()
//server loop
for {
conn, err := listener.Accept()
if err != nil {
continue
}
go handle(conn)
}
func handle(c net.Conn) {
totalBytes, message := connRead(c)
fmt.Println(c.RemoteAddr())
fmt.Println(string(message[:totalBytes]))
c.Write([]byte("Hi"))
fmt.Println("Replied")
c.Close()
}
func connRead(c net.Conn) (int, []byte) {
buffer := make([]byte, 4096)
totalBytes := 0
for {
n, err := c.Read(buffer)
totalBytes += n
if err != nil {
if err != io.EOF {
log.Printf("Read error: %s", err)
}
break
}
}
return totalBytes, buffer
}
Client
tcpAddr, err := net.ResolveTCPAddr("tcp", "localhost:9999")
if err != nil {
log.Fatal(err)
}
conn, err := net.DialTCP("tcp", nil, tcpAddr)
if err != nil {
log.Fatal(err)
}
defer conn.Close()
_, err = conn.Write([]byte("Hello"))
if err != nil {
log.Fatal(err)
}
tBytes, resp := connRead(conn)
fmt.Println(tBytes)
fmt.Println(string(resp[:tBytes]))
func connRead(c net.Conn) (int, []byte) {
buffer := make([]byte, 4096)
totalBytes := 0
for {
fmt.Println("Stuck?")
n, err := c.Read(buffer)
fmt.Println("Stuck.")
totalBytes += n
fmt.Println(totalBytes)
if err != nil {
if err != io.EOF {
log.Printf("Read error: %s", err)
}
break
}
}
return totalBytes, buffer
}
From what I can tell it's not a problem with the server. When I run the client, everything stops right after fmt.Println("Stuck?"). This leads me to belive that it's messing up in the n, err := c.Read(buffer) statement somehow. The server doesn't even print out the messeage length (5) and message ("Hello") untill after I Ctrl-C the client. If I comment out the read and printings in the client, then things run smoothly.
I've tried googling for answers, but nothing has come up.
What am I doing wrong? Am I using conn.Read() wrong in the client?
EDIT:
I actually do have access to Linux, so here are the SIGQUIT dumps for the pertinent functions.
Server
http://pastebin.com/itevngCq
Client
http://pastebin.com/XLiKqkvs
for {
n, err := c.Read(buffer)
totalBytes += n
if err != nil {
if err != io.EOF {
log.Printf("Read error: %s", err)
}
break
}
}
It is because you are reading from connection till EOF error occurs
conn.Write([]byte("Hello"))
The above statement won't reach EOF at all until you actually closes the connection
On pressing ctrl+c client side the connection will be closed, So EOF occurs at server side, That is the reason why it is exiting server side for loop and printing these
127.0.0.1:****
Hello
Replied
If you want to make this work you should not read the connection till EOF
There are many other alternatives for this
Chose a delimiter and read at the server until the delimiter occurs and respond back after that. Check out this link
Send number of bytes to read from client side before sending the actual message, First read number of bytes to read from the server side and then read those many bytes from the connection
I'm trying to download a remote file over ssh
The following approach works fine on shell
ssh hostname "tar cz /opt/local/folder" > folder.tar.gz
However the same approach on golang giving some difference in output artifact size. For example the same folders with pure shell produce artifact gz file 179B and same with go script 178B.
I assume that something has been missed from io.Reader or session got closed earlier. Kindly ask you guys to help.
Here is the example of my script:
func executeCmd(cmd, hostname string, config *ssh.ClientConfig, path string) error {
conn, _ := ssh.Dial("tcp", hostname+":22", config)
session, err := conn.NewSession()
if err != nil {
panic("Failed to create session: " + err.Error())
}
r, _ := session.StdoutPipe()
scanner := bufio.NewScanner(r)
go func() {
defer session.Close()
name := fmt.Sprintf("%s/backup_folder_%v.tar.gz", path, time.Now().Unix())
file, err := os.OpenFile(name, os.O_APPEND|os.O_WRONLY|os.O_CREATE, 0644)
if err != nil {
panic(err)
}
defer file.Close()
for scanner.Scan() {
fmt.Println(scanner.Bytes())
if err := scanner.Err(); err != nil {
fmt.Println(err)
}
if _, err = file.Write(scanner.Bytes()); err != nil {
log.Fatal(err)
}
}
}()
if err := session.Run(cmd); err != nil {
fmt.Println(err.Error())
panic("Failed to run: " + err.Error())
}
return nil
}
Thanks!
bufio.Scanner is for newline delimited text. According to the documentation, the scanner will remove the newline characters, stripping any 10s out of your binary file.
You don't need a goroutine to do the copy, because you can use session.Start to start the process asynchronously.
You probably don't need to use bufio either. You should be using io.Copy to copy the file, which has an internal buffer already on top of any buffering already done in the ssh client itself. If an additional buffer is needed for performance, wrap the session output in a bufio.Reader
Finally, you return an error value, so use it rather than panic'ing on regular error conditions.
conn, err := ssh.Dial("tcp", hostname+":22", config)
if err != nil {
return err
}
session, err := conn.NewSession()
if err != nil {
return err
}
defer session.Close()
r, err := session.StdoutPipe()
if err != nil {
return err
}
name := fmt.Sprintf("%s/backup_folder_%v.tar.gz", path, time.Now().Unix())
file, err := os.OpenFile(name, os.O_APPEND|os.O_WRONLY|os.O_CREATE, 0644)
if err != nil {
return err
}
defer file.Close()
if err := session.Start(cmd); err != nil {
return err
}
n, err := io.Copy(file, r)
if err != nil {
return err
}
if err := session.Wait(); err != nil {
return err
}
return nil
You can try doing something like this:
r, _ := session.StdoutPipe()
reader := bufio.NewReader(r)
go func() {
defer session.Close()
// open file etc
// 10 is the number of bytes you'd like to copy in one write operation
p := make([]byte, 10)
for {
n, err := reader.Read(p)
if err == io.EOF {
break
}
if err != nil {
log.Fatal("err", err)
}
if _, err = file.Write(p[:n]); err != nil {
log.Fatal(err)
}
}
}()
Make sure your goroutines are synchronized properly so output is completeky written to the file.