Why would f.Write() not return any error if I remove the file before I write?
package main
import (
"fmt"
"os"
"time"
)
func main() {
f, err := os.Create("foo")
if err != nil {
panic(err)
}
if err := os.Remove("foo"); err != nil {
panic(err)
}
if _, err := f.Write([]byte("hello")); err != nil {
panic(err) // would expect panic here
}
fmt.Println("no panic?")
}
http://play.golang.org/p/0QllIB6L9O
Apparently this is expected.
When you delete a file you really remove a link to the file (to the inode). If someone already has that file open, they get to keep the file descriptor they have. The file remains on disk, taking up space, and can be written to and read from if you have access to it.
Source: https://unix.stackexchange.com/questions/146929/how-can-a-log-program-continue-to-log-to-a-deleted-file
Related
I am try to use go-bindata and packr, but those packages do not show how to pack an SQLite database file in to a binary file.
I don't need to update the database in any way, I just want to read the data from it on startup.
How can I embed an SQLite database file in a Go binary file?
The SQLite driver can't read a database file from memory (e.g. from a byte slice). But you can write the data to a temporary file, and open that:
//go:generate go run gen.go
package main
import (
"database/sql"
"fmt"
"io/ioutil"
"log"
"os"
_ "github.com/mattn/go-sqlite3"
)
func main() {
// Create temporary file for database.
tmpDB, err := ioutil.TempFile("", "db*.sqlite3")
if err != nil {
log.Fatal(err)
}
// Remove this file after on exit.
defer func() {
err := os.Remove(tmpDB.Name())
if err != nil {
log.Print(err)
}
}()
// Write database to file.
_, err = tmpDB.Write(sqlDB)
if err != nil {
log.Print(err)
}
err = tmpDB.Close()
if err != nil {
log.Print(err)
}
// Open DB.
db, err := sql.Open("sqlite3", tmpDB.Name()+"?mode=ro")
if err != nil {
log.Fatal(err)
}
// Make sure it's loaded correct.
rows, err := db.Query("select * from test")
if err != nil {
log.Fatal(err)
}
for rows.Next() {
var c string
err := rows.Scan(&c)
if err != nil {
log.Fatal(err)
}
fmt.Println(c)
}
}
And you can write the database to db.go with something like:
// +build generate
package main
import (
"fmt"
"io/ioutil"
"log"
"os"
"strings"
)
func main() {
// Read source database file.
d, err := ioutil.ReadFile("source.sqlite3")
if err != nil {
log.Fatal(err)
}
fp, err := os.Create("db.go")
if err != nil {
log.Fatal(err)
}
_, err = fmt.Fprintf(fp, "// Code generated by gen.go; DO NOT EDIT.\n\n"+
"package main\n\n"+
"var sqlDB = %s\n", asbyte(d))
if err != nil {
log.Fatal(err)
}
}
// Write any data as byte array.
func asbyte(s []byte) string {
var b strings.Builder
for i, c := range s {
if i%19 == 0 {
b.WriteString("\n\t\t")
}
b.WriteString(fmt.Sprintf("%#x, ", c))
}
return "[]byte{" + b.String() + "}"
}
You can also use go-bindata or packr for that if you prefer, but I don't really see an advantage.
An alternative way is to use a memory database, which may be faster depending on what you want to do.
Embed the SQL schema and rows you want in your Go binary as strings.
Open a new memory database when your program starts (sql.Open("sqlite3",:memory:`) and create the schema and insert the rows.
There is no disk access with this method, so querying it will probably be a bit faster at the expensive of slower startup times (benchmark to be sure!)
I am trying to get the content of a publicly available file using ioutil.ReadFile() but it doesn't find the file: panic: open http://www.pdf995.com/samples/pdf.pdf: No such file or directory
Here's my code:
// Reading and writing files are basic tasks needed for
// many Go programs. First we'll look at some examples of
// reading files.
package main
import (
"fmt"
"io/ioutil"
)
// Reading files requires checking most calls for errors.
// This helper will streamline our error checks below.
func check(e error) {
if e != nil {
panic(e)
}
}
func main() {
fileInUrl, err := ioutil.ReadFile("http://www.pdf995.com/samples/pdf.pdf")
if err != nil {
panic(err)
}
fmt.Printf("HERE --- fileInUrl: %+v", fileInUrl)
}
Here's a go playground example
ioutil.ReadFile() does not support http.
If you look at the source code(https://golang.org/src/io/ioutil/ioutil.go?s=1503:1549#L42), open the file using os.Open.
I think I can do this coding.
package main
import (
"io"
"net/http"
"os"
)
func main() {
fileUrl := "http://www.pdf995.com/samples/pdf.pdf"
if err := DownloadFile("example.pdf", fileUrl); err != nil {
panic(err)
}
}
func DownloadFile(filepath string, url string) error {
// Get the data
resp, err := http.Get(url)
if err != nil {
return err
}
defer resp.Body.Close()
// Create the file
out, err := os.Create(filepath)
if err != nil {
return err
}
defer out.Close()
// Write the body to file
_, err = io.Copy(out, resp.Body)
return err
}
but, go playgound not protocol(go error dial tcp: Protocol not available).
so, You have to do it PC.
This question already has answers here:
How can I read a whole file into a string variable
(7 answers)
Closed 4 years ago.
I'm trying to read "file.txt" and put the contents into a variable using Golang. Here is what I've tried...
package main
import (
"fmt"
"os"
"log"
)
func main() {
file, err := os.Open("file.txt")
if err != nil {
log.Fatal(err)
}
fmt.Print(file)
}
The file gets read successfully and the return from os.Open returns a type of *os.File
It depends on what you are trying to do.
file, err := os.Open("file.txt")
fmt.print(file)
The reason it outputs &{0xc082016240}, is because you are printing the pointer value of a file-descriptor (*os.File), not file-content. To obtain file-content, you may READ from a file-descriptor.
To read all file content(in bytes) to memory, ioutil.ReadAll
package main
import (
"fmt"
"io/ioutil"
"os"
"log"
)
func main() {
file, err := os.Open("file.txt")
if err != nil {
log.Fatal(err)
}
defer func() {
if err = file.Close(); err != nil {
log.Fatal(err)
}
}()
b, err := ioutil.ReadAll(file)
fmt.Print(b)
}
But sometimes, if the file size is big, it might be more memory-efficient to just read in chunks: buffer-size, hence you could use the implementation of io.Reader.Read from *os.File
func main() {
file, err := os.Open("file.txt")
if err != nil {
log.Fatal(err)
}
defer func() {
if err = file.Close(); err != nil {
log.Fatal(err)
}
}()
buf := make([]byte, 32*1024) // define your buffer size here.
for {
n, err := file.Read(buf)
if n > 0 {
fmt.Print(buf[:n]) // your read buffer.
}
if err == io.EOF {
break
}
if err != nil {
log.Printf("read %d bytes: %v", n, err)
break
}
}
}
Otherwise, you could also use the standard util package: bufio, try Scanner. A Scanner reads your file in tokens: separator.
By default, scanner advances the token by newline (of course you can customise how scanner should tokenise your file, learn from here the bufio test).
package main
import (
"fmt"
"os"
"log"
"bufio"
)
func main() {
file, err := os.Open("file.txt")
if err != nil {
log.Fatal(err)
}
defer func() {
if err = file.Close(); err != nil {
log.Fatal(err)
}
}()
scanner := bufio.NewScanner(file)
for scanner.Scan() { // internally, it advances token based on sperator
fmt.Println(scanner.Text()) // token in unicode-char
fmt.Println(scanner.Bytes()) // token in bytes
}
}
Lastly, I would also like to reference you to this awesome site: go-lang file cheatsheet. It encompassed pretty much everything related to working with files in go-lang, hope you'll find it useful.
Hello again everybody,
I apologize for asking another question so soon, but it just seems to be one thing after another with Go lately.
I have a working web page scraper (thanks to everybody's help) that grabs all the info I want from this wiki page: http://monsterhunter.wikia.com/wiki/MH4U:_Item_List
It then displays everything I want, no hiccups. However, when I go to write to a .txt file I get an error stating: "0 write mh4u.txt: The handle is invalid"
Here is my current code for reference:
package main
import (
"fmt"
"log"
"github.com/PuerkitoBio/goquery"
"os"
"io"
)
func main() {
filename := "mh4u.txt"
file, err := os.Create(filename)
if err!= nil {
fmt.Println(err)
}
doc, err := goquery.NewDocument("http://www.ign.com/wikis/monster-hunter-4/Items")
if err != nil {
log.Fatal(err)
}
doc.Find("tbody").Each(func(i int, s *goquery.Selection) {
s.Find("td").Each(func(j int, s2 *goquery.Selection) {
if s3 := s2.Find("img"); s3 != nil && s3.Length() > 0 {
return
}
fmt.Printf(s2.Text())
n, err := io.WriteString(file, s2.Text())
if err != nil {
fmt.Println(n, err)
}
})
file.Close()
})
}
Testing this code with other web sites leads me to believe that there is maybe some hidden characters that is giving the writer some issues, but that's the only thing I can guess.
Thanks so much for any suggestions/tips/solutions you can offer!
You're calling file.Close() inside a closure. If doc.Find("tbody").Each is called more than one time, you'll end up trying to write to a closed file. You should defer the file closing right after you've created it:
file, err := os.Create(filename)
if err != nil {
log.Fatal(err)
}
defer file.Close()
I'm trying to read a .txt file using Go, but I keep getting told that the program cannot find the specified file, even when I use the complete path to it. However, the code I have has no problem reading .go files.
Help?
package main
import (
"bufio"
"fmt"
"log"
"os"
)
func main() {
// Open an input file, exit on error.
inputFile, err := os.Open("main.go");
if err != nil {
log.Fatal("Error opening input file:", err)
}
defer inputFile.Close()
scanner := bufio.NewScanner(inputFile)
for scanner.Scan() {
fmt.Println(scanner.Text())
}
if err := scanner.Err(); err != nil {
log.Fatal(scanner.Err())
}
}
You can see where you are adding this snippet of code at the beginning of your main function:
http://play.golang.org/p/DqnivLi1Z2
cwd, err := os.Getwd()
if err != nil {
log.Fatal("os.Getwd ", err)
}
log.Println("Current Directory", cwd)
files, err := ioutil.ReadDir(cwd)
if err != nil {
log.Fatal("ioutil.ReadDir", err)
}
for idx, finfo := range files {
log.Println(idx, " ", finfo.Name)
}