Hello again everybody,
I apologize for asking another question so soon, but it just seems to be one thing after another with Go lately.
I have a working web page scraper (thanks to everybody's help) that grabs all the info I want from this wiki page: http://monsterhunter.wikia.com/wiki/MH4U:_Item_List
It then displays everything I want, no hiccups. However, when I go to write to a .txt file I get an error stating: "0 write mh4u.txt: The handle is invalid"
Here is my current code for reference:
package main
import (
"fmt"
"log"
"github.com/PuerkitoBio/goquery"
"os"
"io"
)
func main() {
filename := "mh4u.txt"
file, err := os.Create(filename)
if err!= nil {
fmt.Println(err)
}
doc, err := goquery.NewDocument("http://www.ign.com/wikis/monster-hunter-4/Items")
if err != nil {
log.Fatal(err)
}
doc.Find("tbody").Each(func(i int, s *goquery.Selection) {
s.Find("td").Each(func(j int, s2 *goquery.Selection) {
if s3 := s2.Find("img"); s3 != nil && s3.Length() > 0 {
return
}
fmt.Printf(s2.Text())
n, err := io.WriteString(file, s2.Text())
if err != nil {
fmt.Println(n, err)
}
})
file.Close()
})
}
Testing this code with other web sites leads me to believe that there is maybe some hidden characters that is giving the writer some issues, but that's the only thing I can guess.
Thanks so much for any suggestions/tips/solutions you can offer!
You're calling file.Close() inside a closure. If doc.Find("tbody").Each is called more than one time, you'll end up trying to write to a closed file. You should defer the file closing right after you've created it:
file, err := os.Create(filename)
if err != nil {
log.Fatal(err)
}
defer file.Close()
Related
I am trying to get the content of a publicly available file using ioutil.ReadFile() but it doesn't find the file: panic: open http://www.pdf995.com/samples/pdf.pdf: No such file or directory
Here's my code:
// Reading and writing files are basic tasks needed for
// many Go programs. First we'll look at some examples of
// reading files.
package main
import (
"fmt"
"io/ioutil"
)
// Reading files requires checking most calls for errors.
// This helper will streamline our error checks below.
func check(e error) {
if e != nil {
panic(e)
}
}
func main() {
fileInUrl, err := ioutil.ReadFile("http://www.pdf995.com/samples/pdf.pdf")
if err != nil {
panic(err)
}
fmt.Printf("HERE --- fileInUrl: %+v", fileInUrl)
}
Here's a go playground example
ioutil.ReadFile() does not support http.
If you look at the source code(https://golang.org/src/io/ioutil/ioutil.go?s=1503:1549#L42), open the file using os.Open.
I think I can do this coding.
package main
import (
"io"
"net/http"
"os"
)
func main() {
fileUrl := "http://www.pdf995.com/samples/pdf.pdf"
if err := DownloadFile("example.pdf", fileUrl); err != nil {
panic(err)
}
}
func DownloadFile(filepath string, url string) error {
// Get the data
resp, err := http.Get(url)
if err != nil {
return err
}
defer resp.Body.Close()
// Create the file
out, err := os.Create(filepath)
if err != nil {
return err
}
defer out.Close()
// Write the body to file
_, err = io.Copy(out, resp.Body)
return err
}
but, go playgound not protocol(go error dial tcp: Protocol not available).
so, You have to do it PC.
I will be working on a project connected with GIF images and I tried to do some basic operations on them in Go (such as retrieving frames or creating GIF from a bunch of images). But for now let's do a simple example in which I am only trying to decode a GIF and then to encode it again. I tried to use "image/gif" package, but I am unable to get it to do what I want.
Here is the code :
package main
import (
"os"
"image/gif"
)
func main() {
inputFile , err := os.Open("travolta.gif")
defer inputFile.Close()
if err != nil {
panic(err)
}
g, err := gif.DecodeAll(inputFile)
if err != nil {
panic(err)
}
outputFile, err := os.OpenFile("travolta2.gif", os.O_WRONLY|os.O_CREATE, 0777)
defer outputFile.Close()
if err != nil {
panic(err)
}
err = gif.EncodeAll(outputFile, g)
if err != nil {
panic(err)
}
}
When I run the code it does not panic and another gif is indeed created. Unfortunetely it is corrupted. Moreover the gif's size changes from 3,4MB to 4,4MB. Is it not a way to save/read a gif? What mistake do I make?
EDIT:
By corrupted I mean that when I try to open it an error occurs- screnshot here : http://www.filedropper.com/obrazekpociety.
GIF:
http://vader.joemonster.org/upload/rwu/1539970c1d48acceLw7m.gif
Go version 1.7.4
The problem is you are passing a File* to DecodeAll rather than something that supports the io.Reader interface.
I have adapted your code so that it creates a bufio.Reader from the file and then hands that to DecodeAll as below:
import (
"bufio"
"os"
"image/gif"
)
func main() {
inputFile , err := os.Open("travolta.gif")
defer inputFile.Close()
if err != nil {
panic(err)
}
r := bufio.NewReader(inputFile)
g, err := gif.DecodeAll(r)
if err != nil {
panic(err)
}
// ... remaining code
}
This question already has answers here:
How can I read a whole file into a string variable
(7 answers)
Closed 4 years ago.
I'm trying to read "file.txt" and put the contents into a variable using Golang. Here is what I've tried...
package main
import (
"fmt"
"os"
"log"
)
func main() {
file, err := os.Open("file.txt")
if err != nil {
log.Fatal(err)
}
fmt.Print(file)
}
The file gets read successfully and the return from os.Open returns a type of *os.File
It depends on what you are trying to do.
file, err := os.Open("file.txt")
fmt.print(file)
The reason it outputs &{0xc082016240}, is because you are printing the pointer value of a file-descriptor (*os.File), not file-content. To obtain file-content, you may READ from a file-descriptor.
To read all file content(in bytes) to memory, ioutil.ReadAll
package main
import (
"fmt"
"io/ioutil"
"os"
"log"
)
func main() {
file, err := os.Open("file.txt")
if err != nil {
log.Fatal(err)
}
defer func() {
if err = file.Close(); err != nil {
log.Fatal(err)
}
}()
b, err := ioutil.ReadAll(file)
fmt.Print(b)
}
But sometimes, if the file size is big, it might be more memory-efficient to just read in chunks: buffer-size, hence you could use the implementation of io.Reader.Read from *os.File
func main() {
file, err := os.Open("file.txt")
if err != nil {
log.Fatal(err)
}
defer func() {
if err = file.Close(); err != nil {
log.Fatal(err)
}
}()
buf := make([]byte, 32*1024) // define your buffer size here.
for {
n, err := file.Read(buf)
if n > 0 {
fmt.Print(buf[:n]) // your read buffer.
}
if err == io.EOF {
break
}
if err != nil {
log.Printf("read %d bytes: %v", n, err)
break
}
}
}
Otherwise, you could also use the standard util package: bufio, try Scanner. A Scanner reads your file in tokens: separator.
By default, scanner advances the token by newline (of course you can customise how scanner should tokenise your file, learn from here the bufio test).
package main
import (
"fmt"
"os"
"log"
"bufio"
)
func main() {
file, err := os.Open("file.txt")
if err != nil {
log.Fatal(err)
}
defer func() {
if err = file.Close(); err != nil {
log.Fatal(err)
}
}()
scanner := bufio.NewScanner(file)
for scanner.Scan() { // internally, it advances token based on sperator
fmt.Println(scanner.Text()) // token in unicode-char
fmt.Println(scanner.Bytes()) // token in bytes
}
}
Lastly, I would also like to reference you to this awesome site: go-lang file cheatsheet. It encompassed pretty much everything related to working with files in go-lang, hope you'll find it useful.
Why would f.Write() not return any error if I remove the file before I write?
package main
import (
"fmt"
"os"
"time"
)
func main() {
f, err := os.Create("foo")
if err != nil {
panic(err)
}
if err := os.Remove("foo"); err != nil {
panic(err)
}
if _, err := f.Write([]byte("hello")); err != nil {
panic(err) // would expect panic here
}
fmt.Println("no panic?")
}
http://play.golang.org/p/0QllIB6L9O
Apparently this is expected.
When you delete a file you really remove a link to the file (to the inode). If someone already has that file open, they get to keep the file descriptor they have. The file remains on disk, taking up space, and can be written to and read from if you have access to it.
Source: https://unix.stackexchange.com/questions/146929/how-can-a-log-program-continue-to-log-to-a-deleted-file
I'm trying to read a .txt file using Go, but I keep getting told that the program cannot find the specified file, even when I use the complete path to it. However, the code I have has no problem reading .go files.
Help?
package main
import (
"bufio"
"fmt"
"log"
"os"
)
func main() {
// Open an input file, exit on error.
inputFile, err := os.Open("main.go");
if err != nil {
log.Fatal("Error opening input file:", err)
}
defer inputFile.Close()
scanner := bufio.NewScanner(inputFile)
for scanner.Scan() {
fmt.Println(scanner.Text())
}
if err := scanner.Err(); err != nil {
log.Fatal(scanner.Err())
}
}
You can see where you are adding this snippet of code at the beginning of your main function:
http://play.golang.org/p/DqnivLi1Z2
cwd, err := os.Getwd()
if err != nil {
log.Fatal("os.Getwd ", err)
}
log.Println("Current Directory", cwd)
files, err := ioutil.ReadDir(cwd)
if err != nil {
log.Fatal("ioutil.ReadDir", err)
}
for idx, finfo := range files {
log.Println(idx, " ", finfo.Name)
}