Extract filename from io.ReadCloser - go

I need to get the filename of certain file(s) that receives backend from the frontend. Backend (implemented in Go) will receive the file as io.ReadCloser. Is there way I could extract it from the io.ReadCloser?

Backend (implemented in Go) will receive the file as io.ReadCloser. Is there way I could extract it from the io.ReadCloser?
No.
Take a look at which methods an io.ReadCloser provides by running go doc io.ReadCloser and note that there isn't a method which will provide a name. So unless you know nothing more that that it is an io.ReadCloser you simply cannot do it.

package main
import (
"errors"
"fmt"
"io"
"os"
)
func fatalln(err error) {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
// hasName interface is an interface that expects types
// that implements it to have "Name() string" method.
type hasName interface {
Name() string
}
func open(name string) (io.ReadCloser, error) {
f, err := os.Open(name)
if err != nil {
return nil, err
}
// f implements io.ReadCloser interface as *os.File
// has Read and Close methods.
return f, nil
}
func main() {
// rc is of the type io.ReadCloser
rc, err := open("example.txt")
if err != nil {
fatalln(err)
}
defer rc.Close()
// Type assetion to check rc's underlying type has
// a method "Name() string".
f, ok := rc.(hasName)
if !ok {
fatalln(errors.New("type assertion failed"))
}
// Yay, type assertion succeeded. Print the name!
fmt.Println("Name:", f.Name())
}

The io.ReadCloser here is a reader for runtime reader which reads file from network as the frontend sends it to backend. You'll have to work on request itself to get that file name.
This is an assumption but in most such cases for file upload, the request is a multipart request. If you have the same situation, you can read the headers, typically Content-Disposition to identify the file type. Go native http.Request has ability to parse the details. You can try this :
formFile, handler, err := r.FormFile("file") // read file from network with key "file"
defer formFile.Close()
fileName := handler.Filename // Get file name

By defining an interface which embeds io.Reader you can require a Name() method up front:
package main
import (
"fmt"
"io"
"log"
"os"
)
type NamedReadCloser interface {
io.ReadCloser
Name() string
}
func doThings(f NamedReadCloser) error {
defer f.Close()
b, err := io.ReadAll(f)
if err != nil {
return err
}
fmt.Printf("Name: %s, Content: %s\n", f.Name(), b)
return nil
}
func main() {
f, err := os.Open("/etc/hosts")
if err != nil {
log.Fatal("Cannot open file: ", err)
}
err = doThings(f)
if err != nil {
log.Fatal("Error doing things: ", err)
}
}
This will only work if what is passed in has a name method, like an *os.File. If it does not, then what you are trying to do is not possible.

You'll have to cast it to a type with a Name method:
package main
import (
"io"
"os"
)
func open(name string) (io.ReadCloser, error) {
return os.Open(name)
}
func main() {
c, e := open("file.txt")
if e != nil {
panic(e)
}
defer c.Close()
f := c.(*os.File)
println(f.Name())
}

Related

gob error encoding body: gob: type not registered for interface: in golang

During RPC communication, the server cannot calculate the correct value.
I'm getting this error, and I've been thinking about it for a long time.
Can you give any advice
package main
import (
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
f "fmt"
"log"
"net"
"net/rpc"
)
type Calc int //RPC 서버에 등록하기 위한 임의의 타입정의
type Args struct {
A, B int
}
type Reply struct { //받을 값
PrivateKey ecdsa.PrivateKey
PublicKey []byte
}
func (c *Calc) Get(args Args, reply *Reply) error {
private, public := NewKeyPair()
reply.PrivateKey = private
reply.PublicKey = public
return nil
}
func main() {
rpc.Register(new(Calc))
ln, err := net.Listen("tcp", ":6000")
if err != nil {
f.Println(err)
return
}
defer ln.Close()
for {
conn, err := ln.Accept()
if err != nil {
continue
}
f.Println("Connected")
defer conn.Close()
go rpc.ServeConn(conn)
}
}
func NewKeyPair() (ecdsa.PrivateKey, []byte) {
curve := elliptic.P256()
private, err := ecdsa.GenerateKey(curve, rand.Reader)
if err != nil {
log.Panic(err)
}
pubKey := append(private.PublicKey.X.Bytes(), private.PublicKey.Y.Bytes()...)
return *private, pubKey
}
---- error ----
2022/06/20 19:58:35 rpc: gob error encoding body: gob: type not registered for interface: elliptic.p256Curve
you can register the interface by calling this
gob.Register(elliptic.P256())
but this will lead you to next error.
Currently its not possible to use gob with elliptic.
Use any version below 1.19,
it will work

Semantic way of http.Response receiver functions in Go

I just started learning GO and wrote this piece of code that writes an http.Response.Body to os.Stdout or to a file, but I'm not happy about the semantics of this.
I want the http.Response struct to have these receiver functions, so I can use it more easily throughout the entire app.
I know that the answers might get flagged as opinionated, but I still wonder, is there a better way of writing this?
Is there some sort of best practice?
package main
import (
"fmt"
"io"
"io/ioutil"
"net/http"
"os"
)
type httpResp http.Response
func main() {
res, err := http.Get("http://www.stackoverflow.com")
if err != nil {
fmt.Println("Error: ", err)
os.Exit(1)
}
defer res.Body.Close()
response := httpResp(*res)
response.toFile("stckovrflw.html")
response.toStdOut()
}
func (r httpResp) toFile(filename string) {
str, err := ioutil.ReadAll(r.Body)
if err != nil {
panic(err)
}
ioutil.WriteFile(filename, []byte(str), 0666)
}
func (r httpResp) toStdOut() {
_, err := io.Copy(os.Stdout, r.Body)
if err != nil {
panic(err)
}
}
On a side note, is there a way to make the http.Get method spit out a custom type that already has access to these receiver functions without the need for casting? So i could do something like this:
func main() {
res, err := http.Get("http://www.stackoverflow.com")
if err != nil {
fmt.Println("Error: ", err)
os.Exit(1)
}
defer res.Body.Close()
res.toFile("stckovrflw.html")
res.toStdOut()
}
Thanks!
You don't have to implement these functions. *http.Response already implements io.Writer:
Write writes r to w in the HTTP/1.x server response format, including the status line, headers, body, and optional trailer.
package main
import (
"net/http"
"os"
)
func main() {
r := &http.Response{}
r.Write(os.Stdout)
}
In the example above, the zero value prints:
HTTP/0.0 000 status code 0
Content-Length: 0
Playground: https://play.golang.org/p/2AUEAUPCA8j
In case you need additional business logic in the write methods, you can embed *http.Response in your defined type:
type RespWrapper struct {
*http.Response
}
func (w *RespWrapper) toStdOut() {
_, err := io.Copy(os.Stdout, w.Body)
if err != nil {
panic(err)
}
}
But then you must construct a variable of type RespWrapper with the *http.Response:
func main() {
// resp with a fake body
r := &http.Response{Body: io.NopCloser(strings.NewReader("foo"))}
// or r, _ := http.Get("example.com")
// construct the wrapper
wrapper := &RespWrapper{Response: r}
wrapper.toStdOut()
}
is there a way to make the http.Get method spit out a custom type
No, the return types of http.Get are (resp *http.Response, err error), that's part of the function signature, you can't change it.

Proper pattern to encapsulate log setup in golang

When trying to move log setup code into a separate function I ran into inability to hide the destination file object from the main function. In the following INCORRECT simplified example the attempt is made to setup log writing to both Stderr and a file via a single function call:
package main
import (
"io"
"log"
"os"
)
func SetupLogging() {
logFile, err := os.OpenFile("test.log", os.O_APPEND|os.O_CREATE, 0666)
if err != nil {
log.Panicln(err)
}
defer logFile.Close()
log.SetOutput(io.MultiWriter(os.Stderr, logFile))
}
func main() {
SetupLogging()
log.Println("Test message")
}
Clearly is does not work because defer closes the log file at the end of the SetupLogging function.
A working example below adds extra code and IMHO looses some clarity if repeated in a larger application as a pattern:
package main
import (
"io"
"log"
"os"
)
func SetupLogging() *os.File {
logFile, err := os.OpenFile("test.log", os.O_APPEND|os.O_CREATE, 0666)
if err != nil {
log.Panicln(err)
}
log.SetOutput(io.MultiWriter(os.Stderr, logFile))
return logFile
}
func main() {
logf := SetupLogging()
defer logf.Close()
log.Println("Test message")
}
Is there a different way to fully encapsulate open file management into a function, yet still nicely release the handle?
I have now successfully used the below approach for about a year in multiple projects. The idea is to return a function from the setup call. That resulting function contains the destruction logic. Here is an example:
package main
import (
"fmt"
"io"
"log"
"os"
)
func LogSetupAndDestruct() func() {
logFile, err := os.OpenFile("test.log", os.O_CREATE|os.O_APPEND|os.O_RDWR, 0666)
if err != nil {
log.Panicln(err)
}
log.SetOutput(io.MultiWriter(os.Stderr, logFile))
return func() {
e := logFile.Close()
if e != nil {
fmt.Fprintf(os.Stderr, "Problem closing the log file: %s\n", e)
}
}
}
func main() {
defer LogSetupAndDestruct()()
log.Println("Test message")
}
It is using a closure around the cleanup logic being deferred.
A somewhat more elaborate public example of using this approach is in the Viper code: here is the return from a test initializer, and here it is used to encapsulate the cleanup logic and objects
The proper way of doing this is passing the handle in main to SetupLogging:
func SetupLogging(lf *os.File) {
log.SetOutput(io.MultiWriter(os.Stderr, logFile))
log.Println("Started")
}
func main() {
logFile, err := os.OpenFile("test.log", os.O_APPEND|os.O_CREATE, 0666)
if err != nil {
log.Panicln(err)
}
defer logFile.Close()
SetupLogging(logFile)
log.Println("Test message")
}
Another option is to use runtime.SetFinalizer, but it's not always guaranteed to run before main exits.
func SetupLogging() {
logFile, err := os.OpenFile("test.log", os.O_APPEND|os.O_CREATE, 0666)
if err != nil {
log.Panicln(err)
}
runtime.SetFinalizer(logFile, func(h *os.File) {
h.Close()
})
log.SetOutput(io.MultiWriter(os.Stderr, logFile))
}
You can do this using channels, here is my approach
type InfoLog struct {
InfoChan chan string
CloseChan chan struct{} //empty signal
log *log.Logger
file *os.File
}
func NewInfoLog(file *os.File) *InfoLog {
return &InfoLog{
InfoChan: make(chan string),
CloseChan: make(chan struct{}),
log: log.New(file, "TAG", log.Ldate|log.Ltime),
file: file,
}
}
func (i *InfoLog) listen() {
for {
select {
case infoMsg := <-i.InfoChan:
i.log.Println(infoMsg)
case <-i.CloseChan:
i.file.Close()
close(i.InfoChan)
}
}
}
then in main
func main() {
infoLog := NewInfoLog(ANY_OPEN_FILE_HERE)
go infoLog.listen()
infoLog.InfoChan <- "msg"
infoLog.InfoChan <- "msg"
infoLog.InfoChan <- "msg"
infoLog.CloseChan <- struct{}{}
// exits normaly
}
you can see an asynchronous log system i have made for a complete example: https://github.com/sescobb27/ciudad-gourmet/blob/master/services/log_service.go
in case where multiple "teardown" processes are needed, great solution to this is using google context package (https://blog.golang.org/context). advantage is that you can teardown all currently executing procedures with single context. smth like this:
package main
import (
"fmt"
"io"
"log"
"os"
"golang.org/x/net/context"
)
func LogSetup(ctx context.Context) error {
logFile, err := os.OpenFile("test.log", os.O_CREATE|os.O_APPEND|os.O_RDWR, 0666)
if err != nil {
return err
}
log.SetOutput(io.MultiWriter(os.Stderr, logFile))
// here we could f.ex. execute:
// sendLogOutputToExternalService(ctx)
// and it could have it's own teardown procedure
// which would be called on main context's expiration
go func() {
for _ = range ctx.Done() {
err := logFile.Close()
if err = nil {
fmt.Fprintf(os.Stderr, "Problem closing the log file: %s\n", e)
}
}()
return nil
}
func main() {
var stopAll func()
mainContext, stopAll = context.WithCancel(context.Background())
defer stopAll()
err := LogSetup(mainContext)
if err!=nil {
log.Fatal("error while initializing logging")
}
log.Println("Test message")
}

Save an image from url to file

Very new to Go (first simple project I'm working on).
Question: How do I get an image from URL and then save it to my computer?
Here's what I have so far:
package main
import (
"fmt"
"net/http"
"image"
"io/ioutil"
)
func main() {
url := "http://i.imgur.com/m1UIjW1.jpg"
// don't worry about errors
response, _ := http.Get(url);
defer response.Body.Close()
m, _, err := image.Decode(response.Body)
error := ioutil.WriteFile("/images/asdf.jpg", m, 0644)
}
However, when I run this code, I get cannot use m (type image.Image) as type []byte in function argument
I'm assuming I have to convert image.Image (variable m) into an undefined amount of bytes? Is that the correct way to go about this?
There is no need to decode the file. Simply copy the response body to a file you've opened. Here's the deal in the modified example:
response.Body is a stream of data, and implements the Reader interface - meaning you can sequentially call Read on it, as if it was an open file.
The file I'm opening here implements the Writer interface. This is the opposite - it's a stream you can call Write on.
io.Copy "patches" a reader and a writer, consumes the reader stream and writes its contents to a Writer.
This is one of my favorite things about go - implicit interfaces. You don't have to declare you're implementing an interface, you just have to implement it to be used in some context. This allows mixing and matching of code that doesn't need to know about other code it's interacting with.
package main
import (
"fmt"
"io"
"log"
"net/http"
"os"
)
func main() {
url := "http://i.imgur.com/m1UIjW1.jpg"
// don't worry about errors
response, e := http.Get(url)
if e != nil {
log.Fatal(e)
}
defer response.Body.Close()
//open a file for writing
file, err := os.Create("/tmp/asdf.jpg")
if err != nil {
log.Fatal(err)
}
defer file.Close()
// Use io.Copy to just dump the response body to the file. This supports huge files
_, err = io.Copy(file, response.Body)
if err != nil {
log.Fatal(err)
}
fmt.Println("Success!")
}
package main
import (
"io"
"net/http"
"os"
"fmt"
)
func main() {
img, _ := os.Create("image.jpg")
defer img.Close()
resp, _ := http.Get("http://i.imgur.com/Dz2r9lk.jpg")
defer resp.Body.Close()
b, _ := io.Copy(img, resp.Body)
fmt.Println("File size: ", b)
}
Try this:
package main
import (
"fmt"
"net/http"
"net/url"
"os"
"strings"
)
var (
fileName string
fullUrlFile string
)
func main() {
fullUrlFile = "https://i.imgur.com/m1UIjW1.jpg"
r, e := http.Get(fullUrlFile)
if e != nil {
panic(e)
}
defer r.Body.Close()
buildFileName()
// Create distination
f, e := os.Create(fileName) // "m1UIjW1.jpg"
if e != nil {
panic(e)
}
defer f.Close()
// Fill distination with content
n, e := f.ReadFrom(r.Body)
if e != nil {
panic(e)
}
fmt.Println("File size: ", n)
}
func buildFileName() {
fileUrl, e := url.Parse(fullUrlFile)
if e != nil {
panic(e)
}
path := fileUrl.Path
segments := strings.Split(path, "/")
fileName = segments[len(segments)-1]
println(fileName)
}
You can also use the ReadFrom method:
package main
import (
"net/http"
"os"
)
func main() {
r, e := http.Get("https://i.imgur.com/m1UIjW1.jpg")
if e != nil {
panic(e)
}
defer r.Body.Close()
f, e := os.Create("m1UIjW1.jpg")
if e != nil {
panic(e)
}
defer f.Close()
f.ReadFrom(r.Body)
}
https://golang.org/pkg/os#File.ReadFrom
What is the type of response.Body? You should just convert that into a []byte if it is not and write that to disk. There is no reason to use the image class unless you have some reason to treat the data as an image. Just treat the data as a series of bytes and write it to the disk.

Method declarations with receivers in Go

The following errors:
./main.go:13: c.Set undefined (type redis.Conn has no field or method Set)
./main.go:19: invalid receiver type *redis.Conn (redis.Conn is an interface type)
./main.go:20: red.Send undefined (type *redis.Conn has no field or method Send)
are produced from this code:
package main
import (
"encoding/json"
"github.com/garyburd/redigo/redis"
"github.com/gorilla/mux"
"log"
"net/http"
)
func setHandler(res http.ResponseWriter, req *http.Request) {
c := connectRedis()
c.Set("foo", "bar")
data, _ := json.Marshal("{'order':1, 'weight': 100, 'reps': 5, 'rest': 1}")
res.Header().Set("Content-Type", "application/json; charset=utf-8")
res.Write(data)
}
func (red *redis.Conn) Set(key string, value string) error {
if _, err := red.Send("set", key, value); err != nil {
return err
}
}
func connectRedis() redis.Conn {
c, err := redis.Dial("tcp", ":6379")
if err != nil {
// handle error
}
defer c.Close()
return c
}
func main() {
r := mux.NewRouter()
r.HandleFunc("/sets.json", setHandler)
http.Handle("/", r)
err := http.ListenAndServe(":7000", nil)
if err != nil {
log.Fatal("ListenAndServe: ", err)
}
}
How can I register a method when I have an interface?
You are trying to add a new method to a preexisting structure from another package; you cannot do that.
The right way to do it would be to define a structure in your own package which would alias the preexisting structure and inherit all of its methods. After doing that you will be able to add your new method to your new structure.
After that you can use your own structure everywhere instead so you can access the extra methods.
You might be tempted now to see if you can attach methods and behavior to any type, say like an int or time.Time - not possible. You will be able to add methods for a type only if the type is defined in the same package.
from GoLang Tutorials - Methods on structs

Resources