I use a function that requires a filename as a parameter (of type string). It works fine when providing the filename.
I would like to embed this file in my binary. I can then have the contents as []byte or string but that's not useful. I can also fave it as embed.FS but my understanding is that this is an abstraction that can be used by some functions only.
What I would need is the ability to present this embedded file as a filename (a string) that would then be used by the underlying function to open the (embedded) file.
Is this possible?
Filename that Key accept as string argument is only abstraction on ioutil.ReadFile, see auth.go
What you can do is implement ssh.Auth yourself, here is small example.
package main
import (
_ "embed"
"fmt"
"github.com/melbahja/goph"
"golang.org/x/crypto/ssh"
)
//go:embed id_rsa
var privateKey []byte
func main() {
auth, err := Auth(privateKey, []byte("foobar"))
fmt.Println(auth, err)
}
func Auth(privateKey, pass []byte) (goph.Auth, error) {
signer, err := Singer(privateKey, pass)
if err != nil {
return nil, err
}
return goph.Auth{
ssh.PublicKeys(signer),
}, nil
}
func Singer(privateKey, pass []byte) (ssh.Signer, error) {
if len(pass) != 0 {
return ssh.ParsePrivateKeyWithPassphrase(privateKey, pass)
}
return ssh.ParsePrivateKey(privateKey)
}
Related
I'm trying to organize my code using interfaces in Go.
I have 2 sources of data: FTP and API. In each source, I have several structs that make the logic vary depending on the case.
In this question, I will omit API and stick with FTP.
My problem comes from the impossibility to say: FTPAcq is also an Acquisition
If FetchMeters(), when I do ftp.Decrypt(nil) I would like ftp to be "compatible" with FTPAcq
Here is my code:
package main
import (
"github.com/dutchcoders/goftp"
log "github.com/sirupsen/logrus"
"os"
)
type Acquisition interface {
FetchMeters() ([]Meter, error)
Name() string
}
type FTPAcq interface {
Unzip(file string) string
Decrypt(file string) string
}
//type APIAcq interface {
// FetchMeter(meterID string) (Meter, error)
//}
func main() {
var acqs []Acquisition
ftp, err := NewFTPDriver(os.Getenv("FTP_USER"), os.Getenv("FTP_PASSWD"), os.Getenv("FTP_ADDR"), os.Getenv("FTP_PORT"))
if err != nil {
panic(err)
}
ftp1 := NewFTPDriverSGE(*ftp)
ftp2 := NewFTPDriverTA(*ftp)
acqs = append(acqs, ftp1, ftp2)
for _, acq := range acqs {
tmpMeters, err := acq.FetchMeters()
if err != nil {
log.Warn(acq.Name(), " got error :", err)
}
log.Info(tmpMeters)
}
}
type Meter struct {
ID string
OperationID string
Unit string
}
//FtpSGE is a implementation of acquisition Interface (see driver.go)
type FTP struct {
Username string
Password string
Url string
Port string
client *goftp.FTP
}
type FTPSGE struct {
FTP
}
type FTPTA struct {
FTP
}
func (f FTPSGE) Unzip(path string) []string {
return nil
}
func (f FTPTA) Unzip(path string) []string {
return nil
}
func (f FTPSGE) Decrypt(path string) []string {
return nil
}
func (f FTPTA) Decrypt(path string) []string {
return nil
}
func (ftp FTP) FetchMeters() ([]Meter, error) {
log.Info(ftp.Name(), " is running")
files := ftp.Download(nil)
files = ftp.Decrypt("") // I have several implementation of Decrypt
files = ftp.Unzip("") // I have several implementation of Unzip
log.Info(files)
return nil, nil
}
func (ftp FTP) Name() string {
panic("FTP ")
}
func (ftp FTP) Download(files []string) []string {
panic("implement me")
}
func NewFTPDriver(user, password, url, port string) (*FTP, error) {
var err error
ftp := &FTP{
Username: user,
Password: password,
Url: url,
Port: port,
}
if ftp.client, err = goftp.Connect(url + ":" + port); err != nil {
return nil, err
}
if err = ftp.client.Login(user, password); err != nil {
return nil, err
}
return ftp, nil
}
func NewFTPDriverSGE(f FTP) *FTPSGE {
return &FTPSGE{f}
}
func NewFTPDriverTA(f FTP) *FTPTA {
return &FTPTA{f}
}
In my case, I get:
ftp.Decrypt undefined (type FTP has no field or method Decrypt)
How should I do?
FTP does not implement FTPAcq. It implements only Acquisition. It doesn't even have Decrypt() as a method, interface or not.
FTPSGE and FTPTA implement FTPAcq, but they are not the same type as FTP.
I don't know what you're trying to accomplish, but perhaps something to try is embedding FTP in FTPSGE and FTPTA. This gives those 2 types the fields and methods of the embedded type and still allows you to define additional methods on those types (the methods for FTPAcq in your case).
For example
type FTPSGE {
FTP
}
// OR
type FTPSGE {
*FTP
}
Which you then create as so: x := FTPSGE{ftp1}. Keep in mind that this will create a copy of ftp1 inside x. If ftp1 is type FTP (not a pointer), the entire struct gets copied. If ftp1 is type *FTP (a pointer, which seems to be what you're using), the pointer is copied and x.FTP still points to the same data as ftp1.
This means FTPSGE will implement both Acquisition and FTPAcq.
You'll have to be careful of whether or not interfaces are implemented on the value or pointer: func (a A) Something() vs func (a *A) Somthing().
Here is some reading on methods, interfaces, and embedding.
https://www.ardanlabs.com/blog/2014/05/methods-interfaces-and-embedded-types.html
https://travix.io/type-embedding-in-go-ba40dd4264df
https://golang.org/doc/effective_go.html (and lots of other info)
I've method which I want to provide some interface to make it more easier to test
This is the function
File A
func readFile(s source) ([]byte, error) {
p := fs.GetPath()
file, err := ioutil.ReadFile(p + "/" + s.path + "/" + "rts.yaml")
if err != nil {
return yamlFile, fmt.Errorf("erro reading file : %s", err.Error())
}
return file, err
}
Now I add for it struct
type source struct{
path string
}
And the interface that the readFile is implementing
type fileReader interface {
readFile(path string) ([]byte, error)
}
And now I need to call this function from another file but Im getting error while doing this
File B
type source struct {
path string
}
a := source{}
yamlFile, err := readFile(a)
what am I missing here ?
Import the package containing the source struct in File A and then use that struct to initialize the variable after that pass the variable to the readFile function.
File B
import A
a := A.Source{}
Because source struct in File A is different from source struct in File B. And source struct of File A is implementing the interface that's why you need to import the source struct and then pass it into the function.
One this should be noticed that to make any struct or function exportable you should start the struct or fucntion name with upper case letter.
File A
// make struct exportable
type Source struct{
path string
}
implemented the interface which is different from
File B
type source struct{
path string
}
which does not implemented the interface.
Edited
File A
package main
import (
"fmt"
"io/ioutil"
"os"
)
type Source struct {
Path string
}
type fileReader interface {
readOneFile() ([]byte, error)
}
func(s Source) readOneFile() ([]byte, error) {
cwd, err := os.Getwd()
file, err := ioutil.ReadFile(fmt.Sprintf("%s/file.txt", cwd))
if err != nil {
return nil, fmt.Errorf("erro reading file : %s", err.Error())
}
return file, err
}
File B
package main
import (
"fmt"
)
func main() {
s := Source{}
data, err := s.readOneFile()
if err != nil {
fmt.Errorf("Error in reading the file")
}
fmt.Println(string(data))
}
I am having a function that opens a file and writes to it using the os.OpenFile function.
What I basically want to do is mock the File that is getting returned by it in order to write tests. Because I want to better understand Go, I'd like to do it without using third party packages.
Here is what I have tried:
My Package
package logger
import (
"fmt"
"time"
"sync"
"os"
"strings"
"path/filepath"
"io"
)
const timestampFormat = "2006-01-02 15:04:05.999999999"
type FileOpener interface {
OpenFile(name string, flag int, perm os.FileMode) (*os.File, error)
}
type RotateWriter struct {
fileOpener FileOpener
lock sync.Mutex
filename string
fp *os.File
}
type defaultFileOpener struct{}
func (fo defaultFileOpener) OpenFile(name string, flag int, perm os.FileMode) (*os.File, error) {
return os.OpenFile(name, flag, perm)
}
func CreateRotateWriter(filename string, fileOpener FileOpener) (RotateWriter) {
if (fileOpener == nil) {
return RotateWriter{filename: filename, fileOpener: defaultFileOpener{}}
}
return RotateWriter{filename: filename, fileOpener: fileOpener}
}
func (writer RotateWriter) Write(bytes []byte) (int, error) {
writer.lock.Lock()
defer writer.lock.Unlock()
extension := filepath.Ext(writer.filename)
filename := strings.TrimSuffix(writer.filename, extension)
// There is a specific constants that are used for formatting dates.
// For example 2006 is the YYYYY, 01 is MM and 02 is DD
// Check https://golang.org/src/time/format.go line 88 for reference
fullFilename := filename + time.Now().UTC().Format("-2006-01-02") + extension
// Open the file for the first time if we don't already did so
if writer.fp == nil {
fp, err := os.OpenFile(fullFilename, os.O_APPEND|os.O_WRONLY|os.O_CREATE, 0666)
if err != nil {
return 0, err
}
writer.fp = fp;
}
// We are now pointing to a different file. Close the previous one and open a new one
if fullFilename != writer.fp.Name() {
writer.fp.Close()
fp, err := os.OpenFile(fullFilename, os.O_APPEND|os.O_WRONLY|os.O_CREATE, 0666)
if err != nil {
return 0, err
}
writer.fp = fp;
}
return writer.fp.Write([]byte("[" + time.Now().UTC().Format(timestampFormat) + "] " + string(bytes)))
}
What I was hoping to do in my test package is something like this
type file interface {
io.Closer
io.Reader
io.ReaderAt
io.Seeker
Stat() (os.FileInfo, error)
}
type fileType struct{
fd int
name string
contents string // Where I'll keep the in-memory contents maybe
}
type defaultFileOpener struct{
}
func (fo defaultFileOpener) OpenFile(name string, flag int, perm os.FileMode) (*file, error){
return &fileType{1, name, ""}, nil //Cannot use &fileType{1, name, ""}(type *fileType) as type *file
}
func (f fileType) Write(bytes []byte) (int, error){
f.contents += string(bytes)
return len(bytes), nil
}
I'm most probably misunderstand something, is it even possible to create my own File type in go?
From the snippet it's not clear whether or not the *fileType implements all of the methods of the file interface, but if it doesn't you should first make sure that it does. Because if it doesn't you'll not be able to use it in your tests as you might intend.
Your file opener's OpenFile method should have the interface as its return type, not the pointer to the interface. That is:
func (fo defaultFileOpener) OpenFile(name string, flag int, perm os.FileMode) (file, error) {
This is because *file is not the same type as file, and a value of a type that implements the file interface (e.g. yours *fileType) cannot be used where the pointer to that interface is expected.
And returning a pointer to an interface is almost never what you actually want, maybe it would make sense to do that if you wanted switch the interface value for another using pointer indirection, but that does not seem to be the case here. If you scan through the standard library you'll have a hard time finding functions/methods that return pointers to interface types...
But let's say that that's what you want to do then you have to return a pointer to a value of the interface type, not a pointer to a type that implements the interface.
E.g.
var f file = &fileType{} // f is of type file
return &f, nil // &f is of type *file
Keep in mind that f's type has to be file for the return to work, if it's just *fileType it will not compile.
var f = &fileType{} // f is of type *fileType
return &f, nil // &f is of type **fileType
I've a server in golang who handle folder path like that :
fs := http.FileServer(http.Dir("./assets"))
http.Handle("/Images/", fs)
http.ListenAndServe(":8000", nil)
But in this folder there are privates images, and it shouldn't be possible to access files. So how can i secure image access and prevent anybody to access content of folder.
like that for example :
If you want to block a directory using http package, maybe this will be useful to you :
https://groups.google.com/forum/#!topic/golang-nuts/bStLPdIVM6w
package main
import (
"net/http"
"os"
)
type justFilesFilesystem struct {
fs http.FileSystem
}
func (fs justFilesFilesystem) Open(name string) (http.File, error) {
f, err := fs.fs.Open(name)
if err != nil {
return nil, err
}
return neuteredReaddirFile{f}, nil
}
type neuteredReaddirFile struct {
http.File
}
func (f neuteredReaddirFile) Readdir(count int) ([]os.FileInfo, error) {
return nil, nil
}
func main() {
fs := justFilesFilesystem{http.Dir("/tmp/")}
http.ListenAndServe(":8080", http.FileServer(fs))
}
A little wrapper over FileServer() solves your problem, now you have to add some sort of logic to do Authorization, it looks like you have unique names, that's good, so I just filter the image name for you creating a map of names, now you can add something more dynamic like a key/store(memcached, redis. etc.) Hope you can follow the comments
package main
import (
"log"
"net/http"
"strings"
)
// put the allowed hashs or keys here
// you may consider put them in a key/value store
//
var allowedImages = map[string]bool{
"key-abc.jpg": true,
"key-123.jpg": true,
}
func main() {
http.Handle("/Images/", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// here we can do any kind of checking, in this case we'll just split the url and
// check if the image name is in the allowedImages map, we can check in a DB or something
//
parts := strings.Split(r.URL.Path, "/")
imgName := parts[len(parts)-1]
if _, contains := allowedImages[imgName]; !contains { // if the map contains the image name
log.Printf("Not found image: %q path: %s\n", imgName, r.URL.Path)
// if the image is not found we write a 404
//
// Bonus: we don't list the directory, so nobody can know what's inside :)
//
http.NotFound(w, r)
return
}
log.Printf("Serving allowed image: %q\n", imgName)
fileServer := http.StripPrefix("/Images/", http.FileServer(http.Dir("./assets")))
fileServer.ServeHTTP(w, r) // StripPrefix() and FileServer() return a Handler that implements ServerHTTP()
}))
http.ListenAndServe(":8000", nil)
}
https://play.golang.org/p/ehrd_AWXim
I want to be able to unmarshal yaml files less rigidly. That is, my library has a predefined number of options the yaml file must have. Then, the user should be able to extend this to include any custom options.
Here is what I have
package main
import (
"net/http"
"yamlcms"
"github.com/julienschmidt/httprouter"
)
type Page struct {
*yamlcms.Page
Title string
Date string
}
func getBlogRoutes() {
pages := []*Page{}
yamlcms.ReadDir("html", pages)
}
// This section is a work in progress, I only include it for loose context
func main() {
router := httprouter.New()
//blogRoutes := getBlogRoutes()
//for _, blogRoute := range *blogRoutes {
// router.Handle(blogRoute.Method, blogRoute.Pattern,
// func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {})
//}
http.ListenAndServe(":8080", router)
}
Here is the yamlcms package:
package yamlcms
import (
"io/ioutil"
"os"
"strings"
"gopkg.in/yaml.v2"
)
type Page struct {
Slug string `yaml:"slug"`
File string `yaml:"file"`
}
func (page *Page) ReadFile(file string) (err error) {
fileContents, err := ioutil.ReadFile(file)
if err != nil {
return
}
err = yaml.Unmarshal(fileContents, &page)
return
}
func isYamlFile(fileInfo os.FileInfo) bool {
return !fileInfo.IsDir() && strings.HasSuffix(fileInfo.Name(), ".yaml")
}
func ReadDir(dir string, pages []*Page) (err error) {
filesInfo, err := ioutil.ReadDir(dir)
if err != nil {
return
}
for i, fileInfo := range filesInfo {
if isYamlFile(fileInfo) {
pages[i].ReadFile(fileInfo.Name())
}
}
return
}
There is a compiler issue here:
src/main.go:19: cannot use pages (type []*Page) as type []*yamlcms.Page in argument to yamlcms.ReadDir
My main intent in this question is to learn the idiomatic way of doing this kind of thing in Go. Other 3rd-party solutions may exist but I am not immediately interested in them because I have problems like this frequently in Go having to do with inheritance, etc. So along the lines of what I've presented, how can I best (idiomatically) accomplish what I am going for?
EDIT:
So I've made some changes as suggested. Now I have this:
type FileReader interface {
ReadFile(file string) error
}
func ReadDir(dir string, pages []*FileReader) (err error) {
filesInfo, err := ioutil.ReadDir(dir)
if err != nil {
return
}
for i, fileInfo := range filesInfo {
if isYamlFile(fileInfo) {
(*pages[i]).ReadFile(fileInfo.Name())
}
}
return
}
However, I still get a similar compiler error:
src/main.go:19: cannot use pages (type []*Page) as type []*yamlcms.FileReader in argument to yamlcms.ReadDir
Even though main.Page should be a FileReader because it embeds yamlcms.Page.
EDIT: I forgot that slices of interfaces don't work like that. You'd need to allocate a new slice, convert all pages to FileReaders, call the function, and convert them back.
Another possible solution is refactoring yamlcms.ReadDir to return the contents of the files, so that they could be unmarshaled later:
// In yamlcms.
func ReadYAMLFilesInDir(dir string) ([][]byte, error) { ... }
// In client code.
files := yamlcms.ReadYAMLFilesInDir("dir")
for i := range pages {
if err := yaml.Unmarshal(files[i], &pages[i]); err != nil { return err }
}
The original answer:
There are no such things as inheritance or casting in Go. Prefer composition and interfaces in your designs. In your case, you can redefine your yamlcms.ReadDir to accept an interface, FileReader.
type FileReader interface {
ReadFile(file string) error
}
Both yamlcms.Page and main.Page will implement this, as the latter embeds the former.