prevent access to files in folder with a golang server - go

I've a server in golang who handle folder path like that :
fs := http.FileServer(http.Dir("./assets"))
http.Handle("/Images/", fs)
http.ListenAndServe(":8000", nil)
But in this folder there are privates images, and it shouldn't be possible to access files. So how can i secure image access and prevent anybody to access content of folder.
like that for example :

If you want to block a directory using http package, maybe this will be useful to you :
https://groups.google.com/forum/#!topic/golang-nuts/bStLPdIVM6w
package main
import (
"net/http"
"os"
)
type justFilesFilesystem struct {
fs http.FileSystem
}
func (fs justFilesFilesystem) Open(name string) (http.File, error) {
f, err := fs.fs.Open(name)
if err != nil {
return nil, err
}
return neuteredReaddirFile{f}, nil
}
type neuteredReaddirFile struct {
http.File
}
func (f neuteredReaddirFile) Readdir(count int) ([]os.FileInfo, error) {
return nil, nil
}
func main() {
fs := justFilesFilesystem{http.Dir("/tmp/")}
http.ListenAndServe(":8080", http.FileServer(fs))
}

A little wrapper over FileServer() solves your problem, now you have to add some sort of logic to do Authorization, it looks like you have unique names, that's good, so I just filter the image name for you creating a map of names, now you can add something more dynamic like a key/store(memcached, redis. etc.) Hope you can follow the comments
package main
import (
"log"
"net/http"
"strings"
)
// put the allowed hashs or keys here
// you may consider put them in a key/value store
//
var allowedImages = map[string]bool{
"key-abc.jpg": true,
"key-123.jpg": true,
}
func main() {
http.Handle("/Images/", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// here we can do any kind of checking, in this case we'll just split the url and
// check if the image name is in the allowedImages map, we can check in a DB or something
//
parts := strings.Split(r.URL.Path, "/")
imgName := parts[len(parts)-1]
if _, contains := allowedImages[imgName]; !contains { // if the map contains the image name
log.Printf("Not found image: %q path: %s\n", imgName, r.URL.Path)
// if the image is not found we write a 404
//
// Bonus: we don't list the directory, so nobody can know what's inside :)
//
http.NotFound(w, r)
return
}
log.Printf("Serving allowed image: %q\n", imgName)
fileServer := http.StripPrefix("/Images/", http.FileServer(http.Dir("./assets")))
fileServer.ServeHTTP(w, r) // StripPrefix() and FileServer() return a Handler that implements ServerHTTP()
}))
http.ListenAndServe(":8000", nil)
}
https://play.golang.org/p/ehrd_AWXim

Related

Is it possible to present an embedded file as a filename?

I use a function that requires a filename as a parameter (of type string). It works fine when providing the filename.
I would like to embed this file in my binary. I can then have the contents as []byte or string but that's not useful. I can also fave it as embed.FS but my understanding is that this is an abstraction that can be used by some functions only.
What I would need is the ability to present this embedded file as a filename (a string) that would then be used by the underlying function to open the (embedded) file.
Is this possible?
Filename that Key accept as string argument is only abstraction on ioutil.ReadFile, see auth.go
What you can do is implement ssh.Auth yourself, here is small example.
package main
import (
_ "embed"
"fmt"
"github.com/melbahja/goph"
"golang.org/x/crypto/ssh"
)
//go:embed id_rsa
var privateKey []byte
func main() {
auth, err := Auth(privateKey, []byte("foobar"))
fmt.Println(auth, err)
}
func Auth(privateKey, pass []byte) (goph.Auth, error) {
signer, err := Singer(privateKey, pass)
if err != nil {
return nil, err
}
return goph.Auth{
ssh.PublicKeys(signer),
}, nil
}
func Singer(privateKey, pass []byte) (ssh.Signer, error) {
if len(pass) != 0 {
return ssh.ParsePrivateKeyWithPassphrase(privateKey, pass)
}
return ssh.ParsePrivateKey(privateKey)
}

Runtime access to symbols in Go

In writing a Web server in Go, I'd like to be able to dereference symbols at runtime, to allow me to figure out which functions to call from a configuration file, something like the call to the fictional "eval" function in the example below. That would allow me to select handlers from a library of handlers, and to deploy a new server with just a config file. Is there any way to accomplish this in Go?
config.json
{ "url": "/api/apple", "handler": "Apple", "method": "get" }
{ "url": "/api/banana", "handler": "Banana", "method": "get" }
play.go
package main
import (
"github.com/gorilla/mux"
"net/http"
"encoding/json"
"log"
)
type ConfigEntry struct {
URL string `json:"url"`
Method string `json:"method"`
Handler string `json:"handler"`
}
func main() {
ifp, err := os.Open("config.json")
if err != nil {
log.Fatal(err)
}
dec := json.NewDecoder(ifp)
r := mux.NewRouter()
for {
var config ConfigEntry
if err = dec.Decode(&m); err == io.EOF {
break
} else if err != nil {
log.Fatal(err)
}
r.HandleFunc(config.URL, eval(config.Handler + "Handler")).Methods(config.Method)
}
http.Handle("/", r)
http.ListenAndServe(8080, nil)
}
func AppleHandler(w http.ResponseWriter, r *http.Request) (status int, err error) {
w.Write("Apples!\n")
return http.StatusOK, nil
}
func BananaHandler(w http.ResponseWriter, r *http.Request) (status int, err error) {
w.Write("Bananas!\n")
return http.StatusOK, nil
}
There's nothing like eval in Go, which is a good thing since things like that are very dangerous.
What you can do is have a map mapping the handler strings in your config file to the handler functions in your code:
var handlers = map[string]func(http.ResponseWriter, *http.Request) (int, error){
"Apple": AppleHandler,
"Banana": BananaHandler,
}
Then you can register those handlers by simply doing:
handler, ok := handlers[config.Handler]
if !ok {
log.Fatal(fmt.Errorf("Handler not found"))
}
r.HandleFunc(config.URL, handler).Methods(config.Method)
There's some limited way to access things during runtime with the reflect package. However it doesn't allow you to search for all suitable standalone functions in a package. It would be possible if they are all methods on a known struct type/value.
As an alternative your given example you could simply use a map[string]func(...) to store all handlers, initialize it at startup (during init()) and fetch the handlers from there. But that also more or less what the existing http muxes are doing.

Golang downcasting list of structs

I want to be able to unmarshal yaml files less rigidly. That is, my library has a predefined number of options the yaml file must have. Then, the user should be able to extend this to include any custom options.
Here is what I have
package main
import (
"net/http"
"yamlcms"
"github.com/julienschmidt/httprouter"
)
type Page struct {
*yamlcms.Page
Title string
Date string
}
func getBlogRoutes() {
pages := []*Page{}
yamlcms.ReadDir("html", pages)
}
// This section is a work in progress, I only include it for loose context
func main() {
router := httprouter.New()
//blogRoutes := getBlogRoutes()
//for _, blogRoute := range *blogRoutes {
// router.Handle(blogRoute.Method, blogRoute.Pattern,
// func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {})
//}
http.ListenAndServe(":8080", router)
}
Here is the yamlcms package:
package yamlcms
import (
"io/ioutil"
"os"
"strings"
"gopkg.in/yaml.v2"
)
type Page struct {
Slug string `yaml:"slug"`
File string `yaml:"file"`
}
func (page *Page) ReadFile(file string) (err error) {
fileContents, err := ioutil.ReadFile(file)
if err != nil {
return
}
err = yaml.Unmarshal(fileContents, &page)
return
}
func isYamlFile(fileInfo os.FileInfo) bool {
return !fileInfo.IsDir() && strings.HasSuffix(fileInfo.Name(), ".yaml")
}
func ReadDir(dir string, pages []*Page) (err error) {
filesInfo, err := ioutil.ReadDir(dir)
if err != nil {
return
}
for i, fileInfo := range filesInfo {
if isYamlFile(fileInfo) {
pages[i].ReadFile(fileInfo.Name())
}
}
return
}
There is a compiler issue here:
src/main.go:19: cannot use pages (type []*Page) as type []*yamlcms.Page in argument to yamlcms.ReadDir
My main intent in this question is to learn the idiomatic way of doing this kind of thing in Go. Other 3rd-party solutions may exist but I am not immediately interested in them because I have problems like this frequently in Go having to do with inheritance, etc. So along the lines of what I've presented, how can I best (idiomatically) accomplish what I am going for?
EDIT:
So I've made some changes as suggested. Now I have this:
type FileReader interface {
ReadFile(file string) error
}
func ReadDir(dir string, pages []*FileReader) (err error) {
filesInfo, err := ioutil.ReadDir(dir)
if err != nil {
return
}
for i, fileInfo := range filesInfo {
if isYamlFile(fileInfo) {
(*pages[i]).ReadFile(fileInfo.Name())
}
}
return
}
However, I still get a similar compiler error:
src/main.go:19: cannot use pages (type []*Page) as type []*yamlcms.FileReader in argument to yamlcms.ReadDir
Even though main.Page should be a FileReader because it embeds yamlcms.Page.
EDIT: I forgot that slices of interfaces don't work like that. You'd need to allocate a new slice, convert all pages to FileReaders, call the function, and convert them back.
Another possible solution is refactoring yamlcms.ReadDir to return the contents of the files, so that they could be unmarshaled later:
// In yamlcms.
func ReadYAMLFilesInDir(dir string) ([][]byte, error) { ... }
// In client code.
files := yamlcms.ReadYAMLFilesInDir("dir")
for i := range pages {
if err := yaml.Unmarshal(files[i], &pages[i]); err != nil { return err }
}
The original answer:
There are no such things as inheritance or casting in Go. Prefer composition and interfaces in your designs. In your case, you can redefine your yamlcms.ReadDir to accept an interface, FileReader.
type FileReader interface {
ReadFile(file string) error
}
Both yamlcms.Page and main.Page will implement this, as the latter embeds the former.

Go url parameters mapping

Is there a native way for inplace url parameters in native Go?
For Example, if I have a URL: http://localhost:8080/blob/123/test I want to use this URL as /blob/{id}/test.
This is not a question about finding go libraries. I am starting with the basic question, does go itself provide a basic facility to do this natively.
There is no built in simple way to do this, however, it is not hard to do.
This is how I do it, without adding a particular library. It is placed in a function so that you can invoke a simple getCode() function within your request handler.
Basically you just split the r.URL.Path into parts, and then analyse the parts.
// Extract a code from a URL. Return the default code if code
// is missing or code is not a valid number.
func getCode(r *http.Request, defaultCode int) (int, string) {
p := strings.Split(r.URL.Path, "/")
if len(p) == 1 {
return defaultCode, p[0]
} else if len(p) > 1 {
code, err := strconv.Atoi(p[0])
if err == nil {
return code, p[1]
} else {
return defaultCode, p[1]
}
} else {
return defaultCode, ""
}
}
Well, without external libraries you can't, but may I recommend two excellent ones:
httprouter - https://github.com/julienschmidt/httprouter - is extremely fast and very lightweight. It's faster than the standard library's router, and it creates 0 allocations per call, which is great in a GCed language.
Gorilla Mux - http://www.gorillatoolkit.org/pkg/mux -
Very popular, nice interface, nice community.
Example usage of httprouter:
func Hello(w http.ResponseWriter, r *http.Request, ps httprouter.Params) {
fmt.Fprintf(w, "hello, %s!\n", ps.ByName("name"))
}
func main() {
router := httprouter.New()
router.GET("/hello/:name", Hello)
log.Fatal(http.ListenAndServe(":8080", router))
}
What about trying using regex, and find a named group in your url, like playground:
package main
import (
"fmt"
"net/url"
"regexp"
)
var myExp = regexp.MustCompile(`/blob/(?P<id>\d+)/test`) // use (?P<id>[a-zA-Z]+) if the id is alphapatic
func main() {
s := "http://localhost:8080/blob/123/test"
u, err := url.Parse(s)
if err != nil {
panic(err)
}
fmt.Println(u.Path)
match := myExp.FindStringSubmatch(s) // or match := myExp.FindStringSubmatch(u.Path)
result := make(map[string]string)
for i, name := range myExp.SubexpNames() {
if i != 0 && name != "" {
result[name] = match[i]
}
}
fmt.Printf("id: %s\n", result["id"])
}
output
/blob/123/test
id: 123
Below full code to use it with url, that is receiving http://localhost:8000/hello/John/58 and returning http://localhost:8000/hello/John/58:
package main
import (
"fmt"
"net/http"
"regexp"
"strconv"
)
var helloExp = regexp.MustCompile(`/hello/(?P<name>[a-zA-Z]+)/(?P<age>\d+)`)
func hello(w http.ResponseWriter, req *http.Request) {
match := helloExp.FindStringSubmatch(req.URL.Path)
if len(match) > 0 {
result := make(map[string]string)
for i, name := range helloExp.SubexpNames() {
if i != 0 && name != "" {
result[name] = match[i]
}
}
if _, err := strconv.Atoi(result["age"]); err == nil {
fmt.Fprintf(w, "Hello, %v year old named %s!", result["age"], result["name"])
} else {
fmt.Fprintf(w, "Sorry, not accepted age!")
}
} else {
fmt.Fprintf(w, "Wrong url\n")
}
}
func main() {
http.HandleFunc("/hello/", hello)
http.ListenAndServe(":8090", nil)
}
How about writing your own url generator (extend net/url a little bit) as below.
// --- This is how does it work like --- //
url, _ := rest.NewURLGen("http", "stack.over.flow", "1234").
Pattern(foo/:foo_id/bar/:bar_id).
ParamQuery("foo_id", "abc").
ParamQuery("bar_id", "xyz").
ParamQuery("page", "1").
ParamQuery("offset", "5").
Do()
log.Printf("url: %s", url)
// url: http://stack.over.flow:1234/foo/abc/bar/xyz?page=1&offset=5
// --- Your own url generator would be like below --- //
package rest
import (
"log"
"net/url"
"strings"
"straas.io/base/errors"
"github.com/jinzhu/copier"
)
// URLGen generates request URL
type URLGen struct {
url.URL
pattern string
paramPath map[string]string
paramQuery map[string]string
}
// NewURLGen new a URLGen
func NewURLGen(scheme, host, port string) *URLGen {
h := host
if port != "" {
h += ":" + port
}
ug := URLGen{}
ug.Scheme = scheme
ug.Host = h
ug.paramPath = make(map[string]string)
ug.paramQuery = make(map[string]string)
return &ug
}
// Clone return copied self
func (u *URLGen) Clone() *URLGen {
cloned := &URLGen{}
cloned.paramPath = make(map[string]string)
cloned.paramQuery = make(map[string]string)
err := copier.Copy(cloned, u)
if err != nil {
log.Panic(err)
}
return cloned
}
// Pattern sets path pattern with placeholder (format `:<holder_name>`)
func (u *URLGen) Pattern(pattern string) *URLGen {
u.pattern = pattern
return u
}
// ParamPath builds path part of URL
func (u *URLGen) ParamPath(key, value string) *URLGen {
u.paramPath[key] = value
return u
}
// ParamQuery builds query part of URL
func (u *URLGen) ParamQuery(key, value string) *URLGen {
u.paramQuery[key] = value
return u
}
// Do returns final URL result.
// The result URL string is possible not escaped correctly.
// This is input for `gorequest`, `gorequest` will handle URL escape.
func (u *URLGen) Do() (string, error) {
err := u.buildPath()
if err != nil {
return "", err
}
u.buildQuery()
return u.String(), nil
}
func (u *URLGen) buildPath() error {
r := []string{}
p := strings.Split(u.pattern, "/")
for i := range p {
part := p[i]
if strings.Contains(part, ":") {
key := strings.TrimPrefix(p[i], ":")
if val, ok := u.paramPath[key]; ok {
r = append(r, val)
} else {
if i != len(p)-1 {
// if placeholder at the end of pattern, it could be not provided
return errors.Errorf("placeholder[%s] not provided", key)
}
}
continue
}
r = append(r, part)
}
u.Path = strings.Join(r, "/")
return nil
}
func (u *URLGen) buildQuery() {
q := u.URL.Query()
for k, v := range u.paramQuery {
q.Set(k, v)
}
u.RawQuery = q.Encode()
}
With net/http the following would trigger when calling localhost:8080/blob/123/test
http.HandleFunc("/blob/", yourHandlerFunction)
Then inside yourHandlerFunction, manually parse r.URL.Path to find 123.
Note that if you don't add a trailing / it won't work. The following would only trigger when calling localhost:8080/blob:
http.HandleFunc("/blob", yourHandlerFunction)
As of 19-Sep-22, with go version 1.19, instance of http.request URL has a method called Query, which will return a map, which is a parsed query string.
func helloHandler(res http.ResponseWriter, req *http.Request) {
// when request URL is `http://localhost:3000/?first=hello&second=world`
fmt.Println(req.URL.Query()) // outputs , map[second:[world] first:[hello]]
res.Write([]byte("Hello World Web"))
}
No way without standard library. Why you don't want to try some library? I think its not so hard to use it, just go get bla bla bla
I use Beego. Its MVC style.
how about a simple utility function ?
func withURLParams(u url.URL, param, val string) url.URL{
u.Path = strings.ReplaceAll(u.Path, param, val)
return u
}
you can use it like this:
u, err := url.Parse("http://localhost:8080/blob/:id/test")
if err != nil {
return nil, err
}
u := withURLParams(u, ":id","123")
// now u.String() is http://localhost:8080/blob/123/test
If you need a framework and you think it will be slow because it's 'bigger' than a router or net/http, then you 're wrong.
Iris is the fastest go web framework that you will ever find, so far according to all benchmarks.
Install by
go get gopkg.in/kataras/iris.v6
Django templates goes easy with iris:
import (
"gopkg.in/kataras/iris.v6"
"gopkg.in/kataras/iris.v6/adaptors/httprouter"
"gopkg.in/kataras/iris.v6/adaptors/view" // <-----
)
func main() {
app := iris.New()
app.Adapt(iris.DevLogger())
app.Adapt(httprouter.New()) // you can choose gorillamux too
app.Adapt(view.Django("./templates", ".html")) // <-----
// RESOURCE: http://127.0.0.1:8080/hi
// METHOD: "GET"
app.Get("/hi", hi)
app.Listen(":8080")
}
func hi(ctx *iris.Context){
ctx.Render("hi.html", iris.Map{"Name": "iris"})
}

runtime error:Invalid memory Address or nil pointer dereference-golang

I am new to golang, am trying develop a login page with sesions. the code is building successfully but when I run in browser its saying 404 page not found.can any one help for me. Thanks in advance.
Here is my code
// main.go
package main
import (
_ "HarishSession/routers"
"github.com/astaxie/beego"
"fmt"
"net/http"
"html/template"
"strings"
"log"
"github.com/astaxie/beego/session"
"sync"
)
var globalSessions *session.Manager
var provides = make(map[string]Provider)
func sayhelloName(w http.ResponseWriter, r *http.Request) {
r.ParseForm() // parse arguments, you have to call this by yourself
fmt.Println("the information of form is",r.Form) // print form information in server side
fmt.Println("path", r.URL.Path)
fmt.Println("scheme", r.URL.Scheme)
fmt.Println(r.Form["url_long"])
for k, v := range r.Form {
fmt.Println("key:", k)
fmt.Println("val:", strings.Join(v, ""))
}
fmt.Fprintf(w, "Hello astaxie!") // send data to client side
}
type Manager struct {
cookieName string //private cookiename
lock sync.Mutex // protects session
provider Provider
maxlifetime int64
}
type Provider interface {
SessionInit(sid string) (Session, error)
SessionRead(sid string) (Session, error)
SessionDestroy(sid string) error
SessionGC(maxLifeTime int64)
}
type Session interface {
Set(key, value interface{}) error //set session value
Get(key interface{}) interface{} //get session value
Delete(key interface{}) error //delete session value
SessionID() string //back current sessionID
}
func NewManager(provideName, cookieName string, maxlifetime int64) (*Manager, error) {
provider, ok := provides[provideName]
if !ok {
return nil, fmt.Errorf("session: unknown provide %q (forgotten import?)", provideName)
}
return &Manager{provider: provider, cookieName: cookieName, maxlifetime: maxlifetime}, nil
}
func login(w http.ResponseWriter, r *http.Request) {
sess := globalSessions.SessionStart(w,r)
r.ParseForm()
fmt.Println("method:", r.Method)
if r.Method == "GET" {
t, _ := template.ParseFiles("login.tpl")
w.Header().Set("Content-Type", "text/html")
t.Execute(w,sess.Get("username"))
} else {
//logic part of log in
fmt.Println("username:",r.Form["username"])
fmt.Println("password:",r.Form["password"])
http.Redirect(w,r,"/",302)
}
}
func main() {
var globalSessions *session.Manager
http.HandleFunc("/", sayhelloName)
http.HandleFunc("/login", login)
err := http.ListenAndServe(":8080", nil) // set listen port
if err != nil {
log.Fatal("ListenAndServe the error is: ", err)
}
fmt.Println("hello")
beego.Run()
fmt.Println(globalSessions)
}
//router.go
package routers
import (
"HarishSession/controllers"
"github.com/astaxie/beego"
)
func init() {
beego.Router("/", &controllers.MainController{})
beego.Router("/login", &controllers.MainController{})
}
//default.go
package controllers
import (
"github.com/astaxie/beego"
)
type MainController struct {
beego.Controller
}
func (this *MainController) Get() {
this.Data["Website"] = "beego.me"
this.Data["Email"] = "astaxie#gmail.com"
this.TplNames = "index.tpl"
this.TplNames="login.tpl"
}
You have two variables at different scopes, each called globalSessions. One is in your definition in main.go, which is defined at global scope, and another is defined in the main function, and is defined as a local variable to main. These are separate variables. Your code is making this mistake of conflating them.
You can see this by paying closer attention to the stack trace entry:
github.com/astaxie/beego/session.(*Manager).SessionStart(0x0, 0x151e78, 0xc08212 0000, 0xc082021ad0, 0x0, 0x0)
as this points to globalSessions being uninitialized due to being nil. After that, troubleshooting is a direct matter of looking at the program to see what touches globalSessions.
Note that you should include the stack trace as part of your question. Don't just add it as a comment. It's critical to include this information: otherwise we would not have been able to easily trace the problem. Please improve the quality of your questions to make it easier for people to help you.
Also, you may want to take a serious look at go vet, which is a tool that helps to catch problems like this.
As this is the one line you used in code :
t, _ := template.ParseFiles("login.tpl")
So what you need to check is whether the file login.tpl is at the correct location, where it must be, or not. If not then correct the reference of it and also check same for the other references.
This helped me.

Resources