No such file or directory error in golang - go

I want to specify an html template in one of my golang controller
My directory structure is like this
Project
-com
-src
- controller
-contoller.go
-view
- html
-first.html
I want to load first.html for request /new .I have used NewHandler for url /new and the NewHandler func is executing when /new request comes and is in controller.go. Here is my code
func NewHandler(w http.ResponseWriter, r *http.Request) {
t, err := template.ParseFiles("view/html/first.html")
if err == nil {
log.Println("Template parsed successfully....")
}
err := templates.ExecuteTemplate(w, "view/html/first.html", nil)
if err != nil {
log.Println("Not Found template")
}
// t.Execute(w, "")
}
But I am getting an error
panic: open first.html: no such file or directory
Please help me to remove this error. Thanks in advance

I have solved the issue by giving absolute path of the html. For that I created a class in which the html are parsed
package htmltemplates
import (
"html/template"
"path/filepath"
)
And in the NewHandler method I removed
//Templates is used to store all Templates
var Templates *template.Template
func init() {
filePrefix, _ := filepath.Abs("./work/src/Project/view/html/") // path from the working directory
Templates = template.Must(template.ParseFiles(filePrefix + "/first.html"))
...
//htmls must be specified here to parse it
}
And in the NewHandler I removed first 5 lines and instead gave
err := htmltemplates.Templates.ExecuteTemplate(w, "first.html", nil)
It is now working .But need a better solution if any

Have you included this line in the main function?
http.Handle("/view/", http.StripPrefix("/view/", http.FileServer(http.Dir("view"))))
view is the name of the directory that has to be specified in FileServer function to allow read/write.(view directory has to be kept in the same directory where your binary is present)

Better solution is to give absolute path name i.e instead of this "view/html/first.html",try this
func NewHandler(w http.ResponseWriter, r *http.Request) {
t, err := template.ParseFiles("./work/src/Project/view/html/first.html")
if err == nil {
log.Println("Template parsed successfully....")
}
err := templates.ExecuteTemplate(w, "view/html/first.html", nil)
if err != nil {
log.Println("Not Found template")
}
// t.Execute(w, "")
}

Solution, where I just traversed outside the directory without using libraries to find the path or convert them.
Using "os" package get the present working directory. Like,
import(
"os"
)
your_function(){
pwd,_ := os.pwd()}
Above snippet will return you the path where your program is trying to find the file.
Like
D:\Golang_Projects\POC\Distributed_Application_Go\cmd\teacherportal
After you're sure what path your program is looking at, use ".." to come outside of that directory. Like,
rootTemplate,err := template.ParseFiles("../../teacherportal/students.gohtml")
Alternative solutions are also there but I found this quite simple to implement.

Related

How to detect Hidden Files in a folder in Go - cross-platform approach

I'm iterating through a mounted folder via filePath.Walk method in golang, but it returns the hidden files as well. I have to skip those hidden files.
For MaxOS and Linux, we can detect hidden file via .prefix in the filename, but for windows, when I'm trying to us this method GetFileAttributes, provided by "syscall", it's not detecting these methods and throwing an error.
Using below method to fetch the file
err := filepath.Walk(prefix, func(docPath string, f os.FileInfo, err error) error {
Below is how I'm trying to detect the hidden files
import (
"runtime"
"syscall"
)
func IsHiddenFile(filename string) (bool, error) {
if runtime.GOOS == "windows" {
pointer, err := syscall.UTF16PtrFromString(filename)
if err != nil {
return false, err
}
attributes, err := syscall.GetFileAttributes(pointer)
if err != nil {
return false, err
}
return attributes&syscall.FILE_ATTRIBUTE_HIDDEN != 0, nil
} else {
// unix/linux file or directory that starts with . is hidden
if filename[0:1] == "." {
return true, nil
}
}
return false, nil
}
Error :
.../ undefined: syscall.UTF16PtrFromString
.../ undefined: syscall.GetFileAttributes
.../ undefined: syscall.FILE_ATTRIBUTE_HIDDEN
I added this // +build windows in the start of the file before package name as suggested here : syscall variables undefined but it's sill not working and throwing the same error.
I need to know if go provides some common method to detect if a file is hidden or not? Or is there a way I can fetch all the files/folder in some mounted directory without receiving hidden files in the first place?
Really looking forward on receiving some feedback here, thanks.
EDIT : Fixed the above mentioned issue (please refer to the comments below), I also wants to know how can we detect the hidden file when we are connected with the remote server (SMB), the remoter system could be running any OS, and we compile these method based on the system it's running on. How can we detect Hidden files in that scenario ?
Conditional compilation is the right way to go, but it applies at source file level, so you need two separate files.
For example:
hidden_notwin.go:
//go:build !windows
package main
func IsHiddenFile(filename string) (bool, error) {
return filename[0] == '.', nil
}
hidden_windows.go:
//go:build windows
package main
import (
"syscall"
)
func IsHiddenFile(filename string) (bool, error) {
pointer, err := syscall.UTF16PtrFromString(filename)
if err != nil {
return false, err
}
attributes, err := syscall.GetFileAttributes(pointer)
if err != nil {
return false, err
}
return attributes&syscall.FILE_ATTRIBUTE_HIDDEN != 0, nil
}
Note that //go:build windows tag above is optional - the _windows source file suffix does the magic already. For more details see go command - Build constraints.

Unable to load HTML templates with Gin

I've been having some issues loading html templates using the Gin framework through the r.HTMLRender setting.
It would seem that the templates are not being found.
I have tried to use the following helpers:
GinHTMLRender: https://gist.github.com/madhums/4340cbeb36871e227905
EZ Gin Template: https://github.com/michelloworld/ez-gin-template
Neither of these seem to be working when setting the default path for templates (in my case app/views); for the purposes of getting to this to work my html template file structure looks like this:
/workspace
|-app
|-views
|-layouts
|-application.html
|-test.html
Here is a sample of the Gin loading code:
import (
"github.com/gin-contrib/location"
"github.com/gin-gonic/gin"
"fmt"
"os"
"github.com/gin-gonic/contrib/static"
"github.com/michelloworld/ez-gin-template"
)
//CORSMiddleware ...
func CORSMiddleware() gin.HandlerFunc {
/** CORS middleware **/
}
func Router() {
if os.Getenv("ENVIRONMENT") == "production" {
gin.SetMode(gin.ReleaseMode)
}
// Initialize Gin object
r := gin.Default()
// Cors Middleware
r.Use(CORSMiddleware())
// Rate limiting
rl, err := helpers.RateLimiterMiddleware()
if err != nil {
panic("Rate Limiting Initialization error")
}
r.Use(rl)
// Asset provision
r.Use(static.ServeRoot("/public","app/assets"))
// Get URL information
r.Use(location.Default())
// Attempt with EZ Template, fails
// I ge this error: "runtime error: invalid memory address or nil pointer dereference" when calling c.HTML(...)
render := eztemplate.New()
render.TemplatesDir = "app/views/" // default
render.Layout = "layouts/application" // default
render.Ext = ".html" // default
render.Debug = true // default
r.HTMLRender = render.Init()
// Attempt with GinHTMLRender, fails
// I get this error: https://gist.github.com/madhums/4340cbeb36871e227905#file-gin_html_render-go-L110
/*
htmlRender := GinHTMLRender.New()
htmlRender.TemplatesDir = "app/views/"
htmlRender.Debug = gin.IsDebugging()
htmlRender.Layout = "layouts/application"
log.Println("Dir:"+htmlRender.TemplatesDir)
r.HTMLRender = htmlRender.Create()*/
/** Some Routes **/
// Start web listener
r.Run(":8009") // listen and serve on 0.0.0.0:8080
}
The corresponding render call is code is the following:
/* c is of type *gin.Context */
c.HTML(200, "test", "")
For some reason it seems like the r.HTMLRender function is not taking into account the template path; I have attempted doing this:
_, err := template.ParseFiles("app/views/test.html")
if err != nil {
log.Println("Template Error")
} else {
log.Println("No Template Error")
}
This code consistently displays "No Template Error", which leads me to believe that the HTMLRender assignment is not considering the TemplatesDir set variable.
I've been stuck with this issue for some time, and I am not entirely sure how to get it resolved.
Any help getting this to work would be greatly appreciated.
After doing some further research I found the source of my problem with EZ Gin Template.
I'm hoping this helps anyone experiencing the same issue.
After taking a deeper look at the helper code, I realized that the template matching pattern is strict and does not search for files recursively; ie. it expects a specific file structure to find template files:
In the default setting, EZ Gin Template requires the following file structure to work:
/workspace
- app
- views
- layouts
- some_layout.html
- some_dir
- template_file.html
- _partial_template.html
- partials
- _some_other_partial.html
In order to allow for other file patterns, the helper set of functions needs to be modified.
In my case, I forked the helper code locally to allow matching 1st level template files:
func (r Render) Init() Render {
globalPartials := r.getGlobalPartials()
layout := r.TemplatesDir + r.Layout + r.Ext
// Match multiple levels of templates
viewDirs, _ := filepath.Glob(r.TemplatesDir + "**" + string(os.PathSeparator) + "*" + r.Ext)
// Added the following two lines to match for app/views/some_file.html as well as files on the **/*.html matching pattern
tmp, _ := filepath.Glob(r.TemplatesDir + "*" + r.Ext)
viewDirs = append(viewDirs, tmp...)
// Can be extended by replicating those two lines above and adding search paths within the base template path.
fullPartialDir := filepath.Join(r.TemplatesDir + r.PartialDir)
for _, view := range viewDirs {
templateFileName := filepath.Base(view)
//skip partials
if strings.Index(templateFileName, "_") != 0 && strings.Index(view, fullPartialDir) != 0 {
localPartials := r.findPartials(filepath.Dir(view))
renderName := r.getRenderName(view)
if r.Debug {
log.Printf("[GIN-debug] %-6s %-25s --> %s\n", "LOAD", view, renderName)
}
allFiles := []string{layout, view}
allFiles = append(allFiles, globalPartials...)
allFiles = append(allFiles, localPartials...)
r.AddFromFiles(renderName, allFiles...)
}
}
return r
}
I have not tried a similar solution with GinHTMLRenderer, but I expect that the issue might likely be related to it in terms of the expected file structure.
You can also bind the templates into the code. The jessevdk/go-assets-builder will generate a go file that contains assets within the specified directory. Make sure that the file generated is located where the main package is. Gin also provided this as an example in their Documentation For more Info. It will also include subfolders and its files (i. e. assets) in the binary.
Get The generator tool:
go get github.com/jessevdk/go-assets-builder
Generate:
# go-assets-builder <dir> -o <generated file name>
go-assets-builder app -o assets.go
Note that <generated file name> can also be like cmd/client/assets.go to specify the destination of the file to be generated.
Load Template:
package main
// ... imports
func main() {
r := gin.New()
t, err := loadTemplate()
if err != nil {
panic(err)
}
r.SetHTMLTemplate(t)
r.GET("/", func(c *gin.Context) {
c.HTML(http.StatusOK, "app/views/layouts/application.html", nil)
})
r.GET("/test", func(c *gin.Context) {
c.HTML(http.StatusOK, "app/views/test.html", nil)
})
r.Run(":8080")
}
// loadTemplate loads templates embedded by go-assets-builder
func loadTemplate() (*template.Template, error) {
t := template.New("")
// Assets is the templates
for name, file := range Assets.Files {
if file.IsDir() || !strings.HasSuffix(name, ".html") {
continue
}
h, err := ioutil.ReadAll(file)
if err != nil {
return nil, err
}
t, err = t.New(name).Parse(string(h))
if err != nil {
return nil, err
}
}
return t, nil
}
Here is how I do it. This walks through the directory and collects the files marked with my template suffix which is .html & then I just include all of those. I haven't seen this answer anywhere so I thought Id post it.
// START UP THE ROUTER
router := gin.Default()
var files []string
filepath.Walk("./views", func(path string, info os.FileInfo, err error) error {
if strings.HasSuffix(path, ".html") {
files = append(files, path)
}
return nil
})
router.LoadHTMLFiles(files...)
// SERVE STATICS
router.Use(static.Serve("/css", static.LocalFile("./css", true)))
router.Use(static.Serve("/js", static.LocalFile("./js", true)))
router.Use(static.Serve("/images", static.LocalFile("./images", true)))
routers.LoadBaseRoutes(router)
routers.LoadBlog(router)
router.Run(":8080")
Now they don't have to all be nested at the exact depth like the other suggestions ... the file structure can be uneven

prevent access to files in folder with a golang server

I've a server in golang who handle folder path like that :
fs := http.FileServer(http.Dir("./assets"))
http.Handle("/Images/", fs)
http.ListenAndServe(":8000", nil)
But in this folder there are privates images, and it shouldn't be possible to access files. So how can i secure image access and prevent anybody to access content of folder.
like that for example :
If you want to block a directory using http package, maybe this will be useful to you :
https://groups.google.com/forum/#!topic/golang-nuts/bStLPdIVM6w
package main
import (
"net/http"
"os"
)
type justFilesFilesystem struct {
fs http.FileSystem
}
func (fs justFilesFilesystem) Open(name string) (http.File, error) {
f, err := fs.fs.Open(name)
if err != nil {
return nil, err
}
return neuteredReaddirFile{f}, nil
}
type neuteredReaddirFile struct {
http.File
}
func (f neuteredReaddirFile) Readdir(count int) ([]os.FileInfo, error) {
return nil, nil
}
func main() {
fs := justFilesFilesystem{http.Dir("/tmp/")}
http.ListenAndServe(":8080", http.FileServer(fs))
}
A little wrapper over FileServer() solves your problem, now you have to add some sort of logic to do Authorization, it looks like you have unique names, that's good, so I just filter the image name for you creating a map of names, now you can add something more dynamic like a key/store(memcached, redis. etc.) Hope you can follow the comments
package main
import (
"log"
"net/http"
"strings"
)
// put the allowed hashs or keys here
// you may consider put them in a key/value store
//
var allowedImages = map[string]bool{
"key-abc.jpg": true,
"key-123.jpg": true,
}
func main() {
http.Handle("/Images/", http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// here we can do any kind of checking, in this case we'll just split the url and
// check if the image name is in the allowedImages map, we can check in a DB or something
//
parts := strings.Split(r.URL.Path, "/")
imgName := parts[len(parts)-1]
if _, contains := allowedImages[imgName]; !contains { // if the map contains the image name
log.Printf("Not found image: %q path: %s\n", imgName, r.URL.Path)
// if the image is not found we write a 404
//
// Bonus: we don't list the directory, so nobody can know what's inside :)
//
http.NotFound(w, r)
return
}
log.Printf("Serving allowed image: %q\n", imgName)
fileServer := http.StripPrefix("/Images/", http.FileServer(http.Dir("./assets")))
fileServer.ServeHTTP(w, r) // StripPrefix() and FileServer() return a Handler that implements ServerHTTP()
}))
http.ListenAndServe(":8000", nil)
}
https://play.golang.org/p/ehrd_AWXim

Filepath.Walk in Go not picking up folders under C: drive properly in Windows

I am using the Filepath.Walk in Go to try to get all the folders under C: recursively. However it just returns me sub-folders starting with $Recycle.Bin. What am I doing wrong here?
package main
import (
"fmt"
"path/filepath"
"os"
)
func main() {
array := Subfolders("C:")
for _,value := range array {
fmt.Println(value)
}
}
func Subfolders(path string) (paths []string) {
filepath.Walk(path, func(newPath string, info os.FileInfo, err error) error {
if err != nil {
return err
}
if info.IsDir() {
paths = append(paths, newPath)
}
return nil
})
return paths
}
Output:
C:
C:\$Recycle.Bin
C:\$Recycle.Bin\S-1-5-21-310629982-3373693989-3733510080-1000
C:\$Recycle.Bin\S-1-5-21-310629982-3373693989-3733510080-1000\$R0L9M20
C:\$Recycle.Bin\S-1-5-21-310629982-3373693989-3733510080-1000\$R22ZOD9
C:\$Recycle.Bin\S-1-5-21-310629982-3373693989-3733510080-1000\$R3LS9P4
C:\$Recycle.Bin\S-1-5-21-310629982-3373693989-3733510080-1000\$R4T2IGU
C:\$Recycle.Bin\S-1-5-21-310629982-3373693989-3733510080-1000\$R8TZIET
C:\$Recycle.Bin\S-1-5-21-310629982-3373693989-3733510080-1000\$R9QQZB9
C:\$Recycle.Bin\S-1-5-21-310629982-3373693989-3733510080-1000\$RA71HY3
C:\$Recycle.Bin\S-1-5-21-310629982-3373693989-3733510080-1000\$RBOC0V4.com
C:\$Recycle.Bin\S-1-5-21-310629982-3373693989-3733510080-1000\$RBOC0V4.com\go-fsnotify
C:\$Recycle.Bin\S-1-5-21-310629982-3373693989-3733510080-1000\$REJFS3Z
C:\$Recycle.Bin\S-1-5-21-310629982-3373693989-3733510080-1000\$RF9A1Y6
C:\$Recycle.Bin\S-1-5-21-310629982-3373693989-3733510080-1000\$ROMESWQ
C:\$Recycle.Bin\S-1-5-21-310629982-3373693989-3733510080-1000\$RP4CYID
C:\$Recycle.Bin\S-1-5-21-310629982-3373693989-3733510080-1000\$RQHMPV5
C:\$Recycle.Bin\S-1-5-21-310629982-3373693989-3733510080-1000\$RV0K99H
C:\$Recycle.Bin\S-1-5-21-310629982-3373693989-3733510080-1000\$RX54T04
I have many other folders under C: which donot get picked up by the filepath.walk. I wanted to understand what was the reason behind it.
EDIT
Thanks to the answers I was able to resolve the issue as below:-
func Subfolders(path string) (paths []string) {
filepath.Walk(path, func(newPath string, info os.FileInfo, err error) error {
if err != nil {
log.Println(err)
return filepath.SkipDir
}
if info.IsDir() {
paths = append(paths, newPath)
}
return nil
})
You're blindly returning errors without logging them. Returning a non-nil error from the callback is a signal for filepath.Walk to abort.
Presumably there is some file you don't have access to, or something.
You're returning an error, so it stops walking the paths. As the spec states, whenever an error is returned (besides one special case), processing stops. If you'd like to continue processing and ignore the error, just return nil instead.
Running it on my machine gives me an Access is denied error on one file, which then nukes the walk. The reason is only sees $Recycle.Bin is because it happens to be the first folder in the directory, and the walk is killed before it can get to any others.

Ensure a URI is valid

I'm trying to ensure that URLs passed to my go program are valid. However, I can't seem to work out how to. I thought I could just feed it through url.Parse, but that doesn't seem to do the job.
package main
import (
"fmt"
"net/url"
)
func main() {
url, err := url.Parse("http:::/not.valid/a//a??a?b=&&c#hi")
if err != nil {
panic(err)
}
fmt.Println("It's valid!", url.String())
}
playground
Is there anything along the lines of filter_var I can use?
You can check that your URL has a Scheme, Host, and/or a Path.
If you inspect the URL returned, you can see that the invalid part is inserted into the Opaque data section (so in a sense, it is valid).
url.URL{Scheme:"http", Opaque:"::/not.valid/a//a", Host:"", Path:"", RawQuery:"?a?b=&&c", Fragment:"hi"}
If you parse a URL and don't have a Scheme, Host and Path you can probably assume it's not valid. (though a host without a path is often OK, since it implies /, so you need to check for that)
u, err := url.Parse("http:::/not.valid/a//a??a?b=&&c#hi")
if err != nil {
log.Fatal(err)
}
if u.Scheme == "" || u.Host == "" || u.Path == "" {
log.Fatal("invalid URL")
}
have a try of this package go validator and the IsURL func is what you want.(you can use regexp package as well)
package main
import (
"fmt"
"regexp"
)
const URL string = `^((ftp|http|https):\/\/)?(\S+(:\S*)?#)?((([1-9]\d?|1\d\d|2[01]\d|22[0-3])(\.(1?\d{1,2}|2[0-4]\d|25[0-5])){2}(?:\.([0-9]\d?|1\d\d|2[0-4]\d|25[0-4]))|(((([a-z\x{00a1}-\x{ffff}0-9]+-?-?_?)*[a-z\x{00a1}-\x{ffff}0-9]+)\.)?)?(([a-z\x{00a1}-\x{ffff}0-9]+-?-?_?)*[a-z\x{00a1}-\x{ffff}0-9]+)(?:\.([a-z\x{00a1}-\x{ffff}]{2,}))?)|localhost)(:(\d{1,5}))?((\/|\?|#)[^\s]*)?$`
func Matches(str, pattern string) bool {
match, _ := regexp.MatchString(pattern, str)
return match
}
func main() {
u1 := "http:::/not.valid/a//a??a?b=&&c#hi"
u2 := "http://golang.fastrl.com"
func(us ...string) {
for _, u := range us {
fmt.Println(Matches(u, URL))
}
}(u1, u2)
}
The url.Parse() function will return an error mainly if viaRequest is true, meaning if the URL is assumed to have arrived via an HTTP request.
Which is not the case when you call url.Parse() directly: see the source code.
if url, err = parse(u, false); err != nil {
return nil, err
}
And func parse(rawurl string, viaRequest bool) (url *URL, err error) { only returns err if viaRequest is true.
That is why you never see any error when using url.Parse() directly.
In that latter case, where no err is ever returned, you can check the fields of the URL object returned.
An empty url.Scheme, or an url.Path which isn't the one expected would indicate an error.

Resources