Go: embed static files in binary - go

This might be a very amateur question. I'm trying to embed static files into binary, ie. html. How do I do that with https://github.com/jteeuwen/go-bindata?
So I can access an asset with this https://github.com/jteeuwen/go-bindata#accessing-an-asset, but what do I do with "data", and how to do I parse files, execute template, and serve them in the directory?
I couldn't find any examples online, and will appreciate some help!

5/6 years later, this should be easier with Go 1.16 (Q1 2021), which adds support for embedded files (issue/proposal 41191 )
It will be permitted to use //go:embed naming a single file to initialize a plain string or []byte variable:
//go:embed gopher.png
var gopherPNG []byte
The import is required to flag the file as containing //go:embed lines and needing processing.
Goimports (and gopls etc) can be taught this rule and automatically add the import in any file with a //go:embed as needed.
That sparked a debate on issue 42328 about how to avoid surprising inclusion of "hidden" files when using //go:embed
This as resolved in CL 275092 and commit 37588ff
Decision to exclude files matching .* and _* from embedded directory results when embedding an entire directory tree.
See src/embed/internal/embedtest/embed_test.go
//go:embed testdata/k*.txt
var local embed.FS
testFiles(t, local, "testdata/ken.txt", "If a program is too slow, it must have a loop.\n")
//go:embed testdata/k*.txt
var s string
testString(t, s, "local variable s", "If a program is too slow, it must have a loop.\n")
//go:embed testdata/h*.txt
var b []byte
testString(t, string(b), "local variable b", "hello, world\n")
Note: with CL 281492, cmd/go passes embedcfg to gccgo if supported.
See also (Jan. 2021) issue 43854 "opt-in for //go:embed to not ignore files and empty dirs".

Given a directory structure like so:
example/
main.go
data/hi.html
example/main.go
package main
import (
"html/template"
"log"
"net/http"
"os"
)
var tmpl *template.Template
func init() {
data, err := Asset("data/hi.html")
if err != nil {
log.Fatal(err)
}
tmpl = template.Must(template.New("tmpl").Parse(string(data)))
}
func main() {
// print to stdout
tmpl.Execute(os.Stdout, map[string]string{"Name": "James"})
http.HandleFunc("/", func(w http.ResponseWriter, req *http.Request) {
tmpl.Execute(w, map[string]string{"Name": "James"})
})
log.Fatal(http.ListenAndServe(":8000", nil))
}
example/data/hi.html
<h1>Hi, {{.Name}}</h1>
run like so:
go-bindata data && go build && ./example
Console Output:
<h1>Hi, James</h1>
HTTP output:
Hi, James

Related

Reference to undefined identifier bytes.ReplaceAll

I am trying to build a simple webserver. I want to replace all \n with <br>.
I wanted to use bytes for this, because my page body is stored as an []byte.
I use bytes.ReplaceAll() for this. But it keeps saying that it's a reference to undefined identifier.
Can someone tell me why? I tried the exact same line within an online Compiler and it worked just fine. Do I miss the library?
See my code below:
import (
"bytes"
"html/template"
"io/ioutil"
"log"
"net/http"
"regexp"
)
type Page struct {
Title string
Body []byte
}
func editHandler(w http.ResponseWriter, r *http.Request, title string) {
p, err := loadPage(title)
if err != nil {
p = &Page{Title: title}
}
// THE LINE THAT CAUSES TROUBLE
p.Body = bytes.ReplaceAll(p.Body, []byte("\n"), []byte("<br>"))
renderTemplate(w, "edit", p)
}
The bytes package is part of the standard library, so it is unlikely that you don't have it if you have the go tool available to you.
But do note that bytes.ReplaceAll() was added in Go 1.12, so if you have an older Go SDK, this function will not be available to you.
Execute go version to find out. Get the latest Go from the official site: https://golang.org/dl/
Further to icza's answer,
For the benefit of Go versions prior to 1.12, the following are equivalent:
bytes.ReplaceAll(a, b, c)
and
bytes.Replace(a, b, c, -1)
See the implementation of ReplaceAll

Reflect on struct type from reading a .go file

I'm discovering generator (go generate) and I'm trying to generate Validation function for my struct.
The idea is that I don't want my program to use reflect at runtime, I would rather have a generator use reflect to generate the actual method I want to use.
the problem is I can't import my structs in the generator code, the only way I found so far was to read the .go file from the generator and manually parse the types defined there using regex
I've got something like
models/models.go:
package models
//go:generate go run ../generator.go -file models.go
type MyStruct struct {
...
}
generator.go:
package main
func main() {
f, err := ioutil.ReadFile(fileName) // I read filename from the flag provided
...
// I parse f to generate my stuff
}
I would very much prefer to have an introspection package that would take a go code as a string and give me some information about the struct defined there
Or maybe there is a way to import the file that call go:generate to get directly access to the types
There is no need to specify file name , this code does the same :
//go:generate go run ../generator.go -file $GOFILE
With help of text/template package you are needless of parsing the file. A very simple example would be something like this. This will give you the clue :
package main
import (
"flag"
"os"
"text/template"
)
//go:generate go run main.go -name=A
//go:generate go run main.go -name=B
//go:generate go run main.go -name=C
var name = flag.String("name", "test", "name of struct")
var code = `
package main
type Struct{{.}} struct {}
func (s *Struct{{.}} ) Vailadte() bool {
return true
}
`
func main() {
flag.Parse()
file, _ := os.Create(*name + ".go")
defer file.Close()
tmpl, _ := template.New("test").Parse(code)
tmpl.Execute(file, *name)
}
Maybe you can utilize go/parser and go/ast in your generator.

API to get the module name

Is there an API to get the module name of a project which uses go 1.11 module system?
so I need to get abc.com/a/m from the module definition module abc.com/a/m in go.mod file.
As of this writing, I am not aware of any exposed APIs for that. However, looking at go mod sources, there is a function that can be quite useful in Go mod source file
// ModulePath returns the module path from the gomod file text.
// If it cannot find a module path, it returns an empty string.
// It is tolerant of unrelated problems in the go.mod file.
func ModulePath(mod []byte) string {
//...
}
func main() {
src := `
module github.com/you/hello
require rsc.io/quote v1.5.2
`
mod := ModulePath([]byte(src))
fmt.Println(mod)
}
Which outputs github.com/you/hello
Try this?
package main
import (
"fmt"
"io/ioutil"
"os"
modfile "golang.org/x/mod/modfile"
)
const (
RED = "\033[91m"
RESET = "\033[0m"
)
func main() {
modName := GetModuleName()
fmt.Fprintf(os.Stdout, "modName=%+v\n", modName)
}
func exitf(beforeExitFunc func(), code int, format string, args ...interface{}) {
beforeExitFunc()
fmt.Fprintf(os.Stderr, RED+format+RESET, args...)
os.Exit(code)
}
func GetModuleName() string {
goModBytes, err := ioutil.ReadFile("go.mod")
if err != nil {
exitf(func() {}, 1, "%+v\n", err)
}
modName := modfile.ModulePath(goModBytes)
fmt.Fprintf(os.Stdout, "modName=%+v\n", modName)
return modName
}
If your starting point is a go.mod file and you are asking how to parse it, I would suggest starting with go mod edit -json, which outputs a particular go.mod file in JSON format. Here is the documentation:
https://golang.org/cmd/go/#hdr-Edit_go_mod_from_tools_or_scripts
Alternatively, you could use rogpeppe/go-internal/modfile, which is a go package that can parse a go.mod file, and which is used by rogpeppe/gohack and some other tools from the broader community.
Issue #28101 I think tracks adding a new API to the Go standard library to parse go.mod files.
Here is a snippet of the documentation for go mod edit -json:
The -json flag prints the final go.mod file in JSON format instead of
writing it back to go.mod. The JSON output corresponds to these Go
types:
type Module struct {
Path string
Version string
}
type GoMod struct {
Module Module
Go string
Require []Require
Exclude []Module
Replace []Replace
}
type Require struct {
Path string
Version string
Indirect bool
}
Here is an example snippet of JSON output from go mod edit -json that shows the actual module path (aka module name), which was your original question:
{
"Module": {
"Path": "rsc.io/quote"
},
In this case, the module name is rsc.io/quote.
As of Go 1.12 (for those finding this via a search who are using modules but not necessarily the older version the OP mentioned), the runtime/debug package includes functionality for getting information about the build, including the module name. For example:
import (
"fmt"
"runtime/debug"
)
func main() {
info, _ := debug.ReadBuildInfo()
fmt.Printf("info: %+v", info.Main.Path)
}
You can run this example on the playground: https://go.dev/play/p/5oGbCRxSnjM
For more information, see the documentation for "runtime/debug".BuildInfo

My main.go file cannot see other files

I need some help understanding what is wrong with my file layout in a simple web application.
$GOPATH/src/example.com/myweb
I then have 2 files:
$GOPATH/src/example.com/myweb/main.go
$GOPATH/src/example.com/myweb/api.go
Both files have:
package main
The api.go file looks like:
package main
import (
"bytes"
"encoding/json"
"fmt"
"io"
"net/http"
"time"
)
type API struct {
URI string
Token string
Secret string
client *http.Client
}
...
My main.go file looks like:
package main
import (
"github.com/gorilla/mux"
"html/template"
"net/http"
)
var (
templates = template.Must(template.ParseFiles("views/home.html", "views/history.html", "views/incident.html"))
api = API{
URI: "http://localhost:3000",
Token: "abc",
Secret: "123",
}
)
func renderTemplate(w http.ResponseWriter, tmpl string, hp *HomePage) {
..
..
}
func WelcomeHandler(w http.ResponseWriter, r *http.Request) {
..
..
}
func main() {
r := mux.NewRouter()
r.HandleFunc("/", WelcomeHandler)
r.PathPrefix("/assets/").Handler(
http.StripPrefix("/assets/", http.FileServer(http.Dir("assets/"))))
http.ListenAndServe(":9000", r)
}
In the code I excluded, I basically use structs that are defined in my api.go file, and I get this error when doing:
go run main.go
# command-line-arguments
./main.go:16: undefined: API
./main.go:23: undefined: User
What exactly am I doing wrong here?
I tried changing the package name in api.go to myweb but that didn't help.
Am I suppose to use the package name myweb? Is just 1 file suppose to have main?
You're compiling only the main.go file. You should use:
go run main.go api.go
Or:
go run *.go
If you're writing a complex application, you might add everything to packages in subdirectories and have a single main.go file. For instance, etcd has an etcdmain subdirectory/package along with other subdirectories/packages. Something like:
/alarm
/auth
/cmd
/etcdmain
...
And the main.go file is simply:
package main
import "github.com/coreos/etcd/etcdmain"
func main() {
etcdmain.Main()
}
You are using golang workspace project, which is good for the structure for your application and it also standardize.
When we use the golang workspace, you can not run single go file. You need to call go build / go install.
Install
go install example.com/myweb
The command above will compile your main package on example.com/myweb. And the myweb executable binary will be placed on the GOPATH/bin. And you can run it manually.
Build
go build example.com/myweb
The command is similar to go install but the binary executable file will be placed on the current directory when you call the command, instead of on GOPATH/bin (unless your current directory is GOPATH/bin).
For more information please check this link.

Open a file in the same directory as the .go source file in Go

When in a source file $PWD/dir/src.go I use
os.Open("myfile.txt")
it looks for myfile.txt in $PWD (which looks normal).
Is there way to tell Go to look for myfile.txt in the same directory as src.go ? I need something like __FILE__ in Ruby.
Go is not an interpreted language so looking for a file in the same location as the source file doesn't make any sense. The go binary is compiled and the source file doesn't need to be present for the binary to run. Because of that Go doesn't come with an equivalent to FILE. The runtime.Caller function returns the file name at the time the binary was compiled.
I think perhaps if we understood why you actually wanted this functionality we could advise you better.
A possible substitute skeleton:
func __FILE__() (fn string) {
_, fn, _, _ = runtime.Caller(0)
return
}
Details here.
Use package osext
It's providing function ExecutableFolder() that returns an absolute path to folder where the currently running program executable reside (useful for cron jobs). It's cross platform.
Online documentation
package main
import (
"github.com/kardianos/osext"
"fmt"
"log"
)
func main() {
folderPath, err := osext.ExecutableFolder()
if err != nil {
log.Fatal(err)
}
fmt.Println(folderPath)
}
You can also get full executable path (similar to __FILE__):
package main
import (
"github.com/kardianos/osext"
"fmt"
)
func main() {
exeAbsolutePath, _ := osext.Executable()
fmt.Println(exeAbsolutePath)
}

Resources