Related
I'm working on a small web application in Go that's meant to be used as a tool on a developer's machine to help debug their applications/web services. The interface to the program is a web page that includes not only the HTML but some JavaScript (for functionality), images, and CSS (for styling). I'm planning on open-sourcing this application, so users should be able to run a Makefile, and all the resources will go where they need to go. However, I'd also like to be able to simply distribute an executable with as few files/dependencies as possible. Is there a good way to bundle the HTML/CSS/JS with the executable, so users only have to download and worry about one file?
Right now, in my app, serving a static file looks a little like this:
// called via http.ListenAndServe
func switchboard(w http.ResponseWriter, r *http.Request) {
// snipped dynamic routing...
// look for static resource
uri := r.URL.RequestURI()
if fp, err := os.Open("static" + uri); err == nil {
defer fp.Close()
staticHandler(w, r, fp)
return
}
// snipped blackhole route
}
So it's pretty simple: if the requested file exists in my static directory, invoke the handler, which simply opens the file and tries to set a good Content-Type before serving. My thought was that there's no reason this needs to be based on the real filesystem: if there were compiled resources, I could simply index them by request URI and serve them as such.
Let me know if there's not a good way to do this or I'm barking up the wrong tree by trying to do this. I just figured the end-user would appreciate as few files as possible to manage.
If there are more appropriate tags than go, please feel free to add them or let me know.
Starting with Go 1.16 the go tool has support for embedding static files directly in the executable binary.
You have to import the embed package, and use the //go:embed directive to mark what files you want to embed and into which variable you want to store them.
3 ways to embed a hello.txt file into the executable:
import "embed"
//go:embed hello.txt
var s string
print(s)
//go:embed hello.txt
var b []byte
print(string(b))
//go:embed hello.txt
var f embed.FS
data, _ := f.ReadFile("hello.txt")
print(string(data))
Using the embed.FS type for the variable you can even include multiple files into a variable that will provide a simple file-system interface:
// content holds our static web server content.
//go:embed image/* template/*
//go:embed html/index.html
var content embed.FS
The net/http has support to serve files from a value of embed.FS using http.FS() like this:
http.Handle("/static/", http.StripPrefix("/static/", http.FileServer(http.FS(content))))
The template packages can also parse templates using text/template.ParseFS(), html/template.ParseFS() functions and text/template.Template.ParseFS(), html/template.Template.ParseFS() methods:
template.ParseFS(content, "*.tmpl")
The following of the answer lists your old options (prior to Go 1.16).
Embedding Text Files
If we're talking about text files, they can easily be embedded in the source code itself. Just use the back quotes to declare the string literal like this:
const html = `
<html>
<body>Example embedded HTML content.</body>
</html>
`
// Sending it:
w.Write([]byte(html)) // w is an io.Writer
Optimization tip:
Since most of the times you will only need to write the resource to an io.Writer, you can also store the result of a []byte conversion:
var html = []byte(`
<html><body>Example...</body></html>
`)
// Sending it:
w.Write(html) // w is an io.Writer
Only thing you have to be careful about is that raw string literals cannot contain the back quote character (`). Raw string literals cannot contain sequences (unlike the interpreted string literals), so if the text you want to embed does contain back quotes, you have to break the raw string literal and concatenate back quotes as interpreted string literals, like in this example:
var html = `<p>This is a back quote followed by a dot: ` + "`" + `.</p>`
Performance is not affected, as these concatenations will be executed by the compiler.
Embedding Binary Files
Storing as a byte slice
For binary files (e.g. images) most compact (regarding the resulting native binary) and most efficient would be to have the content of the file as a []byte in your source code. This can be generated by 3rd party toos/libraries like go-bindata.
If you don't want to use a 3rd party library for this, here's a simple code snippet that reads a binary file, and outputs Go source code that declares a variable of type []byte that will be initialized with the exact content of the file:
imgdata, err := ioutil.ReadFile("someimage.png")
if err != nil {
panic(err)
}
fmt.Print("var imgdata = []byte{")
for i, v := range imgdata {
if i > 0 {
fmt.Print(", ")
}
fmt.Print(v)
}
fmt.Println("}")
Example output if the file would contain bytes from 0 to 16 (try it on the Go Playground):
var imgdata = []byte{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15}
Storing as base64 string
If the file is not "too large" (most images/icons qualify), there are other viable options too. You can convert the content of the file to a Base64 string and store that in your source code. On application startup (func init()) or when needed, you can decode it to the original []byte content. Go has nice support for Base64 encoding in the encoding/base64 package.
Converting a (binary) file to base64 string is as simple as:
data, err := ioutil.ReadFile("someimage.png")
if err != nil {
panic(err)
}
fmt.Println(base64.StdEncoding.EncodeToString(data))
Store the result base64 string in your source code, e.g. as a const.
Decoding it is just one function call:
const imgBase64 = "<insert base64 string here>"
data, err := base64.StdEncoding.DecodeString(imgBase64) // data is of type []byte
Storing as quoted string
More efficient than storing as base64, but may be longer in source code is storing the quoted string literal of the binary data. We can obtain the quoted form of any string using the strconv.Quote() function:
data, err := ioutil.ReadFile("someimage.png")
if err != nil {
panic(err)
}
fmt.Println(strconv.Quote(string(data))
For binary data containing values from 0 up to 64 this is how the output would look like (try it on the Go Playground):
"\x00\x01\x02\x03\x04\x05\x06\a\b\t\n\v\f\r\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f !\"#$%&'()*+,-./0123456789:;<=>?"
(Note that strconv.Quote() appends and prepends a quotation mark to it.)
You can directly use this quoted string in your source code, for example:
const imgdata = "\x00\x01\x02\x03\x04\x05\x06\a\b\t\n\v\f\r\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f !\"#$%&'()*+,-./0123456789:;<=>?"
It is ready to use, no need to decode it; the unquoting is done by the Go compiler, at compile time.
You may also store it as a byte slice should you need it like that:
var imgdata = []byte("\x00\x01\x02\x03\x04\x05\x06\a\b\t\n\v\f\r\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f !\"#$%&'()*+,-./0123456789:;<=>?")
The go-bindata package looks like it might be what you're interested in.
https://github.com/go-bindata/go-bindata
It will allow you to convert any static file into a function call that can be embedded in your code and will return a byte slice of the file content when called.
Bundle React application
For example, you have a build output from react like the following:
build/favicon.ico
build/index.html
build/asset-manifest.json
build/static/css/**
build/static/js/**
build/manifest.json
When you use go:embed like this, it will serve the contents as http://localhost:port/build/index.html which is not what we want (unexpected /build).
//go:embed build/*
var static embed.FS
// ...
http.Handle("/", http.FileServer(http.FS(static)))
In fact, we will need to take one more step to make it works as expected by using fs.Sub:
package main
import (
"embed"
"io/fs"
"log"
"net/http"
)
//go:embed build/*
var static embed.FS
func main() {
contentStatic, _ := fs.Sub(static, "build")
http.Handle("/", http.FileServer(http.FS(contentStatic)))
log.Fatal(http.ListenAndServe("localhost:8080", nil))
}
Now, http://localhost:8080 should serve your web application as expected.
Credit to Amit Mittal.
Note: go:embed requires go 1.16 or higher.
also there is some exotic way - I use maven plugin to build GoLang projects and it allows to use JCP preprocessor to embed binary blocks and text files into sources. In the case code just look like line below (and some example can be found here)
var imageArray = []uint8{/*$binfile("./image.png","uint8[]")$*/}
As a popular alternative to go-bindata mentioned in another answer, mjibson/esc also embeds arbitrary files, but handles directory trees particularly conveniently.
I am using google grpc with a json proxy. for some reason i need to remove the omitempty tags from the struct generated in the *.pb.go files.
if i have a proto message like this
message Status {
int32 code = 1;
string message = 2;
}
The generated struct looks like this
type Status struct {
Code int32 `protobuf:"varint,1,opt,name=code" json:"code,omitempty"`
Message string `protobuf:"bytes,2,opt,name=message" json:"message,omitempty"`
}
But My need is to remove the omitempty tag from the generated structs. How can i do this?
If you are using grpc-gateway and you need the default values to be present during json marshaling, you may consider to add the following option when creating your servemux
gwmux := runtime.NewServeMux(runtime.WithMarshalerOption(runtime.MIMEWildcard, &runtime.JSONPb{OrigName: true, EmitDefaults: true}))
Outside of grpc-gateway, if you want to marshal your protocol buffer message, use google.golang.org/protobuf/encoding/protojson (*) package instead of encoding/json
func sendProtoMessage(resp proto.Message, w http.ResponseWriter) {
w.Header().Set("Content-Type", "application/json; charset=utf-8")
m := protojson.Marshaler{EmitDefaults: true}
m.Marshal(w, resp) // You should check for errors here
}
(*) google.golang.org/protobuf replaces the now deprecated github.com/golang/protobuf and its jsonpb package.
A [more] portable solution:
Use sed to strip the tags after generating via protoc.
Example of what I actually use in my go:generate script after having generated the *.pb.go files:
ls *.pb.go | xargs -n1 -IX bash -c 'sed s/,omitempty// X > X.tmp && mv X{.tmp,}'
Note: sed -i (inline-replacement) is not used here because that flag isn't portable between standard OS-X and Linux.
You can try using gogo proto (https://github.com/gogo/protobuf)
With the jsontag extension, your proto message would look like
message Status {
int32 code = 1 [(gogoproto.jsontag) = "code"];
string message = 2 [(gogoproto.jsontag) = "message"];
}
You can also add more tags, if you like.
I found that the omitempty json tag is hard-coded into the protoc-gen-go source around line 1778:
tag := fmt.Sprintf("protobuf:%s json:%q",
g.goTag(message, field, wiretype), jsonName+",omitempty")
it will be easy changing the source and make a new protoc-gen-go binary yourself.
It's worth noting that this is likely inadvisable and not recommended for several reasons, particularly because you will then be responsible for ensuring that the hacked up binary always gets used if the protobufs need to be regenerated.
I am posting an update to DeeSilence's answer that works with the latest protobuf version (at the moment of writing).
import "google.golang.org/protobuf/encoding/protojson"
m := protojson.MarshalOptions{EmitUnpopulated: true}
resp, err := m.Marshal(w)
The Marshaler under the jsonpb package has a EmitDefaults field. Setting this to true, will just ignore the omitempty tag in struct.
https://godoc.org/github.com/golang/protobuf/jsonpb#JSONPBMarshaler
You could use "sed" command to remove this text from files like following
sed -i "" -e "s/,omitempty//g" ./api/proto/*.go
where args:
-i "" is meaning that keep same name of files
-e "s/,omitempty//g" = format to replace like "s/SEARCH/INSERT/g"
you can copy the encoding/json package to your own folder for example my_json, and modify omitEmpty field to false, and use my_json.Marshal() to encode the struct to json string.
To expand on #colm.anseo's edit saying that the google.golang.org/protobuf/encoding/protojson is now deprecated and now google.golang.org/protobuf should be used, here is an example how to do it without modifying the proto.
import (
"google.golang.org/protobuf/encoding/protojson"
)
func main() {
someProto := SomeProto{}
marshaler := protojson.MarshalOptions{EmitUnpopulated: true}
output, _ := marshaler.Marshal(someProto)
...
}
This is the only way to do it if you don't want to modify the generated struct / are importing the struct via buf and the bsr.
I need to parse a Go source code file, find a specific type (by name) and use it in my program.
I already managed to find the type I need using the go/ast package but I don't know how to "load" it into my program so that I can use it.
Question: What's the best way to extract and use a type from an external source code file and use it on runtime?
I can't think of anything except an ugly method to basically copy the file, modify it by injecting a "main" function with my encoding stuff which sends the result to stdOut, execute the it, collect the encoded data from stdout, delete the modified file.
Use case: Analyse go source code and encode the types in a specific format (e.g. json schema)
Edit:
Here is some code. The question is how to encode type allTypes (zero value) and then print it.
package main
import (
"fmt"
"go/ast"
"go/parser"
"go/token"
"encoding/json"
)
var src string = `
package mypack
type allTypes struct{
Brands Brands
Colours Colours
}
type Brands struct{
Sony string
Apple string
}
type Colours struct{
Red string
Green string
}
`
type sometype struct{
Nothing int
}
func main() {
// src is the input for which we want to inspect the AST.
// Create the AST by parsing src.
fset := token.NewFileSet() // positions are relative to fset
f, err := parser.ParseFile(fset, "src.go", src, 0)
if err != nil {
panic(err)
}
// Inspect the AST and find our function
var tp ast.TypeSpec
ast.Inspect(f, func(n ast.Node) bool {
switch x := n.(type) {
case *ast.TypeSpec:
if x.Name.Name == "allTypes"{
tp = *x
}
}
return true
})
fmt.Printf("We found the type: it is %v", tp)
// Encode the zero value of sometype
x := sometype{}
b, _ := json.Marshal(&x)
fmt.Printf("\n Zero value of someType (json) %s", b)
//Next/Question: How to encode the zero value of "allTypes" ???
}
Also on playground
If I understand you are asking for dynamic type loading ala Java's Class.forName(String className). The short answer is Go doesn't support this.
The correct way, as Nick Johnson pointed out, is to parse the tree using ast, and then "generate" the JSON yourself. You will not be able to "load" the type and use JSON.Marshal. It is also worth noting that any type which supports the json.Marshaler interface may generate custom JSON. You also need to ignore, but mark optional "omitempty" behavior. This really prevents you from using the compile it and hack through "stdout" behavior as well.
If you need to extract type information at runtime, you need to use the reflect package. This is the way Go's encoding/json and other similar packages work.
If you want to operate on the types defined in a Go source file, you can use the go.parser package to read and parse a source file into an AST, then traverse the AST for the elements you want to examine.
It seems there is no way to load the type information extracted from the source code(please correct me if I'm wrong). The only solution is to create/generate a package (main), inject it with all the types extracted from the target source code file, build/compile it, execute it and collect the encoded data from stdout.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 7 years ago.
The community reviewed whether to reopen this question 12 months ago and left it closed:
Opinion-based Update the question so it can be answered with facts and citations by editing this post.
Improve this question
What is the preferred way to handle configuration parameters for a Go program (the kind of stuff one might use properties files or ini files for, in other contexts)?
The JSON format worked for me quite well. The
standard library offers methods to write the data structure indented, so it is quite
readable.
See also this golang-nuts thread.
The benefits of JSON are that it is fairly simple to parse and human readable/editable
while offering semantics for lists and mappings (which can become quite handy), which
is not the case with many ini-type config parsers.
Example usage:
conf.json:
{
"Users": ["UserA","UserB"],
"Groups": ["GroupA"]
}
Program to read the configuration
import (
"encoding/json"
"os"
"fmt"
)
type Configuration struct {
Users []string
Groups []string
}
file, _ := os.Open("conf.json")
defer file.Close()
decoder := json.NewDecoder(file)
configuration := Configuration{}
err := decoder.Decode(&configuration)
if err != nil {
fmt.Println("error:", err)
}
fmt.Println(configuration.Users) // output: [UserA, UserB]
Another option is to use TOML, which is an INI-like format created by Tom Preston-Werner. I built a Go parser for it that is extensively tested. You can use it like other options proposed here. For example, if you have this TOML data in something.toml
Age = 198
Cats = [ "Cauchy", "Plato" ]
Pi = 3.14
Perfection = [ 6, 28, 496, 8128 ]
DOB = 1987-07-05T05:45:00Z
Then you can load it into your Go program with something like
type Config struct {
Age int
Cats []string
Pi float64
Perfection []int
DOB time.Time
}
var conf Config
if _, err := toml.DecodeFile("something.toml", &conf); err != nil {
// handle error
}
Viper is a golang configuration management system that works with JSON, YAML, and TOML. It looks pretty interesting.
I usually use JSON for more complicated data structures. The downside is that you easily end up with a bunch of code to tell the user where the error was, various edge cases and what not.
For base configuration (api keys, port numbers, ...) I've had very good luck with the gcfg package. It is based on the git config format.
From the documentation:
Sample config:
; Comment line
[section]
name = value # Another comment
flag # implicit value for bool is true
Go struct:
type Config struct {
Section struct {
Name string
Flag bool
}
}
And the code needed to read it:
var cfg Config
err := gcfg.ReadFileInto(&cfg, "myconfig.gcfg")
It also supports slice values, so you can allow specifying a key multiple times and other nice features like that.
Just use standard go flags with iniflags.
go flags
import "flag"
var nFlag = flag.Int("n", 1234, "help message for flag n")
iniflags
package main
import (
"flag"
...
"github.com/vharitonsky/iniflags"
...
)
var (
flag1 = flag.String("flag1", "default1", "Description1")
...
flagN = flag.Int("flagN", 123, "DescriptionN")
)
func main() {
iniflags.Parse() // use instead of flag.Parse()
}
Standard go flags have the following benefits:
Idiomatic.
Easy to use. Flags can be easily added and scattered across arbitrary packages your project uses.
Flags have out-of-the-box support for default values and description.
Flags provide standard 'help' output with default values and description.
The only drawback standard go flags have - is management problems when the number of flags used in your app becomes too large.
Iniflags elegantly solves this problem: just modify two lines in your main package and it magically gains support for reading flag values from ini file. Flags from ini files can be overriden by passing new values in command-line.
See also https://groups.google.com/forum/#!topic/golang-nuts/TByzyPgoAQE for details.
I have started using Gcfg which uses Ini-like files. It's simple - if you want something simple, this is a good choice.
Here's the loading code I am using currently, which has default settings and allows command line flags (not shown) that override some of my config:
package util
import (
"code.google.com/p/gcfg"
)
type Config struct {
Port int
Verbose bool
AccessLog string
ErrorLog string
DbDriver string
DbConnection string
DbTblPrefix string
}
type configFile struct {
Server Config
}
const defaultConfig = `
[server]
port = 8000
verbose = false
accessLog = -
errorLog = -
dbDriver = mysql
dbConnection = testuser:TestPasswd9#/test
dbTblPrefix =
`
func LoadConfiguration(cfgFile string, port int, verbose bool) Config {
var err error
var cfg configFile
if cfgFile != "" {
err = gcfg.ReadFileInto(&cfg, cfgFile)
} else {
err = gcfg.ReadStringInto(&cfg, defaultConfig)
}
PanicOnError(err)
if port != 0 {
cfg.Server.Port = port
}
if verbose {
cfg.Server.Verbose = true
}
return cfg.Server
}
have a look at gonfig
// load
config, _ := gonfig.FromJson(myJsonFile)
// read with defaults
host, _ := config.GetString("service/host", "localhost")
port, _ := config.GetInt("service/port", 80)
test, _ := config.GetBool("service/testing", false)
rate, _ := config.GetFloat("service/rate", 0.0)
// parse section into target structure
config.GetAs("service/template", &template)
https://github.com/spf13/viper and https://github.com/zpatrick/go-config are a pretty good libraries for configuration files.
Use toml like this article Reading config files the Go way
I wrote a simple ini config library in golang.
https://github.com/c4pt0r/cfg
goroutine-safe, easy to use
package cfg
import (
"testing"
)
func TestCfg(t *testing.T) {
c := NewCfg("test.ini")
if err := c.Load() ; err != nil {
t.Error(err)
}
c.WriteInt("hello", 42)
c.WriteString("hello1", "World")
v, err := c.ReadInt("hello", 0)
if err != nil || v != 42 {
t.Error(err)
}
v1, err := c.ReadString("hello1", "")
if err != nil || v1 != "World" {
t.Error(err)
}
if err := c.Save(); err != nil {
t.Error(err)
}
}
===================Update=======================
Recently I need an INI parser with section support, and I write a simple package:
github.com/c4pt0r/cfg
u can parse INI like using "flag" package:
package main
import (
"log"
"github.com/c4pt0r/ini"
)
var conf = ini.NewConf("test.ini")
var (
v1 = conf.String("section1", "field1", "v1")
v2 = conf.Int("section1", "field2", 0)
)
func main() {
conf.Parse()
log.Println(*v1, *v2)
}
You might also be interested in go-libucl, a set of Go bindings for UCL, the Universal Configuration Language. UCL is a bit like JSON, but with better support for humans: it supports comments and human-readable constructs like SI multipliers (10k, 40M, etc.) and has a little bit less boilerplate (e.g., quotes around keys). It's actually pretty close to the nginx configuration file format, if you're already familiar with that.
I agree with nemo and I wrote a little tool to make it all real easy.
bitbucket.org/gotamer/cfg is a json configuration package
You define your config items in your application as a struct.
A json config file template from your struct is saved on the first run
You can save runtime modifications to the config
See doc.go for an example
I tried JSON. It worked. But I hate having to create the struct of the exact fields and types I might be setting. To me that was a pain. I noticed it was the method used by all the configuration options I could find. Maybe my background in dynamic languages makes me blind to the benefits of such verboseness. I made a new simple config file format, and a more dynamic-ish lib for reading it out.
https://github.com/chrisftw/ezconf
I am pretty new to the Go world, so it might not be the Go way. But it works, it is pretty quick, and super simple to use.
Pros
Super simple
Less code
Cons
No Arrays or Map types
Very flat file format
Non-standard conf files
Does have a little convention built-in, which I now if frowned upon in general in Go community. (Looks for config file in the config directory)
I'm working on a small web application in Go that's meant to be used as a tool on a developer's machine to help debug their applications/web services. The interface to the program is a web page that includes not only the HTML but some JavaScript (for functionality), images, and CSS (for styling). I'm planning on open-sourcing this application, so users should be able to run a Makefile, and all the resources will go where they need to go. However, I'd also like to be able to simply distribute an executable with as few files/dependencies as possible. Is there a good way to bundle the HTML/CSS/JS with the executable, so users only have to download and worry about one file?
Right now, in my app, serving a static file looks a little like this:
// called via http.ListenAndServe
func switchboard(w http.ResponseWriter, r *http.Request) {
// snipped dynamic routing...
// look for static resource
uri := r.URL.RequestURI()
if fp, err := os.Open("static" + uri); err == nil {
defer fp.Close()
staticHandler(w, r, fp)
return
}
// snipped blackhole route
}
So it's pretty simple: if the requested file exists in my static directory, invoke the handler, which simply opens the file and tries to set a good Content-Type before serving. My thought was that there's no reason this needs to be based on the real filesystem: if there were compiled resources, I could simply index them by request URI and serve them as such.
Let me know if there's not a good way to do this or I'm barking up the wrong tree by trying to do this. I just figured the end-user would appreciate as few files as possible to manage.
If there are more appropriate tags than go, please feel free to add them or let me know.
Starting with Go 1.16 the go tool has support for embedding static files directly in the executable binary.
You have to import the embed package, and use the //go:embed directive to mark what files you want to embed and into which variable you want to store them.
3 ways to embed a hello.txt file into the executable:
import "embed"
//go:embed hello.txt
var s string
print(s)
//go:embed hello.txt
var b []byte
print(string(b))
//go:embed hello.txt
var f embed.FS
data, _ := f.ReadFile("hello.txt")
print(string(data))
Using the embed.FS type for the variable you can even include multiple files into a variable that will provide a simple file-system interface:
// content holds our static web server content.
//go:embed image/* template/*
//go:embed html/index.html
var content embed.FS
The net/http has support to serve files from a value of embed.FS using http.FS() like this:
http.Handle("/static/", http.StripPrefix("/static/", http.FileServer(http.FS(content))))
The template packages can also parse templates using text/template.ParseFS(), html/template.ParseFS() functions and text/template.Template.ParseFS(), html/template.Template.ParseFS() methods:
template.ParseFS(content, "*.tmpl")
The following of the answer lists your old options (prior to Go 1.16).
Embedding Text Files
If we're talking about text files, they can easily be embedded in the source code itself. Just use the back quotes to declare the string literal like this:
const html = `
<html>
<body>Example embedded HTML content.</body>
</html>
`
// Sending it:
w.Write([]byte(html)) // w is an io.Writer
Optimization tip:
Since most of the times you will only need to write the resource to an io.Writer, you can also store the result of a []byte conversion:
var html = []byte(`
<html><body>Example...</body></html>
`)
// Sending it:
w.Write(html) // w is an io.Writer
Only thing you have to be careful about is that raw string literals cannot contain the back quote character (`). Raw string literals cannot contain sequences (unlike the interpreted string literals), so if the text you want to embed does contain back quotes, you have to break the raw string literal and concatenate back quotes as interpreted string literals, like in this example:
var html = `<p>This is a back quote followed by a dot: ` + "`" + `.</p>`
Performance is not affected, as these concatenations will be executed by the compiler.
Embedding Binary Files
Storing as a byte slice
For binary files (e.g. images) most compact (regarding the resulting native binary) and most efficient would be to have the content of the file as a []byte in your source code. This can be generated by 3rd party toos/libraries like go-bindata.
If you don't want to use a 3rd party library for this, here's a simple code snippet that reads a binary file, and outputs Go source code that declares a variable of type []byte that will be initialized with the exact content of the file:
imgdata, err := ioutil.ReadFile("someimage.png")
if err != nil {
panic(err)
}
fmt.Print("var imgdata = []byte{")
for i, v := range imgdata {
if i > 0 {
fmt.Print(", ")
}
fmt.Print(v)
}
fmt.Println("}")
Example output if the file would contain bytes from 0 to 16 (try it on the Go Playground):
var imgdata = []byte{0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15}
Storing as base64 string
If the file is not "too large" (most images/icons qualify), there are other viable options too. You can convert the content of the file to a Base64 string and store that in your source code. On application startup (func init()) or when needed, you can decode it to the original []byte content. Go has nice support for Base64 encoding in the encoding/base64 package.
Converting a (binary) file to base64 string is as simple as:
data, err := ioutil.ReadFile("someimage.png")
if err != nil {
panic(err)
}
fmt.Println(base64.StdEncoding.EncodeToString(data))
Store the result base64 string in your source code, e.g. as a const.
Decoding it is just one function call:
const imgBase64 = "<insert base64 string here>"
data, err := base64.StdEncoding.DecodeString(imgBase64) // data is of type []byte
Storing as quoted string
More efficient than storing as base64, but may be longer in source code is storing the quoted string literal of the binary data. We can obtain the quoted form of any string using the strconv.Quote() function:
data, err := ioutil.ReadFile("someimage.png")
if err != nil {
panic(err)
}
fmt.Println(strconv.Quote(string(data))
For binary data containing values from 0 up to 64 this is how the output would look like (try it on the Go Playground):
"\x00\x01\x02\x03\x04\x05\x06\a\b\t\n\v\f\r\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f !\"#$%&'()*+,-./0123456789:;<=>?"
(Note that strconv.Quote() appends and prepends a quotation mark to it.)
You can directly use this quoted string in your source code, for example:
const imgdata = "\x00\x01\x02\x03\x04\x05\x06\a\b\t\n\v\f\r\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f !\"#$%&'()*+,-./0123456789:;<=>?"
It is ready to use, no need to decode it; the unquoting is done by the Go compiler, at compile time.
You may also store it as a byte slice should you need it like that:
var imgdata = []byte("\x00\x01\x02\x03\x04\x05\x06\a\b\t\n\v\f\r\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f !\"#$%&'()*+,-./0123456789:;<=>?")
The go-bindata package looks like it might be what you're interested in.
https://github.com/go-bindata/go-bindata
It will allow you to convert any static file into a function call that can be embedded in your code and will return a byte slice of the file content when called.
Bundle React application
For example, you have a build output from react like the following:
build/favicon.ico
build/index.html
build/asset-manifest.json
build/static/css/**
build/static/js/**
build/manifest.json
When you use go:embed like this, it will serve the contents as http://localhost:port/build/index.html which is not what we want (unexpected /build).
//go:embed build/*
var static embed.FS
// ...
http.Handle("/", http.FileServer(http.FS(static)))
In fact, we will need to take one more step to make it works as expected by using fs.Sub:
package main
import (
"embed"
"io/fs"
"log"
"net/http"
)
//go:embed build/*
var static embed.FS
func main() {
contentStatic, _ := fs.Sub(static, "build")
http.Handle("/", http.FileServer(http.FS(contentStatic)))
log.Fatal(http.ListenAndServe("localhost:8080", nil))
}
Now, http://localhost:8080 should serve your web application as expected.
Credit to Amit Mittal.
Note: go:embed requires go 1.16 or higher.
also there is some exotic way - I use maven plugin to build GoLang projects and it allows to use JCP preprocessor to embed binary blocks and text files into sources. In the case code just look like line below (and some example can be found here)
var imageArray = []uint8{/*$binfile("./image.png","uint8[]")$*/}
As a popular alternative to go-bindata mentioned in another answer, mjibson/esc also embeds arbitrary files, but handles directory trees particularly conveniently.