Example using go-assets in gin - go

I'd like to have a single binary for a go-app, rather than having to bundle static files alongside the deployment.
I'm using a function like this to access PNGs I'm loading:
func getFileList(dir string) (fileList []os.FileInfo, err error) {
// USAGE:
// fileList,_ := getFileList(PNG_DIR)
f, err := os.Open(PNG_DIR)
defer f.Close()
checkErr(err)
fileList, err = f.Readdir(0)
checkErr(err)
return fileList, err
}
I take this list of files and serve it on a static endpoint, with some logic.
I have read the following documentation for using go-assets
https://github.com/jessevdk/go-assets-builder/blob/master/builder.go
https://github.com/jessevdk/go-assets-builder
https://github.com/jessevdk/go-assets/blob/master/generate.go
As well as this gin specific example:
https://github.com/gin-gonic/gin/blob/master/examples/assets-in-binary/assets.go
https://github.com/jessevdk/go-assets/blob/master/example_test.go
https://github.com/gin-gonic/gin/tree/master/examples/assets-in-binary
Which contains the following example:
Prepare Packages
go get github.com/gin-gonic/gin go get
github.com/jessevdk/go-assets-builder
Generate assets.go
go-assets-builder html -o assets.go
Build the server
go build -o assets-in-binary
Run
./assets-in-binary
However, it's not clear to me how to call this file I have built. For example, What do I change in my getFileList() function to now point to whatever I built in the binary, what is it even called and how would I know that?

Usually on gin you would use router.Statuc(path, dir) however you said you first load a list of files and I guess you will later use http.ServeFile.
With go-bindata you have all the files already inside the executable, you can access them using the Asset(file) function...
Basically this is a very simple static handler for gin:
func StaticHandler(c *gin.Context) {
p := c.Param("filepath")
data, err := Assets(p)
if err != nil { return }
c.Writer.Write(data)
}
You can register the static handler into your router:
router.GET("/static/*filepath", StaticHandler)
This allows to access static resources the following way: /static/css/style.css and will load the file css/style.css
You could get the list of files inside your folder, create a map and use that map for the static handler (to limit what files are accesed)

Related

List ClusterServiceVersions using K8S Go client

I'm trying to use the K8S Go client to list the ClusterServiceVersions.
It could be enough to have the raw response body.
I've tried this:
data, err := clientset.RESTClient().Get().Namespace(namespace).
Resource("ClusterServiceVersions").
DoRaw(context.TODO())
if err != nil {
panic(err.Error())
}
fmt.Printf("%v", string(data))
But it returns the following error:
panic: the server could not find the requested resource (get ClusterServiceVersions.meta.k8s.io)
How do I specify to use the operators.coreos.com group?
Looking at some existing code I've also tried to add
VersionedParams(&v1.ListOptions{}, scheme.ParameterCodec)
But it result in this other error:
panic: v1.ListOptions is not suitable for converting to "meta.k8s.io/v1" in scheme "pkg/runtime/scheme.go:100"
It is possible to do a raw request using the AbsPath() method.
path := fmt.Sprintf("/apis/operators.coreos.com/v1alpha1/namespaces/%s/clusterserviceversions", namespace)
data, err := clientset.RESTClient().Get().
AbsPath(path).
DoRaw(ctx)
Also notice that if you want to define clientset using the interface (kubernetes.Interface) instead of the concrete type (*kubernetes.Clientset) the method clientset.RESTClient() is not directly accessible, but you can use the following one:
clientset.Discovery().RESTClient()

How to find a sibling file when the os.Getwd() is different in different environments

myprogram/
|
|-main.go
|-dir1/
|-data/
|-datafile.json
|-runner.go
|-runner_test.go
In runner.go, I have a simple function that reads the datafile.json. Something like
func GetPayload() (string, err) {
dBytes, dErr := ioutil.ReadFile("dir1/data/datafile.json")
if dErr != nil { return nil, dErr}
return dBytes, nil
}
I'm using Go in a Lambda with a structure similar to above. When the Lambda runs in its actual environment, it starts at main.go, and then invokes GetPayload() from runner.go. However, I have a test in a simple worker node machine in runner_test.go that also hits GetPayload() .
During "normal" execution (from main.go) - this works OK. However, when GetPayload() is invoked from runner_test.go, it errors, saying
open dir1/data/datafile.json no such file or directory
This makes sense, because during the test, the working directory is the directory that houses runner_test.go, which is data/, so there is no dir1 as a child of it. I've been trying to play with using os.Getwd() and getting the paths from there like:
pwd, _ := os.Getwd()
dBytes, dErr := ioutil.ReadFile(pwd + "dir1/data/datafile.json")
But again, that won't work, because for runner_test.go pwd is user/myname/myprogram/dir1, but from main.go, it turns up as user/myname/myprogram.
Any idea how I can find and open datafile.json from within GetPayload() in any environment? I could pass an optional parameter to GetPayload() but if possible, it'd be great to avoid that.
If the file is static (meaning that it doesn't need to change after you build the program), you can embed it into the built program. This means you no longer have to worry about run-time file paths.
import (
"embed"
)
//go:embed data/*
var dataFiles embed.FS
func GetPayload() (string, err) {
dBytes, dErr := dataFiles.ReadFile(dataFiles, "data/datafile.json")
if dErr != nil { return nil, dErr}
return dBytes, nil
}
Now the files in your data/ directory are embedded in this variable dataFiles which acts as a read-only file system.
For more info:
Read more about embed in the package documentation overview.
Read my answer about "when to use embed"
For data files that your program needs during runtime, either use a fixed directory and refer to that, or accept a command line argument or some sort of configuration that tells you where the file is.
When running unit tests, the wd is the directory containing the test file. One convention is to use a testdata/ directory under the directory containing the test, and put all data files there. That way you can refer to that file from the test by using testdata/datafile.json.
You can use a copy of the file you need during runtime as your test file, or you can use a symlink from the runtime data file to the test file under the testdata/ dir.
For data files that your program needs during runtime, either use a fixed
directory and refer to that
Someone made this suggestion, which I agree with. To that end, you can use
something like this:
package main
import (
"os"
"path/filepath"
)
func main() {
d, err := os.UserCacheDir()
if err != nil {
panic(err)
}
d = filepath.Join(d, "file.json")
f, err := os.Open(d)
if err != nil {
panic(err)
}
defer f.Close()
os.Stdout.ReadFrom(f)
}
https://golang.org/pkg/os#UserCacheDir
https://golang.org/pkg/os#UserConfigDir

How do I call a function from the main application from a plugin?

I have recently looked into Go plugins instead of manually loading .so files myself.
Basically, I have a game server application, and I want to be able to load plugins (using plugins package) when the server starts. Then, in the plugin itself, I want to be able to call exported functions that are a part of the server.
Say I have this plugin, which is compiled to example_plugin.so using go build -buildmode=plugin:
package main
import "fmt"
func init() {
fmt.Println("Hello from plugin!")
}
Then say I have this server application, which loads the plugin (and ultimately calls the "init" function under the hood):
package main
import (
"fmt"
"plugin"
)
func main() {
fmt.Println("Server started")
if _, err := plugin.Open("example_plugin.so"); err != nil {
panic(err)
}
}
// some API function that loaded plugins can call
func GetPlayers() {}
The output is:
Server started
Hello from plugin!
This works as expected, however I want to be able to call that GetPlayers function (and any other exported functions in the server application, ideally) from the plugin (and any other plugins.) I was thinking about making some sort of library consisting of interfaces containing API functions that the server implements, however I have no idea where to start. I know I will probably need to use a .a file or something similar.
For clarification, I am developing this application for use on Linux, so I am fine with a solution that only works on Linux.
Apologies if this is poorly worded, first time posting on SO.
As mentioned in the comments, there is a Lookup function. In the documentation for the module they have the following example:
// A Symbol is a pointer to a variable or function.
// For example, a plugin defined as
//
// var V int
//
// func F() { fmt.Printf("Hello, number %d\n", V) }
//
// may be loaded with the Open function and then the exported package
// symbols V and F can be accessed
package main
import (
"fmt"
"plugin"
)
func main() {
p, err := plugin.Open("plugin_name.so")
if err != nil {
panic(err)
}
v, err := p.Lookup("V")
if err != nil {
panic(err)
}
f, err := p.Lookup("F")
if err != nil {
panic(err)
}
*v.(*int) = 7
f.(func())() // prints "Hello, number 7"
}
I think the most confusing lines here are
*v.(*int) = 7
f.(func())() // prints "Hello, number 7"
The first one of them performs a type assertion to *int to assert that v is indeed a pointer to int. That is needed since Lookup returns an interface{} and in order to do anything useful with a value, you should clarify its type.
The second line performs another type assertion, this time making sure that f is a function with no arguments and no return values, after which, immediately calls it. Since function F from the original module was referencing V (which we've replaced with 7), this call will display Hello, number 7.

Gcloud functions deployment doesn't find Golang template files

I have written some Golang code which works when tested on my local machine. When I deploy this as a Google Cloud function it fails because it cannot open a template file. The line of code failing is:
t, err := template.New("list.gohtml").ParseFiles("list.gohtml")
After this call err is set to open list.gohtml: no such file or directory
The file is in the same directory as the go source file and is not listed in .gcloudignore or .gitignore. The gcloud functions documentation says all files in the directory will be uploaded unless listed in one of those ignore files and if I run gcloud meta list-files-for-upload then the file list.gohtml is included in the list displayed.
Is there some magic folder layout to make this work, or an option to the gcloud functions deploy command?
Based on #DazWilkin's reply, I now call the function below at the start of the serving function.
Rather than hardwiring the path into the template file names (which would make it fail when tested locally) this simply checks for the presence of the Gcloud source file directory below the current one, and if present, makes it the working directory, so file resolution will now happen exactly as it does when tested locally.
import "os"
const gcloudFuncSourceDir = "serverless_function_source_code"
func fixDir() {
fileInfo, err := os.Stat(gcloudFuncSourceDir)
if err == nil && fileInfo.IsDir() {
_ = os.Chdir(gcloudFuncSourceDir)
}
}
I created a Function that enumerates the uploaded files:
func L(w http.ResponseWriter, r *http.Request) {
var files []string
err := filepath.Walk(".", func(path string, info os.FileInfo, err error) error {
files = append(files, path)
return nil
})
if err != nil {
panic(err)
}
for _, file := range files {
fmt.Fprintln(w, file)
}
}
It output:
.
go.mod
go.sum
main.go
serverless_function_source_code
serverless_function_source_code/f.go
serverless_function_source_code/go.mod
serverless_function_source_code/test.tmpl
Rewriting the template-using function:
tmpl, err := template.New("test.tmpl").ParseFiles("serverless_function_source_code/test.tmpl")
Works!
However, this is undocumented and a hack :-(
The alternative is to embed the templates as strings within Golang files.
I recommend submitting a feature request on Google's Issue Tracker for Cloud Functions
If there's no go.mod file (in other words, if modules aren't enabled), then the behavior changes and the files are under src/<package-name>/, not under serverless_function_source_code/. The main.go file used to drive the cloud function resides in src/serverless_function_app/main/main.go.
For example:
I have a Go application with the following two files:
listfiles/file_system.go
listfiles/tmpl/letter.html
I deployed it with these commands:
cd listfiles
gcloud functions deploy ListFiles --runtime go113 --trigger-http --allow-unauthenticated
The result was that the current directory was set to /srv. Underneath /srv was a typical GOROOT tree with src/ and pkg/ directories. For example:
.googlebuild
.googlebuild/source-code.tar.gz
pkg
pkg/linux_amd64
pkg/linux_amd64/github.com
pkg/linux_amd64/github.com/GoogleCloudPlatform
pkg/linux_amd64/github.com/GoogleCloudPlatform/functions-framework-go
pkg/linux_amd64/github.com/GoogleCloudPlatform/functions-framework-go/funcframework.a
src
src/cloud.google.com
src/cloud.google.com/go
src/cloud.google.com/go/.git
src/cloud.google.com/go/.git/HEAD
src/cloud.google.com/go/.git/branches
[... snip...]
src/go.uber.org/zap/zaptest/testingt_test.go
src/go.uber.org/zap/zaptest/timeout.go
src/go.uber.org/zap/zaptest/timeout_test.go
src/go.uber.org/zap/zaptest/writer.go
src/go.uber.org/zap/zaptest/writer_test.go
src/listfiles
src/listfiles/file_system.go
src/listfiles/tmpl
src/listfiles/tmpl/letter.html
src/serverless_function_app
src/serverless_function_app/main
src/serverless_function_app/main/main.go

How to use golang's built in Untar (golang.org/x/build/internal/untar)

This seems like a really simple issue but for the life of me I cannot figure it out.
All I want to do is unzip and extract the contents of a tar.gz file. On godoc there seems to be a reference to a function that does exactly this. (https://godoc.org/golang.org/x/build/internal/untar).
There are no examples though and I can't seem to figure it out. Worse yet, though, I can't even figure out how to get access to the function.
Right now I have:
package main
import (
"io"
"os"
//???
)
func main() {
f, err := os.Open("foo.tar.gz")
if err != nil {
panic(err)
}
defer f.Close()
var freader io.ReadCloser = f
err = untar.Untar(freader, ".") // <- Compiler doesn't recognize
if err != nil {
panic(err)
}
freader.Close()
}
You can't. Packages with internal as part of their name can't be used by any package outside of the same repo. In other words, golang.org/x/build/internal/untar is for golang.org/x/build only, and the compiler won't allow you to reference it from anywhere else.
You can, if you really like, copy the source of that package into a new file, and use it from there, but you can't use it directly. This gives the developers of the internal untar package complete freedom to break their interface without having to worry about other users.

Resources