How can import a file realtive to the revel basefolder in revel framework. Currently I do the following to get hold of some config values.
file, err := ioutil.ReadFile("conf/config.conf")
...
This results in my server only working if i stand in the app directory when starting revel with
revel run myapp
Is there a way to access the base folder?
There are exported global variables in the revel package, you can use any of these:
var (
// App details
AppName string // e.g. "sample"
BasePath string // e.g. "/Users/revel/gocode/src/corp/sample"
AppPath string // e.g. "/Users/revel/gocode/src/corp/sample/app"
ViewsPath string // e.g. "/Users/revel/gocode/src/corp/sample/app/views"
ImportPath string // e.g. "corp/sample"
SourcePath string // e.g. "/Users/revel/gocode/src"
// Revel installation details
RevelPath string // e.g. "/Users/revel/gocode/src/revel"
// Where to look for templates and configuration.
// Ordered by priority. (Earlier paths take precedence over later paths.)
CodePaths []string
ConfPaths []string
TemplatePaths []string
)
If it is empty for you, that is most likely because you started your app from the base folder.
Note that these paths are set by the Init(mode, importPath, srcPath string) function. It's documentation states:
srcPath - the path to the source directory, containing Revel and the app.
If not specified (""), then a functioning Go installation is required.
Also check out: how to reference a relative file from code and tests
I use this method:
In conf/app.conf add a line with the configuration path of this way:
projectname.path = "/foldersnames/"
and in the controller Write a method like this:
func info(field string) string {
config, err := revel.LoadConfig("app.conf")
if err != nil || config == nil {
log.Fatalln("Failed to load configuration file", err)
}
return config.StringDefault(field, "empty")
}
You can build a helper with this code and take configurations variables from all applications.
You must call of this way:
info("projectname.path")
Related
I would love to be able to set my executable to handle custom Url Scheme (aaa:) so that I can handle deep linking with my program.
You need to modify Windows registry, in order to tell os which protocol to open with which executable.
There are multiple ways you can go about it, depending if you want to add your url scheme handler for all users or for current user.
Easiest way to handle it is to make Windows pass custom url as argument to default executable.
That way both chrome and edge will ask user to open your installed app.
Here is a minimal Golang implementation to set your executable as default url scheme handler:
var k registry.Key
var err error
prefix := "SOFTWARE\\Classes\\"
urlScheme := "aaa"
basePath := prefix + urlScheme
permission := uint32(registry.QUERY_VALUE | registry.SET_VALUE)
baseKey := registry.CURRENT_USER
programLocation := "\"C:\\Windows\\notepad.exe\""
// create key
registry.CreateKey(baseKey, basePath, permission)
// set description
k.SetStringValue("", "Notepad app")
k.SetStringValue("URL Protocol", "")
// set icon
registry.CreateKey(registry.CURRENT_USER, "lumiere\\DefaultIcon", registry.ALL_ACCESS)
k.SetStringValue("", softwareToOpen+",1")
// create tree
registry.CreateKey(baseKey, basePath+"\\shell", permission)
registry.CreateKey(baseKey, basePath+"\\shell\\open", permission)
registry.CreateKey(baseKey, basePath+"\\shell\\open\\command", permission)
// set open command
k.SetStringValue("", softwareToOpen+" \"%1\"")
Getting custom URL after having your app set as default is quite straightforward
You can do this:
func main() {
url := os.Args[1:]
log.Printf("my custom aaa url is: %s", url)
}
I have written some Golang code which works when tested on my local machine. When I deploy this as a Google Cloud function it fails because it cannot open a template file. The line of code failing is:
t, err := template.New("list.gohtml").ParseFiles("list.gohtml")
After this call err is set to open list.gohtml: no such file or directory
The file is in the same directory as the go source file and is not listed in .gcloudignore or .gitignore. The gcloud functions documentation says all files in the directory will be uploaded unless listed in one of those ignore files and if I run gcloud meta list-files-for-upload then the file list.gohtml is included in the list displayed.
Is there some magic folder layout to make this work, or an option to the gcloud functions deploy command?
Based on #DazWilkin's reply, I now call the function below at the start of the serving function.
Rather than hardwiring the path into the template file names (which would make it fail when tested locally) this simply checks for the presence of the Gcloud source file directory below the current one, and if present, makes it the working directory, so file resolution will now happen exactly as it does when tested locally.
import "os"
const gcloudFuncSourceDir = "serverless_function_source_code"
func fixDir() {
fileInfo, err := os.Stat(gcloudFuncSourceDir)
if err == nil && fileInfo.IsDir() {
_ = os.Chdir(gcloudFuncSourceDir)
}
}
I created a Function that enumerates the uploaded files:
func L(w http.ResponseWriter, r *http.Request) {
var files []string
err := filepath.Walk(".", func(path string, info os.FileInfo, err error) error {
files = append(files, path)
return nil
})
if err != nil {
panic(err)
}
for _, file := range files {
fmt.Fprintln(w, file)
}
}
It output:
.
go.mod
go.sum
main.go
serverless_function_source_code
serverless_function_source_code/f.go
serverless_function_source_code/go.mod
serverless_function_source_code/test.tmpl
Rewriting the template-using function:
tmpl, err := template.New("test.tmpl").ParseFiles("serverless_function_source_code/test.tmpl")
Works!
However, this is undocumented and a hack :-(
The alternative is to embed the templates as strings within Golang files.
I recommend submitting a feature request on Google's Issue Tracker for Cloud Functions
If there's no go.mod file (in other words, if modules aren't enabled), then the behavior changes and the files are under src/<package-name>/, not under serverless_function_source_code/. The main.go file used to drive the cloud function resides in src/serverless_function_app/main/main.go.
For example:
I have a Go application with the following two files:
listfiles/file_system.go
listfiles/tmpl/letter.html
I deployed it with these commands:
cd listfiles
gcloud functions deploy ListFiles --runtime go113 --trigger-http --allow-unauthenticated
The result was that the current directory was set to /srv. Underneath /srv was a typical GOROOT tree with src/ and pkg/ directories. For example:
.googlebuild
.googlebuild/source-code.tar.gz
pkg
pkg/linux_amd64
pkg/linux_amd64/github.com
pkg/linux_amd64/github.com/GoogleCloudPlatform
pkg/linux_amd64/github.com/GoogleCloudPlatform/functions-framework-go
pkg/linux_amd64/github.com/GoogleCloudPlatform/functions-framework-go/funcframework.a
src
src/cloud.google.com
src/cloud.google.com/go
src/cloud.google.com/go/.git
src/cloud.google.com/go/.git/HEAD
src/cloud.google.com/go/.git/branches
[... snip...]
src/go.uber.org/zap/zaptest/testingt_test.go
src/go.uber.org/zap/zaptest/timeout.go
src/go.uber.org/zap/zaptest/timeout_test.go
src/go.uber.org/zap/zaptest/writer.go
src/go.uber.org/zap/zaptest/writer_test.go
src/listfiles
src/listfiles/file_system.go
src/listfiles/tmpl
src/listfiles/tmpl/letter.html
src/serverless_function_app
src/serverless_function_app/main
src/serverless_function_app/main/main.go
I need to set a path to my config file. The problem is, for example, my config file is in my_app/config but my working directory is in my_app/workingdir and I should not set an absolute path or a path from my working dir because in production the absolute path and working directory will change. For test I know I can set the path from my code location, but I need a cleaner way.
workingdir, err := os.Getwd()
if err != nil {
logger.Error(err)
}
viper.SetConfigFile(workingdir + "/config/local.yaml")
Don't keep your config file out of your working directory, always make a new folder in your main package like conf and put your file into it.
So, the directory structure will be
my_app/conf/config.yml
Then set the config file into viper through init function of your main.go
//main.go
func init() {
setUpViper()
}
func setUpViper() {
viper.AddConfigPath("./conf")
viper.SetConfigName("config")
}
Use ".." to specify a parent of a directory.
Because relative paths are rooted at the working directory, there's no need to get the working directory in the application code.
Use the following to specify the path to the configuration file:
viper.SetConfigFile("../config/local.yaml")
You can use .. to go to parent directory, and then from there go to config folder.
workingdir, err := os.Getwd()
if err != nil {
logger.Error(err)
}
viper.SetConfigFile(workingdir + "/../config/local.yaml")
I'd like to have a single binary for a go-app, rather than having to bundle static files alongside the deployment.
I'm using a function like this to access PNGs I'm loading:
func getFileList(dir string) (fileList []os.FileInfo, err error) {
// USAGE:
// fileList,_ := getFileList(PNG_DIR)
f, err := os.Open(PNG_DIR)
defer f.Close()
checkErr(err)
fileList, err = f.Readdir(0)
checkErr(err)
return fileList, err
}
I take this list of files and serve it on a static endpoint, with some logic.
I have read the following documentation for using go-assets
https://github.com/jessevdk/go-assets-builder/blob/master/builder.go
https://github.com/jessevdk/go-assets-builder
https://github.com/jessevdk/go-assets/blob/master/generate.go
As well as this gin specific example:
https://github.com/gin-gonic/gin/blob/master/examples/assets-in-binary/assets.go
https://github.com/jessevdk/go-assets/blob/master/example_test.go
https://github.com/gin-gonic/gin/tree/master/examples/assets-in-binary
Which contains the following example:
Prepare Packages
go get github.com/gin-gonic/gin go get
github.com/jessevdk/go-assets-builder
Generate assets.go
go-assets-builder html -o assets.go
Build the server
go build -o assets-in-binary
Run
./assets-in-binary
However, it's not clear to me how to call this file I have built. For example, What do I change in my getFileList() function to now point to whatever I built in the binary, what is it even called and how would I know that?
Usually on gin you would use router.Statuc(path, dir) however you said you first load a list of files and I guess you will later use http.ServeFile.
With go-bindata you have all the files already inside the executable, you can access them using the Asset(file) function...
Basically this is a very simple static handler for gin:
func StaticHandler(c *gin.Context) {
p := c.Param("filepath")
data, err := Assets(p)
if err != nil { return }
c.Writer.Write(data)
}
You can register the static handler into your router:
router.GET("/static/*filepath", StaticHandler)
This allows to access static resources the following way: /static/css/style.css and will load the file css/style.css
You could get the list of files inside your folder, create a map and use that map for the static handler (to limit what files are accesed)
In Node.js I use the nconf module to house environment variables like S3 keys, GCM keys, etc for each of my projects.
I haven't been able to find a similar solution in Go.
What are the generally accepted tools to help manage environment variables for each Go project?
Thanks in advance.
I would strongly recommend using github.com/namsral/flag instead.
It's like the built in flag except you can also supply the parameters via environment variables.
For example, suppose you have this code:
package main
import "fmt"
import "github.com/namsral/flag"
func main() {
var port = 3000
flag.IntVar(&port, "port", port, "Port number")
flag.Parse()
fmt.Println("You seem to prefer", port)
}
Then you can supply the values with either a command line option or an environment variable:
:~/dev/GO$ go run dummy.go
You seem to prefer 3000
:~/dev/GO$ go run dummy.go -port=1234
You seem to prefer 1234
:~/dev/GO$ PORT=4321 go run dummy.go
You seem to prefer 4321
:~/dev/GO$ PORT=4321 go run dummy.go -port=5555
You seem to prefer 5555
This might matter when it's hard to supply command line args. For example, if you use gin to automatically restart a server you have no way to supply command line arguments since gin is just calling go run on the main code without any arguments passed along.
I did some reading on this a while back when I was getting started with Go. According to this link, http://peter.bourgon.org/go-in-production/, they recommend using CLI flags (parameters) instead of environment vars - they even convert environment vars to flags to their CLI apps.
It took some getting used to; but, I really do see the advantages of going pure CLI flags between development, staging and production environments - having specific scripts for each environment.
For example, here's a little web app I wrote recently:
// global flags
var isdebug bool
var port int
var cert string
var key string
var dbdsn string
var dbmaxidle int
var dbmaxopen int
var imguri string
// init is the entry point for the entire web application.
func init() {
log.Println("Starting wwwgo ...")
// setup the flags
//flag.StringVar(&host, "host", "", "Specify a host to redirect to. Use this to redirect all traffic to a single url.")
flag.IntVar(&port, "port", 8080, "Specify the port to listen to.")
flag.BoolVar(&isdebug, "isdebug", false, "Set to true to run the app in debug mode. In debug, it may panic on some errors.")
flag.StringVar(&cert, "cert", "", "Enables listening on 443 with -cert and -key files specified. This must be a full path to the certificate .pem file. See http://golang.org/pkg/net/http/#ListenAndServeTLS for more information.")
flag.StringVar(&key, "key", "", "Enables listening on 443 with -cert and -key files specified. This must be a full path to the key .pem file. See http://golang.org/pkg/net/http/#ListenAndServeTLS for more information.")
flag.StringVar(&dbdsn, "dbdsn", "root:root#tcp(localhost:3306)/dev_db?timeout=5s&tls=false&autocommit=true", "Specifies the MySql DSN connection.")
flag.IntVar(&dbmaxidle, "dbmaxidle", 0, "Sets the database/sql MaxIdleConns.")
flag.IntVar(&dbmaxopen, "dbmaxopen", 500, "Sets the database/sql MaxOpenConns.")
flag.StringVar(&imguri, "imguri", "/cdn/uploads/", "Set this to the full base uri of all images, for example on a remote CDN server or local relative virtual directory.")
flag.Parse()
// log our flags
if isdebug != false {
log.Println("DEBUG mode enabled")
}
if cert != "" && key != "" {
log.Println("Attempting SSL binding with supplied cert and key.")
}
if dbdsn != "" {
log.Printf("Using default dbdsn: %s", dbdsn)
}
...
}
This really becomes nice in staging/production environments.
A quick ./wwwgo -h for "what the heck was that parameter?" gives you full documentation:
admin#dev01:~/code/frontend/src/wwwgo [master]$ ./wwwgo -h
Usage of ./wwwgo:
-cert="": Enables listening on 443 with -cert and -key files specified. This must be a full path to the certificate .pem file. See http://golang.org/pkg/net/http/#ListenAndServeTLS for more information.
-dbdsn="root:root#tcp(localhost:3306)/dev_db?timeout=5s&tls=false&autocommit=true": Specifies the MySql DSN connection.
-dbmaxidle=0: Sets the database/sql MaxIdleConns.
-dbmaxopen=500: Sets the database/sql MaxOpenConns.
-imguri="/cdn/uploads/": Set this to the full base uri of all images, for example on a remote CDN server or local relative virtual directory.
-isdebug=false: Set to true to run the app in debug mode. In debug, it may panic on some errors.
-key="": Enables listening on 443 with -cert and -key files specified. This must be a full path to the key .pem file. See http://golang.org/pkg/net/http/#ListenAndServeTLS for more information.
-port=8080: Specify the port to listen to.
Very nice to have many options at CLI, and no documentation required - it's built into the flags package.
You can clearly see the defaults immediately.
With this type of documentation, I tend to setup all the defaults for common "development environments" that the team uses. We all have root/root access to our local databases. We all are using port 8080 for this particular web app during development, etc. That way, you just have to run:
go build
./wwwgo
And the app runs with all defaults - defaults that are documented. In production, just override the defaults. The built-in flag parsers will panic the application if any parameters are in the wrong format, which is also very nice.
We have used this for a large scale microservice application. This doesn't involve use of any third party libraries, just plain go lang using reflection. It is very simple to use and follows DRY principle. Adding new environment variables and configuring defaults is quite easy. You can also set loading from a config file as a fallback by changing just 2 lines of code. Checkout github page for more info.
package config
import (
"fmt"
"os"
"reflect"
)
/* Tag names to load configuration from environment variable */
const (
ENV = "env"
DEFAULT = "default"
)
type Configuration struct {
Port string `env:"port" default:"3009"`
MongoURL string `env:"MongoUrl" default:"mongodb://localhost:27017/test"`
UserService string `env:"UserService" default:"http://localhost:3005"`
AuthService string `env:"AuthService" default:"http://localhost:3050"`
Debug string `env:"Debug" default:"true"`
}
/* Non-exported instance to avoid accidental overwrite */
var serviceConfig Configuration
func setConfig() {
// ValueOf returns a Value representing the run-time data
v := reflect.ValueOf(serviceConfig)
for i := 0; i < v.NumField(); i++ {
// Get the field tag value
tag := v.Type().Field(i).Tag.Get(ENV)
defaultTag := v.Type().Field(i).Tag.Get(DEFAULT)
// Skip if tag is not defined or ignored
if tag == "" || tag == "-" {
continue
}
a := reflect.Indirect(reflect.ValueOf(serviceConfig))
EnvVar, Info := loadFromEnv(tag, defaultTag)
if Info != "" {
fmt.Println("Missing environment configuration for '" + a.Type().Field(i).Name + "', Loading default setting!")
}
/* Set the value in the environment variable to the respective struct field */
reflect.ValueOf(&serviceConfig).Elem().Field(i).SetString(EnvVar)
}
}
func loadFromEnv(tag string, defaultTag string) (string, string) {
/* Check if the tag is defined in the environment or else replace with default value */
envVar := os.Getenv(tag)
if envVar == "" {
envVar = defaultTag
/* '1' is used to indicate that default value is being loaded */
return envVar, "1"
}
return envVar, ""
}
/*GetConfiguration :Exported function to return a copy of the configuration instance */
func GetConfiguration() Configuration {
return serviceConfig
}
func init() {
setConfig()
fmt.Printf("Service configuration : %+v\n ", serviceConfig)
}
Well I prefer go-arg for setting environment variables. It is easy to use and has nice features.
For example:
package configs
import (
"fmt"
"github.com/alexflint/go-arg"
)
type config struct {
DBHost string `arg:"env:DBHost, -D, --dbhost" help:"Host of the database" placeholder:"DBHost"`
}
var Config config
func main(){
arg.MustParse(&Config)
fmt.Println(Config.DBHost)
}
With this library either you can take the variable from your env or you can pass through args.
export DBHost=127.0.0.1
or
go run ./main.go --dbhost=127.0.0.1
In nodejs, nconf is a great tool. It gives more than storing secret key.
In Golang, as far as i know there is two great package to use .env file easily , godotenv and viper , I prefer godotenv beacuse it's much easier .
Pros: Developers won't see your production secrets. You can use different secrets in dev, test, and production, without having to modify the code.
Cons: Malicious code can read your secrets. The bulk of your application's code is probably open-source libraries. Bad code may creep in without you knowing it.
Example for using godotenv
First run this command inside your terminal,
go get github.com/joho/godotenv
In your .env file
S3_BUCKET=YOURS3BUCKET
SECRET_KEY=YOURSECRETKEYGOESHERE
In your main.go file
package main
import (
"github.com/joho/godotenv"
"log"
"os"
)
func main() {
err := godotenv.Load()
if err != nil {
log.Fatal("Error loading .env file")
}
s3Bucket := os.Getenv("S3_BUCKET")
secretKey := os.Getenv("SECRET_KEY")
// now do something with s3 or whatever
}