Unmarshalling config files in go - go

I am trying to read a config in a json file using the viper library
//config.json
{
"currency": {
"btc": [{
"api_endpoint": "api.blockcypher.com/v1/btc/main/addrs/$address/balance",
"balance_threshold": 234234.34,
"wallet_address": "0xsdrsdf",
"alerts":{
"slack": "put slack config here",
"sms": "put sms config here"
},
"enable_monitoring": true
}],
"eth": [{
"api_endpoint": "some endpoint",
"balance_threshold" : 2234234.234,
"wallet_address": "0xsdrsdf",
"alerts":{
"slack": "put slack config here",
"sms": "put sms config here"
},
"enable_monitoring": true
}]
}
}
and config.go like this
func GetConfig() {
viper.SetConfigName("config") // name of config file (without extension)
viper.SetConfigType("json") // REQUIRED if the config file does not have the extension in the name
viper.AddConfigPath(".") // path to look for the config file in
viper.AddConfigPath("$HOME/") // path to look for the config file in
err := viper.ReadInConfig() // Find and read the config file
if err != nil { // Handle errors reading the config file
panic(fmt.Errorf("Fatal error config file: %s", err))
}
var config config
viper.Unmarshal(&config)
fmt.Println(viper.Get("currency"))
fmt.Printf("Config: %v\n", config["currency"])
}
type coreconfig struct {
APIEndpoint string `json:"api_endpoint"`
BalanceThreshold float64 `json:"balance_threshold"`
WalletAddress string `json:"wallet_address"`
EnableMonitoring bool `json:"enable_monitoring"`
Alerts map[string]alerts
}
type alerts struct {
Sms string `json:"sms"`
Slack string `json:"slack"`
}
type currencyconfig map[string][]coreconfig
type config map[string]currencyconfig
The output is printed as empty config
[map[btc:[map[alerts:[map[slack:put slack config here sms:put sms config here]] api_endpoint:api.blockcypher.com/v1/btc/main/addrs/$address/balance balance_threshold:234234 enable_monitoring:true wallet_address:0xsdrsdf]] eth:[map[alerts:[map[slack:put slack config here sms:put sms config here]] api_endpoint:some endpoint balance_threshold:2.234234e+06 enable_monitoring:true wallet_address:0xsdrsdf]]]] <= This shows that config file is read correctly
Config: map[] <= actual output

Use map[string]string or alerts only in coreconfig struct as your json stucture.
The json tag is used by the json package, not by viper. Viper use github.com/mitchellh/mapstructure package for unmarshaling, so use mapstructure tag.
type coreconfig struct {
APIEndpoint string `mapstructure:"api_endpoint"`
BalanceThreshold float64 `mapstructure:"balance_threshold"`
WalletAddress string `mapstructure:"wallet_address"`
EnableMonitoring bool `mapstructure:"enable_monitoring"`
Alerts alerts
}

Related

How to extract path from user request in golang grpc-gateway

i have a question. Is it possible to extract via metadata path from user request.
Here i have my proto file with defined method.
rpc AllPath(google.protobuf.Empty) returns (google.protobuf.Empty) {
option (google.api.http) = {
get: "/*",
};
}
rpc Auth(google.protobuf.Empty) returns (TokenRender) {
option (google.api.http) = {
get: "/auth"
};
}
}
In AllPath function in my server file im using something like this, found on grpc-gateway ecosystem website.
path := make(map[string]string)
if pattern, ok := runtime.HTTPPathPattern(ctx); ok {
path["pattern"] = pattern // /v1/example/login
}
fmt.Printf("Current path is: %v", path["pattern"])
but my current pattern/path is like i defined in proto file: Current path is: /*
If anyone have idea how to deal with this thing i would appreciate it :)
Best, Kacper
gRPC-Gateway passes various bits of information from the originating HTTP request via gRPC metadata. I don't believe the raw path is supplied, however. It is still possible to get the path passed through by registering a metadata annotator.
When calling github.com/grpc-ecosystem/grpc-gateway/v2/runtime.NewServeMux(), leverage the WithMetadata option func:
mux := runtime.NewServeMux(runtime.WithMetadata(func(_ context.Context, req *http.Request) metadata.MD {
return metadata.New(map[string]string{
"grpcgateway-http-path": req.URL.Path,
})
}))
Then in your gRPC service implementation, you can retrieve the value via the incoming context:
func (s *server) AllPath(ctx context.Context, _ *emptypb.Empty) (*emptypb.Empty, error) {
md, _ := metadata.FromIncomingContext(ctx)
log.Printf("path: %s", md["grpcgateway-http-path"][0])
return &emptypb.Empty{}, nil
}
When hitting, e.g. /foo, this should log:
2022/10/25 15:31:42 path: /foo

How to unmarshall config entry header with Go Viper package?

I have a configuration file that looks like the following:
apps:
customer1:
upload_path: "/opt/uploads/customer1"
local_path: "/opt/ready/customer1"
bucket: "b1"
customer2:
upload_path: /opt/uploads/customer2
local_path: opt/ready/customer2,
bucket: "b2"
I am using Viper to load and read the configuration file.
I am unmarshalling the above config and mapping it to the following struct:
type AppConfig struct {
UploadPath string `mapstructure:"upload_path"`
LocalPath string `mapstructure:"local_path"`
Bucket string `mapstructure:"bucket"`
}
appconfigs []*AppConfig
viper.SetConfigName(configName)
viper.SetConfigType("yaml")
viper.AddConfigPath(".")
viper.UnmarshalKey("apps", &appconfigs)
The problem I am trying to solve is getting the entry header (ie customer1 and customer2) without having to have a redundant field in my config file and ending up with:
apps:
customer1:
name: customer1
upload_path: "/opt/uploads/customer1"
local_path: "/opt/ready/customer1"
bucket: "b1"
The configuration can be unmarshaled to a map:
var appconfigs map[string]*AppConfig
viper.UnmarshalKey("apps", &appconfigs)
You can get the names from the map key.

how to do open api format yaml validation, without using clusters?

I have build a schema in open api format:
type Test_manifest struct {
metav1.TypeMeta
metav1.ObjectMeta
Spec spec
}
type spec struct {
Policies []string
Resources resources
Results []results
Variables variables
}
This is not complete schema, just a part of it.
And here below is the actual yaml file:
apiVersion: cli.kyverno.io/v1beta1
kind: kyvernotest
metadata:
name: test-check-nvidia-gpus
labels:
foolabel: foovalue
annotations:
fookey: foovalue
I'm trying to validate this incoming yaml file from the user, I can convert this yaml to json and then validate the value of the fields, but I'm not getting how to validate the field itself, I mean if user write name1 rather than name then how to show error on it. Basically how to validate the key.
Here's what I've implemented for value validation:
test := "cmd/cli/kubectl-kyverno/test/test.yaml"
yamlFile, err := ioutil.ReadFile(test)
if err != nil {
fmt.Printf("Error: failed to read file %v", err)
}
policyBytes, err1 := yaml.ToJSON(yamlFile)
if err1 != nil {
fmt.Printf("failed to convert to JSON")
}
tests := &kyvernov1.Test_manifest{}
if err := json.Unmarshal(policyBytes, tests); err != nil {
fmt.Printf("failed to decode yaml")
}
if tests.TypeMeta.APIVersion == "" {
fmt.Printf("skipping file as tests.TypeMeta.APIVersion not found")
}
if tests.TypeMeta.Kind == "" {
fmt.Printf("skipping file as tests.TypeMeta.Kind not found")
} else if tests.TypeMeta.Kind != "KyvernoTest" {
fmt.Printf("skipping file as tests.TypeMeta.Kind is not `KyvernoTest`")
}
Also we want this valiadtion to happen outside the cluster.
Two things come to my mind:
I notice that you are trying to build a k8s API extension manually which is a lot of rework, I will suggest you use a framework which will handle this for you. This is the recommended best practice that is used very frequently. It is too complicated to be done manually. Here are some resources. (kube-builder, operator-sdk). These solution are Open-API based as well. They will let you define your schema in a simple template, and generate all the validation and API + controller code for you.
If you want more validation and sanity testing, typically this is done with the help of an admission controller in your cluster. It intercepts the incoming request, and before it is processed by the API server, performs actions on it. (For verification, compliance, policy enforcement, authentication etc.) You can read more about admission controllers here.

How do you marshal a line break in Go YAML?

In a golang CLI I'm programming I collect information on how to configure the tool and I marhsal that as a YAML file. However, I'm not sure how I would add line breaks to make the file more readable?
type Config struct {
Author string `yaml:"author"`
License string `yaml:"license"`
// Marhsal a line break here
Workspace string `yaml:"workspace"`
Description string `yaml:"description"`
// Marhsal a line break here
Target string `yaml:"target"`
}
One way to implement this that allows format (and comments) is to use a template engine.
Here is a running example that generates a string with the formatted yaml, that can be then saved to a .yml file.
No additional libraries are needed and the template is included inside the go file.
package main
import (
"bytes"
"fmt"
"text/template"
)
type Config struct {
Author string
License string
Workspace string
Description string
Target string
}
const cfg_template = `
conf:
author: {{ .Author }}
licence: {{ .License }}
workspace: {{ .Workspace }}
description: {{ .Description }}
# you can even add comments to the template
target: {{ .Target }}
# other hardcoded config
foo: bar
`
func generate(config *Config) string {
t, err := template.New("my yaml generator").Parse(cfg_template)
if err != nil {
panic(err)
}
buf := &bytes.Buffer{}
err = t.Execute(buf, config)
if err != nil {
panic(err)
}
return buf.String()
}
func main() {
c := Config{
Author: "Germanio",
License: "MIT",
Workspace: "/home/germanio/workspace",
Description: "a cool description",
Target: "/home/germanio/target",
}
yaml := generate(&c)
fmt.Printf("yaml:\n%s", yaml)
}
The result looks like this:
$ go run yaml_generator.go
yaml:
conf:
author: Germanio
licence: MIT
workspace: /home/germanio/workspace
description: a cool description
# you can even add comments to the template
target: /home/germanio/target
# other hardcoded config
foo: bar
I'm sure there are better ways to implement it, just want to show a quick working example.
As empty line don't have a meaning in yaml, the default library does not create them, and does not expose an option to do so in the struct field tag.
However, if you want fine grained control of how a type is marshalled in yaml, you can always make it implements yaml.Marshaller by defining a method MarshalYAML() (interface{}, error)

Beego POST method always looks for template file

I am writing a simple login/logout feature using Beego.
My init() in router.go file is given below:
func init() {
beego.Router("/", &controllers.MainController{})
beego.Router("/login", &controllers.AuthController{})
beego.Router("/verify", &controllers.AuthController{}, "post:Verify")
}
In AuthController:
func (c *AuthController) Verify() {
email := c.GetString("email")
password := c.GetString("password")
fmt.Printf("email: %v password: %v", email, password)
}
I just want to print the details to browser (for debugging purpose) and later redirect it to another page if the user is authenticated. But the issue here is that Beego always looks for a template file and throws the below error:
can't find templatefile in the path:views/authcontroller/verify.tpl
How can I stop Beego from acting like that or am I doing something that is "not-beego-like"?
if you don't set response type, beego is always looking for default template path.
if you don't want to render template you can set response type as;
func (c *AuthController) Verify() {
defer c.ServerJSON() // response type
email := c.GetString("email")
password := c.GetString("password")
fmt.Printf("email: %v password: %v", email, password)
}

Resources