How do you marshal a line break in Go YAML? - go

In a golang CLI I'm programming I collect information on how to configure the tool and I marhsal that as a YAML file. However, I'm not sure how I would add line breaks to make the file more readable?
type Config struct {
Author string `yaml:"author"`
License string `yaml:"license"`
// Marhsal a line break here
Workspace string `yaml:"workspace"`
Description string `yaml:"description"`
// Marhsal a line break here
Target string `yaml:"target"`
}

One way to implement this that allows format (and comments) is to use a template engine.
Here is a running example that generates a string with the formatted yaml, that can be then saved to a .yml file.
No additional libraries are needed and the template is included inside the go file.
package main
import (
"bytes"
"fmt"
"text/template"
)
type Config struct {
Author string
License string
Workspace string
Description string
Target string
}
const cfg_template = `
conf:
author: {{ .Author }}
licence: {{ .License }}
workspace: {{ .Workspace }}
description: {{ .Description }}
# you can even add comments to the template
target: {{ .Target }}
# other hardcoded config
foo: bar
`
func generate(config *Config) string {
t, err := template.New("my yaml generator").Parse(cfg_template)
if err != nil {
panic(err)
}
buf := &bytes.Buffer{}
err = t.Execute(buf, config)
if err != nil {
panic(err)
}
return buf.String()
}
func main() {
c := Config{
Author: "Germanio",
License: "MIT",
Workspace: "/home/germanio/workspace",
Description: "a cool description",
Target: "/home/germanio/target",
}
yaml := generate(&c)
fmt.Printf("yaml:\n%s", yaml)
}
The result looks like this:
$ go run yaml_generator.go
yaml:
conf:
author: Germanio
licence: MIT
workspace: /home/germanio/workspace
description: a cool description
# you can even add comments to the template
target: /home/germanio/target
# other hardcoded config
foo: bar
I'm sure there are better ways to implement it, just want to show a quick working example.

As empty line don't have a meaning in yaml, the default library does not create them, and does not expose an option to do so in the struct field tag.
However, if you want fine grained control of how a type is marshalled in yaml, you can always make it implements yaml.Marshaller by defining a method MarshalYAML() (interface{}, error)

Related

how to do open api format yaml validation, without using clusters?

I have build a schema in open api format:
type Test_manifest struct {
metav1.TypeMeta
metav1.ObjectMeta
Spec spec
}
type spec struct {
Policies []string
Resources resources
Results []results
Variables variables
}
This is not complete schema, just a part of it.
And here below is the actual yaml file:
apiVersion: cli.kyverno.io/v1beta1
kind: kyvernotest
metadata:
name: test-check-nvidia-gpus
labels:
foolabel: foovalue
annotations:
fookey: foovalue
I'm trying to validate this incoming yaml file from the user, I can convert this yaml to json and then validate the value of the fields, but I'm not getting how to validate the field itself, I mean if user write name1 rather than name then how to show error on it. Basically how to validate the key.
Here's what I've implemented for value validation:
test := "cmd/cli/kubectl-kyverno/test/test.yaml"
yamlFile, err := ioutil.ReadFile(test)
if err != nil {
fmt.Printf("Error: failed to read file %v", err)
}
policyBytes, err1 := yaml.ToJSON(yamlFile)
if err1 != nil {
fmt.Printf("failed to convert to JSON")
}
tests := &kyvernov1.Test_manifest{}
if err := json.Unmarshal(policyBytes, tests); err != nil {
fmt.Printf("failed to decode yaml")
}
if tests.TypeMeta.APIVersion == "" {
fmt.Printf("skipping file as tests.TypeMeta.APIVersion not found")
}
if tests.TypeMeta.Kind == "" {
fmt.Printf("skipping file as tests.TypeMeta.Kind not found")
} else if tests.TypeMeta.Kind != "KyvernoTest" {
fmt.Printf("skipping file as tests.TypeMeta.Kind is not `KyvernoTest`")
}
Also we want this valiadtion to happen outside the cluster.
Two things come to my mind:
I notice that you are trying to build a k8s API extension manually which is a lot of rework, I will suggest you use a framework which will handle this for you. This is the recommended best practice that is used very frequently. It is too complicated to be done manually. Here are some resources. (kube-builder, operator-sdk). These solution are Open-API based as well. They will let you define your schema in a simple template, and generate all the validation and API + controller code for you.
If you want more validation and sanity testing, typically this is done with the help of an admission controller in your cluster. It intercepts the incoming request, and before it is processed by the API server, performs actions on it. (For verification, compliance, policy enforcement, authentication etc.) You can read more about admission controllers here.

Pass resource from Command To Subcommands in urfave/cli/v2

Is is possible, and if so how, to let a Command initialize a resource and pass it down to its Subcommands. Image an application that takes its arguments like
$ mycmd db --connect <...> create <...>
$ mycmd db --connect <...> update <...>
This may not be a great example but it illustrates the concept. Here db is some resource that all the subcommands depend on. I would like a single function to be responsible for the initialization of the db resource and then pass the initialized resource down to the subcommands. I can't figure out how to do this with urfave/cli/v2 .
You could do it by creating two separate cli.Apps, one that parses the db part of the arguments just to create a context.Context with context.WithValue and then use that context to create the second cli.App which would parse the remainder of the arguments. I'm sure there's a better way to do it.
I'm grateful for any help!
You can achieve this with context values. You set the value in the Before callback of the parent Command. Below code is copied and modified from the subcommands example:
package main
import (
"context"
"fmt"
"log"
"os"
"github.com/urfave/cli/v2"
)
func main() {
app := &cli.App{
Commands: []*cli.Command{
{
Name: "db",
Before: func(c *cli.Context) error {
db := "example"
c.Context = context.WithValue(c.Context, "db", db)
return nil
},
Subcommands: []*cli.Command{
{
Name: "connect",
Action: func(c *cli.Context) error {
db := c.Context.Value("db").(string) // remember to assert to original type
fmt.Println("sub command:", db)
return nil
},
},
},
},
},
}
err := app.Run(os.Args)
if err != nil {
log.Fatal(err)
}
}
This main uses a string so that you can copy paste and run it. You can replace string with your DB object.
How to test:
$ go build -o example
$ ./example db connect
sub command: example

gomplate: executing "<arg>" at <.Values.tpl.organization>: map has no entry for key "Values"

Here is my yaml file
apiVersion: myapi.com/v1
kind: Template
metadata:
name: {{ .Values.tpl.organization }}-template-mkytemplatetemplate
spec:
# Add fields here
organization: {{ .Values.tpl.organization }}
purpose: {{ .Values.tpl.purpose }}
version: {{ .Chart.appVersion }}
location: {{.Values.location}}/template.tgz
name: mkytemplatetemplate
namePattern: ^[a-z0-9\-]{3,25}$
description: "# Please do not use the template\ntest modify run.sh"
author: string
I use gomplate to replace all {{xxx}} to corresponding values in Chart.yaml or Values.yaml. Here is my code
func main() {
log.Println("hello")
//BasicTemplate()
Gomplate()
}
func Gomplate() {
workspace := "/Users/i517131/code/mkytemplatetemplate/.ci/chart"
inBs, _ := ioutil.ReadFile(path.Join(workspace, "templates/template.yaml"))
in := string(inBs)
cfg := &gomplate.Config{
Input: in,
OutputFiles: []string{path.Join(workspace, "result.yaml")},
DataSources: []string{
fmt.Sprintf("%s=%s", "Chart", path.Join(workspace, "Chart.yaml")),
fmt.Sprintf("%s=%s", "Values", path.Join(workspace, "values.yaml")),
},
}
err := gomplate.RunTemplates(cfg)
if err != nil {
panic(err)
}
}
but I receive an error like this panic: template: <arg>:4:18: executing "<arg>" at <.Values.tpl.organization>: map has no entry for key "Values". at err := gomplate.RunTemplates(cfg)
At first, when I run cmd gomplate -f .ci/chart/templates/template.yaml -d Chart=.ci/chart/Chart.yaml -d Values=.ci/chart/values.yaml -o result.yaml, I receive the save error.
I searched the internet and in github, the author suggest us use -c instead of -d
But the gomplate in go can only use gomplate.Config to run templates and do not support -c. What can I do?
result generated by -c cmd
apiVersion: myapi.com/v1
kind: Template
metadata:
name: mky-template-mkytemplatetemplate
spec:
# Add fields here
organization: mky
purpose: prod
version: 1.0.0
location: _/template.tgz
name: mkytemplatetemplate
namePattern: ^[a-z0-9\-]{3,25}$
description: "# Please do not use the template\ntest modify run.sh"
author: string
use Contexts not DataSource
cfg := &gomplate.Config{
Input: in,
OutputFiles: []string{path.Join(workspace, "result.yaml")},
Contexts: []string{
fmt.Sprintf("%s=%s", "Chart", path.Join(workspace, "Chart.yaml")),
fmt.Sprintf("%s=%s", "Values", path.Join(workspace, "values.yaml")),
},
}

how to convert the given code in go templating

I am using go's text/template. I want to do something like:
method = some_var
path = some_other_var
if method is "GET" and "ID" in path
How can I do this in a go template? I am doing it like this.
{{- if and eq .Method "GET" contains "AssetID" .OperationId -}}
EDIT:
The thing is I am working with openAPI to generate a server-code boilerplate. So the templates are in that repo. I am doing it something like:
$ go get github.com/deepmap/oapi-codegen/cmd/oapi-codegen
$ oapi-codegen \
-templates my-templates/ \
-generate types,server \
example-expanded.yaml
above oapi-codegen line is here.
my-templates contains the templates I have changed. These are also provided by oapi-codegen. this dir contains them and I have copied and changed a few of them and followed the steps as directed here.
in one of those templates that I changed, I want to use contains.
What would be the best way to do this?
There is no builtin contains function in templates, so you have to register your function for that. You may use the strings.Contains() function from the standard lib. For reference, the available builtin template functions are listed here: Functions
And you have to group the params of the eq and contains like this:
{{if and (eq .Method "GET") (contains .AssetID .OperationId)}}
true
{{else}}
false
{{end}}
Example code to register the strings.Contains() function, parse the template and execute it:
t := template.Must(template.New("").Funcs(template.FuncMap{
"contains": strings.Contains,
}).Parse(src))
params := map[string]interface{}{
"Method": "GET",
"AssetID": "/some/path/123",
"OperationId": "123",
}
if err := t.Execute(os.Stdout, params); err != nil {
panic(err)
}
params["OperationId"] = "xxx"
if err := t.Execute(os.Stdout, params); err != nil {
panic(err)
}
This will output (try it on the Go Playground):
true
false

Unmarshalling config files in go

I am trying to read a config in a json file using the viper library
//config.json
{
"currency": {
"btc": [{
"api_endpoint": "api.blockcypher.com/v1/btc/main/addrs/$address/balance",
"balance_threshold": 234234.34,
"wallet_address": "0xsdrsdf",
"alerts":{
"slack": "put slack config here",
"sms": "put sms config here"
},
"enable_monitoring": true
}],
"eth": [{
"api_endpoint": "some endpoint",
"balance_threshold" : 2234234.234,
"wallet_address": "0xsdrsdf",
"alerts":{
"slack": "put slack config here",
"sms": "put sms config here"
},
"enable_monitoring": true
}]
}
}
and config.go like this
func GetConfig() {
viper.SetConfigName("config") // name of config file (without extension)
viper.SetConfigType("json") // REQUIRED if the config file does not have the extension in the name
viper.AddConfigPath(".") // path to look for the config file in
viper.AddConfigPath("$HOME/") // path to look for the config file in
err := viper.ReadInConfig() // Find and read the config file
if err != nil { // Handle errors reading the config file
panic(fmt.Errorf("Fatal error config file: %s", err))
}
var config config
viper.Unmarshal(&config)
fmt.Println(viper.Get("currency"))
fmt.Printf("Config: %v\n", config["currency"])
}
type coreconfig struct {
APIEndpoint string `json:"api_endpoint"`
BalanceThreshold float64 `json:"balance_threshold"`
WalletAddress string `json:"wallet_address"`
EnableMonitoring bool `json:"enable_monitoring"`
Alerts map[string]alerts
}
type alerts struct {
Sms string `json:"sms"`
Slack string `json:"slack"`
}
type currencyconfig map[string][]coreconfig
type config map[string]currencyconfig
The output is printed as empty config
[map[btc:[map[alerts:[map[slack:put slack config here sms:put sms config here]] api_endpoint:api.blockcypher.com/v1/btc/main/addrs/$address/balance balance_threshold:234234 enable_monitoring:true wallet_address:0xsdrsdf]] eth:[map[alerts:[map[slack:put slack config here sms:put sms config here]] api_endpoint:some endpoint balance_threshold:2.234234e+06 enable_monitoring:true wallet_address:0xsdrsdf]]]] <= This shows that config file is read correctly
Config: map[] <= actual output
Use map[string]string or alerts only in coreconfig struct as your json stucture.
The json tag is used by the json package, not by viper. Viper use github.com/mitchellh/mapstructure package for unmarshaling, so use mapstructure tag.
type coreconfig struct {
APIEndpoint string `mapstructure:"api_endpoint"`
BalanceThreshold float64 `mapstructure:"balance_threshold"`
WalletAddress string `mapstructure:"wallet_address"`
EnableMonitoring bool `mapstructure:"enable_monitoring"`
Alerts alerts
}

Resources