How to update metric values in prometheus exporter (golang) - go

I'm starting to write an own prometheus exporter using golang.
I think I got the basics but I don't know what to do exactly to get the value of the metric up to date. Using Set only does it once.
It is not changing on runtime.
What I have so far:
package main
import (
"log"
"net/http"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promhttp"
"time"
"io/ioutil"
"github.com/tidwall/gjson"
"strconv"
)
var (
sidekiqProcessed = setGaugeMetric("sidekiq_processed", "Sidekiq Processed", "lable", "lablevalue")
)
func setGaugeMetric(name string, help string, label string, labelvalue string) (prometheusGauge prometheus.Gauge) {
var (
gaugeMetric = prometheus.NewGauge(prometheus.GaugeOpts{
Name: name,
Help: help,
ConstLabels: prometheus.Labels{label: labelvalue},
})
)
return gaugeMetric
}
func getSidekiqProcessed() (sidekiq float64) {
body := getContent("http://example.com/sidekiq/stats")
processed := gjson.Get(body, "sidekiq.processed")
conv, err := strconv.ParseFloat(processed.String(), 64)
if err != nil {
log.Fatal(err)
}
return conv
}
func getContent(url string) (body string) {
httpClient := &http.Client{Timeout: 10 * time.Second}
res, err := httpClient.Get(url)
if err != nil {
log.Fatal(err)
}
content, err := ioutil.ReadAll(res.Body)
res.Body.Close()
if err != nil {
log.Fatal(err)
}
return string(content)
}
func init() {
prometheus.MustRegister(sidekiqProcessed)
}
func main() {
sidekiqProcessed.Set(getSidekiqProcessed())
// The Handler function provides a default handler to expose metrics
// via an HTTP server. "/metrics" is the usual endpoint for that.
http.Handle("/metrics", promhttp.Handler())
log.Fatal(http.ListenAndServe(":8080", nil))
}
Read something about Collector but have no clue how to implement it.
Can somebody help me to complete/correct my code so that the value of the metric also updates at runtime?

Here is an example of custom collector (from https://www.robustperception.io/setting-a-prometheus-counter):
package main
import "github.com/prometheus/client_golang/prometheus"
type MyCollector struct {
counterDesc *prometheus.Desc
}
func (c *MyCollector) Describe(ch chan<- *prometheus.Desc) {
ch <- c.counterDesc
}
func (c *MyCollector) Collect(ch chan<- prometheus.Metric) {
value := 1.0 // Your code to fetch the counter value goes here.
ch <- prometheus.MustNewConstMetric(
c.counterDesc,
prometheus.CounterValue,
value,
)
}
func NewMyCollector() *MyCollector {
return &MyCollector{
counterDesc: prometheus.NewDesc("my_counter_total", "Help string", nil, nil),
}
}
// To hook in the collector: prometheus.MustRegister(NewMyCollector())

You probably want to implement a collector instead, and run the http request when the Prometheus server scrapes. See the best practices.
When implementing the collector for your exporter, you should never use the usual direct instrumentation approach and then update the metrics on each scrape.
Rather create new metrics each time. In Go this is done with MustNewConstMetric in your Update() method. For Python see https://github.com/prometheus/client_python#custom-collectors and for Java generate a List in your collect method, see StandardExports.java for an example.

The github.com/prometheus/client_golang library may be non-trivial to use when writing Prometheus exporters in Go. Try https://pkg.go.dev/github.com/VictoriaMetrics/metrics library instead. It is much easier to use in general. See the following code as an example, which allows dynamically updating sidekiq_processed metric with the given label:
import (
"fmt"
"github.com/VictoriaMetrics/metrics"
)
// UpdateSidekiqProcessed updates `sidekiq_processed{label="<labelValue>"}` metric to the given value
func UpdateSidekiqProcessed(labelValue string, value float64) {
metricName := fmt.Sprintf("sidekiq_processed{label=%q}", labelValue)
metrics.GetOrCreateFloatCounter(metricName).Set(value)
}

Related

Use request ID in all logger

I've some web-application server using go http and I want that each request will have context with uuid, for this I can use http request context https://golang.org/pkg/net/http/#Request.Context
we are using logrus and we initiate it in one file and use the logger instance in other files.
what I need is to print request ID in all the logs but not to add new paremeters to each log print, I want do to it once in each http request (pass the req-id) and all the logs print will have it without doing anything with it
e.g. if the id=123 so log.info("foo") will print
// id=123 foo
I've tried with the following but not sure it's the right way, please advice.
package main
import (
"context"
"errors"
log "github.com/sirupsen/logrus"
)
type someContextKey string
var (
keyA = someContextKey("a")
keyB = someContextKey("b")
)
func main() {
ctx := context.Background()
ctx = context.WithValue(ctx, keyA, "foo")
ctx = context.WithValue(ctx, keyB, "bar")
logger(ctx, nil).Info("did nothing")
err := errors.New("an error")
logger(ctx, err).Fatal("unrecoverable error")
}
func logger(ctx context.Context, err error) *log.Entry {
entry := log.WithField("component", "main")
entry = entry.WithField("ReqID", "myRequestID")
return entry
}
https://play.golang.org/p/oCW09UhTjZ5
Every time you call the logger function you are creating a new *log.Entry and writing the request ID to it again. From your question it sounded like you do not want that.
func main() {
ctx := context.Background()
ctx = context.WithValue(ctx, keyA, "foo")
ctx = context.WithValue(ctx, keyB, "bar")
lg := logger(ctx)
lg.Info("did nothing")
err := errors.New("an error")
lg.WithError(err).Fatal("unrecoverable error")
}
func logger(ctx context.Context) *log.Entry {
entry := log.WithField("component", "main")
entry = entry.WithField("ReqID", "myRequestID")
return entry
}
The downside of this is that you will have to pass the lg variable to every function this request calls and which should also log the request ID.
What we did at our company is create a thin layer around logrus that has an additional method WithRequestCtx so we could pass in the request context and it would extract the request ID itself (which we had written to the context in a middleware). If no request ID was present nothing was added to the log entry. This however did add the request ID to every log entry again as your sample code also did.
Note: our thin layer around logrus had a lot more functionality and default settings to justify the extra effort. In the long run this turned out very helpful to have one place to be able to adjust logging for all our services.
Note2: meanwhile we are in the process of replacing logrus with zerolog to be more lightweight.
Late answer but all you need to do is just call logrus.WithContext(/* your *http.Request.Context() goes here*/).... in your application and logrus will automatically add "id":"SOME-UUID" to each logs. Design is flexible for extracting more key-value from request context if you wanted to.
initialise logger
package main
import (
"path/to/logger"
"path/to/request"
)
func main() {
err := logger.Setup(logger.Config{
ContextFields: map[string]interface{}{
string(request.CtxIDKey): request.CtxIDKey,
}
})
if err != nil {
// ...
}
}
logger
package logger
import (
"github.com/sirupsen/logrus"
)
type Config struct {
Level string
ContextFields map[string]interface{}
}
func Setup(config Config) error {
lev, err := logrus.ParseLevel(config.Level)
if err != nil {
return err
}
logrus.SetLevel(lev)
return nil
}
func (c Config) Fire(e *logrus.Entry) error {
for k, v := range c.StaticFields {
e.Data[k] = v
}
if e.Context != nil {
for k, v := range c.ContextFields {
if e.Context.Value(v) != nil {
e.Data[k] = e.Context.Value(v).(string)
}
}
}
return nil
}
request
package request
import (
"context"
"net/http"
"github.com/google/uuid"
)
type ctxID string
const CtxIDKey = ctxID("id")
func ID(h http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
h.ServeHTTP(w, r.WithContext(context.WithValue(r.Context(), CtxIDKey, uuid.New().String())))
})
}

How to re-evaluate promhttp.Handler on database change?

I'm perhaps abusing promhttp.Handler() to realise the use case for my microservice to tell me the:
version
if it has database connectivity
If there is a better way to monitor my microservices, do let me know!
I'm not sure how to structure the handle in such a way that when /metrics are called, the db.Ping() is re-evaluated.
https://s.natalian.org/2019-06-02/msping.mp4
package main
import (
"log"
"net/http"
"os"
_ "github.com/go-sql-driver/mysql"
"github.com/gorilla/mux"
"github.com/jmoiron/sqlx"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promhttp"
)
const version = "0.0.1"
type App struct {
Router *mux.Router
DB *sqlx.DB
}
func main() {
a := App{}
a.Initialize()
log.Fatal(http.ListenAndServe(":"+os.Getenv("PORT"), a.Router))
}
func (a *App) Initialize() {
connectionString := "root:secret#tcp(localhost:3306)/rest_api_example?multiStatements=true&sql_mode=TRADITIONAL&timeout=5s"
var err error
a.DB, err = sqlx.Open("mysql", connectionString)
if err != nil {
log.Fatal(err)
}
microservicecheck := prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: "mscheck",
Help: "Version with DB ping check",
},
[]string{
"commit",
},
)
if a.DB.Ping() == nil {
microservicecheck.WithLabelValues(version).Set(1)
} else {
microservicecheck.WithLabelValues(version).Set(0)
}
prometheus.MustRegister(microservicecheck)
a.Router = mux.NewRouter()
a.initializeRoutes()
}
func (a *App) initializeRoutes() {
a.Router.Handle("/metrics", promhttp.Handler()).Methods("GET")
}
https://play.golang.org/p/9DdXnz77S55
You could also add a middleware hook that does a preflight routine (i.e. your ping test) before calling promhttp.Handler(). However, at collection time, I think metrics should already have been tallied; and not generated at the instance of collection. So...
Try a separate go-routine that will poll the health of the DB connection at regular intervals. This avoids any messy hooks or custom collectors:
var pingPollingFreq = 5 * time.Second // this should probably match the Prometheus scrape interval
func (a *App) Initialize() {
// ...
prometheus.MustRegister(microservicecheck)
go func() {
for {
if a.DB.Ping() == nil {
microservicecheck.WithLabelValues(version).Set(1)
} else {
microservicecheck.WithLabelValues(version).Set(0)
}
time.Sleep(pingPollingFreq)
}
}()
// ...
}

Golang downcasting list of structs

I want to be able to unmarshal yaml files less rigidly. That is, my library has a predefined number of options the yaml file must have. Then, the user should be able to extend this to include any custom options.
Here is what I have
package main
import (
"net/http"
"yamlcms"
"github.com/julienschmidt/httprouter"
)
type Page struct {
*yamlcms.Page
Title string
Date string
}
func getBlogRoutes() {
pages := []*Page{}
yamlcms.ReadDir("html", pages)
}
// This section is a work in progress, I only include it for loose context
func main() {
router := httprouter.New()
//blogRoutes := getBlogRoutes()
//for _, blogRoute := range *blogRoutes {
// router.Handle(blogRoute.Method, blogRoute.Pattern,
// func(w http.ResponseWriter, r *http.Request, _ httprouter.Params) {})
//}
http.ListenAndServe(":8080", router)
}
Here is the yamlcms package:
package yamlcms
import (
"io/ioutil"
"os"
"strings"
"gopkg.in/yaml.v2"
)
type Page struct {
Slug string `yaml:"slug"`
File string `yaml:"file"`
}
func (page *Page) ReadFile(file string) (err error) {
fileContents, err := ioutil.ReadFile(file)
if err != nil {
return
}
err = yaml.Unmarshal(fileContents, &page)
return
}
func isYamlFile(fileInfo os.FileInfo) bool {
return !fileInfo.IsDir() && strings.HasSuffix(fileInfo.Name(), ".yaml")
}
func ReadDir(dir string, pages []*Page) (err error) {
filesInfo, err := ioutil.ReadDir(dir)
if err != nil {
return
}
for i, fileInfo := range filesInfo {
if isYamlFile(fileInfo) {
pages[i].ReadFile(fileInfo.Name())
}
}
return
}
There is a compiler issue here:
src/main.go:19: cannot use pages (type []*Page) as type []*yamlcms.Page in argument to yamlcms.ReadDir
My main intent in this question is to learn the idiomatic way of doing this kind of thing in Go. Other 3rd-party solutions may exist but I am not immediately interested in them because I have problems like this frequently in Go having to do with inheritance, etc. So along the lines of what I've presented, how can I best (idiomatically) accomplish what I am going for?
EDIT:
So I've made some changes as suggested. Now I have this:
type FileReader interface {
ReadFile(file string) error
}
func ReadDir(dir string, pages []*FileReader) (err error) {
filesInfo, err := ioutil.ReadDir(dir)
if err != nil {
return
}
for i, fileInfo := range filesInfo {
if isYamlFile(fileInfo) {
(*pages[i]).ReadFile(fileInfo.Name())
}
}
return
}
However, I still get a similar compiler error:
src/main.go:19: cannot use pages (type []*Page) as type []*yamlcms.FileReader in argument to yamlcms.ReadDir
Even though main.Page should be a FileReader because it embeds yamlcms.Page.
EDIT: I forgot that slices of interfaces don't work like that. You'd need to allocate a new slice, convert all pages to FileReaders, call the function, and convert them back.
Another possible solution is refactoring yamlcms.ReadDir to return the contents of the files, so that they could be unmarshaled later:
// In yamlcms.
func ReadYAMLFilesInDir(dir string) ([][]byte, error) { ... }
// In client code.
files := yamlcms.ReadYAMLFilesInDir("dir")
for i := range pages {
if err := yaml.Unmarshal(files[i], &pages[i]); err != nil { return err }
}
The original answer:
There are no such things as inheritance or casting in Go. Prefer composition and interfaces in your designs. In your case, you can redefine your yamlcms.ReadDir to accept an interface, FileReader.
type FileReader interface {
ReadFile(file string) error
}
Both yamlcms.Page and main.Page will implement this, as the latter embeds the former.

How to test functions that implement gorilla context

I wrote a functions that save data into redis database server. The challenge is that I want to test these functions and do not know how to test it.
I just start somehow with
Functions
package sessrage
/*
* Save data into redis database. In the common case,
* the data will be only valid during a request. Use
* hash datatype in redis.
*/
import (
"../context"
"github.com/garyburd/redigo/redis"
"net/http"
)
const (
protocol string = "tcp"
port string = ":6379"
)
func connectAndCloseRedis(connectCall func(con redis.Conn)) {
c, err := redis.Dial("tcp", ":6379")
defer c.Close()
if err != nil {
panic(err.Error())
}
connectCall(c)
}
func PostSessionData(r *http.Request, key, value string) {
go connectAndCloseRedis(func(con redis.Conn) {
sessionId := context.Get(r, context.JwtId).(string)
con.Do("HMSET", sessionId, key, value)
})
}
func GetSessionData(r *http.Request, key string) interface{} {
var result interface{}
sessionId := context.Get(r, context.JwtId).(string)
reply, _ := redis.Values(c.Do("HMGET", sessionId, key))
redis.Scan(reply, &result)
return result
}
and the test file
package sessrage
import (
//"fmt"
"../context"
. "github.com/smartystreets/goconvey/convey"
"github.com/stretchr/testify/assert"
"net/http"
"net/http/httptest"
"testing"
"time"
)
var server *httptest.Server
var glrw http.ResponseWriter
var glr *http.Request
func init() {
server = httptest.NewServer(http.HandlerFunc(func(rw http.ResponseWriter, r *http.Request) {
glrw = rw
glr = r
context.Set(glr, context.JwtId, "TestId")
}))
}
func TestPostAndGetSession(t *testing.T) {
Convey("POST and GET data on redis.", t, func() {
PostSessionData(glr, "key1", "value1")
time.Sleep(time.Second * 10)
v := GetSessionData(glr, "key1")
assert.Equal(t, "value1", v)
})
}
when I try to run the test I've got
an't load package: ......./sessrage.go:10:2: local import "../context" in non-local package
and the context package looks like
package context
import (
"github.com/gorilla/context"
"net/http"
)
type contextKey int
const (
LanguageId contextKey = iota
JwtId
)
func Get(r *http.Request, key interface{}) interface{} {
return context.Get(r, key)
}
func Set(r *http.Request, key, val interface{}) {
context.Set(r, key, val)
}
What do I wrong?
That is the first time, I am testing code in conjunction with http. It seems to be very hard to test.
There are a few issues:
Don't use relative import paths.
Use a pool instead of dialing redis on every action.
The call to sessionId := context.Get(r, context.JwtId).(string) in the PostSessionData anonymous function can fail if the mux or something higher in the call chain clears the Gorilla context before the goroutine runs. Do this instead:
func PostSessionData(r *http.Request, key, value string) {
c := pool.Get()
defer c.Close()
sessionId := context.Get(r, context.JwtId).(string)
if err := c.Do("HMSET", sessionId, key, value); err != nil {
// handle error
}
}

runtime error:Invalid memory Address or nil pointer dereference-golang

I am new to golang, am trying develop a login page with sesions. the code is building successfully but when I run in browser its saying 404 page not found.can any one help for me. Thanks in advance.
Here is my code
// main.go
package main
import (
_ "HarishSession/routers"
"github.com/astaxie/beego"
"fmt"
"net/http"
"html/template"
"strings"
"log"
"github.com/astaxie/beego/session"
"sync"
)
var globalSessions *session.Manager
var provides = make(map[string]Provider)
func sayhelloName(w http.ResponseWriter, r *http.Request) {
r.ParseForm() // parse arguments, you have to call this by yourself
fmt.Println("the information of form is",r.Form) // print form information in server side
fmt.Println("path", r.URL.Path)
fmt.Println("scheme", r.URL.Scheme)
fmt.Println(r.Form["url_long"])
for k, v := range r.Form {
fmt.Println("key:", k)
fmt.Println("val:", strings.Join(v, ""))
}
fmt.Fprintf(w, "Hello astaxie!") // send data to client side
}
type Manager struct {
cookieName string //private cookiename
lock sync.Mutex // protects session
provider Provider
maxlifetime int64
}
type Provider interface {
SessionInit(sid string) (Session, error)
SessionRead(sid string) (Session, error)
SessionDestroy(sid string) error
SessionGC(maxLifeTime int64)
}
type Session interface {
Set(key, value interface{}) error //set session value
Get(key interface{}) interface{} //get session value
Delete(key interface{}) error //delete session value
SessionID() string //back current sessionID
}
func NewManager(provideName, cookieName string, maxlifetime int64) (*Manager, error) {
provider, ok := provides[provideName]
if !ok {
return nil, fmt.Errorf("session: unknown provide %q (forgotten import?)", provideName)
}
return &Manager{provider: provider, cookieName: cookieName, maxlifetime: maxlifetime}, nil
}
func login(w http.ResponseWriter, r *http.Request) {
sess := globalSessions.SessionStart(w,r)
r.ParseForm()
fmt.Println("method:", r.Method)
if r.Method == "GET" {
t, _ := template.ParseFiles("login.tpl")
w.Header().Set("Content-Type", "text/html")
t.Execute(w,sess.Get("username"))
} else {
//logic part of log in
fmt.Println("username:",r.Form["username"])
fmt.Println("password:",r.Form["password"])
http.Redirect(w,r,"/",302)
}
}
func main() {
var globalSessions *session.Manager
http.HandleFunc("/", sayhelloName)
http.HandleFunc("/login", login)
err := http.ListenAndServe(":8080", nil) // set listen port
if err != nil {
log.Fatal("ListenAndServe the error is: ", err)
}
fmt.Println("hello")
beego.Run()
fmt.Println(globalSessions)
}
//router.go
package routers
import (
"HarishSession/controllers"
"github.com/astaxie/beego"
)
func init() {
beego.Router("/", &controllers.MainController{})
beego.Router("/login", &controllers.MainController{})
}
//default.go
package controllers
import (
"github.com/astaxie/beego"
)
type MainController struct {
beego.Controller
}
func (this *MainController) Get() {
this.Data["Website"] = "beego.me"
this.Data["Email"] = "astaxie#gmail.com"
this.TplNames = "index.tpl"
this.TplNames="login.tpl"
}
You have two variables at different scopes, each called globalSessions. One is in your definition in main.go, which is defined at global scope, and another is defined in the main function, and is defined as a local variable to main. These are separate variables. Your code is making this mistake of conflating them.
You can see this by paying closer attention to the stack trace entry:
github.com/astaxie/beego/session.(*Manager).SessionStart(0x0, 0x151e78, 0xc08212 0000, 0xc082021ad0, 0x0, 0x0)
as this points to globalSessions being uninitialized due to being nil. After that, troubleshooting is a direct matter of looking at the program to see what touches globalSessions.
Note that you should include the stack trace as part of your question. Don't just add it as a comment. It's critical to include this information: otherwise we would not have been able to easily trace the problem. Please improve the quality of your questions to make it easier for people to help you.
Also, you may want to take a serious look at go vet, which is a tool that helps to catch problems like this.
As this is the one line you used in code :
t, _ := template.ParseFiles("login.tpl")
So what you need to check is whether the file login.tpl is at the correct location, where it must be, or not. If not then correct the reference of it and also check same for the other references.
This helped me.

Resources