I have not yet able to find some good examples in using Gauge, Counter and Histogram in prometheus. Any help on this will do. I tried using the the documentation but I was not able to successfully create a working app.
You could find examples form the prometheus/client_golang. To get you started, you can just get the packages:
$ go get github.com/prometheus/client_golang/prometheus
$ go get github.com/prometheus/client_golang/prometheus/push
The you can just run the following example by setting your correct pushgateway address, which is http://localhost:9091/ in this example:
package main
import (
"fmt"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/push"
)
func ExamplePusher_Push() {
completionTime := prometheus.NewGauge(prometheus.GaugeOpts{
Name: "db_backup_last_completion_timestamp_seconds",
Help: "The timestamp of the last successful completion of a DB backup.",
})
completionTime.SetToCurrentTime()
if err := push.New("http://localhost:9091/", "db_backup").
Collector(completionTime).
Grouping("db", "customers").
Push(); err != nil {
fmt.Println("Could not push completion time to Pushgateway:", err)
}
}
func main() {
ExamplePusher_Push()
}
Run your script:
$ go run pushExample.go
After running your code, you should see the metrics at your gateway (http://localhost:9091/). The interface looks like the following:
I have found this
`
package main
import (
"net/http"
"github.com/prometheus/client_golang/prometheus"
)
var (
cpuTemp = prometheus.NewGauge(prometheus.GaugeOpts{
Name: "cpu_temperature_celsius",
Help: "Current temperature of the CPU.",
})
hdFailures = prometheus.NewCounter(prometheus.CounterOpts{
Name: "hd_errors_total",
Help: "Number of hard-disk errors.",
})
)
func init() {
prometheus.MustRegister(cpuTemp)
prometheus.MustRegister(hdFailures)
}
func main() {
cpuTemp.Set(65.3)
hdFailures.Inc()
http.Handle("/metrics", prometheus.Handler())
http.ListenAndServe(":8080", nil)
}
`
This might be useful to some.
Prometheus is a pull-based system, if you want push-based monitoring, you need to use a gateway of some sort. A minimal example (without actually doing anything useful like starting an HTTP listener, or actually doing anything to a metric) follows:
import (
"github.com/prometheus/client_golang/prometheus"
"net/http"
)
var responseMetric = prometheus.NewHistogram(
prometheus.HistogramOpts{
Name: "request_duration_milliseconds",
Help: "Request latency distribution",
Buckets: prometheus.ExponentialBuckets(10.0, 1.13, 40),
})
func main() {
prometheus.MustRegister(responseMetric)
http.Handle("/metrics", prometheus.Handler())
// Any other setup, then an http.ListenAndServe here
}
You then need to configure Prometheus to scrape the /metrics page your binary provides.
Related
Is there a way to generate OpenAPI v3 specification from go source code? Let's say I have a go
API like the one below and I'd like to generate the OpenAPI specification (yaml file) from it. Something similar to Python's Flask RESTX. I know there are tools that generate go source code from the specs, however, I'd like to do it the other way around.
package main
import "net/http"
func main() {
http.HandleFunc("/hello", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("world\n"))
})
http.ListenAndServe(":5050", nil)
}
You can employ github.com/swaggest/rest to build a self-documenting HTTP REST API. This library establishes a convention to declare handlers in a way that can be used to reflect documentation and schema and maintain a single source of truth about it.
In my personal opinion code first approach has advantages comparing to spec first approach. It can lower the entry bar by not requiring to be an expert in spec language syntax. And it may help to come up with a spec that is well balanced with implementation details.
With code first approach it is not necessary to implement a full service to get the spec. You only need to define the structures and interfaces and may postpone actual logic implementation.
Please check a brief usage example.
package main
import (
"context"
"errors"
"fmt"
"log"
"net/http"
"time"
"github.com/go-chi/chi"
"github.com/go-chi/chi/middleware"
"github.com/swaggest/rest"
"github.com/swaggest/rest/chirouter"
"github.com/swaggest/rest/jsonschema"
"github.com/swaggest/rest/nethttp"
"github.com/swaggest/rest/openapi"
"github.com/swaggest/rest/request"
"github.com/swaggest/rest/response"
"github.com/swaggest/rest/response/gzip"
"github.com/swaggest/swgui/v3cdn"
"github.com/swaggest/usecase"
"github.com/swaggest/usecase/status"
)
func main() {
// Init API documentation schema.
apiSchema := &openapi.Collector{}
apiSchema.Reflector().SpecEns().Info.Title = "Basic Example"
apiSchema.Reflector().SpecEns().Info.WithDescription("This app showcases a trivial REST API.")
apiSchema.Reflector().SpecEns().Info.Version = "v1.2.3"
// Setup request decoder and validator.
validatorFactory := jsonschema.NewFactory(apiSchema, apiSchema)
decoderFactory := request.NewDecoderFactory()
decoderFactory.ApplyDefaults = true
decoderFactory.SetDecoderFunc(rest.ParamInPath, chirouter.PathToURLValues)
// Create router.
r := chirouter.NewWrapper(chi.NewRouter())
// Setup middlewares.
r.Use(
middleware.Recoverer, // Panic recovery.
nethttp.OpenAPIMiddleware(apiSchema), // Documentation collector.
request.DecoderMiddleware(decoderFactory), // Request decoder setup.
request.ValidatorMiddleware(validatorFactory), // Request validator setup.
response.EncoderMiddleware, // Response encoder setup.
gzip.Middleware, // Response compression with support for direct gzip pass through.
)
// Create use case interactor.
u := usecase.IOInteractor{}
// Describe use case interactor.
u.SetTitle("Greeter")
u.SetDescription("Greeter greets you.")
// Declare input port type.
type helloInput struct {
Locale string `query:"locale" default:"en-US" pattern:"^[a-z]{2}-[A-Z]{2}$" enum:"ru-RU,en-US"`
Name string `path:"name" minLength:"3"` // Field tags define parameter location and JSON schema constraints.
}
u.Input = new(helloInput)
// Declare output port type.
type helloOutput struct {
Now time.Time `header:"X-Now" json:"-"`
Message string `json:"message"`
}
u.Output = new(helloOutput)
u.SetExpectedErrors(status.InvalidArgument)
messages := map[string]string{
"en-US": "Hello, %s!",
"ru-RU": "Привет, %s!",
}
u.Interactor = usecase.Interact(func(ctx context.Context, input, output interface{}) error {
var (
in = input.(*helloInput)
out = output.(*helloOutput)
)
msg, available := messages[in.Locale]
if !available {
return status.Wrap(errors.New("unknown locale"), status.InvalidArgument)
}
out.Message = fmt.Sprintf(msg, in.Name)
out.Now = time.Now()
return nil
})
// Add use case handler to router.
r.Method(http.MethodGet, "/hello/{name}", nethttp.NewHandler(u))
// Swagger UI endpoint at /docs.
r.Method(http.MethodGet, "/docs/openapi.json", apiSchema)
r.Mount("/docs", v3cdn.NewHandler(apiSchema.Reflector().Spec.Info.Title,
"/docs/openapi.json", "/docs"))
// Start server.
log.Println("http://localhost:8011/docs")
if err := http.ListenAndServe(":8011", r); err != nil {
log.Fatal(err)
}
}
import (
"github.com/jinzhu/gorm"
_ "github.com/jinzhu/gorm/dialects/mysql"
"server/config"
"gorm.io/plugin/dbresolver"
)
func DB(config *config.Config) {
var err error
config.DB, err = gorm.Open("mysql", config.DBDSN)
if err != nil {
panic(err)
}
if !config.IsDev {
config.DB.Use(dbresolver.Register(dbresolver.Config{
Replicas: []gorm.Dialector{mysql.Open("mysql", config.DBDSN2)},
}))
}
}
I am trying to use GORM's DBresolver to make use of my cloud SQL instance's read replica. I think there is some versioning issue with the GORM package that I use and the DBresolver plugin. When I run the code above I get the following error:
config.DB.Use undefined (type *"github.com/jinzhu/gorm".DB has no
field or method Use)
undefined: "github.com/jinzhu/gorm".Dialect
I could not find any reason for this online and there are very few online resources on GORM's advanced functionalities. I can easily get the replica to work if I create a separate connection to it, but that way I need to specify the DB every-time I interact with the database.
I used the following to implement the code above: https://gorm.io/docs/dbresolver.html
You are using the v1 import path for gorm, but DBResolver is a V2 feature.
You'll want instead to use:
"gorm.io/gorm" for the main import package
"gorm.io/driver/mysql" for the driver import
gorm.Open with mysql.Open for creating the connections.
V2 is mostly backwards compatible, but you'll need to make sure any old code is tested to work on the new version.
import (
"gorm.io/gorm"
"gorm.io/driver/mysql"
"gorm.io/plugin/dbresolver"
"server/config"
)
func DB(config *config.Config) {
var err error
config.DB, err := gorm.Open(mysql.Open(config.DBDSN), &gorm.Config{})
if err != nil {
panic(err)
}
if !config.IsDev {
config.DB.Use(dbresolver.Register(dbresolver.Config{
Replicas: []gorm.Dialector{mysql.Open("mysql", config.DBDSN2)},
}))
}
}
I'm starting to write an own prometheus exporter using golang.
I think I got the basics but I don't know what to do exactly to get the value of the metric up to date. Using Set only does it once.
It is not changing on runtime.
What I have so far:
package main
import (
"log"
"net/http"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promhttp"
"time"
"io/ioutil"
"github.com/tidwall/gjson"
"strconv"
)
var (
sidekiqProcessed = setGaugeMetric("sidekiq_processed", "Sidekiq Processed", "lable", "lablevalue")
)
func setGaugeMetric(name string, help string, label string, labelvalue string) (prometheusGauge prometheus.Gauge) {
var (
gaugeMetric = prometheus.NewGauge(prometheus.GaugeOpts{
Name: name,
Help: help,
ConstLabels: prometheus.Labels{label: labelvalue},
})
)
return gaugeMetric
}
func getSidekiqProcessed() (sidekiq float64) {
body := getContent("http://example.com/sidekiq/stats")
processed := gjson.Get(body, "sidekiq.processed")
conv, err := strconv.ParseFloat(processed.String(), 64)
if err != nil {
log.Fatal(err)
}
return conv
}
func getContent(url string) (body string) {
httpClient := &http.Client{Timeout: 10 * time.Second}
res, err := httpClient.Get(url)
if err != nil {
log.Fatal(err)
}
content, err := ioutil.ReadAll(res.Body)
res.Body.Close()
if err != nil {
log.Fatal(err)
}
return string(content)
}
func init() {
prometheus.MustRegister(sidekiqProcessed)
}
func main() {
sidekiqProcessed.Set(getSidekiqProcessed())
// The Handler function provides a default handler to expose metrics
// via an HTTP server. "/metrics" is the usual endpoint for that.
http.Handle("/metrics", promhttp.Handler())
log.Fatal(http.ListenAndServe(":8080", nil))
}
Read something about Collector but have no clue how to implement it.
Can somebody help me to complete/correct my code so that the value of the metric also updates at runtime?
Here is an example of custom collector (from https://www.robustperception.io/setting-a-prometheus-counter):
package main
import "github.com/prometheus/client_golang/prometheus"
type MyCollector struct {
counterDesc *prometheus.Desc
}
func (c *MyCollector) Describe(ch chan<- *prometheus.Desc) {
ch <- c.counterDesc
}
func (c *MyCollector) Collect(ch chan<- prometheus.Metric) {
value := 1.0 // Your code to fetch the counter value goes here.
ch <- prometheus.MustNewConstMetric(
c.counterDesc,
prometheus.CounterValue,
value,
)
}
func NewMyCollector() *MyCollector {
return &MyCollector{
counterDesc: prometheus.NewDesc("my_counter_total", "Help string", nil, nil),
}
}
// To hook in the collector: prometheus.MustRegister(NewMyCollector())
You probably want to implement a collector instead, and run the http request when the Prometheus server scrapes. See the best practices.
When implementing the collector for your exporter, you should never use the usual direct instrumentation approach and then update the metrics on each scrape.
Rather create new metrics each time. In Go this is done with MustNewConstMetric in your Update() method. For Python see https://github.com/prometheus/client_python#custom-collectors and for Java generate a List in your collect method, see StandardExports.java for an example.
The github.com/prometheus/client_golang library may be non-trivial to use when writing Prometheus exporters in Go. Try https://pkg.go.dev/github.com/VictoriaMetrics/metrics library instead. It is much easier to use in general. See the following code as an example, which allows dynamically updating sidekiq_processed metric with the given label:
import (
"fmt"
"github.com/VictoriaMetrics/metrics"
)
// UpdateSidekiqProcessed updates `sidekiq_processed{label="<labelValue>"}` metric to the given value
func UpdateSidekiqProcessed(labelValue string, value float64) {
metricName := fmt.Sprintf("sidekiq_processed{label=%q}", labelValue)
metrics.GetOrCreateFloatCounter(metricName).Set(value)
}
I’ve application which should use log in state of debug. i.e. all the logs that I want to
provide is like log.debug
I’ve read about it and find the following
https://github.com/Sirupsen/logrus
https://github.com/uber-go/zap
My question is how should I “tell” to the program that now run at debug mode an then
all the logs will be printed since this I believe should come from outside …
example will be very helpful since Im new to golfing .
Ok, a really simple example of the approach I suggested in the comment:
package main
import (
"os"
"github.com/sirupsen/logrus"
)
func init() {
lvl, ok := os.LookupEnv("LOG_LEVEL")
// LOG_LEVEL not set, let's default to debug
if !ok {
lvl = "debug"
}
// parse string, this is built-in feature of logrus
ll, err := logrus.ParseLevel(lvl)
if err != nil {
ll = logrus.DebugLevel
}
// set global log level
logrus.SetLevel(ll)
}
func main() {
logrus.Debug("Will only be visible if the loglevel permits it")
}
The original comment:
Check codebases that are out there. In go, the common way to do that is to load configuration via environment variables (eg LOG_LEVEL, and use os.Getenv or a config package to load the value). That's why logrus for example allows you to set log levels via ints or strings.
Please, please: before you ask a question, read the basic info about the packages you use. Even the github repo for logrus' main README contains an example setting the log level to a specific level:
https://github.com/sirupsen/logrus#example
If the only thing you need is check level before printing you can create you own thin wrapper for standard logger.
This will help to better understand how they work. Feel free to ask any questions.
package main
import (
"log"
)
type MyLog struct {
PrintDebug bool
}
func (m *MyLog) Debug(args ...interface{}) {
if m.PrintDebug {
m.Print(args...)
}
}
func (m *MyLog) Print(args ...interface{}) {
log.Print(args...)
}
func main() {
ml := MyLog{}
ml.Debug("PrintDebig = false, so no oitput")
ml.Print("this will be printed anyway")
ml.PrintDebug = true
ml.Debug("Hello, playground")
}
https://play.golang.org/p/gKxQtC9NqX
I use logrus in all my go apps and recently I started using a context logger. Now I want to "build up" a context during the execution path of my app. See example below, which illustrates what I want.
package main
import (
"github.com/Sirupsen/logrus"
)
func main() {
logrus.Info("normal logger")
cl := logrus.WithFields(
logrus.Fields{
"extra_field_one": "extra_value_one",
})
// some code here
// here I want to add an additional field to to contextlogger cl.
// How do I do that?
}
EDIT
As ymonad mentioned, it's possible by overwriting the contextLogger. Also found out that you can add one additional field:
cl = cl.WithField("key", "value")
You can just call cl.WithFields()
package main
import "github.com/Sirupsen/logrus"
func main() {
cl := logrus.WithFields(
logrus.Fields{
"extra_field_one": "extra_value_one",
})
cl = cl.WithFields(
logrus.Fields{
"extra_field_two": "extra_value_two",
})
cl.Info("hello world")
}
Output is:
INFO[0000] hello world extra_field_one="extra_value_one" extra_field_two="extra_value_two"