Prometheus counters: How to get current value with golang client? - go

I am using counters to count the number of requests. Is there any way to get current value of a prometheus counter?
My aim is to reuse existing counter without allocating another variable.
Golang prometheus client version is 1.1.0.

It's easy, have a function to fetch Prometheus counter value
import (
"github.com/prometheus/client_golang/prometheus"
dto "github.com/prometheus/client_model/go"
"github.com/prometheus/common/log"
)
func GetCounterValue(metric *prometheus.CounterVec) float64 {
var m = &dto.Metric{}
if err := metric.WithLabelValues("label1", "label2").Write(m); err != nil {
log.Error(err)
return 0
}
return m.Counter.GetValue()
}

Currently there is no way to get the value of a counter in the official Golang implementation.
You can also avoid double counting by incrementing your own counter and use an CounterFunc to collect it.
Note: use integral type and atomic to avoid concurrent access issues
// declare the counter as unsigned int
var requestsCounter uint64 = 0
// register counter in Prometheus collector
prometheus.MustRegister(prometheus.NewCounterFunc(
prometheus.CounterOpts{
Name: "requests_total",
Help: "Counts number of requests",
},
func() float64 {
return float64(atomic.LoadUint64(&requestsCounter))
}))
// somewhere in your code
atomic.AddUint64(&requestsCounter, 1)

It is possible to read the value of a counter (or any metric) in the official Golang implementation. I'm not sure when it was added.
This works for me for a simple metric with no vector:
func getMetricValue(col prometheus.Collector) float64 {
c := make(chan prometheus.Metric, 1) // 1 for metric with no vector
col.Collect(c) // collect current metric value into the channel
m := dto.Metric{}
_ = (<-c).Write(&m) // read metric value from the channel
return *m.Counter.Value
}
Update: here's a more general version that works with vectors and on histograms...
// GetMetricValue returns the sum of the Counter metrics associated with the Collector
// e.g. the metric for a non-vector, or the sum of the metrics for vector labels.
// If the metric is a Histogram then number of samples is used.
func GetMetricValue(col prometheus.Collector) float64 {
var total float64
collect(col, func(m dto.Metric) {
if h := m.GetHistogram(); h != nil {
total += float64(h.GetSampleCount())
} else {
total += m.GetCounter().GetValue()
}
})
return total
}
// collect calls the function for each metric associated with the Collector
func collect(col prometheus.Collector, do func(dto.Metric)) {
c := make(chan prometheus.Metric)
go func(c chan prometheus.Metric) {
col.Collect(c)
close(c)
}(c)
for x := range c { // eg range across distinct label vector values
m := dto.Metric{}
_ = x.Write(&m)
do(m)
}
}

While it is possible to obtain counter values in github.com/prometheus/client_golang as pointed at this answer, this looks too complicated. This can be greatly simplified by using an alternative library for exporing Prometheus metrics - github.com/VictoriaMetrics/metrics:
import (
"github.com/VictoriaMetrics/metrics"
)
var requestsTotal = metrics.NewCounter(`http_requests_total`)
//...
func getRequestsTotal() uint64 {
return requestsTotal.Get()
}
E.g. just call Get() function on the needed counter.

Related

How to loop through UUID items

How do I loop through a slice composed of UUIDS? My values comes from db via rows.Next()
Here's how I'm appending my uuid values to my slice (really don't know if its proper)
type Images struct {
image_id uuid.UUID `gorm:"type:uuid;primary_key;"`
}
var new_images []Images
for olds.Next() {
olds.Scan(&oldimages.image_id)
new_images = append(new_images , Images{image_id: oldimages.image_id})
}
olds here is the rows im getting from gorm Rows
olds, err := db.Raw("SELECT images_received.image_id FROM old_pics").Rows()
defer olds.Close()
Heres the function in looping I was given but its for int i dont know how to use this for uuid:
func islice(s []int, n int, f func([]int)) {
for i := 0; i < len(s); i += n {
var section []int
if i > len(s)-n {
section = s[i:]
} else {
section = s[i : i+n]
}
f(section)
}
}
Any idea how I do this? Currently for uuid im using the "github.com/satori/go.uuid" lib
I got the function from another SO question, My goal is to iterate over the rows, but rows.Next() doesnt allow that I guess in order to do that I thought I needed to append them into a slice, so I can get them by fours.
Hence leading to this question.
All you need to do is replace []int with []uuid.UUID everywhere in your islice function, including the parameter types. The functionality of islice() is not bound to []int if thats what your problem is.

concurrent map iteration and map write error in Golang 1.8

So I have this func..
func Set(firstSet map[string][]App, store map[string]*Parsed) map[string][string]struct{} {
s := make(map[string]map[string]struct{})
for dmn, parsed := range store {
for cId, apps := range firstSet {
if _, ok := s[dmn]; !ok {
s[dmn] = make(map[string]struct{})
}
s[dmn][cId] = struct{}{}
}
}
return s
}
Line 3 of that func (for dmn, parsed := range store) is giving me the error concurrent map iteration and map write error in Golang 1.8. any idea?
It looks like Concurrent Map Misuse . Probably your function invoked from different gorotines. Try to enclose function body in mutex.Lock()/Unlock() so that your map is safe for concurrent use.
There is a enhanced concurrent access check added in the Golang 1.8, and this is the source code in runtime/hashmap.go:736,
if h.flags&hashWriting != 0 {
throw("concurrent map iteration and map write")
}

Expanding a slice's size to prevent slice bounds out of range error

I have written the following:
func main() {
//inside main
fileInputBytes, err := ioutil.ReadFile("/tmp/test")
byteSize2 := len(fileInputBytes)
var inputFileByteSlice = fileInputBytes[0:]
var numberOfIndexes = math.Floor(float64(byteSize / indexingOffset))
for i := 1; i <= int(numberOfIndexes); i++ {
// adding i to the indexer insures that we use lookahed to ignore previously inserted indexing values
var v int = (i * indexingOffset) + i
Insert(&inputFileByteSlice, v+i, indexingByteValue)
fmt.Println(i)
}
}
//outside main
//variation of https://blog.golang.org/slices with pointers and such
func Insert(slice *[]byte, index int, value byte) {
// Grow the slice by one element.
(*slice) = (*slice)[0 : len(*slice)+1]
// Use copy to move the upper part of the slice out of the way and open a hole.
copy((*slice)[index+1:], (*slice)[index:])
// Store the new value.
(*slice)[index] = value
// Return the result.
}
The slice bounds out of range error is getting on my nerves. The length of the slice grows outside of the size and overflows, the reason I don't understand is that I thought the call to 'grow' the slice by one element(before copy) will dynamically allocate more space. Since that is not the case, can anyone offer me a better suggestion?
First of all, a slice is already a reference type. So you don't need to pass its pointer around if you are not going to change its capacity. So your main can be simplified as:
func main() {
fileInputBytes, err := ioutil.ReadFile("/tmp/test")
byteSize2 := len(fileInputBytes)
// No need to use pointer to slice. If you want a brand new slice
// that does not affect the original slice values, use copy()
inputFileByteArray := fileInputBytes
var numberOfIndexes = math.Floor(float64(byteSize / indexingOffset))
for i := 1; i <= int(numberOfIndexes); i++ {
var v int = (i * indexingOffset) + i
// Insert needs to return the newly updated slice reference
// which should be assigned in each iteration.
inputFileByteArray = Insert(inputFileByteArray, v+i, indexingByteValue)
fmt.Println(i)
}
}
Then, the Insert function can be simplified simply by using append along with copy and returning the newly created slice:
func Insert(slice []byte, index int, value byte) []byte {
if index >= len(slice) {
// add to the end of slice in case of index >= len(slice)
return append(slice, value)
}
tmp := make([]byte, len(slice[:index + 1]))
copy(tmp, slice[:index])
tmp[index] = value
return append(tmp, slice[index:]...)
}
This may not be the best implementation but it is simple enough. Example usage at: https://play.golang.org/p/Nuq4RX9XQD
Your function only works if the slice happens to have enough initial capacity. If you need more capacity, you can only "grow" the slice using the append function. You can still use the *[]byte pointer argument to modify the slice in place like so:
func Insert(slice *[]byte, index int, value byte) {
*slice = append(*slice, 0)
copy((*slice)[index+1:], (*slice)[index:])
(*slice)[index] = value
}
However, it's more customary to return a new slice value, and reassign it each time. This gives you a similar function signature to the builtin append.
func Insert(slice []byte, index int, value byte) []byte {
slice = append(slice, 0)
copy(slice[index+1:], slice[index:])
slice[index] = value
return slice
}

What happens if I concurrently access a single go map? [duplicate]

When you use a map in a program with concurrent access, is there any need to use a mutex in functions to read values?
Multiple readers, no writers is okay:
https://groups.google.com/d/msg/golang-nuts/HpLWnGTp-n8/hyUYmnWJqiQJ
One writer, no readers is okay. (Maps wouldn't be much good otherwise.)
Otherwise, if there is at least one writer and at least one more either writer or reader, then all readers and writers must use synchronization to access the map. A mutex works fine for this.
sync.Map has merged to Go master as of April 27, 2017.
This is the concurrent Map we have all been waiting for.
https://github.com/golang/go/blob/master/src/sync/map.go
https://godoc.org/sync#Map
I answered your question in this reddit thread few days ago:
In Go, maps are not thread-safe. Also, data requires locking even for
reading if, for example, there could be another goroutine that is
writing the same data (concurrently, that is).
Judging by your clarification in the comments, that there are going to be setter functions too, the answer to your question is yes, you will have to protect your reads with a mutex; you can use a RWMutex. For an example you can look at the source of the implementation of a table data structure (uses a map behind the scenes) which I wrote (actually the one linked in the reddit thread).
You could use concurrent-map to handle the concurrency pains for you.
// Create a new map.
map := cmap.NewConcurrentMap()
// Add item to map, adds "bar" under key "foo"
map.Add("foo", "bar")
// Retrieve item from map.
tmp, ok := map.Get("foo")
// Checks if item exists
if ok == true {
// Map stores items as interface{}, hence we'll have to cast.
bar := tmp.(string)
}
// Removes item under key "foo"
map.Remove("foo")
if you only have one writer, then you can probably get away with using an atomic Value. The following is adapted from https://golang.org/pkg/sync/atomic/#example_Value_readMostly (the original uses locks to protect writing, so supports multiple writers)
type Map map[string]string
var m Value
m.Store(make(Map))
read := func(key string) (val string) { // read from multiple go routines
m1 := m.Load().(Map)
return m1[key]
}
insert := func(key, val string) { // update from one go routine
m1 := m.Load().(Map) // load current value of the data structure
m2 := make(Map) // create a new map
for k, v := range m1 {
m2[k] = v // copy all data from the current object to the new one
}
m2[key] = val // do the update that we need (can delete/add/change)
m.Store(m2) // atomically replace the current object with the new one
// At this point all new readers start working with the new version.
// The old version will be garbage collected once the existing readers
// (if any) are done with it.
}
Why no made use of Go concurrency model instead, there is a simple example...
type DataManager struct {
/** This contain connection to know dataStore **/
m_dataStores map[string]DataStore
/** That channel is use to access the dataStores map **/
m_dataStoreChan chan map[string]interface{}
}
func newDataManager() *DataManager {
dataManager := new(DataManager)
dataManager.m_dataStores = make(map[string]DataStore)
dataManager.m_dataStoreChan = make(chan map[string]interface{}, 0)
// Concurrency...
go func() {
for {
select {
case op := <-dataManager.m_dataStoreChan:
if op["op"] == "getDataStore" {
storeId := op["storeId"].(string)
op["store"].(chan DataStore) <- dataManager.m_dataStores[storeId]
} else if op["op"] == "getDataStores" {
stores := make([]DataStore, 0)
for _, store := range dataManager.m_dataStores {
stores = append(stores, store)
}
op["stores"].(chan []DataStore) <- stores
} else if op["op"] == "setDataStore" {
store := op["store"].(DataStore)
dataManager.m_dataStores[store.GetId()] = store
} else if op["op"] == "removeDataStore" {
storeId := op["storeId"].(string)
delete(dataManager.m_dataStores, storeId)
}
}
}
}()
return dataManager
}
/**
* Access Map functions...
*/
func (this *DataManager) getDataStore(id string) DataStore {
arguments := make(map[string]interface{})
arguments["op"] = "getDataStore"
arguments["storeId"] = id
result := make(chan DataStore)
arguments["store"] = result
this.m_dataStoreChan <- arguments
return <-result
}
func (this *DataManager) getDataStores() []DataStore {
arguments := make(map[string]interface{})
arguments["op"] = "getDataStores"
result := make(chan []DataStore)
arguments["stores"] = result
this.m_dataStoreChan <- arguments
return <-result
}
func (this *DataManager) setDataStore(store DataStore) {
arguments := make(map[string]interface{})
arguments["op"] = "setDataStore"
arguments["store"] = store
this.m_dataStoreChan <- arguments
}
func (this *DataManager) removeDataStore(id string) {
arguments := make(map[string]interface{})
arguments["storeId"] = id
arguments["op"] = "removeDataStore"
this.m_dataStoreChan <- arguments
}

Go how to properly use the for ... range loop

At the moment I have a go program that contains the following code.
package main
import "time"
import "minions/minion"
func main() {
// creating the slice
ms := make([]*minion.Minion, 2)
//populating the slice and make the elements start doing something
for i := range ms {
m := &ms[i]
*m = minion.NewMinion()
(*m).Start()
}
// wait while the minions do all the work
time.Sleep(time.Millisecond * 500)
// make the elements of the slice stop with what they were doing
for i := range ms {
m := &ms[i]
(*m).Stop()
}
}
Here NewMinion() is a constructor that returns a *minion.Minion
The code works perfectly, but having to write m := &ms[i] every time I use a for ... range loop seems to me like there should be a code writer friendlier way to tackle this problem.
Ideally I'd like something like the following to be possible (using the made up &range tag):
package main
import "time"
import "minions/minion"
func main() {
// creating the slice
ms := make([]*minion.Minion, 2)
//populating the slice and make the elements start doing something
for _, m := &range ms {
*m = minion.NewMinion()
(*m).Start()
}
// wait while the minions do all the work
time.Sleep(time.Millisecond * 500)
// make the elements of the slice stop with what they were doing
for _, m := &range ms {
(*m).Stop()
}
}
Unfortunately, this is not a language feature as of yet. Any considerations on what would be the nicest way remove the m := &ms[i] from the code? Or is there no way yet that takes less effort to write than this?
Your first example is a slice of pointers, you don't need to take the address of the pointers in the slice and then dereference the pointers each time. More idiomatic Go would look like (edited slightly to run in the playground without the "minion" package):
http://play.golang.org/p/88WsCVonaL
// creating the slice
ms := make([]*Minion, 2)
//populating the slice and make the elements start doing something
for i := range ms {
ms[i] = NewMinion(i)
ms[i].Start()
// (or equivalently)
// m := MewMinion(i)
// m.Start()
// ms[i] = m
}
// wait while the minions do all the work
time.Sleep(time.Millisecond * 500)
// make the elements of the slice stop with what they were doing
for _, m := range ms {
m.Stop()
}
This is all wrong.
There is absolutely no need to take the address of a pointer in your code. ms is a slice of pointers and you constructor returns a pointer so just assign i directly:
for i := range ms {
ms[i] = minion.NewMinion()
ms[i].Start()
}
Dead simple.

Resources