Golang: implements http server health checking. gocraft/health - go

I want to check the health of my service,having the metrics of each endPoint.
My service calls some other services and recieves a Json code, I make templates with it, and then I send it to a http.ResponseWriter.
I searched and I found this package "gocraft/health" but I didn't really understand how it works.
Is there any other way or package to generate metrics or should I just use "gocraft/health.
Thank you in advance

Finally, I choose "gocraft/health", a great library.
Example of usage:
package main
import (
"log"
"net/http"
"os"
"time"
"github.com/gocraft/health"
)
//should be global Var
var stream = health.NewStream()
func main() {
// Log to stdout!
stream.AddSink(&health.WriterSink{os.Stdout})
// Make sink and add it to stream
sink := health.NewJsonPollingSink(time.Minute*5, time.Minute*20)
stream.AddSink(sink)
// Start the HTTP server! This will expose metrics via a JSON API.
adr := "127.0.0.1:5001"
sink.StartServer(adr)
http.HandleFunc("/api/getVastPlayer", vastPlayer)
log.Println("Listening...")
panic(http.ListenAndServe(":2001", nil))
}
Per the initialization options above, your metrics are aggregated in 5-minute chunks. We'll keep 20 minutes worth of data in memory. Nothing is ever persisted to disk.
You can create as many jobs as you want
func vastPlayer(w http.ResponseWriter, r *http.Request) {
job_1 := stream.NewJob("/api/getVastPlayer")
...
...
if bol {
job_1.Complete(health.Success)
} else {
job_1.Complete(health.Error)
}
}
Once you start your app, this will expose metrics via a JSON API. You can browse the /health endpoint (eg, 127.0.0.1:5001/health) to see the metrics. You will get something like that:
{
"instance_id": "sd-69536.29342",
"interval_duration": 86400000000000,
"aggregations": [
{
"interval_start": "2015-06-11T02:00:00+02:00",
"serial_number": 1340,
"jobs": {
"/api/getVastPlayer": {
"timers": {},
"events": {},
"event_errs": {},
"count": 1328,
"nanos_sum": 140160794784,
"nanos_sum_squares": 9.033775178022173E+19,
"nanos_min": 34507863,
"nanos_max": 2736850494,
"count_success": 62,
"count_validation_error": 1266,
"count_panic": 0,
"count_error": 0,
"count_junk": 0
},
"timers": {},
"events": {},
"event_errs": {}
}
}
]
}
For more information and functionality check this link:
https://github.com/gocraft/health

In case you came across this question because you were looking for exposing a /health endpoint, there is an upcoming RFC for health checks: https://github.com/inadarei/rfc-healthcheck
And there's a Go library health-go for exposing health endpoints compliant with that RFC: https://github.com/nelkinda/health-go
Example:
package main
import (
"github.com/nelkinda/health-go"
"net/http"
)
func main() {
// 1. Create the health Handler.
h := health.New(health.Health{Version: "1", ReleaseID: "1.0.0-SNAPSHOT"})
// 2. Add the handler to your mux/server.
http.HandleFunc("/health", h.Handler)
// 3. Start your server.
http.ListenAndServe(":80", nil)
}
It is extensible and supports a number of built-in checks such as uptime and sysinfo.
Disclaimer: I'm the author of health-go.

Related

Set nested fields in Golang with the Logrus?

I'm using github.com/sirupsen/logrus to make a logger. This simplifies how fields are being initiated in the logger (straightforward from docs):
package main
import "github.com/sirupsen/logrus"
func main() {
logFields := logrus.Fields{
"details": logrus.Fields{
"app_name": "Some name",
"app_version": "Some version",
},
}
logger := logrus.New()
logger.WithFields(logFields)
}
The library seems to allow to add fields with method WithFields only at top level (as in the last line).
But because my API has many steps, I need to add nested fields under details with every step (to accumulate log data), then output it all together in the end.
How can I add these fields?
logrus.Fields is just map[string]interface{}, so you can nest all data in every steps into a struct like
type Details struct {
Step1 string `json:"step1"`
Step2 string `json:"step2"`
Step3 string `json:"step3"`
}
and finally logger.WithFields(logFields).Debugln("some debug info")
Another method, you can create multiple variables to save data from every step, for example
var step1data = "step1"
var step2data = "step2"
and finally
logger.WithFields(logrus.Fields{
"details": logrus.Fields{
"step1": step1data,
"step2": step2data,
},
}).Debugln("some debug info")`

AWS API Gateway HTTP API how to pass string query params?

So I am making an app and need AWS API Gateway. I want to use HTTP API instead of REST API. My code looks like this
package main
import (
"database/sql"
"fmt"
"strings"
"github.com/aws/aws-lambda-go/lambda"
_ "github.com/lib/pq"
)
here I make a connection to the database
func fetch(inte string, input string) string {
if err != nil {
panic(err)
}
switch inte {
case "00":
{
res = append(res, response)
}
switch len(res) {
case 0:
return "401"
}
case "01":
}
switch len(res) {
case 0:
return "402"
}
}
return "404"
}
type LambdaEvent struct {
Req string `json:"req"`
Num string `json:"num"`
}
type LambdaResponse struct {
Res string `json:"res"`
}
func LambdaHandler(event LambdaEvent) (LambdaResponse, error) {
res := fetch(event.Num, event.Req)
return LambdaResponse{
Res: res,
}, nil
}
func main() {
lambda.Start(LambdaHandler)
}
So as you see this is not the full code. I make a connection to the database and and work with the requests string query. so I tried the same with http api but it just gives me the 404 meaning the http api doesn't pass the query string to the lambda so how do I make my api pass the data to the lambda. Rest api works HTTP doesn't.
Thanks for any help.
I'm not familiar with Serverless Frameworks for APIGW, but manipulating QueryString parameters is built into the APIGW Console. Just login to AWS and search for APIGateway. Edit your HTTP API and select Integrations from the menu on the left. Select the integration that maps to your Lambda function and Edit the Parameter Mappings on the right
If you are deploying your lambdas and api-gateway with serverless framework you can do something like this:
hello:
handler: src/hello.handler
name: hello
events:
- http:
path: car/{id}/color/{color}
method: get
Assuming you are planning to use Lambda Proxy Integration in API Gateway, here are the changes that needs to be done to access the query parameters.
import github.com/aws/aws-lambda-go/events (This has all the relevant structs)
Change the lambda handler to func handler(request events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
Now you can access the query parameters as a Map at request.QueryStringParameters and execute your selection logic
When you return a response for API Gateway, ensure you follow the events.APIGatewayProxyResponse struct i.e. at least return a status code along with optional body, headers etc.
No changes/config required at the API Gateway to pass the query parameters with Lambda proxy integration
You can use your own structs for request and response, but they need to use the appropriate keys as defined in the events.APIGatewayProxyRequest and events.APIGatewayProxyResponse.
e.g. add the following in LambdaEvent struct to access the query string parameters.
QueryStringParameters map[string]string `json:"queryStringParameters"`
If you are getting started with AWS Lambda, have a look at AWS SAM to keep things simple.

How to get images path from golang project file

I am a beginner in making a REST full API using golang, what I want to ask is to get an image file from the path in my golang project, before I have successfully uploaded the image and saved it in the path and the path I have saved it in mysql database, how do I do it to be able to display the image in the form of a link, so I can load it on android,
and how to make it like this
"http: // localhost: 3004 / images / imagename.jpg"
inside the json result ( ImgEvent )??
please help me... thanks
func (idb *InDB) GetEvents(c *gin.Context) {
var (
events []structs.Event
newEvents structs.Event
result gin.H
)
getimage := "img_event"
data := idb.DB.Select(getimage).Find(&events)
if data != nil {
result = gin.H{
"message": "Not found",
}
}
newEvents.ImgEvent = getimage
data = idb.DB.Find(&events)
if len(events) <= 0 {
result = gin.H{
"result": nil,
"count": 0,
}
} else {
result = gin.H{
"data": events,
}
}
c.JSON(http.StatusOK, result)
}
========= Respone Json ===========
{
"data": [
{
"ID": 1,
"CreatedAt": "2019-07-28T22:38:20Z",
"UpdatedAt": "2019-08-12T09:51:41Z",
"DeletedAt": null,
"Judul": "test1",
"Isi": "Dalam menyambut ulang tahun Accent-er Regional JawaBarat yang ke 3.\r\n\r\nKami mengundang seluruh member terdaftar maupun non member untuk ikut hadir meramaikan acara ini.\r\n\r\nYang akan di selenggarakan di\r\n\r\nCikole,Lembang Bandung
"ImgEvent": "event-images/event-bukalapak.jpg",
"TotalComments": 1,
"TotalLikes": 5,
"TotalView": 3
},
You should a create a file server to be able create an http link for files. You can use net/http's Fileserver for the same. Fileserver is a good choice if you want it for a small time, like as long as you API server is running.
Another choice would be to create an FTP server and use that to store images.

kubernetes filter objects in Informer

Im writing custom controller for kubernetes.
Im creating shared informer
cache.NewSharedIndexInformer(
&cache.ListWatch{
ListFunc: func(options meta_v1.ListOptions) (k8sruntime.Object, error) {
return client.CoreV1().ConfigMaps(nameSpace).List(options)
},
WatchFunc: func(options meta_v1.ListOptions) (watch.Interface, error) {
return client.CoreV1().ConfigMaps(nameSpace).Watch(options)
},
},
&api_v1.ConfigMap{},
0, //Skip resyncr
cache.Indexers{},
)
I have option to add filtering function into call back functions to further decrease number of objects im working with.
Something like that
options.FieldSelector := fields.OneTermEqualSelector("metadata.name", nodeName).String()
I would like to filter out objects by regular expression. Or by some label at least. Unfortunately documentation is not helping. Could not find anything except for tests for code itself.
Ho do i apply regular expression on filtering mechanism?
Where do i get some examples on this issue?
Its not possible to filter objects by regular expression.
It is possible to filer object by lable
This is the code that will filter by label
labelSelector := labels.Set(map[string]string{"mylabel": "ourdaomain1"}).AsSelector()
informer := cache.NewSharedIndexInformer(
&cache.ListWatch{
ListFunc: func(options meta_v1.ListOptions) (k8sruntime.Object, error) {
options.LabelSelector = labelSelector.String()
return client.CoreV1().ConfigMaps(nameSpace).List(options)
},
WatchFunc: func(options meta_v1.ListOptions) (watch.Interface, error) {
options.LabelSelector = labelSelector.String()
return client.CoreV1().ConfigMaps(nameSpace).Watch(options)
},
},
&api_v1.ConfigMap{},
0, //Skip resyncr
cache.Indexers{},
)
Another thing that is important to remember is how do you add new objects to k8s
I was doing something like
kubectl --namespace==ourdomain1 create configmap config4 -f ./config1.yaml
This is not good. It overwrites all the fields in config map and puts whole file content into data of the new object.
Proper way is
kubectl create -f ./config1.yam

Endpoints Google ID JWT, and Golang oauth2: have id_token, need access_token?

I have an app engine app behind endpoints and I'm having trouble following the documentation around adding authentication. My preference is to allow service accounts within the project through, then perform more granular app-side authorization.
In my case, most clients will be outside GCP and will be automated programs not people, so I'm using JSON key files thinking that is the way to go (correct me if I'm wrong please). I also don't want to have to redeploy the app to change user configs, so I follow the "GOOGLE ID JWT" information in the documentation from here:
https://cloud.google.com/endpoints/docs/openapi/service-to-service-auth
This is the security sections of my swagger JSON:
"securityDefinitions": {
"api_key": {
"type": "apiKey",
"name": "key",
"in": "query"
},
"google_id_token": {
"authorizationUrl": "",
"flow": "implicit",
"type": "oauth2",
"x-google-issuer": "https://accounts.google.com"
}
},
"security": [
{
"api_key": [],
"google_id_token": []
}
]
This deploys OK, but I am stumped on what to do from the client side to make use of the service account JSON.
In Go, I use the following, based on my understanding of the oauth2/google documentation:
package main
import (
"context"
"fmt"
"io/ioutil"
"log"
"os"
"golang.org/x/oauth2/google"
)
func main() {
ctx := context.Background()
apiKey := os.Getenv("API_KEY")
basePath := "https://SERVICE_NAME-dot-PROJECT_NAME.appspot.com" // changed for privacy
creds, _ := google.FindDefaultCredentials(ctx)
jwt, _ := google.JWTConfigFromJSON(creds.JSON, basePath)
client := jwt.Client(ctx)
res, _ := client.Get(fmt.Sprintf("%s/REQUEST_PATH?key=%s", basePath, apiKey)) // changed for privacy
body, _ := ioutil.ReadAll(res.Body)
log.Printf("### http status: %d %s", res.StatusCode, res.Status)
log.Printf("### http body: %s", body)
}
I've removed error-checking for this code paste for brevity, there are no errors when I run this. The result of this being:
2017/10/16 07:32:38 ### http status: 401 401 Unauthorized
2017/10/16 07:32:38 ### http body: {
"code": 16,
"message": "JWT validation failed: Missing or invalid credentials",
"details": [
{
"#type": "type.googleapis.com/google.rpc.DebugInfo",
"stackEntries": [],
"detail": "auth"
}
]
}
Upon inspection of the internals of what's happening, such as calling jwt.TokenSource(ctx).Token(), I see that Google is returning an id_token, but no access_token (spotted by adding some debug logs to the jwt package):
2017/10/16 07:38:47 ### oauth2/jwt -> tokenURL: https://accounts.google.com/o/oauth2/token, form: url.Values{"grant_type":[]string{"urn:ietf:params:oauth:grant-type:jwt-bearer"}, "assertion":[]string{"...cut..."}}
2017/10/16 07:38:47 ### oauth2/jwt <- response: {
"id_token" : "...cut..."
}
2017/10/16 07:38:47 ### oauth2/jwt <- token: &oauth2.Token{AccessToken:"", TokenType:"", RefreshToken:"", Expiry:time.Time{wall:0x0, ext:63643740079, loc:(*time.Location)(0x1424160)}, raw:map[string]interface {}{"id_token":"...cut..."}}
So when the oauth2 HTTP transport tries to add the auth header -- which is tied to the AccessToken field above -- it results in a string like Authorization: Bearer, thus the result fails.
I feel like I'm missing a step here which isn't obvious to me in the documentation.
Thanks for your time.
It would be expected for the oauth client to return id_token instead of access_token. You're using JWTConfigFromJSON, not ConfigFromJSON, which is correct. I think the second argument to JWTConfigFromJSON is incorrect, and you should be specifying scopes instead. In this case, try "email" instead of basePath.

Resources