Zap logger print both to console and to log file - go

I have integrated Zap with my go application, we have logs getting printed in two log files and i am also using Lumberjack for log rotation. But i am trying to display the logs in console as well, but no luck for this case.
Following is my code in logger.go
var (
Logger *zap.Logger
N2n *zap.Logger
)
type WriteSyncer struct {
io.Writer
}
func (ws WriteSyncer) Sync() error {
return nil
}
func InitLogging(mode string) {
var cfg zap.Config
var logName = "abc.log"
var slogName = "n2n.log"
if mode == "production" {
cfg = zap.NewProductionConfig()
cfg.DisableCaller = true
} else {
cfg = zap.NewDevelopmentConfig()
cfg.EncoderConfig.LevelKey = "level"
cfg.EncoderConfig.NameKey = "name"
cfg.EncoderConfig.MessageKey = "msg"
cfg.EncoderConfig.CallerKey = "caller"
cfg.EncoderConfig.StacktraceKey = "stacktrace"
}
cfg.Encoding = "json"
cfg.EncoderConfig.TimeKey = "timestamp"
cfg.EncoderConfig.EncodeTime = zapcore.ISO8601TimeEncoder
cfg.OutputPaths = []string{logName}
sw := getWriteSyncer(logName)
swSugar := getWriteSyncer(slogName)
l, err := cfg.Build(SetOutput(sw, cfg))
if err != nil {
panic(err)
}
defer l.Sync()
ls, err := cfg.Build(SetOutput(swSugar, cfg))
if err != nil {
panic(err)
}
defer ls.Sync()
Logger = l
N2n = ls
}
// SetOutput replaces existing Core with new, that writes to passed WriteSyncer.
func SetOutput(ws zapcore.WriteSyncer, conf zap.Config) zap.Option {
var enc zapcore.Encoder
switch conf.Encoding {
case "json":
enc = zapcore.NewJSONEncoder(conf.EncoderConfig)
case "console":
enc = zapcore.NewConsoleEncoder(conf.EncoderConfig)
default:
panic("unknown encoding")
}
return zap.WrapCore(func(core zapcore.Core) zapcore.Core {
return zapcore.NewCore(enc, ws, conf.Level)
})
}
func getWriteSyncer(logName string) zapcore.WriteSyncer {
var ioWriter = &lumberjack.Logger{
Filename: logName,
MaxSize: 10, // MB
MaxBackups: 3, // number of backups
MaxAge: 28, //days
LocalTime: true,
Compress: false, // disabled by default
}
var sw = WriteSyncer{
ioWriter,
}
return sw
}
I have tried appending the output paths but it is not working.

Since this is one of the first things that appears after looking for this on Google, here's a simple example of how to make logs show on the Console and a log file, which can be any io.Writer as the one used by lumberjack:
func logInit(d bool, f *os.File) *zap.SugaredLogger {
pe := zap.NewProductionEncoderConfig()
fileEncoder := zapcore.NewJSONEncoder(pe)
pe.EncodeTime = zapcore.ISO8601TimeEncoder # The encoder can be customized for each output
consoleEncoder := zapcore.NewConsoleEncoder(pe)
level := zap.InfoLevel
if d {
level = zap.DebugLevel
}
core := zapcore.NewTee(
zapcore.NewCore(fileEncoder, zapcore.AddSync(f), level),
zapcore.NewCore(consoleEncoder, zapcore.AddSync(os.Stdout), level),
)
l := zap.New(core) # Creating the logger
return l.Sugar()
}
I used the default ProductionEncoderConfig but it could be a custom one like the one on OP's code.

Figured out that zapcore has zapcore.NewMultiWriteSyncer which can write the logs in file and also on console using zapcore.addSync(os.stdout).
For example:
swSugar := zapcore.NewMultiWriteSyncer(
zapcore.AddSync(os.Stdout),
getWriteSyncer(logfileName),
)

You can also use zap.CombineWriteSyncers:
CombineWriteSyncers is a utility that combines multiple WriteSyncers into a single, locked WriteSyncer. If no inputs are supplied, it returns a no-op WriteSyncer.
It's provided purely as a convenience; the result is no different from using zapcore.NewMultiWriteSyncer and zapcore.Lock individually.
syncer := zap.CombineWriteSyncers(os.Stdout, getWriteSyncer(logfileName))
core := zapcore.NewCore(enc, syncer, zap.NewAtomicLevelAt(zap.InfoLevel))
zap.New(core)

Related

Parsing prometheus metrics from file and updating counters

I've a go application that gets run periodically by a batch. Each run, it should read some prometheus metrics from a file, run its logic, update a success/fail counter, and write metrics back out to a file.
From looking at How to parse Prometheus data as well as the godocs for prometheus, I'm able to read in the file, but I don't know how to update app_processed_total with the value returned by expfmt.ExtractSamples().
This is what I've done so far. Could someone please tell me how should I proceed from here? How can I typecast the Vector I got into a CounterVec?
package main
import (
"fmt"
"net/http"
"strings"
"time"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promhttp"
dto "github.com/prometheus/client_model/go"
"github.com/prometheus/common/expfmt"
"github.com/prometheus/common/model"
)
var (
fileOnDisk = prometheus.NewRegistry()
processedTotal = prometheus.NewCounterVec(prometheus.CounterOpts{
Name: "app_processed_total",
Help: "Number of times ran",
}, []string{"status"})
)
func doInit() {
prometheus.MustRegister(processedTotal)
}
func recordMetrics() {
go func() {
for {
processedTotal.With(prometheus.Labels{"status": "ok"}).Inc()
time.Sleep(5 * time.Second)
}
}()
}
func readExistingMetrics() {
var parser expfmt.TextParser
text := `
# HELP app_processed_total Number of times ran
# TYPE app_processed_total counter
app_processed_total{status="ok"} 300
`
parseText := func() ([]*dto.MetricFamily, error) {
parsed, err := parser.TextToMetricFamilies(strings.NewReader(text))
if err != nil {
return nil, err
}
var result []*dto.MetricFamily
for _, mf := range parsed {
result = append(result, mf)
}
return result, nil
}
gatherers := prometheus.Gatherers{
fileOnDisk,
prometheus.GathererFunc(parseText),
}
gathering, err := gatherers.Gather()
if err != nil {
fmt.Println(err)
}
fmt.Println("gathering: ", gathering)
for _, g := range gathering {
vector, err := expfmt.ExtractSamples(&expfmt.DecodeOptions{
Timestamp: model.Now(),
}, g)
fmt.Println("vector: ", vector)
if err != nil {
fmt.Println(err)
}
// How can I update processedTotal with this new value?
}
}
func main() {
doInit()
readExistingMetrics()
recordMetrics()
http.Handle("/metrics", promhttp.Handler())
http.ListenAndServe("localhost:2112", nil)
}
I believe you would need to use processedTotal.WithLabelValues("ok").Inc() or something similar to that.
The more complete example is here
func ExampleCounterVec() {
httpReqs := prometheus.NewCounterVec(
prometheus.CounterOpts{
Name: "http_requests_total",
Help: "How many HTTP requests processed, partitioned by status code and HTTP method.",
},
[]string{"code", "method"},
)
prometheus.MustRegister(httpReqs)
httpReqs.WithLabelValues("404", "POST").Add(42)
// If you have to access the same set of labels very frequently, it
// might be good to retrieve the metric only once and keep a handle to
// it. But beware of deletion of that metric, see below!
m := httpReqs.WithLabelValues("200", "GET")
for i := 0; i < 1000000; i++ {
m.Inc()
}
// Delete a metric from the vector. If you have previously kept a handle
// to that metric (as above), future updates via that handle will go
// unseen (even if you re-create a metric with the same label set
// later).
httpReqs.DeleteLabelValues("200", "GET")
// Same thing with the more verbose Labels syntax.
httpReqs.Delete(prometheus.Labels{"method": "GET", "code": "200"})
}
This is taken from the Promethus examples on Github
To use the value of vector you can do the following:
vectorFloat, err := strconv.ParseFloat(vector[0].Value.String(), 64)
if err != nil {
panic(err)
}
processedTotal.WithLabelValues("ok").Add(vectorFloat)
This is assuming you will only ever get a single vector value in your response. The value of the vector is stored as a string but you can convert it to a float with the strconv.ParseFloat method.

zap logging with 1) customized config and 2) lumberjack

I'm trying to build a customized zap logger with 1) customized *zap.Config and 2) lumberjack, but can't find proper example to apply both configurations.
Since config.Build does not accept WriteSync as an input. Do you know how to achieve this?
func genBaseLoggerZap() Logger {
ex, err := os.Executable()
if err != nil {
Fatalf("Failed to get os.Executable, err: %v", err)
}
zlManager.outputPath = path.Join(filepath.Dir(ex), zlManager.outputPath)
// Want to add sync here..
zapcore.AddSync(&lumberjack.Logger{
Filename: zlManager.outputPath + "123",
MaxSize: 500,
MaxBackups: 10,
MaxAge: 28,
})
return genLoggerZap(BaseLogger, genDefaultConfig())
}
// genLoggerZap creates a zapLogger with given ModuleID and Config.
func genLoggerZap(mi ModuleID, cfg *zap.Config) Logger {
logger, err := cfg.Build()
if err != nil {
Fatalf("Failed to generate zap logger, err: %v", err)
}
newLogger := &zapLogger{mi, cfg, logger.Sugar()}
newLogger.register()
return newLogger
}
You can add custom log destinations using the zap.RegisterSink function and the Config.OutputPaths field. RegisterSink maps URL schemes to Sink constructors, and OutputPaths configures log destinations (encoded as URLs).
Conveniently, *lumberjack.Logger implements almost all of the zap.Sink interface already. Only the Sync method is missing, which can be easily added with a thin wrapper type.
package main
import (
"net/url"
"go.uber.org/zap"
lumberjack "gopkg.in/natefinch/lumberjack.v2"
)
type lumberjackSink struct {
*lumberjack.Logger
}
// Sync implements zap.Sink. The remaining methods are implemented
// by the embedded *lumberjack.Logger.
func (lumberjackSink) Sync() error { return nil }
func main() {
zap.RegisterSink("lumberjack", func(u *url.URL) (zap.Sink, error) {
return lumberjackSink{
Logger: &lumberjack.Logger{
Filename: u.Opaque,
// Use query parameters or hardcoded values for remaining
// fields.
},
}, nil
})
config := zap.NewProductionConfig()
// Add a URL with the "lumberjack" scheme.
config.OutputPaths = append(config.OutputPaths, "lumberjack:foo.log")
log, _ := config.Build()
log.Info("test", zap.String("foo", "bar"))
}

What's the idiomatic way of parsing dynamic YAML in Go?

I have some code for handling a YAML config file that's getting a little out-of-control w/ type assertions and I feel like there must be a better way to do this.
Here's the relevant snippet from my config file:
plugins:
taxii20:
default: default
api_roots:
default:
auth:
- ldap
- mutualtls
collections:
all:
selector: g.V().Save("<type>").Save("<created>").All()
selector_query_lang: gizmo
And here's my parsing code:
func parseTaxiiConfig() {
plg.ConfigMutex.Lock()
taxiiConfig := plg.ConfigData.Plugins["taxii20"].(map[interface{}]interface{})
ConfigData = &Config{}
if taxiiConfig["default"] != nil {
ConfigData.DefaultRoot = taxiiConfig["default"].(string)
}
if taxiiConfig["api_roots"] != nil {
ConfigData.APIRoots = make([]model.APIRoot, 0)
iroots := taxiiConfig["api_roots"].(map[interface{}]interface{})
for iname, iroot := range iroots {
root := model.APIRoot{Name: iname.(string)}
authMethods := iroot.(map[interface{}]interface{})["auth"].([]interface{})
root.AuthMethods = make([]string, 0)
for _, method := range authMethods {
root.AuthMethods = append(root.AuthMethods, method.(string))
}
collections := iroot.(map[interface{}]interface{})["collections"].(map[interface{}]interface{})
root.Collections = make([]model.Collection, 0)
for icolName, icollection := range collections {
collection := model.Collection{Name: icolName.(string)}
collection.Selector = icollection.(map[interface{}]interface{})["selector"].(string)
collection.SelectorQueryLang = icollection.(map[interface{}]interface{})["selector_query_lang"].(string)
root.Collections = append(root.Collections, collection)
}
ConfigData.APIRoots = append(ConfigData.APIRoots, root)
}
}
plg.ConfigMutex.Unlock()
// debug
fmt.Println(ConfigData)
}
The code works as intended, but there's just so many type assertions here and I can't shake the feeling that I'm missing a better way.
One possible critical item of note, as the config implies, this is configuration for a Caddy-style plugin system, so the main config parser cannot know ahead of time what the shape of the plugin config will look like. It has to delegate processing of the plugin's portion of the config file to the plugin itself.
Here's what I came up with instead. Much more readable.
// Config represents TAXII 2.0 plugin structure
type Config struct {
DefaultRoot string
APIRoots []model.APIRoot
}
// Intermediate config for mapstructure
type configRaw struct {
DefaultRoot string `mapstructure:"default"`
APIRoots map[string]apiRootRaw `mapstructure:"api_roots"`
}
type apiRootRaw struct {
AuthMethods []string `mapstructure:"auth"`
Collections map[string]collectionRaw `mapstructure:"collections"`
}
type collectionRaw struct {
Selector string `mapstructure:"selector"`
SelectorQueryLang string `mapstructure:"selector_query_lang"`
}
func parseTaxiiConfig() error {
plg.ConfigMutex.Lock()
defer plg.ConfigMutex.Unlock()
taxiiConfig := plg.ConfigData.Plugins["taxii20"].(map[interface{}]interface{})
fmt.Println(taxiiConfig)
ConfigData = &Config{}
raw := &configRaw{}
err := mapstructure.Decode(taxiiConfig, raw)
if err != nil {
return err
}
ConfigData.DefaultRoot = raw.DefaultRoot
ConfigData.APIRoots = make([]model.APIRoot, 0)
for name, root := range raw.APIRoots {
apiRoot := model.APIRoot{Name: name}
apiRoot.AuthMethods = root.AuthMethods
apiRoot.Collections = make([]model.Collection, 0)
for colName, col := range root.Collections {
collection := model.Collection{Name: colName}
collection.Selector = col.Selector
collection.SelectorQueryLang = col.SelectorQueryLang
apiRoot.Collections = append(apiRoot.Collections, collection)
}
ConfigData.APIRoots = append(ConfigData.APIRoots, apiRoot)
}
return nil
}

Output of GET request different to view source

I'm trying to extract match data from whoscored.com. When I view the source on firefox, I find on line 816 a big json string with the data I want for that matchid. My goal is to eventually get this json.
In doing this, I've tried to download every page of https://www.whoscored.com/Matches/ID/Live where ID is the id of the match. I wrote a little Go program to GET request each ID up to a certain point:
package main
import (
"fmt"
"io/ioutil"
"net/http"
"os"
)
// http://www.whoscored.com/Matches/614052/Live is the match for
// Eveton vs Manchester
const match_address = "http://www.whoscored.com/Matches/"
// the max id we get
const max_id = 300
const num_workers = 10
// function that get the bytes of the match id from the website
func match_fetch(matchid int) {
url := fmt.Sprintf("%s%d/Live", match_address, matchid)
resp, err := http.Get(url)
if err != nil {
fmt.Println(err)
return
}
// if we sucessfully got a response, store the
// body in memory
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
fmt.Println(err)
return
}
// write the body to memory
pwd, _ := os.Getwd()
filepath := fmt.Sprintf("%s/match_data/%d", pwd, matchid)
err = ioutil.WriteFile(filepath, body, 0644)
if err != nil {
fmt.Println(err)
return
}
}
// data type to send to the workers,
// last means this job is the last one
// matchid is the match id to be fetched
// a matchid of -1 means don't fetch a match
type job struct {
last bool
matchid int
}
func create_worker(jobs chan job) {
for {
next_job := <-jobs
if next_job.matchid != -1 {
match_fetch(next_job.matchid)
}
if next_job.last {
return
}
}
}
func main() {
// do the eveton match as a reference
match_fetch(614052)
var joblist [num_workers]chan job
var v int
for i := 0; i < num_workers; i++ {
job_chan := make(chan job)
joblist[i] = job_chan
go create_worker(job_chan)
}
for i := 0; i < max_id; i = i + num_workers {
for index, c := range joblist {
if i+index < max_id {
v = i + index
} else {
v = -1
}
c <- job{false, v}
}
}
for _, c := range joblist {
c <- job{true, -1}
}
}
The code seems to work in that it fills a directory called match_data with html. The problem is that this html is completely different to what I get in the browser! Here is the section which I think does this: (from the body of the GET request of http://www.whoscored.com/Matches/614052/Live.
(function() {
var z="";var b="7472797B766172207868723B76617220743D6E6577204461746528292E67657454696D6528293B766172207374617475733D227374617274223B7661722074696D696E673D6E65772041727261792833293B77696E646F772E6F6E756E6C6F61643D66756E6374696F6E28297B74696D696E675B325D3D22723A222B286E6577204461746528292E67657454696D6528292D74293B646F63756D656E742E637265617465456C656D656E742822696D6722292E7372633D222F5F496E63617073756C615F5265736F757263653F4553324C555243543D363726743D373826643D222B656E636F6465555249436F6D706F6E656E74287374617475732B222028222B74696D696E672E6A6F696E28292B222922297D3B69662877696E646F772E584D4C4874747052657175657374297B7868723D6E657720584D4C48747470526571756573747D656C73657B7868723D6E657720416374697665584F626A65637428224D6963726F736F66742E584D4C4854545022297D7868722E6F6E726561647973746174656368616E67653D66756E6374696F6E28297B737769746368287868722E72656164795374617465297B6361736520303A7374617475733D6E6577204461746528292E67657454696D6528292D742B223A2072657175657374206E6F7420696E697469616C697A656420223B627265616B3B6361736520313A7374617475733D6E6577204461746528292E67657454696D6528292D742B223A2073657276657220636F6E6E656374696F6E2065737461626C6973686564223B627265616B3B6361736520323A7374617475733D6E6577204461746528292E67657454696D6528292D742B223A2072657175657374207265636569766564223B627265616B3B6361736520333A7374617475733D6E6577204461746528292E67657454696D6528292D742B223A2070726F63657373696E672072657175657374223B627265616B3B6361736520343A7374617475733D22636F6D706C657465223B74696D696E675B315D3D22633A222B286E6577204461746528292E67657454696D6528292D74293B6966287868722E7374617475733D3D323030297B706172656E742E6C6F636174696F6E2E72656C6F616428297D627265616B7D7D3B74696D696E675B305D3D22733A222B286E6577204461746528292E67657454696D6528292D74293B7868722E6F70656E2822474554222C222F5F496E63617073756C615F5265736F757263653F535748414E45444C3D313536343032333530343538313538333938362C31373139363833393832313930303534313833392C31333935303737313737393531363432383234342C3132363636222C66616C7365293B7868722E73656E64286E756C6C297D63617463682863297B7374617475732B3D6E6577204461746528292E67657454696D6528292D742B2220696E6361705F6578633A20222B633B646F63756D656E742E637265617465456C656D656E742822696D6722292E7372633D222F5F496E63617073756C615F5265736F757263653F4553324C555243543D363726743D373826643D222B656E636F6465555249436F6D706F6E656E74287374617475732B222028222B74696D696E672E6A6F696E28292B222922297D3B";for (var i=0;i<b.length;i+=2){z=z+parseInt(b.substring(i, i+2), 16)+",";}z = z.substring(0,z.length-1); eval(eval('String.fromCharCode('+z+')'));})();
The reason I think this is the case is that the javascript in the page fetches and edits the DOM to what I see on view source. How can I get golang to run the javascript? Is there are library to do this? Better still, could I directly grab the JSON from the servers?
This can be done with https://godoc.org/github.com/sourcegraph/webloop#View.EvaluateJavaScript
Read their main example https://github.com/sourcegraph/webloop
What you need is a "headless browser" in general.
In general it is better to use an Web API vs. scraping. For example, whoscored themselves use OPTA which you should be able to access directly.
http://www.jokecamp.com/blog/guide-to-football-and-soccer-data-and-apis/#opta

Go url parameters mapping

Is there a native way for inplace url parameters in native Go?
For Example, if I have a URL: http://localhost:8080/blob/123/test I want to use this URL as /blob/{id}/test.
This is not a question about finding go libraries. I am starting with the basic question, does go itself provide a basic facility to do this natively.
There is no built in simple way to do this, however, it is not hard to do.
This is how I do it, without adding a particular library. It is placed in a function so that you can invoke a simple getCode() function within your request handler.
Basically you just split the r.URL.Path into parts, and then analyse the parts.
// Extract a code from a URL. Return the default code if code
// is missing or code is not a valid number.
func getCode(r *http.Request, defaultCode int) (int, string) {
p := strings.Split(r.URL.Path, "/")
if len(p) == 1 {
return defaultCode, p[0]
} else if len(p) > 1 {
code, err := strconv.Atoi(p[0])
if err == nil {
return code, p[1]
} else {
return defaultCode, p[1]
}
} else {
return defaultCode, ""
}
}
Well, without external libraries you can't, but may I recommend two excellent ones:
httprouter - https://github.com/julienschmidt/httprouter - is extremely fast and very lightweight. It's faster than the standard library's router, and it creates 0 allocations per call, which is great in a GCed language.
Gorilla Mux - http://www.gorillatoolkit.org/pkg/mux -
Very popular, nice interface, nice community.
Example usage of httprouter:
func Hello(w http.ResponseWriter, r *http.Request, ps httprouter.Params) {
fmt.Fprintf(w, "hello, %s!\n", ps.ByName("name"))
}
func main() {
router := httprouter.New()
router.GET("/hello/:name", Hello)
log.Fatal(http.ListenAndServe(":8080", router))
}
What about trying using regex, and find a named group in your url, like playground:
package main
import (
"fmt"
"net/url"
"regexp"
)
var myExp = regexp.MustCompile(`/blob/(?P<id>\d+)/test`) // use (?P<id>[a-zA-Z]+) if the id is alphapatic
func main() {
s := "http://localhost:8080/blob/123/test"
u, err := url.Parse(s)
if err != nil {
panic(err)
}
fmt.Println(u.Path)
match := myExp.FindStringSubmatch(s) // or match := myExp.FindStringSubmatch(u.Path)
result := make(map[string]string)
for i, name := range myExp.SubexpNames() {
if i != 0 && name != "" {
result[name] = match[i]
}
}
fmt.Printf("id: %s\n", result["id"])
}
output
/blob/123/test
id: 123
Below full code to use it with url, that is receiving http://localhost:8000/hello/John/58 and returning http://localhost:8000/hello/John/58:
package main
import (
"fmt"
"net/http"
"regexp"
"strconv"
)
var helloExp = regexp.MustCompile(`/hello/(?P<name>[a-zA-Z]+)/(?P<age>\d+)`)
func hello(w http.ResponseWriter, req *http.Request) {
match := helloExp.FindStringSubmatch(req.URL.Path)
if len(match) > 0 {
result := make(map[string]string)
for i, name := range helloExp.SubexpNames() {
if i != 0 && name != "" {
result[name] = match[i]
}
}
if _, err := strconv.Atoi(result["age"]); err == nil {
fmt.Fprintf(w, "Hello, %v year old named %s!", result["age"], result["name"])
} else {
fmt.Fprintf(w, "Sorry, not accepted age!")
}
} else {
fmt.Fprintf(w, "Wrong url\n")
}
}
func main() {
http.HandleFunc("/hello/", hello)
http.ListenAndServe(":8090", nil)
}
How about writing your own url generator (extend net/url a little bit) as below.
// --- This is how does it work like --- //
url, _ := rest.NewURLGen("http", "stack.over.flow", "1234").
Pattern(foo/:foo_id/bar/:bar_id).
ParamQuery("foo_id", "abc").
ParamQuery("bar_id", "xyz").
ParamQuery("page", "1").
ParamQuery("offset", "5").
Do()
log.Printf("url: %s", url)
// url: http://stack.over.flow:1234/foo/abc/bar/xyz?page=1&offset=5
// --- Your own url generator would be like below --- //
package rest
import (
"log"
"net/url"
"strings"
"straas.io/base/errors"
"github.com/jinzhu/copier"
)
// URLGen generates request URL
type URLGen struct {
url.URL
pattern string
paramPath map[string]string
paramQuery map[string]string
}
// NewURLGen new a URLGen
func NewURLGen(scheme, host, port string) *URLGen {
h := host
if port != "" {
h += ":" + port
}
ug := URLGen{}
ug.Scheme = scheme
ug.Host = h
ug.paramPath = make(map[string]string)
ug.paramQuery = make(map[string]string)
return &ug
}
// Clone return copied self
func (u *URLGen) Clone() *URLGen {
cloned := &URLGen{}
cloned.paramPath = make(map[string]string)
cloned.paramQuery = make(map[string]string)
err := copier.Copy(cloned, u)
if err != nil {
log.Panic(err)
}
return cloned
}
// Pattern sets path pattern with placeholder (format `:<holder_name>`)
func (u *URLGen) Pattern(pattern string) *URLGen {
u.pattern = pattern
return u
}
// ParamPath builds path part of URL
func (u *URLGen) ParamPath(key, value string) *URLGen {
u.paramPath[key] = value
return u
}
// ParamQuery builds query part of URL
func (u *URLGen) ParamQuery(key, value string) *URLGen {
u.paramQuery[key] = value
return u
}
// Do returns final URL result.
// The result URL string is possible not escaped correctly.
// This is input for `gorequest`, `gorequest` will handle URL escape.
func (u *URLGen) Do() (string, error) {
err := u.buildPath()
if err != nil {
return "", err
}
u.buildQuery()
return u.String(), nil
}
func (u *URLGen) buildPath() error {
r := []string{}
p := strings.Split(u.pattern, "/")
for i := range p {
part := p[i]
if strings.Contains(part, ":") {
key := strings.TrimPrefix(p[i], ":")
if val, ok := u.paramPath[key]; ok {
r = append(r, val)
} else {
if i != len(p)-1 {
// if placeholder at the end of pattern, it could be not provided
return errors.Errorf("placeholder[%s] not provided", key)
}
}
continue
}
r = append(r, part)
}
u.Path = strings.Join(r, "/")
return nil
}
func (u *URLGen) buildQuery() {
q := u.URL.Query()
for k, v := range u.paramQuery {
q.Set(k, v)
}
u.RawQuery = q.Encode()
}
With net/http the following would trigger when calling localhost:8080/blob/123/test
http.HandleFunc("/blob/", yourHandlerFunction)
Then inside yourHandlerFunction, manually parse r.URL.Path to find 123.
Note that if you don't add a trailing / it won't work. The following would only trigger when calling localhost:8080/blob:
http.HandleFunc("/blob", yourHandlerFunction)
As of 19-Sep-22, with go version 1.19, instance of http.request URL has a method called Query, which will return a map, which is a parsed query string.
func helloHandler(res http.ResponseWriter, req *http.Request) {
// when request URL is `http://localhost:3000/?first=hello&second=world`
fmt.Println(req.URL.Query()) // outputs , map[second:[world] first:[hello]]
res.Write([]byte("Hello World Web"))
}
No way without standard library. Why you don't want to try some library? I think its not so hard to use it, just go get bla bla bla
I use Beego. Its MVC style.
how about a simple utility function ?
func withURLParams(u url.URL, param, val string) url.URL{
u.Path = strings.ReplaceAll(u.Path, param, val)
return u
}
you can use it like this:
u, err := url.Parse("http://localhost:8080/blob/:id/test")
if err != nil {
return nil, err
}
u := withURLParams(u, ":id","123")
// now u.String() is http://localhost:8080/blob/123/test
If you need a framework and you think it will be slow because it's 'bigger' than a router or net/http, then you 're wrong.
Iris is the fastest go web framework that you will ever find, so far according to all benchmarks.
Install by
go get gopkg.in/kataras/iris.v6
Django templates goes easy with iris:
import (
"gopkg.in/kataras/iris.v6"
"gopkg.in/kataras/iris.v6/adaptors/httprouter"
"gopkg.in/kataras/iris.v6/adaptors/view" // <-----
)
func main() {
app := iris.New()
app.Adapt(iris.DevLogger())
app.Adapt(httprouter.New()) // you can choose gorillamux too
app.Adapt(view.Django("./templates", ".html")) // <-----
// RESOURCE: http://127.0.0.1:8080/hi
// METHOD: "GET"
app.Get("/hi", hi)
app.Listen(":8080")
}
func hi(ctx *iris.Context){
ctx.Render("hi.html", iris.Map{"Name": "iris"})
}

Resources