I'm trying to make HTTP proxy using go net/http. I made struct type named HTTPProxy to store http.Server object and some hooks to modify Request and Response.
There're two hooks in my proxy(Request/Response hook). The hooks can be set with interface named SetRequestHook and SetResultHook. I checked value of HTTPProxy struct using fmt.Printf("%#v\n", httpProxy) before/after SetRequestHook to check the set of request hook was successful. This is the value of httpProxy before/after SetRequestHook.
webproxy.HTTPProxy{server:(*http.Server)(0xc000154000), IsStarted:false, requestHook:(webproxy.RequestHookFunc)(nil), responseHook:(webproxy.ResponseHookFunc)(nil)}
webproxy.HTTPProxy{server:(*http.Server)(0xc000154000), IsStarted:false, requestHook:(webproxy.RequestHookFunc)(0x75ce60), responseHook:(webproxy.ResponseHookFunc)(nil)}
As you can see address of request hook function stored well in httpProxy struct. But if I call another interface names Start requestHook in struct resets to nil for no reason. This is the result of fmt.Printf("%#v\n", httpProxy) in Start interface.
webproxy.HTTPProxy{server:(*http.Server)(0xc000154000), IsStarted:false, requestHook:(webproxy.RequestHookFunc)(nil), responseHook:(webproxy.ResponseHookFunc)(nil)}
I haven't reset my struct since initialize so I'm not even sure what is happening to my struct. How can I make my request hook to not reset to nil? This is my current code having this problem.
cmd/runserver/main.go
package main
import (
"log"
"net/http"
"test/pkg/webproxy"
)
var a int
func test(request *http.Request) {
log.Printf("New Request: %d\n", a)
a++
}
func main() {
a = 1
httpProxy := webproxy.NewHTTPProxy()
httpProxy.SetRequestHook(test)
httpProxy.Start("", 3000)
}
pkg/webproxy/webproxy.go
// Package webproxy provides http and https proxy with request and response hooks
package webproxy
import (
"context"
"fmt"
"log"
"net"
"net/http"
)
// RequestHookFunc typed functions provided to proxy will modify request before sent to the server.
type RequestHookFunc func(request *http.Request)
// ResponseHookFunc typed functions provided to proxy will modify response before sent to the client.
type ResponseHookFunc func(response *http.Response)
// HTTPProxy will store some items that are used in HTTP proxy and state.
type HTTPProxy struct {
server *http.Server
IsStarted bool
requestHook RequestHookFunc
responseHook ResponseHookFunc
}
// ProxyManager provide methods to manage HTTPProxy and HTTPSProxy.
type ProxyManager interface {
Start(host string, port uint16)
Stop()
GetAddr()
SetRequestHook(hook RequestHookFunc)
SetResponseHook(hook ResponseHookFunc)
}
// NewHTTPProxy creates default HTTPProxy struct.
func NewHTTPProxy() *HTTPProxy {
return &HTTPProxy{
server: new(http.Server),
IsStarted: false,
requestHook: nil,
responseHook: nil,
}
}
// Start serves HTTPProxy on host:port.
func (httpProxy HTTPProxy) Start(host string, port uint16) {
if httpProxy.IsStarted {
log.Println("[HTTP] The proxy is already running.")
return
} else if len(host) != 0 && net.ParseIP(host) == nil {
log.Fatal("[HTTP] Inserted host is not valid.")
}
addr := fmt.Sprintf("%s:%d", host, port)
fmt.Printf("%#v\n", httpProxy) // requestHook = nil. Why?? I called SetRequestHook before run Start
httpProxy.server.Handler = getHandler(httpProxy.requestHook, httpProxy.responseHook)
httpProxy.server.Addr = addr
log.Printf("[HTTP] The proxy will be served on %s:%d\n", host, port)
if err := httpProxy.server.ListenAndServe(); err != http.ErrServerClosed {
log.Fatalf("[HTTP] The proxy ListenAndServer: %v\n", err)
}
}
// Stop will stop HTTPProxy immediately.
func (httpProxy HTTPProxy) Stop() {
if !httpProxy.IsStarted {
log.Println("[HTTP] The proxy is not running.")
return
}
httpProxy.server.Shutdown(context.Background())
}
// GetAddr will return HTTPProxy's host:port.
func (httpProxy HTTPProxy) GetAddr() string {
return httpProxy.server.Addr
}
// SetRequestHook setup request hook into the proxy
func (httpProxy HTTPProxy) SetRequestHook(hook RequestHookFunc) {
if httpProxy.IsStarted {
log.Println("[HTTP] You can't set request hook while the proxy is working.")
return
}
fmt.Printf("%#v\n", httpProxy) // requestHook = nil
httpProxy.requestHook = hook
fmt.Printf("%#v\n", httpProxy) // requestHook = [hook's address]
}
// SetResponseHook setup response hook into the proxy.
func (httpProxy HTTPProxy) SetResponseHook(hook ResponseHookFunc) {
if httpProxy.IsStarted {
log.Println("[HTTP] You can't set response hook while the proxy is working.")
return
}
httpProxy.responseHook = hook
}
pkg/webproxy/proxyhandler.go
package webproxy
import (
"fmt"
"io"
"net/http"
)
type proxyHandler struct {
requestHook RequestHookFunc
responseHook ResponseHookFunc
}
func getHandler(requestHookIn RequestHookFunc, responseHookIn ResponseHookFunc) http.Handler {
fmt.Printf("%#v\n", requestHookIn)
return &proxyHandler{requestHook: requestHookIn, responseHook: responseHookIn}
}
// ServeHttp
func (handler *proxyHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
fmt.Printf("%#v\n", handler.requestHook)
if handler.requestHook != nil {
fmt.Println("Yaha")
handler.requestHook(r)
}
w.Header().Set("Content-Type", "text/plain")
w.WriteHeader(http.StatusOK)
io.WriteString(w, "This HTTP response has both headers before this text and trailers at the end.\n")
}
Related
When I run the program, the output comes like this
1
2
7
8
Server serving at 'http://localhost:3000'
On visiting the localhost:3000/ I get the 404, nothing gets printed on terminal and the server also doesn't crash.
I used fmt.Printf() in various functions to check if the control is going into the function or not and seems like the control never entered the handleRedirect() function
I tried but I'm unable to figure out the issue, I just started learning go land some time ago
package main
import (
"fmt"
"net/http"
"net/http/httputil"
"net/url"
"os"
)
type Server interface {
Address() string
IsAlive() bool
Serve(rw http.ResponseWriter, r *http.Request)
}
type SimpleServer struct {
address string
proxy *httputil.ReverseProxy
}
func newSimpleServer(address string) *SimpleServer {
serverURL, err := url.Parse(address)
handleError(err)
return &SimpleServer{
address: address,
proxy: httputil.NewSingleHostReverseProxy(serverURL),
}
}
type LoadBalancer struct {
port string
roundRobinCount int
servers []Server
}
func newLoadBalancer(port string, servers []Server) *LoadBalancer {
return &LoadBalancer{
port: port,
roundRobinCount: 0,
servers: servers,
}
}
func (s *SimpleServer) Address() string { return s.address }
func (s *SimpleServer) IsAlive() bool { return true }
func (s *SimpleServer) Serve(rw http.ResponseWriter, r *http.Request) {
fmt.Println(6)
s.proxy.ServeHTTP(rw, r)
}
func (lb *LoadBalancer) getNextAvailableServer() Server {
server := lb.servers[lb.roundRobinCount%len(lb.servers)]
for !server.IsAlive() {
lb.roundRobinCount++
server = lb.servers[lb.roundRobinCount%len(lb.servers)]
}
lb.roundRobinCount++
return server
}
func (lb *LoadBalancer) ServeProxy(rw http.ResponseWriter, r *http.Request) {
fmt.Println(4)
targetServer := lb.getNextAvailableServer()
fmt.Printf("forwarding reques to %q\n", targetServer.Address())
fmt.Println(5)
targetServer.Serve(rw, r)
}
func main() {
fmt.Println(1)
servers := []Server{
newSimpleServer("https://www.fast.com/"),
newSimpleServer("https://www.theverge.com/"),
newSimpleServer("https://www.canva.com/"),
}
fmt.Println(2)
lb := newLoadBalancer("3000", servers)
handleRedirect := func(rw http.ResponseWriter, req *http.Request) {
fmt.Println(3)
lb.ServeProxy(rw, req)
}
fmt.Println(7)
http.HandleFunc("/", handleRedirect)
fmt.Println(8)
fmt.Printf("server serving at 'http://localhost:%s'\n", lb.port)
http.ListenAndServe(":"+lb.port, nil)
}
func handleError(err error) {
if err != nil {
fmt.Printf("you've got an error fix it asap %v\n", err)
os.Exit(1)
}
}
in my case, I want to use anothor param instead of callback
my url: http://example.com?id=1&cb=callback1
but I found this in the source code:
// JSONP serializes the given struct as JSON into the response body.
// It adds padding to response body to request data from a server residing in a different domain than the client.
// It also sets the Content-Type as "application/javascript".
func (c *Context) JSONP(code int, obj interface{}) {
callback := c.DefaultQuery("callback", "")
if callback == "" {
c.Render(code, render.JSON{Data: obj})
return
}
c.Render(code, render.JsonpJSON{Callback: callback, Data: obj})
}
how can I use the param cb instead of callback
You can use middleware for gin. Modifying the query before it is parsed.
package main
import (
"fmt"
"github.com/gin-gonic/gin"
"net/url"
)
func main() {
r := gin.Default()
r.Use(func(context *gin.Context) {
req := context.Request
urlCopy, _ := req.URL.Parse(req.RequestURI)
if cb := urlCopy.Query().Get("cb"); cb != "" {
req.URL.RawQuery += fmt.Sprintf("&callback=%s", url.QueryEscape(cb))
}
})
r.GET("/ping", func(c *gin.Context) {
c.JSONP(400, 1)
})
r.Run() // listen and serve on 0.0.0.0:8080 (for windows "localhost:8080")
}
I've some web-application server using go http and I want that each request will have context with uuid, for this I can use http request context https://golang.org/pkg/net/http/#Request.Context
we are using logrus and we initiate it in one file and use the logger instance in other files.
what I need is to print request ID in all the logs but not to add new paremeters to each log print, I want do to it once in each http request (pass the req-id) and all the logs print will have it without doing anything with it
e.g. if the id=123 so log.info("foo") will print
// id=123 foo
I've tried with the following but not sure it's the right way, please advice.
package main
import (
"context"
"errors"
log "github.com/sirupsen/logrus"
)
type someContextKey string
var (
keyA = someContextKey("a")
keyB = someContextKey("b")
)
func main() {
ctx := context.Background()
ctx = context.WithValue(ctx, keyA, "foo")
ctx = context.WithValue(ctx, keyB, "bar")
logger(ctx, nil).Info("did nothing")
err := errors.New("an error")
logger(ctx, err).Fatal("unrecoverable error")
}
func logger(ctx context.Context, err error) *log.Entry {
entry := log.WithField("component", "main")
entry = entry.WithField("ReqID", "myRequestID")
return entry
}
https://play.golang.org/p/oCW09UhTjZ5
Every time you call the logger function you are creating a new *log.Entry and writing the request ID to it again. From your question it sounded like you do not want that.
func main() {
ctx := context.Background()
ctx = context.WithValue(ctx, keyA, "foo")
ctx = context.WithValue(ctx, keyB, "bar")
lg := logger(ctx)
lg.Info("did nothing")
err := errors.New("an error")
lg.WithError(err).Fatal("unrecoverable error")
}
func logger(ctx context.Context) *log.Entry {
entry := log.WithField("component", "main")
entry = entry.WithField("ReqID", "myRequestID")
return entry
}
The downside of this is that you will have to pass the lg variable to every function this request calls and which should also log the request ID.
What we did at our company is create a thin layer around logrus that has an additional method WithRequestCtx so we could pass in the request context and it would extract the request ID itself (which we had written to the context in a middleware). If no request ID was present nothing was added to the log entry. This however did add the request ID to every log entry again as your sample code also did.
Note: our thin layer around logrus had a lot more functionality and default settings to justify the extra effort. In the long run this turned out very helpful to have one place to be able to adjust logging for all our services.
Note2: meanwhile we are in the process of replacing logrus with zerolog to be more lightweight.
Late answer but all you need to do is just call logrus.WithContext(/* your *http.Request.Context() goes here*/).... in your application and logrus will automatically add "id":"SOME-UUID" to each logs. Design is flexible for extracting more key-value from request context if you wanted to.
initialise logger
package main
import (
"path/to/logger"
"path/to/request"
)
func main() {
err := logger.Setup(logger.Config{
ContextFields: map[string]interface{}{
string(request.CtxIDKey): request.CtxIDKey,
}
})
if err != nil {
// ...
}
}
logger
package logger
import (
"github.com/sirupsen/logrus"
)
type Config struct {
Level string
ContextFields map[string]interface{}
}
func Setup(config Config) error {
lev, err := logrus.ParseLevel(config.Level)
if err != nil {
return err
}
logrus.SetLevel(lev)
return nil
}
func (c Config) Fire(e *logrus.Entry) error {
for k, v := range c.StaticFields {
e.Data[k] = v
}
if e.Context != nil {
for k, v := range c.ContextFields {
if e.Context.Value(v) != nil {
e.Data[k] = e.Context.Value(v).(string)
}
}
}
return nil
}
request
package request
import (
"context"
"net/http"
"github.com/google/uuid"
)
type ctxID string
const CtxIDKey = ctxID("id")
func ID(h http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
h.ServeHTTP(w, r.WithContext(context.WithValue(r.Context(), CtxIDKey, uuid.New().String())))
})
}
I'm perhaps abusing promhttp.Handler() to realise the use case for my microservice to tell me the:
version
if it has database connectivity
If there is a better way to monitor my microservices, do let me know!
I'm not sure how to structure the handle in such a way that when /metrics are called, the db.Ping() is re-evaluated.
https://s.natalian.org/2019-06-02/msping.mp4
package main
import (
"log"
"net/http"
"os"
_ "github.com/go-sql-driver/mysql"
"github.com/gorilla/mux"
"github.com/jmoiron/sqlx"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promhttp"
)
const version = "0.0.1"
type App struct {
Router *mux.Router
DB *sqlx.DB
}
func main() {
a := App{}
a.Initialize()
log.Fatal(http.ListenAndServe(":"+os.Getenv("PORT"), a.Router))
}
func (a *App) Initialize() {
connectionString := "root:secret#tcp(localhost:3306)/rest_api_example?multiStatements=true&sql_mode=TRADITIONAL&timeout=5s"
var err error
a.DB, err = sqlx.Open("mysql", connectionString)
if err != nil {
log.Fatal(err)
}
microservicecheck := prometheus.NewGaugeVec(
prometheus.GaugeOpts{
Name: "mscheck",
Help: "Version with DB ping check",
},
[]string{
"commit",
},
)
if a.DB.Ping() == nil {
microservicecheck.WithLabelValues(version).Set(1)
} else {
microservicecheck.WithLabelValues(version).Set(0)
}
prometheus.MustRegister(microservicecheck)
a.Router = mux.NewRouter()
a.initializeRoutes()
}
func (a *App) initializeRoutes() {
a.Router.Handle("/metrics", promhttp.Handler()).Methods("GET")
}
https://play.golang.org/p/9DdXnz77S55
You could also add a middleware hook that does a preflight routine (i.e. your ping test) before calling promhttp.Handler(). However, at collection time, I think metrics should already have been tallied; and not generated at the instance of collection. So...
Try a separate go-routine that will poll the health of the DB connection at regular intervals. This avoids any messy hooks or custom collectors:
var pingPollingFreq = 5 * time.Second // this should probably match the Prometheus scrape interval
func (a *App) Initialize() {
// ...
prometheus.MustRegister(microservicecheck)
go func() {
for {
if a.DB.Ping() == nil {
microservicecheck.WithLabelValues(version).Set(1)
} else {
microservicecheck.WithLabelValues(version).Set(0)
}
time.Sleep(pingPollingFreq)
}
}()
// ...
}
I am new to golang, am trying develop a login page with sesions. the code is building successfully but when I run in browser its saying 404 page not found.can any one help for me. Thanks in advance.
Here is my code
// main.go
package main
import (
_ "HarishSession/routers"
"github.com/astaxie/beego"
"fmt"
"net/http"
"html/template"
"strings"
"log"
"github.com/astaxie/beego/session"
"sync"
)
var globalSessions *session.Manager
var provides = make(map[string]Provider)
func sayhelloName(w http.ResponseWriter, r *http.Request) {
r.ParseForm() // parse arguments, you have to call this by yourself
fmt.Println("the information of form is",r.Form) // print form information in server side
fmt.Println("path", r.URL.Path)
fmt.Println("scheme", r.URL.Scheme)
fmt.Println(r.Form["url_long"])
for k, v := range r.Form {
fmt.Println("key:", k)
fmt.Println("val:", strings.Join(v, ""))
}
fmt.Fprintf(w, "Hello astaxie!") // send data to client side
}
type Manager struct {
cookieName string //private cookiename
lock sync.Mutex // protects session
provider Provider
maxlifetime int64
}
type Provider interface {
SessionInit(sid string) (Session, error)
SessionRead(sid string) (Session, error)
SessionDestroy(sid string) error
SessionGC(maxLifeTime int64)
}
type Session interface {
Set(key, value interface{}) error //set session value
Get(key interface{}) interface{} //get session value
Delete(key interface{}) error //delete session value
SessionID() string //back current sessionID
}
func NewManager(provideName, cookieName string, maxlifetime int64) (*Manager, error) {
provider, ok := provides[provideName]
if !ok {
return nil, fmt.Errorf("session: unknown provide %q (forgotten import?)", provideName)
}
return &Manager{provider: provider, cookieName: cookieName, maxlifetime: maxlifetime}, nil
}
func login(w http.ResponseWriter, r *http.Request) {
sess := globalSessions.SessionStart(w,r)
r.ParseForm()
fmt.Println("method:", r.Method)
if r.Method == "GET" {
t, _ := template.ParseFiles("login.tpl")
w.Header().Set("Content-Type", "text/html")
t.Execute(w,sess.Get("username"))
} else {
//logic part of log in
fmt.Println("username:",r.Form["username"])
fmt.Println("password:",r.Form["password"])
http.Redirect(w,r,"/",302)
}
}
func main() {
var globalSessions *session.Manager
http.HandleFunc("/", sayhelloName)
http.HandleFunc("/login", login)
err := http.ListenAndServe(":8080", nil) // set listen port
if err != nil {
log.Fatal("ListenAndServe the error is: ", err)
}
fmt.Println("hello")
beego.Run()
fmt.Println(globalSessions)
}
//router.go
package routers
import (
"HarishSession/controllers"
"github.com/astaxie/beego"
)
func init() {
beego.Router("/", &controllers.MainController{})
beego.Router("/login", &controllers.MainController{})
}
//default.go
package controllers
import (
"github.com/astaxie/beego"
)
type MainController struct {
beego.Controller
}
func (this *MainController) Get() {
this.Data["Website"] = "beego.me"
this.Data["Email"] = "astaxie#gmail.com"
this.TplNames = "index.tpl"
this.TplNames="login.tpl"
}
You have two variables at different scopes, each called globalSessions. One is in your definition in main.go, which is defined at global scope, and another is defined in the main function, and is defined as a local variable to main. These are separate variables. Your code is making this mistake of conflating them.
You can see this by paying closer attention to the stack trace entry:
github.com/astaxie/beego/session.(*Manager).SessionStart(0x0, 0x151e78, 0xc08212 0000, 0xc082021ad0, 0x0, 0x0)
as this points to globalSessions being uninitialized due to being nil. After that, troubleshooting is a direct matter of looking at the program to see what touches globalSessions.
Note that you should include the stack trace as part of your question. Don't just add it as a comment. It's critical to include this information: otherwise we would not have been able to easily trace the problem. Please improve the quality of your questions to make it easier for people to help you.
Also, you may want to take a serious look at go vet, which is a tool that helps to catch problems like this.
As this is the one line you used in code :
t, _ := template.ParseFiles("login.tpl")
So what you need to check is whether the file login.tpl is at the correct location, where it must be, or not. If not then correct the reference of it and also check same for the other references.
This helped me.