I have customer http service:
s := &http.Server{
Addr: config.Port,
Handler: Controller.Log(http.DefaultServeMux),
ReadTimeout: 3 * time.Second,
WriteTimeout: 3 * time.Second,
}
http.HandleFunc("/exapmle/router/", exampleFunc)
err := s.ListenAndServe()
if err != nil {
log.Critical(err)
os.Exit(1)
}
It's does't work:
go tool pprof http://localhost:8201/debug/pprof/profile
return error:
Failed to fetch http://localhost:8201/debug/pprof/profile?seconds=30
Thanks.
edit:
I think the problem is i rewrite default http server, net/http/pprof package inject http handler:
func init() {
http.Handle("/debug/pprof/", http.HandlerFunc(Index))
http.Handle("/debug/pprof/cmdline", http.HandlerFunc(Cmdline))
http.Handle("/debug/pprof/profile", http.HandlerFunc(Profile))
http.Handle("/debug/pprof/symbol", http.HandlerFunc(Symbol))
}
the handler does not work in my code.
You set "WriteTimeout" less then profile write time.
on pprof.StartCPUProfile() execute, It already start write, see:
http://golang.org/src/pkg/runtime/pprof/pprof.go#L565
http://golang.org/src/pkg/runtime/pprof/pprof.go#L594
So http.WriteTimeout must be greater than profile write time.
Sorry for my poor English.
Related
I just recently started coding in go and thus I am not so skilled yet.
I have a use case where I want to implement a reverse proxy as a gin route.
So my route looks like this:
server.router.POST("/console", server.proxyConsoleUrl)
And my handler function something like this:
func (server *Server) proxyConsoleUrl(ctx *gin.Context) {
director := func(req *http.Request) {
r := ctx.Request
// This is not working, scheme wss is not supported
req.URL.Scheme = "wss"
req.URL.Host = "192.168.******:8006"
// The path which gets proxied should be overriden
req.URL.RawPath = "/api2/json/nodes/something/qemu/123/vncwebsocket?port=5900&vncticket=something"
req.Header["my-header"] = []string{r.Header.Get("my-header")}
// Golang camelcases headers
delete(req.Header, "My-Header")
// This header has to be added to every request which gets proxied
req.Header["Authorization"] = []string{"MYCUSTOMHEADER"}
}
proxy := &httputil.ReverseProxy{Director: director, Transport: &http.Transport{
Proxy: http.ProxyFromEnvironment,
Dial: (&net.Dialer{
Timeout: 30 * time.Second,
KeepAlive: 30 * time.Second,
}).Dial,
TLSHandshakeTimeout: 10 * time.Second,
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}}
proxy.ServeHTTP(ctx.Writer, ctx.Request)
}
So my first problem is, that httputil.ReverseProxy doesn't support web socket as far as I know and noticed when running my code:
httputil: unsupported protocol scheme "wss"
The Second problem is, that I want to override the backend url as well as add custom headers which are added by the proxy.
Maybe someone has an idea hot to implement this, if it's even possible. -Thanks in advance
WebSocket support was added to httputil.ReverseProxy in Go version 1.12.
Use the result of url.Parse("https://192.168.******:8006/api2/json/nodes/something/qemu/123/vncwebsocket?port=5900&vncticket=something") to set the target URL. This fixes the following issues:
The WebSocket protocol uses "http" or "https" on the wire, not "wss".
RawPath is ignored when RawPath is a not a valid escaping of Path. See EscapedPath for the details. Because the RawPath in the question includes a query string, it will never be a valid escaping of Path. The client's path is alway used as is. Ditto for the client's query string.
Create the proxy once and reuse it. The important point is to create an reuse a single transport per the Transport documentation. Reusing the proxy accomplishes that goal.
func createProxy() *httputil.ReverseProxy {
target, _ := url.Parse("https://192.168.******:8006/api2/json/nodes/something/qemu/123/vncwebsocket?port=5900&vncticket=something")
director := func(req *http.Request) {
req.URL = target
req.Header["my-header"] = []string{req.Header.Get("my-header")}
delete(req.Header, "My-Header")
req.Header["Authorization"] = []string{"MYCUSTOMHEADER"}
}
return &httputil.ReverseProxy{Director: director, Transport: &http.Transport{
Proxy: http.ProxyFromEnvironment,
Dial: (&net.Dialer{
Timeout: 30 * time.Second,
KeepAlive: 30 * time.Second,
}).Dial,
TLSHandshakeTimeout: 10 * time.Second,
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
}}
}
var proxy = createProxy()
func (server *Server) proxyConsoleUrl(ctx *gin.Context) {
proxy.ServeHTTP(ctx.Writer, ctx.Request)
}
I use go-diameter as client trying connecting to a server. I follow the example on the repo but found out there's some error happened in connecting. Here is my sample code:
cfg := &sm.Settings{
OriginHost: datatype.DiameterIdentity(originHost),
OriginRealm: datatype.DiameterIdentity(originRealm),
VendorID: diameterVendorID,
ProductName: productName,
FirmwareRevision: firmwareRevision,
HostIPAddresses: []datatype.Address{
datatype.Address(net.ParseIP(viper.GetString("my.ip"))),
},
}
// Create the state machine (it's a diam.ServeMux) and client.
mux := sm.New(cfg)
// CER -> for sending
cli := &sm.Client{
Dict: dict.Default,
Handler: mux,
MaxRetransmits: 3,
RetransmitInterval: time.Second,
EnableWatchdog: true,
WatchdogInterval: 5 * time.Second,
AuthApplicationID: []*diam.AVP{
// Advertise support for credit control application
diam.NewAVP(avp.AuthApplicationID, avp.Mbit, 0, datatype.Unsigned32(diameterAuthAppID)), // RFC 4006
},
}
mux.Handle(diam.CCA, handleCCA())
mux.Handle(diam.DPA, handleDPA())
// Print error reports.
go printErrors(ctx, mux.ErrorReports())
println("diameter connect....")
connect := func() (diam.Conn, error) {
/*I test these two ways, both return same error*/
return cli.DialTimeout(addr, 5*time.Second)
//return dial(cli, addr, "", "","tcp", false)
}
c, err := connect()
if err != nil {
println(err)
}
The code is just like the sample client code on go-diameter repo, but I will got errors as:
diameter connect....
diameter error on x.x.x.x:3868: read tcp x.x.x.x:60766-\u003ex.x.x.x:3868: use of closed network connection
Appreciate if there's any suggestions. Thanks.
Answered by my self. I found the error is happened at network timeout. After I extend the timeout, the connection is OK.
I'm trying to use Go to connect to my RediSearch (Redis module) instance.
Combing through the documentation and Go code that generates the client, I don't see how to pass authentication options to the Redigo client within... Is this an oversight on my part or are we just expected to use RediSearch without securing the database?
I've also tried the following when generating a RediSearch client. Each gives errors regarding the format of the URL:
redisearch.NewClient("mypasswordhere#10.10.10.50:6379", index)
redisearch.NewClient("redis://mypasswordhere#10.10.10.50:6379", index)
Within pool.go, in the NewSingleHostPool function that returns a client, it seems like it should pass DialOptions into the Dial Redigo function instead of nil, and have those options be passed into the RediSearch NewClient function...
Call NewClientWithPool with a pool dial function of your choice.
p := &redis.Pool{
Dial: func() (*Conn, error) {
return redis.DialURL("redis://mypasswordhere#10.10.10.50:6379")
},
MaxIdle: 3, // adjust to taste
IdleTimeout: 240 * time.Second, // adjust to taste
}
c := redissearch.NewClientFromPool(p, index)
A nice way to do this is like so, with built-in constructs for password auth by redigo:
func newPool(server string, password string) *redis.Pool {
return &redis.Pool{
MaxIdle: 3,
IdleTimeout: 240 * time.Second,
Dial: func() (redis.Conn, error) {
return redis.Dial("tcp",
server,
redis.DialPassword(password))
},
TestOnBorrow: func(c redis.Conn, t time.Time) error {
_, err := c.Do("PING")
return err
},
}
}
Your server should look like this: redis-1344.333.us-east-1-mz.ec2.com:17885 and the password is plain text.
I'm trying to implement server-side timeouts for my service. If the request takes longer than X seconds, the server should return 503 Service Unavailable.
I know that this can easily be accomplished by wrapping all of my endpoints in http.TimeoutHandler, but I'm confused why this isn't being done automatically by the Timeout fields of http.Server
Here is a trivial example that I am using for testing. If I cURL or POSTman this server, it hangs forever, rather than the 5 seconds I expect.
package main
import (
"net/http"
"time"
)
func main() {
mux := http.NewServeMux()
mux.HandleFunc("/timeouttest", func(_ http.ResponseWriter, _ *http.Request) {
// busy infinite loop
// for { _ = struct {}{}}
// non-busy infinite loop
select {}
})
srv := &http.Server{
Addr: "localhost:5000",
Handler: mux,
ReadTimeout: 5 * time.Second,
ReadHeaderTimeout: 5 * time.Second,
WriteTimeout: 5 * time.Second,
IdleTimeout: 90 * time.Second,
}
srv.ListenAndServe()
}
EDIT: forgot to link some Cloudflare articles that I have been using as inspiration.
complete guide to golang http timeouts
so you want to expose go on the internet
The answer is to use http.TimeoutHandler. I misunderstood the purpose of the http.Server Timeout fields.
I dont know if it's possible as the std lib does not state anything about the current address being used:
http://golang.org/pkg/net/http/
resp, err := http.Get("http://example.com/")
if err != nil {
// handle error
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
What I'm trying to do is set the source address for that http request, why? because I don't want to use my primary ip address for that kind of stuff...
You can set a custom Dialer in the Client's Transport.
// Create a transport like http.DefaultTransport, but with a specified localAddr
transport := &http.Transport{
Proxy: http.ProxyFromEnvironment,
DialContext: (&net.Dialer{
Timeout: 30 * time.Second,
KeepAlive: 30 * time.Second,
LocalAddr: localAddr,
DualStack: true,
}).DialContext,
MaxIdleConns: 100,
IdleConnTimeout: 90 * time.Second,
TLSHandshakeTimeout: 10 * time.Second,
ExpectContinueTimeout: 1 * time.Second,
}
client := &http.Client{
Transport: transport,
}
Setting up a custom dialer with the specify IP works sometimes, but it is weird since sometimes it failed to work, while curl works normally.
After checking the source code of curl, I figured out why curl is able to specify what network interface to be used: SO_BINDTODEVICE. Unfortunately this is a Linux thing.
So compared to JimB's answer, my method has:
Pro: Behaves as stable as curl, which is irrelevant from routing tables
Con: Only support Linux. So you probably want to learn more about the build tags to write platform-specific code.
dialer := &net.Dialer{
Control: func(network, address string, conn syscall.RawConn) error {
var operr error
if err := conn.Control(func(fd uintptr) {
operr = unix.BindToDevice(int(fd), forceNetworkInterface)
}); err != nil {
return err
}
return operr
},
}
client = http.Client{
Transport = &http.Transport{
DialContext: dialer.DialContext,
},
}
In addition, curl performs SO_BINDTODEVICE which behaves like the above code. And for non-Linux platforms, or when SO_BINDTODEVICE fails, curl sets the local IP address just as JimB's answer does. So you can first try my code and then use JimB's answer as a fallback.
See the source code of curl for details.