How to Create http NewServeMux and run with go routine? - go

I have an issue about http.NewServeMux, I've tried to run with go routine but it's doesn't work.
package main
import (
"context"
"fmt"
"log"
"net/http"
"os"
"os/signal"
"syscall"
"time"
)
func run() (s *http.Server) {
mux := http.NewServeMux()
mux.HandleFunc("/", func(rw http.ResponseWriter, r *http.Request) {
rw.WriteHeader(http.StatusOK)
fmt.Fprint(rw, "under construction")
})
s = &http.Server{
Addr: ":8080",
ReadTimeout: 10 * time.Second,
WriteTimeout: 10 * time.Second,
MaxHeaderBytes: 1 << 20,
}
go func() {
if err := s.ListenAndServe(); err != nil && err != http.ErrServerClosed {
log.Fatalf("listen: %s\n", err)
}
}()
return
}
func main() {
s := run()
quit := make(chan os.Signal)
signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)
<-quit
log.Println("Shutting down server...")
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer func() {
cancel()
}()
if err := s.Shutdown(ctx); err != nil {
log.Fatal("Server forced to shutdown")
}
log.Println("Server exiting")
}
then run with
go run main.go
And access to http://localhost:8080/ that's always to get.
404 page not found
How I can solve this issue?

This is because you're setting up mux but never do anything with it:
// mux is never referenced after you add the default handler
mux := http.NewServeMux()
mux.HandleFunc("/", func(rw http.ResponseWriter, r *http.Request) {
rw.WriteHeader(http.StatusOK)
fmt.Fprint(rw, "under construction")
})
Change your handler to use your mux var instead of http.DefaultServeMux and everything works as expected:
s = &http.Server{
Handler: mux, // This is the important line
Addr: ":8080",
ReadTimeout: 10 * time.Second,
WriteTimeout: 10 * time.Second,
MaxHeaderBytes: 1 << 20,
}

Related

Handle OS signal in live endpoints in case of close connections

I have code that I need to catch OS signal and close the all connections to the database.
In general, below code can handle signal inside main function.
package main
import (
"fmt"
"net/http"
"os"
"os/signal"
"syscall"
"time"
"context"
"log"
"github.com/gin-gonic/gin"
)
func main() {
middleware.
router := gin.Default()
router.GET("/", func(c *gin.Context) {
c.JSON(http.StatusOK, "Welcome to the homepage!")
})
router.GET("/about", func(c *gin.Context) {
c.JSON(http.StatusOK, "This is the about page!")
})
router.GET("/contact", func(c *gin.Context) {
c.JSON(http.StatusOK, "This is the contact page!")
})
router.GET("/help", func(c *gin.Context) {
c.JSON(http.StatusOK, "This is the help page!")
})
srv := &http.Server{
Addr: ":5000",
Handler: router,
ReadHeaderTimeout: 60 * time.Second,
}
go func() {
if err := srv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
fmt.Printf("listen: %s\n", err)
}
}()
quit := make(chan os.Signal, 2)
signal.Notify(quit, syscall.SIGINT, syscall.SIGTERM)
<-quit
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
if err := srv.Shutdown(ctx); err != nil {
log.Fatal("Server forced to shutdown: ", err)
}
log.Print("Application exit")
}
Sometimes endpoints handle bulkdata and is busy.
But I want to be aware how can I handle in live requests if endpoint is busy while server receive SIGTERM.

Reverse Proxy using Go to Cloud Run Instance

I feel like I'm close to having this working but so far I"m running into an issue building a small reverse proxy in Go to a GCP Cloud Run instance. The request 'goes through' but the response from the request is the default GCP Cloud Run 404. It appears when making the request back to Cloud Run the Host header is being ignored and therefore the request is not being routed correction.
What might I be missing here?
package main
import (
"log"
"net/http"
"net/http/httputil"
"net/url"
)
const apiUrl = "MY_CLOUD_RUN.a.run.app"
func main() {
http.HandleFunc("/", proxy)
log.Fatal(http.ListenAndServe(":8081", nil))
}
func proxy(res http.ResponseWriter, req *http.Request) {
// gets past CORS checks
if req.Method == http.MethodOptions {
headers := res.Header()
headers.Add("Access-Control-Allow-Origin", "*")
headers.Add("Vary", "Origin")
headers.Add("Vary", "Access-Control-Request-Method")
headers.Add("Vary", "Access-Control-Request-Headers")
headers.Add("Access-Control-Allow-Headers", "*")
headers.Add("Access-Control-Allow-Methods", "GET,HEAD,PUT,PATCH,POST,DELETE")
res.WriteHeader(http.StatusOK)
return
}
p := httputil.NewSingleHostReverseProxy(&url.URL{
Scheme: "http",
Host: apiUrl,
})
p.Director = func(req *http.Request) {
req.Header.Add("X-Forwarded-Host", req.Host)
req.Header.Add("X-Origin-Host", apiUrl)
req.Header.Add("Host", apiUrl)
req.Header.Add("Access-Control-Allow-Origin", "*")
req.URL.Scheme = "https"
req.URL.Host = apiUrl
}
p.ModifyResponse = func(res *http.Response) error {
res.Header.Set("Access-Control-Allow-Methods", "GET,HEAD,PUT,PATCH,POST,DELETE")
res.Header.Set("Access-Control-Allow-Credentials", "true")
res.Header.Set("Access-Control-Allow-Origin", "*")
res.Header.Set("Access-Control-Allow-Headers", "*")
return nil
}
p.ServeHTTP(res, req)
}
This is a bit more elaborate than the original initial write-up but what we wound up with was as follows.
package main
import (
"context"
"fmt"
"log"
"net/http"
"net/http/httputil"
"net/url"
"os"
"os/signal"
"time"
"golang.org/x/oauth2"
"google.golang.org/api/idtoken"
)
var port = ":8080"
var backend = "[CLOUD_RUN_INSTANCE_TO_PROXY].a.run.app"
func main() {
logger := log.New(os.Stdout, "proxy: ", log.LstdFlags)
logger.Println(fmt.Sprintf("Proxy server is starting for: %s on port: %s", backend, port))
router := http.NewServeMux()
router.Handle("/", proxyHandler())
server := &http.Server{
Addr: port,
Handler: logging(logger)(router),
ErrorLog: logger,
ReadTimeout: 30 * time.Second,
WriteTimeout: 30 * time.Second,
IdleTimeout: 15 * time.Second,
}
done := make(chan bool)
quit := make(chan os.Signal, 1)
signal.Notify(quit, os.Interrupt)
go func() {
<-quit
logger.Println("Proxy server is shutting down...")
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
server.SetKeepAlivesEnabled(false)
if err := server.Shutdown(ctx); err != nil {
logger.Fatalf("Could not gracefully shutdown the server: %v\n", err)
}
close(done)
}()
if err := server.ListenAndServe(); err != nil && err != http.ErrServerClosed {
logger.Fatalf("Could not listen on %s: %v\n", port, err)
}
<-done
logger.Println("Server stopped")
}
func proxyHandler() http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Method == http.MethodOptions {
headers := w.Header()
headers.Add("Access-Control-Allow-Origin", "*")
headers.Add("Access-Control-Allow-Headers", "*")
headers.Add("Access-Control-Allow-Methods", "GET,HEAD,PUT,PATCH,POST,DELETE")
w.WriteHeader(http.StatusOK)
return
}
path := fmt.Sprintf("https://%s%s", backend, r.RequestURI)
at, _ := idTokenTokenSource(path)
p := httputil.NewSingleHostReverseProxy(&url.URL{
Scheme: "https",
Host: backend,
})
p.Director = func(r *http.Request) {
if at != nil {
at.SetAuthHeader(r)
}
}
p.ModifyResponse = func(res *http.Response) error {
res.Header.Set("Access-Control-Allow-Methods", "GET,HEAD,PUT,PATCH,POST,DELETE")
res.Header.Set("Access-Control-Allow-Credentials", "true")
res.Header.Set("Access-Control-Allow-Origin", "*")
res.Header.Set("Access-Control-Allow-Headers", "*")
return nil
}
r.URL.Scheme = "https"
r.URL.Host = backend
r.Header.Set("X-Forwarded-Host", r.Header.Get("Host"))
r.Host = backend
if at != nil {
at.SetAuthHeader(r)
}
p.ServeHTTP(w, r)
})
}
func logging(l *log.Logger) func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
defer func() {
requestId := r.Header.Get("X-Request-Id")
if requestId == "" {
requestId = fmt.Sprintf("%d", time.Now().UnixNano())
}
w.Header().Set("X-Request-Id", requestId)
l.Println(requestId, r.Method, r.URL.Path, r.RemoteAddr, r.UserAgent())
}()
next.ServeHTTP(w, r)
})
}
}
func idTokenTokenSource(audience string) (*oauth2.Token, error) {
ts, err := idtoken.NewTokenSource(context.Background(), audience)
if err != nil {
return nil, err
}
t, err := ts.Token()
if err != nil {
return nil, err
}
return t, nil
}
A good chunk of some of the graceful shutdown, http setup, and logging came from: https://gist.github.com/enricofoltran/10b4a980cd07cb02836f70a4ab3e72d7

Go net/http leaks memory in high load

I am developing an API that calls client URL using the net/http package. There are between 1 and 8 URLs called for each request (POST call) in goroutines concurrently based on user country/os. The app works with low qps of around 1000-1500 requests, but scaling the app to 3k requests there is a sudden increase in the memory even if only 1 client URL is called an app stops responding after a few minute(Response time well above 50sec). I am using Go native net/http package along with gorilla/mux router. Other question on this issue says to close the response body but I have done that using
req, err := http.NewRequest("POST", "client_post_url", bytes.NewBuffer(requestBody))
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Connection", "Keep-Alive")
response, err := common.Client.Do(req)
status := 0
if err != nil {//handle and return}
defer response.Body.Close() //used with/without io.Copy
status = response.StatusCode
body, _ := ioutil.ReadAll(response.Body)
_, err = io.Copy(ioutil.Discard, response.Body)
I need to reuse connection hence I have made http client and transport global variable initialized in init method like this.
common.Transport = &http.Transport{
TLSClientConfig: &tls.Config{
InsecureSkipVerify: true,
},
DialContext: (&net.Dialer{
//Timeout: time.Duration(300) * time.Millisecond,
KeepAlive: 30 * time.Second,
}).DialContext,
//ForceAttemptHTTP2: true,
DisableKeepAlives: false,
//MaxIdleConns: 0,
//IdleConnTimeout: 0,
//TLSHandshakeTimeout: time.Duration(300) * time.Millisecond,
//ExpectContinueTimeout: 1 * time.Second,
}
common.Client = &http.Client{
Timeout: time.Duration(300) * time.Millisecond,
Transport: common.Transport,
}
I have read that using keep alive causes the memory to leak, I have tried a few combination for disabling keep-alive/close request flag on request. But nothing seems to work. Also If I don't make any http call and use time.Sleep(300 * time.Millisecond) in goroutine calling each url concurrently app does work without any leak.
So I am sure It has something to do with client/http package that under high load connection are not released or not used properly.
What should be my approach to achieve this?
Is creating a custom server and custom handler type to accept request and route requests will worked as mentioned in C10K approach in several article?
I can share the sample code with all details if required. Above just added that the part where I feel the issue lies.
this is a representative code
main.go
package main
import (
"./common"
"bytes"
"crypto/tls"
"fmt"
"github.com/gorilla/mux"
"io"
"io/ioutil"
"log"
"math/rand"
"net"
"net/http"
"net/http/pprof"
"os"
"runtime"
"strconv"
"sync"
"time"
)
func init() {
//Get Any command line argument passed
args := os.Args[1:]
numCPU := runtime.NumCPU()
if len(args) > 1 {
numCPU, _ = strconv.Atoi(args[0])
}
common.Transport = &http.Transport{
TLSClientConfig: &tls.Config{
InsecureSkipVerify: true,
},
DialContext: (&net.Dialer{
//Timeout: time.Duration() * time.Millisecond,
KeepAlive: 30 * time.Second,
}).DialContext,
//ForceAttemptHTTP2: true,
DisableKeepAlives: false,
//MaxIdleConns: 0,
//IdleConnTimeout: 0,
//TLSHandshakeTimeout: time.Duration(300) * time.Millisecond,
//ExpectContinueTimeout: 1 * time.Second,
}
common.Client = &http.Client{
Timeout: time.Duration(300) * time.Millisecond,
Transport: common.Transport,
}
runtime.GOMAXPROCS(numCPU)
rand.Seed(time.Now().UTC().UnixNano())
}
func main() {
router := mux.NewRouter().StrictSlash(true)
router.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
_, _ = fmt.Fprintf(w, "Hello!!!")
})
router.HandleFunc("/{name}", func(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
prepareRequest(w, r, vars["name"])
}).Methods("POST")
// Register pprof handlers
router.HandleFunc("/debug/pprof/", pprof.Index)
router.HandleFunc("/debug/pprof/cmdline", pprof.Cmdline)
router.HandleFunc("/debug/pprof/profile", pprof.Profile)
router.HandleFunc("/debug/pprof/symbol", pprof.Symbol)
router.HandleFunc("/debug/pprof/trace", pprof.Trace)
routerMiddleWare := http.TimeoutHandler(router, 500*time.Millisecond, "Timeout")
srv := &http.Server{
Addr: "0.0.0.0:" + "80",
/*ReadTimeout: 500 * time.Millisecond,
WriteTimeout: 500 * time.Millisecond,
IdleTimeout: 10 * time.Second,*/
Handler: routerMiddleWare,
}
log.Fatal(srv.ListenAndServe())
}
func prepareRequest(w http.ResponseWriter, r *http.Request, name string) {
//other part of the code and call to goroutine
var urls []string
results, s, c := callUrls(urls)
finalCall(w, results, s, c)
}
type Response struct {
Status int
Url string
Body string
}
func callUrls(urls []string) ([]*Response, []string, []string) {
var wg sync.WaitGroup
wg.Add(len(urls))
ch := make(chan func() (*Response, string, string), len(urls))
for _, url := range urls {
go func(url string) {
//decide if request is valid for client to make http call using country/os
isValid := true //assuming url to be called
if isValid {
//make post call
//request body have many more paramter, just sample included.
//instead of creating new request, time.Sleep for 300ms doesn't cause any memory leak.
req, err := http.NewRequest("POST", url, bytes.NewBuffer([]byte(`{"body":"param"}`)))
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Connection", "Keep-Alive")
//req.Close = true
response, err := common.Client.Do(req)
if err != nil {
wg.Done()
ch <- func() (*Response, string, string) {
return &Response{Status: 500, Url: url, Body: ""}, "error", "500"
}
return
}
defer response.Body.Close()
body, _ := ioutil.ReadAll(response.Body)
_, err = io.Copy(ioutil.Discard, response.Body)
//Close the body, forced this
//Also tried without defer, and only wothout following line
response.Body.Close()
//do something with response body replace a few string etc.
//and return
wg.Done()
ch <- func() (*Response, string, string) {
return &Response{Status: 200, Url: url, Body: string(body)}, "success", "200"
}
} else {
wg.Done()
ch <- func() (*Response, string, string) {
return &Response{Status: 500, Url: url, Body: ""}, "invalid", "500"
}
}
}(url)
}
wg.Wait()
var (
results []*Response
msg []string
status []string
)
for {
r, x, y := (<-ch)()
if r != nil {
results = append(results, r)
msg = append(msg, x)
status = append(status, y)
}
if len(results) == len(urls) {
return results, msg, status
}
}
}
func finalCall(w http.ResponseWriter, results []*Response, msg []string, status []string){
fmt.Println("response", "response body", results, msg, status)
}
vars.go
package common
import (
"net/http"
)
var (
//http client
Client *http.Client
//http Transport
Transport *http.Transport
)
pprof: Profiled app with 4 client url on average of around 2500qps.
Top command:
After 2minutes:
Without calling client url, by keeping isValid = false and time.Sleep(300* time.Millisecond) no leaks happens.
this code is not leaking.
To demonstrate, lets update it ** slightly so the post is reproducible.
main.go
package main
import (
"bytes"
"crypto/tls"
_ "expvar"
"fmt"
"io"
"io/ioutil"
"log"
"math/rand"
"net"
"net/http"
_ "net/http/pprof"
"os"
"runtime"
"strconv"
"sync"
"time"
"github.com/gorilla/mux"
)
var (
//http client
Client *http.Client
//http Transport
Transport *http.Transport
)
func init() {
go http.ListenAndServe("localhost:6060", nil)
//Get Any command line argument passed
args := os.Args[1:]
numCPU := runtime.NumCPU()
if len(args) > 1 {
numCPU, _ = strconv.Atoi(args[0])
}
Transport = &http.Transport{
TLSClientConfig: &tls.Config{
InsecureSkipVerify: true,
},
DialContext: (&net.Dialer{
//Timeout: time.Duration() * time.Millisecond,
KeepAlive: 30 * time.Second,
}).DialContext,
//ForceAttemptHTTP2: true,
DisableKeepAlives: false,
//MaxIdleConns: 0,
//IdleConnTimeout: 0,
//TLSHandshakeTimeout: time.Duration(300) * time.Millisecond,
//ExpectContinueTimeout: 1 * time.Second,
}
Client = &http.Client{
// Timeout: time.Duration(300) * time.Millisecond,
Transport: Transport,
}
runtime.GOMAXPROCS(numCPU)
rand.Seed(time.Now().UTC().UnixNano())
}
func main() {
router := mux.NewRouter().StrictSlash(true)
router.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
_, _ = fmt.Fprintf(w, "Hello!!!")
})
router.HandleFunc("/{name}", func(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
prepareRequest(w, r, vars["name"])
}).Methods("POST", "GET")
// Register pprof handlers
// router.HandleFunc("/debug/pprof/", pprof.Index)
// router.HandleFunc("/debug/pprof/cmdline", pprof.Cmdline)
// router.HandleFunc("/debug/pprof/profile", pprof.Profile)
// router.HandleFunc("/debug/pprof/symbol", pprof.Symbol)
// router.HandleFunc("/debug/pprof/trace", pprof.Trace)
routerMiddleWare := http.TimeoutHandler(router, 500*time.Millisecond, "Timeout")
srv := &http.Server{
Addr: "localhost:8080",
/*ReadTimeout: 500 * time.Millisecond,
WriteTimeout: 500 * time.Millisecond,
IdleTimeout: 10 * time.Second,*/
Handler: routerMiddleWare,
}
log.Fatal(srv.ListenAndServe())
}
func prepareRequest(w http.ResponseWriter, r *http.Request, name string) {
// go func() {
// make(chan []byte) <- make([]byte, 10024)
// }()
//other part of the code and call to goroutine
var urls []string
urls = append(urls,
"http://localhost:7000/",
"http://localhost:7000/",
)
results, s, c := callUrls(urls)
finalCall(w, results, s, c)
}
type Response struct {
Status int
Url string
Body string
}
func callUrls(urls []string) ([]*Response, []string, []string) {
var wg sync.WaitGroup
wg.Add(len(urls))
ch := make(chan func() (*Response, string, string), len(urls))
for _, url := range urls {
go func(url string) {
//decide if request is valid for client to make http call using country/os
isValid := true //assuming url to be called
if isValid {
//make post call
//request body have many more paramter, just sample included.
//instead of creating new request, time.Sleep for 300ms doesn't cause any memory leak.
req, err := http.NewRequest("POST", url, bytes.NewBuffer([]byte(`{"body":"param"}`)))
if err != nil {
wg.Done()
ch <- func() (*Response, string, string) {
return &Response{Status: 500, Url: url, Body: ""}, err.Error(), "500"
}
return
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Connection", "Keep-Alive")
//req.Close = true
response, err := Client.Do(req)
if err != nil {
wg.Done()
ch <- func() (*Response, string, string) {
return &Response{Status: 500, Url: url, Body: ""}, err.Error(), "500"
}
return
}
defer response.Body.Close()
body, _ := ioutil.ReadAll(response.Body)
io.Copy(ioutil.Discard, response.Body)
//Close the body, forced this
//Also tried without defer, and only wothout following line
response.Body.Close()
//do something with response body replace a few string etc.
//and return
wg.Done()
ch <- func() (*Response, string, string) {
return &Response{Status: 200, Url: url, Body: string(body)}, "success", "200"
}
} else {
wg.Done()
ch <- func() (*Response, string, string) {
return &Response{Status: 500, Url: url, Body: ""}, "invalid", "500"
}
}
}(url)
}
wg.Wait()
var (
results []*Response
msg []string
status []string
)
for {
r, x, y := (<-ch)()
if r != nil {
results = append(results, r)
msg = append(msg, x)
status = append(status, y)
}
if len(results) == len(urls) {
return results, msg, status
}
}
}
func finalCall(w http.ResponseWriter, results []*Response, msg []string, status []string) {
fmt.Println("response", "response body", results, msg, status)
}
k/main.go
package main
import "net/http"
func main() {
y := make([]byte, 100)
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
w.Write(y)
})
http.ListenAndServe(":7000", nil)
}
Install additional visualization tool, and use ab to simulate some load, it will do the job for that intuitive demonstration.
go get -u github.com/divan/expvarmon
go run main.go &
go run k/main.go &
ab -n 50000 -c 2500 http://localhost:8080/y
# in a different window, for live preview
expvarmon -ports=6060 -i 500ms
At that point you read the output of expvarmon, if it was live you have something like
you can see the stuff waving, the gc is being actively working.
the app is loaded, the memory is being consumed, wait for the server to release its conn and the gc to clean them
You can see the memstats.Alloc, memstats.HeapAlloc, memstats.HeapInuse are now reduced, as expected when the gc does his job and that no leak exists.
If you were to check for go tool pprof -inuse_space -web http://localhost:6060/debug/pprof/heap, right after ab ran
It shows that the app is using 177Mb of memory.
Most of it 102Mb is being used by net/http.Transport.getConn.
Your handler is accouting for 1Mb, the rest is various things required.
If you were to take the screenshot after the server has released and the gc too, you would see an even smaller graph. not demonstrated here.
Now let us generate a leak and see it using both tools again.
In the code uncomment in,
func prepareRequest(w http.ResponseWriter, r *http.Request, name string) {
go func() {
make(chan []byte) <- make([]byte, 10024)
}()
//...
restart apps (press q in expvarmon, although it is not required)
go get -u github.com/divan/expvarmon
go run main.go &
go run k/main.go &
ab -n 50000 -c 2500 http://localhost:8080/y
# in a different window, for live preview
expvarmon -ports=6060 -i 500ms
it shows
In expvarmon you can see the same behavior, only the numbers has changed, and at rest state, after it has been gced, it still consumed a lot of memory, a lot more than a void golang http server to take a comparison point.
Again, screenshot the heap, it shows that your handler is now consuming most of the memory ~450Mb, notice the arrows, it shows that there is for 452mb of 10kb allocations, and 4.50Mb of 96b. They respectively correspond to the []byte slice being pushed to the chan []byte.
Finally, you can check your stack traces to look for dead goroutines, and thus leaking memory, open http://localhost:6060/debug/pprof/goroutine?debug=1
goroutine profile: total 50012
50000 # 0x43098f 0x4077fa 0x4077d0 0x4074bb 0x76b85d 0x45d281
# 0x76b85c main.prepareRequest.func1+0x4c /home/mh-cbon/gow/src/test/oom/main.go:101
4 # 0x43098f 0x42c09a 0x42b686 0x4c3a3b 0x4c484b 0x4c482c 0x57d94f 0x590d79 0x6b4c67 0x5397cf 0x53a51d 0x53a754 0x6419ef 0x6af18d 0x6af17f 0x6b5f33 0x6ba4fd 0x45d281
# 0x42b685 internal/poll.runtime_pollWait+0x55 /home/mh-cbon/.gvm/gos/go1.12.7/src/runtime/netpoll.go:182
# 0x4c3a3a internal/poll.(*pollDesc).wait+0x9a /home/mh-cbon/.gvm/gos/go1.12.7/src/internal/poll/fd_poll_runtime.go:87
// more...
It tells us that the programs is hosting 50 012 goroutines, then it lists them grouped by file positions, where the first number is the count of instances running, 50 000 in the first group of this example. It is followed by the stack trace that lead to the goroutine to exist.
You can see there is a bunch of system thing, that in your case, you should not worry much about it.
You got to look for those that you believe you should not be live if your program was working as you think it should.
However, overall your code is not satisfying and could be, and probably, should be improved with a thorough review about its allocations and overall design conception.
** This is a summary of the changes applied to the original source code.
It adds a new program k/main.go to act as a backend server.
It adds _ "expvar" import statement
It starts the std api HTTP server instance that pprof registers onto during init phase with go http.ListenAndServe("localhost:6060", nil)
The client timeout is disabled Timeout: time.Duration(300) * time.Millisecond,, otherwise the load test does not return 200s
The server address is set to Addr: "localhost:8080",
The urls values created within prepareRequest are set to a static list of len=2
It adds error checking for req, err := http.NewRequest("POST", url, bytes.NewBuffer([]byte({"body":"param"})))
It disalbles error checking in io.Copy(ioutil.Discard, response.Body)
I have solved it by replacing net/http package with fasthttp. Earlier I haven't used it because I was not able find timeout method on fasthttp client but I see that there is indeed a method DoTimeout for fasthttp client which timedout the request after specified duration.
Here the updated code:
in vars.go ClientFastHttp *fasthttp.Client
main.go
package main
import (
"./common"
"crypto/tls"
"fmt"
"github.com/gorilla/mux"
"github.com/valyala/fasthttp"
"log"
"math/rand"
"net"
"net/http"
"net/http/pprof"
"os"
"runtime"
"strconv"
"sync"
"time"
)
func init() {
//Get Any command line argument passed
args := os.Args[1:]
numCPU := runtime.NumCPU()
if len(args) > 1 {
numCPU, _ = strconv.Atoi(args[0])
}
common.Transport = &http.Transport{
TLSClientConfig: &tls.Config{
InsecureSkipVerify: true,
},
DialContext: (&net.Dialer{
//Timeout: time.Duration() * time.Millisecond,
KeepAlive: 30 * time.Second,
}).DialContext,
//ForceAttemptHTTP2: true,
DisableKeepAlives: false,
//MaxIdleConns: 0,
//IdleConnTimeout: 0,
//TLSHandshakeTimeout: time.Duration(300) * time.Millisecond,
//ExpectContinueTimeout: 1 * time.Second,
}
common.Client = &http.Client{
Timeout: time.Duration(300) * time.Millisecond,
Transport: common.Transport,
}
runtime.GOMAXPROCS(numCPU)
rand.Seed(time.Now().UTC().UnixNano())
}
func main() {
router := mux.NewRouter().StrictSlash(true)
router.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
_, _ = fmt.Fprintf(w, "Hello!!!")
})
router.HandleFunc("/{name}", func(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
prepareRequest(w, r, vars["name"])
}).Methods("POST")
// Register pprof handlers
router.HandleFunc("/debug/pprof/", pprof.Index)
router.HandleFunc("/debug/pprof/cmdline", pprof.Cmdline)
router.HandleFunc("/debug/pprof/profile", pprof.Profile)
router.HandleFunc("/debug/pprof/symbol", pprof.Symbol)
router.HandleFunc("/debug/pprof/trace", pprof.Trace)
routerMiddleWare := http.TimeoutHandler(router, 500*time.Millisecond, "Timeout")
srv := &http.Server{
Addr: "0.0.0.0:" + "80",
/*ReadTimeout: 500 * time.Millisecond,
WriteTimeout: 500 * time.Millisecond,
IdleTimeout: 10 * time.Second,*/
Handler: routerMiddleWare,
}
log.Fatal(srv.ListenAndServe())
}
func prepareRequest(w http.ResponseWriter, r *http.Request, name string) {
//other part of the code and call to goroutine
var urls []string
results, s, c := callUrls(urls)
finalCall(w, results, s, c)
}
type Response struct {
Status int
Url string
Body string
}
func callUrls(urls []string) ([]*Response, []string, []string) {
var wg sync.WaitGroup
wg.Add(len(urls))
ch := make(chan func() (*Response, string, string), len(urls))
for _, url := range urls {
go func(url string) {
//decide if request is valid for client to make http call using country/os
isValid := true //assuming url to be called
if isValid {
//make post call
//request body have many more paramter, just sample included.
//instead of creating new request, time.Sleep for 300ms doesn't cause any memory leak.
req := fasthttp.AcquireRequest()
req.SetRequestURI(url)
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Connection", "Keep-Alive")
req.Header.SetMethod("POST")
req.SetBody([]byte(`{"body":"param"}`))
resp := fasthttp.AcquireResponse()
defer fasthttp.ReleaseRequest(req) // <- do not forget to release
defer fasthttp.ReleaseResponse(resp) // <- do not forget to release
//err := clientFastHttp.Do(req, response)
//endregion
t := time.Duration(300)
err := common.ClientFastHttp.DoTimeout(req, resp, t*time.Millisecond)
body := resp.Body()
if err != nil {
wg.Done()
ch <- func() (*Response, string, string) {
return &Response{Status: 500, Url: url, Body: ""}, "error", "500"
}
return
}
/*defer response.Body.Close()
body, _ := ioutil.ReadAll(response.Body)
_, err = io.Copy(ioutil.Discard, response.Body)
//Close the body, forced this
//Also tried without defer, and only wothout following line
response.Body.Close()*/
//do something with response body replace a few string etc.
//and return
wg.Done()
ch <- func() (*Response, string, string) {
return &Response{Status: 200, Url: url, Body: string(body)}, "success", "200"
}
} else {
wg.Done()
ch <- func() (*Response, string, string) {
return &Response{Status: 500, Url: url, Body: ""}, "invalid", "500"
}
}
}(url)
}
wg.Wait()
var (
results []*Response
msg []string
status []string
)
for {
r, x, y := (<-ch)()
if r != nil {
results = append(results, r)
msg = append(msg, x)
status = append(status, y)
}
if len(results) == len(urls) {
return results, msg, status
}
}
}
func finalCall(w http.ResponseWriter, results []*Response, msg []string, status []string) {
fmt.Println("response", "response body", results, msg, status)
}

Error: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)

About 3~4minutes,Some Errors would happen in my log.
net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
I try to find out where it takes time Using httptrace.
httptrace.GetConn
httptrace.GotConn
I think it runs out of time before httptrace.GotConn.
So errors happend
request canceled while waiting for connection
My machine is ok.and this is my netstat.
LAST_ACK 2
CLOSE_WAIT 7
ESTABLISHED 108
SYN_SENT 3
TIME_WAIT 43
package main
import (
"crypto/md5"
"encoding/hex"
"fmt"
"io/ioutil"
"math/rand"
"net"
"net/http"
"net/http/httptrace"
"os"
"sync"
"time"
)
var Client *http.Client = &http.Client{
Transport: &http.Transport{
DisableKeepAlives:true,
Proxy: http.ProxyFromEnvironment,
DialContext: (&net.Dialer{
Timeout: 3 * time.Second, // 连接超时
KeepAlive: 10 * time.Second,
DualStack: true,
}).DialContext,
IdleConnTimeout: 120 * time.Second,
ResponseHeaderTimeout: 60 * time.Second,
ExpectContinueTimeout: 1 * time.Second,
},
Timeout: 500 * time.Millisecond,
}
func GenLogId() string {
h2 := md5.New()
rand.Seed(time.Now().Unix())
str := fmt.Sprintf("%d%d%d", os.Getpid(), time.Now().UnixNano(), rand.Int())
h2.Write([]byte(str))
uniqid := hex.EncodeToString(h2.Sum(nil))
return uniqid
}
func main() {
var (
wg sync.WaitGroup
maxParallel int = 50
parallelChan chan bool = make(chan bool, maxParallel)
)
for {
parallelChan <- true
wg.Add(1)
go func() {
defer func() {
wg.Done()
<-parallelChan
}()
testHttp2()
}()
}
wg.Wait()
}
func testHttp2() {
url := "http://10.33.108.39:11222/index.php"
req, _ := http.NewRequest("GET", url, nil)
uniqId := GenLogId()
trace := &httptrace.ClientTrace{
GetConn: func(hostPort string) {
fmt.Println("GetConn id:", uniqId, time.Now().UnixNano(), hostPort)
},
GotConn: func(connInfo httptrace.GotConnInfo) {
fmt.Println("GotConn id:", uniqId, time.Now().UnixNano(), connInfo.Conn.LocalAddr())
},
ConnectStart: func(network, addr string) {
fmt.Println("ConnectStart id:", uniqId, time.Now().UnixNano(), network, addr)
},
ConnectDone: func(network, addr string, err error) {
fmt.Println("ConnectDone id:", uniqId, time.Now().UnixNano(), network, addr, err)
},
}
req = req.WithContext(httptrace.WithClientTrace(req.Context(), trace))
resp, err := Client.Do(req)
if err != nil {
fmt.Println("err: id", uniqId, time.Now().UnixNano(), err)
return
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
fmt.Println("error", string(body))
}
return
}
You can reproduce using my code. I am so confuse about the bug...
Thank you.
You need to increase the client Timeout value for your test.
net/http: request canceled (Client.Timeout exceeded while awaiting headers)
This means your Client.Timeout value is less than your server response time, due to many reasons ( e.g. Busy, CPU overload, many requests per second you generated here, ...).
Here a simple way to explain it and regenerate it:
Run this server (which waits for 2 * time.Second then sends back the response):
package main
import (
"io"
"log"
"net/http"
"time"
)
func main() {
http.HandleFunc(`/`, func(w http.ResponseWriter, r *http.Request) {
log.Println("wait a couple of seconds ...")
time.Sleep(2 * time.Second)
io.WriteString(w, `Hi`)
log.Println("Done.")
})
log.Println(http.ListenAndServe(":8080", nil))
}
Then run this client which times out in 1 * time.Second:
package main
import (
"io/ioutil"
"log"
"net/http"
"time"
)
func main() {
log.Println("HTTP GET")
client := &http.Client{
Timeout: 1 * time.Second,
}
r, err := client.Get(`http://127.0.0.1:8080/`)
if err != nil {
log.Fatal(err)
}
defer r.Body.Close()
bs, err := ioutil.ReadAll(r.Body)
if err != nil {
log.Fatal(err)
}
log.Println("HTTP Done.")
log.Println(string(bs))
}
The output is (Client.Timeout exceeded while awaiting headers):
2019/10/30 11:05:08 HTTP GET
2019/10/30 11:05:09 Get http://127.0.0.1:8080/: net/http: request canceled (Client.Timeout exceeded while awaiting headers)
exit status 1
Note:
You need to change these two settings accordingly (http.Transport.ResponseHeaderTimeout and http.Client.Timeout).
You have set ResponseHeaderTimeout: 60 * time.Second, while Client.Timeout to half a second.
Suppose anyone wants to capture theses errors please use,
os.IsTimeout(err) -> it will return true for context deadlined
For capturing dial i/o timeout issue,
netErr, ok := err.(net.Error); (ok && netErr.Timeout()) -> it will return true for dial i/o timeout

How to write a simple custom HTTP server in Go?

I am new to Go and trying to write a custom HTTP server. I'm getting a compilation error. How can I implement the ServeHTTP method in my code?
My Code:
package main
import (
"net/http"
"fmt"
"io"
"time"
)
func myHandler(w http.ResponseWriter, req *http.Request) {
io.WriteString(w, "hello, world!\n")
}
func main() {
// Custom http server
s := &http.Server{
Addr: ":8080",
Handler: myHandler,
ReadTimeout: 10 * time.Second,
WriteTimeout: 10 * time.Second,
MaxHeaderBytes: 1 << 20,
}
err := s.ListenAndServe()
if err != nil {
fmt.Printf("Server failed: ", err.Error())
}
}
Error while compiling:
.\hello.go:21: cannot use myHandler (type func(http.ResponseWriter, *http.Request)) as type http.Handler in field value:
func(http.ResponseWriter, *http.Request) does not implement http.Handler (missing ServeHTTP method)
You either use a struct and define ServeHTTP on it or simply wrap your function in a HandlerFunc
s := &http.Server{
Addr: ":8080",
Handler: http.HandlerFunc(myHandler),
ReadTimeout: 10 * time.Second,
WriteTimeout: 10 * time.Second,
MaxHeaderBytes: 1 << 20,
}
in order to work properly, myHandler should be an object that satisfy the Handler Interface, in other words myHandler should have method called ServeHTTP.
For example, let's say that myHandler is custom handler for showing the current time. The code should be like this
package main
import (
"fmt"
"net/http"
"time"
)
type timeHandler struct {
zone *time.Location
}
func (th *timeHandler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
tm := time.Now().In(th.zone).Format(time.RFC1123)
w.Write([]byte("The time is: " + tm))
}
func newTimeHandler(name string) *timeHandler {
return &timeHandler{zone: time.FixedZone(name, 0)}
}
func main() {
myHandler := newTimeHandler("EST")
//Custom http server
s := &http.Server{
Addr: ":8080",
Handler: myHandler,
ReadTimeout: 10 * time.Second,
WriteTimeout: 10 * time.Second,
MaxHeaderBytes: 1 << 20,
}
err := s.ListenAndServe()
if err != nil {
fmt.Printf("Server failed: ", err.Error())
}
}
run this code and access http://localhost:8080/ in your browser. you should see formatted text like this
The time is: Sat, 30 Aug 2014 18:19:46 EST
(you should see different time.)
hope this help,
Further more reading
A Recap of Request Handling in Go
You could also define a mux and add handlers to it before assigning it to the server like below.
func myHandler(w http.ResponseWriter, req *http.Request) {
io.WriteString(w, "hello, world!\n")
}
func main(){
// define a serveMux to handle routes
mux := http.NewServeMux()
// assign a route/todo to a handler myHandler
mux.HandleFunc("/todo", myHandler)
// assign a route/todo/notes to an anonymous func
mux.HandleFunc("/todo/notes", func(w http.ResponseWriter, req *http.Request) {
w.Write([]byte("Hello again."))
})
s := &http.Server{
Addr: ":8080",
Handler: mux,
ReadTimeout: 10 * time.Second,
WriteTimeout: 10 * time.Second,
MaxHeaderBytes: 1 << 20,
}
if err := s.ListenAndServe(); err != nil {
log.Fatalf("server failed to start with error %v", err.Error())
}
}

Resources