I am writing a simple TCP proxy server that is started, stopped, and reconfigured by an HTTP handler. The proxy part works fine.
The problem I am having is starting it again each time the HTTP handler runs. i.e. it works on the first pass. But the second pass panics because the listener is already running.
My problem seems to be how to detect that the goroutine is already running, so I can stop it and start it again with new parameters. i.e. I can't figure out how to get a persistent handle on it from each execution of the web handler. It ends up crashing because the listener is already listening...
In another language, I would make it a Singleton. But what is the best way to do it in Go?
My Server has a NewServer(), Server.Stop, using a channel, etc. And they work if I keep them within the one block of code.
i.e. this works
ps = utils.NewProxyServer(listening, target[0])
time.Sleep(...)
ps.Stop()
How can I persist the handle on it between passes through the handler?
var ps *utils.Server
func MyHandler(w http.ResponseWriter, r *http.Request) {
switch r.Method {
case "GET":
...
// how can I know that ps exists so I can close it?
// and create a new one? Elegantly...?
ps.Stop() // this panics
ps = utils.NewServer(listening, target[0])
log.Print(ps.Status)
http.Redirect(w, r, redirurl, http.StatusSeeOther)
} else {
http.Redirect(w, r, "/network", http.StatusSeeOther)
}
}
}
I'd recommend having a goroutine manage ps, and using a channel to communicate with it:
var ps *utils.Server
// Buffer size of 1 to prevent blocking assuming it's not triggered frequently
var psControl = make(chan psConfig,1)
type psConfig struct {
// Correct the types as needed
listening string
target string
}
// This probably shouldn't be in an init(), put it wherever is appropriate
func init() {
go func() {
for cfg := range psControl {
if ps != nil {
ps.Stop()
}
ps = utils.NewServer(cfg.listening, cfg.target)
log.Print(ps.Status)
}
}()
}
func MyHandler(w http.ResponseWriter, r *http.Request) {
switch r.Method {
case "GET":
...
psControl <- psConfig{listening, target[0]}
http.Redirect(w, r, redirurl, http.StatusSeeOther)
} else {
http.Redirect(w, r, "/network", http.StatusSeeOther)
}
}
}
Related
I'm trying to find a way to add a correlation/request id for logs in our project to make it easier to navigate through them and debug when some issues occur. I found this article. From the example there, there is a middleware to add the correlationID and then retrieve it in some handler function.
Middleware function:
const ContextKeyRequestID ContextKey = "requestID"
func reqIDMiddleware1(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
id := uuid.New()
ctx = context.WithValue(ctx, ContextKeyRequestID, id.String())
r = r.WithContext(ctx)
log.Debugf("Incoming request %s %s %s %s", r.Method, r.RequestURI, r.RemoteAddr, id.String())
next.ServeHTTP(w, r)
log.Debugf("Finished handling http req. %s", id.String())
})
}
Handler:
const LogFieldKeyRequestID = "requestID"
func handleSomeRequest() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
ctx := r.Context()
reqIDRaw := ctx.Value(ContextKeyRequestID) // reqIDRaw at this point is of type 'interface{}'
reqID, ok := reqIDRaw.(string)
if !ok {
// handler error
}
// if reached here, reqID is ready to be used
// let's use it with logrus FieldLogger!
logger := log.WithField(LogFieldKeyRequestID, reqID)
// Do something, then log what you did
logger.Debugf("What I just did!")
// Do more, log more. Handle this request seriously
}
}
But I was wondering if there is a way to achieve this without having to refactor all the existing handlers and changing the logging functionality, through some automatic configuration that would add id for each log, in my case our project is quite big, and doing it in the way described above would require a lot of changes.
Have you looked at WithContext and Hooks?
You still have to modify your code but you can centralize some behaviours.
https://go.dev/play/p/4YxJMK6Zl5D
I'm trying to integrate saml using crewjam library with an open-source app in go.
After authentication test using samltest.id, I want to be redirected to the home page.
I have tried several ways, but nothing works well, i'm using gorilla/mux router:
func login(w http.ResponseWriter, r *http.Request) {
s := samlsp.SessionFromContext(r.Context())
if s == nil {
return
}
sa, ok := s.(samlsp.SessionWithAttributes)
if !ok {
return
}
fmt.Fprintf(w, "Token contents, %+v!", sa.GetAttributes())
w.Header().Add("Location", "http://localhost:8080/")
w.WriteHeader(http.StatusFound)
}
I have also tested :
http.Redirect(w, r, "http://localhost:8080/", http.StatusFound)
Can someone help me please?
Thanks :)
Calling w.Write or writing into it using Fmt.Fprintf requires HTTP status code to be set before, otherwise it sets default StatusOK
Server.go
// If WriteHeader is not called explicitly, the first call to Write
// will trigger an implicit WriteHeader(http.StatusOK).
Setting the status code multiple times throws superfluous log.
Therefore, Your code is setting the HTTP status code to 200 (http.StatusOk), so the redirect after that is simply impossible.
Solution:
func login(w http.ResponseWriter, r *http.Request) {
s := samlsp.SessionFromContext(r.Context())
if s == nil {
return
}
sa, ok := s.(samlsp.SessionWithAttributes)
if !ok {
return
}
// this line is removed
// fmt.Fprintf(w, "Token contents, %+v!", sa.GetAttributes())
w.Header().Add("Location", "http://localhost:8080/")
w.WriteHeader(http.StatusFound)
// Or Simply
// http.Redirect(w, r, "http://localhost:8080/", http.StatusFound)
}
Try to send your headers before writing content.
And optionally use a relative Location
w.Header().Add("Location", "/")
w.WriteHeader(http.StatusFound)
fmt.Fprintf(w, "Token contents, %+v!", sa.GetAttributes())
I am writing a web application where I have a long running goroutine.
I want to delegate all HTTP requests to this goroutine via channels.
The pattern that I have come across is:
// Internal long running goroutine
for{
select{
case e := <-event: //web request
req := e.req
// do something
....
select {
case <-ctx.Done():
//log
default:
e.replyTo <- result
}
}
}
// Web handler
http.HandleFunc("/bar", func(w http.ResponseWriter, r *http.Request) {
//decode request etc
...
replyTo := make(chan interface{}, 1)
ctx, cancel := context.WithCancel(context.BackGround())
event <- Event{req: req, ctx: ctx, replyTo: replyTo}
select{
case <-time.After(time.Second):
cancel()
//return 500
case r := <-replyTo:
// return some response
}
})
I do see that there is one single go-routine at the end, so parallelism is lost but I am okay with it.
Is this pattern the right way of doing this?
What other approaches can be suggested?
Is this pattern the right way of doing this?
Assuming you are trying to manage where state in a single go routine, I would say no. I think it would be better to have some form of a state manager that is responsible for thread safety. Therefore the handler should take in something that can manage the state and simply expose a few methods to the handler.
type State interface{
Load() (string, error)
Save(something string) error
}
Decoupling the code will pay off for you later on. It will also allow unit tests for both the handler and the State that can be focused and readable.
My program are running fine with one connection per time, but not with concurrent connections.
I need all connections being rendered by one function, which will have all data I need in my service, and that is not working fine, so I ilustrated with the simple code below:
package main
import (
"encoding/json"
"fmt"
"github.com/gorilla/mux"
"github.com/rs/cors"
"net/http"
"reflect"
"time"
)
var Out struct {
Code int `json:"status"`
Message []interface{} `json:"message"`
}
func Clear(v interface{}) {
p := reflect.ValueOf(v).Elem()
p.Set(reflect.Zero(p.Type()))
}
func YourHandler(w http.ResponseWriter, r *http.Request) {
Clear(&Out.Message)
Out.Code = 0
// w.Header().Set("Content-Type", "application/json; charset=UTF-8")
w.Header().Set("Access-Control-Allow-Origin", "*")
w.Header().Set("Access-Control-Allow-Headers","Content-Type,access-control-allow-origin, access-control-allow-headers")
w.WriteHeader(http.StatusOK)
for i:=0; i<10; i++ {
Out.Code = Out.Code + 1
Out.Message = append(Out.Message, "Running...")
time.Sleep(1000 * time.Millisecond)
if err := json.NewEncoder(w).Encode(Out)
err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
}
}
func main() {
r := mux.NewRouter()
r.StrictSlash(true);
r.HandleFunc("/", YourHandler)
handler := cors.New(cors.Options{
AllowedOrigins: []string{"*"},
AllowCredentials: true,
Debug: true,
AllowedHeaders: []string{"Content-Type"},
AllowedMethods: []string{"GET"},
}).Handler(r)
fmt.Println("Working in localhost:5000")
http.ListenAndServe(":5000", handler)
}
If you run this code, you won't see anything wrong in one connection per time, but if you run it in another tab/browser/etc, at same time, because of the delay, the status code will not be from 1 to 10, but it will be shuffled with all calls.
So I guess that means it's not stateless, and I need it to be, so even with 300 connections at same time, it will always return status code from 1 to 10 in each one.
How can I do it? (As I said, it's a simple code, the structure and the render functions are in separeted packages from each other and of all data collection and)
Handlers are called concurrently by the net/http server. The server creates a goroutine for each client connection and calls handlers on those goroutines.
The Gorilla Mux is passive with respect to concurrency. The mux calls through to the registered application handler on whatever goroutine the mux is called on.
Use a sync.Mutex to limit execution to one goroutine at a time:
var mu sync.Mutex
func YourHandler(w http.ResponseWriter, r *http.Request) {
mu.Lock()
defer mu.Unlock()
Clear(&Out.Message)
Out.Code = 0
...
This is not a good solution given the time.Sleep calls in the handler. The server will process at most one request every 10 seconds.
A better solution is to declare Out as a local variable inside the handler function. With this change, here's no need for the mutex or to clear Out:
func YourHandler(w http.ResponseWriter, r *http.Request) {
var Out struct {
Code int `json:"status"`
Message []interface{} `json:"message"`
}
// w.Header().Set("Content-Type", "application/json; charset=UTF-8")
w.Header().Set("Access-Control-Allow-Origin", "*")
...
If it's not possible to move the declaration of Out, then copy the value to a local variable:
func YourHandler(w http.ResponseWriter, r *http.Request) {
Out := Out // local Out is copy of package-level Out
Clear(&Out.Message)
Out.Code = 0
...
Gorilla Mix uses Go's net/http server to process your http requests. Go creates a Go routine to service each of these incoming requests. If I understand your question correctly, you expect that the Go responses will have your custom status codes in order from 1 to 10 since you were expecting each request coming in synchronously in that order. Go routine's parallelism doesn't guarantee order of execution just like Java threads are if you're familiar with Java. So if Go routines were spawned for each of the requests created in the for 1-to-10 loop then, the routines will execute on its own without regard for order who goes and complete first. Each of these Go routines will serve your requests as it finishes. If you want to control the order of these requests processed in parallel but in order then you can use channels. Look at this link to control synchonization between your 10 Go routines for each of those http requests. https://gobyexample.com/channel-synchronization
First I would like to thanks ThunderCat and Ramil for the help, yours answers gave me a north to find the correctly answer.
A short answer is: Go don't have stateless connections, so I can't do what I was looking for.
Once that said, the reason why I think (based on RFC 7230) it doesn't have is because:
In a traditional web server application we have a program that handle the connections (Apache, nginx etc) and open a thread to the routed application, while in Go we have both in same application, so anything global are always shared between connections.
In languages that may work like Go (the application that opens a port and stay listen it), like C++, they are Object Oriented, so even public variables are inside a class, so you won't share it, since you have to create an instance of the class each time.
Create a thread would resolve the problem, but Go don't have it, instead it have Goroutines, more detail about it in:
https://translate.google.com/translate?sl=ko&tl=en&u=https%3A%2F%2Ftech.ssut.me%2F2017%2F08%2F20%2Fgoroutine-vs-threads%2F
After days on that and the help here, I'll fix it changing my struct to type and put it local, like that:
package main
import (
"encoding/json"
"fmt"
"github.com/gorilla/mux"
"github.com/rs/cors"
"net/http"
"reflect"
"time"
)
type Out struct {
Code int `json:"status"`
Message []interface{} `json:"message"`
}
func Clear(v interface{}) {
p := reflect.ValueOf(v).Elem()
p.Set(reflect.Zero(p.Type()))
}
func YourHandler(w http.ResponseWriter, r *http.Request) {
localOut := Out{0,nil}
// w.Header().Set("Content-Type", "application/json; charset=UTF-8")
w.Header().Set("Access-Control-Allow-Origin", "*")
w.Header().Set("Access-Control-Allow-Headers","Content-Type,access-control-allow-origin, access-control-allow-headers")
w.WriteHeader(http.StatusOK)
for i:=0; i<10; i++ {
localOut.Code = localOut.Code + 1
localOut.Message = append(localOut.Message, "Running...")
time.Sleep(1000 * time.Millisecond)
if err := json.NewEncoder(w).Encode(localOut)
err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
}
}
func main() {
r := mux.NewRouter()
r.StrictSlash(true);
r.HandleFunc("/", YourHandler)
handler := cors.New(cors.Options{
AllowedOrigins: []string{"*"},
AllowCredentials: true,
Debug: true,
AllowedHeaders: []string{"X-Session-Token","Content-Type"},
AllowedMethods: []string{"GET","POST","PUT","DELETE"},
}).Handler(r)
fmt.Println("Working in localhost:5000")
http.ListenAndServe(":5000", handler)
}
Of course that will take some weeks, so for now I put my application behind nginx and now it works as expected.
I have a very weird output ... let me post my code first then I will explain:
Under main function I declared
manageMux.HandleFunc("/info", info)
first I log in and redirect from "/login" to page "/":
func login(w http.ResponseWriter, r *http.Request) {
if r.Method == "GET" {
t, err := template.ParseFiles("manage/login.html")
checkError(err)
t.Execute(w, nil)
} else { //POST
r.ParseForm()
//do some authentications here
http.Redirect(w, r, "/", http.StatusFound)
}
}
Then I redirect to another page "/info" from current page "/" (which has only buttons):
func manage(w http.ResponseWriter, r *http.Request) {
t, err := template.ParseFiles("manage/manage.html")
checkError(err)
t.Execute(w, nil)
r.ParseForm()
if r.Form["Button"] != nil { //to get only POST actions from buttons
if r.Form["Button"][0] == "Log" {
http.Redirect(w, r, "/info", http.StatusFound)
}
}
}
At last, I made a template and would like to show on client side:
const tpl=`stuff inside`
type InfoDefault struct {
//stuff inside
}
func info(w http.ResponseWriter, r *http.Request) {
info := InfoDefault{
//stuff inside
}
t, err := template.New("info").Parse(tpl)
checkError(err)
err = t.Execute(os.Stdout, info)
checkError(err)
}
Now, the weird thing is, when I click the button on page "/", I got the error "http: multiple response.WriteHeader calls". At the same time a link called "found" shows up on the bottom of my page (weird!), and when I click the link "found", I got all my parsed template printed on the server side instead of webpage.
Does anyone know why...? And how to fix the error and print stuff on client webpage? Thank you!!!
As JimB already pointed out: your server gets confused because there are different status codes associated with both writing to http.ResponseWriter and the redirect. You can't do both at the same time.
I would actually like to expand more on how you can carry data over to the next page (assuming you are redirecting).
Headers
You can write some information to the request object and receive it on the destination page. Example:
func myHandler(w http.ResponseWriter, r *http.Request) {
w.Header().Set("My-Awesome-Header", "Rocks")
...
}
Session:
You are talking about access control as far as I can see, and I think persisting user data is better done through a session. Example: you can use a database or a session handler like https://github.com/gorilla/sessions. Check out this thread: Best practice with sessions (gorilla/sessions).
Cookies:
I'm not sure what kind of front-end you are using, but storing non-sensitive data on the cookie could be an option? Nothing beats this one (it has real choc-chip cookies in the examples ;-) ): https://astaxie.gitbooks.io/build-web-application-with-golang/content/en/06.1.html.
In your manage handler, you're executing the template which will write to the http.ResponseWriter and trigger an http.StatusOK (200) status code. You can't redirect after that, since that requires sending a different response code.
If you need to redirect, do it before executing the template.