I have a function that when run will give a user a URL to use to start an OAuth flow for a token exchange with my client.
I need to run a local HTTP server to accept the callback for the user. But I'm not sure how to then shutdown the HTTP server and close the function and move on once the flow is done.
func OAuth(request *OAuthRequest) {
http.HandleFunc("/callback", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte(`{"response": "unable to handle request"}`))
// Do something
// Finish up and shutdown HTTP server
})
err := http.ListenAndServe(net.JoinHostPort("","8080"), nil)
if err != nil {
log.Fatalln(err)
}
fmt.Println("Login at: www.example.com/123abc")
// Wait for user to finish flow
}
User calls .OAuth()
Local server is started and exposes /callback
User is given URL to use in browser
Remote server sends callback to localhost/callback
Local HTTP server handles response
Job done, shutdown local HTTP server
.OAuth() is complete
You need to create the HTTP server via http.Server if you want to be able to shut it down:
package main
import (
"context"
"fmt"
"net/http"
)
func main() {
mux := http.NewServeMux()
srv := http.Server{
Addr: "localhost:8080",
Handler: mux,
}
mux.HandleFunc("/callback", func(w http.ResponseWriter, r *http.Request) {
w.Write([]byte(`{"response": "unable to handle request"}`))
go srv.Shutdown(context.Background())
})
fmt.Println("Login at: www.example.com/123abc")
if err := srv.ListenAndServe(); err != http.ErrServerClosed {
panic(err)
}
}
Related
I'm setting up an OAuth2 flow from a CLI application, I'm working on. I need to create a temporary HTTP server for the provider to send the callback to, e.g. localhost:8080/callback
Once the provider has sent the details I need, I want to be able to shut the HTTP server down, just to keep everything clean. I think what I'm looking for is Routines and Wait Groups, but I'm still quite new to this area.
This is what I have so far. I have redacted the part that sends the user to the provider, as my main issue is simply how to shut down the HTTP server once the token variable has been captured.
Server starts
User is directed to authorization URL at the provider site
User approves the request
Provider directs user back to localhost:8080/callback
URL includes client-side only params so I have to server HTML to use JS to capture the values and send it back to the server
Server receives token and can then shutdown
package main
import (
"fmt"
"log"
"net/http"
"sync"
)
func main() {
// Start local HTTP serevr to listen for response
serverDone := &sync.WaitGroup{}
serverDone.Add(1)
Start(serverDone)
// ... Process to start OAuth2 flow
// User is directed to provider website
// User approves
// Provider direct user back to localhost/callback
serverDone.Wait()
}
func Start(wg *sync.WaitGroup) {
srv := &http.Server{Addr: ":8080"}
http.HandleFunc("/callback", func(w http.ResponseWriter, r *http.Request) {
token := r.URL.Query().Get("token")
if token != "" {
fmt.Println("Found Token:", token)
// Shut down server here
} else {
// Server HTML page to fetch token and return to server at /callback
}
})
go func() {
// let main know we are done cleaning up
defer wg.Done()
// ErrServerClosed on graceful close
if err := srv.ListenAndServe(); err != http.ErrServerClosed {
log.Fatalf("ListenAndServe(): %v", err)
}
}()
}
Use:
var ctxShutdown, cancel = context.WithCancel(context.Background())
Then:
cancel() // to say sorry, above.
// graceful-shutdown
err := srv.Shutdown(context.Background())
Try this:
package main
import (
"context"
"fmt"
"log"
"net/http"
"sync"
)
func main() {
serverDone := &sync.WaitGroup{}
serverDone.Add(1)
Start(serverDone)
serverDone.Wait()
fmt.Println("Done that.")
}
var ctxShutdown, cancel = context.WithCancel(context.Background())
func Start(wg *sync.WaitGroup) {
srv := &http.Server{Addr: ":8080"}
http.HandleFunc("/callback", func(w http.ResponseWriter, r *http.Request) {
select {
case <-ctxShutdown.Done():
fmt.Println("Sorry: Shuting down ...")
return
default:
}
token := r.URL.Query().Get("token")
if token != "" {
fmt.Println("Found Token:", token)
fmt.Println("Shuting down ...")
// Shut down server here
cancel() // to say sorry, above.
// graceful-shutdown
err := srv.Shutdown(context.Background())
if err != nil {
log.Println("server.Shutdown:", err)
}
} else {
fmt.Fprintln(w, "Hi") // Server HTML page to fetch token and return to server at /callback
}
})
go func() {
defer wg.Done()
if err := srv.ListenAndServe(); err != http.ErrServerClosed {
log.Fatalf("ListenAndServe(): %v", err)
}
fmt.Println("Bye.")
}()
}
Run and open http://127.0.0.1:8080/callback?token=2
Output:
Found Token: 2
Shuting down ...
Bye.
Done that.
I've googled all over for this but can't find anything.
I have a struct that takes in a http.Client and it sends several GET requests. In my tests I want to mock the responses so it's not sending real requests.
Currently I've figured out how to only serve 1 request, like this:
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusOK)
file, err := os.Open("./testdata/1.html")
if err != nil {
t.Error(err)
}
bytes, err := ioutil.ReadAll(file)
if err != nil {
t.Error(err)
}
w.Write(bytes)
}))
ts.Client() // Now I can inject this client into my struct.
So once that response is mocked out and the http client is performs a new request, my tests are sending out real requests after that.
How do I allow for several handlers so I can mock several responses upon calling http.Client.Get(...)?
Since the original question uses httptest.NewServer - you can register a ServeMux on the httptest.Server function, and then you can add several routes to that mux:
mux := http.NewServeMux()
mux.HandleFunc("/someroute/", func(res http.ResponseWriter, req *http.Request) {
...do some stuff...
})
mux.HandleFunc("/someotherroute/", func(res http.ResponseWriter, req *http.Request) {
...do other stuff...
})
ts := httptest.NewServer(mux)
defer ts.Close()
ServeMux.Handle can be used to setup a server to handle multiple requests like in this example.
package main
import (
"log"
"net/http"
)
const addr = "localhost:12345"
func main() {
mux := http.NewServeMux()
mux.HandleFunc("/hello", HandleHello)
// other handlers can be assigned to separate paths
log.Printf("Now listening on %s...\n", addr)
server := http.Server{Handler: mux, Addr: addr}
log.Fatal(server.ListenAndServe())
}
func HandleHello(w http.ResponseWriter, r *http.Request) {
log.Printf("Hello!")
}
But to be honest you probably just want to abstract the http.Client behind an interface that you've created, and then stub that with a test implementation that returns exactly what you want. By doing this you avoid the overhead of http communication in your tests.
I wrote a small checkbook ledger in Go as a server that runs in localhost and opens the default browser to a small web app front-end (https://bitbucket.org/grkuntzmd/checks-and-balances).
To automatically shut down the server when the browser tab is closed, I have the browser call a "heartbeat" URL every few seconds. If that heartbeat does not arrive, the server uses (*Server) Shutdown to stop running.
Is there any way to do the same thing using contexts (https://golang.org/pkg/context/)? It is my understanding from watching this episode of JustForFunc that a context passed down to a handler will be notified if the client cancels a request.
Instead of sending a "heartbeat" request every so often, you could take advantage of server-sent events.
Server-sent events is a web technology where the browser makes a HTTP request and keeps the connection open to receive events from the server. This could replace your need for repeated heartbeat requests by having the server shutdown when the connection to the event source is closed.
Here's a basic server implementation in Go:
package main
import (
"fmt"
"log"
"net/http"
"time"
)
func main() {
http.HandleFunc("/heartbeat", func(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "text/event-stream")
w.Header().Set("Cache-Control", "no-cache")
w.Header().Set("Connection", "keep-alive")
flusher, ok := w.(http.Flusher)
if !ok {
http.Error(w, "your browser doesn't support server-sent events")
return
}
// Send a comment every second to prevent connection timeout.
for {
_, err := fmt.Fprint(w, ": ping")
if err != nil {
log.Fatal("client is gone, shutting down")
return
}
flusher.Flush()
time.Sleep(time.Second)
}
})
fmt.Println(http.ListenAndServe(":1323", nil))
}
See using server-sent events for a guide on the client side.
Open a websocket connection from the page. On the server, exit when the websocket connection is closed:
func wsHandler(w http.ResponseWriter, r *http.Request) {
conn, err := upgrader.Upgrade(w, r, nil)
if err != nil {
log.Println(err)
return
}
defer c.Close()
// Shutdown on exit from handler
defer server.Shutdown(context.Background())
// Read messages from client. NextReader returns an error
// when the client shuts down.
for {
if _, _, err := c.NextReader(); err != nil {
break
}
}
}
Add this line of code to the client application:
var conn = new WebSocket("ws://" + document.location.host + "/ws");
Ensure that conn has lifetime of the page.
This is somewhat similar to the SSE solution suggested in another answer, but has the advantage of working in more browsers.
There's no need to send a heartbeat because this is on local host.
I'm in the process of building a little command line based Go bot that interacts with the Instagram API.
The Instagram API is OAuth based, and so not overly great for command line based apps.
To get around this, I am opening the appropriate authorization URL in the browser and using a local server I spin up for the redirect URI - this way I can capture and gracefully show the access token as opposed to the user needing to get this from the URL manually.
So far so good, the application can successfully open the browser to the authorisation URL, you authorise it and it redirects you to the local HTTP server.
Now, I have no need for the HTTP server after the access token has been displayed to the user and so I am wanting to manually shut the server down after doing this.
To do this, I drew inspiration from this answer and drummed up the below:
package main
import (
"fmt"
"io"
"log"
"net/http"
"os/exec"
"runtime"
"time"
)
var client_id = "my_client_id"
var client_secret = "my_client_secret"
var redirect_url = "http://localhost:8000/instagram/callback"
func main() {
srv := startHttpServer()
openbrowser(fmt.Sprintf("https://api.instagram.com/oauth/authorize/?client_id=%v&redirect_uri=%v&response_type=code", client_id, redirect_url))
// Backup to gracefully shutdown the server
time.Sleep(20 * time.Second)
if err := srv.Shutdown(nil); err != nil {
panic(err) // failure/timeout shutting down the server gracefully
}
}
func showTokenToUser(w http.ResponseWriter, r *http.Request, srv *http.Server) {
io.WriteString(w, fmt.Sprintf("Your access token is: %v", r.URL.Query().Get("code")))
if err := srv.Shutdown(nil); err != nil {
log.Fatal(err) // failure/timeout shutting down the server gracefully
}
}
func startHttpServer() *http.Server {
srv := &http.Server{Addr: ":8000"}
http.HandleFunc("/instagram/callback", func(w http.ResponseWriter, r *http.Request) {
showTokenToUser(w, r, srv)
})
go func() {
if err := srv.ListenAndServe(); err != nil {
// cannot panic, because this probably is an intentional close
log.Printf("Httpserver: ListenAndServe() error: %s", err)
}
}()
// returning reference so caller can call Shutdown()
return srv
}
func openbrowser(url string) {
var err error
switch runtime.GOOS {
case "linux":
err = exec.Command("xdg-open", url).Start()
case "windows":
err = exec.Command("rundll32", "url.dll,FileProtocolHandler", url).Start()
case "darwin":
err = exec.Command("open", url).Start()
default:
err = fmt.Errorf("unsupported platform")
}
if err != nil {
log.Fatal(err)
}
}
However, the above causes this error:
2017/11/23 16:02:03 Httpserver: ListenAndServe() error: http: Server closed
2017/11/23 16:02:03 http: panic serving [::1]:61793: runtime error: invalid memory address or nil pointer dereference
If I comment out these lines in the handler then it works flawlessly, albeit without shutting down the server when I hit the callback route:
if err := srv.Shutdown(nil); err != nil {
log.Fatal(err) // failure/timeout shutting down the server gracefully
}
Where am I going wrong? What do I need to change so that I can shut the server down when I hit the callback route, after displaying the text to the user.
You may use context.WithCancel:
package main
import (
"context"
"io"
"log"
"net/http"
)
func main() {
ctx, cancel := context.WithCancel(context.Background())
http.HandleFunc("/quit", func(w http.ResponseWriter, r *http.Request) {
io.WriteString(w, "Bye\n")
cancel()
})
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
io.WriteString(w, "Hi\n")
})
srv := &http.Server{Addr: ":8080"}
go func() {
err := srv.ListenAndServe()
if err != http.ErrServerClosed {
log.Println(err)
}
}()
<-ctx.Done() // wait for the signal to gracefully shutdown the server
// gracefully shutdown the server:
// waiting indefinitely for connections to return to idle and then shut down.
err := srv.Shutdown(context.Background())
if err != nil {
log.Println(err)
}
log.Println("done.")
}
The same Context may be passed to functions running in different goroutines:
"Contexts are safe for simultaneous use by multiple goroutines."
You may use the same context - if you don't want to wait extera:
package main
import (
"context"
"io"
"log"
"net/http"
)
func main() {
ctx, cancel := context.WithCancel(context.Background())
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
io.WriteString(w, "Hi\n")
})
http.HandleFunc("/quit", func(w http.ResponseWriter, r *http.Request) {
io.WriteString(w, "Bye\n")
cancel()
})
srv := &http.Server{Addr: ":8080"}
go func() {
if err := srv.ListenAndServe(); err != nil {
log.Printf("Httpserver: ListenAndServe() error: %s", err)
}
}()
<-ctx.Done()
// if err := srv.Shutdown(ctx); err != nil && err != context.Canceled {
// log.Println(err)
// }
log.Println("done.")
}
Server.Shutdown:
Shutdown gracefully shuts down the server without interrupting any active connections. Shutdown works by first closing all open listeners, then closing all idle connections, and then waiting indefinitely for connections to return to idle and then shut down. If the provided context expires before the shutdown is complete, Shutdown returns the context's error, otherwise it returns any error returned from closing the Server's underlying Listener(s).
When Shutdown is called, Serve, ListenAndServe, and ListenAndServeTLS immediately return ErrServerClosed. Make sure the program doesn't exit and waits instead for Shutdown to return.
Shutdown does not attempt to close nor wait for hijacked connections such as WebSockets. The caller of Shutdown should separately notify such long-lived connections of shutdown and wait for them to close, if desired. See RegisterOnShutdown for a way to register shutdown notification functions.
Once Shutdown has been called on a server, it may not be reused; future calls to methods such as Serve will return ErrServerClosed.
Shutdown function accepts parameter ctx context.Context. Try to pass it an empty context.
ctx := context.Background()
Also:
When Shutdown is called, Serve, ListenAndServe, and ListenAndServeTLS immediately return ErrServerClosed. Make sure the program doesn't exit and waits instead for Shutdown to return.
I am creating a streaming API similar to the Twitter firehose/streaming API.
As far as I can gather this is based on HTTP connections that are kept open and when the backend gets data it then writes to the chucked HTTP connection. It seems that any code I write closes the HTTP connection as soon as anything connects.
Is there a way to keep this open at all?
func startHTTP(pathPrefix string) {
log.Println("Starting HTTPS Server")
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
// Wait here until a write happens to w
// Or we timeout, we can reset this timeout after each write
})
log.Print("HTTPS listening on :5556")
log.Fatal(http.ListenAndServeTLS(":5556", pathPrefix+".crt", pathPrefix+".key", nil))
}
When you want to send HTTP response to client not immediately but after some event, it's called long polling.
Here's simple example of long polling with request cancellation on client disconnect:
package main
import (
"context"
"fmt"
"net/http"
"time"
)
func longOperation(ctx context.Context, ch chan<- string) {
// Simulate long operation.
// Change it to more than 10 seconds to get server timeout.
select {
case <-time.After(time.Second * 3):
ch <- "Successful result."
case <-ctx.Done():
close(ch)
}
}
func handler(w http.ResponseWriter, _ *http.Request) {
notifier, ok := w.(http.CloseNotifier)
if !ok {
panic("Expected http.ResponseWriter to be an http.CloseNotifier")
}
ctx, cancel := context.WithCancel(context.Background())
ch := make(chan string)
go longOperation(ctx, ch)
select {
case result := <-ch:
fmt.Fprint(w, result)
cancel()
return
case <-time.After(time.Second * 10):
fmt.Fprint(w, "Server is busy.")
case <-notifier.CloseNotify():
fmt.Println("Client has disconnected.")
}
cancel()
<-ch
}
func main() {
http.HandleFunc("/", handler)
http.ListenAndServe("localhost:8080", nil)
}
URLs:
anonymous struct and empty struct.
Send a chunked HTTP response from a Go server.
Go Concurrency Patterns: Context.
Gists:
Golang long polling example.
Golang long polling example with request cancellation.