How to exchange context value between servers? - go

I'm studying Golang and have question on context.
I want to send request from server1 to server2 with context and want to read context value set by server1.
But it seems context of sercer2 doesn't carry server1's context value.
When I send request to server1 by curl http://localhost:8080
server1's console.
Send request to http://localhost:8082
server2's console.
request coming in
<nil>
How can I retrieve context value set by server1?
And if it is possible, I also want to know whether it is a correct way of exchanging value's like authentication between servers.
Middleware pattern is more desirable?
Thank you.
Codes
package main
import (
"context"
"fmt"
"net/http"
"github.com/go-chi/chi/v5"
)
func main() {
r := chi.NewRouter()
r.HandleFunc("/", hello)
fmt.Println("Starting listening on port 8080...")
http.ListenAndServe(":8080", r)
}
// Context's key.
type Sample string
var sampleKey Sample = "sample"
func hello(w http.ResponseWriter, r *http.Request) {
fmt.Println("Send request to http://localhost:8082")
ctx := context.WithValue(context.Background(), sampleKey, "1234")
req, err := http.NewRequestWithContext(ctx, http.MethodGet, "http://localhost:8082", nil)
if err != nil {
fmt.Println("Error while sending request: ", err)
}
// Send request.
var c http.Client
c.Do(req)
}
package main
import (
"fmt"
"net/http"
"github.com/go-chi/chi/v5"
)
func main() {
r := chi.NewRouter()
r.HandleFunc("/", receive)
fmt.Println("Start listening on port 8082...")
http.ListenAndServe(":8082", r)
}
type Sample string
var sampleKey Sample
func receive(w http.ResponseWriter, r *http.Request) {
fmt.Println("request coming in")
fmt.Println(r.Context().Value(sampleKey))
}

The context is supposed to be used in a function call stack to share information and cancellation point between these functions. There is no automatic mechanism to serialize the value bound in the context over http. If you want to send data over http between these two servers, you mostly have three solution, you can:
encode that as an url parameter in the path somewhere
GET /hello/1234
send that in the http body, for example in JSON
POST /hello
{
"sample": 1234
}
as url key/value encoded as url parameters
/hello?sample=1234

Related

Golang: Intercepting and Mocking an HTTP Response with httptest

I've looked into various different tools that can be used for mock testing in golang, but I'm trying to accomplish this task using httptest. In particular, I have a function as such:
type contact struct {
username string
number int
}
func getResponse(c contact) string {
url := fmt.Sprintf("https://mywebsite/%s", c.username)
req, err := http.NewRequest(http.MethodGet, url, nil)
// error checking
resp, err := http.DefaultClient.Do(req)
// error checking
return response
}
A lot of the documentation I've read seems to require creating a client interface or a custom transport. Is there no way to mock a response in a test file without changing this main code at all? I want to keep my client, response, and all the related details within the getResponse function. I could have the wrong idea, but I'm trying to find a way to intercept the http.DefaultClient.Do(req) call and return a custom response, is that possible?
https://pkg.go.dev/net/http/httptest#example-Server is a good example for your use case with a small refactoring of your code.
You just have to change the getResponse() by getResponse(url string) to be able to give the server mock url.
I've read seems to require creating a client interface
without changing this main code at all
Keeping your code clean is a good practice and you'll finally get used to it, a testable code is cleaner and a cleaner code is more testable, so don't worry to change your code (using interfaces) so it can accept mock objects.
Your code in its simplest form can be like this:
package main
import (
"fmt"
"net/http"
)
type contact struct {
username string
number int
}
type Client interface {
Do(req *http.Request) (*http.Response, error)
}
func main() {
getResponse(http.DefaultClient, contact{})
}
func getResponse(client Client, c contact) string {
url := fmt.Sprintf("https://mywebsite/%s", c.username)
req, _ := http.NewRequest(http.MethodGet, url, nil)
// error checking
resp, _ := http.DefaultClient.Do(req)
// error checking and response processing
return response
}
And your test can be like this:
package main
import (
"net/http"
"testing"
)
type mockClient struct {
}
// Do function will cause mockClient to implement the Client interface
func (tc mockClient) Do(req *http.Request) (*http.Response, error) {
return &http.Response{}, nil
}
func TestGetResponse(t *testing.T) {
client := new(mockClient)
getResponse(client, contact{})
}
But if you prefer to use httptest:
package main
import (
"fmt"
"io"
"net/http"
"net/http/httptest"
)
type contact struct {
username string
number int
}
func main() {
fmt.Println(getResponse(contact{}))
}
func getResponse(c contact) string {
// Make a test server
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
fmt.Fprintln(w, "your response")
}))
defer ts.Close()
// You should still set your base url
base_url := ts.URL
url := fmt.Sprintf("%s/%s", base_url, c.username)
req, _ := http.NewRequest(http.MethodGet, url, nil)
// Use ts.Client() instead of http.DefaultClient in your tests.
resp, _ := ts.Client().Do(req)
// Processing the response
response, _ := io.ReadAll(resp.Body)
resp.Body.Close()
return string(response)
}

Make httputil.ReverseProxy Foward Response Stream Correctly

I have reverse proxies in my main web-server that are dedicated to a certain micro-service and handle forward requests to their appropriate micro-services.
func newTrimPrefixReverseProxy(target *url.URL, prefix string) *httputil.ReverseProxy {
director := func(req *http.Request) {
// ... trims prefix from request path and prepends the path of the target url
}
return &httputil.ReverseProxy{Director: director}
}
This has worked perfectly for pure JSON responses, but I have ran into issues recently when trying to serve content (stream responses) through the reverse proxy. The means for serving the content is irrelevant, the (video) content is served as intended when the service is accessed directly and not through the reverse proxy.
Serving the content:
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
http.ServeContent(w, r, "video.mp4", time.Now().Add(time.Hour*24*365*12*9*-1), videoReadSeeker)
})
Again, the videoReadSeeker and how the content is served is not the issue, the issue is having my response relayed as intended to the requester through the reverse proxy; when accessing the service directly, the video shows up and I can scrub it to my heart's content.
Note that the response for data the content is received (http status, headers), but the content stream in the response body is not.
How can I make sure that the reverse proxy handles streamed responses as intended for the content?
Do you get the same results when using:
package main
import (
"log"
"net/http"
"net/http/httputil"
"net/url"
)
func main() {
u, err := url.Parse("http://localhost:8080/asdfasdf")
if err != nil {
log.Fatal("url.Parse: %v", err)
}
proxy := httputil.NewSingleHostReverseProxy(u)
log.Printf("Listening at :8081")
if err := http.ListenAndServe(":8081", proxy); err != nil {
log.Fatal("ListenAndServe: %v", err)
}
}
Ultimately these are the same implementation under the hood, but the director provided here ensures that some of the expected headers exist that you will need for some proxy features to function as expected.

How to make stateless connections with gorilla mux?

My program are running fine with one connection per time, but not with concurrent connections.
I need all connections being rendered by one function, which will have all data I need in my service, and that is not working fine, so I ilustrated with the simple code below:
package main
import (
"encoding/json"
"fmt"
"github.com/gorilla/mux"
"github.com/rs/cors"
"net/http"
"reflect"
"time"
)
var Out struct {
Code int `json:"status"`
Message []interface{} `json:"message"`
}
func Clear(v interface{}) {
p := reflect.ValueOf(v).Elem()
p.Set(reflect.Zero(p.Type()))
}
func YourHandler(w http.ResponseWriter, r *http.Request) {
Clear(&Out.Message)
Out.Code = 0
// w.Header().Set("Content-Type", "application/json; charset=UTF-8")
w.Header().Set("Access-Control-Allow-Origin", "*")
w.Header().Set("Access-Control-Allow-Headers","Content-Type,access-control-allow-origin, access-control-allow-headers")
w.WriteHeader(http.StatusOK)
for i:=0; i<10; i++ {
Out.Code = Out.Code + 1
Out.Message = append(Out.Message, "Running...")
time.Sleep(1000 * time.Millisecond)
if err := json.NewEncoder(w).Encode(Out)
err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
}
}
func main() {
r := mux.NewRouter()
r.StrictSlash(true);
r.HandleFunc("/", YourHandler)
handler := cors.New(cors.Options{
AllowedOrigins: []string{"*"},
AllowCredentials: true,
Debug: true,
AllowedHeaders: []string{"Content-Type"},
AllowedMethods: []string{"GET"},
}).Handler(r)
fmt.Println("Working in localhost:5000")
http.ListenAndServe(":5000", handler)
}
If you run this code, you won't see anything wrong in one connection per time, but if you run it in another tab/browser/etc, at same time, because of the delay, the status code will not be from 1 to 10, but it will be shuffled with all calls.
So I guess that means it's not stateless, and I need it to be, so even with 300 connections at same time, it will always return status code from 1 to 10 in each one.
How can I do it? (As I said, it's a simple code, the structure and the render functions are in separeted packages from each other and of all data collection and)
Handlers are called concurrently by the net/http server. The server creates a goroutine for each client connection and calls handlers on those goroutines.
The Gorilla Mux is passive with respect to concurrency. The mux calls through to the registered application handler on whatever goroutine the mux is called on.
Use a sync.Mutex to limit execution to one goroutine at a time:
var mu sync.Mutex
func YourHandler(w http.ResponseWriter, r *http.Request) {
mu.Lock()
defer mu.Unlock()
Clear(&Out.Message)
Out.Code = 0
...
This is not a good solution given the time.Sleep calls in the handler. The server will process at most one request every 10 seconds.
A better solution is to declare Out as a local variable inside the handler function. With this change, here's no need for the mutex or to clear Out:
func YourHandler(w http.ResponseWriter, r *http.Request) {
var Out struct {
Code int `json:"status"`
Message []interface{} `json:"message"`
}
// w.Header().Set("Content-Type", "application/json; charset=UTF-8")
w.Header().Set("Access-Control-Allow-Origin", "*")
...
If it's not possible to move the declaration of Out, then copy the value to a local variable:
func YourHandler(w http.ResponseWriter, r *http.Request) {
Out := Out // local Out is copy of package-level Out
Clear(&Out.Message)
Out.Code = 0
...
Gorilla Mix uses Go's net/http server to process your http requests. Go creates a Go routine to service each of these incoming requests. If I understand your question correctly, you expect that the Go responses will have your custom status codes in order from 1 to 10 since you were expecting each request coming in synchronously in that order. Go routine's parallelism doesn't guarantee order of execution just like Java threads are if you're familiar with Java. So if Go routines were spawned for each of the requests created in the for 1-to-10 loop then, the routines will execute on its own without regard for order who goes and complete first. Each of these Go routines will serve your requests as it finishes. If you want to control the order of these requests processed in parallel but in order then you can use channels. Look at this link to control synchonization between your 10 Go routines for each of those http requests. https://gobyexample.com/channel-synchronization
First I would like to thanks ThunderCat and Ramil for the help, yours answers gave me a north to find the correctly answer.
A short answer is: Go don't have stateless connections, so I can't do what I was looking for.
Once that said, the reason why I think (based on RFC 7230) it doesn't have is because:
In a traditional web server application we have a program that handle the connections (Apache, nginx etc) and open a thread to the routed application, while in Go we have both in same application, so anything global are always shared between connections.
In languages that may work like Go (the application that opens a port and stay listen it), like C++, they are Object Oriented, so even public variables are inside a class, so you won't share it, since you have to create an instance of the class each time.
Create a thread would resolve the problem, but Go don't have it, instead it have Goroutines, more detail about it in:
https://translate.google.com/translate?sl=ko&tl=en&u=https%3A%2F%2Ftech.ssut.me%2F2017%2F08%2F20%2Fgoroutine-vs-threads%2F
After days on that and the help here, I'll fix it changing my struct to type and put it local, like that:
package main
import (
"encoding/json"
"fmt"
"github.com/gorilla/mux"
"github.com/rs/cors"
"net/http"
"reflect"
"time"
)
type Out struct {
Code int `json:"status"`
Message []interface{} `json:"message"`
}
func Clear(v interface{}) {
p := reflect.ValueOf(v).Elem()
p.Set(reflect.Zero(p.Type()))
}
func YourHandler(w http.ResponseWriter, r *http.Request) {
localOut := Out{0,nil}
// w.Header().Set("Content-Type", "application/json; charset=UTF-8")
w.Header().Set("Access-Control-Allow-Origin", "*")
w.Header().Set("Access-Control-Allow-Headers","Content-Type,access-control-allow-origin, access-control-allow-headers")
w.WriteHeader(http.StatusOK)
for i:=0; i<10; i++ {
localOut.Code = localOut.Code + 1
localOut.Message = append(localOut.Message, "Running...")
time.Sleep(1000 * time.Millisecond)
if err := json.NewEncoder(w).Encode(localOut)
err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
}
}
func main() {
r := mux.NewRouter()
r.StrictSlash(true);
r.HandleFunc("/", YourHandler)
handler := cors.New(cors.Options{
AllowedOrigins: []string{"*"},
AllowCredentials: true,
Debug: true,
AllowedHeaders: []string{"X-Session-Token","Content-Type"},
AllowedMethods: []string{"GET","POST","PUT","DELETE"},
}).Handler(r)
fmt.Println("Working in localhost:5000")
http.ListenAndServe(":5000", handler)
}
Of course that will take some weeks, so for now I put my application behind nginx and now it works as expected.

How do you send websocket data after a page is rendered in Golang?

I am new to Golang and am trying to send data using web-sockets to a page. I have a handler and I want to be able to serve a file and after it is rendered send it a message. This is some code that I have now.
package main
import (
"github.com/gorilla/websocket"
"log"
"fmt"
"net/http"
)
var upgrader = websocket.Upgrader{
ReadBufferSize: 1024,
WriteBufferSize: 1024,
}
func serveRoot(w http.ResponseWriter, r *http.Request) {
http.ServeFile(w, r, "views/index.html")
_, err := upgrader.Upgrade(w, r, nil)
if err != nil {
log.Println(err)
return
}
}
func main() {
http.HandleFunc("/", serveRoot)
fmt.Println("Started")
if err := http.ListenAndServe(":9090", nil); err != nil {
log.Fatal("ListenAndServe:", err)
}
}
The problem is that using the gorilla library I have no idea how to send data and I am getting some output when I load the page.
2018/01/23 08:35:24 http: multiple response.WriteHeader calls
2018/01/23 08:35:24 websocket: the client is not using the websocket protocol: 'upgrade' token not found in 'Connection' header
2018/01/23 08:35:24 http: multiple response.WriteHeader calls
2018/01/23 08:35:24 websocket: 'Origin' header value not allowed
Intention: Send some data after the page is rendered, then (later) hook it up to stdin/stderr
Disclaimer: I am just learning to code, so it would be a great help is you could take that into consideration and not be too vague.
So, as some of the comments mentioned, you can't upgrade a connection that has already been served html. The simple way to do this is just have one endpoint for your websockets, and one endpoint for your html.
So in your example, you might do:
http.HandleFunc("/", serveHtml)
http.HandleFunc("/somethingElse", serveWebsocket)
Where serveHtml has your http.ServeFile call, and serveWebsocket has the upgrading and wotnot.

Gorilla Mux router from inside handler only works once then gives 404 page not found

I'm using Gorilla mux as my router and I'm having a very strange behaviour. On the first request to the server, I get a valid response. But on subsequent requests, I receive a 404 page not found. There are no errors in the console.
My code is pretty straightforward (it can be copy-pasted to test it right out):
package main
import (
"fmt"
"github.com/gorilla/mux"
"log"
"net/http"
)
func main() {
router := mux.NewRouter()
router.HandleFunc("/", RootHandler).Name("root")
http.Handle("/", router)
log.Println("Listening on port 1337...")
if err := http.ListenAndServe(":1337", nil); err != nil {
log.Fatal("http.ListenAndServe: ", err)
}
}
func RootHandler(w http.ResponseWriter, r *http.Request) {
content := "Welcome to "
rootUrl, err := mux.CurrentRoute(r).Subrouter().Get("root").URL()
if err != nil {
log.Printf("mux.CurrentRoute(r).Subrouter().Get(\"root\").URL(): ", err)
}
response := content + rootUrl.String()
fmt.Fprintf(w, response)
}
After some code commenting and tests, it seems the following line is the culprit:
rootUrl, err := mux.CurrentRoute(r).Subrouter().Get("root").URL()
This method of getting the router inside the handler using the current request comes from another StackOverflow post: How to call a route by its name from inside a handler?
But for a strange reason, it only works once:
shell-1$ go run servertest.go
2014/10/30 13:31:34 Listening on port 1337...
shell-2$ curl http://127.0.0.1:1337
Welcome to /
shell-2$ curl http://127.0.0.1:1337
404 page not found
As you can see, there are no errors in the console.
Does someone have an idea of why it only works once ?
The problem is Subrouter() isn't made to return the router, but to create one, thus it changes the matcher of the router it is called on, making you lose the handler.
You could try passing the router to the handler using closures instead.
func RootHandler(router *mux.Router) func(http.ResponseWriter, *http.Request) {
return func(w http.ResponseWriter, r *http.Request) {
...
}
}
i run to this problem and fixe it by re initiating the methods
//create a subrouter separately \
subRoute := mux.CurrentRoute(req).Subrouter() \
//Call the Route u want and store the URL
url, err := subRoute.Get("check_authorization").URL("id", key, "password", token)
// re-initiate the method to GET or whatever u had before
subRoute.Methods(http.MethodGet)

Resources