echo server distorting image response body - go

Trying to write a simple echo server for images, but it distorts the file. What's going wrong?
package main
import (
"fmt"
"io"
"net/http"
)
type FlushWriter struct {
w io.Writer
}
func (fw *FlushWriter) Write(bytes []byte) (int, error) {
count, e := fw.w.Write(bytes)
fw.w.(http.Flusher).Flush()
return count, e
}
func main() {
http.HandleFunc("/", index)
fmt.Println("listening on 8000")
http.ListenAndServe(":8000", nil)
}
func index(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "image/jpeg")
fw := &FlushWriter{ w: w }
io.Copy(fw, r.Body)
}
And to test it.
$ curl --data-binary #image.jpg -o test.jpg localhost:8000

You're ignoring errors in your code, and missing a ErrBodyReadAfterClose.
You can't read from the http.Request.Body once you start writing to the http.ResponseWriter
http://golang.org/pkg/net/http/#pkg-variables
ErrBodyReadAfterClose is returned when reading a Request or Response
Body after the body has been closed. This typically happens when the
body is read after an HTTP Handler calls WriteHeader or Write on its
ResponseWriter
You'll need to buffer the image on the server before writing it back.
Aside from the fact that Go won't let you do this, even if you were to make a handler that worked, most clients would cause this to deadlock with images larger than the sum of all the buffers involved. This requires a client that can simultaniously send and receive, which very few, if any would do.

Related

How to exchange context value between servers?

I'm studying Golang and have question on context.
I want to send request from server1 to server2 with context and want to read context value set by server1.
But it seems context of sercer2 doesn't carry server1's context value.
When I send request to server1 by curl http://localhost:8080
server1's console.
Send request to http://localhost:8082
server2's console.
request coming in
<nil>
How can I retrieve context value set by server1?
And if it is possible, I also want to know whether it is a correct way of exchanging value's like authentication between servers.
Middleware pattern is more desirable?
Thank you.
Codes
package main
import (
"context"
"fmt"
"net/http"
"github.com/go-chi/chi/v5"
)
func main() {
r := chi.NewRouter()
r.HandleFunc("/", hello)
fmt.Println("Starting listening on port 8080...")
http.ListenAndServe(":8080", r)
}
// Context's key.
type Sample string
var sampleKey Sample = "sample"
func hello(w http.ResponseWriter, r *http.Request) {
fmt.Println("Send request to http://localhost:8082")
ctx := context.WithValue(context.Background(), sampleKey, "1234")
req, err := http.NewRequestWithContext(ctx, http.MethodGet, "http://localhost:8082", nil)
if err != nil {
fmt.Println("Error while sending request: ", err)
}
// Send request.
var c http.Client
c.Do(req)
}
package main
import (
"fmt"
"net/http"
"github.com/go-chi/chi/v5"
)
func main() {
r := chi.NewRouter()
r.HandleFunc("/", receive)
fmt.Println("Start listening on port 8082...")
http.ListenAndServe(":8082", r)
}
type Sample string
var sampleKey Sample
func receive(w http.ResponseWriter, r *http.Request) {
fmt.Println("request coming in")
fmt.Println(r.Context().Value(sampleKey))
}
The context is supposed to be used in a function call stack to share information and cancellation point between these functions. There is no automatic mechanism to serialize the value bound in the context over http. If you want to send data over http between these two servers, you mostly have three solution, you can:
encode that as an url parameter in the path somewhere
GET /hello/1234
send that in the http body, for example in JSON
POST /hello
{
"sample": 1234
}
as url key/value encoded as url parameters
/hello?sample=1234

How to make stateless connections with gorilla mux?

My program are running fine with one connection per time, but not with concurrent connections.
I need all connections being rendered by one function, which will have all data I need in my service, and that is not working fine, so I ilustrated with the simple code below:
package main
import (
"encoding/json"
"fmt"
"github.com/gorilla/mux"
"github.com/rs/cors"
"net/http"
"reflect"
"time"
)
var Out struct {
Code int `json:"status"`
Message []interface{} `json:"message"`
}
func Clear(v interface{}) {
p := reflect.ValueOf(v).Elem()
p.Set(reflect.Zero(p.Type()))
}
func YourHandler(w http.ResponseWriter, r *http.Request) {
Clear(&Out.Message)
Out.Code = 0
// w.Header().Set("Content-Type", "application/json; charset=UTF-8")
w.Header().Set("Access-Control-Allow-Origin", "*")
w.Header().Set("Access-Control-Allow-Headers","Content-Type,access-control-allow-origin, access-control-allow-headers")
w.WriteHeader(http.StatusOK)
for i:=0; i<10; i++ {
Out.Code = Out.Code + 1
Out.Message = append(Out.Message, "Running...")
time.Sleep(1000 * time.Millisecond)
if err := json.NewEncoder(w).Encode(Out)
err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
}
}
func main() {
r := mux.NewRouter()
r.StrictSlash(true);
r.HandleFunc("/", YourHandler)
handler := cors.New(cors.Options{
AllowedOrigins: []string{"*"},
AllowCredentials: true,
Debug: true,
AllowedHeaders: []string{"Content-Type"},
AllowedMethods: []string{"GET"},
}).Handler(r)
fmt.Println("Working in localhost:5000")
http.ListenAndServe(":5000", handler)
}
If you run this code, you won't see anything wrong in one connection per time, but if you run it in another tab/browser/etc, at same time, because of the delay, the status code will not be from 1 to 10, but it will be shuffled with all calls.
So I guess that means it's not stateless, and I need it to be, so even with 300 connections at same time, it will always return status code from 1 to 10 in each one.
How can I do it? (As I said, it's a simple code, the structure and the render functions are in separeted packages from each other and of all data collection and)
Handlers are called concurrently by the net/http server. The server creates a goroutine for each client connection and calls handlers on those goroutines.
The Gorilla Mux is passive with respect to concurrency. The mux calls through to the registered application handler on whatever goroutine the mux is called on.
Use a sync.Mutex to limit execution to one goroutine at a time:
var mu sync.Mutex
func YourHandler(w http.ResponseWriter, r *http.Request) {
mu.Lock()
defer mu.Unlock()
Clear(&Out.Message)
Out.Code = 0
...
This is not a good solution given the time.Sleep calls in the handler. The server will process at most one request every 10 seconds.
A better solution is to declare Out as a local variable inside the handler function. With this change, here's no need for the mutex or to clear Out:
func YourHandler(w http.ResponseWriter, r *http.Request) {
var Out struct {
Code int `json:"status"`
Message []interface{} `json:"message"`
}
// w.Header().Set("Content-Type", "application/json; charset=UTF-8")
w.Header().Set("Access-Control-Allow-Origin", "*")
...
If it's not possible to move the declaration of Out, then copy the value to a local variable:
func YourHandler(w http.ResponseWriter, r *http.Request) {
Out := Out // local Out is copy of package-level Out
Clear(&Out.Message)
Out.Code = 0
...
Gorilla Mix uses Go's net/http server to process your http requests. Go creates a Go routine to service each of these incoming requests. If I understand your question correctly, you expect that the Go responses will have your custom status codes in order from 1 to 10 since you were expecting each request coming in synchronously in that order. Go routine's parallelism doesn't guarantee order of execution just like Java threads are if you're familiar with Java. So if Go routines were spawned for each of the requests created in the for 1-to-10 loop then, the routines will execute on its own without regard for order who goes and complete first. Each of these Go routines will serve your requests as it finishes. If you want to control the order of these requests processed in parallel but in order then you can use channels. Look at this link to control synchonization between your 10 Go routines for each of those http requests. https://gobyexample.com/channel-synchronization
First I would like to thanks ThunderCat and Ramil for the help, yours answers gave me a north to find the correctly answer.
A short answer is: Go don't have stateless connections, so I can't do what I was looking for.
Once that said, the reason why I think (based on RFC 7230) it doesn't have is because:
In a traditional web server application we have a program that handle the connections (Apache, nginx etc) and open a thread to the routed application, while in Go we have both in same application, so anything global are always shared between connections.
In languages that may work like Go (the application that opens a port and stay listen it), like C++, they are Object Oriented, so even public variables are inside a class, so you won't share it, since you have to create an instance of the class each time.
Create a thread would resolve the problem, but Go don't have it, instead it have Goroutines, more detail about it in:
https://translate.google.com/translate?sl=ko&tl=en&u=https%3A%2F%2Ftech.ssut.me%2F2017%2F08%2F20%2Fgoroutine-vs-threads%2F
After days on that and the help here, I'll fix it changing my struct to type and put it local, like that:
package main
import (
"encoding/json"
"fmt"
"github.com/gorilla/mux"
"github.com/rs/cors"
"net/http"
"reflect"
"time"
)
type Out struct {
Code int `json:"status"`
Message []interface{} `json:"message"`
}
func Clear(v interface{}) {
p := reflect.ValueOf(v).Elem()
p.Set(reflect.Zero(p.Type()))
}
func YourHandler(w http.ResponseWriter, r *http.Request) {
localOut := Out{0,nil}
// w.Header().Set("Content-Type", "application/json; charset=UTF-8")
w.Header().Set("Access-Control-Allow-Origin", "*")
w.Header().Set("Access-Control-Allow-Headers","Content-Type,access-control-allow-origin, access-control-allow-headers")
w.WriteHeader(http.StatusOK)
for i:=0; i<10; i++ {
localOut.Code = localOut.Code + 1
localOut.Message = append(localOut.Message, "Running...")
time.Sleep(1000 * time.Millisecond)
if err := json.NewEncoder(w).Encode(localOut)
err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
}
}
}
func main() {
r := mux.NewRouter()
r.StrictSlash(true);
r.HandleFunc("/", YourHandler)
handler := cors.New(cors.Options{
AllowedOrigins: []string{"*"},
AllowCredentials: true,
Debug: true,
AllowedHeaders: []string{"X-Session-Token","Content-Type"},
AllowedMethods: []string{"GET","POST","PUT","DELETE"},
}).Handler(r)
fmt.Println("Working in localhost:5000")
http.ListenAndServe(":5000", handler)
}
Of course that will take some weeks, so for now I put my application behind nginx and now it works as expected.

golang server middlerware request cancellation

I created in the example at the bottom a little server which is running on port 3000. You can access it over "htto://localhost:3000/time". The whole Request is covered with two middlewares. First "cancelHandler" and Second "otherHandler" is called - which is responding with some dummy data after 4 seconds.
To my problem: When i request the page in a browser and then cancel the request (before the 4sec). The server is still handling the goroutine/request in the background. I spent already hours to find a solution on google but i can just not wrap my head around the context. (context.WithCancel()) I get that i have to create a chan and listen to it but how does this work with the requests. thats already a goroutine, do i have to create another goroutine in the request/goroutine? Also another question is, should i really use Context for that or is there an easier solution with the cancelNotifier?
Maybe someone can describe it for me and others which maybe have the same understanding problem.
Solution should be that the cancel Handler is stopping the goroutine/request, when the browser cancels the request.
Thank you very much for your time!
package main
import (
"log"
"net/http"
"time"
"fmt"
)
func otherHandler(format string) http.Handler {
fn := func(w http.ResponseWriter, r *http.Request) {
time.Sleep(time.Duration(4)*time.Second)
tm := time.Now().Format(format)
w.Write([]byte("The time is: " + tm))
fmt.Println("response:", "The time is: "+tm)
}
return http.HandlerFunc(fn)
}
func cancelHandler(h http.Handler) http.Handler {
fn := func(w http.ResponseWriter, r *http.Request) {
fmt.Println("start: called cancelHandler")
h.ServeHTTP(w, r)
fmt.Println("end: called cancelHandler")
}
return http.HandlerFunc(fn)
}
func main() {
mux := http.NewServeMux()
th := otherHandler(time.RFC1123)
mux.Handle("/time", cancelHandler(th))
log.Println("Listening...")
http.ListenAndServe(":3000", mux)
}
The only way to "stop" a function is to return from it. Thus, time.Sleep cannot be interrupted. Use a select statement instead:
package main
import (
"fmt"
"net/http"
"time"
)
func main() {
http.ListenAndServe(":3000", otherHandler(time.RFC1123))
}
func otherHandler(format string) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
select {
case <-time.After(4 * time.Second):
// time's up
case <-r.Context().Done():
// client gave up
return
}
tm := time.Now().Format(format)
w.Write([]byte("The time is: " + tm))
fmt.Println("response:", "The time is: "+tm)
}
}
In general, check the request context (or one that is derived from it) in strategic places. If the context is canceled, don't proceed any further and return.

Golang - User of package without selector

Please, I searched this a lot and after not been able to find, I am writing and not that I didn't try to search all over first. Couldn't get the right answer. I even tried to check Revel's function and couldn't get the answer from there as well.
When I run this program I get this error for line
./test.go:11: use of package http without selector
This error points at the line below where I have written
*http
inside the struct
Confusing part is that with test and dot I even get auto complete with VIM. So I don't know why is the error. Is it that it has to be somewhat like
*(net/http)
or something like that ?
package main
import (
"fmt"
"net/http"
)
type HandleHTTP struct {
*http
}
func handler(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Path is %s", r.URL.Path[1:])
}
func main() {
test := HandleHTTP{}
test.http.HandleFunc("/", handler)
test.http.ListenAndServe(":8080", nil)
}
If you want to have two or more instances serving from different ports you need to spin up two, or more, server. Would something like this, perhaps, work for you?
package main
import (
"fmt"
"net/http"
)
type HandleHTTP struct {
http *http.Server
}
func handler(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Path is %s", r.URL.Path[1:])
}
func main() {
mux1 := http.NewServeMux()
mux1.HandleFunc("/", handler)
test1 := HandleHTTP{http:&http.Server{Addr:":8081", Handler:mux1}}
mux2 := http.NewServeMux()
mux2.HandleFunc("/", handler)
test2 := HandleHTTP{http:&http.Server{Addr:":8082", Handler:mux2}}
// run the first one in a goroutine so that the second one is executed
go test1.http.ListenAndServe()
test2.http.ListenAndServe()
}

Go: Read method in http.Request

I'm reading Go Web Programming by Sau Sheong Chang. Here is a sample code for reading data from a request body:
import (
"fmt"
"net/http"
)
func bodyfunc(w http.ResponseWriter, r *http.Request) {
len := r.ContentLength
body := make([]byte, len)
r.Body.Read(body)
fmt.Fprintln(w, string(body))
}
func main() {
server := http.Server{
Addr: "127.0.0.1:8080",
}
http.HandleFunc("/body", bodyfunc)
server.ListenAndServe()
}
By definition, the Body field in the Request struct is actually an
io.ReadCloser interface. My question is: The Read method in this interface is just declared but not implemented. Meanwhile the code works well. The implementation of the Read method must have been done somewhere. Where is it?
The implementation is in https://golang.org/src/net/http/transfer.go#L637.
I used the delve debugger to find it.

Resources