I am fairly new to golang and in one of the handler functions I'm collecting the data using channels from different goroutines and now wanted to return the array of result as a response object
So I have given a return type as the struct details but it's throwing an error
if this is not the way to return the slice of struct as response then how can I return my results array as a response to post my request
Error:
cannot use homePage (type func(http.ResponseWriter, *http.Request) []details) as type func(http.ResponseWriter, *http.Request) in argument to http.HandleFunc
Handler Func:
func homePage(w http.ResponseWriter, r *http.Request) []details{
var wg sync.WaitGroup
for _, url := range urls {
out, err := json.Marshal(url)
if err != nil {
panic (err)
}
wg.Add(1)
go do_calc(ch,client,string(out),&wg)
}
fmt.Println("Returning Response")
go func() {
for v := range ch {
results = append(results, v)
}
}()
wg.Wait()
close(ch)
return results
}
So, your question is two fold. Firstly the reason for the error is because if you look at the documentation here you can see that http.HandleFunc has the following definition.
func HandleFunc(pattern string, handler func(ResponseWriter, *Request))
Since your function has a return with []details it does not meet the requirements.
So going off the other part of your question;
if this is not the way to return the slice of struct as response then how can I return my results array as a response to post my request
To solve your problem we need to write your data back to the response, you'll notice in the arguments passed to your HandleFunc you have a ResponseWriter, where you can use the Write() method to return your response
Not entirely sure how you want to display your result but you could do it with JSON easy enough.
b, err := json.Marshal(results)
if err != nil {
// Handle Error
}
w.Write(b)
Related
I'm using the net/http package and wondering how I can exit the handler from anywhere in the code. Say I have this code:
func main() {
http.HandleFunc("/", handler)
http.ListenAndServe(":8080", nil)
}
func handler(w http.ResponseWriter, r *http.Request){
err := checkSomeThing(w, r)
if err != nil {
return
}
fmt.Println("End of Handler.")
return
}
func checkSomeThing(w http.ResponseWriter, r *http.Request) error{
http.Error(w, "Bad Request!", http.StatusBadRequest)
return errors.New("bad request")
}
Ideally I'd like to exit the handler from within the checkSomeThing function without having to return and then return again up a level, which will get worse as the application grows. This is purely for code readability.
The idiomatic approach is to check error returns up the call chain.
To exit the the handler from anywhere, use panic and recover following the pattern in the encoding/json package.
Define a unique type for panic:
type httpError struct {
status int
message string
}
Write a function to be used in a defer statement. The function checks for the type and handles the error as appropriate. Otherwise, the function continues the panic.
func handleExit(w http.ResponseWriter) {
if r := recover(); r != nil {
if he, ok := r.(httpError); ok {
http.Error(w, he.message, he.status)
} else {
panic(r)
}
}
}
Write a helper function for the call to panic:
func exit(status int, message string) {
panic(httpError{status: status, message: message})
}
Use the functions like this:
func example() {
exit(http.StatusBadRequest, "Bad!")
}
func someHandler(w http.ResponseWriter, r *http.Request) {
defer handleExit(w)
example()
}
My answer:
First, the common pattern established in Golang is to have errors "bubble up" from callee back to caller as a return value. It has a lot of advantages with regards to readability and re-use. The side effect is that there's a lot of if err != nil {return} checks.
My suggestion if you really want to break from the norm
I'm going to pitch an idea, that I don't think is common or standard with respect to golang coding styles and patterns. But I didn't see anything online suggesting this was catastrophic. Let's see what I get in the comments to say this is awful.
You could use runtime.Goexit() to achieve what you want. The handler just waits on another goroutine to do the work. If the inner code running in the go-routine wants to abort processing, it can call Goexit(). It has the advantage that all defer statements will still execute.
This just seems like a weak version of exception handling that Golang currently doesn't support. But I'm throwing it out there.
func handler(w http.ResponseWriter, r *http.Request) {
var cleanExit bool = false
var ch = make(chan bool)
// the actual handler implementation in a goroutine
go func() {
defer close(ch)
handlerImpl(w, r)
cleanExit = true // if handlerImpl invokes goExit, this line doesn't execute
}()
// wait for goroutine to exit
<-ch
if cleanExit {
fmt.Println("Handler exited normally")
} else {
fmt.Println("Hanlder was aborted")
}
}
func handlerImpl(w http.ResponseWriter, r *http.Request) {
checkSomeThing(w, r)
}
func checkSomeThing(w http.ResponseWriter, r *http.Request) {
http.Error(w, "Bad Request!", http.StatusBadRequest)
runtime.Goexit()
}
If checkSomeThing() is specific to that route, you should probably keep going with the code sample you pasted.
If checkSomeThing() is a function common to all your routes (or to a subset of routes), you can choose a way to run a middleware before calling the handler for specific routes.
See for example this answer or this answer, or here is a way to do it using only code from the standard http package :
func checkSomething(...) error {
...
}
func WrapWithCheck(handler http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
err := checkSomething(w, req)
if err != nil {
return
}
handler.ServeHTTP(w, req)
})
}
func setupRouter() http.Handler {
mux := http.NewServeMux()
mux.HandleFunc("/foo/", handleFoo)
mux.HandleFunc("/bar/", handleBar)
mux.HandleFunc("/baz/", handleBaz)
// add your common call to 'checkSomething' here :
handler := WrapWithCheck(mux)
return handler
}
playground
note : I tried using httptest in the playground above, and for some reason it deadlocks in the playground. It works fine if you copy/paste this code in a sample.go file and use go run sample.go
Let say I have a function that handles request body in general
func GetReqBody(r *http.Request) (interface {}, error){
var data interface{}
decorder := json.NewDecoder(r.Body)
decorder.DisallowUnknownFields()
err := decorder.Decode(&data)
return data, err
}
Then in the controller, I will have to do type assertion
func post(w http.ResponseWriter, r *http.Request) {
data, err := utils.GetReqBody(r)
//req.User is a struct
newUser, ok := data.(req.User)
// ...
}
Is it possible to encapsulate the type assertion login inside the GetReqBody function? To do that I will need to pass the struct into the function, yet as it is not a value I am unable to do so.
"Is it possible to encapsulate the type assertion login inside the GetReqBody function?" -- No, it's not possible, not in any useful way.
However you could simplify your code thus:
func GetReqBody(r *http.Request, data interface{}) error {
decorder := json.NewDecoder(r.Body)
decorder.DisallowUnknownFields()
return decorder.Decode(data)
}
func post(w http.ResponseWriter, r *http.Request) {
var newUser req.User
if err := utils.GetReqBody(r, &newUser); err != nil {
// handle err
}
// ...
}
I have go web server running on port and handling post request which internally calls different url to fetch response using goroutine and proceed.
I have divided the whole flow to different method. Draft of the code.
package main
import (
"bytes"
"fmt"
"github.com/gorilla/mux"
"log"
"net/http"
"time"
)
var status_codes string
func main() {
router := mux.NewRouter().StrictSlash(true)
/*router := NewRouter()*/
router.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
_, _ = fmt.Fprintf(w, "Hello!!!")
})
router.HandleFunc("/{name}", func(w http.ResponseWriter, r *http.Request) {
vars := mux.Vars(r)
prepare(w, r, vars["name"])
}).Methods("POST")
log.Fatal(http.ListenAndServe(fmt.Sprintf(":%d", 8080), router))
}
func prepare(w http.ResponseWriter, r *http.Request, name string) {
//initializing for the current request, need to maintain this variable for each request coming
status_codes = ""
//other part of the code and call to goroutine
var urls []string
//lets say all the url loaded, call the go routine func and wait for channel to respond and then proceed with the response of all url
results := callUrls(urls)
process(w, results)
}
type Response struct {
status int
url string
body string
}
func callUrls(urls []string) []*Response {
ch := make(chan *Response, len(urls))
for _, url := range urls {
go func(url string) {
//http post on url,
//base on status code of url call, add to status code
//some thing like
req, err := http.NewRequest("POST", url, bytes.NewBuffer(somePostData))
req.Header.Set("Content-Type", "application/json")
req.Close = true
client := &http.Client{
Timeout: time.Duration(time.Duration(100) * time.Second),
}
response, err := client.Do(req)
if err != nil {
status_codes += "200,"
//do other thing with the response received
} else {
status_codes += "500,"
}
// return to channel accordingly
ch <- &Response{200, "url", "response body"}
}(url)
}
var results []*Response
for {
select {
case r := <-ch:
results = append(results, r)
if len(results) == len(urls) {
//Done
close(ch)
return results
}
}
}
}
func process(w http.ResponseWriter, results []*Response){
//read those status code received from all urls call for the given request
fmt.Println("status", status_codes)
//Now the above line keep getting status code from other request as well
//for eg. if I have called 5 urls then it should have
//200,500,204,404,200,
//but instead it is
//200,500,204,404,200,204,404,200,204,404,200, and some more keep growing with time
}
The above code does:
Variable declare globally, Initialized in prepare function.
append value in go routine callUrls function
read those variable in process function
Now should I pass those variable declared globally to each function call to make them local as it won't be shared then?(I would hate to do this.)
Or is there any other approach to achieve the same thing without adding more argument to function being called.
As I will have few other string and int value as well that will be used across the program and in go routine function as well.
What will be the correct way of making them thread safe and only 5 codes for each request coming on port simultaneously.
Don't use global variables, be explicit instead and use function arguments. Moreover, you have a race condition on status_codes because it is accessed by multiple goroutines without any mutex lock.
Take a look at my fix below.
func prepare(w http.ResponseWriter, r *http.Request, name string) {
var urls []string
//status_codes is populated by callUris(), so let it return the slice with values
results, status_codes := callUrls(urls)
//process() needs status_codes in order to work, so pass the variable explicitely
process(w, results, status_codes)
}
type Response struct {
status int
url string
body string
}
func callUrls(urls []string) []*Response {
ch := make(chan *Response, len(urls))
//In order to avoid race condition, let's use a channel
statusChan := make(chan string, len(urls))
for _, url := range urls {
go func(url string) {
//http post on url,
//base on status code of url call, add to status code
//some thing like
req, err := http.NewRequest("POST", url, bytes.NewBuffer(somePostData))
req.Header.Set("Content-Type", "application/json")
req.Close = true
client := &http.Client{
Timeout: time.Duration(time.Duration(100) * time.Second),
}
response, err := client.Do(req)
if err != nil {
statusChan <- "200"
//do other thing with the response received
} else {
statusChan <- "500"
}
// return to channel accordingly
ch <- &Response{200, "url", "response body"}
}(url)
}
var results []*Response
var status_codes []string
for !doneRes || !doneStatus { //continue until both slices are filled with values
select {
case r := <-ch:
results = append(results, r)
if len(results) == len(urls) {
//Done
close(ch) //Not really needed here
doneRes = true //we are done with results, set the corresponding flag
}
case status := <-statusChan:
status_codes = append(status_codes, status)
if len(status_codes) == len(urls) {
//Done
close(statusChan) //Not really needed here
doneStatus = true //we are done with statusChan, set the corresponding flag
}
}
}
return results, status_codes
}
func process(w http.ResponseWriter, results []*Response, status_codes []string) {
fmt.Println("status", status_codes)
}
I have this middleware func:
func errorMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
defer func() {
if err := recover(); err != nil {
log.Error("Caught error in defer/recover middleware: ", err)
w.WriteHeader(http.StatusInternalServerError)
json.NewEncoder(w).Encode(struct {
ID string
}{
err.Error(),
})
}
}()
next.ServeHTTP(w, r)
})
}
I use it like so:
router := mux.NewRouter()
router.Use(errorMiddleware)
however I am getting a compilation error, it says:
Anybody know what that's about? I am just trying to convert err to a string, ultimately, serialize it for the client etc.
recover() returns an interface with no methods to proxy any value sent by panic(). In the defer block, you're trying to access the Error() method of a pure, has-no-method interface. If you want to distinguish the built-in error type, you'd have to assert its type like:
realErr, ok := err.(error)
if ok {
// here you can use realErr.Error().
}
So that it'll give you a real value of type error. If you check out the built-in types, you'll see that error is to implement an Error() string method.
Type assertions: https://tour.golang.org/methods/15
I have a method that returns Json from the database and it works correctly, however when I try to make it concurrent it does not return anything and does not give any errors. For example this is the method working correctly
func Listing_Expiration(w http.ResponseWriter, r *http.Request) {
db,err := sql.Open("DB_Connect")
if err != nil {
log.Fatal(err)
println(err)
}
var result string
errr := db.QueryRow("select json_build_object('Expiration', array_to_json(array_agg(t))) from (select fullname,ad_end from profiles where id=32)t").Scan(&result)
defer db.Close()
switch {
case errr != nil:
log.Fatal(errr)
}
fmt.Fprintf(w,result)
}
The above method works correctly and the data is returned to the browser then I try to make this method async
func Listing_Expiration(w http.ResponseWriter, r *http.Request) {
go func(){
db,err := sql.Open("DB_Connect")
if err != nil {
log.Fatal(err)
println(err)
}
var result string
errr := db.QueryRow("select json_build_object('Expiration', array_to_json(array_agg(t))) from (select fullname,ad_end from profiles where id=32)t").Scan(&result)
defer db.Close()
switch {
case errr != nil:
log.Fatal(errr)
}
fmt.Fprintf(w,result)
}()
}
The async above returns nothing the only thing I have changed is that I added the Go func() inside the method so that it is async and everything else is the same . I checked and the database is returning the same content back just have no idea why the async is not printing the results back to the browser.
The net/http server completes the response when the handler returns. Anything written to the response after the handler returns is ignored.
The handler function Listing_Expiration is returning before the anonymous function executes fmt.Fprintf(w,result).
To fix this, use a sync.WaitGroup:
func Listing_Expiration(w http.ResponseWriter, r *http.Request) {
var wg sync.WaitGroup
wg.Add(1)
go func(){
defer wg.Done()
// same code here as in question
}()
wg.Wait()
}
The net/http server starts a goroutine for each connection and runs handlers in those goroutines. If the code in the question is the complete handler, then there's no reason to start yet another goroutine.