Let's say I have a handler that makes a request and gets the latest data on the selected stock:
func (ss *stockService) GetStockInfo(ctx *gin.Context) {
code := ctx.Param("symbol")
ss.logger.Info("code", code)
url := fmt.Sprintf("URL/%v", code)
ss.logger.Info(url)
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
errs.HTTPErrorResponse(ctx, &ss.logger, errs.New(errs.Internal, err))
return
}
resp, err := http.DefaultClient.Do(req)
if err != nil {
errs.HTTPErrorResponse(ctx, &ss.logger, errs.New(errs.Internal, err))
return
}
defer resp.Body.Close()
var chart ChartResponse
err = json.NewDecoder(resp.Body).Decode(&chart)
if err != nil {
errs.HTTPErrorResponse(ctx, &ss.logger, errs.New(errs.Internal, err))
return
}
ctx.JSON(http.StatusOK, chart)
}
And I want to add caching here. Since I don't have a lot of experience right now, I'm interested in proper interaction with the cache.
I think that if, for example, it is not possible to save to the cache for some reason, then you can simply make a request to the api. Then I wonder if it would be right to save to the cache in a separate goroutine and immediately return the response:
func (ss *stockService) GetStockInfo(ctx *gin.Context) {
code := ctx.Param("symbol")
stockInfo, err := ss.cache.Get(code)
if err == nil {
// FIND
...
ctx.JSON(http.StatusOK, chart)
} else {
ss.logger.Info("code", code)
url := fmt.Sprintf("URL/%v", code)
ss.logger.Info(url)
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
...
err = json.NewDecoder(resp.Body).Decode(&chart)
// IS IT A GOOD WAY ?
go ss.cache.Save(code,chart,expireAt)
ctx.JSON(http.StatusOK, chart)
}
}
I use redis as a cache.
I will be glad if someone says what is wrong with this approach.
Related
I copy-pasted the code from an API (https://api.magiceden.dev/). This code gets the link and prints a slice. Here's the code:
func main() {
url := "https://api-mainnet.magiceden.dev/v2/wallets/6xX3z7uxTNB68izZW2GHKnzno49dizqeVVc5ncVzdjFM/activities?offset=0&limit=100"
method := "GET"
client := &http.Client{}
req, err := http.NewRequest(method, url, nil)
if err != nil {
fmt.Println(err)
return
}
res, err := client.Do(req)
if err != nil {
fmt.Println(err)
return
}
defer res.Body.Close()
body, err := ioutil.ReadAll(res.Body)
if err != nil {
fmt.Println(err)
return
}
fmt.Println(string(body))
}
I'm new to Go, and I know about empty return statements in other functions, but what is returned in main function? That's the question and I still haven't found the answer.
I tried googling it, but I couldn't find any info or examples of empty return statements in main function.
When there is no return type in the function signature the return in such a function just stops the processing of the function at this point. No further statement are run then, but the registered defer functions are processed in the reverse order they have been registered.
I've been playing around with the spotify api and came to an Problem. context.Context gets used and therefore the functions just "randomly" execute. The OAuth function should check if the Code is invalid but If I don't do this with an channel the last part of the code gets executed directly without even the first/second function finishing. Because of that I made an second goroutine that checks if the channel is received and then write an response. But now I get this error http: wrote more than the declared Content-Length how can I correct the Content-Lenght? Why is context even used?
My Code:
// Wrapper: github.com/zmb3/spotify/v2
func WriteResponse(w http.ResponseWriter, h chan *spotify.Client) {
client := <-h
user, err := client.CurrentUser(context.Background())
fmt.Println(user.User.DisplayName)
if err != nil {
_, err := fmt.Fprint(w, "Couldn't get user sorry :(")
if err != nil {
return
}
}
_, err = fmt.Fprintf(w, "Logged in as %s!", user.User.DisplayName)
if err != nil {
log.Println(err)
return
}
}
func OAuth(w http.ResponseWriter, r *http.Request) {
ch := make(chan *spotify.Client)
tok, err := auth.Token(r.Context(), state, r)
if err != nil {
w.WriteHeader(503)
_, err := fmt.Fprint(w, "Couldn't get token sorry :(")
if err != nil {
return
}
}
if st := r.FormValue("state"); st != state {
http.NotFound(w,r)
log.Fatalf("State mismatch: %s != %s\n", st, state)
}
go WriteResponse(w, ch)
client := spotify.New(auth.Client(r.Context(), tok))
ch <- client
}
You forgot to return..
if err != nil {
w.WriteHeader(503)
_, err := fmt.Fprint(w, "Couldn't get token sorry :(")
if err != nil {
return
}
// here
return
}
I have use case where a Go Client with a file-watcher listens to changes in a file and sends these changes to a Go server. If the Client can't reach the server during the POST requests with the payload of the file changes, the Client will keep trying every 3 seconds to send the request until err := http.NewRequest() dosen't return a non-nil err
But If the Client is currently trying every 3 seconds to send a POST request but at the same time a new change occurs to the file under file-watch, I want the current POST requests's payload to be overwritten by the new payload(new changes from the file)
How Do I archive this best?
Client code for sending an HTTP requests
func makeRequest(method string, body io.Reader) (*http.Response, error) {
client := &http.Client{}
request, err := http.NewRequest(.., .., ..)
if err != nil {
log.Println("Error: Couldn't make a new Request:", err)
return nil, err
}
response, err := client.Do(request)
if err != nil {
log.Printf("Error: Couldn't execute %s request:%s", method, err)
}
return response, err
}
The function that retries until err !=nil
func autoRetry(f func() error) {
if err := backoff.Retry(f, getBackOff()); err != nil {
log.Println("Error: Couldn't execute exponential backOff retries: ", err)
}
}
autoRetry() is just a function which takes a function and uses ExponentialBackOff to calculate the amount of tries until err !=nil
The call to the method doing the POST request with retries
func postTodo() {
autoRetry(func() error {
r, err := makeRequest("POST", getFileData())
if err != nil {
return err
}
if r.StatusCode != 200 {
return errors.New("Error:" + r.Status)
}
return nil
})
}
I have built a Backend API in Go, it works however I want refactor the code for the DB access layer into a function - idiomatically.
// Get the form data entered by client; FirstName, LastName, phone Number,
// assign the person a unique i.d
// check to see if that user isn't in the database already
// if they are send an error message with the a 'bad' response code
// if they aren't in db add to db and send a message with success
func CreateStudentAccountEndpoint(response http.ResponseWriter, request *http.Request){
client, err := mongo.NewClient("mongodb://localhost:27017")
if err != nil {
log.Fatalf("Error connecting to mongoDB client Host: Err-> %v\n ", err)
}
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
defer cancel()
err = client.Connect(ctx)
if err != nil {
log.Fatalf("Error Connecting to MongoDB at context.WtihTimeout: Err-> %v\n ", err)
}
response.Header().Set("Content-Type", "application/json")
studentCollection := client.Database(dbName).Collection("students")
_, err = studentCollection.InsertOne(context.Background(),data)
if err != nil {
response.WriteHeader(501)
response.Write([]byte(`{ "message": "` + err.Error() + `" }`))
}
// encoding json object for returning to the client
jsonStudent, err := json.Marshal(student)
if err != nil {
http.Error(response, err.Error(), http.StatusInternalServerError)
}
response.Write(jsonStudent)
}
I understand that I can create a method which returns (*mongoClient, err) as I utilise the client local variable later on in the code.
However I am lost as to how to implement the defer cancel() part because it executes once the method CreateStudenAccountEndpoint is at the end. But I am at a loss on how to implement this defer section in a method that will recognise that I want the defer to happen at the end of the function that calls the DB access layer method e.g CreateStudentAccountEndpoint not the actual db access method itself.
As I understand it, the connection should be long-lived and set up as a part of a constructor, i.e. not part of the request flow.
This will typically look something like this:
type BackendAPI struct {
client *mongo.Client
}
func NewBackendAPI(mongoURI string) (*BackendAPI, error) {
client, err := mongo.NewClient(mongoURI)
if err != nil {
return nil, err
}
ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
defer cancel()
err = client.Connect(ctx)
if err != nil {
return nil, err
}
return &BackendAPI{client}, nil
}
func (api *BackendAPI) func CreateStudentAccountEndpoint(response http.ResponseWriter, request *http.Request) {
response.Header().Set("Content-Type", "application/json")
// note the use of the long-lived api.client, which is connected already.
studentCollection := api.client.Database(dbName).Collection("students")
_, err = studentCollection.InsertOne(context.Background() ,data)
if err != nil {
response.WriteHeader(501)
response.Write([]byte(`{ "message": "` + err.Error() + `" }`))
return // at this point, the method should return
}
// encoding json object for returning to the client
jsonStudent, err := json.Marshal(student)
if err != nil {
http.Error(response, err.Error(), http.StatusInternalServerError)
}
response.Write(jsonStudent)
}
If you worry about losing the connection, you could implement a call to api.client.Ping in there, but in my opinion this should only be attempted if you encounter a failure you believe you can recover from by reconnecting.
Is there a way to download a large file using Go that will store the content directly into a file instead of storing it all in memory before writing it to a file? Because the file is so big, storing it all in memory before writing it to a file is going to use up all the memory.
I'll assume you mean download via http (error checks omitted for brevity):
import ("net/http"; "io"; "os")
...
out, err := os.Create("output.txt")
defer out.Close()
...
resp, err := http.Get("http://example.com/")
defer resp.Body.Close()
...
n, err := io.Copy(out, resp.Body)
The http.Response's Body is a Reader, so you can use any functions that take a Reader, to, e.g. read a chunk at a time rather than all at once. In this specific case, io.Copy() does the gruntwork for you.
A more descriptive version of Steve M's answer.
import (
"os"
"net/http"
"io"
)
func downloadFile(filepath string, url string) (err error) {
// Create the file
out, err := os.Create(filepath)
if err != nil {
return err
}
defer out.Close()
// Get the data
resp, err := http.Get(url)
if err != nil {
return err
}
defer resp.Body.Close()
// Check server response
if resp.StatusCode != http.StatusOK {
return fmt.Errorf("bad status: %s", resp.Status)
}
// Writer the body to file
_, err = io.Copy(out, resp.Body)
if err != nil {
return err
}
return nil
}
The answer selected above using io.Copy is exactly what you need, but if you are interested in additional features like resuming broken downloads, auto-naming files, checksum validation or monitoring progress of multiple downloads, checkout the grab package.
Here is a sample. https://github.com/thbar/golang-playground/blob/master/download-files.go
Also I give u some codes might help you.
code:
func HTTPDownload(uri string) ([]byte, error) {
fmt.Printf("HTTPDownload From: %s.\n", uri)
res, err := http.Get(uri)
if err != nil {
log.Fatal(err)
}
defer res.Body.Close()
d, err := ioutil.ReadAll(res.Body)
if err != nil {
log.Fatal(err)
}
fmt.Printf("ReadFile: Size of download: %d\n", len(d))
return d, err
}
func WriteFile(dst string, d []byte) error {
fmt.Printf("WriteFile: Size of download: %d\n", len(d))
err := ioutil.WriteFile(dst, d, 0444)
if err != nil {
log.Fatal(err)
}
return err
}
func DownloadToFile(uri string, dst string) {
fmt.Printf("DownloadToFile From: %s.\n", uri)
if d, err := HTTPDownload(uri); err == nil {
fmt.Printf("downloaded %s.\n", uri)
if WriteFile(dst, d) == nil {
fmt.Printf("saved %s as %s\n", uri, dst)
}
}
}