I have golang lambda that prepares ES request, send it to external system and returns its response. Currently, I haven't found a better approach than an unmarshalling response to interface{}.
func HandleRequest(ctx context.Context, searchRequest SearchRequest) (interface{}, error) {
// ... some data preparation and client initalisation
resp, err := ctxhttp.Post(ctx, &client, url, "application/json", buffer)
if err != nil {
return "", err
}
var k interface{}
all, err := ioutil.ReadAll(resp.Body)
err = json.Unmarshal(all, &k)
return k, err
}
I'm not sure it is the fastest and the most performant way to forward response due to that extra ReadAll and Unmarshall. Is there a more performant approach?
I looked at events.APIGatewayProxyResponse{}, but body in it - string and same manipulations are needed
You can handle the response in many different way
If lambda implements additional search response handling, it might be worth defining the response data type contract with respective marshaling/unmarshaling and additional handling logic.
If lambda functionality is to only proxy response from ES search, you may just passed search response payload ([]byte) directly to APIGatewayProxyResponse.Body as []byte and may need to base64 if the payload has binary data.
Code:
func handleRequest(ctx context.Context, apiRequest events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
request, err := newSearchRequest(apiRequest)
if err != nil {
return handleError(err)
}
responseBody, err := proxySearch(ctx, request)
if err != nil {
return handleError(err)
}
return events.APIGatewayProxyResponse{
StatusCode: http.StatusOK,
Body: string(responseBody),
}, nil
}
func proxySearch(ctx context.Context, searchRequest SearchRequest) ([]byte, error) {
// ... some data preparation and client initalisation
resp, err := ctxhttp.Post(ctx, &client, url, "application/json", buffer)
if err != nil {
return nil, err
}
responseBody, err := ioutil.ReadAll(resp.Body)
return responseBody, err
}
Related
There are almost two identical functions that do approximately the same thing. What would be the right way to organize the code to avoid repetition in this case? The httpGetter() function accesses cloud platform API and gets JSON response, which I then parsed in another function and based on it I form terraform manifest from the template. The getToken() function does almost the same thing, just gets a token, which is then used in the httpGetter() function.
var accessToken = getToken()
func httpGetter(method, url string) (*http.Response, []byte) {
client := &http.Client{}
req, err := http.NewRequest(method, url, nil)
if err != nil {
fmt.Println(err)
}
req.Header.Add("Accept", "application/json;version=35.0")
req.Header.Add("Authorization", "Bearer "+accessToken)
res, err := client.Do(req)
if err != nil {
fmt.Println(err)
}
defer res.Body.Close()
body, err := ioutil.ReadAll(res.Body)
if err != nil {
fmt.Println(err)
}
return res, body
}
func getToken() string {
url := "https://cloud-platform-api.com/api/sessions"
method := "POST"
client := &http.Client{}
req, err := http.NewRequest(method, url, nil)
if err != nil {
fmt.Println(err)
}
req.Header.Add("Accept", "application/*+xml;version=35.0")
req.Header.Add("Authorization", "Basic <auth-hash>")
res, err := client.Do(req)
if err != nil {
fmt.Println(err)
}
defer res.Body.Close()
if err != nil {
fmt.Println(err)
}
accessToken := res.Header.Get("x-vmware-vcloud-access-token")
return accessToken
}
First thing first if you know the method is a getter then you don't need to pass the method param so the signature would become something like below. Also, you are already returning a *http.Response back to the caller now it will be the callers decision on what to do with the response + the caller should decide what to do in case of if the HTTP call fails so return error and let the caller decide.
func HttpGet(url string) (*http.Response, error)
Now you also want POST method with body (in some cases) so have another function
func HttpPost(URL string, body []byte) (*http.Response, error)
Now to manage both signature and have a common code you could have a private method that will be just used in this file or you could also expose that method (it is up to you)
type Headers map[string]string
func http(method, URL string, body []byte, headers Headers) (*http.Response, error) { // we pass headers here so that the caller can pass custom headers for request
client := &http.Client{}
req, err := http.NewRequest(method, url, body)
if err != nil {
return nil, err
}
req.Header.Add("Accept", "application/json;version=35.0") // common static header you can keep as it is
for key, value := range headers {
req.Header.Add(key, value)
}
return client.Do(req)
}
Using this your call from the two methods would look like
func HttpGet(url string, headers Headers) (*http.Response, error) {
return http(http.MethodGet, URL, nil, headers)
}
func HttpPost(url string, body []byte, headers Headers) (*http.Response, error) {
return http(http.MethodPost, url, body, headers)
}
Now you can use this to pass the auth token from the caller like:
func getToken() {
res, err := httpPost("https://cloud-platform-api.com/api/sessions", nil,
map[string]string{
"Authorization": "Basic <auth-hash>",
}
if err != nil {
// do something with error
}
if res.StatusCode == http.StatusCreated {
// do what you want with the success response like unmarshalling to JSON
}
}
and for cases where you don't need to pass header, you can do
res, err := httpGet("some-url", nil) // you pass header as nil
I've got an HTTP Post method, which successfully posts data to an external third party API and returns a response.
I then need data returned from this response to post to my database.
The response contains a few piece of data, but I only need the 'access_token' and 'refresh_token' from it.
As a result, what I'm attempting to do is convert the response from a string into individual components in a new data struct I've created - to then pass to my database.
However, the data is showing as blank, despite it successfully being written to my browser. I'm obviously doing something fundamentally wrong, but not sure what..
Here's my code:
type data struct {
Access_token string `json:"access_token"`
Refresh_token string `json:"refresh_token"`
}
func Fetch(w http.ResponseWriter, r *http.Request) {
client := &http.Client{}
q := url.Values{}
q.Add("grant_type", "authorization_code")
q.Add("client_id", os.Getenv("ID"))
q.Add("client_secret", os.Getenv("SECRET"))
q.Add("redirect_uri", "https://callback-url.com")
q.Add("query", r.URL.Query().Get("query"))
req, err := http.NewRequest("POST", "https://auth.truelayer-sandbox.com/connect/token", strings.NewReader(q.Encode()))
if err != nil {
log.Print(err)
fmt.Println("Error was not equal to nil at first stage.")
os.Exit(1)
}
req.Header.Add("Content-Type", "application/x-www-form-urlencoded")
resp, err := client.Do(req)
if err != nil {
fmt.Println("Error sending request to server")
os.Exit(1)
}
respBody, _ := ioutil.ReadAll(resp.Body)
d := data{}
err = json.NewDecoder(resp.Body).Decode(&d)
if err != nil {
fmt.Println(err)
}
fmt.Println(d.Access_token)
fmt.Println(d.Refresh_token)
w.WriteHeader(resp.StatusCode)
w.Write(respBody)
}
With ioutil.ReadAll you read the body, already. The second time you pass to NewDecoder(resp.Body) the stream was consumed.
You can use instead json.Unmarshal(respBody, &d).
One more advice, don't ignore the error on ioutil.ReadAll
I'm trying to test a Go function which performs a call to an external service. Here's the function:
func (gs *EuGameService) retrieveGames(client model.HTTPClient) (model.EuGamesResponse, error) {
req, err := http.NewRequest(http.MethodGet, gs.getGamesEndpoint, nil)
if err != nil {
log.Fatal("Error while creating request ", err)
return nil, err
}
resp, err := client.Do(req)
if err != nil {
log.Fatal("Error while retrieving EU games", err)
return nil, err
}
var euGames model.EuGamesResponse
decoder := json.NewDecoder(resp.Body)
decoder.Decode(&euGames)
return euGames, nil
}
to properly test it, I'm trying to inject a mock client.
type HTTPClient interface {
Do(req *http.Request) (*http.Response, error)
}
type mockClient struct{}
func (mc *mockClient) Do(req *http.Request) (*http.Response, error) {
mock, _ := os.Open("../stubs/eugames.json")
defer mock.Close()
r := ioutil.NopCloser(bufio.NewReader(mock))
return &http.Response{
Status: string(http.StatusOK),
StatusCode: http.StatusOK,
Body: r,
}, nil
}
the file eugames.json contains a couple of games. But for some reason, the body is always empty! What am I missing here? I tried to use a constant with the file content and it works, games are decoded correctly. So I'm assuming there's a problem with my use of the file.
I have a function to create user which is working properly. Now I have to mock Prepare and SaveUser function inside CreateUser. But that CreateUser require json data as request parameter.
Below is my CreateUser function.
func (server *Server) CreateUser(c *gin.Context) {
errList = map[string]string{}
user := models.User{}
if err := c.ShouldBindJSON(&user); err != nil {
log.Println(err)
c.JSON(http.StatusBadRequest, gin.H{"error": err.Error()}) **//every time return from here with error -> invalid request**
return
}
user.Prepare()
userCreated, err := sqlstore.SaveUser(&user)
if err != nil {
formattedError := formaterror.FormatError(err.Error())
errList = formattedError
c.JSON(http.StatusInternalServerError, gin.H{
"status": http.StatusInternalServerError,
"error": errList,
})
return
}
c.JSON(http.StatusCreated, gin.H{
"status": http.StatusCreated,
"response": userCreated,
})
}
This is required json data as request parameter for above create user. I want to pass below data while mocking.
{"firstname":"test","email":"test#test.com"}
Below is test case to mock above create user function.
type UserMock struct {
mock.Mock
}
func (u *UserMock) Prepare() (string, error) {
args := u.Called()
return args.String(0), args.Error(1)
}
func (u *UserMock) SaveUser() (string, error) {
args := u.Called()
return args.String(0), args.Error(1)
}
func TestCreateUser(t *testing.T) {
gin.SetMode(gin.TestMode)
w := httptest.NewRecorder()
c, _ := gin.CreateTestContext(w)
**//how to json data as request parameter**
uMock := UserMock{}
uMock.On("Prepare").Return("mocktest", nil)
server := Server{}
server.CreateUser(c)
if w.Code != 201 {
t.Error("Unexpected status code found : ",w.Code)
}
}
Thanks in advance.
You need to add a strings.NewReader(string(myjson)) on a new request. Please check this and take it as a template on your current GIN Code.
// TestCreateUser new test
func TestCreateUser(t *testing.T) {
// Setup Recorder, TestContext, Router
router := getRouter(true)
w := httptest.NewRecorder()
c, _ := gin.CreateTestContext(w)
// set JSON body
jsonParam := `{"firstname":"test","email":"test#test.com"}`
// Mock HTTP Request and it's return
req, err := http.NewRequest("POST", "/user", strings.NewReader(string(jsonParam)))
// make sure request was executed
assert.NoError(t, err)
// Serve Request and recorded data
router.ServeHTTP(w, req)
// Test results
assert.Equal(t, 200, w.Code)
assert.Equal(t, nil, w.Body.String())
// check your response since nil isn't valid return
}
I have use case where a Go Client with a file-watcher listens to changes in a file and sends these changes to a Go server. If the Client can't reach the server during the POST requests with the payload of the file changes, the Client will keep trying every 3 seconds to send the request until err := http.NewRequest() dosen't return a non-nil err
But If the Client is currently trying every 3 seconds to send a POST request but at the same time a new change occurs to the file under file-watch, I want the current POST requests's payload to be overwritten by the new payload(new changes from the file)
How Do I archive this best?
Client code for sending an HTTP requests
func makeRequest(method string, body io.Reader) (*http.Response, error) {
client := &http.Client{}
request, err := http.NewRequest(.., .., ..)
if err != nil {
log.Println("Error: Couldn't make a new Request:", err)
return nil, err
}
response, err := client.Do(request)
if err != nil {
log.Printf("Error: Couldn't execute %s request:%s", method, err)
}
return response, err
}
The function that retries until err !=nil
func autoRetry(f func() error) {
if err := backoff.Retry(f, getBackOff()); err != nil {
log.Println("Error: Couldn't execute exponential backOff retries: ", err)
}
}
autoRetry() is just a function which takes a function and uses ExponentialBackOff to calculate the amount of tries until err !=nil
The call to the method doing the POST request with retries
func postTodo() {
autoRetry(func() error {
r, err := makeRequest("POST", getFileData())
if err != nil {
return err
}
if r.StatusCode != 200 {
return errors.New("Error:" + r.Status)
}
return nil
})
}