Hi I am creating a POST request in GO but it terminates before actually finishing, for example I am trying to pull a docker image and this can be done with curl by the following:
curl -X POST https://address/images/create?fromImage=imagename
this returns the following
{"status":"Pulling from imagename","id":"latest"}
{"status":"Already exists","progressDetail":{},"id":"3d30e94188f7"}
{"status":"Already exists","progressDetail":{},"id":"bf4e27765153"}
{"status":"Already exists","progressDetail":{},"id":"67280fd39fba"}
.... many of those
{"status":"Pull complete","progressDetail":{},"id":"21c062e2346f"}
{"status":"Digest: sha256:24f26a1344fca6d5ee1adcdsf2d01b20d7823c560ed9d2193466e36bd1f049088"}
{"status":"Pulling from imagename","id":"20161005"}
{"status":"Digest: sha256:f527dsfds88676eb25d8f7de5406f46cbc3a995345ddb4bb3d08fcf110458fe3cf"}
{"status":"Status: Downloaded newer image for imagename"}
and the image is pulled successfully
but If I try from go
func PullImage(imagename string, uuid string) error {
logFields := log.Fields{
"handler": "PullImage",
"uuid": uuid,
}
log.WithFields(logFields).Debugf("imagename:%v", imagename)
url := fmt.Sprintf("https://%s/images/create?fromImage%s", sconf.Docker.Endpoint, imagename)
req, err := http.NewRequest("POST", url, nil)
req.Header.Set("Content-Type", "application/json")
resp, err := client.Do(req)
log.WithFields(logFields).Infof("call url:%s", url)
if err != nil {
log.WithFields(logFields).Errorf("Error in call url (%s) :%s", url, err)
return errors.New(fmt.Sprintf("Error in call url (%s) :%s", url, err))
}
var pullresbody interface{}
err = json.NewDecoder(resp.Body).Decode(&pullresbody)
if err != nil {
log.WithFields(logFields).Errorf("Could not unmarshal json: %s", err)
return errors.New(fmt.Sprintf("Could not unmarshal json: %s", err))
}
log.WithFields(logFields).Infof("response of url %s:%+v", url, resp)
log.WithFields(logFields).Infof("response body:%+v", pullresbody)
return nil
}
then I get this in the logs:
map[status:Pulling from imagename]
and the image is not pulled so the connection is stopped before really finishing how can I fix this?
The Decoder will only decode one object from a stream at a time. To get all objects, you'll need something like
var pullRespBody interface{}
dec := json.NewDecoder(resp.Body)
var err error
for err == nil {
err = dec.Decode(&pullRespBody)
// Check err...
log.WithFields(logFields).Infof("response body:%+v", pullRespBody)
// Do something else with pullRespBody...
}
// Deal with err...
Related
I've got an HTTP Post method, which successfully posts data to an external third party API and returns a response.
I then need data returned from this response to post to my database.
The response contains a few piece of data, but I only need the 'access_token' and 'refresh_token' from it.
As a result, what I'm attempting to do is convert the response from a string into individual components in a new data struct I've created - to then pass to my database.
However, the data is showing as blank, despite it successfully being written to my browser. I'm obviously doing something fundamentally wrong, but not sure what..
Here's my code:
type data struct {
Access_token string `json:"access_token"`
Refresh_token string `json:"refresh_token"`
}
func Fetch(w http.ResponseWriter, r *http.Request) {
client := &http.Client{}
q := url.Values{}
q.Add("grant_type", "authorization_code")
q.Add("client_id", os.Getenv("ID"))
q.Add("client_secret", os.Getenv("SECRET"))
q.Add("redirect_uri", "https://callback-url.com")
q.Add("query", r.URL.Query().Get("query"))
req, err := http.NewRequest("POST", "https://auth.truelayer-sandbox.com/connect/token", strings.NewReader(q.Encode()))
if err != nil {
log.Print(err)
fmt.Println("Error was not equal to nil at first stage.")
os.Exit(1)
}
req.Header.Add("Content-Type", "application/x-www-form-urlencoded")
resp, err := client.Do(req)
if err != nil {
fmt.Println("Error sending request to server")
os.Exit(1)
}
respBody, _ := ioutil.ReadAll(resp.Body)
d := data{}
err = json.NewDecoder(resp.Body).Decode(&d)
if err != nil {
fmt.Println(err)
}
fmt.Println(d.Access_token)
fmt.Println(d.Refresh_token)
w.WriteHeader(resp.StatusCode)
w.Write(respBody)
}
With ioutil.ReadAll you read the body, already. The second time you pass to NewDecoder(resp.Body) the stream was consumed.
You can use instead json.Unmarshal(respBody, &d).
One more advice, don't ignore the error on ioutil.ReadAll
I have two golang servers running on localhost.
They are using different ports.
I want to create a post request on one that sends a JSON object to the other one.
I am using the echo framework (if this matters)
The error I am getting is when I try to marshal the object for the post object:
2-valued json.Marshal(data) (value of type ([]byte, error)) where single value is expected
server 1:
type SendEmail struct {
SenderName string `json:"senderName,omitempty" bson:"senderName,omitempty" validate:"required,min=3,max=128"`
SenderEmail string `json:"senderEmail" bson:"senderEmail" validate:"required,min=10,max=128"`
Subject string `json:"subject" bson:"subject" validate:"required,min=10,max=128"`
RecipientName string `json:"recipientName" bson:"recipientName" validate:"required,min=3,max=128"`
RecipientEmail string `json:"recipientEmail" bson:"recipientEmail" validate:"required,min=10,max=128"`
PlainTextContent string `json:"plainTextContent" bson:"plainTextContent" validate:"required,min=10,max=512"`
}
func resetPassword(c echo.Context) error {
email := c.Param("email")
if email == "" {
return c.String(http.StatusNotFound, "You have not supplied a valid email")
}
data := SendEmail{
RecipientEmail: email,
RecipientName: email,
SenderEmail: “test#test”,
SenderName: “name”,
Subject: "Reset Password",
PlainTextContent: "Here is your code to reset your password, if you did not request this email then please ignore.",
}
// error here
req, err := http.NewRequest("POST", "127.0.0.1:8081/", json.Marshal(data))
if err != nil {
fmt.Println(err)
}
defer req.Body.Close()
return c.JSON(http.StatusOK, email)
}
server 2:
e.GET("/", defaultRoute)
func defaultRoute(c echo.Context) (err error) {
u := SendEmail{}
if err = c.Bind(u); err != nil {
return
}
return c.JSON(http.StatusOK, u)
}
It's always nice to meet a Gopher. A few things you might want to know, Go supports multi-value returns in that a function can return more than one value.
byteInfo, err := json.Marshal(data) // has two values returned
// check if there was an error returned first
if err != nil{
// handle your error here
}
Now the line below in your code
// error here
req, err := http.NewRequest("POST", "127.0.0.1:8081/", json.Marshal(data))
Will become this
// error here
req, err := http.NewRequest("POST", "127.0.0.1:8081/", bytes.NewBuffer(byteInfo))
And you can continue with the rest of your code. Happy Coding!
json.Marshal returns []byte and error which means you're passing 4 values to http.NewRequest.
You should call json.Marshal first and then use the result for http.NewRequest.
body, err := json.Marshal(data)
if err != nil {
// deal with error
}
req, err := http.NewRequest("POST", "127.0.0.1:8081/", body)
I have use case where a Go Client with a file-watcher listens to changes in a file and sends these changes to a Go server. If the Client can't reach the server during the POST requests with the payload of the file changes, the Client will keep trying every 3 seconds to send the request until err := http.NewRequest() dosen't return a non-nil err
But If the Client is currently trying every 3 seconds to send a POST request but at the same time a new change occurs to the file under file-watch, I want the current POST requests's payload to be overwritten by the new payload(new changes from the file)
How Do I archive this best?
Client code for sending an HTTP requests
func makeRequest(method string, body io.Reader) (*http.Response, error) {
client := &http.Client{}
request, err := http.NewRequest(.., .., ..)
if err != nil {
log.Println("Error: Couldn't make a new Request:", err)
return nil, err
}
response, err := client.Do(request)
if err != nil {
log.Printf("Error: Couldn't execute %s request:%s", method, err)
}
return response, err
}
The function that retries until err !=nil
func autoRetry(f func() error) {
if err := backoff.Retry(f, getBackOff()); err != nil {
log.Println("Error: Couldn't execute exponential backOff retries: ", err)
}
}
autoRetry() is just a function which takes a function and uses ExponentialBackOff to calculate the amount of tries until err !=nil
The call to the method doing the POST request with retries
func postTodo() {
autoRetry(func() error {
r, err := makeRequest("POST", getFileData())
if err != nil {
return err
}
if r.StatusCode != 200 {
return errors.New("Error:" + r.Status)
}
return nil
})
}
token, err := googleOauthConfig.Exchange(context.Background(), code)
if err != nil {
fmt.Fprintf(w, "Err: %+v", err)
}
The output of the fprintf is:
Err: oauth2: cannot fetch token: 401 Unauthorized
Response: {"error":"code_already_used","error_description":"code_already_used"}
I want to check if "error" = "code_already_used". For the life of me, I can't sort out how.
How do I check/return/read "error" or "error_description" of err?
I've looked at the oauth2 code and it's a bit above me.
// retrieveToken takes a *Config and uses that to retrieve an *internal.Token.
// This token is then mapped from *internal.Token into an *oauth2.Token which is returned along
// with an error..
func retrieveToken(ctx context.Context, c *Config, v url.Values) (*Token, error) {
tk, err := internal.RetrieveToken(ctx, c.ClientID, c.ClientSecret, c.Endpoint.TokenURL, v)
if err != nil {
if rErr, ok := err.(*internal.RetrieveError); ok {
return nil, (*RetrieveError)(rErr)
}
return nil, err
}
return tokenFromInternal(tk), nil
}
How guess I'm trying to see the (*RetrieveError) part. Right?
THANK YOU!
The expression:
(*RetrieveError)(rErr)
converts therErr's type from *internal.RetrieveError to *RetrieveError. And since RetrieveError is declared in the oauth2 package, you can type assert the error you receive to *oauth2.RetrieveError to get the details. The details are contained in that type's Body field as a slice of bytes.
Since a slice of bytes is not the best format to be inspected and in your case it seems like the bytes contain a json object you can make your life easier by predefining a type into which you can unmarshal those details.
That is:
type ErrorDetails struct {
Error string `json:"error"`
ErrorDescription string `json:"error_description"`
}
token, err := googleOauthConfig.Exchange(context.Background(), code)
if err != nil {
fmt.Fprintf(w, "Err: %+v", err)
if rErr, ok := err.(*oauth2.RetrieveError); ok {
details := new(ErrorDetails)
if err := json.Unmarshal(rErr.Body, details); err != nil {
panic(err)
}
fmt.Println(details.Error, details.ErrorDescription)
}
}
Can do like this.
arr := strings.Split(err.Error(), "\n")
str := strings.Replace(arr[1], "Response: ", "", 1)
var details ErrorDetails
var json = jsoniter.ConfigCompatibleWithStandardLibrary
err := json.Unmarshal([]byte(str), &details)
if err == nil {
beego.Debug(details.Error)
beego.Debug(details.ErrorDescription)
}
I am trying to upload photos to Twitter. I created a multipart writer and creating a file field using that named media but when I send my request to Twitter it keeps responding missing media field.
Am I missing something?
Here is my code
f, err := os.Open("/Users/nikos/Desktop/test.png")
errored:
if nil != err {
fmt.Println(err)
return
}
var img = new(bytes.Buffer)
enc := base64.NewEncoder(base64.StdEncoding, img)
_, err = io.Copy(enc, f)
if nil != err {
goto errored
}
body := new(bytes.Buffer)//Multipart body
writer := multipart.NewWriter(body)
cl, err := twitter.OauthClient.MakeHttpClient(&oauth.AccessToken{
Token: "xxx",
Secret: "yyy",
})
err = writer.WriteField("media_data", img.String())//base64 version of the image (i tried both binary and base64 versions neither will work)
if nil != err {
goto errored
}
part, err := writer.CreateFormFile("media", "test.png")//actual binary file multiparted and it is named media.
if nil != err {
goto errored
}
_, err = io.Copy(part, f)
if nil != err {
goto errored
}
req, err := http.NewRequest("POST",
"https://upload.twitter.com/1.1/media/upload.json",
body)
if nil != err {
goto errored
}
res, err := cl.Do(req)
if nil != err {
goto errored
}
//and twitter responds that there is no field attached named media
_, err = io.Copy(os.Stdout, res.Body)
fmt.Println(res)
if nil != err {
goto errored
}
Updates: Just referred Twitter API Upload parameter. As per your code snippet you're using both fields media and media_data. You have to use only one -
Upload using base64 -> field name is media_data
Upload using raw -> field name is media
And, you have to add Content-Type header.
req, err := http.NewRequest("POST",
"https://upload.twitter.com/1.1/media/upload.json",
body)
req.Header.Set("Content-Type", writer.FormDataContentType())
if err := writer.Close(); err != nil {
log.Println(err)
}
// Now fire the http request
PS: While composing an answer, in 30 secs gap, #cerise-limón added comment, also close the multipart writer as mentioned by #cerise-limón.
Asked in the comment:
Twitter accepts application/octet-stream, you may not need below approach.
Adding multi-part with user supplied Content-Type instead of application/octet-stream. Basically you have to do same implementation as convenience wrapper with your content-type.
writer := multipart.NewWriter(body)
h := make(textproto.MIMEHeader)
h.Set("Content-Disposition", fmt.Sprintf(`form-data; name="%s"; filename="%s"`,
escapeQuotes(fieldname), escapeQuotes(filename)))
h.Set("Content-Type", "image/png")
part, err := writer.CreatePart(h)
// use part same as before
Definition of escapeQuotes from multiple-part package.
var quoteEscaper = strings.NewReplacer("\\", "\\\\", `"`, "\\\"")
func escapeQuotes(s string) string {
return quoteEscaper.Replace(s)
}