Closed. This question needs debugging details. It is not currently accepting answers.
Edit the question to include desired behavior, a specific problem or error, and the shortest code necessary to reproduce the problem. This will help others answer the question.
Closed 3 years ago.
Improve this question
I'm making an API post request to bigpanda using Go.
https://docs.bigpanda.io/reference#create-plan
I have below code and when I try to make API post getting name is undefined on object error.
2019/08/23 18:38:04 {
"status" : "failure",
"error" : "{\"obj\":[{\"msg\":[\"'name' is undefined on object: {\\\"maintenance_plan\\\":{\\\"name\\\":\\\"\\\\\\\"name\\\\\\\": \\\\\\\"scheduled host maintenance\\\\\\\",\\\",\\\"condition\\\":\\\"\\\\\\\"condition\\\\\\\": {\\\\\\\"=\\\\\\\": [\\\\\\\"host\\\\\\\", \\\\\\\"prod-api-1\\\\\\\"]},\\\",\\\"start\\\":\\\"\\\\\\\"start\\\\\\\": \\\\\\\"1566514810\\\\\\\",\\\",\\\"end\\\":\\\"\\\\\\\"end\\\\\\\": \\\\\\\"156651600\\\\\\\"\\\"}}\"],\"args\":[]}]}"
}
package main
import (
"bytes"
"encoding/json"
"fmt"
"io/ioutil"
"log"
"net/http"
)
type Maintenace_Plan struct {
Name string `json:"name"`
//Condition map[string]map[string][]string `json:condition`
Condition string `json:"condition"`
Start string `json:"start"`
End string `json:"end"`
}
type Payload struct {
Maintenace_Plan Maintenace_Plan `json:"maintenance_plan"`
}
func main() {
name := `"name": "scheduled host maintenance",`
create_plan := `"condition": {"=": ["host", "prod-api-1"]},`
start_time := `"start": "1566514810",`
end_time := `"end": "156651600"`
data := Payload{
Maintenace_Plan: Maintenace_Plan{
Name: name,
Condition: create_plan,
Start: start_time,
End: end_time,
},
}
payloadBytes, err := json.Marshal(data)
if err != nil {
fmt.Println(err)
}
body := bytes.NewReader(payloadBytes)
req, err := http.NewRequest("POST", "https://api.bigpanda.io/resources/v2.0/maintenance-plans", body)
if err != nil {
log.Fatal(err)
}
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Authorization", "Bearer <token>")
resp, err := http.DefaultClient.Do(req)
defer resp.Body.Close()
body_1, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatal(err)
}
log.Println(string(body_1))
}
Looks like body is incorrect.
Is there any way to fix the code.
That error message is probably being returned from the API Call you're making and not the JSON Marshal. I suspect it's due to the way you're marshalling your payload - you're writing JSON to the fields and then JSON marshalling it so you end up with a payload that looks like:
{"maintenance_plan":{"name":"\"name\": \"scheduled host maintenance\",","condition":"\"condition\": {\"=\": [\"host\", \"prod-api-1\"]},","start":"\"start\": \"1566514810\",","end":"\"end\": \"156651600\""}}
Notice the double "name: "\name\".
The way to fix it would be to do something like:
data := Payload{
MaintenancePlan: MaintenancePlan{
Name: "scheduled host maintenance",
Condition: map[string][]string{
"=": []string{"host", "prod-api-1"},
},
StartTime: "1566514810",
EndTime: "156651600",
},
}
var buf bytes.Buffer
err := json.NewEncoder(&buf).Encode(data)
if err != nil {
// Handle me
}
req, err := http.NewRequest(http.MethodPost, "https://foo/bar", &buf)
// continue
Example: https://play.golang.org/p/J6wrsLYkvwF
Related
This question already has answers here:
POST data using the Content-Type multipart/form-data
(7 answers)
Closed 8 months ago.
I hope I can explain this right. I'm trying to make an HTTP post request that contains binary data (a file). This is for DeepStack image processing. In Python I have the following working:
image_data = open(file,"rb").read()
try:
response = requests.post("http://deepstack.local:82/v1/vision/detection",files={"image":image_data},timeout=15).json()
In Go, I started with the basic example from here: https://golangtutorial.dev/tips/http-post-json-go/
Modifying this a bit for my use, the relevant lines are:
data, err := ioutil.ReadFile(tempPath + file.Name())
if err != nil {
log.Print(err)
}
httpposturl := "http://deepstack.local:82/v1/vision/custom/combined"
fmt.Println("HTTP JSON POST URL:", httpposturl)
var jsonData = []byte(`{"image": ` + data + `}`)
request, error := http.NewRequest("POST", httpposturl, bytes.NewBuffer(jsonData))
request.Header.Set("Content-Type", "application/json; charset=UTF-8")
This results in an error:
invalid operation: `{"image": ` + data (mismatched types untyped string and []byte)`
the "data" variable at this point is []uint8 ([]byte). I realize, at a high level, what is wrong. I'm trying to join two data types that are not the same. That's about it though. I've tried a bunch of stuff that I'm pretty sure anyone familiar with Go would immediately realize was wrong (declaring jsonData as a byte, converting data to a string, using os.Open instead of ioutil.ReadFile, etc.). I'm just kind of stumbling around blind though. I can't find an example that doesn't use a plain string as the JSON data.
I would appreciate any thoughts.
--- ANSWER ---
I'm marking Dietrich Epp's answer as accepted, because he gave me what I asked for. However, RedBlue in the comments gave me what I actually needed. Thank you both. The code below is modified just a bit from this answer: https://stackoverflow.com/a/56696333/2707357
Change the url variable to your DeepStack server, and the file name to one that actually exists, and the response body should return the necessary information.
package main
import (
"bytes"
"fmt"
"io"
"io/ioutil"
"mime/multipart"
"net/http"
"os"
)
func createMultipartFormData(fieldName, fileName string) (bytes.Buffer, *multipart.Writer) {
var b bytes.Buffer
var err error
w := multipart.NewWriter(&b)
var fw io.Writer
file := mustOpen(fileName)
if fw, err = w.CreateFormFile(fieldName, file.Name()); err != nil {
fmt.Println("Error: ", err)
}
if _, err = io.Copy(fw, file); err != nil {
fmt.Println("Error: ", err)
}
w.Close()
return b, w
}
func mustOpen(f string) *os.File {
r, err := os.Open(f)
if err != nil {
pwd, _ := os.Getwd()
fmt.Println("PWD: ", pwd)
panic(err)
}
return r
}
func main() {
url := "http://deepstack.local:82/v1/vision/custom/combined"
b, w := createMultipartFormData("image", "C:\\go_sort\\temp\\person.jpg")
req, err := http.NewRequest("POST", url, &b)
if err != nil {
return
}
// Don't forget to set the content type, this will contain the boundary.
req.Header.Set("Content-Type", w.FormDataContentType())
client := &http.Client{}
response, error := client.Do(req)
if err != nil {
panic(error)
}
defer response.Body.Close()
fmt.Println("response Status:", response.Status)
fmt.Println("response Headers:", response.Header)
body, _ := ioutil.ReadAll(response.Body)
fmt.Println("response Body:", string(body))
}
It's really such a small error. This is all your question boils down to, as far as I can tell:
var data []byte // with some value
jsonData := []byte(`{"image": ` + data + `}`)
All you have to do is change this to use append() or something similar:
jsonData := append(
append([]byte(`{"image": `), data...),
'}')
The reason is that you can't use + to concatenate []byte in Go. You can use append(), though.
I have been unable to solve the problem into elasticsearch Bulk method for several days, since I am not strong in Go and started learning it not so long ago, while executing the code :
package main
import (
"bytes"
json "encoding/json"
"fmt"
"io/ioutil"
"log"
"net/http"
)
type BulkInsertMetaData struct {
Index []BulkInsertIndex `json:"index"`
}
type BulkInsertIndex struct {
Index string `json:"_index"`
ID string `json:"_id"`
}
type BulInsertData struct {
Url string `json:"url"`
}
func main() {
dataMeta := BulkInsertMetaData{
Index: []BulkInsertIndex{
{
Index: "Test",
ID: "1234567890",
},
},
}
data := BulInsertData{
Url: "http://XXXX.XX",
}
TojsBulInsertData, _ := json.Marshal(data)
TojsBulkInsertMetaData, _ := json.Marshal(dataMeta)
BulkMetaData := bytes.NewBuffer(append(TojsBulkInsertMetaData, []byte("\n")...))
BulkData := bytes.NewBuffer(append(TojsBulInsertData, []byte("\n")...))
log.Println(BulkMetaData)
log.Println(BulkData)
respMetaData, err := http.Post("http://127.0.0.1:9200/_bulk", "application/json", BulkMetaData)
if err != nil {
log.Println(err)
}
body, err := ioutil.ReadAll(respMetaData.Body)
if err != nil {
log.Println(err)
}
fmt.Println(string(body))
respBulkData, err := http.Post("http://127.0.0.1:9200/_bulk", "application/json", BulkData)
if err != nil {
log.Println(err)
}
body2, err := ioutil.ReadAll(respBulkData.Body)
if err != nil {
log.Println(err)
}
fmt.Println(string(body2))
}
but i get an error:
2022/02/09 14:37:02 {"index":[{"_index":"Test","_id":"1234567890"}]}
2022/02/09 14:37:02 {"url":"http://XXXX.XX"}
{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Malformed action/metadata line [1], expected START_OBJECT or END_OBJECT but found [START_ARRAY]"}],"type":"illegal_argument_exception","reason":"Malformed action/metadata line [1], expected START_OBJECT or END_OBJECT but found [START_ARRAY]"},"status":400}
please help and explain what I'm doing wrong, I searched the Internet for the answer to my question but did not find
I test insert when using REST client passes without problems
BulkMetaData should be {"index":{"_index":"Test","_id":"1234567890"}} (without []) and it should be sent to /_bulk together with BulkData, as a single payload:
{"index":{"_index":"Test","_id":"1234567890"}}
{"url":"http://XXXX.XX"}
Sorry for kinda necroing but I also recently needed to design a Bulk connector in our codebase and the fact that there are NO NDJSON encoder/decoders out on the web is appalling. Here is my implementation:
func ParseToNDJson(data []map[string]interface{}, dst *bytes.Buffer) error {
enc := json.NewEncoder(dst)
for _, element := range data {
if err := enc.Encode(element); err != nil {
if err != io.EOF {
return fmt.Errorf("failed to parse NDJSON: %v", err)
}
break
}
}
return nil
}
Driver code to test:
func main() {
var b bytes.Buffer
var data []map[string]interface{}
// pointless data generation...
for i, name := range []string{"greg", "sergey", "alex"} {
data = append(data, map[string]interface{}{name: i})
}
if err := ParseToNDJson(query, &body); err != nil {
return nil, fmt.Errorf("error encoding request: %s", err)
}
res, err := esapi.BulkRequest{
Index: "tasks",
Body: strings.NewReader(body.String()),
}.Do(ctx, q.es)
Hope this helps someone
I have this template
<data><datos name={{.Name}} phone={{.Phone}} email={{.Email}}></data>
And I'm running it like this:
buf := &bytes.Buffer{}
t := template.Must(template.New("dataPdf").Parse("<data><datos name={{.Name}} phone={{.Phone}} email={{.email}}></data>"))
err := t.Execute(buf, data)
if err != nil {
panic(err)
}
return buf.String()
All values are overridden, but for some reason the < and > characters remain in unicode
\u003cdatos\u003e\
I update:
Verify and the error doesn't occur until this string value is assigned to a specific Struct object and then I try to convert it to json.
I cannot reproduce this. See a working example on the Go Playground https://play.golang.org/p/xKmjgzZxeES.
The output of the code below is
<data><datos name=Joe phone=123-456-7890 email=hunter2#a.com></data>
package main
import (
"bytes"
"fmt"
"text/template"
)
type Person struct {
Name string
Phone string
Email string
}
func main() {
buf := &bytes.Buffer{}
t := template.Must(template.New("dataPdf").Parse("<data><datos name={{.Name}} phone={{.Phone}} email={{.Email}}></data>"))
data := Person{Name: "Joe", Phone: "123-456-7890", Email: "hunter2#a.com"}
err := t.Execute(buf, data)
if err != nil {
panic(err)
}
fmt.Printf(buf.String())
}
Following code:
Lobby := map[string]interface{}{
"table_id" :"new table id",
"Status" : true,
"name" : "shivam",
"array":[]interface{}{0,1,3},// this replace existing array with new values
}
result, err := client.Collection("lobbies").Doc("12").Set(ctx,Lobby,firestore.MergeAll)
I just want to update 2nd element in array with the new value
There is no way to inform a database to replace a specific element in a slice or array.
Your array field stores a slice and is not a map, so you need to implement your own code that will regenerate that slice in the way you need and replace it in the document, something like:
Lobby := map[string]interface{} {
"table_id" : "new table id",
"Status" : true,
"name" : "shivam",
"array" : []interface{}{0,1,3},
}
new_slice, err := change_my_slice(Lobby["array"])
if err != nil {
log.Errorf("Error message goes here")
return nil, err
}
Lobby["array"] = new_slice
result, err := client.Collection("lobbies").Doc("12").Set(ctx,Lobby,firestore.MergeAll)
This is currently possible, although advertised as "Not yet supported" at firebase.
Here's a slightly modified example.
type User struct {
Address string `firestore:"address"`
Hobbies []Stickers `firestore:"hobbies"`
}
package main
import (
"cloud.google.com/go/firestore"
"context"
"fmt"
)
func main() {
ctx := context.Background()
client, err := firestore.NewClient(ctx, "project-id")
if err != nil {
log.Fatalln("Client Error", err)
}
defer client.Close()
co := client.Doc("User/Frank")
_, err := co.Update(ctx, []firestore.Update{
{Path: "hobbies", Value: firestore.ArrayUnion("Simping")},
})
if err != nil {
log.Println("Update error", err)
}
}
You can check the documentation of ArrayUnion and ArrayRemove at pkg
I'm trying to follow the Google Sheets API quickstart here:
https://developers.google.com/sheets/api/reference/rest/v4/spreadsheets.values/batchUpdate
(scroll down to "Examples" then click "GO")
This is how I tried to update a spreadsheet:
package main
// BEFORE RUNNING:
// ---------------
// 1. If not already done, enable the Google Sheets API
// and check the quota for your project at
// https://console.developers.google.com/apis/api/sheets
// 2. Install and update the Go dependencies by running `go get -u` in the
// project directory.
import (
"errors"
"fmt"
"log"
"net/http"
"golang.org/x/net/context"
"google.golang.org/api/sheets/v4"
)
func main() {
ctx := context.Background()
c, err := getClient(ctx)
if err != nil {
log.Fatal(err)
}
sheetsService, err := sheets.New(c)
if err != nil {
log.Fatal(err)
}
// The ID of the spreadsheet to update.
spreadsheetId := "1diQ943LGMDNkbCRGG4VqgKZdzyanCtT--V8o7r6kCR0"
var jsonPayloadVar []string
monthVar := "Apr"
thisCellVar := "A26"
thisLinkVar := "http://test.url"
jsonRackNumberVar := "\"RACKNUM01\""
jsonPayloadVar = append(jsonPayloadVar, fmt.Sprintf("(\"range\": \"%v!%v\", \"values\": [[\"%v,%v)\"]]),", monthVar, thisCellVar, thisLinkVar, jsonRackNumberVar))
rb := &sheets.BatchUpdateValuesRequest{"ValueInputOption": "USER_ENTERED", "data": jsonPayloadVar}
resp, err := sheetsService.Spreadsheets.Values.BatchUpdate(spreadsheetId, rb).Context(ctx).Do()
if err != nil {
log.Fatal(err)
}
fmt.Printf("%#v\n", resp)
}
func getClient(ctx context.Context) (*http.Client, error) {
// https://developers.google.com/sheets/quickstart/go#step_3_set_up_the_sample
//
// Authorize using the following scopes:
// sheets.DriveScope
// sheets.DriveFileScope
sheets.SpreadsheetsScope
return nil, errors.New("not implemented")
}
Output:
hello.go:43: invalid field name "ValueInputOption" in struct initializer
hello.go:43: invalid field name "data" in struct initializer
hello.go:58: sheets.SpreadsheetsScope evaluated but not used
There are 2 things that aren't working:
It's not obvious how to enter the fields into variable rb
I need to use sheets.SpreadsheetsScope
Can anyone provide a working example that does a BatchUpdate?
References:
This article shows how to do an update that is not a BatchUpdate: Golang google sheets API V4 - Write/Update example?
Google's API reference - see the ValueInputOption section starting at line 1437: https://github.com/google/google-api-go-client/blob/master/sheets/v4/sheets-gen.go
This article shows how to do a BatchUpdate in Java: Write data to Google Sheet using Google Sheet API V4 - Java Sample Code
How about the following sample script? This is a simple sample script for updating sheet on Spreadsheet. So if you want to do various update, please modify it. The detail of parameters for spreadsheets.values.batchUpdate is here.
Flow :
At first, in ordet to use the link in your question, please use Go Quickstart. In my sample script, the script was made using the Quickstart.
The flow to use this sample script is as follows.
For Go Quickstart, please do Step 1 and Step 2.
Please put client_secret.json to the same directory with my sample script.
Copy and paste my sample script, and create it as new script file.
Run the script.
When Go to the following link in your browser then type the authorization code: is shown on your terminal, please copy the URL and paste to your browser. And then, please authorize and get code.
Put the code to the terminal.
When Done. is displayed, it means that the update of spreadsheet is done.
Request body :
For Spreadsheets.Values.BatchUpdate, BatchUpdateValuesRequest is required as one of parameters. In this case, the range, values and so on that you want to update are included in BatchUpdateValuesRequest. The detail information of this BatchUpdateValuesRequest can be seen at godoc. When it sees BatchUpdateValuesRequest, Data []*ValueRange can be seen. Here, please be carefull that Data is []*ValueRange. Also ValueRange can be seen at godoc. You can see MajorDimension, Range and Values in ValueRange.
When above infomation is reflected to the script, the script can be modified as follows.
Sample script :
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"log"
"net/http"
"os"
"golang.org/x/net/context"
"golang.org/x/oauth2"
"golang.org/x/oauth2/google"
"google.golang.org/api/sheets/v4"
)
// getClient uses a Context and Config to retrieve a Token
// then generate a Client. It returns the generated Client.
func getClient(ctx context.Context, config *oauth2.Config) *http.Client {
cacheFile := "./go-quickstart.json"
tok, err := tokenFromFile(cacheFile)
if err != nil {
tok = getTokenFromWeb(config)
saveToken(cacheFile, tok)
}
return config.Client(ctx, tok)
}
// getTokenFromWeb uses Config to request a Token.
// It returns the retrieved Token.
func getTokenFromWeb(config *oauth2.Config) *oauth2.Token {
authURL := config.AuthCodeURL("state-token", oauth2.AccessTypeOffline)
fmt.Printf("Go to the following link in your browser then type the "+
"authorization code: \n%v\n", authURL)
var code string
if _, err := fmt.Scan(&code); err != nil {
log.Fatalf("Unable to read authorization code %v", err)
}
tok, err := config.Exchange(oauth2.NoContext, code)
if err != nil {
log.Fatalf("Unable to retrieve token from web %v", err)
}
return tok
}
// tokenFromFile retrieves a Token from a given file path.
// It returns the retrieved Token and any read error encountered.
func tokenFromFile(file string) (*oauth2.Token, error) {
f, err := os.Open(file)
if err != nil {
return nil, err
}
t := &oauth2.Token{}
err = json.NewDecoder(f).Decode(t)
defer f.Close()
return t, err
}
func saveToken(file string, token *oauth2.Token) {
fmt.Printf("Saving credential file to: %s\n", file)
f, err := os.Create(file)
if err != nil {
log.Fatalf("Unable to cache oauth token: %v", err)
}
defer f.Close()
json.NewEncoder(f).Encode(token)
}
type body struct {
Data struct {
Range string `json:"range"`
Values [][]string `json:"values"`
} `json:"data"`
ValueInputOption string `json:"valueInputOption"`
}
func main() {
ctx := context.Background()
b, err := ioutil.ReadFile("client_secret.json")
if err != nil {
log.Fatalf("Unable to read client secret file: %v", err)
}
config, err := google.ConfigFromJSON(b, "https://www.googleapis.com/auth/spreadsheets")
if err != nil {
log.Fatalf("Unable to parse client secret file to config: %v", err)
}
client := getClient(ctx, config)
sheetsService, err := sheets.New(client)
if err != nil {
log.Fatalf("Unable to retrieve Sheets Client %v", err)
}
spreadsheetId := "### spreadsheet ID ###"
rangeData := "sheet1!A1:B3"
values := [][]interface{}{{"sample_A1", "sample_B1"}, {"sample_A2", "sample_B2"}, {"sample_A3", "sample_A3"}}
rb := &sheets.BatchUpdateValuesRequest{
ValueInputOption: "USER_ENTERED",
}
rb.Data = append(rb.Data, &sheets.ValueRange{
Range: rangeData,
Values: values,
})
_, err = sheetsService.Spreadsheets.Values.BatchUpdate(spreadsheetId, rb).Context(ctx).Do()
if err != nil {
log.Fatal(err)
}
fmt.Println("Done.")
}
Result :
References :
The detail infomation of spreadsheets.values.batchUpdate is here.
The detail infomation of Go Quickstart is here.
The detail infomation of BatchUpdateValuesRequest is here.
The detail infomation of ValueRange is here.
If I misunderstand your question, I'm sorry.