How to translate this curl call into Go? - go

I have the following curl call:
curl \
--request POST \
--header "Content-Type: application/x-www-form-urlencoded" \
--header "Authorization: Basic XXXX" \
--data-urlencode 'A=B' \
--data-urlencode 'C=D' \
"https://www.myexample.com/mypath?param1=foobar"
How can I write Go code that will do the exact same thing?
So far, I have successfully constructed the full URL and am sending it to http.NewRequestWithContext()
u, err := url.Parse("https://www.myexample.com" + "/mypath")
if err != nil {
return err
}
q := u.Query()
u.RawQuery = q.Encode()
http.NewRequestWithContext(myContext, http.MethodPost, u.String(), nil)
response, err := http.NewRequestWithContext(myContext, http.MethodPost, s.cfg.BaseDRAPI + oAuthPath, nil)
if err != nil {
return err
}
However, I do not know where/how to specify the headers and the two data fields.
Please help!

Create the request and send it as below. Handle the error and response according to your application.
endpoint := "https://www.myexample.com/mypath?param1=foobar"
data := url.Values{}
data.Set("A", "B")
data.Set("C", "D")
//create new POST request to the url and encoded form Data
r, err := http.NewRequest("POST", endpoint, strings.NewReader(data.Encode())) // URL-encoded payload
if err != nil {
log.Fatal(err)
}
//set headers to the request
r.Header.Add("Content-Type", "application/x-www-form-urlencoded") //this is a must for form data encoded request
r.Header.Add("Authorization", "Basic XXXX")
//send request and get the response
client := &http.Client{}
res, err := client.Do(r)
if err != nil {
//handle error
log.Fatal(`error: `,err)
}
log.Println(res.Status)
//handle response part here

Related

Go test to validate connect2id gives "invalid_client" error

I'm trying to validate the Connect2id set up with Go test and I'm getting following error.
"Client authentication failed: Missing client authentication","error":"invalid_client"
The full scenario output is look like below.
Feature: Test Identity Provider
Scenario: # features/idp.feature:3
Given identity provider 'hf1-idp.json' # main_test.go:72 -> *Scaffold
When I request an access token as 'cc_test' # main_test.go:83 -> *Scaffold
oauth2: cannot fetch token: 401 Unauthorized
Response: {"error_description":"Client authentication failed: Missing client authentication","error":"invalid_client"}
Then the token should have a claim 'scope' # main_test.go:92 -> *Scaffold
And the token should have a claim 'sub' with value 'dips-mra' # main_test.go:106 -> *Scaffold
And the token should have a claim 'hso:userid' with value 'dips-mra' # main_test.go:106 -> *Scaffold
My hf1-idp.json file look like below.
{
"kind": "PING",
"issuer": "https://my.issuer.com/c2id",
"insecure": true,
"clients": {
"cc_test": {
"flow": "clientcredentials",
"id": "clientId",
"secret": "",
"scopes": ["openid", "solr"],
"values": {
"resource": ["https://my.solrnode1.com/solr/", "https://my.solrnode2.com/solr/"]
}
},
Connect2id works fine in the set up environment. As an example I get expected result when I run following Curl command with correct values
curl -k -s -H "Content-Type: application/json" -XPOST https://my.issuer.com/c2id/direct-authz/rest/v2 \
-H "Authorization: Bearer ztucBearerToken" \
-d '{
"sub_session" : { "sub" : "alice" },
"client_id" : "clientId",
"scope" : [ "openid", "solr" ],
"claims" : [ "name", "email", "email_verified", "access_token:hso:subject:system", "access_token:hso:subject:id", "access_token:hso:subject:name", "access_token:hso:subject:role:system", "access_token:hso:subject:role:id", "access_token:hso:subject:role:name", "access_token:hso:subject:organization:system", "access_token:hso:subject:organization:id", "access_token:hso:subject:organization:name", "access_token:hso:subject:organization:child-organization:system", "access_token:hso:subject:organization:child-organization:id", "access_token:hso:subject:organization:child-organization:name", "access_token:hso:purpose:system", "access_token:hso:purpose:id", "access_token:hso:purpose:description", "access_token:hso:resource:system", "access_token:hso:resource:id" ]
}'
Updated with following codes
main_test.go
func (sc *Scaffold) readIdentityProvider(filename string) error {
idp, err := idp.ReadIdentityProvider(context.Background(), "testdata/"+filename)
// More code goes here
}
provider.go
func ReadIdentityProvider(ctx context.Context, filename string) (*IdentityProvider, error) {
config, err := readIdentityProvider(filename)
if err != nil {
return nil, err
}
return NewIdentityProvider(ctx, config)
}
func NewIdentityProvider(ctx context.Context, config *Config) (*IdentityProvider, error) {
ctx = context.WithValue(ctx, oauth2.HTTPClient, &http.Client{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{
InsecureSkipVerify: config.Insecure,
},
},
})
provider, err := oidc.NewProvider(ctx, config.Issuer)
// More code goes here
}
oidc.go
func NewProvider(ctx context.Context, issuer string) (*Provider, error) {
wellKnown := strings.TrimSuffix(issuer, "/") + "/direct-authz/rest/v2"
req, err := http.NewRequest("POST", wellKnown, nil)
if err != nil {
return nil, err
}
resp, err := doRequest(ctx, req) // Herer I get 401 Unauthorized
if err != nil {
return nil, err
}
// More code goes here
}
You are missing the Bearer token in the request. (https://connect2id.com/products/server/docs/integration/direct-authz#error-401)
When you curl you use the -H parameter to add the Authorization Bearer token
-H "Authorization: Bearer ztucBearerToken"
You need to do the same in your Go application.
func NewProvider(ctx context.Context, issuer string) (*Provider, error) {
wellKnown := strings.TrimSuffix(issuer, "/") + "/direct-authz/rest/v2"
req, err := http.NewRequest("POST", wellKnown, nil)
bearer := "Bearer ztucBearerToken"
req.Header.Add("Authorization", bearer)
if err != nil {
return nil, err
}
resp, err := doRequest(ctx, req) // Herer I get 401 Unauthorized
if err != nil {
return nil, err
}
// More code goes here
}
Some supportive facts
Similar SO discussion goes here.
To get rid of the
400 Bad Request: {"error_description":"Bad request: Invalid JSON:
Unexpected token at position 0.","error":"invalid_request"}
what you have to do is pass necessary request body like below.
req, err := http.NewRequest("POST", wellKnown, bytes.NewBuffer(bytesRepresentation))
Fore more info visit, Golang documentation related to net/http

Converting curl request to Golang

So this curl request to the paypal payouts API works:
curl --silent -v 'https://api.sandbox.paypal.com/v1/oauth2/token' \
-H "Accept: application/json" \
-H "Accept-Language: en_US" \
-u "${client_id}:${client_secret}" \
-d "grant_type=client_credentials"
one thing I am confused about: the -d option is for data in the body of the HTTP request - does the -d option make it a POST request or is the curl request above a GET request with a body? I would guess that latter but I am unsure given the output of curl --help.
In golang I have:
package main
import (
"encoding/json"
"fmt"
"io/ioutil"
"log"
"net/http"
)
const (
PayPalTestClientID = "Aeit5RskDRN8eUUMB0Ud3RjA_z6feWMUHktwlJZMeQMo9A9ulbKK"
PayPalTestSecret = "EAAqyzrOTUWf-OFJCB4BxgXT4xuravL7pnkC8Tn20HYtZExd1mFO"
)
func main() {
//reader := bytes.NewBuffer()
req, err := http.NewRequest("GET", "https://api.sandbox.paypal.com/v1/oauth2/token", nil)
if err != nil {
log.Fatal(err)
}
req.Header.Set("Accept", "application/json")
req.Header.Set("Accept-Language", "en_US")
req.Header.Set("Authorization", fmt.Sprintf("Basic %s:%s", PayPalTestClientID, PayPalTestSecret))
client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
log.Fatal(err)
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatal(err)
}
var v interface{}
err = json.Unmarshal(body, &v);
if err != nil {
log.Fatal(err)
}
log.Print(v)
}
the client/secret were obfuscated, so they won't work as written above. But using the real creds I get:
2020/01/31 16:05:07 map[error:invalid_client error_description:Client
Authentication failed]
The real creds do work with the curl command tho.
Note: The credentials provided are valid? Cause i receive a sonorus 401, Authentication Failed.
NOTE: Using the -d in cURL, you are going to send a POST request instead of a GET. Due to this behaviour, your proably want to send a POST request instead of a GET
You can use my little http library: https://github.com/alessiosavi/Requests/
package main
import requests "github.com/alessiosavi/Requests"
func main() {
req, err := requests.InitRequest("https://postman-echo.com/basic-auth", "GET", []byte{}, false, false)
if err != nil {
fmt.Println("ERROR! ", err)
}
req.CreateHeaderList("Accept", "application/json", "Accept-Language", "en_US", "Authorization", "postman:password")
client := &http.Client{}
resp := req.ExecuteRequest(client)
fmt.Println(resp.Dump())
}
You can change the data (URL, post data, headers) with the one that you need for authenticate to the service.
In your case, will be something like this:
package main
import requests "github.com/alessiosavi/Requests"
const (
ID= "Aeit5RskDRN8eUUMB0Ud3RjA_z6feWMUHktwlJZMeQMo9A9ulbKK"
SECRET= "EAAqyzrOTUWf-OFJCB4BxgXT4xuravL7pnkC8Tn20HYtZExd1mFO"
)
func main() {
req, err := requests.InitRequest("https://api.sandbox.paypal.com/v1/oauth2/token", "GET", []byte{"grant_type=client_credentials"}, false, false)
if err != nil {
fmt.Println("ERROR! ", err)
}
req.CreateHeaderList("Accept", "application/json", "Accept-Language", "en_US", "Authorization", ID+":"+SECRET)
client := &http.Client{}
resp := req.ExecuteRequest(client)
fmt.Println(resp.Dump())
}

golan-gin api return invalid URL escape %

I want to create an api for uploading images in my go-gin application. This is my current api code :
func (s *Service) ImageCreate(c *gin.Context) {
token := c.MustGet(tokenKey).(*models.Token)
now := time.Now()
file, err := c.FormFile("file")
if err != nil {
c.JSON(http.StatusBadRequest, gin.H{"error": "01> " + err.Error()})
return
}
}
When I call my application from the browser it returns this error:
{"error":"01\u003e invalid URL escape \"%$\\xf4\""}
In postman api it's working without error. What is wrong? How I can fix it for the browser?
Don't use Postman.
curl -X POST http://localhost:8080/server-image/upload-file -F "file=#/Users/shikuanxu/Downloads/images/course-outline.png" -H "Content-Type: multipart/form-data"
Try again

equivalent to curl --data in go http request payload

For the purpose of setting a K/V in a NoSQL storage, I need to create an equivalent to the following curl command in go:
curl -H "Content-Type: text/xml" --data '[...]' http://localhost:8000/test/testrow/test:testcolumn
I am trying to use something in the lines of the following code for that purpose, although I am unable to find how to set the binary data []byte(value) as a POST payload.
func setColumn(table string, key string, col string, value string) {
url := "http://localhost:8123/" + table + "/" + key + "/" + col
req, err := http.NewRequest("POST", url, nil)
req.Header.Set("Content-Type", "application/octet-stream")
data = []byte(value)
client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
// handle error
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
fmt.Println(string(body))
}
So, how to map the data payload within the POST request ? Any indicators are welcome.
In order to post any data you should pass the value instead of nil
in the Newrequest method.
This should work
func setColumn(table string, key string, col string, value string) {
url := "http://localhost:8123/" + table + "/" + key + "/" + col
**data := bytes.NewReader([]byte(value))**
req, err := http.NewRequest("POST", url, **data**)
req.Header.Set("Content-Type", "application/octet-stream")
client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
// handle error
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
fmt.Println(string(body))
}
Most of the NoSQL Servers has GO API use them instad of creating your own HTTP request.

http.Post data-binary, curl equivalent in golang

I'm trying to use net/http to post a json file to ElasticSearch. Normally in Curl I would do the following:
curl -XPOST localhost:9200/prod/aws -d #aws.json
In golang I've used an example but it has not worked. I can see it posting but something must be set incorrectly. I've tested the JSON file I am using and it's good to go.
Go code:
target_url := "http://localhost:9200/prod/aws"
body_buf := bytes.NewBufferString("")
body_writer := multipart.NewWriter(body_buf)
jsonfile := "aws.json"
file_writer, err := body_writer.CreateFormFile("upfile", jsonfile)
if err != nil {
fmt.Println("error writing to buffer")
return
}
fh, err := os.Open(jsonfile)
if err != nil {
fmt.Println("error opening file")
return
}
io.Copy(file_writer, fh)
body_writer.Close()
http.Post(target_url, "application/json", body_buf)
If you want to read json from file then use .
jsonStr,err := ioutil.ReadFile("filename.json")
if(err!=nil){
panic(err)
}
Simple way to post json in http post request.
req, err := http.NewRequest("POST", url, bytes.NewBuffer(jsonStr))
req.Header.Set("Content-Type", "application/json")
client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
panic(err)
}
defer resp.Body.Close()
fmt.Println("response Status:", resp.Status)
body, _ := ioutil.ReadAll(resp.Body)
fmt.Println("response Body:", string(body))
This should work
Note that you can Post with an io.Reader as the body:
file, err := os.Open("./aws.json")
resp, err := http.Post(targetUrl, "application/json", file)
// TODO: handle errors
This might work better than reading the file contents into memory first, especially if the file is very large.

Resources