According to Coinbase pro API docs:
The CB-ACCESS-SIGN header is generated by creating a sha256 HMAC using the base64-decoded secret key on the prehash string timestamp +
method + requestPath + body (where + represents string concatenation)
and base64-encode the output. The timestamp value is the same as the
CB-ACCESS-TIMESTAMP header.
The body is the request body string or omitted if there is no request
body (typically for GET requests).
The method should be UPPER CASE.
I borrowed a signing function from a better programmer and feed it something like this:
1619383731POST/reports{{"end_date":"2021-01-02T11:59:59Z","start_date":"2020-01-01T00:00:00Z","type":"account"}}
But keep getting invalid signature from Coinbase.
Signing function for reference:
// sign
func (e *exchange) sign(msg string) string {
key, err := base64.StdEncoding.DecodeString(e.http.secret)
if e.checkErr(err) {
return "bad_sig"
}
signature := hmac.New(sha256.New, key)
_, err = signature.Write([]byte(msg))
if e.checkErr(err) {
return "bad_sig"
}
return base64.StdEncoding.EncodeToString(signature.Sum(nil))
}
Where am I screwing up?
Have you checked out the go-coinbase github repo implementing this as:
https://github.com/preichenberger/go-coinbasepro/blob/master/client.go
h := make(map[string]string)
h["CB-ACCESS-KEY"] = c.Key
h["CB-ACCESS-PASSPHRASE"] = c.Passphrase
h["CB-ACCESS-TIMESTAMP"] = timestamp
message := fmt.Sprintf(
"%s%s%s%s",
timestamp,
method,
url,
data,
)
sig, err := generateSig(message, c.Secret)
if err != nil {
return nil, err
}
h["CB-ACCESS-SIGN"] = sig
return h, nil
It turns out using req.Body directly is the culprit. Why I don't know and something I need to find out, but reading into a []byte and then casting to a string solves part of it.
Related
I'm using range to loop through an array of structs to extract data which will be used as a URL parameter for my API calls. Within this loop, I'm trying to push response data from one struct to another.
I'm able to get everything working, except for moving data from one struct to another, but not entirely sure how to solve for the errors I keep getting. I've tried multiple methods and seem to be stuck in the mud here for something I don't consider to be too hard, until now... In my code I'm using the append method but I'm not so sure that might be the correct way to proceed.
Presenting my code:
models.go
//Here is my existing struct, with populated data that I get from a CSV
type TravelItenaries struct {
Origin string
Destination string
Flight_num string
Origin_latitude string
Origin_longitude string
Destination_latitude string
Destination_longitude string
Origin_weather string
Destination_weather string
Coordinates_ori string
Coordinates_dest string
Temp_c_ori string
Temp_f_ori string
Temp_c_dest string
Temp_f_dest string
}
//Here is the response data that I'm expected to get from my API calls.
//I'm trying to "push" Temp_c_dest and Temp_f_dest data into TravelItenaries.Temp_f_dest and TravelItenaries.Temp_c_dest
//While also changing the data types to fit above.
type Response struct {
Current struct {
LastUpdatedEpoch int `json:"last_updated_epoch"`
LastUpdated string `json:"last_updated"`
Temp_c_dest float64 `json:"temp_c"`
Temp_c_dest float64 `json:"temp_f"`
IsDay int `json:"is_day"`
} `json:"current"
}
weather.go
func (s *Server) getWeather(w http.ResponseWriter, r *http.Request) {
// open file
f, err := os.Open("challenge_dataset.csv")
if err != nil {
responses.ERROR(w, http.StatusBadRequest, fmt.Errorf("helpful error"))
return
}
// remember to close the file at the end of the program
defer f.Close()
// read csv values using csv.Reader
csvReader := csv.NewReader(f)
data, err := csvReader.ReadAll()
if err != nil {
responses.ERROR(w, http.StatusBadRequest, fmt.Errorf("helpful error"))
return
}
// convert records to array of structs
travelItenaries := createTravelItenaries(data)
// remove duplicate flight records
cleanTravelItenaries:= remDupKeys(travelItenaries)
// set up params for API get request
params := url.Values{
"key": []string{"xxx"},
"q": []string{""},
}
// Construct URL for API request
u := &url.URL{
Scheme: "https",
Host: "api.weatherapi.com",
Path: "/v1/current.json",
RawQuery: params.Encode(),
}
client := &http.Client{}
// Will need this to populate the params using a range over a struct
values := u.Query()
// loop through cleaned data set
for _, service := range cleanTravelItenaries {
// dynamically acquire data from struct to pass as parameter
values.Set("q", service.Coordinates_dest)
u.RawQuery = values.Encode()
req, err := http.NewRequest("GET", u.String(), nil)
if err != nil {
responses.ERROR(w, http.StatusBadRequest, fmt.Errorf("helpful error"))
return
}
resp, err := client.Do(req)
if err != nil {
responses.ERROR(w, http.StatusBadRequest, fmt.Errorf("helpful error"))
return
}
body, err := ioutil.ReadAll(resp.Body)
// create empty struct to parse response data with using Inmarshal
var responseData models.Response
json.Unmarshal(body, &responseData)
// Here is the issue, I don't think append might be the correct procedure here?
// I simply just need to pass this response data to my already existing struct
service.Temp_c_dest = append(responseData.Current.Temp_c_dest , cleanTravelItenaries )
service.Temp_f_dest = append(responseData.Current.Temp_f_dest , cleanTravelItenaries )
}
}
The errors I get are related to both append statements at the end of the range function.
first argument to append must be slice; have float64
first argument to append must be slice; have float64
for both append methods.
Also, note how type TravelItenaries struct uses string type for:
Temp_c_dest string
Temp_f_dest string
Hence why I also need to do some field type conversion from Float64 to string.
How can I extract the fields Temp_c_dest and Temp_f_dest from API response struct to TravelItenaries struct fields while changing datatypes?
EDIT:
I've managed to get this somewhat working, but only inside the for loop. The data is not being saved outside the function.
service.Temp_f_dest = strconv.FormatFloat(responseData.Current.Temp_f_dest, 'g', -1, 64)
service.Temp_c_dest = strconv.FormatFloat(responseData.Current.Temp_c_dest, 'g', -1, 64)
I'm trying to write simple JWT implementation with these functionalities:
Generating token using HMAC
Validating token (if signature is correct or exp is not timed out)
Decode token and getting claims
from scratch for better understanding how does it work in depth.
So far I found this article how to build an authentication microservice in golang from scratch. One chapter is dedicated to implementation JWT from scratch. I used it go generate token, however when I paste token in https://jwt.io I've got invalid signature and following warnings:
Warning: Looks like your JWT signature is not encoded correctly using base64url (https://tools.ietforg/html/rfc4648#section-5). Note that padding ("=") must be omitted as per https://tools.ietf.org/html/rfc7515#section-2
Warning: Looks like your JWT header is not encoded correctly using base64url (https://tools.ietforg/html/rfc4648#section-5). Note that padding ("=") must be omitted as per https://tools.ietf.org/html/rfc7515#section-2
Warning: Looks like your JWT payload is not encoded correctly using base64url (https://tools.ietforg/html/rfc4648#section-5). Note that padding ("=") must be omitted as per https://tools.ietf.org/html/rfc7515#section-2
Token I paste look like below:
eyAiYWxnIjogIkhTMjU2IiwgInR5cCI6ICJKV1QiIH0=.eyJhdWQiOiJmcm9udGVuZC5rbm93c2VhcmNoLm1sIiwiZXhwIjoiMTY1MTIyMjcyMyIsImlzcyI6Imtub3dzZWFyY2gubWwifQ==.SqCW8Hxakzck9Puzl0BEOkREPDyl38g2Fd4KFaDazV4=
My JWT code implementation:
package main
import (
"crypto/hmac"
"crypto/sha256"
"encoding/base64"
"encoding/json"
"fmt"
"strings"
"time"
)
func GenerateToken(header string, payload map[string]string, secret string) (string, error) {
h := hmac.New(sha256.New, []byte(secret))
header64 := base64.StdEncoding.EncodeToString([]byte(header))
payloadstr, err := json.Marshal(payload)
if err != nil {
return "", err
}
payload64 := base64.StdEncoding.EncodeToString(payloadstr)
message := header64 + "." + payload64
unsignedStr := header + string(payloadstr)
h.Write([]byte(unsignedStr))
signature := base64.StdEncoding.EncodeToString(h.Sum(nil))
tokenStr := message + "." + signature
return tokenStr, nil
}
func ValidateToken(token string, secret string) (bool, error) {
splitToken := strings.Split(token, ".")
if len(splitToken) != 3 {
return false, nil
}
header, err := base64.StdEncoding.DecodeString(splitToken[0])
if err != nil {
return false, err
}
payload, err := base64.StdEncoding.DecodeString(splitToken[1])
if err != nil {
return false, err
}
unsignedStr := string(header) + string(payload)
h := hmac.New(sha256.New, []byte(secret))
h.Write([]byte(unsignedStr))
signature := base64.StdEncoding.EncodeToString(h.Sum(nil))
fmt.Println(signature)
if signature != splitToken[2] {
return false, nil
}
return true, nil
}
func main() {
claimsMap := map[string]string{
"aud": "frontend.knowsearch.ml",
"iss": "knowsearch.ml",
"exp": fmt.Sprint(time.Now().Add(time.Second * 2).Unix()),
}
secret := "Secure_Random_String"
header := `{ "alg": "HS256", "typ": "JWT" }`
tokenString, err := GenerateToken(header, claimsMap, secret)
if err != nil {
fmt.Println(err)
return
}
fmt.Println("token: ", tokenString)
isValid, _ := ValidateToken(tokenString, secret)
fmt.Println("is token valid: ", isValid)
duration := time.Second * 4
time.Sleep(duration)
isValid, _ = ValidateToken(tokenString, secret)
fmt.Println("is token valid: ", isValid)
}
What's wrong with implementation above and how to fix it and get rid of warnings?
I decided to use Golang for implementation, however examples in any other languages very appreciated.
JWT specification requires that all padding = characters are removed:
Base64 encoding using the URL- and filename-safe character set
defined in Section 5 of RFC 4648 [RFC4648], with all trailing '='
characters omitted (as permitted by Section 3.2) and without the
inclusion of any line breaks, whitespace, or other additional
characters.
You can use base64.RawURLEncoding , which creates Base64Url encoding without padding, instead of base64.StdEncoding.
You can see the differences between the StdEncoding, RawStdEncodingand RawURLEncoding in this short Go Playground example.
Also, I strongly recommend to use a JWT library if it's not for learning exercise.
I am trying to implement JWT for REST
I'm just curious on this code below
var mySigningKey = []byte("mysecret")
token, err := jwt.Parse(r.Header["Token"][0], func(token *jwt.Token) (interface{}, error) {
if _, ok := token.Method.(*jwt.SigningMethodHMAC); !ok {
return nil, fmt.Errorf("There was an error")
}
return mySigningKey, nil
})
fmt.Println(token)
I expect token will return mySigningKey value but its return Token value
that shoud return value inside mySigningKey right?
Or did I misunderstand some concept?
By calling Parse method on your header Token value you are creating the token struct. If you want to get your SigningString you need to call SigningString() method on your token variable.
// Generate the signing string. This is the
// most expensive part of the whole deal. Unless you
// need this for something special, just go straight for
// the SignedString.
func (t *Token) SigningString() (string, error) {
var err error
parts := make([]string, 2)
for i, _ := range parts {
var source map[string]interface{}
if i == 0 {
source = t.Header
} else {
source = t.Claims
}
var jsonValue []byte
if jsonValue, err = json.Marshal(source); err != nil {
return "", err
}
parts[i] = EncodeSegment(jsonValue)
}
return strings.Join(parts, "."), nil
}
The function which is the second argument to Parse is responsible for providing the signing key which will be used to verify the JWT signature is correct.
In the case illustrated in the question, the signing key is a constant because HMAC is used to sign it.
However, if you had a RSA keypair, the token header would be useful to pick the right public key to check the signature, perhaps pulling it down from a well known JWKS endpoint.
I am trying to create a buildURL function that can generate an url. However I have run into a problem when facing a dynamic url.
type ShopifyDownloader struct {
Domain string
AccessToken string
}
func (sd *ShopifyDownloader) BuildURL(Path string, Query map[string]string) (string, error) {
u, err := url.Parse("https://123.myshopify.com/admin/orders/count.json?access_token=123")
if err != nil {
fmt.Println("Cannot parse", err)
return "", err
}
u.Host = sd.Domain
u.Path = Path
params := url.Values{}
for key, value := range Query {
params.Add(key, value)
}
u.RawQuery = params.Encode()
nu := u.String()
var buffer bytes.Buffer
buffer.WriteString(nu)
buffer.WriteString("?access_token=")
buffer.WriteString(sd.AccessToken)
bu := buffer.String()
return bu, err
}
It takes domain, token, path and queries and generate an url in string. However it does not work with a dynamic path (with id) like
this /admin/orders/#{id}.json or this /admin/products/#{id}/images/#{id}.json
Does anybody have any suggestion?
Why not just build a url with something simpler at the point where you use it?
url := fmt.Sprintf("%s/admin/orders/count.json?access_token=%s&custom=%s",sd.Domain,sd.Token,customparam)
Then use it. This has the virtue of being completely clear as to intent and the url used in each case while sharing the important stuff (shopify domain etc). Then for your question about ids it becomes easier:
url := fmt.Sprintf("%s/admin/orders/%d.json?access_token=%s",sd.Domain,orderID,sd.Token)
and
url := fmt.Sprintf("%s/admin/products/%d/images/%d.json?access_token=%s",sd.Domain,productID,imageID,sd.Token)
if you feel your params are super complex (for me this would have to be at least 5), just put them in an url.Values (including the token):
params := url.Values{"myparam":{"123"}, "token": {sd.Token}}
url := fmt.Sprintf("%s/admin/orders/count.json?%s", sd.Domain,
params.Encode())
but this is only worthwhile if you have a significant number of params.
If you want to add functions to ShopifyDownloader, why force the caller to define the url structure? Try to make it simpler for them. You don't need to parse urls or write to a bytes buffer, http.Get expects a string, so work with strings. If you want to build functions, consider instead having separate functions which know about the url formats, to abstract this away from the caller, so :
func (sd *ShopifyDownloader) OrderURL(orderID int, params url.Values) string {
params["token"] = []string{sd.AccessToken}
return fmt.Sprintf("%s/admin/orders/%d.json?%s",sd.Domain,orderID,params)
}
The caller should not know about the specific format of Shopify urls, which may change (just as the endpoint may change).
I'm sending POST request:
req, err := http.NewRequest("POST", link, bytes.NewBuffer(jsonStr))
client := &http.Client{Timeout: tm}
resp, err := client.Do(req)
I receive resp.Header in format with type http.Header
I need to something like this:
[
"Server: nginx/1.4.4",
"Date: Wed, 24 Feb 2016 19:09:49 GMT"
]
I don't know how to approach this problem, because I don't know how to deal with http.Header datatype. Could someone help please
resp.Header is of type http.Header. You can see in the documentation that this type is also a map, so you can access it in two different ways:
1) By using http.Header's methods:
serverValue := resp.Header().Get("Server")
dataValue := resp.Header().Get("Date")
If the header exists, you'll get its first value (keep in mind that there might be multiple values for a single header name); otherwise you'll get an empty string.
2) By using map's methods:
serverValue, ok := resp.Header()["Server"]
dataValue, ok := resp.Header()["Date"]
If the header exists, ok will be true (i.e. the header exists) and you'll get a slice of strings containing all the values for that header; otherwise, ok will be false (i.e. the header doesn't exist).
Use whichever method you prefer.
If you need to iterate over all the header values, you can do it with something like:
for name, value := range resp.Header() {
fmt.Printf("%v: %v\n", name, value)
}
You could use a function like this one:
func HeaderToArray(header http.Header) (res []string) {
for name, values := range header {
for _, value := range values {
res = append(res, fmt.Sprintf("%s: %s", name, value))
}
}
return
}
It should return an array like the one you want.
This solutions is for go version go1.13 windows/amd64.
The request object from http.Request contains Header object. We are using net/http package here. You can get values of all headers by name using following method:
import(
"net/http"
)
type RequestHeaders struct {
ContentType string `json: "content-type"`
Authorization string `json: "authorization"`
}
func getHeaders(r *http.Request) RequestHeaders {
contentType := r.Header.Get("Content-Type")
authorization := r.Header.Get("Authorization")
headers := RequestHeaders{
Content-Type: contentType,
Authorization: authorization}
return headers
}
You can see that we are using r.Header.Get("Content-Type") method to get value of the header.
If the header is missing the Get() method will return empty string.
You can retrieve the response header's first value with resp.Header().Get(), which returns "" if the header key has no values.
Hence, in your case
var a [2]string
a[0] = resp.Header().Get("server")
a[1] = resp.Header().Get("date")