I need to retrieve the values of sub-claims from a JWT in Go.
I have (legacy) JWTs I need to parse in go, which contain a custom claim "data" which holds an Json-Object consisting of some fields (userid, username), so
{ [...standard claims]..., "data":{"id":"123", "name":"JohnDoe"} }
With using github.com/dgrijalva/jwt-go, I can parse the token and access the claims with this:
keyfunc := func(token *jwt.Token) (interface{}, error) {
return tknkey, nil
}
tkn, err := jwt.Parse(tknStr, keyfunc)
cl, _ := tkn.Claims.(jwt.MapClaims)
This works fine for the standard claims, and I also get the field names from the Json-Sub-Object in the "data" claim, but not the field values (all empty strings). I also tried setting up structs matching the claim hierarchy (outer and inner struct), with no success.
What would be the way to access the values of the sub-claims?
You can use jwt.MapClaims with "data": map[string]string with the following steps.
Steps 1.1 and 1.2 create the token
Steps 2.1 and 2.2 parse the token and extract the sub-claim values.
In the below example, jwt is github.com/golang-jwt/jwt/v4. Running code for this example is at github.com/grokify/goauth/examples/jwt/main.go.
Step 1.1: Create the claims
Create the custom MapClaims with a data map. Add a custom data.name property which we'll extract below.
claims := &jwt.MapClaims{
"iss": "issuer",
"exp": time.Now().Add(time.Hour).Unix(),
"data": map[string]string{
"id": "123",
"name": "JohnDoe",
},
}
Step 1.2: Create the JWT
For this example, we'll use a symmetric key.
token := jwt.NewWithClaims(
jwt.SigningMethodHS256,
claims)
secretKey := "foobar"
tokenString, err := token.SignedString([]byte(secretKey))
Step 2.1: Parse the token and cast claims to MapClaims.
Use the secretKey again since this example uses HS256.
token, err := jwt.Parse(tokenString, func(token *jwt.Token) (interface{}, error) {
return []byte(secretKey), nil
})
claims := token.Claims.(jwt.MapClaims)
Step 2.2: Extract custom sub-claim
Cast data to map[string]interface{} and cast data["name"] to string.
data := claims["data"].(map[string]interface{})
name := data["name"].(string)
Try this
type UserData struct {
Id string `json:"id"`
Name string `json:"name"`
}
type JWTClaim struct {
Data UserData `json:"data"`
jwt.StandardClaims
}
token, err := jwt.ParseWithClaims(
signedToken,
&JWTClaim{},
func(token *jwt.Token) (interface{}, error) {
return []byte(jwtKey), nil
},
)
Related
I'm trying to write simple JWT implementation with these functionalities:
Generating token using HMAC
Validating token (if signature is correct or exp is not timed out)
Decode token and getting claims
from scratch for better understanding how does it work in depth.
So far I found this article how to build an authentication microservice in golang from scratch. One chapter is dedicated to implementation JWT from scratch. I used it go generate token, however when I paste token in https://jwt.io I've got invalid signature and following warnings:
Warning: Looks like your JWT signature is not encoded correctly using base64url (https://tools.ietforg/html/rfc4648#section-5). Note that padding ("=") must be omitted as per https://tools.ietf.org/html/rfc7515#section-2
Warning: Looks like your JWT header is not encoded correctly using base64url (https://tools.ietforg/html/rfc4648#section-5). Note that padding ("=") must be omitted as per https://tools.ietf.org/html/rfc7515#section-2
Warning: Looks like your JWT payload is not encoded correctly using base64url (https://tools.ietforg/html/rfc4648#section-5). Note that padding ("=") must be omitted as per https://tools.ietf.org/html/rfc7515#section-2
Token I paste look like below:
eyAiYWxnIjogIkhTMjU2IiwgInR5cCI6ICJKV1QiIH0=.eyJhdWQiOiJmcm9udGVuZC5rbm93c2VhcmNoLm1sIiwiZXhwIjoiMTY1MTIyMjcyMyIsImlzcyI6Imtub3dzZWFyY2gubWwifQ==.SqCW8Hxakzck9Puzl0BEOkREPDyl38g2Fd4KFaDazV4=
My JWT code implementation:
package main
import (
"crypto/hmac"
"crypto/sha256"
"encoding/base64"
"encoding/json"
"fmt"
"strings"
"time"
)
func GenerateToken(header string, payload map[string]string, secret string) (string, error) {
h := hmac.New(sha256.New, []byte(secret))
header64 := base64.StdEncoding.EncodeToString([]byte(header))
payloadstr, err := json.Marshal(payload)
if err != nil {
return "", err
}
payload64 := base64.StdEncoding.EncodeToString(payloadstr)
message := header64 + "." + payload64
unsignedStr := header + string(payloadstr)
h.Write([]byte(unsignedStr))
signature := base64.StdEncoding.EncodeToString(h.Sum(nil))
tokenStr := message + "." + signature
return tokenStr, nil
}
func ValidateToken(token string, secret string) (bool, error) {
splitToken := strings.Split(token, ".")
if len(splitToken) != 3 {
return false, nil
}
header, err := base64.StdEncoding.DecodeString(splitToken[0])
if err != nil {
return false, err
}
payload, err := base64.StdEncoding.DecodeString(splitToken[1])
if err != nil {
return false, err
}
unsignedStr := string(header) + string(payload)
h := hmac.New(sha256.New, []byte(secret))
h.Write([]byte(unsignedStr))
signature := base64.StdEncoding.EncodeToString(h.Sum(nil))
fmt.Println(signature)
if signature != splitToken[2] {
return false, nil
}
return true, nil
}
func main() {
claimsMap := map[string]string{
"aud": "frontend.knowsearch.ml",
"iss": "knowsearch.ml",
"exp": fmt.Sprint(time.Now().Add(time.Second * 2).Unix()),
}
secret := "Secure_Random_String"
header := `{ "alg": "HS256", "typ": "JWT" }`
tokenString, err := GenerateToken(header, claimsMap, secret)
if err != nil {
fmt.Println(err)
return
}
fmt.Println("token: ", tokenString)
isValid, _ := ValidateToken(tokenString, secret)
fmt.Println("is token valid: ", isValid)
duration := time.Second * 4
time.Sleep(duration)
isValid, _ = ValidateToken(tokenString, secret)
fmt.Println("is token valid: ", isValid)
}
What's wrong with implementation above and how to fix it and get rid of warnings?
I decided to use Golang for implementation, however examples in any other languages very appreciated.
JWT specification requires that all padding = characters are removed:
Base64 encoding using the URL- and filename-safe character set
defined in Section 5 of RFC 4648 [RFC4648], with all trailing '='
characters omitted (as permitted by Section 3.2) and without the
inclusion of any line breaks, whitespace, or other additional
characters.
You can use base64.RawURLEncoding , which creates Base64Url encoding without padding, instead of base64.StdEncoding.
You can see the differences between the StdEncoding, RawStdEncodingand RawURLEncoding in this short Go Playground example.
Also, I strongly recommend to use a JWT library if it's not for learning exercise.
I am trying to implement JWT for REST
I'm just curious on this code below
var mySigningKey = []byte("mysecret")
token, err := jwt.Parse(r.Header["Token"][0], func(token *jwt.Token) (interface{}, error) {
if _, ok := token.Method.(*jwt.SigningMethodHMAC); !ok {
return nil, fmt.Errorf("There was an error")
}
return mySigningKey, nil
})
fmt.Println(token)
I expect token will return mySigningKey value but its return Token value
that shoud return value inside mySigningKey right?
Or did I misunderstand some concept?
By calling Parse method on your header Token value you are creating the token struct. If you want to get your SigningString you need to call SigningString() method on your token variable.
// Generate the signing string. This is the
// most expensive part of the whole deal. Unless you
// need this for something special, just go straight for
// the SignedString.
func (t *Token) SigningString() (string, error) {
var err error
parts := make([]string, 2)
for i, _ := range parts {
var source map[string]interface{}
if i == 0 {
source = t.Header
} else {
source = t.Claims
}
var jsonValue []byte
if jsonValue, err = json.Marshal(source); err != nil {
return "", err
}
parts[i] = EncodeSegment(jsonValue)
}
return strings.Join(parts, "."), nil
}
The function which is the second argument to Parse is responsible for providing the signing key which will be used to verify the JWT signature is correct.
In the case illustrated in the question, the signing key is a constant because HMAC is used to sign it.
However, if you had a RSA keypair, the token header would be useful to pick the right public key to check the signature, perhaps pulling it down from a well known JWKS endpoint.
I'm reading the example folder for JWT i'm a bit unsure how things work for validating the token.
func ExampleNewWithClaims_customClaimsType() {
mySigningKey := []byte("AllYourBase")
type MyCustomClaims struct {
Foo string `json:"foo"`
jwt.StandardClaims
}
// Create the Claims
claims := MyCustomClaims{
"bar",
jwt.StandardClaims{
ExpiresAt: 15000,
Issuer: "test",
},
}
token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
ss, err := token.SignedString(mySigningKey)
fmt.Printf("%v %v", ss, err)
//Output: eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJmb28iOiJiYXIiLCJleHAiOjE1MDAwLCJpc3MiOiJ0ZXN0In0.HE7fK0xOQwFEr4WDgRWj4teRPZ6i3GLwD5YCm6Pwu_c <nil>
}
Here it's straight forward and the token is being signed here token.SignedString(mySigningKey) using "mySigningKey"
Now there unparsing is much less clear for me :
func ExampleParseWithClaims_customClaimsType() {
tokenString := "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJmb28iOiJiYXIiLCJleHAiOjE1MDAwLCJpc3MiOiJ0ZXN0In0.HE7fK0xOQwFEr4WDgRWj4teRPZ6i3GLwD5YCm6Pwu_c"
type MyCustomClaims struct {
Foo string `json:"foo"`
jwt.StandardClaims
}
// sample token is expired. override time so it parses as valid
at(time.Unix(0, 0), func() {
token, err := jwt.ParseWithClaims(tokenString, &MyCustomClaims{}, func(token *jwt.Token) (interface{}, error) {
return []byte("AllYourBase"), nil
})
if claims, ok := token.Claims.(*MyCustomClaims); ok && token.Valid {
fmt.Printf("%v %v", claims.Foo, claims.StandardClaims.ExpiresAt)
} else {
fmt.Println(err)
}
})
// Output: bar 15000
}
Is order to validate that the signature of the token string given back by the client is valid would you need to
decode the claim ( done in &MyCustomClaims{} )
validate that the signature part of the decoded claim is valid against the " pub key included in the token" using token.Valid.
But the example is just decoding the key and by "magic" the return is the secret/signing key?
It doesn't make sense to me at all. Also returning a valid claim for a public key is useless as it could have been done by any private key.
What am i missing ?
Validation is not done by the public key included in the token.
HS256 is symmetric, so whatever key you used to sign the token, you have to use the same key to validate the signature, and that's what's happening. The function passed to ParseWithClaims is returning the same key used to sign the token.
If an asymmetric signing algorithm was used, you'd use the private key to sign the token, and then the public key to validate it, and that nested function would have to return the public key so that ParseWithClaims can validate it.
It sounds like the part that confused you is:
jwt.ParseWithClaims(tokenString, &MyCustomClaims{}, func(token *jwt.Token) (interface{}, error) {
return []byte("AllYourBase"), nil
})
The nested function passed to ParseWithClaims is supposed to inspect the token passed, and return the correct key that can be used to verify the signature. For HS256, it is the same as the key used to sign it. For RSxxx, it'd be the public key, and which public key it should return can be retrieved from the token passed in. It usually contains a public key id so you can select the correct key.
func Login(c echo.Context) error {
user := &users.User{}
if err := c.Bind(&user); err != nil {
return err
}
return token.SigIn(c, user.Email, user.Password)
}
This is my Login function that retrieve the token when the user send the requests.
the Signin func that handle the token
func SigIn(c echo.Context, email, password string) error {
user := users.User{}
db := database.SetUp()
if err := db.Where("email = ?", email).First(&user).Error; gorm.IsRecordNotFoundError(err) {
restErr := errors.NewBadRequestError("Invalid credentials")
return c.JSON(http.StatusBadRequest, restErr)
}
if user.VerifyPassword(password) != nil {
restErr := errors.NewUnauthorizedError("Couldn't log you in with these credentials")
return c.JSON(http.StatusUnauthorized, restErr)
}
//user is successfull
return CreateToken(c)
}
the CreateToken func is as follow
type TokenJWT struct {
Token string `json:"token"`
}
func CreateToken(c echo.Context) error {
token := jwt.New(jwt.SigningMethodHS256)
claims := token.Claims.(jwt.MapClaims)
claims["authorized"] = true
claims["name"] = "Pascal Gaetan"
claims["exp"] = time.Now().Add(time.Hour * 1).Unix()
// Generate encoded token and send it as response.
t, err := token.SignedString([]byte("my_secret_key"))
if err != nil {
return err
}
return c.JSON(http.StatusOK, TokenJWT{
Token: t,
})
}
when everyhting is succesfull, i would like to get the authenticated user through an URL /api/me that calls a Me function
Let me split your question into two parts: the first one is how to easily encode and decode user in or from JWT token and the second part is how to write a generic code which can retrieve user from everywhere.
From your example I mentioned that you created a MapClaims but to reduce parsing complexity it will be better to create a token using a custom claims type. If you are using dgrijalva/jwt-go, then according to documentation you can do something like that
type UserClaims struct {
Name string `json:"name"`
jwt.StandardClaims
}
// encode it as before, but with your created type
t := jwt.New(signer)
userClaims := &UserClaims{Name: "Burmese"}
t.Claims = userClaims
tokenString, err = t.SignedString(]byte("my_secret_key"))
then you can parse your user in your router/framework middleware with
tokenString := "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJmb28iOiJiYXIiLCJleHAiOjE1MDAwLCJpc3MiOiJ0ZXN0In0.HE7fK0xOQwFEr4WDgRWj4teRPZ6i3GLwD5YCm6Pwu_c"
token, err := jwt.ParseWithClaims(tokenString, &UserClaims{}, func(token *jwt.Token) (interface{}, error) {
return []byte("my_secret_key"), nil
})
if claims, ok := token.Claims.(*UserClaims); ok && token.Valid {
fmt.Printf("%v %v", claims.Name, claims.StandardClaims.ExpiresAt)
} else {
fmt.Println(err)
}
This example was adopted from an official documentation here
Now you know how to parse authenticated user struct with ease and the next logic move is to wrap it into your middleware. Whether there are a lot of implementation details like you can retrieve JWT from cookie, header or query, also defining some ordering on them, the gist the following: you should have wrapped abovementioned code into your middleware and after parsing the struct you can pass it via your request context. I don't use echo and other frameworks, but for pure net/http you can pass your parsed struct from middleware with
context.WithValue(ctx, UserCtxKey, claims)
Hope it helps!
This is a fairly common design pattern to create an authenticated client and then call various action methods on it. You could do something like the following:
type Client struct {
... // other members
token string // unexported unless there is a special reason to do otherwise
}
func NewClient(c echo.Context, email, password string) (*Client, error) {
user := users.User{}
cl := Client{}
... // your original method
cl.token = token
return &cl, nil
}
func (c *Client) DoSomething(...) ... { ... }
I am trying to test some golang code and I have a method that calls several other methods from its body. All these methods perform some kind of operations using an elastic search client. I wanted to know whether it will be a good practice if I used a test server for testing this method that will write different responses depending upon the request method and path it received from the request that is made when the methods inside the body execute and make calls to the elasticsearch client that sends the requests to my test server?
Update:
I am testing an elasticsearch middleware. It implements a reindex service like this
type reindexService interface {
reindex(ctx context.Context, index string, mappings, settings map[string]interface{}, includes, excludes, types []string) error
mappingsOf(ctx context.Context, index string) (map[string]interface{}, error)
settingsOf(ctx context.Context, index string) (map[string]interface{}, error)
aliasesOf(ctx context.Context, index string) ([]string, error)
createIndex(ctx context.Context, name string, body map[string]interface{}) error
deleteIndex(ctx context.Context, name string) error
setAlias(ctx context.Context, index string, aliases ...string) error
getIndicesByAlias(ctx context.Context, alias string) ([]string, error)
}
I can easily test all the methods using this pattern. Creating a simple elastic search client using a httptest server url and making requests to that server
var createIndexTests = []struct {
setup *ServerSetup
index string
err string
}{
{
&ServerSetup{
Method: "PUT",
Path: "/test",
Body: `null`,
Response: `{"acknowledged": true, "shards_acknowledged": true, "index": "test"}`,
},
"test",
"",
},
// More test cases here
}
func TestCreateIndex(t *testing.T) {
for _, tt := range createIndexTests {
t.Run("Should successfully create index with a valid setup", func(t *testing.T) {
ctx := context.Background()
ts := buildTestServer(t, tt.setup)
defer ts.Close()
es, _ := newTestClient(ts.URL)
err := es.createIndex(ctx, tt.index, nil)
if !compareErrs(tt.err, err) {
t.Fatalf("Index creation should have failed with error: %v got: %v instead\n", tt.err, err)
}
})
}
}
But in case of reindex method this approach poses a problem since reindex makes calls to all the other methods inside its body. reindex looks something like this:
func (es *elasticsearch) reindex(ctx context.Context, indexName string, mappings, settings map[string]interface{}, includes, excludes, types []string) error {
var err error
// Some preflight checks
// If mappings are not passed, we fetch the mappings of the old index.
if mappings == nil {
mappings, err = es.mappingsOf(ctx, indexName)
// handle err
}
// If settings are not passed, we fetch the settings of the old index.
if settings == nil {
settings, err = es.settingsOf(ctx, indexName)
// handle err
}
// Setup the destination index prior to running the _reindex action.
body := make(map[string]interface{})
body["mappings"] = mappings
body["settings"] = settings
newIndexName, err := reindexedName(indexName)
// handle err
err = es.createIndex(ctx, newIndexName, body)
// handle err
// Some additional operations
// Reindex action.
_, err = es.client.Reindex().
Body(reindexBody).
Do(ctx)
// handle err
// Fetch all the aliases of old index
aliases, err := es.aliasesOf(ctx, indexName)
// handle err
aliases = append(aliases, indexName)
// Delete old index
err = es.deleteIndex(ctx, indexName)
// handle err
// Set aliases of old index to the new index.
err = es.setAlias(ctx, newIndexName, aliases...)
// handle err
return nil
}
For testing the reindex method I have tried mocking and DI but that turns out to be hard since the methods are defined on a struct instead of passing an interface as an argument to them. (So now I want to keep the implementation same since it would require making changes to all the plugin implementations and I want to avoid that)
I wanted to know whether I can use a modified version of my build server funtion (the one I am using is given below) to return responses for different methods for the reindex service which will write the appropriate responses based on the HTTP method and the request path that is used by that method?
type ServerSetup struct {
Method, Path, Body, Response string
HTTPStatus int
}
// This function is a modified version of: https://github.com/github/vulcanizer/blob/master/es_test.go
func buildTestServer(t *testing.T, setup *ServerSetup) *httptest.Server {
handlerFunc := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
requestBytes, _ := ioutil.ReadAll(r.Body)
requestBody := string(requestBytes)
matched := false
if r.Method == setup.Method && r.URL.EscapedPath() == setup.Path && requestBody == setup.Body {
matched = true
if setup.HTTPStatus == 0 {
w.WriteHeader(http.StatusOK)
} else {
w.WriteHeader(setup.HTTPStatus)
}
_, err := w.Write([]byte(setup.Response))
if err != nil {
t.Fatalf("Unable to write test server response: %v", err)
}
}
// TODO: remove before pushing
/*if !reflect.DeepEqual(r.URL.EscapedPath(), setup.Path) {
t.Fatalf("wanted: %s got: %s\n", setup.Path, r.URL.EscapedPath())
}*/
if !matched {
t.Fatalf("No requests matched setup. Got method %s, Path %s, body %s\n", r.Method, r.URL.EscapedPath(), requestBody)
}
})
return httptest.NewServer(handlerFunc)
}
Something like this function but it takes a map of request methods and past mapped to appropriate responses and writes them to the writer?