In my pure Ruby app one of the components to create a token for my request authentication to an external API is to create signature which is HMAC value that is created using the api_key and the secret_key. The signature contains the following elements that are each separated by a new line \n (except the last line) and are in the same order as below list:
ts = '1529342939277'
nonce = '883b170c-a768-41a1-ae6d-c626323aa128'
host = 'ws.idms.lexisnexis.com'
resource_path = '/restws/identity/v3/accounts/11111/workflows/rdp.test.workflow/conversations'
body_hash = 'fQoIAs0IO4vNleZVE9tcI3Ni7h+niT+GrrgEHsKZOyM='
API_KEY = '6njQLkz7uCiz1ZeJ1bFCWX4DFVTfKQXa'
SECRET_KEY = 'CcdaZEt7co647iJoEc5G29CHtlo7T9M3'
# create string signature separated by new line
signature = [ts, nonce, host, resource_path, body_hash].join("\n")
# create HMAC for signature
mac = Base64.strict_encode64(OpenSSL::HMAC.hexdigest('SHA256', API_KEY, signature))
2.7.0 :146 > mac
=> "ZDE4NDQxZDdiNmZkODNiODgyODI4Nzc2OTQ3OGFlMjVhZTMyNThhZTZlMTRiMjkxMzI0NmQ5NzljNDJkZWVhZg=="
According to the docs the signature should be Syb6i+sRygAGCgxLQJ4NwwKcT5Mnkh4r3QXgwZ3vmcE= but I'm getting ZDE4NDQxZDdiNmZkODNiODgyODI4Nzc2OTQ3OGFlMjVhZTMyNThhZTZlMTRiMjkxMzI0NmQ5NzljNDJkZWVhZg== instead. Where did I go wrong?
I've got an example how to do it in Java if something will be unclear: https://gist.github.com/mrmuscle1234/20c9d46d163fee66528449c0ea8419a7
Related
I have a Nuxt application that needs to retrieve some information from a Spring Boot-based auth service.
Right now I sign a text message on the Nuxt app (the auth server is aware of that text message), using node-forge, and then I send it encrypted and with the signature for verification on the auth service.
The problem is that the auth service keeps telling me that the size of the signature is wrong, with a java.security.SignatureException: Signature length not correct: got 3XX but was expecting 256.
Here is the code generating the encrypted message and signature on the Nuxt side:
var md = forge.md.sha256.create();
md.update("123"); // for example purposes
var sign = pPrivateKey.sign(md);
var digestBytes = md.digest().bytes();
console.log("Signature:", sign );
console.log("Encrypted:", digestBytes);
console.log("Encrypted B64:", Buffer.from(digestBytes).toString("base64"));
var keyAuthB64Url = Buffer.from(digestBytes).toString("base64url");
var signB64Url = Buffer.from(sign).toString("base64url");
var jwt = await axios.get(process.env.URL + "/auth", { params: { encrypted: keyAuthB64Url, signature: signB64Url } });
On the auth service I have the following code:
byte[] messageBytes = Base64.getUrlDecoder().decode(encryptedMessage);
byte[] signatureBytes = Base64.getUrlDecoder().decode(signature);
Signature sign = Signature.getInstance("SHA256withRSA");
sign.initVerify(certPublicKey);
sign.update(messageBytes);
boolean verified = sign.verify(signatureBytes);
if (!verified) {
throw new Exception("Not verified!");
}
From all the debugging I have done, it seems like the Spring Boot app has a problem with the signature generated by node-forge on the Nuxt side, with a signature generated in the Spring Boot app the verification works.
There are several issues:
First, the bug that was already mentioned in the comment: While the NodeJS code does not hash implicitly, the Java side does. Therefore, hashing must not be done explicitly on the Java side:
byte[] messageBytes = "123".getBytes("utf-8");
...
sign.update(messageBytes); // Fix 1: Don't hash
Also, in the NodeJS code, sign() returns the data as a bytes string, which must therefore be imported into a NodeJS buffer as a 'binary':
var keyAuthB64Url = Buffer.from(digestBytes, "binary").toString("base64url"); // Fix 2: Import via 'binary' encoding
Without explicit specification of the encoding, a UTF-8 encoding is performed by default, which irreversibly corrupts the data.
And third, latin1 is implicitly used as encoding when generating the hash in the NodeJS code. Other encodings must be specified explicitly, e.g. for the common UTF-8 with utf8:
md.update("123", "utf8"); // Fix 3: Specify the encoding
For the example data 123 used here, this fix has no effect, which changes as soon as characters with a Unicode value larger than 0x7f are included, e.g. 123ยง. Note that there is little margin for error in the specification of the encoding, e.g. utf-8 would be ignored (because of the hyphen) and latin1 would be used silently.
With these fixes, verification with the Java code works.
I'm attempting to port the GitHub Apps sample starter code from Ruby to Python, but I'm running into trouble whilst generating the required JWT. The Ruby script looks like this, and works fine:
require 'openssl'
require 'jwt' # https://rubygems.org/gems/jwt
private_pem = File.read(YOUR_PATH_TO_PEM)
private_key = OpenSSL::PKey::RSA.new(private_pem)
payload = {
iat: Time.now.to_i,
exp: Time.now.to_i + (10 * 60),
iss: GITHUB_APP_IDENTIFIER
}
jwt = JWT.encode(payload, private_key, "RS256")
puts jwt
My Python script is as follows and produces the A JSON web token could not be decoded error when used against the GitHub API:
import os
import time
import jwt
APP_IDENTIFIER = os.environ["GITHUB_APP_IDENTIFIER"]
with open('./PRIVATE_KEY.pem', 'r') as f:
PRIVATE_KEY = f.read()
payload = {"iat": int(time.time()),
"exp": int(time.time()) + (10*60),
"iss": APP_IDENTIFIER}
print(jwt.encode(payload, PRIVATE_KEY, algorithm='RS256'))
When I tried printing the private keys from both scripts, I found that the Ruby version had an extra newline character. I tried adding this to the private key in the Python script, but it didn't change the output.
My best guess is that the difference is something to do with the OpenSSL::PKey::RSA.new call, but I'm not sure what that does to the key.
jwt.encode() will return you bytes in Python 3 which probably ends up having str() called on it somewhere in the sending pipeline. Calling str() on bytes objects in Python 3 can result in potentially surprising behaviour:
>>> a = b'hello'
>>> str(a)
"b'hello'"
The correct way to turn bytes into a string in Python 3 is to use:
>>> a.decode('utf-8')
'hello'
I added the call to decode on the end of my jwt.encode line and the API suddenly had no problems decoding the JWT.
I'm trying to secure a Sinatra API.
I'm using ruby-jwt to create the JWT, but I don't know exactly what to sign it with.
I'm trying to use the user's BCrypt password_digest, but every time password_digest is called it changes, making the signature invalid when I go to verify it.
Use any kind of application secret key, not a user's bcrypt password digest.
For example, use the dot env gem and a .env file, with an entry such as:
JWT_KEY=YOURSIGNINGKEYGOESHERE
I personally generate a key by using a simple random hex string:
SecureRandom.hex(64)
The hex string contains just 0-9 and a-f, so the string is URL safe.
For RS256 public and private key strategy you can use Ruby OpenSSL lib:
Generating keys:
key = OpenSSL::PKey::RSA.new 2048
open 'private_key.pem', 'w' do |io| io.write key.to_pem end
open 'public_key.pem', 'w' do |io| io.write key.public_key.to_pem end
Load key from .pem file to sign token:
priv_key = OpenSSL::PKey::RSA.new File.read 'private_key.pem'
token = JWT.encode payload, priv_key, 'RS256'
Load key from .pem file to Verify token(Create a middleware for this):
begin
# env.fetch gets http header
bearer = env.fetch('HTTP_AUTHORIZATION').slice(7..-1)
pub_key = OpenSSL::PKey::RSA.new File.read 'public_key.pem'
payload = JWT.decode bearer, pub_key, true, { algorithm: 'RS256'}
# access your payload here
#app.call env
rescue JWT::ExpiredSignature
[403, { 'Content-Type' => 'text/plain' }, ['The token has expired.']]
rescue JWT::DecodeError
[401, { 'Content-Type' => 'text/plain' }, ['A token must be passed.']]
rescue JWT::InvalidIssuerError
[403, { 'Content-Type' => 'text/plain' }, ['The token does not have a valid issuer.']]
rescue JWT::InvalidIatError
[403, { 'Content-Type' => 'text/plain' }, ['The token does not have a valid "issued at" time.']]
end
To use RSA key in your .env instead of loading a file, you will need to use gem 'dotenv' and import the key as a single line variable with the use of newline '\n'. check this question on how to do it. example:
PUBLIC_KEY="-----BEGIN PUBLIC KEY-----\nmineminemineminemine\nmineminemineminemine\nmineminemine...\n-----END PUBLIC KEY-----\n"
as an .env PUBLIC_KEY variable, loading the key will change to this:
key = OpenSSL::PKey::RSA.new ENV['PUBLIC_KEY']
According to wikipedia, a secret key used in cryptography is basically just that, a key to open the lock. The key should be consistent and reliable, but not easy to duplicate, just like a key you would use on your home.
As stated in this answer, secret keys should be randomly-generated. However, you still want the key to be retained for use across the application. By using the password digest from bcrypt, you are actually using a hashed key that was derived from a base secret key (the password). Because the hash is random, this is not a reliable secret key to use, as you stated.
The previous answer using SecureRandom.hex(64) is a great way to create an initial base application key. However, in a production system, you should be taking this in as a configuration variable and storing it for consistent use across multiple runs of your application (for example following a server reboot, you should not invalidate all of your user's JWTs) or across multiple distributed servers. This article gives an example of pulling in the secret key from an environment variable for rails.
I decode (secret_key,client_id, path) into signature by following code :
require 'rubygems'
require 'base64'
require 'cgi'
require 'hmac-sha1'
#client_id = "asdkasdlda"
#secret = "3fdsdsfxds"
binary_key = Base64.decode64(#secret)
params.update({"client" => #client_id})
path = uri_path + "?" + params.collect{|k,v| "#{k}=#{v}"}.inject{|initial,cur| initial + "&" + cur}
digest = HMAC::SHA1.new(binary_key).update(path).digest
digest = Base64.encode64(digest).gsub(/[+\/]/, {"+" => "-", "/" => "_"}).delete("=")
return "#{path}&sig=#{digest}"
So, this code generates sig and path. we send request to server in following way:
/api/v1/customers/sign_in.json?user[email]=amit1656789#gmail.com&user[password]=[FILTERED]&client=asdkasdlda&sig=JSdP5xUHhgS8ZbKApBOIlsJKg_Q
Now, on server side, i want to decode this params["sign"] into app_id, secret_key and path means reverse process of above code. But i am not found any reverse process of this. Means
(app_id, secret, path) => "signature"
"signature" => (app_id, secret, path) /* Here i stuck */
First thing you should know:
"signature" => (app_id, secret, path)
This is not possible. It is not how MACs of any kind work. The signature does not contain the data. Signatures are meant to be sent alongside the message that they sign.
For secure HMAC, you should never send the secret with the message that you sign. It is also not possible to figure out a secret from the signature, except by repeatedly guessing what the value might be.
The usual way to confirm a signature is to follow the same process on the server, signing the same message, using the same secret (which the server should already have), and compare the signatures. You have made it difficult for yourself because you have signed the params as you sent them, and then put the signature on the end. You have to re-construct the message.
First, you need to use whatever web server library you can to get the request URI including the query string
signed_uri = "/api/v1/customers/sign_in.json?user[email]=amit1656789#gmail.com&user[password]=[FILTERED]&client=asdkasdlda&sig=JSdP5xUHhgS8ZbKApBOIlsJKg_Q"
Then split it into the message and its signature (I'll leave that to you, but just a regular expression ought to work):
message = "/api/v1/customers/sign_in.json?user[email]=amit1656789#gmail.com&user[password]=[FILTERED]&client=asdkasdlda"
signature = "JSdP5xUHhgS8ZbKApBOIlsJKg_Q"
To decode this signature back to the original digest (for easy comparison), just reverse the replace and encoding you did at the end on the client:
client_digest = Base64.decode64(
signature.gsub(/[-_]/, {"-" => "+", "_" => "/"}) )
Then on the server (where you should already have a value for #secret), calculate what you expect the signature to be:
#secret = '3fdsdsfxds'
binary_key = Base64.decode64(#secret)
server_digest = HMAC::SHA1.new(binary_key).update( message ).digest
if server_digest == client_digest
puts "The message was signed correctly"
else
puts "ERROR: The message or signature is not correct!"
end
I was going through the Amazon Product Advertising API REST signature docs and I got stuck at #8
Calculate an RFC 2104-compliant HMAC with the SHA256 hash algorithm using the string above with our "dummy" Secret Access Key: 1234567890. For more information about this step, see documentation and code samples for your programming language.
I managed to get it on one more try with the help of Calculating a SHA hash with a string + secret key in python.
The following creates the correct signature:
require 'openssl'
secret_key = '1234567890'
query = 'AWSAccessKeyId=AKIAIOSFODNN7EXAMPLE&ItemId=0679722769&Operation=ItemLookup&ResponseGroup=ItemAttributes%2COffers%2CImages%2CReviews&Service=AWSECommerceService&Timestamp=2009-01-01T12%3A00%3A00Z&Version=2009-01-06'
data = ['GET', 'ecs.amazonaws.com', '/onca/xml', query].join("\n")
sha256 = OpenSSL::Digest::SHA256.new
sig = OpenSSL::HMAC.digest(sha256, secret_key, data)
signature = Base64.encode64(sig)
Adding to AJcodez answer:
I would do:
...
signature_raw = Base64.strict_encode64(sig)
signature = CGI::escape(signature_raw)
encode64adds a newline at the end, strict_encode64() does not.
https://stackoverflow.com/a/2621023/2760406
Amazon wants you to "URL encode the plus (+) and equal (=) characters in the signature" #9 - won't work now if you don't.
http://docs.aws.amazon.com/AWSECommerceService/latest/DG/rest-signature.html#rest_detailedexample
You can calculate a keyed-hash message authentication code (HMAC-SHA256) signature with your secret access key by using cryptoJs
First install cryptoJs locally in your system by typing
npm install crypto-js
to install it globally you node a flag -g to the above command. Then add this code and run it.
var CryptoJS = require("crypto-js");
// Calculate an RFC 2104-compliant HMAC with the SHA256 hash algorithm
var exampleString =
"GET\n" +
"webservices.amazon.com\n" +
"/onca/xml\n" +
"AWSAccessKeyId=AKIAIOSFODNN7EXAMPLE&AssociateTag=mytag-20&ItemId=0679722769&Operation=ItemLookup&ResponseGroup=Images%2CItemAttributes%2COffers%2CReviews&Service=AWSECommerceService&Timestamp=2014-08-18T12%3A00%3A00Z&Version=2013-08-01";
var signature = CryptoJS.HmacSHA256(exampleString, "1234567890");
console.log("test signature", signature.toString(CryptoJS.enc.Base64));