I decode (secret_key,client_id, path) into signature by following code :
require 'rubygems'
require 'base64'
require 'cgi'
require 'hmac-sha1'
#client_id = "asdkasdlda"
#secret = "3fdsdsfxds"
binary_key = Base64.decode64(#secret)
params.update({"client" => #client_id})
path = uri_path + "?" + params.collect{|k,v| "#{k}=#{v}"}.inject{|initial,cur| initial + "&" + cur}
digest = HMAC::SHA1.new(binary_key).update(path).digest
digest = Base64.encode64(digest).gsub(/[+\/]/, {"+" => "-", "/" => "_"}).delete("=")
return "#{path}&sig=#{digest}"
So, this code generates sig and path. we send request to server in following way:
/api/v1/customers/sign_in.json?user[email]=amit1656789#gmail.com&user[password]=[FILTERED]&client=asdkasdlda&sig=JSdP5xUHhgS8ZbKApBOIlsJKg_Q
Now, on server side, i want to decode this params["sign"] into app_id, secret_key and path means reverse process of above code. But i am not found any reverse process of this. Means
(app_id, secret, path) => "signature"
"signature" => (app_id, secret, path) /* Here i stuck */
First thing you should know:
"signature" => (app_id, secret, path)
This is not possible. It is not how MACs of any kind work. The signature does not contain the data. Signatures are meant to be sent alongside the message that they sign.
For secure HMAC, you should never send the secret with the message that you sign. It is also not possible to figure out a secret from the signature, except by repeatedly guessing what the value might be.
The usual way to confirm a signature is to follow the same process on the server, signing the same message, using the same secret (which the server should already have), and compare the signatures. You have made it difficult for yourself because you have signed the params as you sent them, and then put the signature on the end. You have to re-construct the message.
First, you need to use whatever web server library you can to get the request URI including the query string
signed_uri = "/api/v1/customers/sign_in.json?user[email]=amit1656789#gmail.com&user[password]=[FILTERED]&client=asdkasdlda&sig=JSdP5xUHhgS8ZbKApBOIlsJKg_Q"
Then split it into the message and its signature (I'll leave that to you, but just a regular expression ought to work):
message = "/api/v1/customers/sign_in.json?user[email]=amit1656789#gmail.com&user[password]=[FILTERED]&client=asdkasdlda"
signature = "JSdP5xUHhgS8ZbKApBOIlsJKg_Q"
To decode this signature back to the original digest (for easy comparison), just reverse the replace and encoding you did at the end on the client:
client_digest = Base64.decode64(
signature.gsub(/[-_]/, {"-" => "+", "_" => "/"}) )
Then on the server (where you should already have a value for #secret), calculate what you expect the signature to be:
#secret = '3fdsdsfxds'
binary_key = Base64.decode64(#secret)
server_digest = HMAC::SHA1.new(binary_key).update( message ).digest
if server_digest == client_digest
puts "The message was signed correctly"
else
puts "ERROR: The message or signature is not correct!"
end
Related
In my pure Ruby app one of the components to create a token for my request authentication to an external API is to create signature which is HMAC value that is created using the api_key and the secret_key. The signature contains the following elements that are each separated by a new line \n (except the last line) and are in the same order as below list:
ts = '1529342939277'
nonce = '883b170c-a768-41a1-ae6d-c626323aa128'
host = 'ws.idms.lexisnexis.com'
resource_path = '/restws/identity/v3/accounts/11111/workflows/rdp.test.workflow/conversations'
body_hash = 'fQoIAs0IO4vNleZVE9tcI3Ni7h+niT+GrrgEHsKZOyM='
API_KEY = '6njQLkz7uCiz1ZeJ1bFCWX4DFVTfKQXa'
SECRET_KEY = 'CcdaZEt7co647iJoEc5G29CHtlo7T9M3'
# create string signature separated by new line
signature = [ts, nonce, host, resource_path, body_hash].join("\n")
# create HMAC for signature
mac = Base64.strict_encode64(OpenSSL::HMAC.hexdigest('SHA256', API_KEY, signature))
2.7.0 :146 > mac
=> "ZDE4NDQxZDdiNmZkODNiODgyODI4Nzc2OTQ3OGFlMjVhZTMyNThhZTZlMTRiMjkxMzI0NmQ5NzljNDJkZWVhZg=="
According to the docs the signature should be Syb6i+sRygAGCgxLQJ4NwwKcT5Mnkh4r3QXgwZ3vmcE= but I'm getting ZDE4NDQxZDdiNmZkODNiODgyODI4Nzc2OTQ3OGFlMjVhZTMyNThhZTZlMTRiMjkxMzI0NmQ5NzljNDJkZWVhZg== instead. Where did I go wrong?
I've got an example how to do it in Java if something will be unclear: https://gist.github.com/mrmuscle1234/20c9d46d163fee66528449c0ea8419a7
Reading the HTTP API docs. My requests fail though for bad signature. From error message I can see that my string to sign is correct but looks like I can't generate the correct HMAC-SHA1 (seriously why use SHA1 still??).
So I decided to try replicate the signature of the sample inside same document.
[47] pry(main)> to_sign = "GET&%2F&AccessKeyId%3Dtestid&Action%3DDescribeRegions&Format%3DXML&SignatureMethod%3DHMAC-SHA1&SignatureNonce%3D3ee8c1b8-83d3-44af-a94f-4e0ad82fd6cf&SignatureVersion%3D1.0&Timestamp%3D2016-02-23T12%253A46%253A24Z&Version%3D2014-05-26"
[48] pry(main)> Base64.encode64 OpenSSL::HMAC.digest("sha1", "testsecret", to_sign)
=> "MLAxpXej4jJ7TL0smgWpOgynR7s=\n"
[49] pry(main)> Base64.encode64 OpenSSL::HMAC.digest("sha1", "testsecret&", to_sign)
=> "VyBL52idtt+oImX0NZC+2ngk15Q=\n"
[50] pry(main)> Base64.encode64 OpenSSL::HMAC.hexdigest("sha1", "testsecret&", to_sign)
=> "NTcyMDRiZTc2ODlkYjZkZmE4MjI2NWY0MzU5MGJlZGE3ODI0ZDc5NA==\n"
[51] pry(main)> Base64.encode64 OpenSSL::HMAC.hexdigest("sha1", "testsecret", to_sign)
=> "MzBiMDMxYTU3N2EzZTIzMjdiNGNiZDJjOWEwNWE5M2EwY2E3NDdiYg==\n"
[52] pry(main)> OpenSSL::HMAC.hexdigest("sha1", "testsecret&", to_sign)
=> "57204be7689db6dfa82265f43590beda7824d794"
[53] pry(main)> OpenSSL::HMAC.hexdigest("sha1", "testsecret", to_sign)
=> "30b031a577a3e2327b4cbd2c9a05a93a0ca747bb"
As evident none of these matches the example signature of CT9X0VtwR86fNWSnsc6v8YGOjuE=. Any idea what is missing here?
Update: taking tcpdump from the Golang client tool I see that it does a POST request like:
POST /?AccessKeyId=**********&Action=DescribeRegions&Format=JSON&RegionId=cn-qingdao&Signature=aHZVpIMb0%2BFKdoWSIVaFJ7bd2LA%3D&SignatureMethod=HMAC-SHA1&SignatureNonce=c29a0e28964c470a8997aebca4848b57&SignatureType=&SignatureVersion=1.0&Timestamp=2018-07-16T19%3A46%3A33Z&Version=2014-05-26 HTTP/1.1
Host: ecs.aliyuncs.com
User-Agent: Aliyun-CLI-V3.0.3
Content-Length: 0
Content-Type: application/x-www-form-urlencoded
x-sdk-client: golang/1.0.0
x-sdk-core-version: 0.0.1
x-sdk-invoke-type: common
Accept-Encoding: gzip
When I take parameters from the above request and generate signature it does match. So I tried all tree: GET, POST with URL params and POST with params in body. Every time I am getting a signature error. If I redo the request with exact same params as the golang tool, I'm getting nonce already used error (as expected).
Finally got this working. The main issue in my case was that I have been double-percent-encoding the signature parameter thus it turned out invalid. What helped me most was running the aliyun cli utility and capturing traffic, then running a query with exactly the same parameters to compare the exact query string.
But let me list some key points for me:
once hmac-sha1 sig is generated, do not percent-encode it, just add it to the query with normal form www encoding
order of parameters in the HTTP query is not significant; order of parameters in the signing string is significant though
I find all the following types of requests to work: GET, POST with parameters in URL query, POST with parameters in request body form www encoded; I'm using GET per documentation but I see aliyun using POST vs query params and ordered params in the query
you must add & character to the end of the secret key when generating HMAC-SHA1
generate HMAC-SHA1 in binary form, then encode as Base64 (no hex values)
some parameters might be case insensitive, e.g. Format works both as json and JSON
I see aliyun, #wanghq and John using UUID 4 for SignatureNonce but I deferred to plain random (according to docs) because it seems to be only a replay attack protection. So cryptographically secure random number must unnecessary.
The special encoding rules for +, * and ~ seem to only apply to string for signing, not actually to encode data in such a way in the HTTP query.
I decided to not use #wanghq's wrapper as it didn't work for me as well disables certificate validation but maybe it's going to be fixed. Just I thought that queries are simple enough once signature is figured out and an additional layer of indirection is not worth it. +1 to his answer though as it was helpful to get my signature right.
Here's example ruby code to make a simple request:
require 'base64'
require 'cgi'
require 'openssl'
require 'time'
require 'rest-client'
# perform a request against Alibaba Cloud API
# #see https://www.alibabacloud.com/help/doc-detail/25489.htm
def request(action:, params: {})
api_url = "https://ecs.aliyuncs.com/"
# method = "POST"
method = "GET"
process_params!(http: method, action: action, params: params)
RestClient::Request.new(method: method, url: api_url, headers: {params: params})
# RestClient::Request.new(method: method, url: api_url, payload: params)
# RestClient::Request.new(method: method, url: api_url, payload: params.map{|k,v| "#{k}=#{CGI.escape(v)}"}.join("&"))
end
# generates the required common params for a request and adds them to params
# #return undefined
# #see https://www.alibabacloud.com/help/doc-detail/25490.htm
def process_params!(http:, action:, params:)
params.merge!({
"Action" => action,
"AccessKeyId" => config[:auth][:key_id],
"Format" => "JSON",
"Version" => "2014-05-26",
"Timestamp" => Time.now.utc.iso8601
})
sign!(http: http, action: action, params: params)
end
# generate request signature and adds to params
# #return undefined
# #see https://www.alibabacloud.com/help/doc-detail/25492.htm
def sign!(http:, action:, params:)
params.delete "Signature"
params["SignatureMethod"] = "HMAC-SHA1"
params["SignatureVersion"] = "1.0"
params["SignatureNonce"] = "#{rand(1_000_000_000_000)}"
# params["SignatureNonce"] = SecureRandom.uuid.gsub("-", "")
canonicalized_query_string = params.sort.map { |key, value|
"#{key}=#{percent_encode value}"
}.join("&")
string_to_sign = %{#{http}&#{percent_encode("/")}&#{percent_encode(canonicalized_query_string)}}
params["Signature"] = hmac_sha1(string_to_sign)
end
# #param data [String]
# #return [String]
def hmac_sha1(data, secret: config[:auth][:key_secret])
Base64.encode64(OpenSSL::HMAC.digest('sha1', "#{secret}&", data)).strip
end
# encode strings per Alibaba cloud rules for signing
# #return [String] encoded string
# #see https://www.alibabacloud.com/help/doc-detail/25492.htm
def percent_encode(str)
CGI.escape(str).gsub(?+, "%20").gsub(?*, "%2A").gsub("%7E", ?~)
end
## example call
request(action: "DescribeRegions")
Code can be simplified a little but decided to keep it very close to documentation instructions.
P.S. not sure why John deleted his answer but leaving a link above to his web page for any python guys looking for example code
Seems this aliyun ruby sdk (non official, just for reference) works. You may want to check how it's implemented.
Check how its string_to_sign looks like. I did a run and seems it's slightly different than what you provided. The params are concatenated with & instead of %26.
GET&%2F&AccessKeyId%3Dtestid&Action%3DDescribeRegions&Format%3DXML&SignatureMethod%3DHMAC-SHA1&SignatureNonce%3D3ee8c1b8-83d3-44af-a94f-4e0ad82fd6cf&SignatureVersion%3D1.0&Timestamp%3D2016-02-23T12%253A46%253A24Z&Version%3D2014-05-26
require 'rubygems'
require 'aliyun'
$DEBUG = true
options = {
:access_key_id => "k",
:access_key_secret => "s",
:service => :ecs
}
service = Aliyun::Service.new options
puts service.DescribeRegions({})
wanted to share a library I found (Python) that does everything for me w/o the need to sign the request myself.
It can also help those who wants to just copy their functions and still construct the signature on their own
I'm using this:
from aliyunsdkcore.client import AcsClient
from aliyunsdkvpc.request.v20160428.DescribeEipAddressesRequest import DescribeEipAddressesRequest
client = AcsClient(access_key, secret_key, region)
request = DescribeEipAddressesRequest()
request.set_accept_format('json')
response = client.do_action_with_exception(request) # FYI returned as Bytes
print(response)
Each section in Alibaba Cloud has its own library (just like I used: aliyunsdkvpc for EIP addresses)
And they are all listed here:
https://develop.aliyun.com/tools/sdk?#/python
I'm trying to verify a link that will expire in a week. I have an activator_token stored in the database, which will be used to generate the link in this format: http://www.example.com/activator_token. (And not activation tokens generated by Devise or Authlogic.)
Is there a way to make this activator token expire (in a week or so) without comparing with updated_at or some other date. Something like an encoded token, which will return nil when decoded after a week. Can any existing modules in Ruby do this? I don't want to store the generated date in the database or in an external store like Redis and compare it with Time.now. I want it to be very simple, and wanted to know if something like this already exists, before writing the logic again.
What you want to use is: https://github.com/jwt/ruby-jwt .
Here is some boilerplate code so you can try it out yourself.
require 'jwt'
# generate your keys when deploying your app.
# Doing so using a rake task might be a good idea
# How to persist and load the keys is up to you!
rsa_private = OpenSSL::PKey::RSA.generate 2048
rsa_public = rsa_private.public_key
# do this when you are about to send the email
exp = Time.now.to_i + 4 * 3600
payload = {exp: exp, discount: '9.99', email: 'user#example.com'}
# when generating an invite email, this is the token you want to incorporate in
# your link as a parameter
token = JWT.encode payload, rsa_private, 'RS256'
puts token
puts token.length
# this goes into your controller
begin
#token = params[:token]
decoded_token = JWT.decode token, rsa_public, true, { :algorithm => 'RS256' }
puts decoded_token.first
# continue with your business logic
rescue JWT::ExpiredSignature
# Handle expired token
# inform the user his invite link has expired!
puts "Token expired"
end
I'm trying to secure a Sinatra API.
I'm using ruby-jwt to create the JWT, but I don't know exactly what to sign it with.
I'm trying to use the user's BCrypt password_digest, but every time password_digest is called it changes, making the signature invalid when I go to verify it.
Use any kind of application secret key, not a user's bcrypt password digest.
For example, use the dot env gem and a .env file, with an entry such as:
JWT_KEY=YOURSIGNINGKEYGOESHERE
I personally generate a key by using a simple random hex string:
SecureRandom.hex(64)
The hex string contains just 0-9 and a-f, so the string is URL safe.
For RS256 public and private key strategy you can use Ruby OpenSSL lib:
Generating keys:
key = OpenSSL::PKey::RSA.new 2048
open 'private_key.pem', 'w' do |io| io.write key.to_pem end
open 'public_key.pem', 'w' do |io| io.write key.public_key.to_pem end
Load key from .pem file to sign token:
priv_key = OpenSSL::PKey::RSA.new File.read 'private_key.pem'
token = JWT.encode payload, priv_key, 'RS256'
Load key from .pem file to Verify token(Create a middleware for this):
begin
# env.fetch gets http header
bearer = env.fetch('HTTP_AUTHORIZATION').slice(7..-1)
pub_key = OpenSSL::PKey::RSA.new File.read 'public_key.pem'
payload = JWT.decode bearer, pub_key, true, { algorithm: 'RS256'}
# access your payload here
#app.call env
rescue JWT::ExpiredSignature
[403, { 'Content-Type' => 'text/plain' }, ['The token has expired.']]
rescue JWT::DecodeError
[401, { 'Content-Type' => 'text/plain' }, ['A token must be passed.']]
rescue JWT::InvalidIssuerError
[403, { 'Content-Type' => 'text/plain' }, ['The token does not have a valid issuer.']]
rescue JWT::InvalidIatError
[403, { 'Content-Type' => 'text/plain' }, ['The token does not have a valid "issued at" time.']]
end
To use RSA key in your .env instead of loading a file, you will need to use gem 'dotenv' and import the key as a single line variable with the use of newline '\n'. check this question on how to do it. example:
PUBLIC_KEY="-----BEGIN PUBLIC KEY-----\nmineminemineminemine\nmineminemineminemine\nmineminemine...\n-----END PUBLIC KEY-----\n"
as an .env PUBLIC_KEY variable, loading the key will change to this:
key = OpenSSL::PKey::RSA.new ENV['PUBLIC_KEY']
According to wikipedia, a secret key used in cryptography is basically just that, a key to open the lock. The key should be consistent and reliable, but not easy to duplicate, just like a key you would use on your home.
As stated in this answer, secret keys should be randomly-generated. However, you still want the key to be retained for use across the application. By using the password digest from bcrypt, you are actually using a hashed key that was derived from a base secret key (the password). Because the hash is random, this is not a reliable secret key to use, as you stated.
The previous answer using SecureRandom.hex(64) is a great way to create an initial base application key. However, in a production system, you should be taking this in as a configuration variable and storing it for consistent use across multiple runs of your application (for example following a server reboot, you should not invalidate all of your user's JWTs) or across multiple distributed servers. This article gives an example of pulling in the secret key from an environment variable for rails.
Is there any library in Ruby that generates the Signature, 'X-PAYPAL-AUTHORIZATION' header that is required to make calls on behalf of the account holder who has authorized us through the paypal Permissions API.
I am done with the permissions flow and get the required access token, tokenSecret. I feel I am generating the signature incorrectly as all my calls with the the generated 'X-PAYPAL-AUTHORIZATION' fail. They give the following errors:
For NVP call I get:
You do not have permissions to make this API call
And for the GetBasicPersonalData call I get:
Authentication failed. API credentials are incorrect.
Has anyone gone through this in Ruby? What is best way to generate signature. Paypal has just provided some SDK in Paypal, Java, but not the algorithm to generate signature.
Thanks,
Nilesh
Take a look at the PayPal Permissions gem.
https://github.com/moshbit/paypal_permissions
Specifically lib/paypal_permissions/x_pp_authorization.rb
require 'cgi'
require 'openssl'
require 'base64'
class Hash
def to_paypal_permissions_query
collect do |key, value|
"#{key}=#{value}"
end.sort * '&'
end
end
module ActiveMerchant #:nodoc:
module Billing #:nodoc:
module XPPAuthorization
public
def x_pp_authorization_header url, api_user_id, api_password, access_token, access_token_verifier
timestamp = Time.now.to_i.to_s
signature = x_pp_authorization_signature url, api_user_id, api_password, timestamp, access_token, access_token_verifier
{ 'X-PAYPAL-AUTHORIZATION' => "token=#{access_token},signature=#{signature},timestamp=#{timestamp}" }
end
public
def x_pp_authorization_signature url, api_user_id, api_password, timestamp, access_token, access_token_verifier
# no query params, but if there were, this is where they'd go
query_params = {}
key = [
paypal_encode(api_password),
paypal_encode(access_token_verifier),
].join("&")
params = query_params.dup.merge({
"oauth_consumer_key" => api_user_id,
"oauth_version" => "1.0",
"oauth_signature_method" => "HMAC-SHA1",
"oauth_token" => access_token,
"oauth_timestamp" => timestamp,
})
sorted_query_string = params.to_paypal_permissions_query
base = [
"POST",
paypal_encode(url),
paypal_encode(sorted_query_string)
].join("&")
base = base.gsub /%([0-9A-F])([0-9A-F])/ do
"%#{$1.downcase}#{$2.downcase}" # hack to match PayPal Java SDK bit for bit
end
digest = OpenSSL::HMAC.digest('sha1', key, base)
Base64.encode64(digest).chomp
end
# The PayPalURLEncoder java class percent encodes everything other than 'a-zA-Z0-9 _'.
# Then it converts ' ' to '+'.
# Ruby's CGI.encode takes care of the ' ' and '*' to satisfy PayPal
# (but beware, URI.encode percent encodes spaces, and does nothing with '*').
# Finally, CGI.encode does not encode '.-', which we need to do here.
def paypal_encode str
s = str.dup
CGI.escape(s).gsub('.', '%2E').gsub('-', '%2D')
end
end
end
end
Sample parameters:
url = 'https://svcs.sandbox.paypal.com/Permissions/GetBasicPersonalData'
api_user_id = 'caller_1234567890_biz_api1.yourdomain.com'
api_password = '1234567890'
access_token = 'YJGjMOmTUqVPlKOd1234567890-jdQV3eWCOLuCQOyDK1234567890'
access_token_verifier = 'PgUjnwsMhuuUuZlPU1234567890'
The X-PAYPAL-AUTHORIZATION header [is] generated with URL "https://svcs.paypal.com/Permissions/GetBasicPersonalData". (see page 23, and chapter 7, at the link)
NVP stating "You do not have permissions to make this API call" means your API credentials are correct, just that your account does not have permission for the particular API you are trying to call. Something between the two calls you are submitting is not using the same API credentials.
For NVP call I get:
What NVP call?
TransactionSearch (see comments below)
Also, if you haven't already done so, you will want to use the sandbox APP-ID for testing in the sandbox, and you will need to apply for an app-id with Developer Technical Services (DTS) at PayPal to get an App-ID for live.
EDIT:
To use the TransactionSearch API, all you should be submitting is below. You do not need to specify any extra headers.
USER=xxxxxxxxxxxxxxxxxx
PWD=xxxxxxxxxxxxxxxxxx
SIGNATURE=xxxxxxxxxxxxxxxxxx
METHOD=TransactionSearch
VERSION=86.0
STARTDATE=2009-10-11T00:00:00Z
TRANSACTIONID=1234567890
//And for submitting API calls on bob's behalf, if his PayPal email was bob#bob.com:
SUBJECT=bob#bob.com